Wednesday, 2018-03-14

cburgessmriedem overachiever, I take it you all are finally removing nova-network from the tree?00:09
*** germs has quit IRC00:15
*** salv-orlando has joined #openstack-nova00:15
*** germs has joined #openstack-nova00:17
*** gouthamr has joined #openstack-nova00:19
*** gyee has quit IRC00:20
*** claudiub has quit IRC00:20
*** salv-orlando has quit IRC00:21
*** chyka has quit IRC00:21
*** pchavva has joined #openstack-nova00:24
*** germs has quit IRC00:32
*** germs has joined #openstack-nova00:42
*** germs has quit IRC00:42
*** germs has joined #openstack-nova00:42
*** Zames has joined #openstack-nova00:44
*** Dinesh_Bhor has joined #openstack-nova00:47
*** germs has quit IRC00:48
*** liuzz has joined #openstack-nova00:48
*** germs has joined #openstack-nova00:49
*** yamahata_ has quit IRC00:49
*** yamahata_ has joined #openstack-nova00:50
*** Dinesh_Bhor has quit IRC00:51
*** germs has quit IRC00:53
*** odyssey4me has quit IRC00:53
*** odyssey4me has joined #openstack-nova00:53
*** Dinesh_Bhor has joined #openstack-nova00:54
*** gjayavelu has quit IRC00:55
*** wolverineav has quit IRC00:56
*** wolverineav has joined #openstack-nova00:57
*** jichen has joined #openstack-nova00:57
*** mingyu has quit IRC00:57
artomSpaz-Home, you have a weird schedule ;)00:58
*** Dinesh__Bhor has joined #openstack-nova00:59
*** Dinesh__Bhor has quit IRC00:59
*** zhaochao has joined #openstack-nova00:59
*** Dinesh_Bhor has quit IRC01:00
*** Zames has quit IRC01:01
*** phuongnh has joined #openstack-nova01:02
*** wolverineav has quit IRC01:02
*** r-daneel has quit IRC01:02
*** Dinesh_Bhor has joined #openstack-nova01:03
mnaseris os_vif under nova or neutron's world01:03
mnaser -- im trying to understand why the unplug under linux is noop for ovs01:03
jichenmriedem: hi do you know where to find the etherpad that discussed in PTG to put patches of previously approved spec so that it can get more review and help? I want to put into it. thanks01:03
*** pchavva has quit IRC01:06
Kevin_Zhengmriedem Hi, saw you guys discussing about the quota thing, any conclusion?01:09
*** hongbin has joined #openstack-nova01:12
*** itlinux has quit IRC01:12
*** salv-orlando has joined #openstack-nova01:17
*** harlowja has quit IRC01:20
*** salv-orlando has quit IRC01:21
*** yangyapeng has joined #openstack-nova01:22
*** hshiina has joined #openstack-nova01:24
*** suresh12 has quit IRC01:25
*** edmondsw has joined #openstack-nova01:25
*** suresh12 has joined #openstack-nova01:25
openstackgerritArtom Lifshitz proposed openstack/nova-specs master: Live migration with CPU pinning
*** yangyapeng has quit IRC01:26
*** gongysh has quit IRC01:27
*** sdague has quit IRC01:28
*** edmondsw has quit IRC01:29
*** suresh12 has quit IRC01:30
*** Tom-Tom has joined #openstack-nova01:36
*** gbarros has quit IRC01:36
*** gongysh has joined #openstack-nova01:39
*** lei-zh has joined #openstack-nova01:39
*** yangyapeng has joined #openstack-nova01:41
*** tiendc has joined #openstack-nova01:55
*** suresh12 has joined #openstack-nova02:00
*** suresh12 has quit IRC02:04
*** zhaochao has quit IRC02:04
*** suresh12 has joined #openstack-nova02:05
jianghuaw_jichen, is this what you're looking for?
Spaz-Homelol indeed artom , Night shift, so I work Asia hours :p02:12
*** vladikr has quit IRC02:13
*** vladikr has joined #openstack-nova02:13
artomSpaz-Home, ah! I did night shift in a previous life, working support for a web host. It was... interesting02:14
Spaz-HomeThat it is.. i've been doing it for about 10 years..  It can be extremely unhealthy, Just in the paste 2 years or so i've started to prioritize my health in the situation.02:14
Spaz-HomeBut also.. I get to bother jianghuaw_ all night, which is probably the best benefit :P02:14
jianghuaw_Spaz-Home, good morning:-)02:15
Spaz-HomeWelcome home, sir.02:15
artomI only did it for a few months, it definitely messed with me, I can't imagine doing it for years. We're not cats.02:15
jichenjianghuaw_: thanks, I missed line 98 in the etherpad... :)02:15
Spaz-HomeHehe yes sir.. I've honestly seem people's minds go a little crazy .. some people just are not made for that transition.02:16
jianghuaw_jichen, np:-)02:16
jianghuaw_Spaz-Home, thanks. Good job on  :-)02:18
Spaz-HomeHey jianghuaw_  before I lose ya,  Dan is asking for a Xen Subteam member to attend Nova Meeting this week for
Spaz-HomeHaha thanks sir, made me happy02:18
jianghuaw_Spaz-Home, yes. Naichuan or myself will attend this week's meeting.02:19
jianghuaw_Hope will get that BP be approved.02:19
Spaz-HomeSounds good.  I'd also like to discuss that with you a bit today or tomorrow if you have the time02:19
Spaz-HomeI do thin kthey plan to approve02:19
jianghuaw_Yes, we had some discussion in the PTG.02:20
jianghuaw_Spaz-Home, so what do you want to to chat on that topic?02:20
*** germs has joined #openstack-nova02:20
*** germs has quit IRC02:20
*** germs has joined #openstack-nova02:20
Spaz-HomeWell I understand the reason for the patch and it's basically just moving away from aggregates into the more refined pools, so I cannot speak anythign bad about the patch itself as it's the same situation but a different method02:21
Spaz-HomeHowever I still dislike that we're forcing an optional feature into the xenapi drives for LM to work02:21
Spaz-HomeMy apologies, Live Migration02:22
Spaz-HomeI guess the best example would be my own environment we work in,  We do not use Aggregates or Pooling, and just allow VMs to migrate around within the same Cell02:23
Spaz-HomeSince Networks are assigned through the cell and inventory is similar.. My thought however is how common that swetup would be rather than using pooling or aggregates.02:23
jianghuaw_Yes, the original plan for this feature is to support XAPI pool which is the direction which we think is better than the aggregation pool.02:23
openstackgerritMerged openstack/nova master: Reparent placement objects to oslo_versionedobjects
Spaz-HomeI can certainly agree on that point02:24
jianghuaw_But later we noticed the existing aggregation pool is broken when introducing cells.02:24
*** yangyapeng has quit IRC02:25
*** r-daneel has joined #openstack-nova02:26
jianghuaw_I guess the use case you mentioned should be still supported.02:27
Spaz-HomeWell, as of right now it's definately not since it will fail pulling the aggregate, and I think it will now as well trying to pull a pool master.02:27
*** Dinesh_Bhor has quit IRC02:27
jianghuaw_As you're not using pool or aggregation, so ML should still work.02:28
*** amodi has joined #openstack-nova02:29
*** Dinesh_Bhor has joined #openstack-nova02:30
Spaz-Homeyeah I don't think it will work looking at the new is_same_pool_host02:30
Spaz-HomeIT's going to look for that dest in the pool and fail just like the aggregation check does.02:30
Spaz-HomeI'll need to fire up a host to confirm that, but should throw an error02:30
Spaz-HomeI guess in the end I had two thoughts.  If  (when we make this final change to change the checks themselves) we could check if the pool is empty AND if CONF [cells]->Enable=True  to figure out if a destination is acceptable02:31
Spaz-HomeOr if you guys do decide to jsut rely on pools, I would like to submit a followup patch to increase parity with block-migration.  Right now block-migration is very much not aligned, as this is how we currently live migrate to avoid the aggregation check.02:32
Spaz-HomeThat way things remain even and clean.  We could use that opportunity to always run the assert_will_migrate and ensure that block-migration checks for the pool st atus as well.02:33
jianghuaw_I think it should go to this branch.02:34
jianghuaw_It's not able to use shared SR to speed-up live-migration.02:35
*** yamahata_ has quit IRC02:35
*** vladikr has quit IRC02:35
*** AlexeyAbashkin has joined #openstack-nova02:36
jianghuaw_If want to use shared SR, surely must add hosts into the same pool.02:36
*** mriedem has quit IRC02:37
*** vladikr has joined #openstack-nova02:38
jianghuaw_Spaz-Home, also please note we will have a follow-up patch to remove the this upcall - _ensure_host_in_aggregate.02:38
Spaz-HomeYeah got ya02:38
Spaz-HomeIt's wierd that we're using block_migration like that.. but I actually hadn't looked at this line before02:39
Spaz-HomeSicne block_migration is intended purely for iscsi stuff.. it's itneresting that we're using it to reroute the code lol02:39
*** AlexeyAbashkin has quit IRC02:40
Spaz-HomeOk.. with that line I am fine with this then sir02:41
jianghuaw_Indeed. I agree. At sometime we should refactor this part to make it more reasonable.02:41
Spaz-HomeI think we need to just refactor the process overall.. I was thinking in a month or so just getting a whiteboard and seeing what I can do and send you pictures lol02:41
Spaz-HomeBut a project for another time02:41
*** amodi has quit IRC02:42
*** hongbin_ has joined #openstack-nova02:42
jianghuaw_Great. yes. just ping me if you have any thoughts at any time.02:42
jianghuaw_I'm happy that you're interested at it.02:42
*** hongbin has quit IRC02:43
*** germs has quit IRC02:43
Spaz-HomeAbsolutely :D   I'll submit a commit here this week as well to fix that assert_can_migrate as well so evertything asserts, and keep bug diving :P02:43
Spaz-HomeEnjoy the rest of your shift sir, let me know if you need anything02:43
*** germs has joined #openstack-nova02:43
*** germs has quit IRC02:43
*** germs has joined #openstack-nova02:43
jianghuaw_Thanks. It's too late for you.02:43
jianghuaw_time for bed02:44
*** fragatina has quit IRC02:46
*** psachin has joined #openstack-nova02:47
*** germs has quit IRC02:48
*** fragatina has joined #openstack-nova02:49
Spaz-HomeHehe nah I stay on my night shift on my weekends.  Will be up until well after you're asleep.02:50
Spaz-HomeWith my wife in Korea, I have to stay on the same hours as you basically :p02:50
*** fragatina has quit IRC02:51
*** Dinesh_Bhor has quit IRC02:52
*** Dinesh_Bhor has joined #openstack-nova02:52
openstackgerritZhenyu Zheng proposed openstack/nova-specs master: Allow abort live migrations in queued status
*** suresh12 has quit IRC02:54
openstackgerritZhenyu Zheng proposed openstack/nova-specs master: Allow abort live migrations in queued status
Spaz-HomeMoving my second commit back down to subteam review.  I self -1'ed it for the moment.  Will fix unit tests tomorrow.02:56
*** Dinesh_Bhor has quit IRC02:58
*** psachin has quit IRC02:59
*** Dinesh_Bhor has joined #openstack-nova03:00
*** psachin has joined #openstack-nova03:03
*** psachin has quit IRC03:05
*** psachin has joined #openstack-nova03:08
*** edmondsw has joined #openstack-nova03:14
*** vivsoni has quit IRC03:15
*** gongysh has quit IRC03:17
*** psachin has quit IRC03:17
*** edmondsw has quit IRC03:18
*** zhaochao has joined #openstack-nova03:18
*** salv-orlando has joined #openstack-nova03:18
*** salv-orlando has quit IRC03:23
*** nicolasbock has quit IRC03:26
*** vivsoni has joined #openstack-nova03:27
*** suresh12 has joined #openstack-nova03:28
*** hshiina2 has joined #openstack-nova03:31
*** hshiina has quit IRC03:31
*** esberglu has quit IRC03:32
*** suresh12 has quit IRC03:32
*** Zames has joined #openstack-nova03:32
*** psachin has joined #openstack-nova03:35
*** AlexeyAbashkin has joined #openstack-nova03:36
*** AlexeyAbashkin has quit IRC03:40
*** vivsoni has quit IRC03:45
*** yamahata_ has joined #openstack-nova03:51
*** Zames has quit IRC03:53
*** david-lyle has joined #openstack-nova03:55
*** vivsoni has joined #openstack-nova03:58
*** gongysh has joined #openstack-nova04:00
*** hongbin_ has quit IRC04:02
*** Zames has joined #openstack-nova04:02
*** janki has joined #openstack-nova04:02
*** esberglu has joined #openstack-nova04:05
*** udesale has joined #openstack-nova04:05
*** Dinesh_Bhor has quit IRC04:07
*** fragatina has joined #openstack-nova04:08
*** namnh has joined #openstack-nova04:09
*** suresh12 has joined #openstack-nova04:09
*** fragatina has quit IRC04:09
*** esberglu has quit IRC04:09
*** fragatina has joined #openstack-nova04:10
*** harlowja has joined #openstack-nova04:12
*** suresh12 has quit IRC04:14
*** Zames has quit IRC04:16
*** suresh12 has joined #openstack-nova04:17
*** wolverineav has joined #openstack-nova04:17
*** dave-mccowan has quit IRC04:17
*** salv-orlando has joined #openstack-nova04:19
*** wolverineav has quit IRC04:21
*** links has joined #openstack-nova04:22
*** salv-orlando has quit IRC04:24
*** moshele has joined #openstack-nova04:28
*** lei-zh has quit IRC04:28
*** diga has joined #openstack-nova04:29
*** moshele has quit IRC04:36
*** chyka has joined #openstack-nova04:38
*** harlowja has quit IRC04:40
*** Dinesh_Bhor has joined #openstack-nova04:42
*** chyka has quit IRC04:43
*** abhishekk has joined #openstack-nova04:45
*** andreas_s has joined #openstack-nova04:48
*** andreas_s has quit IRC04:52
*** lei-zh has joined #openstack-nova04:54
*** Dinesh_Bhor has quit IRC04:57
*** esberglu has joined #openstack-nova04:59
*** vladikr has quit IRC05:00
*** Dinesh_Bhor has joined #openstack-nova05:00
*** alexchadin has joined #openstack-nova05:03
*** esberglu has quit IRC05:03
*** ratailor has joined #openstack-nova05:14
*** Dinesh_Bhor has quit IRC05:17
*** salv-orlando has joined #openstack-nova05:20
*** salv-orlando has quit IRC05:24
*** mdnadeem has joined #openstack-nova05:31
*** sidx64 has joined #openstack-nova05:31
*** esberglu has joined #openstack-nova05:34
*** esberglu has quit IRC05:38
openstackgerritOpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements
*** claudiub has joined #openstack-nova05:48
*** Spazmotic has joined #openstack-nova05:48
*** sidx64 has quit IRC05:49
*** Spaz-Home has quit IRC05:49
*** sridharg has joined #openstack-nova05:50
*** sidx64 has joined #openstack-nova05:51
openstackgerritOpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements
*** sidx64 has quit IRC05:59
*** suresh12 has quit IRC05:59
*** sidx64 has joined #openstack-nova06:00
openstackgerritOpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata
*** salv-orlando has joined #openstack-nova06:21
*** lajoskatona has joined #openstack-nova06:23
*** salv-orlando has quit IRC06:25
*** esberglu has joined #openstack-nova06:28
*** alexchadin has quit IRC06:28
*** alexchadin has joined #openstack-nova06:29
*** esberglu has quit IRC06:32
*** mdnadeem_ has joined #openstack-nova06:32
*** moshele has joined #openstack-nova06:33
*** mdnadeem has quit IRC06:33
openstackgerritZhenyu Zheng proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware
*** salv-orlando has joined #openstack-nova06:34
*** Dinesh_Bhor has joined #openstack-nova06:35
*** kholkina has joined #openstack-nova06:37
*** sidx64_ has joined #openstack-nova06:38
*** mdnadeem_ has quit IRC06:39
*** sidx64 has quit IRC06:40
*** trinaths has joined #openstack-nova06:41
*** dims has quit IRC06:41
*** sidx64 has joined #openstack-nova06:42
*** dims has joined #openstack-nova06:43
*** sidx64_ has quit IRC06:45
*** dims has quit IRC06:48
*** dims has joined #openstack-nova06:49
*** edmondsw has joined #openstack-nova06:50
*** mdnadeem_ has joined #openstack-nova06:50
*** deepak_ has quit IRC06:53
*** edmondsw has quit IRC06:54
*** tojuvone has joined #openstack-nova06:56
*** sahid has joined #openstack-nova06:58
*** pcaruana has joined #openstack-nova07:04
*** wangqwsh has joined #openstack-nova07:09
*** sidx64_ has joined #openstack-nova07:12
*** sidx64 has quit IRC07:13
*** andreas_s has joined #openstack-nova07:14
*** Dinesh_Bhor has quit IRC07:15
*** Dinesh_Bhor has joined #openstack-nova07:16
*** alexchadin has quit IRC07:16
*** sidx64_ has quit IRC07:17
*** alexchadin has joined #openstack-nova07:17
*** sidx64 has joined #openstack-nova07:17
*** sar has joined #openstack-nova07:19
*** Dinesh_Bhor has quit IRC07:20
*** Dinesh_Bhor has joined #openstack-nova07:27
*** tovin07 has joined #openstack-nova07:28
*** salv-orlando has quit IRC07:28
*** OctopusZhang has joined #openstack-nova07:29
*** andreas_s has quit IRC07:31
*** danpawlik has joined #openstack-nova07:31
*** andreas_s has joined #openstack-nova07:32
*** OctopusZhang_ has joined #openstack-nova07:32
*** OctopusZhang has quit IRC07:32
*** OctopusZhang_ is now known as yufei07:33
*** salv-orlando has joined #openstack-nova07:34
*** OctopusZhang has joined #openstack-nova07:35
*** Dinesh_Bhor has quit IRC07:37
*** trinaths has quit IRC07:38
*** yufei has quit IRC07:39
*** OctopusZhang is now known as yufei07:39
*** alexchadin has quit IRC07:39
*** chyka has joined #openstack-nova07:42
*** chyka has quit IRC07:47
*** alexchadin has joined #openstack-nova07:47
*** tssurya has joined #openstack-nova07:51
*** AlexeyAbashkin has joined #openstack-nova07:57
*** alexchadin has quit IRC07:57
*** alexchadin has joined #openstack-nova07:58
*** suresh12 has joined #openstack-nova07:59
*** mdnadeem has joined #openstack-nova08:02
*** suresh12 has quit IRC08:04
*** mdnadeem_ has quit IRC08:05
*** diga has quit IRC08:08
*** tesseract has joined #openstack-nova08:14
openstackgerritsahid proposed openstack/nova master: libvirt: slow live-migration to ensure network is ready
*** Dinesh_Bhor has joined #openstack-nova08:17
*** salv-orlando has quit IRC08:17
*** damien_r has joined #openstack-nova08:21
*** yamahata_ has quit IRC08:24
*** jpena|off is now known as jpena08:25
openstackgerritJie Li proposed openstack/nova-specs master: Support volume-backed server rebuild
*** hoonetorg has quit IRC08:30
*** mingyu has joined #openstack-nova08:30
*** mdbooth has quit IRC08:33
*** ansiwen has quit IRC08:33
*** amoralej|off is now known as amoralej08:36
openstackgerritsahid proposed openstack/nova-specs master: libvirt: add support for virtio-net rx/tx queue sizes
*** edmondsw has joined #openstack-nova08:38
*** Dinesh_Bhor has quit IRC08:38
*** edmondsw has quit IRC08:43
*** hoonetorg has joined #openstack-nova08:44
*** ragiman has joined #openstack-nova08:51
*** ralonsoh has joined #openstack-nova08:54
*** ccamacho has joined #openstack-nova08:55
*** cdent has joined #openstack-nova08:58
*** sidx64 has quit IRC09:03
openstackgerritChris Dent proposed openstack/nova master: Optional separate database for placement API
openstackgerritChris Dent proposed openstack/nova master: Isolate placement database config
openstackgerritChris Dent proposed openstack/nova master: Move placement exceptions into the placement package
*** salv-orlando has joined #openstack-nova09:07
kashyapmdbooth: Saw the change last evening was letting Zuul go through its course.09:08
*** lucas-afk is now known as lucasagomes09:11
*** Dinesh_Bhor has joined #openstack-nova09:15
*** Dinesh_Bhor has quit IRC09:16
*** yamamoto_ has joined #openstack-nova09:18
*** hemna_ has quit IRC09:20
openstackgerritChris Dent proposed openstack/nova master: Move placement exceptions into the placement package
openstackgerritChris Dent proposed openstack/nova master: Optional separate database for placement API
openstackgerritChris Dent proposed openstack/nova master: Isolate placement database config
*** Dinesh_Bhor has joined #openstack-nova09:21
*** trinaths has joined #openstack-nova09:21
*** yamamoto has quit IRC09:22
*** gongysh has quit IRC09:22
*** mgoddard_ has joined #openstack-nova09:24
bauzasman, I forgot my manners09:27
bauzasgood morning Nova09:27
*** udesale_ has joined #openstack-nova09:27
*** udesale__ has joined #openstack-nova09:29
*** udesale has quit IRC09:29
*** Dinesh_Bhor has quit IRC09:30
*** udesale_ has quit IRC09:32
*** derekh has joined #openstack-nova09:34
*** lpetrut has joined #openstack-nova09:34
openstackgerritYikun Jiang (Kero) proposed openstack/nova-specs master: Complex (Anti)-Affinity Policies
pooja_jadhavcdent: Hi09:37
cdentgood morning pooja_jadhav09:38
pooja_jadhavcdent: actually i want discuss with you regarding shared resource provider thing.09:39
pooja_jadhavcdent: very gm :)09:39
cdentpooja_jadhav: sure, but you should be aware that share providers are still a bit of a work in progress. Most of the functionality is nearly there, but nothing really uses it yet.09:40
cdentWhat's your plan?09:41
pooja_jadhavcdent: planning to test correct disk usage or not on shared storage09:42
pooja_jadhavcdent : cdent: i have configured nfs backend, and created shared resource provider, but when i booted the instance it failing at line [1]
cdentpooja_jadhav: is there an aggregate in place that associates the shared storage provider with one or more compute nodes that use it?09:43
pooja_jadhavyes, aggregate is there09:43
pooja_jadhavits not getting allocation request09:44
pooja_jadhavwhen i create the instance, that instance going into error state09:44
pooja_jadhavactually, where am i missing something, i am not getting :(09:46
cdenttetsuro recently wrote an email with some updated information on the state of sharing providers, let me find that. It may be that what you're trying to do simply doesn't work in /allocation_candidates yet09:47
cdentthis message:
cdentThat first item he lists, about the resource class existing in both places, may be part of the problem? Is the compute node reporting disk inventory ?09:48
*** lei-zh has quit IRC09:50
*** Tom-Tom has quit IRC09:52
*** Tom-Tom has joined #openstack-nova09:52
pooja_jadhavi have checked the db, inventory record i have created that exists09:52
*** wangqwsh has quit IRC09:52
pooja_jadhavbut to check that compute node is reporting it or not?09:52
pooja_jadhavbut how to check that compute node is reporting that inventory or not?09:53
cdentpooja_jadhav: there's an osc-placement client now, so you can use that if you like, or use curl or another tool to interact with the placement api, or look in the database (there will be inventory rows that are associated with resource providers by resource provider id)09:55
pooja_jadhavyes, i have checked in the database, there is inventory which i have created with the resource provider which is created by me.09:56
*** Tom-Tom has quit IRC09:57
*** jichen has quit IRC09:58
*** namnh has quit IRC10:00
cdentpooja_jadhav: do you have disk inventory from both a compute node and the shared provider or only the shared provider?10:00
openstackgerritChris Dent proposed openstack/nova master: Update contributor/placement.rst to contemporary reality
cdentstephenfin, gibi : quick doc improvement ^10:01
*** yufei has quit IRC10:02
cdentstephenfin: I also reordered some of the remaining placement-extraction code that you recently helped merge so that the current "next one" is ahead of the optional db stuff:
pooja_jadhavcdebt : disk inventory from both a compute node and the shared provider (bcz I have created only one inventory record for resource provider id 2 but in that table already 3 more records exists means 3 inventory records from resource provider id 1)10:04
cdentpooja_jadhav: in that case I think you're hitting the problem described here it is possible for you try again with that code in place?10:05
cdentI'm going to get some more coffee but will be back shortly10:05
pooja_jadhavcdent: sure10:08
pooja_jadhavi will try10:10
*** hshiina2 has quit IRC10:11
*** rcernin has quit IRC10:13
*** sidx64 has joined #openstack-nova10:17
*** alexchadin has quit IRC10:18
cdentpooja_jadhav: let me know how it goes, I suspect there will be a few more issues, as shared providers hasn't received the same attention (yet) that other use cases have10:18
*** mingyu has quit IRC10:18
*** sidx64 has quit IRC10:24
*** nicolasbock has joined #openstack-nova10:25
openstackgerritChris Dent proposed openstack/nova master: Fix allocation_candidates not to ignore shared RPs
openstackgerritChris Dent proposed openstack/nova master: Test alloc_cands with indirectly sharing RPs
openstackgerritChris Dent proposed openstack/nova master: Support relay RP for allocation candidates
*** priteau has joined #openstack-nova10:26
cdentpooja_jadhav: in that ^ stack is an update version that resolves the merge conflicts, which you'll want if you're working from today's master10:26
*** edmondsw has joined #openstack-nova10:26
pooja_jadhavcdent : i have applied the patch, but facing same issue10:27
*** fanzhang has quit IRC10:28
*** fanzhang has joined #openstack-nova10:28
*** sdague has joined #openstack-nova10:28
pooja_jadhavwhen i hit nova show <instance_uuid> then it shows error in fault section [1]
*** phuongnh has quit IRC10:28
cdentIf you can write up a bug that might be the best thing at this point. I havent got a clear picture of exactly what you're doing and having the replication strategy written down in a bug will make it easier to understand10:29
*** jmlowe has quit IRC10:30
cdentah, I hadnt understood you were using a custom resource class, that's an important bit of data. Do you have a log of the requests made to the placement service, it would be useful to see what the GET /allocation_candidartes query is10:30
*** vivsoni has quit IRC10:30
cdentpooja_jadhav: this bug may be related:
openstackLaunchpad bug 1705231 in OpenStack Compute (nova) "Placement returns no allocation candidate for request that needs both compute resources and custom shared resources" [High,Fix released] - Assigned to Chris Dent (cdent)10:31
*** edmondsw has quit IRC10:31
*** esberglu has joined #openstack-nova10:41
*** Zames has joined #openstack-nova10:41
*** mingyu has joined #openstack-nova10:42
openstackgerritStephen Finucane proposed openstack/nova master: conf: Correct documentation for '[pci] passthrough_whitelist'
stephenfinlyarwood, sean-k-mooney: ^10:43
stephenfinI'll probably look for that to be backported if all is ok10:43
*** sidx64 has joined #openstack-nova10:44
*** esberglu has quit IRC10:45
lyarwoodstephenfin: cool yeah that would be great, will need a bug if you have time, happy to quickly write one if not, I did mean to do this yesterday :|10:46
stephenfinlyarwood: I'm working on the NUMA-aware vSwitch spec this morning, so if you're happy to write up said bug I'd appreciate it :)10:47
*** sidx64 has quit IRC10:48
*** alexchadin has joined #openstack-nova10:48
*** Zames has quit IRC10:49
*** trinaths has quit IRC10:49
*** alexchadin has quit IRC10:51
*** mikal_ has joined #openstack-nova10:52
*** alexchadin has joined #openstack-nova10:53
*** mikal has quit IRC10:53
*** sidx64 has joined #openstack-nova10:55
*** alexchadin has quit IRC10:55
*** belmoreira has joined #openstack-nova10:56
*** cdent has quit IRC10:56
*** Zames has joined #openstack-nova10:56
*** sidx64 has quit IRC10:58
*** esberglu has joined #openstack-nova10:58
*** Zames has quit IRC10:59
*** Zames has joined #openstack-nova11:00
*** sidx64 has joined #openstack-nova11:01
*** esberglu has quit IRC11:03
*** alexchadin has joined #openstack-nova11:04
*** sidx64 has quit IRC11:09
*** janki has quit IRC11:10
*** sidx64 has joined #openstack-nova11:11
*** brault has joined #openstack-nova11:19
*** HW-Peter has quit IRC11:22
*** jmlowe has joined #openstack-nova11:23
*** dtantsur|afk is now known as dtantsur11:26
*** jmlowe has quit IRC11:27
*** abhishekk has quit IRC11:29
*** chyka has joined #openstack-nova11:30
*** chyka has quit IRC11:34
*** sambetts|afk is now known as sambetts11:36
*** Zames has quit IRC11:36
gibiafternoon nova11:39
* gibi was in an internal meeting all morning11:39
gibicdent: thanks for the placement doc update, I +2d it11:39
*** janki has joined #openstack-nova11:43
*** ratailor has quit IRC11:43
*** salv-orlando has quit IRC11:44
*** udesale__ has quit IRC11:45
*** cdent has joined #openstack-nova11:49
*** tiendc has quit IRC11:52
openstackgerritMerged openstack/nova master: Update contributor/placement.rst to contemporary reality
*** lucasagomes is now known as lucas-hungry12:03
*** tetsuro has quit IRC12:06
*** amodi has joined #openstack-nova12:07
*** pchavva has joined #openstack-nova12:14
*** yamamoto_ has quit IRC12:17
*** yamamoto has joined #openstack-nova12:18
*** chyka has joined #openstack-nova12:23
*** edmondsw has joined #openstack-nova12:24
*** chyka has quit IRC12:27
artombauzas, sahid, could you take a look at when you get a chance? It's the infamouse live migration with CPU pinning spec12:27
*** jpena is now known as jpena|lunch12:27
artomsahid, you weren't at PTG, but we basically agreed to start over, with a spec, since Nikola's patch is too hard to review/merge at this point12:27
artomjaypipes, ^^ dunno if you're around this early, but your input would be appreciated as well12:28
*** mdnadeem_ has joined #openstack-nova12:31
*** dave-mccowan has joined #openstack-nova12:32
*** mdnadeem has quit IRC12:35
openstackgerritNguyen Hai proposed openstack/nova-specs master: Enhance nova-specs webpage and clean up repo
*** maciejjozefczyk has quit IRC12:37
*** maciejjozefczyk has joined #openstack-nova12:38
*** yamamoto has quit IRC12:38
*** claudiub has quit IRC12:39
*** yamamoto has joined #openstack-nova12:39
*** derekh has quit IRC12:41
*** yamamoto has quit IRC12:44
*** salv-orlando has joined #openstack-nova12:44
jaypipesartom: yep, will look shortly.12:46
jaypipesthx for the heads up12:46
artomjaypipes, cheers :)12:47
*** odyssey4me has quit IRC12:47
*** odyssey4me has joined #openstack-nova12:47
*** vladikr has joined #openstack-nova12:49
*** salv-orlando has quit IRC12:49
*** r-daneel has quit IRC12:53
*** yamamoto has joined #openstack-nova12:55
*** edmondsw_ has joined #openstack-nova12:55
*** mriedem has joined #openstack-nova12:57
*** edmondsw has quit IRC12:58
*** yamamoto has quit IRC12:59
*** alexchadin has quit IRC13:01
*** artom has quit IRC13:01
*** alexchadin has joined #openstack-nova13:06
*** lucas-hungry is now known as lucasagomes13:07
*** yamamoto has joined #openstack-nova13:10
openstackgerritEric Fried proposed openstack/nova master: placement: generation in provider aggregate APIs
openstackgerritEric Fried proposed openstack/nova master: placement: Return new provider from POST /rps
openstackgerritEric Fried proposed openstack/nova master: Stop assuming initial provider generation is 0
jaypipesefried: ^ that ready to go now?13:12
efriedjaypipes: First two, yes.13:12
efriedjaypipes: Quick, before edleafe's!13:12
*** cmm_ has joined #openstack-nova13:13
*** sree has joined #openstack-nova13:13
*** yamamoto has quit IRC13:15
*** amoralej is now known as amoralej|lunch13:15
*** Tom-Tom has joined #openstack-nova13:19
*** lyan has joined #openstack-nova13:21
*** lyan is now known as Guest764113:21
edleafeefried: no rush, working on alex_xu_'s comments13:22
*** felipemonteiro_ has joined #openstack-nova13:23
*** yamamoto has joined #openstack-nova13:25
*** eharney has joined #openstack-nova13:25
*** jpena|lunch is now known as jpena13:27
*** edmondsw_ is now known as edmondsw13:27
*** pchavva has quit IRC13:29
*** mdnadeem has joined #openstack-nova13:29
*** yamamoto has quit IRC13:29
*** sidx64 has quit IRC13:31
*** pchavva has joined #openstack-nova13:32
*** sidx64 has joined #openstack-nova13:32
*** mdnadeem_ has quit IRC13:32
*** sahid has quit IRC13:36
*** yamamoto has joined #openstack-nova13:40
*** derekh has joined #openstack-nova13:40
*** yamamoto has quit IRC13:44
*** sridharg has quit IRC13:44
*** burt has joined #openstack-nova13:44
alex_xu_edleafe: the only help I can give is to review efried's patch :)13:44
*** salv-orlando has joined #openstack-nova13:45
*** Zames has joined #openstack-nova13:46
*** awaugama has joined #openstack-nova13:46
sarSo I had an issue where i couldn't delete an instance attached to a previously migrated volume. I get an error where it says it can't find the volume id. Turns out it doesn't update the volume_id in the json stored in block_device_mapping during volume migration. Can someone help me verify if this can be considered a bug? See around line 5656 here:
sarShouldn't there be a line such as this? : new_cinfo['volume_id'] = save_volume_id13:49
*** crushil has joined #openstack-nova13:50
*** salv-orlando has quit IRC13:50
*** felipemonteiro_ has quit IRC13:51
*** amoralej|lunch is now known as amoralej13:52
*** Zames has quit IRC13:54
*** yamamoto has joined #openstack-nova13:54
*** yamamoto has quit IRC13:55
*** yamamoto has joined #openstack-nova13:55
*** yamamoto has quit IRC13:55
*** sridharg has joined #openstack-nova13:57
*** gbarros has joined #openstack-nova13:58
*** liverpooler has joined #openstack-nova14:00
*** esberglu has joined #openstack-nova14:00
edleafealex_xu_: should have an update soon. Until then, have at efried!14:01
*** liverpooler has quit IRC14:02
*** liverpooler has joined #openstack-nova14:02
mriedemgibi: i'm going through if you want to hold off on updating it14:04
*** shaohe_feng has quit IRC14:04
cdentthanks jaypipes for saying what you did on the low-level cache spec14:04
jaypipescdent: yw14:05
cdentI tried to read that whitepaper that's reference before I made a judgement and dissolved in a sea of acronyms14:06
cdentbut my gut reaction was "oh, you've got to be kidding me"14:06
*** cdent has quit IRC14:07
openstackgerritDan Smith proposed openstack/nova master: Add --by-service to discover_hosts
*** sidx64 has quit IRC14:10
*** sidx64 has joined #openstack-nova14:14
stephenfinjaypipes: Low-level cache spec?14:15
* stephenfin is intrigued14:15
*** gouthamr has quit IRC14:16
*** sahid has joined #openstack-nova14:18
stephenfinI change my mind14:18
dansmithI'm going tp propose a spec soon to let you reserve a single byte of physical memory14:18
*** kholkina has quit IRC14:18
dansmithhope that's cool14:18
jaypipesdansmith: totes. go for it.14:18
dansmithI've always been partial to memory location 0xdeadbeef14:18
mriedemjaypipes: just abandoned that spec - it was still targeting pike14:18
jaypipesdansmith: a single bit would be better, though.14:18
dansmithand I desire my byte to be stored there14:18
ShilpaSDstephenfin: Hi14:19
stephenfinShilpaSD: o/14:19
ShilpaSDstephenfin: Had one query on same topic what we discussed yesterday14:19
mnaserso i was working with the ODL folks and it looks like has their vif plugging.  i dug in deeper and it looks like the unplug operation in os_vif with ovs is noop (so odl never really sees the port unplugged to change state) and then when the server is started again, it expects a network-vif-plugged event which never comes because the port is already plugged based on odl's14:19
ShilpaSDstephenfin: Instead of doing changes in manager to update access URL, can we add the url in novncproxy_base_url14:19
mnaseras this is being backported, it's breaking branch by branch unfortunately14:20
ShilpaSDstephenfin: novncproxy_base_url=http://<ip address>/vnc.html?path=websocikfy14:20
mnaserunfortunately the port type is still 'ovs' when using ODL.. should the fix be making os_vif actually unplug things rather than noop?14:21
*** shaohe_feng has joined #openstack-nova14:21
jaypipesmnaser: I will take a look at it as soon as I'm done with the cyborg demo.14:21
mnaserjaypipes: cool, thank you, i spent a lot of time digging around so i can point to a few things i've seen14:22
Kevin_Zhengmriedem Hi saw you guys were talking about the quota issue yesterday, any conclusion?14:26
mriedemKevin_Zheng: no, just that it's still a problem14:26
openstackgerritBalazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider
gibimriedem: sorry14:26
mriedemefried: is there a spec or talk about adding a generation to ?14:26
mriedemgibi: !14:26
gibimriedem: I just saw your ping14:27
gibimriedem: I promise I will read your comments even if it is on older ps14:27
mriedemwell ok then14:27
*** salv-orlando has joined #openstack-nova14:27
mriedemefried: oh i guess there is an optional generation in PUT /allocations/{consumer_id} in 1.1214:28
mriedemso nvm14:28
gibimriedem: sorry. I will not be available until Monday (national holiday in Hungary) and I have to leave soon14:29
*** mlavalle has joined #openstack-nova14:29
Kevin_Zhengmriedm OK, I will be intrested if we dicide to fix it14:29
mriedemKevin_Zheng: melwitt sounded semi interested in fixing it, so probably want to talk to her14:29
Kevin_ZhengCool, good to know14:29
efriedmriedem: No, there's nothing.14:30
mriedemefried: so is wrong?14:30
*** moshele has quit IRC14:30
*** psachin has quit IRC14:30
Kevin_Zhenggibi, I have a quick question about versioned notification14:31
efriedIt's ignored14:31
efriedgeneration (Optional)bodyintegerA consistent view marker that assists with the management of concurrent resource provider updates. The value is ignored; it is present to preserve symmetry between read and write representations.14:32
mriedemoh gdi14:32
*** Tom-Tom has quit IRC14:32
mriedemok, well, probably going to be important when both nova and neutron are changing allocatoins for the same consumer14:32
*** alexchadin has quit IRC14:32
*** Tom-Tom has joined #openstack-nova14:32
Kevin_Zhenggibi, I saw that InstanceActionPayload is a parent object of alot other payload objects, and seems some of the child object also got version bump when the parent object got bumped?14:32
efriedmriedem: Yes.  We talked about it in Dublin.  On Wednesday IIRC.  And agreed to add a generation field to the allocations table.  jaypipes probably has more of it in his head.  Not sure who's on the hook to do the spec/work.14:33
mriedemKevin_Zheng: yes that's not new14:33
gibiKevin_Zheng: if you add something to the parent then that will appeare in the children payloads therefore you need the bump14:33
mriedemyou'll have to update all of the children14:33
Kevin_Zhengso many children14:33
mriedemclimb that mountain14:33
* mriedem inserts inspirational huawei training video speech14:34
gibiKevin_Zheng: the parent-child relationship is not visible in the serialized payload, as it only contains the child class name14:34
gibiKevin_Zheng: therefore the version of the child should reflect the overall structure14:34
Kevin_Zhenggibi ack14:35
gibiKevin_Zheng: dont worry I think the unit test will catch if you miss some of those children14:35
*** germs has joined #openstack-nova14:35
*** germs has quit IRC14:35
*** germs has joined #openstack-nova14:35
Kevin_Zhengmriedem you got trainning too, I thought it was just for us LOL14:35
gibiKevin_Zheng: as the signature of the children classes will change if you add a field to the parent14:36
Kevin_Zhenggibi, yeah thats true14:37
*** Tom-Tom has quit IRC14:37
openstackgerritEd Leafe proposed openstack/nova master: Add 'member_of' param to GET /allocation_candidates
edleafealex_xu_: ^^ now you can ignore efried14:37
stephenfinShilpaSD: and that works as expected, for both noVNC 0.6 and 1.0?14:38
*** sean-k-mooney has quit IRC14:38
ShilpaSDyes, if we manage at configuration level, no need to do changes at nova-compute14:38
stephenfinmriedem: Thoughts on that? ^14:39
mriedemstephenfin: huh?14:39
*** suresh12 has joined #openstack-nova14:39
stephenfinmriedem: This is for the breaking change in noVNC 1.0. Apparently we can set the config option to use 'vnc.html' with a parameter and this works with both noVNC 0.6 and 1.014:39
stephenfin'vnc.html' instead of 'vnc_auto.html' for 0.6 and 'vnc_lite.html' for 1.014:40
*** mvk has quit IRC14:40
mriedemwell that seems like the thing to do for the default then, but does that also work for 0.6?14:40
mriedemif that doesn't work for 0.6, then you'd be regressing the default for anyone <1.014:41
stephenfinmriedem: According to ShilpaSD, it does, yes14:41
stephenfinYup, same as changing the default to 'vnc_lite.html'14:41
mriedemsure seems fine then, accompanied with a release note that the default is changing probably14:41
stephenfinShilpaSD: If you fancy making that change to the default, we can see if DevStack is happy. That will ensure we're OK with 0.614:42
*** alex_xu has joined #openstack-nova14:42
*** sar has quit IRC14:43
*** danpawlik has quit IRC14:43
stephenfinShilpaSD: I already have a DevStack change up to bump noVNC 1.0. I can make this change 'Depends-on' your one14:43
*** mrjk__ has quit IRC14:44
*** mrjk has joined #openstack-nova14:46
ShilpaSDstephenfin: that will be great, but still one more query14:48
ShilpaSDstephenfin: /opt/stack/nova/nova/tests/functional/api_sample_tests/api_samples/os-remote-consoles/ also need to make that change? since functionaly TC using that14:49
*** felipemonteiro_ has joined #openstack-nova14:49
stephenfinShilpaSD: Already done
stephenfinWell, those are wrong. You can take that patch and fix it up, if you like14:50
stephenfinOr I'll rebase it onto whatever you do. You just need to modify nova/conf/ and add a release note14:50
*** felipemonteiro__ has joined #openstack-nova14:51
stephenfinlyarwood: Regarding, I think that's a bug in oslo_config.sphinxext. The rST is correct.14:51
stephenfinIf I were to guess, we're not doing a nested parse14:51
* gibi is going away back on Monday14:52
*** sean-k-mooney has joined #openstack-nova14:52
*** cdent has joined #openstack-nova14:52
*** shaohe_feng has quit IRC14:52
ShilpaSDstephenfin: thnak you for clarification, will get back to you on what action i am taking aginst this14:53
*** suresh12 has quit IRC14:54
sean-k-mooneymriedem: stephenfin just looking at the few runs of kuryr-kubernetes-tempest-daemon-octavia i have see so far against os-vif seam a little flaky.14:54
*** artom has joined #openstack-nova14:54
sean-k-mooneymriedem: stephenfin we may want to consider makeing it non voting if it contiues. ill keep an eye on it14:55
*** germs has quit IRC14:55
*** felipemonteiro_ has quit IRC14:55
*** yamamoto has joined #openstack-nova14:55
*** hongbin has joined #openstack-nova14:55
*** mvk has joined #openstack-nova14:55
cdentefried: re [t 1LbC] wasn't it to the consumer table?14:56
purplerbot<efried> mriedem: Yes.  We talked about it in Dublin.  On Wednesday IIRC.  And agreed to add a generation field to the allocations table.  jaypipes probably has more of it in his head.  Not sure who's on the hook to do the spec/work. [2018-03-14 14:33:08.634568] [n 1LbC]14:56
efriedcdent: Could be, sure.  I'm not very familiar with the tables related to allocations.14:56
efriedand don't remember the conversation exactly14:56
efriedbut I bet it's in the etherpad.14:56
sean-k-mooneystephenfin: mriedem  for example it failed becase the tempest regex elminated all tests and it did not publish results on at all.14:57
stephenfinsean-k-mooney: Indeed. There was a thing on openstack-dev about it earlier in the week. Apparently some neutron (?) change has broken it14:58
stephenfinsean-k-mooney: Agreed though. Let's keep an eye on it14:58
sean-k-mooneystephenfin: well those two need other work to be mergable first but i would prefer to make it non voteing instead of blocking other changes to os-vif that may be need in the future.14:59
*** chyka has joined #openstack-nova14:59
sean-k-mooneystephenfin: its not blocking anything currently hence lets wait and see. legacy-tempest-dsvm-nova-os-vif on the other had we might want to make voteing or rework for zuul v315:00
*** yamamoto has quit IRC15:02
*** mdnadeem has quit IRC15:02
Kevin_Zhengmriedem gibi thanks alot15:06
*** ayoung has quit IRC15:08
*** jbernard has quit IRC15:09
*** itlinux has joined #openstack-nova15:12
*** germs has joined #openstack-nova15:13
*** germs has quit IRC15:14
*** germs has joined #openstack-nova15:15
*** germs has quit IRC15:15
*** germs has joined #openstack-nova15:15
*** jbernard has joined #openstack-nova15:15
*** ragiman has quit IRC15:18
*** links has quit IRC15:18
*** ayoung has joined #openstack-nova15:20
openstackgerritSylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs
bauzasstephenfin: edleafe: jaypipes: dansmith: efried: you could be interested in (NUMA topology with RPs)15:25
bauzasI just have a shitty docs problem that I don't see why15:25
efriedbauzas: Added to review list.15:26
efriedbauzas: You're having trouble getting the doc to build?15:26
bauzasnvm, found the issue15:27
efriedyep, indent that graphic15:27
bauzasyeah, missed the code directive15:28
efried...and the next one.15:28
efriedOh, or that.15:28
efriedand an extra newline around L13815:28
efriedand 15015:29
efriedand 17815:29
efriedbauzas: With those fixed, it builds for me.15:29
efriedbauzas: btw, not sure how you're building locally, but I use this to cut build time down to a sub-second:15:30
*** tianhui has quit IRC15:30
efriedspecs ()15:30
efried    rele=${1:-rocky};15:30
efried    ( . .tox/docs/bin/activate || return;15:30
efried    set -x;15:30
efried    python build_sphinx -s doc/source/specs/$rele -c doc/source || return;15:30
efried    find doc/build/html -type f -name '*.html' | xargs realpath )15:30
stephenfinbauzas: Also, single-backticks aren't really valid rST. They mean default role which just happens to be italics in current Sphinx. No reason that won't change going forward though (they do tend to break stuff often)15:30
stephenfinBut that's a big nit :) Also placed on my review queue15:30
bauzasthanks both of you folks15:32
openstackgerritChris Dent proposed openstack/nova-specs master: Spec for isolating configuration of placement database
bauzasstephenfin: so, for targeting links, you would recommend double-backticks ?15:32
*** Tom-Tom has joined #openstack-nova15:33
*** esberglu has quit IRC15:33
stephenfinbauzas: What do you mean?15:33
bauzas(16:30:24) stephenfin: bauzas: Also, single-backticks aren't really valid rST. T15:33
stephenfinSomething like `xyz`_ or :role:`test` is valid. What's not valid is `xyz`15:33
bauzasI used single backticks for explicit targeting15:33
bauzasstephenfin: yeah, so I thought I did that everywhere, were have you seen a single-backtick without a link ?15:34
*** felipemonteiro__ has quit IRC15:34
*** moshele has joined #openstack-nova15:34
openstackgerritSylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs
*** felipemonteiro__ has joined #openstack-nova15:35
stephenfinbauzas: Line 13115:35
stephenfin257 too15:35
dansmithedleafe: hey, I just noticed this went in and I'm a tad confused by it:
stephenfinthat's about it though, actually :) Like I said, it's the nitiest of nits15:36
openstackgerritSylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs
edleafedansmith: on a call now15:36
dansmithedleafe: the uuid warning from o.vo has been around for a long time now and it seems like this is a bit of a heavy-handed way to resolve a warning15:36
dansmithedleafe: ack15:36
dansmithgibi: stephenfin ^15:36
stephenfindansmith: Heavy handed, perhaps, but it did resolve the issue and made sense to me15:37
*** Tom-Tom has quit IRC15:38
dansmithstephenfin: it resolves the warning in unit tests by storing an obfuscated uuid in the database which would make it really hard to examine what is going on15:38
dansmiththat seems not sensical to me15:38
dansmithand would break anyone who was halfway through a mapping and then walked over this15:39
dansmithlike someone doing an FFU15:39
dansmithand.. instance uuid _is_ a uuid, so if there's a warning about the format of it, maybe the warning is wrong or something else needs to change15:40
dansmithbut munging the values (which I'm not even sure is legit uuid behavior) seems kinda silly15:40
openstackgerritMerged openstack/nova master: Updated from global requirements
dansmithif you're not doing the rotation on all uuids then you're increasing your chances for collision, and since this doesn't (actual instance mappings have unrotated uuids, only the marker is rotated)15:42
*** lpetrut has quit IRC15:42
*** sidx64 has quit IRC15:43
dansmithsince this is munging data in the database I think I'm going to exercise fast revert15:43
*** tblakes has joined #openstack-nova15:44
stephenfindansmith: Gimme a sec. I'm trying to re-review this and see what my reasoning for letting it in was15:44
cdentbauzas: I may have missed some discussion in Dublin, but what's the different between a NUMA_CORE and a VCPU and why keep track of both of them?15:44
*** tblakes_ has joined #openstack-nova15:46
bauzascdent: good question15:46
bauzascdent: see my alternatives section15:46
bauzasif we shard VCPUs across NUMA nodes, then an instance spreading its cores amongst multilple NUMA nodes could have problems15:47
dansmithmriedem: I forget, in my revert patch should I explain the problems I think there are or just do a straight revert and then discuss on a re-propose15:47
cdentbauzas: yeah, saw that, but that doesn't really answer the question. If you are counting NUMA_CORES (as a separate thing) why would you record VCPU at all?15:47
bauzasdansmith: fast-revert and discuss later IMO15:47
dansmithbauzas: okay that's what I was thinking too, but it's been a while15:47
bauzascdent: because operators that wouldn't care about NUMA things wouldn't then need to use a NUMA specific RC15:47
*** esberglu has joined #openstack-nova15:48
*** hemna_ has joined #openstack-nova15:48
bauzasdansmith: stephenfin: so I need a bit of context in order to chime in15:48
cdentI'll have to think on that some more. Having to track the same thing multiple times doesn't seem simple to me. It may, however, be that simple is not possible.15:49
bauzascdent: np, I'll be on PTO till thurs starting EOB15:49
dansmithbauzas: after further examination, rotating one uuid out of a set is not a legit thing to do to a uuid and maintain uniqueness, so I think it's flawed just on that basis alone15:49
bauzascdent: so take your time15:49
bauzascdent: that spec is maybe a strawman15:50
*** tblakes has quit IRC15:50
dansmithi.e. if your key space is 0-4, and you have keys 1, 2, 3, you can't just rotate 2, or you will end up with a conflict15:50
bauzascdent: so I'm open to alternatives15:50
dansmithyou have to rotate them all equally15:50
*** tblakes_ is now known as tblakes15:50
stephenfindansmith: So my understanding of this was that we're creating something akin to a fake InstanceMapping so we could keep track of where we were in the process15:50
bauzasdansmith: right, I don't like touching things15:50
dansmithstephenfin: that's the marker functionality, yes15:50
bauzasbut that's not the only marker we had, right?15:51
bauzaslike, I provided a marker IIRC for the RequestSpec migration thing15:51
bauzasnow the code is gone, but lemme see how I did that and if that can help15:51
stephenfindansmith: Right, so if were re-using the UUID of an existing instance we were going to bump into the UNIQUE constraint, as noted on line 1127 there15:51
dansmithbauzas: right, and also, this is storing an _actual_ uuid in this field15:51
dansmithbauzas: so if the warning is complaining about it, then it's wrong15:52
stephenfinand to work around that, alaski put in a patch that munged the instance UUID to avoid said constraint15:52
stephenfindansmith: UUIDs need to have hyphens in them, no?15:52
*** tblakes has quit IRC15:52
bauzasaaaaah I see15:52
bauzasstephenfin: the RFC doesn't really mention that IIRC15:53
bauzasit's just a python uuid thing IIRC15:53
dansmithstephenfin: ah, I see what's going on15:53
dansmithstephenfin: it does't change the fact that we're rotating one uuid in a given namespace, and storing it in the DB, just to get past a uuid format warning15:54
mriedemdansmith: i always try to explain why i'm reverting something15:54
mriedembecause the later patch might never come15:54
dansmithmriedem: okay, I just didn't want to argue over the rationale and delay the revert, but fair enough15:54
stephenfindansmith: It seemed no worse than stripping out the spaces. It was still very much a reversible operation15:54
stephenfinHowever, you're suggesting that it's not actually valid thing to do. I didn't know that and that makes a revert the right thing to do, if so15:55
dansmithstephenfin: it's completely worse because (a) it could create a conflict, (b) it breaks the format that people in the middle of this operation may have already stored and (c) it's much harder to manually inspect if there is a problem15:55
cfriesencdent: for the VCPU vs NUMA_CORES thing, also bear in mind the proposed specs for shared/dedicated instances on the same host, and even on the same instance, as well as sahid's spec to run the emulator threads on a separate pool of host pCPUs15:55
stephenfinAre UUID conflicts a thing? Aren't there, like, a bajillion permutations?15:55
cdentcfriesen: yeah, it makes my head swim15:56
cfriesencdent: make that shared/dedicated vCPUs within the same instance15:56
bauzasokay, so I marked the last Requestspec I checked by using a NULL UUID for the instance
stephenfinThat was noted in the commit message and I agreed with it
dansmithstephenfin: but it's wrong15:56
bauzasnow, I need to understand why the marker is different for instance_mappings15:56
cfriesencdent: if we could figure out an efficient generic way to do dynamic allocation of pCPUs for shared vs dedicated it would simplify things, but it seems too complicated to do generically15:56
bauzascfriesen: context is
openstackgerritDan Smith proposed openstack/nova master: Revert "Make the InstanceMapping marker UUID-like"
*** yamamoto has joined #openstack-nova15:58
dansmithbauzas: mriedem stephenfin ^15:58
bauzascfriesen: cdent: what I'd like is to keep the root RP responsible for the general resource classes for upgrade and consistency purposes, and just add NUMA-specific resources to the children15:58
bauzasthe fact that the operator has to handle flavors accordingly (ie. having a NUMA_CORES value that matches with the flavor VCPU) is something important to note, but not a big deal15:59
*** lajoskatona has quit IRC16:00
*** felipemonteiro_ has joined #openstack-nova16:01
*** felipemonteiro_ has quit IRC16:02
*** suresh12 has joined #openstack-nova16:02
bauzasstephenfin: see
*** yamahata has joined #openstack-nova16:03
stephenfinbauzas: Good to know :)16:03
*** felipemonteiro_ has joined #openstack-nova16:03
*** diga has joined #openstack-nova16:04
*** yamamoto has quit IRC16:04
cfriesenbauzas: so what would the extra-specs look like?  would we drop the existing "dedicated" cpu policy?16:04
*** arvindn05 has quit IRC16:04
*** felipemonteiro__ has quit IRC16:04
dansmithstephenfin: wanna discuss why #1 is an issue?16:04
cfriesenbauzas: and what about the "isolate" hyperthreading policy, how would we handle the sibling LCPUs?16:05
dansmithor are you saying it's so unlikely that you don't care?16:05
bauzascfriesen: I imagine some time being where two things would continue to work16:05
bauzascfriesen: I'm not trying to boil the ocean and have all the NUMA policies be done in Placement16:05
*** tbachman has joined #openstack-nova16:06
stephenfindansmith: Sure. I'd been going under the assumption that UUID conflicts were night-on impossible. Shuffling bits around on an existing UUID wouldn't change that16:06
*** felipemonteiro__ has joined #openstack-nova16:06
*** Tom-Tom has joined #openstack-nova16:06
* edleafe is off the call and reading back16:06
cfriesenbauzas: agreed, I just don't have a good picture of what the extra specs would look like when asking for dedicated cpus16:07
dansmithstephenfin: "collisions can occur only when an implementation varies from the standards, either inadvertently or intentionally"16:07
*** suresh12 has quit IRC16:07
dansmithstephenfin: mutating one uuid (even predictably) within a given namespace of others that are not so mutated would amount to a different implementation right?16:07
bauzasdansmith: right, like I said, nothing in the RFC tells "-" is mandatory AFAIK16:07
dansmithbauzas: well, there's also that16:07
dansmithstephenfin: uuids boil down to a very large number space16:08
edleafedansmith: so is your main concern the possibility of collisions?16:08
*** arvindn05 has joined #openstack-nova16:08
dansmithstephenfin: if you reduce that to a human-conceivable range and then populate it with some values, rotating one within the space, you can easily cause a conflict16:08
dansmithedleafe: my main concerns are in the revert.. collisions are one aspect16:08
bauzascfriesen: from
dansmithedleafe: totally agree it's very unlikely but it's fundamentally a wrong thing to do to mutate one like that16:09
*** felipemonteiro_ has quit IRC16:09
arvindn05mriedem: wanted to discuss your review on Are you going to be online an hr from now?16:10
edleafedansmith: just to be clear: it wasn't to pretty up the unit tests. When o.vo switches from warning to raising, it will break this code16:10
stephenfindansmith: I imagine it would, yes16:10
edleafeAnd adding spaces to a UUID is worse munging, IMO16:11
stephenfinand I certainly can't prove it wouldn't16:11
mriedemarvindn05: yes16:11
stephenfinYeah, that was my thinking ^16:11
stephenfinIf we're going to munge, do it right16:11
*** Tom-Tom has quit IRC16:11
dansmithedleafe: which may never happen, if you look at the discussion around doing the uuid format checking a couple years ago..16:11
stephenfinbauzas: What was this about using a null UUID as the marker?16:11
arvindn05mriedem: thanks!16:11
dansmithstephenfin: edleafe: but changing dashes to spaces doesn't change the unique elements of the uuid, it only changes the uniqueness of the string as a SQL hack16:11
bauzasstephenfin: I was walking over the RequestSpec tablre16:12
dansmithusing the null uuid is a completely legit way to do this, as bauzas did for reqspec16:12
dansmithbut it doesn't help as much in this case because of where the constraints lie16:12
cfriesenbauzas: I understand the current extra specs, just wasn't sure what it would look like with the placement stuff added to it.16:13
*** tbachman has quit IRC16:13
openstackgerritChris Dent proposed openstack/nova-specs master: Spec for isolating configuration of placement database
bauzasdansmith: yeah, I'm trying to load again the instance mapping context in mind16:15
*** r-daneel has joined #openstack-nova16:15
bauzasand why it has to be different from my walkthrough over reqspec16:15
dansmithbauzas: instance mapping is storing the uuids of the instances, so it can't store a duplicate one since there is a unique constraint on that column16:15
* bauzas looks at code16:15
dansmiththere's no room inside it to stash the uuid elsewhere and use the null uuid as the marker key16:16
bauzasdisclaimer: I have goblins at home that make noise, I need to take my Bose headset16:16
*** zhaochao has quit IRC16:18
bauzasdansmith: mmmm, I was just considering the Nul UUID for storing the last RequestSpec record I was walking thru16:20
*** felipemonteiro__ has quit IRC16:20
bauzasnow, I'm looking at instance_mappings16:20
*** udesale has joined #openstack-nova16:20
dansmithbauzas: you were storing the instance uuid inside a blob field, which was not required to be unique (nor would be) and thus it didn't conflict with the actual reqspec for the instance16:22
*** diga has quit IRC16:22
dansmithbauzas: and using the nil uuid to find the marker each time16:22
bauzasyeah, also because I was wrong 5 mins ago16:22
dansmithin instance mapping, we can't store the instance's uuid exactly again because it's a UC field16:23
bauzasI wasn't walking over reqspec, but rather over the instances table16:23
*** Tom-Tom has joined #openstack-nova16:23
bauzasdansmith: I wasn't storing the instance field, I was storing the request spec of the last instance I checkedc16:24
*** tbachman has joined #openstack-nova16:24
bauzasnow, I'm looking at the instance mappings migration script16:24
*** tbachman has quit IRC16:24
dansmithwe could have used the nil uuid here and then stashed the instance's real uuid in the project field or something like that, but that would be rather nasty as well and could have affected runtime in other ways16:24
dansmithand, this is done and in people's systems16:25
*** udesale has quit IRC16:26
bauzasnow the thing is done16:26
cdentedleafe: is member_of is a state of ready for review?16:26
bauzasgibi: so, I just saw
*** Tom-Tom has quit IRC16:28
bauzasgibi: like I said earlier, I bet that valid UUIDs with spaces instead of hypens are valid per-RFC16:28
mriedemi believe the gibster is currently celebrating the national holiday of the rubik's cube16:29
*** andreas_s has quit IRC16:29
edleafecdent: yep16:29
melwittdansmith, mriedem, tssurya: I could use your eyeballs on this cells session recap to add/correct anything I might have missed before I send it to the dev ML
dansmithbauzas: that's really unfortunate16:29
*** andreas_s has joined #openstack-nova16:30
cdentthanks edleafe16:30
*** AlexeyAbashkin has quit IRC16:30
bauzasdansmith: I need some tests on my box16:30
bauzasdansmith: but I think o.vo badly coerces, that's it16:31
dansmithbauzas: hmm?16:31
dansmithbauzas: it should only be emitting a warning (now error) if the format doesn't match16:31
dansmithbauzas: making it required formatting would be breaking our RPC API16:31
bauzasdansmith: I'm just saying that '52ec5cae 1654 42ad bc38 ebab73fbb161' is a valid UUID16:32
bauzasdansmith: so o.vo shouldn't complain at all16:32
dansmithbauzas: ah, about that I seee16:32
dansmithbauzas: well, I was against enforcing any one of the many ways to write a uuid in the first place16:33
dansmiththe warning was the compromise16:33
dansmithmelwitt: looks okay to me16:33
melwittcool, thanks16:33
bauzasdansmith: right, I remember the early and shiny days of nova/objects/ and the UUID() field type :)16:33
tssuryamelwitt: did a small change, otherwise looks good to me16:34
melwittsweet, thanks16:35
bauzasdansmith: stephenfin: gibi: so I'm wrapping my head around
bauzasbecause this is wrong16:36
dansmithit's opinionated16:36
stephenfinbauzas: I think the RFC is wrong16:37
bauzasit says 16 octets, period.16:37
dansmithbauzas: note it allows removing the dashes :)16:37
dansmithbauzas: because it thinks that's okay :)16:37
dansmithwhich we could also have done16:37
bauzaswe could have done many things16:39
stephenfinbauzas: It didn't matter much before anyway since we were undoing it. This will only start to bite us if/when o.vo decides to make that warning an error
bauzasbut I'm trying to see how we could better coerce in o.vo so that would make both not changing the DB, and make edleafe and stephenfin happy16:40
edleafebauzas: I really don't care16:40
dansmithstephenfin: that would (a) be a change to our (and others') RPC APIs, but also (b) we could easily handle this in _from_db_obj(), or by doing the migration process with the low-level routines instead of objects16:40
edleafeI was just trying to fix a potential issue16:40
bauzasI'm litterally 20 mins away from a long holiday period16:40
bauzaswould those 20 mins well spent in fixing that then ?16:41
edleafebut spaces in a UUID are not valid. Removing the dashes is fine16:41
*** crushil has quit IRC16:41
bauzas c'on16:41
bauzasit's a *string*16:41
dansmitha UUID is a number16:41
dansmiththe string representation of it can be many things16:41
dansmithmicrosoft encloses them in {} to make them stand out16:41
bauzashow the string translates to an hex is something fine even with spaces16:41
*** gjayavelu has joined #openstack-nova16:42
*** yamahata has quit IRC16:42
edleafehey, I wanted to store all uuids are 128-bit integers, but got out-voted by the human-readable people16:42
*** tbachman has joined #openstack-nova16:43
bauzasedleafe: it's comprehensive16:43
bauzasedleafe: but the o.vo coercing method shouldn't care at all about the formatting16:43
bauzasit should just assume Good Faith (c)16:44
cfriesenon a totally different topic...does nova wait to ensure vifs are actually plugged when doing a live migration?  I see it calling self.virtapi.wait_for_instance_event() on instance spawn and cold migration, but not for live.  I assume the flow is somewhat different?16:46
*** janki has quit IRC16:47
stephenfinbauzas: Before you go, fancy pushing these two patches through?
stephenfinAdditional shuffling things around/adding docstring patches16:47
bauzasmy review stats are poor this week16:47
bauzasthanks to the NUMA spec16:47
*** tbachman has quit IRC16:48
*** tbachman has joined #openstack-nova16:48
bauzas(and the f*** tax-credit document I had to write)16:48
*** andreas_s has quit IRC16:48
stephenfinbauzas: No better time, in that case16:48
bauzas "When a string of hex digits is given, curly braces, hyphens, and a URN prefix are all optional.'16:48
bauzasso, yeah, really the space carries a lot more, but meh16:49
dansmithtssurya: ah, yeah, kudos for spotting it in the initial patch :)16:49
*** andreas_s has joined #openstack-nova16:50
bauzasdansmith: planning to do a Cells meeting in 10 mins ?16:50
bauzascool, I can attend it for once \o/16:50
dansmithbauzas: I thought you were leaving in 20 minutes? :)16:50
bauzasfor the first time since 6 months I can attend a cells meeting, I won't skip it16:51
dansmithheh okay16:51
bauzasif I need to leave, I'll16:51
dansmithbauzas: you will leave only if I dismiss you!16:51
bauzasfine, I'll tell my children I can't go to Disneyland because of you16:52
* dansmith nods16:52
bauzasthey'll understaznd16:52
dansmithdansmith -- keeping children from happiness since 198116:52
*** tbachman has quit IRC16:53
*** suresh12 has joined #openstack-nova16:54
*** andreas_s has quit IRC16:54
openstackgerritEric Berglund proposed openstack/nova master: PowerVM Driver: Snapshot
openstackgerritDan Smith proposed openstack/nova master: Add --by-service to discover_hosts
*** yamamoto has joined #openstack-nova16:59
*** suresh12 has quit IRC17:00
*** suresh12 has joined #openstack-nova17:00
dansmithmelwitt: mriedem tssurya bauzas cells meeting?17:01
*** sapd_ has joined #openstack-nova17:01
mriedemmelwitt: comments in your etherpad17:03
*** suresh12 has quit IRC17:03
*** Tom-Tom has joined #openstack-nova17:03
*** jamesdenton has joined #openstack-nova17:04
*** sapd has quit IRC17:05
*** yamamoto has quit IRC17:05
openstackgerritChris Dent proposed openstack/nova master: DNM: Demo code for microversion parse extraction
arvindn05mriedem: i've added comments to the review17:06
cdentjaypipes: if you like this (multi workers for scheduler) can you kick it in. Seems like one of those things we'd like to exercise for as long as possible17:07
arvindn05mriedem: i think the only change would be to remove the workitem for ImageExtraSpecsFilter17:07
stephenfinmriedem: Gerrit keeps scrolling up when I try to reply to comments. Did you have a workaround for that?17:08
*** ralonsoh has quit IRC17:08
*** Tom-Tom has quit IRC17:08
mriedemstephenfin: i've noticed that today again too - the fix used to be to change the render setting to 'slow'17:08
mriedemarvindn05: in the cells meeting atm17:08
*** suresh12 has joined #openstack-nova17:11
jaypipescdent: done17:12
*** claudiub has joined #openstack-nova17:14
*** yamahata has joined #openstack-nova17:15
*** gbarros has quit IRC17:16
cfriesencdent: jaypipes: I'm nervous about defaulting to multiple workers...seems like a good way to hit races even with placement17:17
*** gbarros has joined #openstack-nova17:17
*** suresh12 has quit IRC17:17
cdentcfriesen: a) how?, b) that's why we're merging early17:17
*** pcaruana has quit IRC17:18
openstackgerritStephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec
cfriesencdent: numa resources aren't allocated until you actually hit the compute node, so can fail late.   Also, do we model server group policies in placement currently?17:19
cfriesencdent: in order to fix server group policy stuff we had to serialize scheduling of instances in the same server group, otherwise we hit races even with just a single scheduler worker.17:19
jaypipescfriesen: no, we do not model server group policies in placement.17:21
*** ayoung has quit IRC17:21
*** gbarros has quit IRC17:21
cfriesencdent: jaypipes: the problem with server groups is that the group membership isn't updated until the instance actually hits the node, so there's a big window from when the scheduler made the decision until the membership changes17:21
cfriesen(in the DB)17:22
*** moshele has quit IRC17:22
cfriesensorry, not group membership but the list of compute nodes being used by the group members17:22
jaypipescfriesen: yes, that's group membership.17:22
*** moshele has joined #openstack-nova17:23
jaypipescfriesen: so because of a crappy affinity implementation and crappy numa resource tracking, we'll continue to slow down the rest of the world...17:23
jaypipescfriesen: can't we tell folks using that functionality to run a single worker?17:24
sean-k-mooneycfriesen: defulting to muliple works has other issue. like we used to hit the db max conncetion limits because several service in openstack defualted to use multiple works for things and defulted to 1 worker per cpu.17:24
cfriesenjaypipes: just pointing out that this could cause unexpected races if it's enabled by default.  We might want to put something in the release notes.17:24
cdentalso, if the aforementioned serialization is already present, will it help?17:25
*** Swami has joined #openstack-nova17:25
jaypipessean-k-mooney: that's unrelated.17:25
cfriesencdent: I don't think that serialization is upstream yet.  How would we serialize across multiple workers?17:25
*** moshele has quit IRC17:25
sean-k-mooneyjaypipes: proably i am just skimming the scoll back now17:25
cdentcfriesen: ah, sorry, I had misunderstood which "we" you meant :)17:26
sean-k-mooneyi was in meeting all day up until this point17:26
*** gbarros has joined #openstack-nova17:26
jaypipescfriesen: a single nova boot request is only handled by a single worker, no? the case you're worrying about is multiple clients calling nova boot with the same server group.17:27
jaypipescfriesen: as for the NUMA issues, I don't think that changing to multiple workers will make a difference in the number of retries that happen.17:27
sean-k-mooneyjaypipes: or perhaps the nova multi boot support we you say boot x instance of y flavour on network z ?17:27
jaypipessean-k-mooney: I don't understand how that's relevant?17:28
jaypipessean-k-mooney: that would be handled in a single thread.17:28
jaypipessean-k-mooney: the only situation cfriesen is worried about is when multiple nova boot requests involving the same server group were executed simultaneously.17:29
*** awaugama_ has joined #openstack-nova17:29
sean-k-mooneyjaypipes: i was wondering if that was a single boot request form the api point of view or x independet ones and a client feature17:29
jaypipessean-k-mooney: a single nova boot is a single thread of execution.17:29
sean-k-mooneyjaypipes: ya that makes sense. and nova boot --min 3 --max 3 --server-group... is considered a singel boot request17:30
sean-k-mooneyjaypipes: so the only race would be if two client tried to boot servers in the same group concureently without the server group already having running instances17:31
*** awaugama has quit IRC17:32
jaypipessean-k-mooney: yes, that is considered a single boot request.17:32
jaypipessean-k-mooney: almost. the only race is two clients concurrently attempting to add instances to the same server group (regardless of whether the server group has members). and that is a situation I find pretty rare and not worth shooting the rest of the world in the foot for.17:33
cfriesenjaypipes: the scenario I'm worried about is where two instances race to schedule and get put onto the same compute node, but then one of them claims a mix of resources that causes the other to fail it's resource allocation.  With a single sched worker this is less likely (though still possible).17:35
jaypipescfriesen: unless those are NUMA resources, it's not possible to do that.17:35
mriedemarvindn05: ok what's up17:36
cfriesenjaypipes: yes, numa resources.  CPUs, hugepages, PCI devices with strict affinity17:36
*** Tom-Tom has joined #openstack-nova17:37
mriedemarvindn05: oh i'll read your replies17:37
cfriesenI'm totally fine with the change to enable multiple workers, I just thing we might want to mention in the release notes that it could increase races in some cases.17:37
bauzasjaypipes: cfriesen: I'm totally up for deprecating y'know17:39
melwittmriedem: on this bp, when you say you'd like to see the proposed code for deleting a server, did you mean you want to wait for proposed code before approving the bp?
bauzasit creates more problems than it solves17:39
cfriesenjaypipes: for server groups there's also the case where you resize/migrate/evacuate instances in a server group at the same time, or at the same time a new instance is created.  I don't think we have late validation for all those scenarios.17:39
bauzasand when you say --min 1 --max 5, you'd expect that if you have room for 1, then only one would be created and 4 in ERROR, but that's not the case17:40
cfriesenbauzas: me too. :)  but last time I brought it up people wanted to keep it.17:40
mriedemmelwitt: kind of yeah17:40
bauzasbecause min/max are only relevant wrt quotas17:40
cfriesenbauzas: actually I'd expect that 1 would be created and the other 4 would be like they never existed17:40
arvindn05mriedem: thanks....any questions for me?17:40
bauzascfriesen: that's absolutely not the current behaviour17:40
melwittmriedem: okay, I'm cool with that. just wanted to make sure I understood17:40
cfriesenbauzas: I know, but it should be given how it's documented in the API17:41
mriedemarvindn05: not yet, still reading17:41
bauzasthe current behaviour is that if you asked for 10 instances but your quota only tells 2 left, then it will just check that17:41
*** tidwellr has joined #openstack-nova17:41
arvindn05mriedem: great....just let me know when you have questions.17:41
bauzasbut honestly, who cares ?17:41
mriedembauzas: heh "Error handling for multiple create is not as consistent as for single server create, and there is no guarantee that all the servers will be built. This call should generally be avoided in favor of clients doing direct individual server creates."17:41
mriedemso actually,17:41
*** Tom-Tom has quit IRC17:41
mriedemconcurrent creates is actually a problem for the affinity scenarios17:41
mriedembut not a single multi-create request17:41
mriedemdamned if you do, damned if yo udon't17:42
bauzasI'm just saying that the "minimum" value is useless17:42
bauzasyou ask for a max of 10, the scheduler will try to find you room for 10, or kick all of them17:42
mriedemmaybe we should amend that to say, "unless of course you're creating servers in a server group, then because of race issues we have'nt fixed yet, you should use multi-create"17:42
cfriesenbauzas: and it shouldn't kick all of them if some of them pass17:43
bauzascfriesen: I'm not sure17:43
bauzascfriesen: it's a capacity problem17:43
cfriesenbauzas: the user has said "give me as many instances as you can, up to a max of 10"17:43
bauzasand we shouldn't leak the capacity to the user17:43
mriedemhow is the min useless? give me at least 2, but 10 if you can17:43
bauzasmriedem: the "if you can" is only quota-wise17:43
cfriesenbauzas: no, it's in the boot request17:44
*** imacdonn has quit IRC17:44
bauzascfriesen: but the info we give to the scheduler is "how many the user asked"17:44
*** sree has quit IRC17:44
*** imacdonn has joined #openstack-nova17:44
bauzascfriesen: not "how many the user is agreeing to only have"17:44
mriedemalso depends on the port quota, don't forget17:44
cfriesenbauzas: I totally agree it's all messed up.  but the API describes it as the min/max nubmer of servers to be created17:45
bauzasthe API docs are wrong17:45
*** sree has joined #openstack-nova17:45
cfriesenthe API docs are by definition correct, the implementation is wrong. :)17:45
cfriesenthe user wants at least 2 and at most 10 instances.  If the quota is 4, we should give them 417:45
bauzasthat's an interesting PoV17:45
cfriesenIf we can only schedule 3, we should give them 317:45
mriedemi'm pretty sure we've said in the past that the docs don't define the api17:46
dansmithcfriesen: I think the actual behavior, which people may have built dependencies on, is the canonical thing17:46
mriedemthe current behavior defines the api17:46
dansmith(man we're dorks)17:46
mriedemremember the docs of yore for the compute api were written by people that didn't actually code up nova api17:46
*** moshele has joined #openstack-nova17:46
melwittI think we've said in the past that it makes more sense for the --min to be honored if we can but if we change that, we have to do it with a microversion17:46
mriedemand until sdague lifted them in tree they were all sorts of wrong17:46
mriedemand still are in parts17:47
*** dtantsur is now known as dtantsur|afk17:47
bauzasanyway, time to say goodbye17:47
cfriesenhave fun17:47
*** awaugama has joined #openstack-nova17:48
bauzas"adieu" has a very strong meaning of a final goodbye :)17:48
bauzasI just hope to be back in 5 days :)17:48
cfriesenau revoir17:48
*** tssurya has quit IRC17:48
cfriesenfor what it's worth, it seems odd to say that nobody can trust our published API specification, and instead they have to observe the current behaviour.17:49
*** sree has quit IRC17:50
cfriesenbut I get the argument about people already depending on the current behaviour17:50
*** itlinux has quit IRC17:50
*** awaugama_ has quit IRC17:50
*** sar has joined #openstack-nova17:52
*** claudiub|2 has joined #openstack-nova17:53
*** moshele has quit IRC17:54
melwittI wonder if it only behaves that way for quota. like, what happens if you do --min 2 --max 10 and there's only compute capacity for 2. I wonder if that would pass scheduling and give back only 2 instances17:54
dansmithcfriesen: I think it's odd to say that we'd jump through hoops which might be very difficult to implement something the way someone guessed it may have worked long ago17:54
dansmith...just because they put that in a doc17:54
mriedemarvindn05: ok thanks for the clarifications. replies inline for what i'd like to see changed, but it should be trivial17:55
*** claudiub has quit IRC17:55
mriedemcfriesen: if the docs are inaccurate, let's clean up the docs17:55
mriedemcoincidentally i was just looking at again17:55
openstackLaunchpad bug 1684261 in OpenStack Compute (nova) "AggregateImagePropertiesIsolation example doesn't actually indicate how it works" [Low,Confirmed]17:55
arvindn05mriedem: awesome...will review them and let you know17:56
mriedemwould be really nice if someone would take on fixing the doc for that filter17:57
*** wolverineav has joined #openstack-nova17:57
*** links has joined #openstack-nova17:57
*** itlinux has joined #openstack-nova17:58
*** suresh12 has joined #openstack-nova17:58
arvindn05mriedem: sure. Updating the docs should be straitforward...will look into it17:58
mriedemfamous, last, words17:59
arvindn05mriedem: :)17:59
cfriesenthere's a truth table for that filter in
cfriesenmight be handy to reference when reworking the docs18:00
melwittlet's put it in the docs!18:00
*** yamamoto has joined #openstack-nova18:01
*** jpena is now known as jpena|off18:01
melwittmriedem: thanks for the comments on the cells summary, updated it and will send it out18:02
*** lucasagomes is now known as lucas-afk18:02
*** wolverineav has quit IRC18:03
*** derekh has quit IRC18:04
openstackgerritChris Dent proposed openstack/nova master: Move placement exceptions into the placement package
openstackgerritChris Dent proposed openstack/nova master: Optional separate database for placement API
openstackgerritChris Dent proposed openstack/nova master: Isolate placement database config
*** gyee has joined #openstack-nova18:05
*** yamamoto has quit IRC18:06
*** mgoddard_ has quit IRC18:07
*** AlexeyAbashkin has joined #openstack-nova18:07
*** mrjk has quit IRC18:08
*** mrjk has joined #openstack-nova18:09
*** wolverineav has joined #openstack-nova18:09
*** harlowja has joined #openstack-nova18:11
*** AlexeyAbashkin has quit IRC18:11
*** gbarros has quit IRC18:12
*** felipemonteiro_ has joined #openstack-nova18:12
*** alex_xu has quit IRC18:12
mnaserjaypipes: don't want to bother you too much but if you have some free time about the vif plugging on reboot issue18:12
*** wolverineav has quit IRC18:14
*** felipemonteiro__ has joined #openstack-nova18:15
*** fragatina has quit IRC18:15
*** fragatina has joined #openstack-nova18:16
*** gbarros has joined #openstack-nova18:17
jaypipesmnaser: crap, sorry man. got distracted. looking now while tests are running locally.18:19
*** felipemonteiro_ has quit IRC18:19
mnaserjaypipes: np! i can get you a bit to what point i reached from my debugging18:19
jaypipessure thing.18:20
jaypipesgot for it.18:20
mnaserjaypipes: so which was merged in master and in the process of getting backported changed behavior (after another change) where now, nova expects a vif-network-plugged event even on reboots18:21
mnasernow, it looks like linuxbridge broke because it didn't send that event, so a little work around was added in that patch above to skip that.  now, opendaylight is broken because it doesn't send a notification either.18:21
*** sahid has quit IRC18:21
mnaserupon digging on *why* it doesn't send one, it looks like unplug with ovs on linux is a noop in os_vif18:22
mnaserwhich means that the port is never really unplugged in odl, so when it tries to 'plug' it, neutron never really does anything because the port state never changes, and the server times out booting because vif_plugging_timeout18:22
mnasernoop unplug here =
jaypipesmnaser: and this is only on hard reboot, yes?18:23
mnaserjaypipes: correct, but afaik a clean state is a hard reboot too in the nova codebase?18:23
mnaserclean start*18:23
mnaserso first time you start an instance, it'll be okay, but stop and start again will cause it to want a plugged event18:23
jaypipesmnaser: no. a clean start does not issue a call to unplug() in the os-vif API.18:23
mnaserjaypipes: right, but a clean start means the port is created the first time and the vif plugged event comes through fine, but when the instance is stopped, nova calls unplug (but nothing happens) and when it starts again, it waits for vif plugged event  (those patches changed that behaviour)18:24
mnaserthis was noticed when the tempest start_stop tests failed because the startup would time out18:25
mnaser (you can look at nova logs there, but pretty much test_stop_start_server / test_reboot_server_hard / etc are the ones that arent working now)18:26
mnaserand failure is .. "Details: (ServerActionsTestJSON:test_stop_start_server) Server da9cead9-c217-495e-97a8-65a6adacf37c failed to reach ACTIVE status and task state "None" within the required time (196 s). Current status: SHUTOFF. Current task state: powering-on." .. nova logs shows it timing out after 5 minutes18:26
jaypipesmnaser: k. just a minute. reading through these patches...18:28
mnasersure thing18:28
*** lpetrut has joined #openstack-nova18:28
*** felipemonteiro__ has quit IRC18:30
*** felipemonteiro__ has joined #openstack-nova18:30
jaypipesmnaser: ok, done. so are you suggesting the fix here is to add VIF_TYPE_OVS to line 5392 here?
mnaserjaypipes: that would be a fix, or os_vif *actually* unplugging things could be a fix too18:36
mnaseri do feel if that list starts growing, it might start become confusing for users :<18:36
jaypipesmnaser: agreed.18:36
jaypipesmnaser: for os-vif linux bridge, though, I'm not entirely sure what should be done on unplug...18:37
mnaserjaypipes: yeah.. that's beyond me, but for openvswitch, im not sure if there is a link state, or maybe just deleting the actual port if its 'unplugged'18:37
mnaserim sure there's a really good reason why it's not being deleted though, but i don't know why18:38
jaypipesmnaser: I'm also not sure whether that would *ensure* that a vif-unplugged *event* from Neutron would be received...18:38
jaypipessean-k-mooney: you still around?18:38
mnaserjaypipes: well, nova doesnt care about vif-unplugged afaik, issue is surrounding vif-plugged event18:39
mnaserjaypipes: based on my reading, the nova notifications are a hook to the db model in neutron when an object changes.  so if it goes from active => active, nothing is being updated and it never sent anything18:39
mnasernow, maybe neutron-openvswitch-agent does things differently to trigger a change, i'm not sure.18:40
mriedemlyarwood: melwitt: probably want to listen to ^18:40
jaypipesmnaser: not the neutron DB. the ovsdb...18:40
jaypipesmnaser: and that's only for OVS of course.18:40
melwittmriedem: thanks18:40
*** tbachman has joined #openstack-nova18:40
*** mvk has quit IRC18:41
mriedemadding VIF_TYPE_OVS to the blacklist would defeat the purpose of the change, which is apparently working for ovs18:41
mriedemwe use ovs in our normal gate jobs18:41
mriedembuilding a giant list of which vifs types we should not wait on is going to be shitty18:42
melwittyeah, we were under the impression that ovs *does* send events for the plug in the hard reboot case18:42
melwittand that it was only linuxbridge that doesn't18:42
mnaserright, but for context, i'm talking about the case when opendaylight is used here18:42
mriedemdoes that use vif type ovs?18:42
jaypipesmriedem: you mean like this?
jaypipesmriedem: welcome to our own private Idahell.18:43
mriedemjaypipes: heh shitty indeed18:43
mriedemcan i be keanu?18:43
melwittoriginally, the patch never waited for any events because we thought during a hard reboot, the neutron agent would never detect us doing a os-vif unplug/plug18:43
*** wolverineav has joined #openstack-nova18:43
mnaserso afaik odl just 'notices' ports appearing, and in neutron-server there is a web socket connection to odl which notifies it that a port is appeared, so it sets it to active18:43
melwittthen we added another patch to only not wait if linuxbridge because we thought ovs was detecting the unplug/plug18:44
mnaserand so what happens is that, after the hard reboot, the port is always still plugged and didnt get a state change, so nothing happens in neutron and no notification goes out18:44
jaypipesmnaser: wow. I had no idea there were hard-coded hooks in the neutron base db plugin thing. :(18:44
mnaserwell, the unplug in os_vif when using ovs literally does nothing18:44
mnaserjaypipes: yeah, TIL yesterday18:44
mnaseror YIL i guess18:44
mnaseri thought it was a bit more complex than that18:44
melwittyeah, but the plug? we asked people in #openstack-neutron and the consensus at the time was that the ovs agent would detect things and send events18:45
mnasernow question is does anyone know why os_vif_ovs does nothing when nova uses the 'unplug' method in it? win32 seems to remove the port from the bridge18:45
mriedemmelwitt: this isn't 'ovs' this,18:45
mriedemit's opendaylight18:45
melwittokay, so things are working fine for ovs then18:45
mriedemapparently same vif type,18:46
mnaser^ and that too18:46
mriedemdifferent backend implementation18:46
mriedemdoes the vif type == networking API or something?18:46
*** abalutoiu_ has joined #openstack-nova18:46
mriedemand odl implements the ovs api?18:46
melwittsorry, I got confused by the earlier mention of adding VIF_TYPE_OVS to the blacklist?18:46
melwittis that because opendaylight falls under that vif type?18:46
mriedemmelwitt: yes18:46
jaypipesmnaser: I'll be honest. I don't know why for OVS we don't do anything for OVS when !Windows.18:46
melwittokay, gotcha now18:46
mriedemif we had to add ovs to that blacklist, then there is really no point in even waiting for vif plug events during a hard reboot with the libvirt driver18:47
*** abalutoiu__ has joined #openstack-nova18:47
mnaserpretty much :\18:47
mnaserbecause i think if os_vif_ovs did an actual unplug, the state in neutron would be updated, and on the actual plug later, the state would be updated again and a notification goes out18:47
jaypipesmnaser: I'm going to git blame it... one sec18:47
mnaseri'm *assuming* neutron-openvswitch-agent magically realizes things have been unplugged and removes them18:47
mnaserjaypipes: i tried to git blame for a while and didn't get anywhere productive but my git-fu isn't strong18:48
*** tidwellr has quit IRC18:48
*** tidwellr has joined #openstack-nova18:48
*** mgoddard_ has joined #openstack-nova18:48
*** abalutoiu has quit IRC18:48
*** gbarros has quit IRC18:49
jaypipesmnaser: it was Rawlin Peters who added this code.18:49
mnaseragain it could be something odl should be handling, does nova contact neutron for the unplug at the api layer?18:49
jaypipesmnaser: I don't know Rawlin.18:49
mriedemmnaser: when we were talking about the differences between openvswith and linuxbridge agents here, sean-k-mooney was saying that ovs will send a notification which the neutron agent proxies to the server which is why ovs works and we don't miss plug events, but LB does polling and there could be a window where we miss it and then timeout waiting for an event that neutron never sends18:49
*** abalutoiu has joined #openstack-nova18:49
mriedemmnaser: no it's not a REST API call18:50
jaypipesperhaps we should ask danpb and sean-k-mooney since they approved this patch.18:50
mriedemit's a change on the host which the neutron agent routes up to neutron server18:50
sean-k-mooneyi saw my name18:50
*** abalutoiu_ has quit IRC18:50
mnaserok i see, so given that neutron agent doesnt exist on the host, it would never end up hitting neutron18:50
melwittmnaser: yeah, we only do a unplug and plug through os-vif, no calls to neutron (unless os-vif does any, which I don't think it does)18:50
mriedemmelwitt: it doesn't18:51
jaypipesmelwitt: it does not, no.18:51
mriedemit's like os-brick, just all host level stuff18:51
jaypipesmriedem: it's like os-brick, only it bricked the host.18:51
mnaserso maybe if we actually unplugged in os-vif, then it probably wont "hurt" the ovs-agent and things like odl would notice that the port is gone18:51
mnaserand then update neutron in their own little mechanisms18:51
sean-k-mooneyoh yes the neutron l2 agent subscribes to notification from the ovs db instead of polling so it gets notified when we do plug and unplug on a hard reboot18:51
*** Tom-Tom has joined #openstack-nova18:51
mnasersean-k-mooney: but os_vif unplug with ovs does nothing18:52
mnaserso im not sure how n-ovs-agent even knows something changed18:52
*** abalutoiu__ has quit IRC18:52
mriedemjaypipes: rimshot'ed18:53
sean-k-mooneymnaser: yes libvirt does the port plug on the ovs bridge in that case18:53
jaypipesmriedem: :)18:53
mnasersean-k-mooney: ok, so pretty much it doesn't know things were unplugged, but it knows when they're plugged18:53
melwitthow come odl doesn't have its own plugin for os-vif?18:53
sean-k-mooneyodl is not a network backend18:53
sean-k-mooneyodl mananges ovs18:53
jaypipesit's an SDN controller.18:53
sean-k-mooneysame for ovn18:53
melwittso odl manages ovs, but in the setup there are no ovs agents running on compute hosts18:54
mnasermelwitt: correct18:54
mriedemit's possible there is something in the port binding profile that tells us it's odl and not native ovs, but then again if we had to check that level of detail in the driver logic, it's going to suck18:55
mriedemgiven all of the different neutron backends18:55
sean-k-mooneyneutron is repsocible for managing l2 and up nova/os-vif handel l1 netowrking for instances18:55
melwittyeah, I didn't realize the ovs agent was optional if the backend is ovs18:55
mnaserbackend is odl* :)18:56
mriedemi didn't realize you could have different backends for the same vif type either18:56
sean-k-mooneymelwitt: well the reference implentation use the agent but, ovn,onos and odl can all be used instead of the agents18:56
melwittsean-k-mooney said odl isn't a network backend. sorry, I lack knowledge on these18:56
*** Tom-Tom has quit IRC18:56
melwittokay, so they replace the agents. thanks18:56
* mnaser is just slowly learning more about this type of stuff anyways18:56
sean-k-mooneymelwitt: for a neutron vif_type perspctive its nots18:57
*** itlinux_ has joined #openstack-nova18:57
*** itlinux has quit IRC18:57
*** ociuhandu has joined #openstack-nova18:57
sean-k-mooneyby the way i think there might be a way to get rid of the polling in the linux bridge agent if you use pyinotify but i have not run the idea past any of the neutron folks18:58
mriedemactually there isn't really anything in the port information that would help us distinguish odl vifs
sean-k-mooneymriedem: correct infact that is by design18:58
sean-k-mooneymriedem: if there was the tenant could also tell18:59
mriedemwelp, we should probably not wait for vif plugged events on hard reboot in the libvirt driver then18:59
mriedemif we can't determine accurately when that will work18:59
mriedemwhich means reverting all of these
mriedemdansmith: ^ fyi19:00
sean-k-mooneyi missed the start of this convo. are ye discussing the fact that odl says it is finished wiring up the port imediatly when you bind the port19:00
dansmithyeah I'm following19:00
dansmiththat does indeed suck19:00
mriedemsean-k-mooney: odl doesn't send the vif plugged event19:00
*** salv-orlando has quit IRC19:00
mnasermriedem: it does send it, but only when the port is first created19:01
mriedembut is the ovs vif type19:01
*** salv-orl_ has joined #openstack-nova19:01
mriedemmnaser: ok19:01
mnasermriedem: the port is never ever unplugged from the openvswitch port on shutdown (when nova calls unplug())19:01
sean-k-mooneymriedem: it send it when the port is bound not when the port is wired up19:01
mnaserso when the vm is started again, it wont send it, because the port is already uo19:01
*** fragatina has quit IRC19:01
melwittsean-k-mooney: we were trying to Do The Right Thing and wait for vif plug events when we unplug and replug vifs during a hard reboot. but that's messing up for odl because it doesn't send a plug event in that scenario19:01
melwittand by unplug and replug we mean through os-vif only19:01
sean-k-mooneymelwitt: to make that work with odl we would have to bind the port again19:01
mriedemand according to that other patch from lyarwood, there are other cases19:01
* melwitt nods19:02
mnaseryeah so that's gonna be a little list of 'things that don't send notifications'19:02
*** yamamoto has joined #openstack-nova19:02
mnaser(i can imagine that you'll have other closed source sdns showing up soon too)19:02
mriedemsean-k-mooney: and that would be a bigger change, to re-bind the port19:02
sean-k-mooneymriedem: yes it would. we could explore that at some point but prehaps not now19:02
melwittyeah. I was wondering, is there a call to neutron we could make that would be mostly a no-op, not actually rebind it but update it to the same binding profile or something like that?19:03
*** ociuhandu has quit IRC19:03
melwittI don't know if neutron would emit an event if nothing changed19:03
mnasermelwitt: it would not, i looked into the logic, its a db hook that checks if states changed19:03
sean-k-mooneyas context the odl ml2 works the way it does today because odl does not have acess to rabitmq to be able to notify nova/neutron when its finished wireing things up19:03
melwittmnaser: ah, rats19:03
mnaserand it explicitely ignores noops19:04
*** crushil has joined #openstack-nova19:04
mriedemnova would have to unbind/bind or change the state of the binding (i think)19:05
mriedemand that all seems scary and hacky19:05
mnasermriedem: and not something you'd want to backport all the way to stable/ocata either19:05
melwittyeah, I don't want to do that19:05
sean-k-mooneymriedem: ya we would have to set the host_id in the binding profile to None then back to the hostname to get it to work19:05
mriedemwell the good news is the original change didn't make it to newton19:06
sean-k-mooneymriedem: maybe we can use the multiple port bindings neutron are adding to adress this in the future19:07
melwittI was thinking if we could not actually change it, update it to the same, and it would emit a notification, that might be cool. but I definitely don't want to unbind and rebind it just for this. guess it all depends on what's the impact of not waiting for the event. instance says ACTIVE when it might not actually have networking yet?19:07
mriedemmelwitt: yes19:07
*** yamamoto has quit IRC19:07
mriedemdoing that for hard reboot is less shitty than initial create, at least from our CI system19:07
sean-k-mooneymriedem: how would you feel about nova calling activate on the port binding again on a hard reboot?19:08
openstackgerritArvind Nadendla proposed openstack/nova-specs master: Support traits in Glance
mriedemany places in tempest that fallout as a result could wait on the port being ACTIVE in addition to the server being ACTIVE19:08
*** salv-orl_ has quit IRC19:08
mriedemsean-k-mooney: calling activate?19:08
*** salv-orlando has joined #openstack-nova19:08
melwittyeah, what's that do?19:08
sean-k-mooneyin the multiple port binding spec there is a new api call to activate a binding as part of live migration19:09
mriedemif it relies on the new porting binding api, that doesn't exist yet, and isn't backportable :)19:09
*** mvk has joined #openstack-nova19:09
mriedemsean-k-mooney: you just mean changing the binding status to 'active' right?19:09
mriedemsean-k-mooney: for a hard-reboot with a single host binding, it would already be active19:09
sean-k-mooneywell my taught would be allow that to tell neutron to validate its wired up again because did something(reboot) you man not have seen19:09
melwittthat sounds like what we want, but as mriedem said, not backportable19:10
mriedemmnaser: other good news is ocata isn't going anywhere :)
sean-k-mooneymriedem: yes it would. but since that api is not in a release yet we could ammend it to say calling activate on an active binding forces it to wire it up again or at least check its in the active state19:11
*** andreas_s has joined #openstack-nova19:11
mriedemsean-k-mooney: that's pretty hacky,19:11
melwittand validating it would cause an event to be sent?19:11
mriedemi think you'd have to deactivate the binding, and then activate it19:11
mriedem/ports/{id}/binding/{host}/action :)19:12
sean-k-mooneymelwitt: validating would sent the event if it actully changed something19:12
mriedemi can see it now19:12
melwittsean-k-mooney: yeah, I guess that seems similar to just unbinding it and binding it again19:12
mriedemso for present day options,19:12
mriedemi think we're talking revert right?19:12
sean-k-mooneymelwitt: yep it is19:12
mriedemback through queens and pike19:12
melwittI guess so. doesn't seem like we have any choice19:13
sean-k-mooneymriedem: revert waiting. yes i think so19:13
mriedemdansmith: jaypipes: how are you feeling about a revert19:13
mriedemi don't see other very good options19:13
dansmithyeah I mean, are there other options?19:13
mriedemnot outside of changing the port binding on reboot19:13
sean-k-mooneyi think we can/shoudl fix it for rocky so we can reenable waiting19:13
mriedemwhich is a bigger change19:13
melwittunbinding the port and binding it again19:14
mriedemsean-k-mooney: i think that would also require talking to the neutron team about whether or not that would do what we need19:14
*** idlemind has joined #openstack-nova19:14
mriedemi can only talk to miguel once per day, he said19:14
cfriesenare there any races that come from not waiting for vifs to actually be plugged?19:14
mriedemmnaser: is there a bug report for the ODL issue?19:15
mriedemcfriesen: yes19:15
sean-k-mooney"in general" un binding and binding the port "should" be safe but it can fail19:15
mriedemtempest races assuming it can ssh into the guest19:15
melwittcfriesen: instance can go to ACTIVE state without having networking yet19:15
mnasermriedem: in ODL world yes, but not in nova world, but i can make one19:15
mriedemmnaser: yes please for tracking19:15
melwittwell, for ssh I know from using devstack that the instance says ACTIVE before ssh daemon is ready19:15
sean-k-mooneycfriesen: odl has no mechanisium to notify nova today that they have been plugged19:15
melwittI thought19:15
sean-k-mooney* notify neutron19:16
mriedemmelwitt: if that were true,19:16
mriedemour CI would shit its pants daily19:16
sean-k-mooneymriedem: we get around that by a retry loop19:16
dansmithmriedem:  we retry waiting for ssh19:16
mriedemah yes19:16
dansmithbut that's a totally different problem than not having networking and DHCP ready during boot19:17
*** tssurya has joined #openstack-nova19:17
mriedemso we hide the poo19:17
*** gbarros has joined #openstack-nova19:17
dansmithit's totally legit to not expect the instance is ready when the state goes to active,19:17
dansmithit's not legit for the instance to boot up and expect networking but have none19:17
sean-k-mooneytempest waits for active then wiats for ping to work then ssh's i think19:17
dansmithand then tries ssh a few times with long timeouts19:17
mriedemsean-k-mooney: yes i forgot about the latter19:17
mnaseri mean, dhcp could be slow too, so it doesnt matter if internet is wired up..19:17
mnaserstart up of the OS could be slow19:18
mnaserunless you setup something inside the OS that tells nova the vm is ready, we'll always have ACTIVE vms that arent accessible19:18
melwittyeah, +1 dansmith. could this screw boot up entirely? if DHCP not ready while it's coming up? or will it always recover19:18
sean-k-mooneydansmith: active and ready in an ironic case is partcallarly different things. vms tend to be ready soon after they are active19:18
*** salv-orl_ has joined #openstack-nova19:18
dansmithmelwitt: yes if networking isn't running19:18
dansmithmelwitt: some OSes may try DHCP briefly, and if it gets none, then it never retries19:18
dansmithif you are a cloud image and don't know what kind of cloud you're on,19:19
melwittthat would majorly suck to just have a junk instance if this is hit19:19
dansmithyou really have to just poke around until you figure it out because it could be a variety of things19:19
mnaser...but then again this is something that has been in nova for 3-4 releases, so a fix would be nice, but i dont think its a issue thats happening often enough?19:19
*** salv-orlando has quit IRC19:19
sean-k-mooneymelwitt: also oftend cloud images will wait for cloud init to finish before starting ssh so that can take a while too19:19
mnaserconsidering the port is never really unplugged in ovs, most things are already wired up19:19
melwittI see19:19
dansmithmnaser: yeah less of a concern on reboot for that reason19:20
sean-k-mooneymnaser: on hard reboot it is unpluged19:20
sean-k-mooneymnaser: on soft reboot it is not19:20
*** esberglu has quit IRC19:20
mnasersean-k-mooney: but the unplug in os_vif seems to be noop from what i see (or maybe im misunderstanding things)19:20
mnasersean-k-mooney: nothing happens here...19:20
sean-k-mooneymnaser: that only because in that code path the plugin is done by libvirt19:20
*** tesseract has quit IRC19:20
mnaseroh okay sorry i'm following you now19:20
mnaserthe actual unplug happens by libvirt which is then picked up by n-ovs-agent19:21
*** Tom-Tom has joined #openstack-nova19:21
sean-k-mooneyyep when we do the domain destroy on hard reboot libvirt remove the interface form ovs and deletes teh tap19:21
mnaserok i understand now19:21
mnaserthank youy19:21
cfriesenmy organization has a local patch to make plug_vifs() optionally wait until the vifs are actually plugged in order to ensure that things work reliably during a live migration...we added that a long time ago and I was wondering if it's still needed, but it sounds like it is.19:22
sean-k-mooneycfriesen: when you say actully plugged what do you wait for19:22
dansmithwe can't know that the plumbing is done behind the neutron curtain without the event right?19:23
*** esberglu has joined #openstack-nova19:23
sean-k-mooneycfriesen: if its external-ids:iface-status=active in the ovs db we hard code that here
sean-k-mooneydansmith: technically for ovs and ovs agents only we shoudl be able to use external-ids:iface-status=active but i dont know for odl19:24
dansmithbut that's a very specific case19:24
sean-k-mooneydansmith: for lb or vpp or whatever else we have no way to tell19:24
dansmithfor that one type19:24
*** andreas_s has quit IRC19:25
mnasermriedem: i tried to write a basic description and referenced this conversation but yeah feel free to update it with any lacking details19:25
openstackLaunchpad bug 1755890 in OpenStack Compute (nova) "Instances fail to hard reboot when using OpenDaylight" [Undecided,New]19:25
cfriesensean-k-mooney: actually, never mind.  the only one with a useful query is for our custom networking thing, the others are assumed to always be up.  My bad.19:25
melwittI have to run for now but I'm cool with reverting those patches if that's the best option we have. bbl19:25
arvindn05mriedem: jaypipes: updated the spec based on comments Patch set 9 should address the issues
*** Tom-Tom has quit IRC19:26
sean-k-mooneycfriesen: the dpdk based titainium server vswitch i assuem19:27
cfriesenit'll query the vswitch API19:28
sean-k-mooneycfriesen: ya the issue is that bar seeting external-ids:iface-status=active which is an ovs agent specif thing there is no way to tell for ovs in general.19:29
sean-k-mooneywhen odl first intregrated support into the ml2 framework this gap was discussed back in icehouse but they had no way syncronis state in odl with neutron without a intoducing an sdn constrolller specific api, b having odl emit notifcation on the rabbitmq bus or be haveing the neutron server poll odl for state change19:31
*** suresh12 has quit IRC19:32
sean-k-mooneythe chose to do non of the above an just report vif_plugged on port binding instead and its been that way ever since19:32
mnasersean-k-mooney: maybe this is a networking-odl bug?  i know that right now it creates a websocket that listens for state updates19:32
openstackLaunchpad bug 1686023 in networking-odl "networking-odl dynamic port status update full support missing" [Low,In progress]19:33
mriedemjaypipes: replied in - need bauzas to probably elaborate at this point since i don't remember this being talked about at the ptg19:33
mnaseri guess they support DOWN => ACTIVE but not ACTIVE=>DOWN19:33
*** sridharg has quit IRC19:33
*** links has quit IRC19:34
mnaseri wonder if there's an easy way to let neutron know that the port has gone down in there19:34
mnaserand avoid the revert19:34
sean-k-mooneymnaser: well i think its a ml2 framework feature request. e.g. allow agentless backend notify neutron of state changes19:34
mriedemif we can blame this all on an incomplete neutron backend then that works for me19:35
sean-k-mooneymnaser: that bug seams to focus on admin state which is different19:35
*** moshele has joined #openstack-nova19:35
mnasersean-k-mooney: ever that little bit?  the bug talks about admin state but it seems to get the actual port state19:36
sean-k-mooneymnaser: yest that should adress this but it seams that odl is not detecting the removal of the port and readding it19:37
sean-k-mooneymnaser: perhaps the networking-odl ml2 driver is just not sending the notification to nova when it recives the notification from odl19:38
mriedemshouldn't the driver go through the normal notificatoin code in neutron?19:38
mnasersean-k-mooney: its certainly not sending it when its being unplugged, but it looks like the code to watch for state doesnt even update it to 'down' or 'unplugged' or whatever state it should be in neutron19:39
*** Tom-Tom has joined #openstack-nova19:39
sean-k-mooneymriedem: that notification code is triggered by port status updates on the rpc bus19:39
sean-k-mooneymriedem: these update from odl are from the websocket19:39
*** moshele has quit IRC19:40
*** itlinux_ has quit IRC19:40
sean-k-mooneymriedem: since we are not storing the state changes from in the db i dont think will be invoked19:41
*** moshele has joined #openstack-nova19:41
mnasersean-k-mooney: is there an example of a plugin that is storing/updating state changes in db which might be good to reference19:41
sean-k-mooneymaybe look at the ovs agent code.19:42
*** hemna_ has quit IRC19:43
mnaseri guess i can use that as a reference and try submitting something to networking-odl19:43
*** Tom-Tom has quit IRC19:43
*** pchavva has quit IRC19:44
sean-k-mooneyi dont think we actully need to store it in the db. we just need to send the notification to nova19:44
mnaseri guess one thing though is talking about the revert is... if this is going to affect other drivers too19:45
mnaseraka we fix odl but find 4 other vif types that are affected by that issue19:45
*** amoralej is now known as amoralej|off19:45
sean-k-mooneymnaser: we are already special casing lb and we need to skip waithing i think for sriov too so basically we can only wait for ovs with agents today19:46
sean-k-mooneymaybe we can wait for ovn? but not sure.19:46
*** mgoddard_ has quit IRC19:48
mriedemunless the port profile/details add some specific flag that says we can expect events on replug, i don't think it's safe to just have the blanket wait that we have today19:49
*** salv-orl_ has quit IRC19:50
openstackgerritEd Leafe proposed openstack/nova master: Add 'member_of' param to GET /allocation_candidates
edleafejaypipes: cdent: ^^ Addressed your comments19:50
*** salv-orlando has joined #openstack-nova19:50
sean-k-mooneymriedem: that is true. i think this is something that first needs to made a requirement for all neutron backend before nova can start to depend on it.19:50
sean-k-mooneymriedem: one option is add a nova config opetion to the compute node? we wait there correct? but not ideal19:51
*** awaugama has quit IRC19:52
*** suresh12 has joined #openstack-nova19:52
mriedemit's an option yes but not a good one imo19:52
mriedemwe already have vif_plugging_timeout and vif_plugging_is_fatal19:52
mriedemwe'd have to add a vif_plugging_reboot_if_hard_and_libvirt_and_thursday_timeout19:53
*** itlinux has joined #openstack-nova19:53
*** crushil has quit IRC19:53
*** sambetts is now known as sambetts|afk19:53
*** salv-orlando has quit IRC19:54
sean-k-mooneyhehe well it could be vif_wait_for_network but ya only ovs + reference agents could set it to true19:54
sean-k-mooneyit would have to default to false and proably should be deprecated from the start19:55
mnaseri guess a revert seems like the cleanest possible choice? :x19:56
sean-k-mooneymnaser: ya19:56
jaypipesedleafe: +2 from me.19:56
mnaseri'm not sure how a revert would happen because this seems to be 2 patches.  a patch that reverts both referencing both commits?19:56
mnaserunless someone else volunteers to do this19:56
*** suresh12 has quit IRC19:57
sean-k-mooneywhat were the 2 commits again. it will be next week if i do it at the earliest.19:57
mriedemi think it's just this one
mriedemthat's the one that adding the wait back in19:58
mnaserwhat about ?19:58
mriedemthat's what you want to happen, not wait19:58
mriedemthere were 3 changes:19:58
mriedem1. full hard reboot blasting everything away and re-plug; that caused issues because we were waiting for something that wouldn't happen19:59
mriedem2. to workaround ^, we stopped waiting19:59
mriedem3. because not waiting when it will work for some (ovs but not lb), the final patch was added which added the bridge conditional19:59
mriedembut as we've now seen, that doesn't work for ovs (odl)19:59
mriedemso revert #3 to go back to not waiting20:00
mriedemwhich is
sean-k-mooneymriedem: right skip waiting for lb and then larwood was extending it for sriov then odl came up20:00
*** suresh12 has joined #openstack-nova20:00
mriedemyeah so it's whack-a-mole at this point20:00
mriedemheh, we could just revert all of them20:01
mnaserso revert master and then cherry pick the reverts once they land?20:01
mriedemgoing back to
mriedemmnaser: cherry picking reverts is weird,20:01
sean-k-mooneymriedem: so one patch to just remove waithing on reboot and backporting that is proable the simpelst thing20:01
mriedemi'd just revert individually on all branches20:01
mnaserok i see20:01
*** artom has quit IRC20:01
sean-k-mooneymriedem: or that.20:01
mriedemi was never fully comfortable with
mriedemit was trying to fix a problem with encrypted volumes and overshot probably to include vifs20:02
mnaserrelated-bug or closes-bug to ?20:02
openstackLaunchpad bug 1755890 in OpenStack Compute (nova) "Instances fail to hard reboot when using OpenDaylight" [Undecided,New]20:02
mnaser(for the revert)20:02
*** yamamoto has joined #openstack-nova20:03
openstackgerritMohammed Naser proposed openstack/nova master: Revert "Refine waiting for vif plug events during _hard_reboot"
sean-k-mooneymriedem: we have to be carful not to reintroduce by reverting
openstackLaunchpad bug 1724573 in OpenStack Compute (nova) "encrypted volumes are directly attached to instances after a compute host reboot" [Medium,Fix released] - Assigned to Matthew Booth (mbooth-9)20:04
openstackgerritMohammed Naser proposed openstack/nova stable/queens: Revert "Refine waiting for vif plug events during _hard_reboot"
openstackgerritMohammed Naser proposed openstack/nova stable/pike: Revert "Refine waiting for vif plug events during _hard_reboot"
mnasercan someone 'block' that from merging20:05
mnaserdansmith: maybe remove your +W?20:05
sean-k-mooneymnaser: it has a zuul -2 because its in merge conflict20:05
mnasersean-k-mooney: figure just in case someone doesnt notice it :)20:06
mriedemsean-k-mooney: yeah if we reverted all the way, we'd have to do a partial revert of and only remove the changes for the vif plugging part20:07
*** yamamoto has quit IRC20:08
sean-k-mooneywell looking at i think it solve the imidiate odl issue at least for now20:08
sean-k-mooneythat seams like the minimal change at the cost of never waiting20:09
*** sidx64 has joined #openstack-nova20:10
openstackgerritMatt Riedemann proposed openstack/nova master: Make nova-cells-v1 run with neutron
mriedemif we wanted to go back to,20:14
mriedemwe could pass destroy_vifs=False through self.destroy -> self.cleanup20:14
*** Tom-Tom has joined #openstack-nova20:14
*** suresh12 has quit IRC20:14
mriedembecause that was the change in behavior in that patch that caused a lot of the trouble20:14
mriedemas i said, was meant to fix an encrypted volume issue and turned into "let's just fully destroy the damn thing and everything associated with it except disks"20:15
*** sidx64 has quit IRC20:15
*** suresh12 has joined #openstack-nova20:15
*** sidx64 has joined #openstack-nova20:16
*** Tom-Tom has quit IRC20:18
sean-k-mooneyim going to head off soon. FYI ill be online intermitely tommorw and then off until tuesday.20:21
*** mgoddard_ has joined #openstack-nova20:25
efriedYou don't like me anymore, jaypipes?20:31
dansmithefried: we took a vote20:32
dansmithsad to say, you're off the island20:32
efriedAnd edleafe won???20:32
efriedCome ON!20:32
* dansmith doesn't even know the context20:32
efriedI mean, he does have seniority.20:33
efriedSERIOUS seniority.20:33
jaypipesefried: your torch has been extinguished.20:33
dansmithyou must leave the tribal council area IMMEDIATELY20:34
jaypipesdansmith: edleafe has won the microversion battle being currently waged.20:34
* efried feels he is missing some pop culture reference, but suspect it has to do with survival shows.20:34
dansmithah heh20:34
efrieddansmith: It's not too late!  You can saaaaave me!20:34
jaypipesefried: that is correct. when a person is voted off the island in the survivor show, their torch is extinguished.20:35
dansmithI wouldn't mind seeing a fire making challenge between the two of you20:35
* efried whips out trusty Zippo20:35
jaypipesefried: or at least, that's the way it was the last time I saw that show, which would have been around 2005.20:35
*** jmlowe has joined #openstack-nova20:35
*** awaugama has joined #openstack-nova20:35
dansmithefried: nah, you have to use flint and coconut husk20:35
*** AlexeyAbashkin has joined #openstack-nova20:36
efriedor my high-velocity eyeballs?20:36
jaypipesdansmith: you can't make fried_rice without fire.20:36
* dansmith turns to the judges20:36
dansmithbah dum.20:36
openstackgerritMerged openstack/nova master: Revert "Make the InstanceMapping marker UUID-like"
* mriedem hands edleafe a single rose20:36
*** fragatina has joined #openstack-nova20:39
*** AlexeyAbashkin has quit IRC20:40
* edleafe blushes20:41
mriedemi assume edleafe is an avid bachelor(ette) watcher20:41
mriedemalthough it does come on pretty late...20:41
dansmithum, aren't we all?20:41
edleafeNah, no cable in this house20:41
mriedemit's not cable!20:41
dansmithit's not on cable dude :)20:41
mriedemrabbit ears20:41
edleafeguess I'll have to pick some up to see what all the fuss is about20:42
mriedemor, your texas-sized satellite in the backyard should pick it up20:42
edleafemaybe it's on hulu?20:42
mriedemidk, but i wouldn't advice trying to find it20:42
dansmithI bet it's on hulu,20:42
dansmithbut we're not serious about watching/liking it20:43
edleafeoh sure20:43
*** salv-orlando has joined #openstack-nova20:43
edleafeyou're just probably embarrassed to admit it20:43
edleafeit *is* on hulu20:44
edleafewait - season 22??20:44
dansmithoh boy, edleafe is in for a treat tonight20:44
dansmithoh yeah man, it's a major deal20:44
*** idlemind_ has joined #openstack-nova20:44
dansmiththat's why I figured it'd be on hulu.. they couldn't have a subscriber base without it20:44
*** idlemind_ has quit IRC20:44
*** idlemind has quit IRC20:44
edleafeoh, they have all sorts of trashy TV: the voice, top model, the kardashians20:45
edleafeI'm so culturally deprived20:45
efriedculturally depraved if you watch that crap.  Take your pick20:46
edleafeefried: good point20:47
*** pchavva has joined #openstack-nova20:48
*** eharney has quit IRC20:49
*** Tom-Tom has joined #openstack-nova20:50
openstackgerritMatt Riedemann proposed openstack/nova master: libvirt: handle DiskNotFound during update_available_resource
*** esberglu has quit IRC20:52
openstackgerritMatt Riedemann proposed openstack/nova master: libvirt: handle DiskNotFound during update_available_resource
*** Tom-Tom has quit IRC20:55
*** esberglu_ has joined #openstack-nova20:55
*** esberglu_ has quit IRC20:56
*** moshele has quit IRC20:56
*** esberglu_ has joined #openstack-nova20:56
cfriesengot something wierd, looking for ideas.  With libvirt, when you power off a node it calls _destroy() which will loop over the serial ports and call serial_console.release_port(). Then in power_on() we end up calling _destroy() again, which could end up releasing serial ports that are now in use by another instance.20:58
cfriesen(I think these are basically TCP ports on the host.)21:00
mriedemmnaser: left a comment in - how do you feel about working that into the commit message before we start a revert party21:01
mnasermriedem: are you okay with me copypasta-ing that comment and adding co-authored because you seem to have summarized it well21:03
mriedemthat's fine21:04
mriedemdon't really need the co-author21:04
mnaserokay cool, let me see21:04
*** yamamoto has joined #openstack-nova21:04
*** gagehugo has joined #openstack-nova21:06
openstackgerritsean mooney proposed openstack/nova master: WIP add mtu to libvirt xml for ethernet and bridge types
openstackgerritMohammed Naser proposed openstack/nova master: Revert "Refine waiting for vif plug events during _hard_reboot"
mnasermriedem: is that ok?  if thats okay ill update the other ones21:07
mriedemyeah lgtm21:08
*** rcernin has joined #openstack-nova21:08
sean-k-mooneyi have not tested the mtu ptach so ignore it for now. ill test it tomorrow and remove the WIP once i add unit test and check it actully works.21:08
mriedemuse the same change id in the stable branch ones too21:08
mnaserwill do21:08
*** tojuvone has quit IRC21:08
*** tojuvone has joined #openstack-nova21:09
*** sar has quit IRC21:09
arvindn05mriedem: jaypipes: updated the spec based on comments Patch set 9 should address the issues
openstackgerritMohammed Naser proposed openstack/nova stable/queens: Revert "Refine waiting for vif plug events during _hard_reboot"
*** yamamoto has quit IRC21:10
openstackgerritEric Fried proposed openstack/nova master: Stop assuming initial provider generation is 0
efriedcdent: jaypipes: That 'un is ready now too ^21:10
openstackgerritMohammed Naser proposed openstack/nova stable/pike: Revert "Refine waiting for vif plug events during _hard_reboot"
mnasermriedem: apparently changing the id in change-id: <foo> doesn't do it?21:11
mnasermaybe i need to recommit or rebase (i did edited with the ui)21:12
mriedemi figured you could use the ui21:17
*** tojuvone_ has joined #openstack-nova21:20
*** tojuvone has quit IRC21:20
*** tidwellr has quit IRC21:21
sean-k-mooneyjaypipes: just on the nic feature based schduling spec
*** lpetrut has quit IRC21:22
*** sidx64 has quit IRC21:22
sean-k-mooneyi just responed to you comments . i can resping if you want but many of the nits you raised are due to the fact that half of this feature merged in pike since it was feature complete since then21:22
sean-k-mooneymoving the nic feature to placement would be cool but they have been stored in the nova db for 2 release now so i would like to finish the use fo them first then port to placement21:23
*** dave-mccowan has quit IRC21:26
*** Tom-Tom has joined #openstack-nova21:28
jaypipessean-k-mooney: I'm already +2 on that. no need.21:30
*** tssurya has quit IRC21:30
*** andreas_s has joined #openstack-nova21:31
arvindn05jaypipes: quick question on one of your comments21:33
*** Tom-Tom has quit IRC21:33
*** dave-mccowan has joined #openstack-nova21:33
*** awaugama has quit IRC21:33
arvindn05 - Don't forget you will need to modify the existing ImageExtraSpecsFilter  to ignore keys that start with "trait:", since clearly the placement API  will have already filtered out hosts without the required traits...21:33
arvindn05for the comment mriedem pointed out there is no ImageExtraSpecsFilter. Was there a different filter you had in mind?21:33
*** pchavva has quit IRC21:34
jaypipesarvindn05: yeah, it's ImagePropertiesFilter, sorry about that21:34
jaypipesarvindn05: I always forget that it's named differently.21:35
mriedemImagePropertiesFilter only cares about 3 specific image props21:35
mriedemshould probably be renamed21:35
*** tbachman has quit IRC21:35
arvindn05^what mriedem said21:35
arvindn05actually 4 properties looks like..21:36
jaypipesefried: "bogosity". well played.21:36
arvindn05if i trust the documentation :)21:36
*** andreas_s has quit IRC21:36
efriedjaypipes: I calls 'em like I sees 'em21:36
* arvindn05 less changes yay21:37
jaypipesarvindn05: don't forget the illustrative "i_am_nfv_and_do_what_want" property.21:38
openstackgerritMatt Riedemann proposed openstack/nova master: Change TestNewtonCellsCheck to not rely on objects
mriedemthe value is a wildcard right?21:38
*** esberglu_ has quit IRC21:38
mriedemdansmith: melwitt: ^ is the thing tssurya needed21:38
jaypipesmriedem: on Tuesdays. on Wednesdays it's a reverse wildcard. On Thursday it's a regex and on Friday it's all whitespace.21:39
melwittjaypipes: been meaning to ask you, were you gonna update this unit test to do the BFV thing? or I can try to help with that if you want. if possible, I want to backport that all the way to ocata where it broke
jaypipesmelwitt: I'd definitely appreciate a hand on that one.21:39
*** itlinux has quit IRC21:40
melwittjaypipes: cool, I'll take a stab at it21:40
jaypipesmelwitt: tyvm21:40
*** edmondsw has quit IRC21:41
*** belmoreira has quit IRC21:45
*** liverpooler has quit IRC21:48
jaypipesefried: issue in
* cdent has shame21:50
efriedjaypipes: Oh - cdent and I discussed this at the ptg21:50
jaypipesefried: oh?21:50
* cdent has less shame21:51
efriedhe assured me the exception wasn't possible anymore21:51
mriedemarvindn05: +2 on now, thanks21:51
cdentI did not assure you! I said I couldn't see how it could happen.21:51
cdentEntirely different, sirrah.21:51
cdentI may be blind21:51
jaypipesefried, cdent: pretty sure it's still possible.21:52
efriedYou totally assured me21:52
efriedlisten to the tape21:52
* cdent would totally punch efried except that's really scary21:52
*** esberglu has joined #openstack-nova21:52
jaypipesefried, cdent: plus, defensive coding and all that...21:52
efriedshrug, okay21:53
arvindn05mriedem: thanks for the +2 :)21:53
cdentit was conversation that would have been nice to have jaypipes at because we were both struggling to conceptualize how the transaction was operating21:53
cdentand that without that it was all speculation and we didn't know how/if to even test it21:54
arvindn05now just need to bother jaypipes for his original +2 :)21:54
jaypipescdent: it's not necessarily about the transaction itself (or how it operates). it's about the consistent read view. another process could have modified the same resource provider in between the time when we originally began the transaction and when we go to incremenet the generation.21:55
cdentso, jaypipes, if we ever get a clear moment to talk through that, it would be cool. probably wants a whiteboard though, so perhaps vancouver21:55
cdentwell that's exactly the part we couldn't conceptualize how/when does the read view change?21:56
jaypipesarvindn05: +W21:56
*** Matias has quit IRC21:57
cdentif we have at some point in our transaction done a select on the resource provider (as is the case here), isn't that going to be "true" for the duration of the transaction?21:57
cdentor does that require additional syntax?21:58
jaypipescdent: read view == "before I began trying to set these aggregates, the provider generation was 101". we then start the transaction and begin inserting and deleting records from the resource_provider_aggregates table. after our transaction began, another process can modify the resource provider record and update the generation. that is an indication that the state we viewed at the beginning of the transaction has changed and we will need to21:59
jaypipesre-read our view.21:59
*** Matias has joined #openstack-nova21:59
*** wolverineav has quit IRC21:59
*** wolverineav has joined #openstack-nova22:00
*** Tom-Tom has joined #openstack-nova22:01
jaypipescdent: when another process calls COMMIT on a transaction, the changes made in that transaction are now viewable by other transactions that are in process. Changes made in another process's transaction *before* the COMMIT are generally *not* viewable by another process' transactions unless the READ UNCOMMITTED transaction isolation level is set.22:01
*** wolverineav has quit IRC22:01
*** wolverin_ has joined #openstack-nova22:01
*** tbachman has joined #openstack-nova22:02
* cdent parses22:02
openstackgerritMatt Riedemann proposed openstack/nova master: Change TestNewtonCellsCheck to not rely on objects
openstackgerritMatt Riedemann proposed openstack/nova master: Add disabled column to cell_mappings table.
openstackgerritMatt Riedemann proposed openstack/nova master: [WIP] Add disabled field to CellMapping object
openstackgerritMatt Riedemann proposed openstack/nova master: [WIP] Add CellMappingList.get_all_enabled() query method
openstackgerritMatt Riedemann proposed openstack/nova master: [WIP] Allow scheduling only to enabled cells (Filter Scheduler)
efriedMeaning the operation as a whole is atomic, but non-locking.22:02
efriedIt's a race: whoever gets to COMMIT first, wins.22:03
*** burt has quit IRC22:03
efriedBut the stuff happening within one transaction isn't fuddling the other.22:03
*** chyka has quit IRC22:03
cdentefried: I seem to recall you had concerns that needed to be addressed if this is how things turned out, or is "this" some middle ground between the extremes you were thinking of?22:04
*** Matias has quit IRC22:04
efriedthe latter22:04
efriedBecause I was concerned that the state would change between when we read things an when we updated them.22:04
*** owalsh_ has joined #openstack-nova22:06
efriedI guess that's still true, but the code is set up to raise ConcurrentUpdateException if that happens - assuming whatever operation we're doing is incrementing the generation.22:06
*** Tom-Tom has quit IRC22:06
efriedthough this may actually mean we have holes in things that don't muck with generations...22:06
*** yamamoto has joined #openstack-nova22:06
cdentright, the only reason it's not a problem is becsuse incrementing the generation is an update22:06
* cdent makes a slight adjustment to his mental model22:07
efriedspecifically because we have this code to check for generation mismatch.22:07
efriedThough technically there's still a hole - if we check for mismatch riiiight before the other transaction commits.22:08
openstackgerritEric Fried proposed openstack/nova master: placement: generation in provider aggregate APIs
openstackgerritEric Fried proposed openstack/nova master: placement: Return new provider from POST /rps
openstackgerritEric Fried proposed openstack/nova master: Stop assuming initial provider generation is 0
efriedjaypipes: Howzat? ^22:09
cdentefried: I don't think that's the case, because we're not checking for mismatch. We are doing an update that can fail, which is different (at least as far as I understand things, but as this conversation has demonstrated there are some holes in my understanding)22:09
efriedNever mind about the hole, I think I see how it works.22:10
*** owalsh has quit IRC22:10
*** owalsh_ has quit IRC22:11
*** yamamoto has quit IRC22:12
*** mgoddard_ has quit IRC22:14
*** priteau has quit IRC22:15
*** owalsh has joined #openstack-nova22:16
*** felipemonteiro_ has joined #openstack-nova22:20
*** felipemonteiro__ has quit IRC22:23
*** fragatina has quit IRC22:23
*** mlavalle has quit IRC22:23
*** fragatina has joined #openstack-nova22:24
openstackgerritGiridhar Jayavelu proposed openstack/nova-specs master: VMware: place instances on resource pool
openstackgerritMerged openstack/nova-specs master: Support traits in Glance
*** cmm_ has quit IRC22:28
*** fragatina has quit IRC22:29
*** Guest7641 has quit IRC22:30
*** tasker has quit IRC22:30
*** gjayavelu has quit IRC22:30
*** owalsh has quit IRC22:32
*** owalsh has joined #openstack-nova22:33
*** fragatina has joined #openstack-nova22:36
*** fragatina has quit IRC22:36
*** Tom-Tom has joined #openstack-nova22:38
*** fragatina has joined #openstack-nova22:38
*** fragatina has quit IRC22:40
*** danpawlik has joined #openstack-nova22:41
*** Tom-Tom has quit IRC22:43
openstackgerritChris Dent proposed openstack/nova master: Optional separate database for placement API
openstackgerritChris Dent proposed openstack/nova master: Isolate placement database config
*** fragatina has joined #openstack-nova22:46
*** dave-mccowan has quit IRC22:47
*** Tom-Tom has joined #openstack-nova22:47
melwittmriedem: here's the nova/cinder session summary, if you have anything to add/correct
*** felipemonteiro_ has quit IRC22:49
*** danpawlik has quit IRC22:52
*** Tom-Tom has quit IRC22:52
*** shaohe_feng has joined #openstack-nova22:57
*** hongbin has quit IRC22:59
*** cdent has quit IRC23:05
*** tbachman has quit IRC23:07
*** yamamoto has joined #openstack-nova23:07
*** liverpooler has joined #openstack-nova23:08
*** wolverin_ has quit IRC23:09
*** wolverineav has joined #openstack-nova23:10
*** wolverineav has quit IRC23:11
*** wolverin_ has joined #openstack-nova23:11
*** yamamoto has quit IRC23:12
*** artom has joined #openstack-nova23:17
*** Tom-Tom has joined #openstack-nova23:19
*** david-lyle has quit IRC23:21
*** Tom-Tom has quit IRC23:24
*** david-lyle has joined #openstack-nova23:26
*** gjayavelu has joined #openstack-nova23:27
*** liverpooler has quit IRC23:27
Spazmotic Morning folks.23:28
*** david-lyle has quit IRC23:28
mriedemmelwitt: do you want to link patches in there for stuff that's in progress now?23:29
melwittmriedem: sure, why not. I wasn't really thinking about it23:30
*** david-lyle has joined #openstack-nova23:30
mriedemmelwitt: done23:33
*** shaohe_feng has quit IRC23:33
melwittthank ye23:33
*** kmalloc has quit IRC23:38
openstackgerritGiridhar Jayavelu proposed openstack/nova-specs master: VMware: place instances on resource pool
*** Matias has joined #openstack-nova23:43
*** itlinux has joined #openstack-nova23:47
*** esberglu has quit IRC23:50
*** Tom-Tom has joined #openstack-nova23:51
*** gjayavelu has quit IRC23:51
*** hamzy has quit IRC23:56
*** Tom-Tom has quit IRC23:56
*** hamzy has joined #openstack-nova23:58
*** liverpooler has joined #openstack-nova23:59

Generated by 2.15.3 by Marius Gedminas - find it at!