Thursday, 2021-01-21

*** hoonetorg has joined #openstack-nova00:01
*** artom has quit IRC00:11
*** sapd1 has joined #openstack-nova00:26
*** macz_ has quit IRC00:30
*** mlavalle has quit IRC00:49
*** sapd1 has quit IRC01:11
*** dave-mccowan has quit IRC01:24
*** jamesdenton has quit IRC01:33
*** jamesden_ has joined #openstack-nova01:34
*** hamalq has quit IRC01:40
openstackgerritBrin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List SG API  https://review.opendev.org/c/openstack/nova/+/76672601:40
openstackgerritBrin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from Flavor Access APIs  https://review.opendev.org/c/openstack/nova/+/76770401:40
openstackgerritBrin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List/Show usage APIs  https://review.opendev.org/c/openstack/nova/+/76850901:40
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenants* with projects* of policies  https://review.opendev.org/c/openstack/nova/+/76531501:40
openstackgerritBrin Zhang proposed openstack/nova master: Replace os-simple-tenant-usage with os-simple-project-usage  https://review.opendev.org/c/openstack/nova/+/76885201:40
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenant_id with project_id in os-quota-sets path  https://review.opendev.org/c/openstack/nova/+/76885101:40
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenant_id with project_id in Limits API  https://review.opendev.org/c/openstack/nova/+/76886201:41
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenant* with project* in codes  https://review.opendev.org/c/openstack/nova/+/76932901:41
*** xinranwang has joined #openstack-nova02:01
*** amodi has joined #openstack-nova02:04
*** swp20 has joined #openstack-nova02:06
*** martinkennelly has quit IRC02:08
*** lbragstad_ is now known as lbragstad02:15
*** hemanth_n has joined #openstack-nova02:18
openstackgerritLance Bragstad proposed openstack/placement master: Pass context objects to oslo.policy directly  https://review.opendev.org/c/openstack/placement/+/76678502:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for resource providers  https://review.opendev.org/c/openstack/placement/+/76024002:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for aggregates  https://review.opendev.org/c/openstack/placement/+/76023502:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for allocations  https://review.opendev.org/c/openstack/placement/+/76023602:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for allocation candidates  https://review.opendev.org/c/openstack/placement/+/76023702:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for inventories  https://review.opendev.org/c/openstack/placement/+/76023802:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for resource classes  https://review.opendev.org/c/openstack/placement/+/76023902:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for traits  https://review.opendev.org/c/openstack/placement/+/76024102:26
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for usage  https://review.opendev.org/c/openstack/placement/+/76024202:26
*** swp20 has quit IRC02:29
*** swp20 has joined #openstack-nova02:31
*** nweinber has joined #openstack-nova02:34
*** nweinber has quit IRC02:42
*** songwenping_ has joined #openstack-nova03:03
*** swp20 has quit IRC03:07
*** songwenping_ has quit IRC03:08
*** songwenping_ has joined #openstack-nova03:09
*** mkrai has joined #openstack-nova03:25
*** jmlowe has joined #openstack-nova03:36
*** zenkuro has quit IRC04:01
*** macz_ has joined #openstack-nova04:21
*** macz_ has quit IRC04:25
openstackgerritmelanie witt proposed openstack/nova master: Add --sleep option for archive_deleted_rows --until-complete  https://review.opendev.org/c/openstack/nova/+/77174904:26
openstackgerritBrin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List SG API  https://review.opendev.org/c/openstack/nova/+/76672604:28
*** ratailor has joined #openstack-nova04:35
openstackgerritHemanth N proposed openstack/nova stable/stein: Update pci stat pools based on PCI device changes  https://review.opendev.org/c/openstack/nova/+/76172704:45
*** mgariepy has quit IRC04:53
*** mgariepy has joined #openstack-nova04:53
*** mkrai has quit IRC05:01
*** mkrai has joined #openstack-nova05:01
*** xinranwang has quit IRC05:07
*** zzzeek has quit IRC05:41
*** zzzeek has joined #openstack-nova05:42
*** songwenping_ has quit IRC05:51
*** songwenping_ has joined #openstack-nova05:51
*** whoami-rajat__ has joined #openstack-nova05:56
openstackgerritBrin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from Flavor Access APIs  https://review.opendev.org/c/openstack/nova/+/76770405:56
openstackgerritBrin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List/Show usage APIs  https://review.opendev.org/c/openstack/nova/+/76850905:57
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenants* with projects* of policies  https://review.opendev.org/c/openstack/nova/+/76531505:57
*** vishalmanchanda has joined #openstack-nova05:58
openstackgerritBrin Zhang proposed openstack/nova master: Replace os-simple-tenant-usage with os-simple-project-usage  https://review.opendev.org/c/openstack/nova/+/76885205:58
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenant_id with project_id in os-quota-sets path  https://review.opendev.org/c/openstack/nova/+/76885106:00
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenant_id with project_id in Limits API  https://review.opendev.org/c/openstack/nova/+/76886206:01
openstackgerritBrin Zhang proposed openstack/nova master: Replace tenant* with project* in codes  https://review.opendev.org/c/openstack/nova/+/76932906:01
*** rcernin has quit IRC06:07
*** nightmare_unreal has joined #openstack-nova06:41
*** gyee has quit IRC06:46
*** mkrai has quit IRC07:02
*** LinPeiWen52 has quit IRC07:09
*** lpetrut has joined #openstack-nova07:10
openstackgerritYongli He proposed openstack/nova master: Smartnic support - cyborg drive  https://review.opendev.org/c/openstack/nova/+/77136207:13
openstackgerritYongli He proposed openstack/nova master: smartnic support - new vnic type  https://review.opendev.org/c/openstack/nova/+/77136307:13
openstackgerritYongli He proposed openstack/nova master: smartnic support  https://review.opendev.org/c/openstack/nova/+/75894407:13
*** LinPeiWen has joined #openstack-nova07:18
*** tkajinam_ has joined #openstack-nova07:19
*** ratailor_ has joined #openstack-nova07:19
*** ratailor has quit IRC07:19
*** tkajinam has quit IRC07:20
*** LinPeiWen has quit IRC07:20
*** ralonsoh has joined #openstack-nova07:41
*** LinPeiWen has joined #openstack-nova07:48
*** slaweq has joined #openstack-nova07:58
*** ociuhandu has joined #openstack-nova08:00
*** luksky has joined #openstack-nova08:03
*** mkrai has joined #openstack-nova08:04
*** ociuhandu has quit IRC08:05
*** jamesden_ has quit IRC08:05
*** jamesdenton has joined #openstack-nova08:05
*** rpittau|afk is now known as rpittau08:11
*** tosky has joined #openstack-nova08:16
gibilyarwood: hi! looking at the ppc64 unit test results here https://oplab9.parqtec.unicamp.br/pub/ppc64el/openstack/nova/periodic/openstack-tox-py39/2021-01-19-0058-38c70ae/job-output.txt08:23
gibiit seems that the failed tests are arch dependent08:23
gibias os_brick does not define connector for NVME and SCALEIO on ppc64 platform08:24
*** andrewbonney has joined #openstack-nova08:24
gibiI remember you mentioned that you think we missed some arch mocking in our test. Did you mean these? Or in this case we should skip these tests on ppc64?08:25
*** tesseract has joined #openstack-nova08:26
*** xek has joined #openstack-nova08:31
*** bbowen_ has joined #openstack-nova08:44
*** bbowen has quit IRC08:46
songwenping_gibi, sean-k-mooney: hi, do you have any ideas why we need detach pci devices when we suspend the guest, and if the the pci devices detached, can other guest attach them?08:53
*** brinzhang has quit IRC08:55
*** brinzhang has joined #openstack-nova08:55
*** brinzhang has quit IRC08:57
*** brinzhang has joined #openstack-nova08:57
*** ociuhandu has joined #openstack-nova08:58
gibisongwenping_: I don't know why we need to detach for suspend. If we only detach but does not remove the pci claim then another instance cannot claim the same device08:58
gibiyou can look at the pci_devices table to see the state the the device after the suspend08:59
*** tesseract has quit IRC08:59
songwenping_gibi: thanks for your guide.08:59
*** tesseract has joined #openstack-nova09:00
gibihm, it seems that the nova-ceph-multisore is full red since last evening https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack/nova09:04
*** ociuhandu has quit IRC09:04
lyarwoodgibi: morning09:05
gibilyarwood: o/09:05
lyarwoodgibi: yeah that's an example but I can't recall if nova or os-brick are checking the arch here tbh09:06
lyarwoodgibi: either way unit tests shouldn't be checking so it needs to be mocked09:06
gibilyarwood: yes it does check the arc09:06
lyarwoodgibi: and I'm not sure that we should be allowing calls to reach down into os-brick in our unit tests anyway right?09:06
gibiit has different list of connectors on different platform09:06
lyarwoodgibi: we should be mocking things outside of tree09:07
lyarwoodof our*09:07
gibilyarwood: yeah, hitting os_brick is also a bit dirty09:07
gibiin unit test09:07
bauzaslyarwood: sorry was on PTO yesterday09:07
gibilyarwood: did you have a patch that poisons the arch checking code?09:07
gibithat would make these errors visible in our gate too09:08
lyarwoodgibi: https://review.opendev.org/c/openstack/nova/+/769920 but in this case it's os-brick looking up the arch09:08
lyarwoodgibi: so to fix this we just need to mock out the os-brick calls, I'll file a bug and submit something later today09:08
gibilyarwood: thanks09:09
lyarwoodbauzas: np, was that ping about the spec? https://review.opendev.org/c/openstack/nova-specs/+/771474 reviews still welcome if you have time today09:09
lyarwoodgibi: ^ also btw09:09
gibithere is a mail on the ML from aditi Dukle running the ppc64 unit test09:09
bauzaslyarwood: indeed, and sure09:09
gibis09:09
*** ociuhandu has joined #openstack-nova09:10
* lyarwood checks09:11
*** ociuhandu has quit IRC09:14
gibithe gate is broken https://bugs.launchpad.net/nova/+bug/1912607\09:19
openstackLaunchpad bug 1912607 in OpenStack Compute (nova) "test_attach_cloned_encrypted_volume fails in nova-ceph-multistore job permanently " [Critical,New]09:19
lyarwoodgibi: looking09:23
lyarwoodgibi: can you add a direct link to the example failure logs in the bug?09:23
lyarwoodthis smells like a cinder bug tbh09:24
gibilyarwood: done09:24
gibilyarwood: there was not much patch merged in cinder yesterday09:25
lyarwoodyeah that test only landed yesterday09:25
lyarwoodwhere they don't have coverage of our nova-ceph-multistore job09:27
gibiohh, so do we need to revert the test?09:28
gibistephenfin: do you have a blueprint for https://review.opendev.org/q/topic:%22bp%252Fsmarter-usb-devices%22+(status:open%20OR%20status:merged) ?09:28
*** martinkennelly has joined #openstack-nova09:29
openstackgerritMerged openstack/nova-specs master: libvirt: Add support for flavor and image defined ephemeral encryption  https://review.opendev.org/c/openstack/nova-specs/+/77147409:30
stephenfingibi: Hmm, I don't. That was more bugfix than feature. I can create one now though?09:32
stephenfin(despite the topic)09:33
*** lpetrut_ has joined #openstack-nova09:34
lyarwoodgibi: https://github.com/openstack/nova/blob/b9c48afd1516023839dd32b96f3eece36b164a8c/.zuul.yaml#L379-L382 - I guess we add it to the blacklist09:34
*** xek_ has joined #openstack-nova09:34
*** tosky has quit IRC09:36
*** tosky_ has joined #openstack-nova09:36
*** songwenping__ has joined #openstack-nova09:37
*** xek has quit IRC09:37
*** lpetrut has quit IRC09:37
*** songwenping_ has quit IRC09:39
*** tosky_ is now known as tosky09:43
openstackgerritLee Yarwood proposed openstack/nova master: zuul: Skip test_attach_cloned_encrypted_volume in nova-ceph-multistore  https://review.opendev.org/c/openstack/nova/+/77177709:43
lyarwoodgibi: ^ yup lets skip it09:44
*** derekh has joined #openstack-nova09:46
lyarwoodgibi: https://review.opendev.org/c/openstack/nova/+/754702 - would you mind hitting this today so we can start on the above removals and version bumps please?09:46
*** manuvakery1 has joined #openstack-nova09:48
*** zoharm has joined #openstack-nova09:58
bauzasany help for the gate issue ?09:59
bauzaslyarwood: gibi: ^09:59
*** jamesdenton has quit IRC10:00
*** jamesdenton has joined #openstack-nova10:01
lyarwoodbauzas: reviews on https://review.opendev.org/c/openstack/nova/+/754702 would help thanks10:01
bauzasack on it10:01
gibistephenfin: no worries if this is a bugfix then lets handle that that way10:01
* bauzas opened your spec, fwiw10:01
bauzaslyarwood: why isn't it just a revert of https://review.opendev.org/c/openstack/nova/+/746210 ?10:02
bauzasI guess because of https://review.opendev.org/c/openstack/nova/+/746981 ?10:03
lyarwoodconflicts iirc10:07
lyarwoodoh wait sorry that wasn't the gate issue10:07
lyarwoodhttps://review.opendev.org/c/openstack/nova/+/771777 is for the gate issue10:07
stephenfingibi: Actually, the stuff that's merged is a bugfix but "libvirt: Add support for virtio-based input devices" isn't really. I can create a BP for that10:07
gibilyarwood: thanks, fast approved https://review.opendev.org/c/openstack/nova/+/77177710:09
bauzasgibi: already added my +2 ;)10:10
stephenfingibi: https://blueprints.launchpad.net/nova/+spec/smarter-usb-devices10:10
stephenfingibi: related discussion here http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018713.html10:11
*** tesseract has quit IRC10:15
*** tesseract has joined #openstack-nova10:16
stephenfinlyarwood: Any chance you could re-add your +2 here https://review.opendev.org/c/openstack/nova/+/756551/3 The discussion on the mailing list concluded. melwitt deferred to others on whether the 'hw_input_bus' idea (the next patch) was a good idea or not. I still think it is, obv10:17
gibibauzas: you stole my justified fast approve :)10:18
gibistephenfin: thanks10:18
*** ociuhandu has joined #openstack-nova10:18
*** mkrai has quit IRC10:21
*** tesseract-RH has joined #openstack-nova10:24
*** tesseract-RH has quit IRC10:25
gibistephenfin: wiil you bring it up the smarter-usb-device bp on the today's meeting for a last minute approval?10:25
*** tesseract-RH has joined #openstack-nova10:25
stephenfinsure10:25
gibicool10:25
*** tesseract has quit IRC10:27
openstackgerritMark Goddard proposed openstack/nova master: Fix PCI passthrough race on reschedule (claims)  https://review.opendev.org/c/openstack/nova/+/71084710:29
openstackgerritMark Goddard proposed openstack/nova master: Fix PCI passthrough race on reschedule (refresh)  https://review.opendev.org/c/openstack/nova/+/71084810:29
openstackgerritMark Goddard proposed openstack/nova master: Functional test test_boot_reschedule_with_proper_pci_device_count  https://review.opendev.org/c/openstack/nova/+/76035410:34
*** LinPeiWen has quit IRC10:40
*** mkrai has joined #openstack-nova10:50
*** sean-k-mooney has joined #openstack-nova10:57
*** dtantsur|afk is now known as dtantsur10:58
*** tesseract-RH has quit IRC11:09
*** LinPeiWen has joined #openstack-nova11:10
sean-k-mooneystephenfin: i can quickly update the upgrage impact section in the vdpa spec if you like11:21
stephenfinfine by me11:21
sean-k-mooneywithout hte triat it just become None11:21
stephenfinfollow-up is also good11:21
sean-k-mooneyok ill do it in a patch on top11:21
*** tesseract has joined #openstack-nova11:22
*** rcernin has joined #openstack-nova11:23
openstackgerritsean mooney proposed openstack/nova-specs master: [FUP] remove final reference to trait in vdpa spec  https://review.opendev.org/c/openstack/nova-specs/+/77179611:26
stephenfinlyarwood, gibi: when live migrating on API microversion >= 2.25, is there any reason to set block_migration to any value *except* auto?11:26
stephenfini.e. to explicitly request block or shared migration11:26
sean-k-mooneystephenfin: not that i no of11:27
stephenfinokay, so would it make sense to simply ignore the '--block-migration' and '--shared-migration' flags of 'openstack server migrate --live-migration' on suitable microversions11:27
stephenfinwith a log, of course11:27
*** Mamduhala has joined #openstack-nova11:29
*** k_mouza has joined #openstack-nova11:29
* stephenfin plays it safe and simply defaults to auto11:30
lyarwoodstephenfin: I guess as a workaround when auto doesn't detect that things either are or are not shared?11:38
*** mkrai has quit IRC11:38
stephenfinDoes that happen?11:38
* stephenfin isn't familiar with how that machinery works11:38
lyarwoodI've never seen it, just playing devils advocate11:38
stephenfinfair. I've gone with an overrideable default so we should be good11:39
lyarwoodI guess that's why we allow three values instead of just dropping the param and always going auto under the covers11:39
lyarwoodwhich FWIW I'd be fine with11:39
lyarwoodand if that fails raise a bug11:39
lyarwoodbut that's another microversion11:40
sean-k-mooneyi think we just did not remove it because we need the code for the lower microverions so did not see the point11:43
lyarwoodyeah that's fair11:43
sean-k-mooneystephenfin: you should not eed to pass --block-migration or --shared-migration however when using --live-migration i think11:44
sean-k-mooneye.g. if osc requires either today we should dorp that requirement with microversion >=2.2511:44
*** songwenping__ has quit IRC11:48
*** songwenping__ has joined #openstack-nova11:49
*** bbowen_ has quit IRC11:53
*** bbowen has joined #openstack-nova11:53
*** Mamduhala has quit IRC11:55
*** tbachman has quit IRC11:57
*** rcernin has quit IRC11:57
*** jamesdenton has quit IRC12:18
*** jamesdenton has joined #openstack-nova12:19
gibistephenfin: defaulting to auto works for me12:30
stephenfinCool. That's here https://review.opendev.org/c/openstack/python-openstackclient/+/77180112:30
stephenfinAlong with what I hope to be the remainder of the feature gap closing patches for OSC. My doc of novaclient gaps is now empty save for things I don't think we need12:31
gibistephenfin: ack, will try to get to it12:37
gibisean-k-mooney: I approved the vdpa spec and the fup. but I have a question12:37
gibisean-k-mooney: do you have some link to a ticket that tracks the live migration support for vdpa in QEMU?12:38
gibi(suddenly my downstream product mgr got really interested in all the vdpa stuff)12:38
*** raildo has joined #openstack-nova12:38
sean-k-mooneyah good question. there are mail threads on that topic let me see if we have something in bugzilla. i get periodic updates form our qemu folks but not directly tracking it myself12:40
sean-k-mooneyalso thanks :)12:40
sean-k-mooneygibi: by the way vdpa will support block devices at some point to12:41
sean-k-mooneyso that will evenruatly become and rfe12:41
sean-k-mooneyit provides a way to offload thinks like nvme over rdma to the nic and pass it though to the guest12:42
gibisounds interesting12:42
sean-k-mooneybut also provides a way to take a local ssd and use sriov/vdpa to devide it up12:42
sean-k-mooneythat the qemu support for that https://bugzilla.redhat.com/show_bug.cgi?id=1886123 still looking for the migration12:43
openstackbugzilla.redhat.com bug 1886123 in qemu-kvm "Supporting vDPA block in QEMU" [Medium,New] - Assigned to sgarzare12:43
sean-k-mooneygibi: thre will be dpdk support for vdpa eventually too12:44
gibiyeah, I found a slideset about that ^^12:44
*** ratailor_ has quit IRC12:45
openstackgerritMerged openstack/nova-specs master: libvirt: support vdpa kernel networking  https://review.opendev.org/c/openstack/nova-specs/+/76499912:45
openstackgerritMerged openstack/nova-specs master: [FUP] remove final reference to trait in vdpa spec  https://review.opendev.org/c/openstack/nova-specs/+/77179612:46
sean-k-mooneythey are currenlty reworking the original vdpa support that was protype wiht dpdk to work with what landed in the kernel12:46
sean-k-mooneygibi: im not imidetly seeing the live migration tracking bz so ill ask our pm where its being tracked and ill let you know12:47
gibisean-k-mooney: thanks12:48
*** ociuhandu has quit IRC12:49
*** LinPeiWen has quit IRC12:57
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Load and cache volume drivers on-demand  https://review.opendev.org/c/openstack/nova/+/74154513:00
openstackgerritLee Yarwood proposed openstack/nova master: tests: Posion nova.objects.fields.Architecture.from_host  https://review.opendev.org/c/openstack/nova/+/76992013:00
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Stop NVMe and ScaleIO unit tests from calling os-brick  https://review.opendev.org/c/openstack/nova/+/77180613:00
lyarwoodgibi: ^ went down a rabbit hole for a while with the ppc64 stuff but I think https://review.opendev.org/c/openstack/nova/+/741545 is required now.13:01
*** hemanth_n has quit IRC13:02
gibilyarwood: will look at it after my call13:02
lyarwoodgibi: ack thanks13:03
gibiI thank you13:03
lyarwoodstephenfin: https://review.opendev.org/c/openstack/nova/+/771711 - not sure if you saw my ping about this last night btw, finally copied some notes from Matt into our docs13:03
*** tosky has quit IRC13:06
*** tosky has joined #openstack-nova13:06
*** tbachman has joined #openstack-nova13:11
brinzhangsean-k-mooney, gibi: https://review.opendev.org/c/openstack/tempest/+/769120 this patch based on https://review.opendev.org/c/openstack/tempest/+/769857, but why the fist patch still run the stable branch zuul job?13:16
brinzhangdo you have time to check this?13:17
brinzhangjobs tempest-full-py3 , tempest-full-victoria-py3, tempest-full-ussuri-py3, and tempest-full-train-py3 already failed, because of https://review.opendev.org/c/openstack/nova/+/76885213:18
brinzhangand another issue is the nova patch and tempest patch depends-on each other, I have no idea to deal them :(13:19
brinzhangif you have time to help, please left comment or said in irc, I will back home now, thanks13:21
*** ociuhandu has joined #openstack-nova13:23
sean-k-mooneywhy are you skiping the usage tests?13:26
sean-k-mooneytempest is ment to be branchless right and work with any version of openstack13:28
sean-k-mooneyi think what we need to do is update the tempest tests such that they use the old microversion when doing usage checks13:30
sean-k-mooneynot skip13:30
gibiyepp ^^ commented the same thing13:30
sean-k-mooneybrinzhang: that will break the cylic depency as the tempest change is not depent on the nova one then13:31
*** ratailor has joined #openstack-nova13:46
*** sapd1 has joined #openstack-nova13:47
*** owalsh has quit IRC13:48
*** zenkuro has joined #openstack-nova14:03
*** tbachman has quit IRC14:04
*** owalsh has joined #openstack-nova14:08
elodlyarwood: as I added some info to Nova Meeting's stable section (feel free to extend it :)) I realized, that rocky is still blocked. could you have a look at this easy patch some time to unblock it? >>> https://review.opendev.org/c/openstack/nova/+/76649214:09
lyarwoodelod: ack will do14:10
elodlyarwood: thx in advance :]14:10
stephenfinlyarwood: I did but hadn't got to it yet. Looking14:14
lyarwoodstephenfin: ack sorry didn't mean to spam14:15
stephenfinnope, all good14:15
*** hemanth_n has joined #openstack-nova14:21
*** macz_ has joined #openstack-nova14:25
*** markmcclain has quit IRC14:26
*** artom has joined #openstack-nova14:28
*** macz_ has quit IRC14:30
*** vishalmanchanda has quit IRC14:38
openstackgerritDan Smith proposed openstack/nova master: Try harder to exclude encrypted volumes  https://review.opendev.org/c/openstack/nova/+/77182214:40
dansmithlyarwood: we're also suddenly running cinder encrypted tests in that multistore job despite the blacklist regex, which are coming from the cinder tempest plugin14:40
dansmithlyarwood: ^ this tries to hit them too14:40
*** dave-mccowan has joined #openstack-nova14:41
sean-k-mooneyhehe i like the title14:43
*** ratailor has quit IRC14:43
dansmithI dunno why that has started failing all the sudden,14:43
dansmithlike if a name changed and we started running those or something14:43
*** ratailor has joined #openstack-nova14:43
dansmithbut they end up putting an image int he wrong glance repo, which doesn't work14:43
dansmithoh, heh14:44
sean-k-mooneyhttps://github.com/openstack/cinder-tempest-plugin/commit/5ab3fef76283ca91b92fa130ba2685886bd8af6814:44
dansmithlooks like there's already something in gate14:44
sean-k-mooneymaybe that14:45
*** ratailor has quit IRC14:45
dansmithsean-k-mooney: yuuup14:45
*** ratailor has joined #openstack-nova14:45
sean-k-mooneyso ya just new test added14:46
*** ratailor has quit IRC14:46
*** markmcclain has joined #openstack-nova14:46
dansmithlyarwood: sorry, just saw yours from 1am.. I had half figured this out last night but ran out of time, thanks for grabbing that14:47
dansmiththat makes three distinct and legit non-nova, non-glance issues with that job in a week :)14:47
sean-k-mooneywell at least there is now testing for this so its still progress14:53
sean-k-mooneyeven if there are teething issues14:53
*** nweinber has joined #openstack-nova14:54
dansmithlyarwood: I assume that snuck in because we run the whole tempest-cinder-plugin on that job, but they don't gate on it.. so my first thought was just to not load their tempest plugin in there to prevent this from happening in the future.. thoughts on that?14:59
dansmithsean-k-mooney: yes, especially good when they're actual problems and not just flaky or unreliable tests14:59
*** brinzhang has quit IRC15:01
*** brinzhang has joined #openstack-nova15:01
lyarwooddansmith: morning, sorry was afk15:03
lyarwooddansmith: yeah correct, I was going to see if we could add this job into their gate tbh15:03
dansmithlyarwood: np, it took a night of sleep to realize that test was from a tempest plugin, thanks for nailing that whilst I slept15:03
dansmithlyarwood: that also works15:04
*** brinzhang has quit IRC15:04
*** brinzhang has joined #openstack-nova15:04
lyarwoodso what's the deal with these project specific tempest plugins15:04
lyarwoodfrom what I've seen most of this should live in core tempest anyway15:05
dansmithI dunno, asking for trouble :)15:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach  https://review.opendev.org/c/openstack/nova/+/77024615:05
lyarwoodhehe15:05
lyarwoodI just don't get the need for loads of different test repos all doing the same time tbh, I'll go away and read up15:05
dansmithwithout co-gating it's just an attack vector15:05
*** spatel has joined #openstack-nova15:12
lyarwooddansmith: multistore attempts to cover some edge usecases right?15:12
*** masterpe has quit IRC15:19
*** rouk has quit IRC15:19
dansmithlyarwood: the whole arrangement in devstack is edgy yeah15:21
*** irclogbot_3 has quit IRC15:21
lyarwooddansmith: cool, so looking at the actual test that borked here this might be something we can fix on their side15:21
lyarwooddansmith: for whatever reason the test is creating a unique image for the instance to use15:22
lyarwooddansmith: when it really isn't required as the test is about cloning an encrypted volume attached to the instance15:22
*** irclogbot_2 has joined #openstack-nova15:22
lyarwoodI'll submit a few changes to the cinder plugin before adding our job to their gate somehow15:23
dansmithyeah, and it creates it in the wrong glance store, which is the problem15:23
lyarwoodwe might be able to drop the blacklist around encrypted volumes after all15:23
lyarwoodskiplist*15:23
dansmithbrownlist15:23
dansmithlike, turds15:23
*** hemanth_n has quit IRC15:25
lyarwood^_^15:25
spatelnoonedeadpunk: In your designate example where is pool_id? http://paste.openstack.org/show/801787/15:26
*** tbachman has joined #openstack-nova15:26
noonedeadpunko_O15:26
spateli had issue that issue and when i put pool id then it works15:27
spatelwhat is the workaround for that?15:27
spateldoes OSA generate random pool id ?  (because default isn't valid)15:27
spatelsorry folks wrong channel15:28
*** masterpe has joined #openstack-nova15:29
sean-k-mooneyi had issue with the pool id in kolla ansible too15:34
sean-k-mooneyfor me i had to both mess with the config and the db to eventulaly get it to work15:35
sean-k-mooneybut i remmeber id did not jsut work for me when it was deployed15:36
sean-k-mooneyi added15:36
sean-k-mooney[service:central]15:36
sean-k-mooneydefault_pool_id = 6a67bd3a8e654ddeb3675b8534899f1615:36
sean-k-mooneyworkers = 515:36
sean-k-mooneyand then i think i needed to edit the db to make it match15:36
sean-k-mooneyit was strange15:37
*** lpetrut_ has quit IRC15:42
openstackgerritLee Yarwood proposed openstack/nova master: zuul: Allow encryption tests in nova-ceph-multistore again  https://review.opendev.org/c/openstack/nova/+/77183715:46
lyarwooddansmith: ^ hopefully this work15:46
lyarwoodthat links through to a fix and another change adding the job to gate of the tempest plugin15:46
lyarwoodah I think I also need to fix some core tempest tests15:47
lyarwoodmy bad15:47
dansmithyeah, there are core tests too, and that's why last night I was like "wtf, how is this just now breaking when nothing has changed here"15:47
lyarwoodack kk, I'll let this fail and then fix them up, it's likely the same bogus image creation copy pasted between tempest and the plugin tbh15:48
dansmithwouldn't be surprised, yeah15:49
dansmithwhich is further support for "why is this separate?"15:49
lyarwoodyup15:49
gibinova weekly meeting starts in 10 minutes in #openstack-meeting-315:50
bauzasgibi: given the curfew we have at 6pm, I'll stay in the meeting until 5.30pm our time15:50
*** mlavalle has joined #openstack-nova15:51
gibibauzas: O15:54
gibiK15:54
* gibi has curfew from 8pm15:55
*** macz_ has joined #openstack-nova15:59
*** macz_ has quit IRC15:59
*** manuvakery1 has quit IRC16:00
*** macz_ has joined #openstack-nova16:00
dansmithhey sean-k-mooney, you like devstacking.. would you run this locally and compare before/after numbers? https://review.opendev.org/c/openstack/devstack/+/77150516:06
sean-k-mooney" know this is going to be controversial, but hear me out." L)16:07
sean-k-mooney* :)16:08
sean-k-mooneysure i can try that16:08
dansmithI actually maybe should remove that,16:08
dansmithbecause so far it has been super stable for me, and easy to debug things16:08
dansmithinitial attempt interleaved all the parallel logging, which I knew would be hard to swallow for people16:08
*** dklyle has joined #openstack-nova16:08
sean-k-mooneywe still have some native calls to osc by the way that dont use osc wrap16:09
dansmithbut now it looks very clean, just potentially different orderings16:09
sean-k-mooneyif we fixed up those it would also help16:09
dansmithsean-k-mooney: oscwrap is just for time accounting16:09
dansmithit doesn't make it any faster16:09
dansmithand it sets up an alias, so everything should be using it, AFAI16:09
sean-k-mooneywaith i tought it used osc in itneractive mode via a socket16:09
dansmithno16:09
sean-k-mooneydid we not merge danpbs prototype of that16:09
sean-k-mooneyoh ok16:09
dansmithI tried doing that myself this week, but getting the return code is going to be non-trivial and we need that in a *lot* of places in devstack16:10
sean-k-mooneywell ya ill pull that into one of my vms and let it run16:10
dansmithwe do lots of single-threaded things in devstack, waiting on osc is just the dumbest one :)16:11
dansmithsean-k-mooney: ooh, thanks I'd definitely appreciate  run in a vm16:11
dansmithI'm on baremetal16:11
sean-k-mooneywell i ment run it in one of my openstack vms so that will be similar to the upstream ci runs16:12
dansmiththat's cool16:12
dansmithperformance in the actual nodepool workers varies so much from minute to minute it's hard to prove with linkable numbers that it's better16:12
dansmithso more anecdotal evidence would be good16:12
dansmithI'm also on spinning rust currently, and I feel like I could be getting better parallel performance by overlapping the db syncs if I wasn't16:14
sean-k-mooneyi rebooted my cloud at the weekend so need to start up my third paryt ci again but i can pproably have it do a few full tempest runs too but ill take a look at that spereatly16:14
sean-k-mooneysince my thridparty ci just runs on one host its pretty consitnet16:14
dansmithso far it hasn't failed a tempest run, so I'm pretty confident that it's still building a consistent devstack16:14
artomDo we... do we have a config option to log in JSON somehow?16:16
sean-k-mooneyoh so "    async_inner $name $* &" is how your doing the async run16:17
artomI'm looking at some downstream logs, and they're in format {"message": <blah>, "asctime": <blah>, "name": <blah>, "msg": <etc> }16:17
artomFirst time I see something like that16:17
sean-k-mooneyartom: yes you can do this16:18
dansmithartom: I think you could use one of the libs for it and configure it thusly16:18
sean-k-mooneyvia standard python logger config16:19
dansmithso probably doable without changes to the openstack code16:19
dansmithyeah ^16:19
artomMakes all of our usual debugging tools (read: os-log-merger) completely useless16:19
sean-k-mooneyone of our aisa based customer does it16:19
* artom tries lnav I guess?16:19
sean-k-mooneylave i think will work with the time stamps but really it depend on how they did it16:20
artomsean-k-mooney, and it's a finicky external events evacuation issue as well16:20
artomWhere you need to correlate timestampe and requests across Neutron and Nova :(16:21
sean-k-mooneyoh ya i started looking at that but did not open the sos report16:23
sean-k-mooneythey should be aware that hw:numa_mempolicy or whatever it is is not a thing16:24
sean-k-mooneyoh wait imight be mixing too issues16:24
sean-k-mooneyyour looking at https://bugzilla.redhat.com/show_bug.cgi?id=191767516:25
openstackbugzilla.redhat.com bug 1917675 in openstack-nova "Nova evacuate fails due to timeout waiting for a network-vif-plugged event for instance" [High,New] - Assigned to nova-maint16:25
sean-k-mooneythe one with the fake extra spec is a different one i was look at breifly too16:25
artomsean-k-mooney, yeah, that one. We can move downstream to discuss the specifics, tho16:25
sean-k-mooneyhehe ya i know just confirm there was not a duplicate16:26
sean-k-mooneyi just glanced at it so have not looked at the logs or anything16:26
artomTheir format makes them unworkable :(16:26
*** amodi has quit IRC16:29
*** amodi has joined #openstack-nova16:30
*** gyee has joined #openstack-nova16:32
*** lpetrut has joined #openstack-nova16:42
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Load and cache volume drivers on-demand  https://review.opendev.org/c/openstack/nova/+/74154516:47
*** ociuhandu_ has joined #openstack-nova16:49
*** lpetrut has quit IRC16:50
openstackgerritStephen Finucane proposed openstack/placement master: tests: Suppress policy deprecation warnings  https://review.opendev.org/c/openstack/placement/+/77185216:50
*** ociuhandu has quit IRC16:53
stephenfinlbragstad: When you've a chance, I've a question left on https://review.opendev.org/c/openstack/placement/+/76023516:53
*** ociuhandu_ has quit IRC16:54
* lyarwood shakes fist at stephenfin and his typing fetish16:56
lyarwoodnow I have to fix all of these broken tests16:56
lyarwoodgah16:56
stephenfintests broke?16:57
stephenfinhuh?16:57
lyarwoodpassing in strings instead of dicts etc16:57
lyarwoodat least I think that's what this error means16:58
stephenfinOh, those won't get picked up by mypy unless you run it on them. Don't run mypy on tests :)16:58
stephenfinHmm, I doubt it16:58
stephenfinPaste?16:58
lyarwoodTypeError: Parameters to generic types must be types. Got 0.16:58
lyarwoodhttp://paste.openstack.org/show/801843/16:58
* stephenfin looks16:59
lyarwoodself.volume_drivers = ty.Dict[str, volume.LibvirtBaseVolumeDriver]  is what I have for that dict17:00
stephenfinyup, I just pulled down to see what's going on17:00
* lyarwood should likely ready https://docs.python.org/3/library/typing.html before going any further17:01
lyarwoodread*17:01
stephenfinlyarwood: oh /o\17:01
stephenfinself.volume_drivers = ty.Dict[str, volume.LibvirtBaseVolumeDriver]17:01
stephenfinshould be17:01
stephenfinself.volume_drivers: ty.Dict[str, volume.LibvirtBaseVolumeDriver] = {}17:01
stephenfinvar: type = value17:01
lyarwoodaaaaaah17:01
stephenfinsorry, I missed that17:02
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Load and cache volume drivers on-demand  https://review.opendev.org/c/openstack/nova/+/74154517:05
lyarwoodstephenfin: np thanks again ^17:05
*** tesseract has quit IRC17:06
openstackgerritMerged openstack/nova master: Add regression test for bug #1899835  https://review.opendev.org/c/openstack/nova/+/75823917:08
openstackbug 1899835 in OpenStack Compute (nova) "n-cpu attempts to disconnect volumes after early pre_live_migration failures on the destination during a live migration" [Medium,In progress] https://launchpad.net/bugs/1899835 - Assigned to Lee Yarwood (lyarwood)17:08
lbragstadstephenfin done17:09
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for aggregates  https://review.opendev.org/c/openstack/placement/+/76023517:10
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for allocations  https://review.opendev.org/c/openstack/placement/+/76023617:10
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for allocation candidates  https://review.opendev.org/c/openstack/placement/+/76023717:10
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for inventories  https://review.opendev.org/c/openstack/placement/+/76023817:10
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for resource classes  https://review.opendev.org/c/openstack/placement/+/76023917:10
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for traits  https://review.opendev.org/c/openstack/placement/+/76024117:10
openstackgerritLance Bragstad proposed openstack/placement master: Implement secure RBAC for usage  https://review.opendev.org/c/openstack/placement/+/76024217:10
*** dcapone2004 has joined #openstack-nova17:20
*** dave-mccowan has quit IRC17:31
*** manoj_kumar_kata has joined #openstack-nova17:32
*** hamalq has joined #openstack-nova17:33
*** lpetrut has joined #openstack-nova17:40
*** rpittau is now known as rpittau|afk17:42
*** k_mouza has quit IRC17:48
*** ralonsoh has quit IRC17:52
*** rcernin has joined #openstack-nova17:54
*** lpetrut has quit IRC17:55
*** k_mouza has joined #openstack-nova17:57
*** rcernin has quit IRC17:59
manoj_kumar_katahi network experts, we have 4-node openstack cluster and a recent lab poweroff resulted in losing network configuration. we worked on it and now the controller node could create vms but VMs are not getting the ip assigned .18:00
manoj_kumar_kataappreciate your help here18:00
*** dtantsur is now known as dtantsur|afk18:32
*** dcapone2004 has quit IRC18:41
*** nightmare_unreal has quit IRC18:42
*** jamesdenton has quit IRC18:43
*** jamesdenton has joined #openstack-nova18:43
*** tbachman has quit IRC18:46
*** tbachman has joined #openstack-nova18:47
*** andrewbonney has quit IRC18:55
*** _mlavalle_1 has joined #openstack-nova18:59
*** sapd1 has quit IRC19:02
*** k_mouza has quit IRC19:03
*** mlavalle has quit IRC19:03
*** sapd1 has joined #openstack-nova19:06
openstackgerritMerged openstack/nova master: zuul: Skip test_attach_cloned_encrypted_volume in nova-ceph-multistore  https://review.opendev.org/c/openstack/nova/+/77177719:08
*** tbachman has quit IRC19:16
*** haleyb has quit IRC19:27
*** f0o has quit IRC19:28
*** f0o has joined #openstack-nova19:28
*** haleyb has joined #openstack-nova19:28
*** xek_ has quit IRC19:30
*** xek_ has joined #openstack-nova19:31
openstackgerritLee Yarwood proposed openstack/nova stable/queens: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0  https://review.opendev.org/c/openstack/nova/+/74736219:34
lyarwoodmelwitt / elod ; https://review.opendev.org/c/openstack/nova/+/747362/ would you mind looking at this stack of stable/queens changes again, should be good to go now.19:35
* lyarwood tilts head19:35
lyarwoodI just tried to rebase that in the new gerrit UI and it has ended up with a merge conflict, wat?!19:36
*** martinkennelly has quit IRC19:38
lyarwoodha, it just didn't report an actual conflict during the rebase attempt, fun19:38
openstackgerritLee Yarwood proposed openstack/nova stable/queens: sync_guest_time: use the proper errno  https://review.opendev.org/c/openstack/nova/+/74242319:40
openstackgerritLee Yarwood proposed openstack/nova stable/queens: libvirt: Remove reference to transient domain when detaching devices  https://review.opendev.org/c/openstack/nova/+/60818619:40
openstackgerritLee Yarwood proposed openstack/nova stable/queens: libvirt: Handle VIR_ERR_DEVICE_MISSING when detaching devices  https://review.opendev.org/c/openstack/nova/+/74242419:40
openstackgerritLee Yarwood proposed openstack/nova stable/queens: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0  https://review.opendev.org/c/openstack/nova/+/74736219:40
lyarwoodthere rebased and hopefully ready19:40
*** tbachman has joined #openstack-nova19:53
*** slaweq has quit IRC19:55
*** dave-mccowan has joined #openstack-nova19:55
*** zoharm has quit IRC20:00
*** spatel has quit IRC20:00
elodlyarwood: ack, I will review tomorrow :)20:02
*** jamesdenton has quit IRC20:06
*** jamesdenton has joined #openstack-nova20:06
openstackgerritLee Yarwood proposed openstack/nova master: docs: Add reference docs for internal block device structures  https://review.opendev.org/c/openstack/nova/+/77171120:12
*** manoj_kumar_kata has quit IRC20:12
*** irclogbot_2 has quit IRC20:14
*** tosky has quit IRC20:14
*** tosky has joined #openstack-nova20:15
*** k_mouza has joined #openstack-nova20:16
*** irclogbot_3 has joined #openstack-nova20:17
*** k_mouza has quit IRC20:20
*** tbachman has quit IRC20:26
*** yankcrime has quit IRC20:31
melwittsean-k-mooney: I read the backscroll from the nova meeting but I don't understand how it answers the question I had on the review, how can an operator prevent a user from leveraging numa aware vswitch?20:34
melwitt"As an operator i wish to utilize numa aware vswitch but still be able to disable it for individual vm interfaces."20:34
sean-k-mooneymelwitt: what i ment by that was and still allow the tenant to disable it for indivigual vm interfaces20:40
*** dcapone2004 has joined #openstack-nova20:40
sean-k-mooneyi had a other part of the spec that i planned to do but have not included too20:40
sean-k-mooneywhich was to add a config option for the default polcy for numa vswitches20:40
melwittsean-k-mooney: ok, so tenant instead of operator. and that means there's a numa_affinity_policy that means disable? or by disable do you mean opt out20:41
sean-k-mooneyline 179 https://etherpad.opendev.org/p/nova-wallaby-ptg20:41
sean-k-mooneymelwitt: so if you have numa vsiwtchs configured thne you get strict affinity for every numa vm on the host20:41
sean-k-mooneywiththe port policy you coudl set it to prefer20:42
sean-k-mooneywhich is basically best effort or no affinity enforced20:42
melwittok, so there's not a concept of "disabling" it i.e. preventing it20:42
sean-k-mooneythe prefer policy is the closet thing20:43
sean-k-mooneythere is not way to request anti affinity20:43
sean-k-mooneyor disable all numa checks20:43
melwittok, thanks20:43
sean-k-mooneyyou can just say for this 1 interface do X20:43
*** whoami-rajat__ has quit IRC21:04
*** nweinber has quit IRC21:11
*** nweinber has joined #openstack-nova21:20
openstackgerritMerged openstack/nova-specs master: add port scoped numa affinity spec  https://review.opendev.org/c/openstack/nova-specs/+/76590121:21
*** owalsh has quit IRC21:26
*** tbachman has joined #openstack-nova21:28
*** derekh has quit IRC21:37
*** raildo has quit IRC21:41
*** owalsh has joined #openstack-nova21:44
*** nweinber has quit IRC21:49
*** rcernin has joined #openstack-nova21:54
*** rcernin has quit IRC21:59
*** xek_ has quit IRC22:05
*** derekh has joined #openstack-nova22:06
*** derekh has quit IRC22:06
*** owalsh has quit IRC22:16
*** owalsh has joined #openstack-nova22:25
*** rcernin has joined #openstack-nova22:38
*** rcernin has quit IRC22:56
*** rcernin has joined #openstack-nova22:57
*** jobewan has joined #openstack-nova22:59
*** songwenping__ has quit IRC23:00
*** brinzhang has quit IRC23:00
*** ociuhandu has joined #openstack-nova23:24
*** ociuhandu has quit IRC23:29
*** brinzhang has joined #openstack-nova23:34
*** luksky has quit IRC23:51
*** lemko has quit IRC23:51
*** lemko6 has joined #openstack-nova23:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!