Wednesday, 2018-12-05

*** gyee has quit IRC00:33
*** mlavalle has quit IRC00:33
*** mlavalle has joined #openstack-nova00:33
*** mlavalle has quit IRC00:33
*** rodolof has joined #openstack-nova00:37
*** wolverineav has quit IRC00:45
*** wolverineav has joined #openstack-nova00:48
*** ileixe has joined #openstack-nova01:05
*** Nel1x has joined #openstack-nova01:16
*** wolverineav has quit IRC01:24
*** wolverineav has joined #openstack-nova01:25
*** wolverineav has quit IRC01:30
*** wolverineav has joined #openstack-nova01:31
*** hongbin has joined #openstack-nova01:36
*** dakshina-ilangov has joined #openstack-nova01:39
*** k_mouza has joined #openstack-nova01:43
openstackgerritzhongshengping proposed openstack/nova master: Update mailinglist from dev to discuss  https://review.openstack.org/62182701:46
*** jamesdenton has joined #openstack-nova01:46
*** qwebirc716083 has joined #openstack-nova01:47
*** rambo_li has joined #openstack-nova01:48
*** k_mouza has quit IRC01:48
qwebirc716083join #openstack-nova01:51
*** qwebirc716083 has quit IRC01:52
*** cfriesen has quit IRC02:05
*** wolverineav has quit IRC02:07
*** dklyle has quit IRC02:12
*** dklyle has joined #openstack-nova02:13
*** Dinesh_Bhor has joined #openstack-nova02:13
*** mrsoul has joined #openstack-nova02:14
*** wolverineav has joined #openstack-nova02:14
*** qwebirc783492 has joined #openstack-nova02:18
*** wolverineav has quit IRC02:21
*** qwebirc584207 has joined #openstack-nova02:25
*** qwebirc783492 has quit IRC02:28
*** qwebirc584207 has quit IRC02:30
*** Zer0Byte_ has quit IRC02:36
*** mhen has quit IRC02:37
*** mhen has joined #openstack-nova02:37
openstackgerritTushar Patil proposed openstack/nova-specs master: Support filtering of allocation_candidates by forbidden aggregates  https://review.openstack.org/60996002:38
*** Dinesh_Bhor has quit IRC02:39
openstackgerritJack Ding proposed openstack/nova master: Improve libvirt image and snapshot handling  https://review.openstack.org/61669202:42
*** psachin has joined #openstack-nova02:44
*** Dinesh_Bhor has joined #openstack-nova02:48
openstackgerritZhenyu Zheng proposed openstack/nova master: Handle tags in _bury_in_cell0  https://review.openstack.org/62185602:51
*** imacdonn has quit IRC02:52
*** imacdonn has joined #openstack-nova02:53
*** igordc has quit IRC02:56
*** hongbin has quit IRC03:07
*** Nel1x has quit IRC03:16
openstackgerritMerged openstack/nova master: Remove utils.execute() calls from xenapi.  https://review.openstack.org/61970003:21
*** Dinesh_Bhor has quit IRC03:56
*** rodolof has quit IRC03:57
*** rambo_li has quit IRC04:00
*** wolverineav has joined #openstack-nova04:08
*** wolverineav has quit IRC04:12
*** itlinux has joined #openstack-nova04:12
*** rodolof has joined #openstack-nova04:19
*** wolverineav has joined #openstack-nova04:19
*** ileixe has quit IRC04:24
*** hongbin has joined #openstack-nova04:24
*** ileixe has joined #openstack-nova04:28
*** wolverineav has quit IRC04:37
*** janki has joined #openstack-nova04:40
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Body verification for the lock action  https://review.openstack.org/62283504:41
*** mmethot has quit IRC04:41
*** rodolof has quit IRC04:55
*** mordred has quit IRC04:57
*** mordred has joined #openstack-nova04:57
*** Dinesh_Bhor has joined #openstack-nova05:02
*** hongbin has quit IRC05:19
*** ileixe has quit IRC05:31
*** ileixe has joined #openstack-nova05:31
*** licanwei has joined #openstack-nova05:43
*** izza_ has quit IRC05:44
*** gouthamr has quit IRC05:46
*** izza_ has joined #openstack-nova05:47
izza_hi guys05:47
izza_can anyone help us here on windows provisioning on openstack05:48
*** sridharg has joined #openstack-nova05:48
*** gouthamr has joined #openstack-nova05:53
*** wolverineav has joined #openstack-nova05:56
*** ratailor has joined #openstack-nova06:00
*** wolverineav has quit IRC06:00
*** gouthamr has quit IRC06:15
*** gouthamr has joined #openstack-nova06:18
*** gouthamr has quit IRC06:29
openstackgerritYongli He proposed openstack/nova master: Adds the server group info into show server detail API.  https://review.openstack.org/62147406:30
*** gouthamr has joined #openstack-nova06:32
openstackgerritYongli He proposed openstack/nova master: Add numa topology and pci devices into server detail API  https://review.openstack.org/62147606:34
*** tetsuro has joined #openstack-nova06:41
*** Luzi has joined #openstack-nova06:42
*** tetsuro has quit IRC06:43
*** Dinesh_Bhor has quit IRC06:57
*** gouthamr has quit IRC06:58
*** ralonsoh has joined #openstack-nova06:58
*** ileixe has quit IRC07:01
*** ileixe has joined #openstack-nova07:02
*** gouthamr has joined #openstack-nova07:07
*** pcaruana has joined #openstack-nova07:10
openstackgerritYongli He proposed openstack/nova-specs master: add 'show-server-group' spec  https://review.openstack.org/61225507:13
*** Dinesh_Bhor has joined #openstack-nova07:16
*** takamatsu has joined #openstack-nova07:21
openstackgerritYongli He proposed openstack/nova-specs master: add spec "show-server-numa-topology"  https://review.openstack.org/61225607:26
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Body verification for the lock action  https://review.openstack.org/62283507:30
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Body verification for the lock action  https://review.openstack.org/62283507:39
*** dpawlik has joined #openstack-nova07:40
*** ileixe has quit IRC07:40
*** gouthamr has quit IRC07:40
*** gouthamr has joined #openstack-nova07:42
*** slaweq has joined #openstack-nova07:48
*** sahid has joined #openstack-nova07:56
*** brault has joined #openstack-nova07:57
*** takashin has left #openstack-nova08:01
*** ttsiouts has joined #openstack-nova08:05
openstackgerritZhenyu Zheng proposed openstack/nova master: Handle tags in _bury_in_cell0  https://review.openstack.org/62185608:10
*** ttsiouts has quit IRC08:16
*** ttsiouts has joined #openstack-nova08:17
*** spatel has joined #openstack-nova08:19
*** spatel has quit IRC08:19
*** awalende has joined #openstack-nova08:20
*** priteau has joined #openstack-nova08:21
*** ttsiouts has quit IRC08:21
*** helenafm has joined #openstack-nova08:22
izza_hi08:25
izza_can anyone help us here how to install windows in trial version in openstack08:25
izza_i saw procedure but it is looking for a license windows08:25
*** tssurya has joined #openstack-nova08:32
*** awalende_ has joined #openstack-nova08:33
tssuryamriedem: yea here https://bugs.launchpad.net/nova/+bug/180598408:33
openstackLaunchpad bug 1805984 in OpenStack Compute (nova) "Placement is not aware of disable compute nodes" [Undecided,New]08:33
*** ttsiouts has joined #openstack-nova08:34
*** awalende has quit IRC08:35
*** jmlowe has quit IRC08:35
*** ttsiouts has quit IRC08:37
izza_hi08:38
openstackgerritRui Zang proposed openstack/nova-specs master: Virtual persistent memory libvirt driver implementation  https://review.openstack.org/62289308:38
izza_is it possible to install windows trial version on openstack?08:39
*** k_mouza has joined #openstack-nova08:43
*** k_mouza has quit IRC08:48
*** ttsiouts has joined #openstack-nova08:50
openstackgerritRui Zang proposed openstack/nova-specs master: support virtual persistent memory  https://review.openstack.org/60159608:51
*** ttsiouts has quit IRC09:01
*** ttsiouts has joined #openstack-nova09:01
*** brault has quit IRC09:03
*** brault has joined #openstack-nova09:04
*** maciejjozefczyk has quit IRC09:04
*** dpawlik has quit IRC09:04
*** maciejjozefczyk has joined #openstack-nova09:04
*** wolverineav has joined #openstack-nova09:05
*** dpawlik has joined #openstack-nova09:05
*** ttsiouts has quit IRC09:06
*** ttsiouts has joined #openstack-nova09:07
*** wolverineav has quit IRC09:09
*** ccamacho has joined #openstack-nova09:20
*** xek has joined #openstack-nova09:21
*** Dinesh_Bhor has quit IRC09:25
*** trident has quit IRC09:29
*** k_mouza has joined #openstack-nova09:29
*** Dinesh_Bhor has joined #openstack-nova09:30
*** trident has joined #openstack-nova09:31
*** k_mouza has quit IRC09:32
*** k_mouza has joined #openstack-nova09:33
*** ttsiouts has quit IRC09:35
*** ttsiouts has joined #openstack-nova09:36
*** rcernin has quit IRC09:38
*** ttsiouts has quit IRC09:40
*** ttsiouts has joined #openstack-nova09:46
*** derekh has joined #openstack-nova09:49
*** maciejjozefczyk has quit IRC10:07
*** maciejjozefczyk has joined #openstack-nova10:08
openstackgerritBhagyashri Shewale proposed openstack/nova master: PCPU: Define numa dedicated CPU resource class  https://review.openstack.org/56177010:18
openstackgerritBhagyashri Shewale proposed openstack/nova master: NUMACell, InstanceNUMACell: Adopt 'PCPU' changes  https://review.openstack.org/57602110:18
openstackgerritBhagyashri Shewale proposed openstack/nova master: Report PCPU to placement  https://review.openstack.org/57703810:18
openstackgerritBhagyashri Shewale proposed openstack/nova master: Virt driver: Build guest xml  https://review.openstack.org/57737210:18
*** maciejjozefczyk has quit IRC10:27
*** izza_ has quit IRC10:30
*** maciejjozefczyk has joined #openstack-nova10:31
*** Dinesh_Bhor has quit IRC10:31
openstackgerritTheodoros Tsioutsias proposed openstack/nova master: Introduce the PENDING instance state  https://review.openstack.org/56647310:34
*** cdent has joined #openstack-nova10:37
*** k_mouza has quit IRC10:39
*** k_mouza has joined #openstack-nova10:40
openstackgerritStephen Finucane proposed openstack/osc-placement master: tox: Hide deprecation warnings from stdlib  https://review.openstack.org/61188410:42
*** dtantsur|afk is now known as dtantsur10:50
*** mschuppert has joined #openstack-nova10:51
*** ttsiouts has quit IRC10:52
*** ttsiouts has joined #openstack-nova10:53
openstackgerritMerged openstack/nova master: Always read-deleted=yes on lazy-load  https://review.openstack.org/57519010:54
*** Dinesh_Bhor has joined #openstack-nova10:54
*** erlon has joined #openstack-nova10:57
*** ttsiouts has quit IRC10:57
*** Dinesh_Bhor has quit IRC10:59
*** kashyap has joined #openstack-nova11:02
kashyapstephenfin: Hi, I got (again) logged out of IRC here.  So if you pinged me, I haven't noticed.11:03
kashyapBut yes, this is on the top of my pile: https://review.openstack.org/#/c/575735/11:03
* kashyap looks now11:06
zigoI'm getting this in my nova-compute.log: http://paste.openstack.org/show/736696/ Where should I look to fix this?11:10
zigo"Placement API returning an error response: JSON does not validate: 0 is less than the minimum of 1" in nova-placement-api.log too ...11:11
*** xek has quit IRC11:12
zigobauzas: ^ help ? :)11:12
* bauzas looks11:12
gibizigo: '[{'DISK_GB': {'max_unit': 0, 'step_size': 1, 'min_unit': 1, 'reserved': 0, 'allocation_ratio': 1.0, 'total': 0}' it is an invalid part as total needs to be >= 111:14
gibizigo: if there is no resouce of DISK_GB then we should not include the resource class to the request at all11:14
zigogibi: Ok, so it lacks space on the Ceph backend, or something?11:15
kashyapgibi: Hi, I'm just doing another test for this: https://review.openstack.org/#/c/575735/11:16
gibizigo: somehow the nova-compute sees 0 disk11:16
zigoRight.11:16
kashyapgibi: And then add a comment there today.  Was off yesterday; sorry for the terrible delay.11:16
zigoI just wonder how/where it looks.11:16
gibikashyap: no worries, ping me if your fine with that patch an I will upgrade my vote11:17
*** k_mouza_ has joined #openstack-nova11:17
kashyapYeah, Georg is right; the patch is correct.  I just want to match my test w/ Georg's11:17
gibikashyap: sure, do your test :)11:17
kashyapIn progress; nested KVM envs...11:18
zigoOh, I believe there's an issue in my Ceph cluster.11:19
zigoNone of my PGs are active...11:19
gibizigo: https://github.com/openstack/nova/blob/3a8dd02c81e4d8393f5bb5e105bbff0e628594d6/nova/compute/resource_tracker.py#L72111:19
gibizigo: this is the place where the compute node calls the virt driver to get the available resourves11:19
zigogibi: Thanks !11:20
*** k_mouza has quit IRC11:20
*** k_mouza_ has quit IRC11:21
*** xek has joined #openstack-nova11:26
openstackgerritLee Yarwood proposed openstack/nova stable/pike: Update docs for _destroy_evacuated_instances  https://review.openstack.org/62120311:34
openstackgerritLee Yarwood proposed openstack/nova stable/pike: Add regression test for bug #1764883  https://review.openstack.org/62120411:34
openstackbug 1764883 in OpenStack Compute (nova) "Evacuation fails if the source host returns while the migration is still in progress" [Undecided,Fix released] https://launchpad.net/bugs/1764883 - Assigned to Lee Yarwood (lyarwood)11:34
openstackgerritLee Yarwood proposed openstack/nova stable/pike: compute: Ensure pre-migrating instances are destroyed during init_host  https://review.openstack.org/62120511:34
*** xek has quit IRC11:38
*** tbachman has quit IRC11:44
kashyapThere goes another 18 mintes for `make stack`.11:49
*** maciejjozefczyk has quit IRC11:49
*** maciejjozefczyk has joined #openstack-nova11:49
* kashyap wonders how 'fast' are people's DevStack setups here; e.g. how quickly (in minutes) can you spin up a new DevStack?11:50
gibikashyap: Total runtime        63211:54
gibikashyap: ubuntu 18.04, RECLONE=False, OFFLINE=False, horizon and tempest disabled11:55
kashyapgibi: Is that in seconds?11:55
kashyap(That 632)11:55
gibikashyap: yes11:55
kashyapgibi: And is that on baremetal?  Or in a VM (that's capable of nested virt)11:56
gibikashyap: in a kvm VM, with 8G ram and 2 vcpu11:56
kashyapInteresting.  I'm doing something same (on a pre-prod Haswell hardware from 5 years ago)11:57
kashyap  ENABLED_SERVICES=g-api,g-reg,key,n-api,n-cpu,placement-api,n-sch,n-cond,n-sproxy,mysql,rabbit,dstat,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta11:57
kashyapgibi: The above are my services ^11:57
kashyapI think I'll get a different timing if I run it on an SSD disk.11:57
gibikashyap: this is my local conf http://paste.openstack.org/show/736698/11:58
* kashyap clicks11:58
gibikashyap: I'm running this on a fairly new hp 840 G511:58
sean-k-mooneykashyap: devstack is still one of the fasest installes i have used11:59
kashyapgibi: That's mine: http://paste.openstack.org/show/736699/12:00
sean-k-mooneyin genreal even with RECLONE=True i rarely exceed 15Mins on baremetal or 20 on a vm12:00
kashyapsean-k-mooney: Oh, agreed.  It is "the" fastest, IMO12:00
sean-k-mooneykashyap: kolla-ansible is fater if you have the images12:00
kashyapI don't have RECLONE enabled.  (I'll assume it defaults to 'false')12:00
sean-k-mooneybut not by much12:01
sean-k-mooneykashyap: it does12:01
sean-k-mooneyi normally run in offline mode however12:01
kashyapWhat does 'OFFLINE' do, remind me again, please12:01
* cdent wonders when it becomes time to make a devstack-that-containerizes12:01
sean-k-mooneyit skips cloning and uses the git repos you have and also skps installing packages12:02
sean-k-mooneyit just configres teh services/dbs and runs them12:02
gibikashyap: my runs from a nvme ssd12:02
sean-k-mooneyon baremetal with OFFLINE=True a compute node will stack in under a miniute12:03
kashyapYeah, I NVMe on my laptop as well; mine is running on a slow remote machine; /me refreshes his local env.12:03
kashyap[OT] Meanwhile, check this out folks: http://qemu-advent-calendar.org/12:03
kashyapToday's image is by yours truly :D12:03
kashyap(If you liked it, we also organized it in 2016: http://www.qemu-advent-calendar.org/2016/)12:03
kashyapAnd this year is 15 years of QEMU.12:04
kashyapIf you're wondering WTH is this QEMU Advent, then read the "About" paragraph (https://www.qemu-advent-calendar.org/2018/#about)12:04
kashyap[quote]12:05
kashyapThe QEMU Advent Calendar 2018 features a QEMU disk image each day of December until Christmas. Each day a new package becomes available for download12:05
kashyap[...]12:05
kashyapThe disk images contain interesting operating systems and software that run under the QEMU emulator. Some of them are well-known or not-so-well-known operating systems, old and new, others are custom demos and neat algorithms.12:05
kashyap[/quote]12:05
kashyapIf anyone has ideas for images, more than welcome!12:05
gibikashyap: :)12:06
jangutterkashyap: Will December 24's image be a virtualization platform capable of running December 24's image?12:06
kashyapgibi: Some 3 years ago, I tried to compress an OpenStack DevStack install under 200 MB.  But I could only come up to 618 MB. :-(12:07
kashyap(The artifact is still here: https://kashyapc.fedorapeople.org/virt/advent-qemu/)12:07
kashyapjangutter: 24th / 25th is usually a custom surprise.12:07
kashyapjangutter: And even custom-coded by some community members.  Not going to ruin the surprise further :D12:07
jangutterkashyap: I still fondly remember the QNX boot floppies.12:07
kashyapSo here's an open challenge: I wonder if anyone can really get a "xz-compressed" DevStack disk image that is _under_ 280 MB.12:08
kashyapjangutter: Heh12:08
kashyapAn extremely minimal DevStack that fits under 280 MB in a disk image requires ugly hacks like removing directories (e.g doc, man pages) from the OS.  Tried some of those and gave up as it was a really uphill battle12:10
sean-k-mooneykashyap: i mean it does not sound that hard12:11
kashyapTry it.12:11
sean-k-mooneyespially if i dont have to include the git repos12:11
kashyapI dare you, double-dare you :D12:11
*** sayalilunkad has quit IRC12:11
kashyapsean-k-mooney: Yeah, that's also I considered12:12
kashyapRemoving the 'git repos'12:12
gibiI root for you, sean-k-mooney12:12
kashyapIt's going to take a full week of concentrated effort to prepare this image.  If one attempts at all12:12
* kashyap spent many evenings fighting with compression algos, doing dirty hacks in the OS (Operating System), trying to compress a few more MBs ...12:13
kashyap(In the past, i.e.)12:13
sean-k-mooneykashyap: well what base os did you start with12:13
kashyapsean-k-mooney: Bare-minimal Fedora -- with @base (300 packages or so)12:13
jangutterkasyap: you just want someone to port  OpenStack to OpenEmbedded12:14
kashyapThat's the kickstart: https://kashyapc.fedorapeople.org/virt/fed-minimal.ks12:14
kashyaps/base/core/12:14
*** ttsiouts has joined #openstack-nova12:14
jangutterkasyap: it should be pretty trivial if the image just loads a 1GB blob from the network into RAM.12:14
jangutterkashyap: use the "curl XX | sudo bash" compression method.12:15
kashyapHaha, yikes12:15
sean-k-mooneyjangutter: i mean that is how you install docker right :p12:16
sean-k-mooneyactully i wonder if my old docker devstack still works...12:18
*** ratailor has quit IRC12:19
jangutterkashyap: you know the history behind tcc? ( https://en.wikipedia.org/wiki/Tiny_C_Compiler )12:20
kashyapjangutter: Hehe, funny you mention that!  That was one of the images I prepared for the 2016 Advent Calendar!12:20
sean-k-mooneyjangutter: wasnt it for an obfucation compition intally12:20
kashyapjangutter: Checkout the day-16 image: https://www.qemu-advent-calendar.org/2016/#day-1612:21
janguttersean-k-mooney: yep. It just goes to show you that in every thousand cool hacks at least once something really useful emerges.12:22
*** khomesh has joined #openstack-nova12:23
sean-k-mooneyi used to use this as my primay dev env in a vm for about 6-9 months12:23
sean-k-mooneyhttps://github.com/SeanMooney/docker-devstack12:23
jangutterkashyap: Ah, day 13 of 2016 - turtles all the way down.12:24
sean-k-mooneyits 2 docker continers one of which had a dev environment e.g. ide, wireshark firefoxd, rdp server and another that run systemd and was where devstack woudl be run12:24
sean-k-mooneyjangutter: well i had a 10 node openstack dev cluster already12:25
kashyapjangutter: Yep, do read the README before you run it :D12:25
kashyapBecause it will create a "loop" of nested guests until your system runs out of memory (a.k.a crashes)12:26
jangutterkashyap: In other words it emulates nearly all my OpenStack setups perfectly.12:27
sean-k-mooneywhy do i get the feeling ill have to learn to debug casandra with all the edge stuff in the future12:29
janguttersean-k-mooney: You're just making doomed predictions that nobody believes.... like Cassandra did.12:30
sean-k-mooneyno im reading the mail tread that jsut said we need a weakly consitent backend for openstack to replace mysql and casandra is the only "db" i know of that is weakly consitent12:32
sean-k-mooneyjangutter: i was reading http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000606.html12:32
*** ttsiouts has quit IRC12:35
*** ttsiouts has joined #openstack-nova12:36
janguttersean-k-mooney: and yet.... python still has the GIL.12:36
*** ttsiouts has quit IRC12:36
*** ttsiouts_ has joined #openstack-nova12:36
*** sayalilunkad has joined #openstack-nova12:37
sean-k-mooneyjangutter: well that serves a different  usecase12:38
sean-k-mooneyeventual consitence does not mean you allow data races12:38
sean-k-mooneyyou just may be working on old data12:39
sean-k-mooneybut it was still valid data12:39
janguttersean-k-mooney: yeah, like RCU.12:39
sean-k-mooneyyep12:40
kashyapgibi: stephenfin: I think this can be merged, FWIW; added my +1 with notes: https://review.openstack.org/#/c/575735/ (handle logfiles for tcp-based consoles)12:40
gibikashyap: thanks. I upgrade my vote12:42
*** ttsiouts_ has quit IRC12:50
*** ttsiouts has joined #openstack-nova12:51
*** tbachman has joined #openstack-nova12:54
*** ttsiouts_ has joined #openstack-nova12:55
*** ttsiouts has quit IRC12:55
openstackgerritMerged openstack/nova master: Deprecate the nova-console service  https://review.openstack.org/61007513:05
*** janki has quit IRC13:08
mdboothThis is interesting. Profiling the execution with testtools.run of a single functional test, top results by individual call time: http://paste.openstack.org/show/736703/13:08
mdboothAm I reading that right, do we call Compute.API.__init__() 616 times?13:09
cdentsure looks that way13:09
mdboothThat seems suboptimal13:10
cdentalso a tshirt13:10
mdboothHehe13:10
*** k_mouza has joined #openstack-nova13:11
mdboothIn wallclock time, it's db access which is slow.13:12
sean-k-mooneywell at least its not a global13:12
*** ttsiouts_ has quit IRC13:14
mdboothI wonder where we're creating new Compute.API objects. It's not the cheapest operation.13:14
*** ttsiouts has joined #openstack-nova13:14
openstackgerritMerged openstack/nova master: Deprecate the nova-xvpvncproxy service  https://review.openstack.org/61007613:18
openstackgerritMerged openstack/nova stable/rocky: hardware: fix memory check usage for small/large pages  https://review.openstack.org/59941313:18
*** ttsiouts has quit IRC13:19
*** ttsiouts has joined #openstack-nova13:19
cdentmdbooth: you are likely on the edge of a deep deep hole. One which, if you choose to enter, will lead to riches beyond your dreams. But you might be there a while.13:22
*** udesale has joined #openstack-nova13:26
mdboothcdent: Hehe. Yeah, I'm going to reduce the scope to establishing why mriedem had to revert my functional test.13:34
mdboothAnd for that I need a stand-out, obvious culprit.13:34
mdboothThe above was just a surprising observation.13:34
*** dtantsur is now known as dtantsur|brb13:39
*** dpawlik has quit IRC13:41
*** dpawlik has joined #openstack-nova13:44
*** diliprenkila has joined #openstack-nova13:48
*** dpawlik has quit IRC13:49
*** diliprenkila has quit IRC13:49
*** k_mouza has quit IRC13:49
*** priteau has quit IRC13:50
*** dpawlik has joined #openstack-nova13:50
*** khomesh has quit IRC13:51
*** mriedem has joined #openstack-nova13:52
kaisersmdbooth:  regarding https://review.openstack.org/#/c/554195/ - some time ago you were'nt happy with the issue details, i tried to shed a bit more light on things, could you revisit this at some point pls?14:01
mdboothkaisers: Is the fix still required in the context of the systemd bugfix?14:03
kaisersmdbooth: yes, the problem ist that we added the --user flag. that fails for multiple reasons14:04
kaisers(i listed them in the commit message and the bugs new description14:04
kaisers)14:04
*** rodolof has joined #openstack-nova14:05
*** priteau has joined #openstack-nova14:05
mdboothkaisers: Which version of CentOS doesn't support --user?14:06
kaiserslatest as of two days ago14:06
mdboothWhich operating systems don't ship with the systemd fix?14:06
mdboothkashyap: So CentOS 7?14:06
kaisersThe problem ist not the fix for systemd but that it ships with an older systemd that does not support the --user flag14:06
kaiserscentos 7 comes with v219, the bug came up with v229 (i think) and was fixed with 23x14:08
kaisersFixed in v238 according to https://github.com/systemd/systemd/blob/master/NEWS14:09
kaisersmmh, sorry, bug described  with v233 in that news link, not v22914:10
*** bzhao__ has quit IRC14:18
openstackgerritChris Dent proposed openstack/nova master: Correct lower-constraints.txt and the related tox job  https://review.openstack.org/62297214:20
cdentstephenfin: ^ has an issue that needs to be resolved before it can be merged, but gets close14:20
gibistephenfin, mriedem: hi! the last two notification transformation patch is good to go from my perspective https://review.openstack.org/#/q/topic:bp/versioned-notification-transformation-stein+status:open14:25
*** psachin has quit IRC14:27
stephenfincdent: I'd kill the test, personally. lbragstad added it in 6244a442784 and, while I get what it's doing, I don't think the lower-constraints thing is more of an issue14:27
stephenfinGood to get a second opinion though14:27
stephenfinkaisers: The bug's introduced in v233 and fixed in v238?14:28
kaisersstephenfin: I'm not sure when it was introduced but that's the version for which it is reported in the systemd changelog.14:29
kashyapmdbooth: Hi, was AFK for a break; reading the scroll14:29
mriedemgibi: ack14:29
kaisersstephenfin: But as stated earlier, the more direct problem is that centos 7 comes with v219 and e.g. xenial with 229, both not supporting the --user flag14:29
kashyapmdbooth: Okay, I see kaisers already mentioned which version of systemd ships in CentOS and which one you care about14:30
kaisersstephenfin: scratch that last comment, v229 does support it14:30
kaisersso the old version issue only comes up with centos14:30
stephenfinkaisers: Yup, gotcha14:30
mdboothmriedem: Hey, about that functional test. Appreciate it made sense to take it out, but I'm trying to work out how we can safely get it back in. Problem is that we basically have no idea why it was failing and now way of finding out that I can think of without running it in CI. Could I perhaps fire up a non-voting job which enabled DEBUG output?14:31
mdboothI responded on the bug that I *suspect* a performance issue in CI.14:31
gibimdbooth: does it only fail in CI?14:31
mdboothgibi: Well it never fails locally. I've run it hundreds of times.14:31
stephenfinkaisers: OK, I can't speak for mdbooth but the last thing I'd like to see is a comment in the code itself explaining that this is only required for CentOS 714:32
mriedemmdbooth: i suspect globals14:32
gibimdbooth: let me try it and see if I can make it fail14:32
mriedemcould try the REQUIRES_LOCKING variable or whatever14:32
stephenfinPresumably once CentOS 8 (or 8.1) is out and starts gaining traction, we can remove this again14:32
mdboothstephenfin kaisers: Also there's a better way to detect systemd14:32
mriedembut i also suggesting writing a libvirt driver extension which overrides methods rather than mock them14:32
mriedem*suggested14:32
kaisersstephenfin: mdbooth: I'm still not sure why the third issue comes up, the --user flag failing when run via oslo_concurrency.processutils.execute. The error shown is similar to running --user as root but the python call should be executed as user nova afaics. I'm still digging into this but currently this fails, too14:32
mdboothmriedem: What makes you think globals?14:32
aspiersdansmith, mriedem: any thoughts on latest patch set of SEV spec? I tried to capture the results of our Berlin conversations, and sean-k-mooney seems happier now IIUC14:33
mriedemmdbooth: i'd have to look at the bug report again14:33
kaisersmdbooth: regarding the systemd detection i'm all ears, i'm using the method danpb recommended to me some time ago14:33
mriedemin the middle of a different evacuate issue with CI14:33
mriedemaspiers: i haven't looked yet14:33
*** janki has joined #openstack-nova14:33
aspiersmriedem: for your convenience :) https://review.openstack.org/#/c/609779/14..15/specs/stein/approved/amd-sev-libvirt-support.rst14:33
kaisersstephenfin: I'll add the CentOS / systemd release version info to the change14:34
stephenfingibi: ack. I need to get some reviews done. Only just finished reading 3 weeks of emails (yes, it is already Wednesday :-\)14:34
aspiersmriedem: it looks like only 16MB will need to be accounted for per VM in CONF.reserved_host_memory_mb14:35
gibistephenfin: no worries, this is the last two patches and we have still time til milestone 2 :)14:35
*** k_mouza has joined #openstack-nova14:35
mdboothmriedem: Interesting. What difference would it make to override rather than mock? Are you thinking class variables?14:36
mriedemaspiers: and that's something the operator needs to calculate and configure that option based on how many sev instances they think will be on a given host?14:36
kaisersstephenfin: oh, i misread, you do _not_ want that change pointed out to be specifically for centos 7, sry14:36
mriedemmdbooth: i've just seen enough weird mock failures with the libvirt driver to know i don't trust it14:37
mriedemrunning tests in isolation is fine,14:37
aspiersmriedem: yes, although I suspect in practice that value would be large enough anyway to swallow that amount without extra thought by the operator14:37
mriedembut then running full libvirt driver test suites i've seen weird mock failures14:37
mriedemaspiers: and why can't the driver just report the overhead value if it's a sev-enabled vm?14:37
mriedem*of14:37
mriedemgah14:38
aspiers:)14:38
stephenfinkaisers: No, I do want it14:38
kaisersstephenfin: ok, I'll add14:38
stephenfinkaisers: ta14:38
mriedemaspiers: so yeah, why can't the driver just report the overhead calculation per vm?14:39
aspiersmriedem: it could do that, although as I said in Berlin this is not an increased overhead compared to the non-SEV case14:40
mriedemmaybe we won't use it b/c of placement we won't do the ram claim14:40
*** k_mouza has quit IRC14:40
mriedemaspiers: yeah i still don't understand that,14:40
aspiersmriedem: if you read the latest patch set hopefully all will become clear :)14:40
mriedemi'm very confused by saying, "well operators might need to bump the reserved ram option if the compute hosts these types of instances, but at the same time maybe not because there is no increase in ram usage on the host"14:41
aspiersI've tried very hard to explain this in detail in patchset 1514:41
aspiersplease take a look, and if it's still not clear then let me know14:41
dansmithmriedem: did you see the libvirt statement that the overhead amount is unknowable?14:41
mriedemi've seen that before yes14:42
mriedembut aspiers just said it's about 16MB per Vm14:42
aspiersdansmith: it turns out that it's a more knowable than we thought :) again, patchset goes into great detail on this :)14:42
aspiers*patchset 1514:42
* dansmith throws his hands up14:42
mriedemright, so,14:42
mriedemat this point, i will likely just +2 out of exhaustion on the spec and we'll push it through and doc the hell out of this in the admin guide14:43
mriedemb/c it's a snowflake feature14:43
mriedembeing brutally honest14:43
mriedemnot because i'm against it,14:43
mriedemi'm just too dumb to understand it14:43
aspiershaha, I highly doubt that :)14:43
mriedemno really i'm not very smart14:44
aspiersbut totally understand that you have limited time for it14:44
aspiersrest assured I will be doc'ing the hell out of it14:44
aspiersthat intention is also captured in the spec14:44
aspiershttp://logs.openstack.org/79/609779/15/check/openstack-tox-docs/eaf9d54/html/specs/stein/approved/amd-sev-libvirt-support.html#documentation-impact14:45
lbragstadstephenfin cdent hitting issues with I7cd27fe8c39ddfc6ec20f4cfe4d62912d4cebaa4 ?14:45
*** mlavalle has joined #openstack-nova14:46
cdentlbragstad: causes issues in https://review.openstack.org/#/c/622972/ (see commit message)14:46
lbragstadyeah - i don't have a strong attachment to that test...14:48
lbragstadi changed something though so i wanted to test it14:48
*** eharney has joined #openstack-nova14:48
mriedemstill looking for another core on the allocation ratio data migration patch https://review.openstack.org/#/c/613499/ it's pretty easy14:48
mriedemand we can flush this runway bp today14:48
dansmithyeah, I've had that open since middle of yesterday14:49
dansmithI'll get some protein to go with my caffeine and see if I can make myself do it14:51
mriedempower through14:53
mriedembauzas: i'm removing reshaper from the runway queue again14:53
mriedemdec 4 came and went14:53
stephenfinmriedem, dansmith, cdent, gibi, anyone else around: are you happy with me doing this? https://review.openstack.org/62298714:55
*** amodi has quit IRC14:55
stephenfinI'm removing osc-placement notifications from this channel14:55
cdentwfm14:55
stephenfin*gerritbot notifications14:55
mriedemsure14:56
mriedemreminds me that osc-placement is falling behind on parity14:56
gibistephenfin: I think the original reason to keep that was to know when we need to keep placement changes in sync in placement code in nova and in the separated placement14:56
mriedemosc-placement14:57
mriedemnot placement14:57
*** awaugama has joined #openstack-nova14:59
*** Sundar has joined #openstack-nova14:59
bauzasmriedem: ack, ok14:59
*** cfriesen has joined #openstack-nova14:59
bauzasmriedem: I'll tho ping people later about reviewing it ;)15:01
*** yan0s has joined #openstack-nova15:02
yan0shi guys, I'm having a trouble with policy.json file in /etc/nova/15:04
mriedembauzas: you also have some review comments to address it looks like15:04
yan0show can I debug if it actually being parsed?15:04
bauzasmriedem: just the ones from artom but yeah :)15:04
bauzasI was waiting for other comments15:04
mriedemyan0s: interesting question which someone was having trouble with last week, i don't have a great answer...15:06
mriedemi'm not sure if enabling debug logging will dump anything from oslo.policy when the check is performed?15:06
yan0sI have set  "compute:create": "is_admin:True" and  "compute:create": "is_admin:True"15:06
mriedemthose are the same15:07
yan0sbut users of any role can still create instances15:07
yan0sthat's wrong right?15:07
yan0s"os_compute_api:servers:create": "is_admin:True"15:07
yan0sthat was the second rule15:07
mriedemwhat is your is_admin definition?15:07
mriedemand which release is this?15:07
yan0srocky15:07
yan0sbut with juju15:07
mriedemand [oslo_policy]/policy_file in nova.conf is not changed from the default policy.json?15:08
mriedemand policy.json is in /etc/nova/policy.json or somewhere else?15:08
mriedemwe do have this CLI https://github.com/openstack/nova/blob/master/nova/cmd/policy.py#L73 but it's not really what you're looking for probably15:09
yan0spolicy.json is in /etc/nova/15:09
yan0sis there a field in nova.conf that defines the policy.json directory?15:10
mriedemthe options are in this group https://docs.openstack.org/nova/latest/configuration/config.html#oslo-policy15:11
*** lpetrut has joined #openstack-nova15:12
*** mmethot has joined #openstack-nova15:12
*** dtantsur|brb is now known as dtantsur15:13
openstackgerritMatt Riedemann proposed openstack/nova master: Add docs for (initial) allocation ratio configuration  https://review.openstack.org/62258815:16
yan0s[oslo_policy] was missing from the nova.conf15:16
yan0sthanks mriedem!15:16
mriedemyan0s: well, it doesn't need to be in nova.conf if you're not changing defaults15:17
mriedemif the defaults are what you need anyway, it should just work15:17
yan0sadding policy_file = policy.json under it fixed it15:17
mriedemhmm, that shouldn't matter, that's already the default in code...15:17
mriedemlbragstad: ^15:17
yan0sno I need a specific configuration15:17
mriedemyan0s: why?15:18
yan0sI need user roles with different access rights15:18
mriedemsure, that's in policy.json though,15:19
mriedemnova.conf is already pointing at policy_file=policy.json by default15:19
yan0sis there a better way of doing this?15:19
mriedembecause of defaults in the code15:19
mriedemhttps://docs.openstack.org/nova/latest/configuration/config.html#oslo_policy.policy_file Default:policy.json15:19
mriedemoverriding policy rules is fine and expected, i just wonder why the config defaults weren't working15:20
mriedembecause someone was having a similar issue last week15:20
yan0scould it be a juju charm specific issue?15:20
mriedemi think the person last week was using kolla but it's hard to remember15:20
* lbragstad reads scrollback15:20
mriedemlbragstad: apparently [oslo_policy]/policy_file default not being read for some reason in some deployments15:21
mriedemexplicitly putting policy_file=policy.json in nova.conf fixed it, but that shouldn't be necessary given it's already the default value15:21
lbragstadas far as the logging goes (which doesn't seem to be the root issue) we did just release a new version of oslo.policy that allows for better logging15:21
lbragstadhttps://review.openstack.org/#/c/619260/15:22
*** itlinux has quit IRC15:22
lbragstadand the default from policy.json isn't changing for what's documented in code?15:22
mriedemno they are using /etc/nova/policy.json15:23
mriedem"adding policy_file = policy.json under it fixed it"15:23
mriedemwe don't test policy overrides in tempest/dsvm jobs, but patrole should, i should find one of those jobs and see if they have to explicitly configure this in nova.conf15:25
kashyapIf anyone has a few spare minutes, this already has a +2: https://review.openstack.org/#/c/620327/ -- "libvirt: Refactor handling of PCIe root ports"15:25
stephenfinmriedem: Would you mind taking a look at this when you have time? https://review.openstack.org/#/c/554523/15:26
stephenfinkashyap: on it15:26
kashyapstephenfin: Thanks!  The more important bit I need to work out is a clean solution for long "TODO" in the code...15:27
stephenfinjust reading that atm15:27
dansmithmriedem: when you get a sec, check my comment on the initial ratio migration patch15:28
dansmith(please)15:28
*** munimeha1 has joined #openstack-nova15:29
* kashyap gets back to writing an ugly script that sets up TLS for live migration w/o shared storage with DevStack15:29
stephenfinkashyap: I think you've already stated the fix, no? (third paragraph of that TODO)15:29
*** amodi has joined #openstack-nova15:29
kashyapstephenfin: Yep, so you actually read it, then :D15:30
stephenfinI'm 100% for detailed code comments ;)15:30
stephenfin...so long as they're kept up-to-date15:30
kashyapThanks!  I spent unjustifiable time to write that comment :D15:31
stephenfinkashyap: In any case, that's a future problem. +Wd15:31
kashyapstephenfin: The "hidden" bit is to workout XML wrangling to get the machine type from the right XML bit15:31
kashyapThanks!15:31
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove port allocation during detach  https://review.openstack.org/62242115:31
kashyapstephenfin: Thanks!  That's a near-term problem, as "certain Linux distributions" might / will change default machine types15:33
yan0salso, does any service neds to be restarted for updates in policy.json to take effect?15:33
mriedemdansmith: replied15:33
mriedemjust tell me what you want me to do and i'll make the change15:33
*** amodi has quit IRC15:35
*** amodi_ has joined #openstack-nova15:35
mriedemyikun: fyi, https://blueprints.launchpad.net/nova/+spec/live-migration-force-after-timeout is now in a runway slot until dec 1915:35
*** amodi_ is now known as amodi15:35
*** awalende_ has quit IRC15:35
dansmithmriedem: well, I was going to volunteer to fix it since it's my complaint, but I think I'd rather it not tweak updated_at unless it's going to do a full refresh so it's consistent15:35
mriedemi'll do it, i've got the series local already - i've been tweaking it along the way for my minor stuff15:36
mriedemcan you take a look at the next in line before i do?15:36
mriedemthe one that adds the cli hook15:36
dansmithokay, other than that I'm good with it15:36
dansmithokay15:36
*** amodi has quit IRC15:36
dansmithmriedem: that migration is going to do deleted records too, is that the intent?15:37
dansmithmriedem: two comments on there15:39
mriedemew, no i dont think we want that15:40
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Body verification for the lock action  https://review.openstack.org/62283515:40
*** amodi has joined #openstack-nova15:40
mriedemi'll address those as well15:41
mriedemstephenfin: done15:41
stephenfinThank you15:41
mriedemstephenfin: note that the pike backport hasn't been released yet15:41
mriedemeven though it was done on august15:42
mriedem*merged15:42
mriedemso....probably time to think about doing an os-vif pike release15:42
mriedemstephenfin: do you want to queue that up?15:42
stephenfinmriedem: Sure thing15:43
*** ttsiouts has quit IRC15:45
openstackgerritMatt Riedemann proposed openstack/nova master: Use tempest [compute]/build_timeout in evacuate tests  https://review.openstack.org/62301115:47
*** k_mouza has joined #openstack-nova15:47
*** lpetrut has quit IRC15:49
*** k_mouza has quit IRC15:49
*** k_mouza has joined #openstack-nova15:50
*** tssurya has quit IRC15:54
stephenfinmriedem: https://review.openstack.org/62301615:54
openstackgerritMerged openstack/os-vif stable/ocata: Check if interface belongs to a Linux Bridge before removing  https://review.openstack.org/55452315:54
mriedemwoohoo, summit and ptg are separate, $700 summit only with no lunch, ptg is $400 w/ lunch (oh thank you), and $1200 for both15:55
mriedemso longer week, more expensive, no lunch, what's not to love?!15:55
tbachmando they have gift-bags?15:55
mriedembyogb15:56
dansmithcome on, even for atcs?15:56
tbachmandarn15:56
mriedemno idea about atcs15:56
*** ttsiouts has joined #openstack-nova15:56
stephenfinmriedem: Oh, I wonder if we should include this too, if we're doing a release? https://review.openstack.org/#/c/505731/15:56
stephenfinI'll prepare an ocata release too. Nothing else on the backlog15:56
cdentefried: is https://review.openstack.org/#/c/613613/ still alive (kill safe connect)15:58
efriedcdent: I think so. Just needs core reviews.15:58
cdentroger, just doing a to do list review, and it came round16:00
*** slaweq has quit IRC16:02
*** tbachman has quit IRC16:03
mriedemstephenfin: question in https://review.openstack.org/#/c/505731/416:04
stephenfinlooking16:04
melwitto/16:06
*** awalende has joined #openstack-nova16:08
*** tbachman has joined #openstack-nova16:08
mriedemstephenfin: i guess it's always been there? https://kernel.googlesource.com/pub/scm/linux/kernel/git/shemminger/bridge-utils/+/38837a8e19540690c80f5a3b0971e0e1495a4a3e/brctl/brctl_cmd.c#12416:08
stephenfinmriedem: :D I was looking at the exact same thing16:09
lbragstadyan0s i think it depends on how the service init's the oslo.policy enforcer object16:09
openstackgerritChris Dent proposed openstack/nova master: Correct lower-constraints.txt and the related tox job  https://review.openstack.org/62297216:10
lbragstadyan0s there is a configuration option to reload policy rules when they change on disk, i believe16:10
lbragstads/configuration option/option parameter to the enforcer/16:10
mriedemlbragstad: that would be here https://github.com/openstack/nova/blob/master/nova/policy.py#L6816:10
stephenfinmriedem: It's in Ubuntu 16.04 http://manpages.ubuntu.com/manpages/xenial/man8/brctl.8.html and there's no changelog section, so I assume it's always been there too16:11
lbragstadmriedem yeah - some documentation on it here https://docs.openstack.org/oslo.policy/latest/reference/api/oslo_policy.policy.html#oslo_policy.policy.Enforcer16:11
*** jmlowe has joined #openstack-nova16:12
*** awalende has quit IRC16:12
mriedemhmm, so nova-api isn't dynamically reloading rules?16:13
mriedemi only see ENFORCER.load_rules() called when we initialize it16:13
mriedemyan0s: did you restart nova-api when you changed nova.conf to set the policy_file option?16:14
sean-k-mooneyo/16:14
gibimdbooth: I run ~30 consecutive full nova functional tests with random test case order to reproduce the test failure but I failed16:15
gibimdbooth: so I think it is more likely a timing issue as the gate runs the test slower than I16:16
sean-k-mooneystephenfin: brctl?16:16
stephenfinyes16:16
sean-k-mooneyi was wondering whtat the context was16:17
*** pcaruana has quit IRC16:18
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Add workaround to remove instance dir during cleanup with rbd  https://review.openstack.org/61847816:19
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove port allocation during detach  https://review.openstack.org/62242116:20
*** maciejjozefczyk has quit IRC16:21
*** maciejjozefczyk has joined #openstack-nova16:23
openstackgerritMerged openstack/os-vif stable/pike: ovs-hybrid: should permanently keep MAC entries  https://review.openstack.org/50573116:27
*** amodi has quit IRC16:27
stephenfingibi: I know no one wants to see that patch, but the notification samples would be so much easier to grok if they were a consistently sorted 😅 (especially the common payloads)16:28
yan0syes I did restart it16:29
yan0snova-api-os-compute16:29
yan0snova-api-os-compute.service16:29
*** janki has quit IRC16:30
mriedemyan0s: ok so it might have just been that you restarted the service that reloaded the policy file rather than needing to configure nova.conf16:31
mriedemi thought we always dynamically read the policy file on each auth check16:31
*** amodi_ has joined #openstack-nova16:31
mriedembut i guess not16:31
mdboothgibi: Yeah, I *suspect* that it's just a canary. I think that create is taking close to the 5 second timeout, and when the gate is clogged its the first thing to get pushed over.16:31
cfriesensean-k-mooney: regarding https://review.openstack.org/#/c/620959/, I believe the traits are already defined as part of https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/report-cpu-features-as-traits.html16:32
mdboothgibi: Problem is I can't prove it. I spent time this morning looking at a profiler and I can see a couple of opportunities for making that test slightly faster, but if I guessed wrong it's just going to be merge ping-pong.16:32
*** Luzi has quit IRC16:33
mdboothgibi: The other thing, of course, would be to increase the timeout.16:33
*** gyee has joined #openstack-nova16:34
*** tbachman has quit IRC16:35
openstackgerritMatt Riedemann proposed openstack/nova master: Add ratio online data migration when load compute node  https://review.openstack.org/61349916:36
openstackgerritMatt Riedemann proposed openstack/nova master: Add compute_node ratio online data migration script  https://review.openstack.org/60999516:36
openstackgerritMatt Riedemann proposed openstack/nova master: Note the aggregate allocation ratio restriction in scheduler docs  https://review.openstack.org/62071316:36
openstackgerritMatt Riedemann proposed openstack/nova master: Add docs for (initial) allocation ratio configuration  https://review.openstack.org/62258816:36
mriedemdansmith: done ^16:36
*** ttsiouts has quit IRC16:36
*** itlinux has joined #openstack-nova16:37
*** ttsiouts has joined #openstack-nova16:37
dansmithmriedem: got 'em16:39
mriedemthanks16:39
mriedemapproving the non-docs changes16:39
mdboothmriedem: What do you think would be acceptable criteria for re-proposing that patch, btw?16:40
mriedemthe racy test?16:40
dansmithmriedem: I'll look at the following ones in a bit16:40
mriedemdansmith: thanks16:40
mdboothmriedem: Yeah.16:40
mriedemmdbooth: idk, maybe not using mocks/stubs on the libvirt driver in functional tests?16:40
mriedemlike i said, might be better to subclass the libvirt driver and override what you need16:41
mdboothmriedem: I suspect the *race* is against a slow test worker.16:41
mdboothI can't see any evidence of an ordering thing.16:41
mriedemcould also add the REQUIRES_LOCKING = True variable, but i don't know if that will help anything16:41
mdboothWhich isn't to say that it's not an ordering thing, but without being able to reproduce it we're really just trying stuff.16:41
mdboothSo... when have we tried enough stuff?16:41
mriedemjust, doing file i/o and mocks and such in a functional test like that with a real driver - i'm not really surprised there were side effects16:41
*** ttsiouts has quit IRC16:42
mriedemmdbooth: i can't answer that16:42
mdboothmriedem: Me neither :/16:42
*** dpawlik has quit IRC16:42
mdboothHowever, like I say I don't see any evidence of side effects, or globals, or anything like that.16:42
mdboothThe failures look like it was just too slow to me.16:43
mdboothI can post performance improvements.16:43
*** dpawlik has joined #openstack-nova16:44
efriedMy boss just made me look at my remaining vacation time for the year. Turns out... I'm gone until January, starting this afternoon. melwitt mriedem Sundar edmondsw FYI.16:45
mdboothefried: \o/16:45
aspiersefried: whoa nice, enjoy ;-)16:45
efried:) thanks16:45
lbragstadmriedem yan0s fwiw - i keystone doesn't either, but i can't really think of a reason not to?16:46
lbragstads/i//16:46
mriedemlbragstad: perf?16:46
melwittefried: thanks for the heads up. enjoy your time off :)16:46
*** artom has quit IRC16:46
*** udesale has quit IRC16:46
mriedemefried: ack16:46
lbragstadmriedem possibly - i could time it16:46
*** artom has joined #openstack-nova16:47
*** udesale has joined #openstack-nova16:47
mriedemefried: i'll just rebase and abuse any changes of yours that i need16:47
efriedmriedem: It would be easier just to merge them right now.16:47
mriedembah16:47
*** tbachman has joined #openstack-nova16:47
mriedemmy queue is already deep and i haven't started on either of the 2 things i said i'd do today16:47
Sundarefried: Have fun and Happy Holidays!16:49
efriedThanks16:49
openstackgerritEric Fried proposed openstack/nova master: Add cyborg client to requirements  https://review.openstack.org/62302616:51
openstackgerritEric Fried proposed openstack/nova master: WIP: Cyborg PCI handling  https://review.openstack.org/62302716:51
*** awalende has joined #openstack-nova16:55
yan0sNo I definitely needed to change the nova.conf16:56
*** awalende has quit IRC16:56
yan0sI was restarting the service in every try16:56
yan0s@lbragstad16:57
yan0s@mriedem16:57
mriedemhmm, well i'm stumped then16:58
lbragstadyeah - by default, oslo.policy isn't going to reload policy files on disk16:58
mriedemlbragstad: but they shouldn't have had to explicitly configure nova.conf with [oslo_policy]/policy_file = policy.json16:58
mriedemsince that's the default in code16:58
*** priteau has quit IRC16:59
mriedemanyway, probably just something i'd need to mess with in devstack to see if i can recreate it16:59
lbragstadthis sounds like two different issues16:59
lbragstadyan0s if you're in #openstack-oslo this might be more relevant to talk about there17:03
*** priteau has joined #openstack-nova17:07
openstackgerritEric Fried proposed openstack/nova master: Add cyborg client to requirements  https://review.openstack.org/62302617:07
openstackgerritEric Fried proposed openstack/nova master: WIP: Cyborg PCI handling  https://review.openstack.org/62302717:07
*** slaweq has joined #openstack-nova17:08
*** priteau has quit IRC17:08
*** priteau has joined #openstack-nova17:10
*** yan0s has quit IRC17:16
*** wolverineav has joined #openstack-nova17:17
*** slaweq has quit IRC17:20
*** slaweq has joined #openstack-nova17:21
mriedemgibi: not sure if you saw my comment in that notifications patch, but if we planned on switching the default format to unversioned, marking it as deprecated would be weird17:25
*** helenafm has quit IRC17:28
openstackgerritStephen Finucane proposed openstack/nova master: Address nits on I08991796aaced2abc824f608108c0c786181eb65  https://review.openstack.org/61432217:29
openstackgerritStephen Finucane proposed openstack/nova master: Address nits on I1f1fa1d0f79bec5a4101e03bc2d43ba581dd35a0  https://review.openstack.org/61432317:29
stephenfinmriedem: I'd suggested going the other way, leaving it to deployers to manually set things if some tool can't consume versioned notifications. Maybe that's too severe though17:31
*** sahid has quit IRC17:32
mriedemstephenfin: by some tool you mean *all* tools?17:32
mriedemat least all tools in openstack17:32
mriedemas far as i know, nothing in openstack which consumes nova notifications uses versioned notifications17:33
mriedemand it's on us to add that support to those already understaffed projects17:33
stephenfinApparently so. I thought there were some, based on some mailing list post from a while back17:33
sean-k-mooneydidnt we say we were removing the unversioned notification in denver17:33
mriedemthere are lots of projects that consume nova's notifications, but not versioned17:33
mriedemno17:33
mriedemwe said we'd never remove them17:34
stephenfinNot removing - just deprecating17:34
stephenfinYeah17:34
stephenfinuntil they eventually break, presumably17:34
sean-k-mooneysorry your right we said we woudl keep them but not add new unversioned notifications17:34
mriedemwe haven't been adding new unversioned notifications since...we had versioned notifications17:35
mriedemgetting telemetry over the hump is probably the biggest hurdle17:35
mriedemwithin openstack17:35
sean-k-mooneyright but i gues what i ment is freezing the unversioned code.17:35
mriedemit's already frozen17:35
sean-k-mooneyincluding bug fixes17:36
mriedemdeprecation would just be signaling, don't write new stuff with this17:36
mriedemi can't remember the last time we had a bug fix that dealt with unversioned notifications17:36
mriedemmy concern is https://bugs.launchpad.net/nova/+bug/180565917:37
openstackLaunchpad bug 1805659 in OpenStack Compute (nova) "nova notifications hammering the message bus" [Low,Confirmed]17:37
mriedemwhich jmlowe brought up17:37
*** k_mouza_ has joined #openstack-nova17:37
mriedemtoday we default to send both, versioned and unversioned, which kills rpc17:37
mriedems/kills/doesn't help/17:37
mriedemif nothing is consuming versioned yet, then that is a shitty default imo17:37
mriedemi know we want to get people off the old stuff, but no one is doing that17:38
sean-k-mooneyya just readign that makes sense but im kind of reluctent to default to old17:38
mriedemand honestly without a schema that consumers can use, it's probably not really something they care about17:38
*** cdent has quit IRC17:38
mriedemsince they can't do version negotiation17:38
sean-k-mooneycan we default to versioned going forward instead and tell peopel to enable both/unversions if they have a service that uses them?17:39
mriedemwe can do whatever17:39
mriedemme? i'm going to go make lunch and try to think about something else.17:40
*** mriedem is now known as mriedem_away17:40
sean-k-mooneyok :)17:40
*** k_mouza has quit IRC17:40
*** k_mouza_ has quit IRC17:41
*** mrhillsman has quit IRC17:51
*** zioproto has quit IRC17:51
*** mrhillsman has joined #openstack-nova17:52
*** zioproto has joined #openstack-nova17:52
cfriesensean-k-mooney: I replied to your comments on https://review.openstack.org/#/c/620959/17:54
*** Sundar has quit IRC17:55
sean-k-mooneycfriesen: yep i know the feature flags are already in placement but they currently mean find ma a host wtih X not enable X on the vm17:55
sean-k-mooneycfriesen: qemu does not always requrie the host to have X to enable X for the guest17:55
sean-k-mooneyAVX is an exampel you can enable AVX in a guest as long as the host has SSE4 suport17:56
cfriesensean-k-mooney: not quite true.  according to the spec the exposed traits represent the features of the CPU model that you would get in the guest.17:56
sean-k-mooneythe new spec17:57
cfriesenIn https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/report-cpu-features-as-traits.html it says "The libvirt virt-driver should only return the CPU features which are available to the guest."17:57
sean-k-mooneycfriesen: we currently report all cpu feature on the host i belive17:57
sean-k-mooneyi dont think we filter by the configured cpu model17:58
*** ttsiouts has joined #openstack-nova17:58
cfriesensean-k-mooney: see _get_cpu_traits() in virt/libvirt/driver.py17:59
openstackgerritMerged openstack/nova master: Add a bug tag for nova doc  https://review.openstack.org/61943417:59
cfriesensean-k-mooney: "if mode is 'custom', use cpu_model to generate CPU features"18:00
*** N3l1x has joined #openstack-nova18:01
sean-k-mooneywhen the traits were defiend in os tratis they were ment to model the host capablity though18:02
cfriesenso on x86-64 specifying a trait currently *should* result in a guest with the specified CPU feature present, as far as I can tell18:02
*** sridharg has quit IRC18:02
sean-k-mooneycfriesen: that seams to be the case but that was not the intent of the trait18:02
cfriesenwhy would you care whether a host has a feature if you can't get it in the guest?18:02
*** derekh has quit IRC18:03
sean-k-mooneythe trait was ment to allow you to ensure it was not emulated in software18:03
sean-k-mooneythe way it currently works i could get avx emulated in software using sse4 which will allow the application to run but will have worese perfromance18:04
cfriesenso I guess you'd need to still use the ComputeCapabilitiesFilter  for that?18:04
cfriesencombine that with requesting the trait in the flavor, and you'd get both, no?18:04
sean-k-mooneyit still misses the point that HW_CPU_X86_AVX is ment to only be used if the host support it in hardware18:06
cfriesensean-k-mooney: if ComputeCapabilitiesFilter fails hosts that don't have it in hardware, and specifying the trait requests that it's available in the guest, that seems like it'd work.18:07
cfriesenThe other option is to simply not specify CPU models with features unsupported by your hardware...which is up to the operator.18:07
sean-k-mooneyit would work but i realy dislike that we are not useing the HW e.g. hardware namespace to model only hardware things18:08
cfriesensean-k-mooney: have you got any specs/docs for the intent of the trait?18:08
sean-k-mooneythat was my intent when i asked for namespacing in os traits https://github.com/openstack/os-traits/commit/23d81d4451dd29c23150b29b8fa9d3025ee8878f#diff-f290aedb8b7fdc21b2b04be76222f6c318:09
*** dave-mccowan has joined #openstack-nova18:10
sean-k-mooneythe HW namespace was for hardware features18:10
cfriesenwe use the "hw" namespace for all sorts of virtual hardware stuff18:10
sean-k-mooneywe only started doing that this cylce with the vtpm spec18:11
cfriesennumber of numa nodes, cpu distribution between numa nodes, etc18:11
cfriesenoh, you mean for trait18:11
sean-k-mooneyyes18:11
sean-k-mooneyanyway looks like we approved and implmented https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/report-cpu-features-as-traits.html#libvirt so we are stuck with it18:12
sean-k-mooneyi was hopping not to make the same mistakes we did with flavors in os-traits but if i can rely on traits to modele this correctly i will always need filters18:13
cfriesenI think it'd be worth adding something to the release notes warning operators to consider this when adding CPU models to the list.18:13
cfriesenIn practice, I would expect that most operators advertising high-performance would not want emulated features, no?18:13
sean-k-mooneyya or better to the config option so its in the docs for setting the models/extra flags18:14
cfriesenright, that makes sense18:14
sean-k-mooneycfriesen: yes but as a tenat i can no longer rely on it being a hardware feature in a public cloud18:14
sean-k-mooneyi get the opertor usecase but this originally came form the mano folks18:15
cfriesengotta run for lunch18:15
sean-k-mooneyo/18:15
sean-k-mooneyill comment back on the spec.18:16
sean-k-mooneysince we are already missuing them wwe might as well continue18:16
*** ralonsoh has quit IRC18:21
*** alex_xu has quit IRC18:24
*** alex_xu has joined #openstack-nova18:26
*** wolverineav has quit IRC18:34
*** wolverineav has joined #openstack-nova18:37
*** priteau has quit IRC18:38
*** ccamacho has quit IRC18:40
*** wolverineav has quit IRC18:41
*** wolverineav has joined #openstack-nova18:43
*** dpawlik has quit IRC18:45
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Per aggregate scheduling weight (spec)  https://review.openstack.org/59930818:45
*** dpawlik has joined #openstack-nova18:46
mriedem_awayjohnthetubaguy: bauzas: ^ when you're around, i'm +2 on ^ now - seems like a good compromise from the per-flavor weights it started from18:46
mriedem_awaymgagne_: since you said you had something like this downstream already ^ it would be good if you can ack that works for you as well18:46
*** mriedem_away is now known as mriedem18:46
*** manjeets_ is now known as manjeets18:47
*** alex_xu has quit IRC18:47
*** alex_xu has joined #openstack-nova18:50
*** ccamacho has joined #openstack-nova18:50
*** eharney has quit IRC18:52
mgagne_mriedem: done. thanks18:52
*** ccamacho has quit IRC19:00
*** wolverineav has quit IRC19:01
*** betherly has joined #openstack-nova19:01
*** betherly has quit IRC19:06
*** wolverineav has joined #openstack-nova19:16
*** wolverineav has quit IRC19:16
*** wolverineav has joined #openstack-nova19:17
*** wolverineav has quit IRC19:29
*** ttsiouts has quit IRC19:30
*** ttsiouts has joined #openstack-nova19:30
*** wolverineav has joined #openstack-nova19:31
openstackgerritJack Ding proposed openstack/nova master: Preserve UEFI NVRAM variable store  https://review.openstack.org/62164619:32
*** ttsiouts has quit IRC19:35
*** wolverineav has quit IRC19:39
*** wolverineav has joined #openstack-nova19:43
*** wolverineav has quit IRC19:46
*** wolverineav has joined #openstack-nova19:46
*** tbachman has quit IRC19:53
*** spatel has joined #openstack-nova20:03
*** jaosorior has joined #openstack-nova20:03
*** eharney has joined #openstack-nova20:05
*** dtantsur is now known as dtantsur|afk20:06
*** wolverineav has quit IRC20:06
*** wolverineav has joined #openstack-nova20:07
*** wolverineav has quit IRC20:18
*** peeejayz has quit IRC20:22
*** k_mouza has joined #openstack-nova20:30
*** Swami has joined #openstack-nova20:32
*** k_mouza has quit IRC20:34
*** erlon has quit IRC20:35
*** tbachman has joined #openstack-nova20:43
*** dpawlik has quit IRC20:44
*** maciejjozefczyk has quit IRC20:48
*** slaweq has quit IRC20:49
*** wolverineav has joined #openstack-nova20:56
*** wolverineav has quit IRC20:58
*** wolverineav has joined #openstack-nova20:58
*** dpawlik has joined #openstack-nova20:59
*** tssurya has joined #openstack-nova20:59
*** dpawlik has quit IRC21:03
efriedjaypipes, mriedem: Are you waiting for CERN to deploy on ironic nodes before reviewing series at https://review.openstack.org/#/c/615677/ ?21:09
mriedemefried: not really21:10
mriedemi'm not intentionally avoiding it21:11
efriedCool, I know you're stretched pretty thin.21:11
mriedembut my waist line continues to grow21:11
mriedemhere is an easy gate related fix https://review.openstack.org/#/c/623011/21:13
mriedemefried: i've been ignoring some other stuff for awhile so trying to burn that list down first21:15
mriedemreviewing your series will be my xmas gift to you21:15
efriedmriedem: +2 on the tempest timeout21:15
mriedemthanks21:15
efriedThanks :)21:15
*** sambetts|afk has quit IRC21:31
*** sambetts_ has joined #openstack-nova21:33
*** priteau has joined #openstack-nova21:33
efriedmriedem: reserved=total for which resource? And how do you stop the virt driver from overwriting that? (Re ML post about CERN workaround for low alloc candidates limit)21:38
mriedemVCPU? all of them?21:40
dansmithfor the compute node21:40
dansmithfor all of them yeah21:40
mriedemthe compute would probably need to know if it's service is disabled and if so, not ovewrite it21:40
mriedem*its21:41
dansmithmriedem: I think we discussed just having an rpc cast to compute to have it do it, vs. a periodic21:41
mriedemdansmith: sure, but the next update_available_resource periodic would overwrite it21:41
mriedemwhen reporting inventory21:41
dansmiththe compute can stop calling the virt driver's update method if it's disabled I would think21:41
sean-k-mooneyefried: the virt dirver really should not be touching the reserved value excpet when its first creating the RP21:41
mriedemsean-k-mooney: the ironic driver does all the time21:41
mriedemwhen the node is being cleaned21:41
dansmithsean-k-mooney: uh, why?21:41
dansmiththe reserved amount is owned by the virt driver, IMHO21:41
mriedemthat's exactly why the placement api change was made so that reserved can equal total21:42
dansmithonly the virt driver knows what it should be21:42
sean-k-mooneydansmith: well for the same reason as teh have the cpu allocation ratios vs inital cpu allocation ratios spec21:42
efriedagreed, all the inventory values ought to be owned by the virt driver, period. We start making exceptions, we end up with messes like allocation ratio... and reserved.21:42
sean-k-mooneycontroling via api or config21:42
efriedbut making that contract stick for this kind of workaround is going to be tricksy.21:42
dansmithsean-k-mooney: that's not an example that helps your case I think :)21:42
mriedemthe alternative was a trait i think21:43
efriedunless we do it with a handy-dandy provider config yaml file https://review.openstack.org/#/c/612497/21:43
mriedemand a pre-request placement filter in nova-scheduler21:43
mriedemor something like that21:43
mriedemthis isn't blues clues21:43
dansmithmriedem: yeah, that's also an option21:43
mriedemso api sets a trait (or removes it), virt doesn't overwrite it, and scheduler filters on it (essentially it becomes the ComputeFilter)21:44
efriedomg, now you totally remind me of Steve from blues clues.21:44
sean-k-mooneyefried: provider config yaml for what?21:44
mriedemefried: there were a couple of steves21:44
mriedemor was it one and then another guy?21:44
dansmithmriedem: not scheduler filters, request filter21:44
efriedno, there was only one steve.21:44
sean-k-mooneyhow does that allw you to set it via the api and not have the compute overriede21:44
efriedthe other guy was...21:44
efriedjoe?21:44
mriedemyes joe21:45
mriedemhttp://www.gstatic.com/tv/thumb/persons/250022/250022_v9_ba.jpg21:45
* mriedem swoons21:45
mriedemsean-k-mooney: b/c the compute/virt driver isn't supposed to overwrite externally set traits21:45
mriedemand the api can set traits just like it mirrors aggregates21:46
sean-k-mooneyanyway if we say only the entity that created the RP may set the reserved value on the invetories im fine with that if we document it21:46
dansmithI thought there was some other reason for doing the rpc cast on disable, but I can't remember what it was21:46
dansmithmaybe so ironic driver can do something?21:47
sean-k-mooneyon disableing a compute node. i think some dirver can take actions on disable and ironic might be one of them but i dont remember eiter21:47
efriedwe can ask each driver's upt to do21:48
efriedif self.disabled:21:48
efried  <do a right thing to disable the inventory>21:48
dansmithefried: we still have to do it as soon as the api is called though,21:48
dansmithso waiting for the next periodic is not good enough21:48
efriedmm21:48
dansmithoh yeah, so..21:49
efrieddo we have a ComputeDriver hook that we call on disable?21:49
mriedemright i could disable an entire cell,21:49
dansmithdisable is on the service not the node, right?21:49
efriedor could/should we implement one?21:49
mriedemer half a cell, and then live migrate stuff within the cell21:49
dansmithso if you disable the thing running on ironic, the nodes will still be schedule-able21:49
mriedemdansmith: yes on the service21:49
mriedemefried: we don't have one for os-services disable no21:49
dansmithso you have to do the service->nodes translation, and set the trait on all the nodes21:49
mriedemwe had one for os-hosts update status, but that was something else that only xen implemented21:50
dansmithbut if you rpc to the compute, it gets to do that itself21:50
mriedemdansmith: yup21:50
efriedum, or we could make upt that thing - i.e. call it immediately on disable21:50
*** takashin has joined #openstack-nova21:50
dansmithmriedem: that also gets wonky if you disable a service and then they rebalance, but you have set the trait on some nodes that rebalanced out21:50
sean-k-mooneymriedem: so the trait way woudl be add status_down trait and change placment query to include status_down forbien trait?21:50
*** priteau has quit IRC21:50
dansmithI guess they'll eventually un-disable, but.. ick21:50
dansmithwell, not actualy21:50
dansmithbecause we just said virt has to leave those alone if the api set them21:51
mriedemsean-k-mooney: essentially yes21:51
mriedemif ComputeFilter is enabled or something21:51
dansmithefried: right point being we don't have an rpc call for disable right now, but I think we probably should21:51
mriedemor some new boolean that replaces ComputeFilter21:51
efriedoh, because the disabling is being prompted not by the compute itself.21:51
* efried crawls back into hole21:52
mriedemcorrect21:52
dansmithit's an api call21:52
dansmithin most cases21:52
mriedemhttps://developer.openstack.org/api-ref/compute/#compute-services-os-services21:52
dansmithhowever,21:52
sean-k-mooneywell the trait could elminate the host before the filster so we dont need the computeFilter anymore21:52
mriedemif the hypervisor dies the libvirt driver auto-disables it's own compute service21:52
dansmithmriedem: computes can self-disable, so you'd also have to have them be able to set/clear the trait21:52
efriedthis would be a great use case for sean-k-mooney's reverse-required traits :P21:52
mriedemi smell spec21:52
dansmithif you just let the compute always manage this in response to an rpc call, the ownership of it is a lot more clear I think21:52
efried++21:52
mriedemsure i'm fine with rpc call to compute to do the thing21:53
sean-k-mooneyefried: you mean the " you cant be schduled to this RP without asking for trait X"21:53
mriedembtw this was the old thing from the os-hosts API https://github.com/openstack/nova/blob/master/nova/compute/api.py#L502621:53
efriedsean-k-mooney: yeah, that one.21:53
mriedemwe could maybe re-use that21:53
mriedemthat was only used by this before https://developer.openstack.org/api-ref/compute/#update-host-status21:54
mriedemwhich is deprecated21:54
dansmithI wonder if jaypipes is okay with a status_enabled or status_disabled trait on compute nodes21:55
sean-k-mooney mriedem was that repalce by the mark host down thing that we added for opnfv21:55
mriedemsean-k-mooney: on21:55
mriedem*no21:55
dansmithm-h-d is not just for opnfv21:56
mriedemsean-k-mooney: https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/deprecate-os-hosts.html21:56
sean-k-mooneydansmith: ya i know but they where the group i had the most conversation with21:56
sean-k-mooneymriedem the more you know. we have/had a lot of duplicate/legacy apis21:57
*** rcernin has joined #openstack-nova22:03
*** betherly has joined #openstack-nova22:06
*** rodolof has quit IRC22:11
*** rodolof has joined #openstack-nova22:11
*** betherly has quit IRC22:11
*** awaugama has quit IRC22:13
*** rodolof has quit IRC22:17
*** efried is now known as efried_out_til_j22:17
*** efried_out_til_j is now known as efried_cya_jan22:17
*** wolverineav has quit IRC22:24
*** wolverineav has joined #openstack-nova22:24
*** udesale has quit IRC22:25
*** mchlumsky has quit IRC22:30
cfriesenjust saw mriedem's mail on placement and disabled computes...rather than mucking around with inventory values why not just explicitly add a separate field that says whether a given RP is enabled or not?22:30
mriedemfield?22:33
mriedemomg have you talked to jay or chris ever?22:34
mriedemit's a trait at best22:34
mriedembut jay might not even like that22:34
edleafecfriesen: because that feels like having placement mirror nova's design. IMO it would be cleaner to set reserved=total22:34
mriedemsince it's not a capability22:34
edleafemriedem: although it would be fun watching Jay foam at the mouth22:35
*** dave-mccowan has quit IRC22:35
mriedemyes i was hoping dan pinging him would summon the beast22:35
sean-k-mooneywell the cappablity is its abiltiy to provide resouces.22:35
sean-k-mooneythere is proably a general usecase there but ya likely not a new field22:35
mriedemhttps://assets.rbl.ms/14621189/980x.jpg22:36
*** wolverineav has quit IRC22:36
sean-k-mooneyfor a second i tought it was a crock and wasl like aw its a lab22:37
cfriesenedleafe: I don't like modifying "reserved" when nova hasn't really reserved anything, and I'm concerned that the math makes no sense if we set "reserved" equal to total but have already-allocated resources as well22:37
cfriesensome sort of flag saying "don't consider this RP" makes more sense, I think22:38
sean-k-mooneycfriesen: if it was a non nova specific one like avaiable then maybe22:38
cfriesenoh, sure.  I don't care about the actual name. :)22:38
edleafecfriesen: I don't like it at all; I just hate it much, much less than adding a nova-specific field22:39
sean-k-mooneycfriesen: if you want jay and other to buy in to it you should :)22:39
edleafecfriesen: It should be a nova filter ideally22:39
mriedemthat's what the trait would do22:39
mriedemprovide filtering22:39
cfriesenedleafe: it is already22:39
cfriesenthe problem is that if placement only returns disabled compute nodes then nova filters them all out and ends up with novalidhost22:40
sean-k-mooneymriedem: yes and it would just be using required/forben traits which already exist so no placment change requried22:40
*** itlinux has quit IRC22:40
sean-k-mooneywe would be jsut establishing it as a patteren22:40
*** owalsh_ has joined #openstack-nova22:40
*** amodi_ has quit IRC22:41
sean-k-mooneythe issue is that we are usign placement to store metadata.. again about the resouce which is kindo fo what a trait is but not entirely22:41
mriedemif people don't think that lots of external services, like vcenter agents and such, are going to be dynamically adding/removing traits to coerce scheduling behavior in nova,...22:41
mriedemhow many people have already talked about adding basically encoding into traits for the lack of metadata22:42
*** owalsh has quit IRC22:42
edleafemriedem: "disabled" is a nova concept, not a placement one. If you don't want to use a scheduler filter on the nova side, then create a "disabled" agg, and add/remove compute nodes as needed, then use the upcoming negative member_of.22:42
sean-k-mooneymriedem: i was talking to an onap person at one point and when i told them what placement was they ask if the could use traits to tag the compute nodes with there uuids so they could say where to boot...22:42
sean-k-mooneyedleafe: there are paralles in other service22:43
mriedemedleafe: even using forbidden aggregates means nova needs to know to filter on that22:43
mriedemwhich is tpatil's spec22:43
mriedemhttps://review.openstack.org/#/c/609960/22:43
mriedemwhich is totally complicated for me btw22:44
mriedembut again, i might just be dumb22:44
sean-k-mooneyedleafe: you can mark a neutron dataplane as down i would be surprised if cinder does not have something similar22:44
edleafemriedem: Sure, just include that in every request, since you never want to schedule to a disabled host22:44
mriedemthat's what we'd do with a disabled trait as well22:44
mriedemhttps://developer.openstack.org/api-ref/block-storage/v3/?expanded=freeze-a-cinder-backend-host-detail#freeze-a-cinder-backend-host22:45
mriedemsean-k-mooney: you mean like that ^22:45
sean-k-mooney yep22:45
sean-k-mooneyi think it woudl be common usecase that other servcies woudl use if we figured out a generic way of doing it in nova22:46
sean-k-mooney* in placement22:46
*** wolverineav has joined #openstack-nova22:47
cfriesenhow about a "resource unavailable" trait?22:47
sean-k-mooneycfriesen: that is the trait we were suggesting by another name22:47
cfriesenyes22:48
cfriesenjust trying to come up with a suitably generic name22:49
sean-k-mooneyedleafe: any comment on using a trait?22:50
*** wolverineav has quit IRC22:51
edleafesean-k-mooney: You mean besides the design that a trait is not for status?22:51
sean-k-mooneyedleafe: yep beside the general object ot using placement as a metdata store / etcd standin22:52
*** rcernin has quit IRC22:52
edleafesean-k-mooney: :)22:52
*** rcernin has joined #openstack-nova22:52
*** wolverineav has joined #openstack-nova22:53
mriedemhere are some sufficiently out of band proxy via nova via placement things you guys will probably all like then https://review.openstack.org/#/c/552190/ https://review.openstack.org/#/c/549067/22:54
sean-k-mooneyvmware are aware that openstack in not vspher/vcloud right ...22:57
sean-k-mooneythe vmware-vm-groups sound kindof like the tenant aggreate affintiy usecase22:58
sean-k-mooneyoh actully22:59
sean-k-mooneyok the usecase is compulte different to what i was expecte form the intro and problem descripteion22:59
*** wolverineav has quit IRC23:05
*** munimeha1 has quit IRC23:07
*** wolverineav has joined #openstack-nova23:12
*** mlavalle has quit IRC23:16
*** spatel has quit IRC23:22
*** owalsh_ is now known as owalsh23:25
*** wolverineav has quit IRC23:27
sean-k-mooneydoes blazer not use its specs repos23:27
*** ttsiouts has joined #openstack-nova23:30
sean-k-mooneyits in tree never mind23:30
*** mriedem has quit IRC23:31
*** ttsiouts has quit IRC23:32
tssuryamriedem: we have filed the bug already regarding placement not knowing about disabled computes: https://bugs.launchpad.net/nova/+bug/180598423:40
openstackLaunchpad bug 1805984 in OpenStack Compute (nova) "Placement is not aware of disable compute nodes" [Undecided,New]23:40
sean-k-mooneytssurya: its not really a bug its a new feature that shoudl live in nova23:41
sean-k-mooneywell not quite23:41
sean-k-mooneyits not currently expected that placement would know23:41
sean-k-mooneyso its no a bug in that sence23:41
*** _alastor_ has quit IRC23:44
*** wolverineav has joined #openstack-nova23:45
tssuryasean-k-mooney: right agreed its not a bug since it was decided as a won't fix on the placement side (https://bugs.launchpad.net/nova/+bug/1708958)23:49
openstackLaunchpad bug 1708958 in OpenStack Compute (nova) "disabling a compute service does not disable the resource provider" [Wishlist,Won't fix]23:49
tssuryabut mriedem wanted a bug (was replying to his ML post)23:50
sean-k-mooneytssurya: i set it to triage and wish list with the suggestion of converting to a spec or specless blueprint23:50
tssuryato just track the issue23:50
sean-k-mooneytssurya: if it was neutron i would have just set the RFE tag on the bug but nova does not use that23:50
sean-k-mooneyits definetly an edge case an i can see it causeing issues for larger deployments23:51
sean-k-mooneydid you see the conversation we had on this an hour ago23:52
tssuryasean-k-mooney: was just scrolling back up to read it :D23:56
tssuryaI can see a couple of plausible solutions being proposed23:56
sean-k-mooneyyep there are 2 or 3 options23:56
tssuryaprobably the one where reserved=total + disabling periodic updates on disabled computes to avoid overwrite is the better option23:57
sean-k-mooneyit looks like we should be able to come up with somethign23:57
tssuryaconsidering adding another periodic task just for this might be a overhead23:57
sean-k-mooneytssurya: im not sure about that23:57
sean-k-mooneyi guess we will see the traits approch or using a disabled aggreate seam like a clean solution23:58
tssuryaoh disabled aggregates! didn't see that part yet, reading now23:58
sean-k-mooneytssurya: well a disable aggreage is just an aggrage that we "nova" adds host that are down too and we jsut use the not member_of proposal to avoid23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!