| *** elod has joined #openstack-nova | 00:02 | |
| *** markvoelker has joined #openstack-nova | 00:03 | |
| *** dpawlik has joined #openstack-nova | 00:06 | |
| *** markvoelker has quit IRC | 00:07 | |
| *** dpawlik has quit IRC | 00:11 | |
| *** dave-mccowan has quit IRC | 00:11 | |
| *** sapd1_x has joined #openstack-nova | 00:11 | |
| *** tetsuro has joined #openstack-nova | 00:14 | |
| openstackgerrit | Merged openstack/nova master: qemu: Make disk image conversion dramatically faster https://review.openstack.org/640781 | 00:14 |
|---|---|---|
| *** gyee has quit IRC | 00:15 | |
| *** luksky has quit IRC | 00:15 | |
| *** sapd1_x has quit IRC | 00:21 | |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Remove unnecessary default provider_tree when getting traits https://review.openstack.org/644452 | 00:30 |
| *** markvoelker has joined #openstack-nova | 00:35 | |
| *** markvoelker has quit IRC | 00:49 | |
| *** slaweq has quit IRC | 00:58 | |
| *** slaweq has joined #openstack-nova | 01:10 | |
| *** hongbin has joined #openstack-nova | 01:12 | |
| *** dustinc has quit IRC | 01:22 | |
| *** slaweq has quit IRC | 01:24 | |
| *** wolverineav has joined #openstack-nova | 01:24 | |
| *** whoami-rajat has joined #openstack-nova | 01:25 | |
| *** wolverineav has quit IRC | 01:30 | |
| *** eharney_ has quit IRC | 01:40 | |
| *** dave-mccowan has joined #openstack-nova | 01:52 | |
| openstackgerrit | Rui Zang proposed openstack/nova-specs master: support virtual persistent memory https://review.openstack.org/601596 | 01:54 |
| *** lbragstad has quit IRC | 01:56 | |
| *** nicolasbock has quit IRC | 02:01 | |
| *** dpawlik has joined #openstack-nova | 02:07 | |
| *** lbragstad has joined #openstack-nova | 02:09 | |
| *** dpawlik has quit IRC | 02:12 | |
| *** dave-mccowan has quit IRC | 02:23 | |
| *** wolverineav has joined #openstack-nova | 02:36 | |
| *** itlinux has joined #openstack-nova | 03:05 | |
| *** zzzeek has quit IRC | 03:10 | |
| *** zzzeek has joined #openstack-nova | 03:16 | |
| *** cfriesen has quit IRC | 03:31 | |
| *** igordc has quit IRC | 03:32 | |
| *** janki has joined #openstack-nova | 03:44 | |
| *** wolverineav has quit IRC | 03:45 | |
| *** sapd1_x has joined #openstack-nova | 03:52 | |
| *** sapd1_x has quit IRC | 04:04 | |
| *** dpawlik has joined #openstack-nova | 04:08 | |
| *** hongbin has quit IRC | 04:12 | |
| *** dpawlik has quit IRC | 04:13 | |
| *** udesale has joined #openstack-nova | 04:14 | |
| *** diablo_rojo has quit IRC | 04:15 | |
| *** wolverineav has joined #openstack-nova | 04:18 | |
| *** cfriesen has joined #openstack-nova | 04:20 | |
| *** sridharg has joined #openstack-nova | 04:25 | |
| *** tetsuro has quit IRC | 04:33 | |
| openstackgerrit | chenker proposed openstack/python-novaclient master: Optimize limit option docs string description for novaclient https://review.openstack.org/643035 | 04:41 |
| *** dpawlik has joined #openstack-nova | 04:44 | |
| *** dklyle has quit IRC | 04:44 | |
| *** dklyle has joined #openstack-nova | 04:45 | |
| openstackgerrit | chenker proposed openstack/python-novaclient master: Optimize limit option docs string description for novaclient https://review.openstack.org/643035 | 04:46 |
| *** dpawlik has quit IRC | 04:48 | |
| *** wolverineav has quit IRC | 04:49 | |
| openstackgerrit | Rui Zang proposed openstack/nova-specs master: Virtual persistent memory libvirt driver implementation https://review.openstack.org/622893 | 05:11 |
| *** udesale has quit IRC | 05:31 | |
| *** wolverineav has joined #openstack-nova | 05:42 | |
| *** sajauddin has joined #openstack-nova | 05:45 | |
| *** ratailor has joined #openstack-nova | 05:46 | |
| *** wolverineav has quit IRC | 05:51 | |
| *** ravirjn has quit IRC | 05:57 | |
| *** liuyulong has quit IRC | 06:01 | |
| *** dpawlik has joined #openstack-nova | 06:15 | |
| *** brinzhang has joined #openstack-nova | 06:17 | |
| *** ivve has joined #openstack-nova | 06:19 | |
| *** markvoelker has joined #openstack-nova | 06:20 | |
| *** markvoelker has quit IRC | 06:25 | |
| *** minmin has joined #openstack-nova | 06:29 | |
| Kevin_Zheng | ping gmann | 06:35 |
| *** ccamacho has quit IRC | 06:37 | |
| *** lbragstad has quit IRC | 06:38 | |
| *** markvoelker has joined #openstack-nova | 06:51 | |
| *** cfriesen has quit IRC | 06:51 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add a prelude release note for the 19.0.0 Stein GA https://review.openstack.org/644412 | 06:53 |
| *** jaosorior has quit IRC | 06:55 | |
| *** jaosorior has joined #openstack-nova | 06:57 | |
| *** itlinux has quit IRC | 06:57 | |
| *** itlinux has joined #openstack-nova | 06:58 | |
| *** Luzi has joined #openstack-nova | 06:59 | |
| *** pcaruana has joined #openstack-nova | 07:14 | |
| *** zkzmmrc has joined #openstack-nova | 07:18 | |
| *** dpawlik has quit IRC | 07:19 | |
| *** itlinux has quit IRC | 07:21 | |
| *** wolverineav has joined #openstack-nova | 07:22 | |
| *** tosky has joined #openstack-nova | 07:25 | |
| *** dpawlik has joined #openstack-nova | 07:29 | |
| *** pcaruana has quit IRC | 07:33 | |
| *** pcaruana has joined #openstack-nova | 07:34 | |
| *** slaweq has joined #openstack-nova | 07:38 | |
| *** maciejjozefczyk has joined #openstack-nova | 07:46 | |
| *** owalsh has quit IRC | 07:48 | |
| *** owalsh has joined #openstack-nova | 07:54 | |
| openstackgerrit | Merged openstack/nova master: Remove stale aggregates notes from scheduler evolution doc https://review.openstack.org/643613 | 07:56 |
| *** rpittau|afk is now known as rpittau | 07:56 | |
| *** wolverineav has quit IRC | 07:57 | |
| *** tetsuro has joined #openstack-nova | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-nova | 08:02 | |
| *** jangutter has quit IRC | 08:04 | |
| *** minmin has quit IRC | 08:06 | |
| *** luksky has joined #openstack-nova | 08:07 | |
| *** ttsiouts has joined #openstack-nova | 08:08 | |
| *** udesale has joined #openstack-nova | 08:09 | |
| *** awalende has joined #openstack-nova | 08:11 | |
| *** jangutter has joined #openstack-nova | 08:12 | |
| *** ccamacho has joined #openstack-nova | 08:14 | |
| *** luksky has quit IRC | 08:16 | |
| *** tesseract has joined #openstack-nova | 08:16 | |
| *** dtantsur|afk is now known as dtantsur | 08:18 | |
| *** tssurya has joined #openstack-nova | 08:19 | |
| *** alex_xu has quit IRC | 08:21 | |
| *** phasespace has joined #openstack-nova | 08:22 | |
| *** alex_xu has joined #openstack-nova | 08:23 | |
| *** brault has joined #openstack-nova | 08:24 | |
| *** ttsiouts has quit IRC | 08:25 | |
| *** mdbooth_ has joined #openstack-nova | 08:26 | |
| kashyap | efried: Thank you for putting it through. | 08:26 |
| *** mdbooth has quit IRC | 08:30 | |
| *** helenafm has joined #openstack-nova | 08:32 | |
| *** priteau has joined #openstack-nova | 08:38 | |
| openstackgerrit | Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765 | 08:39 |
| *** wolverineav has joined #openstack-nova | 08:48 | |
| *** ttsiouts has joined #openstack-nova | 08:49 | |
| openstackgerrit | Seyeong Kim proposed openstack/nova stable/rocky: Extract compute API _create_image to compute.utils https://review.openstack.org/643852 | 08:56 |
| openstackgerrit | Seyeong Kim proposed openstack/nova stable/rocky: Share snapshot image membership with instance owner https://review.openstack.org/643853 | 08:56 |
| *** Luzi has quit IRC | 08:59 | |
| *** zhongjun2 has joined #openstack-nova | 09:03 | |
| *** panda|pto is now known as panda | 09:04 | |
| *** dklyle has quit IRC | 09:08 | |
| *** dklyle has joined #openstack-nova | 09:09 | |
| *** kukacz has quit IRC | 09:11 | |
| *** xek has joined #openstack-nova | 09:15 | |
| *** alex_xu has quit IRC | 09:18 | |
| *** tkajinam has quit IRC | 09:18 | |
| *** luksky has joined #openstack-nova | 09:19 | |
| *** wolverineav has quit IRC | 09:19 | |
| *** wolverineav has joined #openstack-nova | 09:27 | |
| *** kukacz has joined #openstack-nova | 09:28 | |
| *** wolverineav has quit IRC | 09:29 | |
| *** kukacz has quit IRC | 09:29 | |
| *** kukacz has joined #openstack-nova | 09:30 | |
| *** ttsiouts has quit IRC | 09:31 | |
| *** ttsiouts has joined #openstack-nova | 09:32 | |
| openstackgerrit | zhangyangyang proposed openstack/nova master: Drop py35 jobs and add py36 jobs https://review.openstack.org/643871 | 09:32 |
| *** wolverineav has joined #openstack-nova | 09:35 | |
| *** kukacz has quit IRC | 09:39 | |
| *** kukacz has joined #openstack-nova | 09:39 | |
| *** derekh has joined #openstack-nova | 09:40 | |
| *** sajauddin has quit IRC | 09:54 | |
| *** IvensZambrano has joined #openstack-nova | 09:55 | |
| *** tbachman has joined #openstack-nova | 09:57 | |
| *** dustinc has joined #openstack-nova | 10:03 | |
| gibi | melwitt: I don't think https://bugs.launchpad.net/nova/+bug/1820588 is rc potential, I left some reasoning in the bug | 10:05 |
| openstack | Launchpad bug 1820588 in OpenStack Compute (nova) "Bandwidth resource is leaked if a bound port is deleted in neutron" [Low,Confirmed] | 10:05 |
| sean-k-mooney | gibi: that will be fixed by handeling the networks-status event in the periodic jobs right? | 10:06 |
| sean-k-mooney | gibi: have you been working on a fix for that already or is it still todo? | 10:06 |
| sean-k-mooney | i vaguely rememebr talking about it a week or two ago that we would have to do something to handel this edgecase | 10:07 |
| kashyap | aspiers: No problem; I myself will be off next week | 10:07 |
| sean-k-mooney | gibi: even if its not rc potential which i kind of agree with i think this should be backportable shortly after GA | 10:09 |
| *** wolverineav has quit IRC | 10:09 | |
| gibi | sean-k-mooney: when the bound port is deleted in neutron nova gets the notification and detaches the port from the server. I don't know how the periodic is involved here | 10:11 |
| gibi | sean-k-mooney: moreover I till not conviced that why the admin cannot detach the port in nova first and only delete it after in neutron | 10:11 |
| gibi | detach + delete works properly with bandwidth | 10:12 |
| *** mikal has quit IRC | 10:15 | |
| sean-k-mooney | gibi: when a bound port is delete neutron send a notification to nova which the periodic jobs will recive | 10:19 |
| sean-k-mooney | gibi: this also does not require admin acess | 10:20 |
| sean-k-mooney | a tenant can simple do a openstack port delete to cause this issue | 10:20 |
| aspiers | kashyap: you wanted me to split out the addition of getDomainCapabilities() from the <sev> parsing, right? | 10:21 |
| kashyap | aspiers: Yes, please | 10:21 |
| aspiers | kashyap: the problem is, that code doesn't parse anything else in the output | 10:22 |
| sean-k-mooney | gibi: but you are right that this should be done by a nova interface detach in general | 10:22 |
| aspiers | kashyap: so the first patch would introduce nonsensical code | 10:22 |
| sean-k-mooney | we should however be able to process the network-status event we get form neutron and heal the placement allocation regardless of why it was done so its not something i would worry too much about | 10:23 |
| aspiers | kashyap: the only thing it parses is <sev> under <features> | 10:23 |
| aspiers | kashyap: so splitting it would result in 3 new LibvirtConfigObject classes which do absolutely nothing | 10:23 |
| kashyap | Hmm | 10:24 |
| aspiers | I could make it parse something else just for the sake of it | 10:24 |
| kashyap | Why would it be "nonsensical code" to split out a logically separate part? | 10:24 |
| aspiers | Not sure whether that makes sense though | 10:24 |
| aspiers | Because that separate part is non-functional noop code | 10:25 |
| aspiers | There'd be nothing to test | 10:25 |
| gibi | sean-k-mooney: I disagree. Why end user wants to delete a bound port instead of deatching it? | 10:26 |
| gibi | sean-k-mooney: what is the use case for this? | 10:26 |
| sean-k-mooney | gibi: its something that we have supported for many releases. there is not a defined usecase but neutron has never prevented it and until now nova has alreays correctly detached the port | 10:27 |
| sean-k-mooney | the only other fix we could do is modify neutron to make deleteing a bound port illegal | 10:27 |
| *** cdent has joined #openstack-nova | 10:27 | |
| sean-k-mooney | we could do that but that is a seperate issue | 10:28 |
| aspiers | kashyap: unless you can think of something to test? | 10:28 |
| aspiers | but if it's not parsing anything I can't really see how | 10:28 |
| gibi | sean-k-mooney: to support bandwidth cleanup at bound port delete is not a trivial thing. It adds complexity for someting that has no clear use case and I don't like that | 10:28 |
| kashyap | aspiers: /me needs to fully re-load this context; but if you think it is not helping in anyway (even for review), then leave it as-is | 10:29 |
| kashyap | aspiers: And note this reasoning somewhere in the commit message or the change | 10:29 |
| sean-k-mooney | gibi: that is true but the only way to prevent needeing to handle this is to modify the neutron api to prevent deletion of bound ports which will require a spec and my have unintended sideffects | 10:29 |
| sean-k-mooney | anyway we could fix it in the periodic job but that does not mean we have too | 10:30 |
| aspiers | kashyap: I've split out the move of machine type calculation to utils.py | 10:30 |
| aspiers | kashyap: which already helps a bit | 10:30 |
| gibi | sean-k-mooney: I'm not sure we can easly fix it in the periodic job. Also I think it would make sense to discuss why neutron allows deleteing a bound port | 10:30 |
| kashyap | aspiers: Yeah, that helps | 10:31 |
| aspiers | kashyap: take a look at https://review.openstack.org/#/c/633855/11/nova/virt/libvirt/config.py | 10:31 |
| aspiers | kashyap: if I remove parsing of <sev>, then LibvirtConfigDomainCapsFeatures becomes a noop (no features ever get parsed and added to self._features) | 10:32 |
| aspiers | and in that case LibvirtConfigDomainCaps is also a noop | 10:32 |
| aspiers | parse_dom() would do absolutely nothing | 10:32 |
| sean-k-mooney | gibi: yep we porablly should bring it up in the cross project session | 10:32 |
| gibi | sean-k-mooney: OK, I'm adding it to the etherpad | 10:33 |
| sean-k-mooney | gibi: the asymitry has always annoyed me that we can detach port via neutron but not attach them | 10:33 |
| aspiers | kashyap: like I said, I could make it parse something simple like <path> and/or <arch> just so that there is something to test | 10:33 |
| kashyap | aspiers: Dealing with 4 pings, please bear with me. | 10:33 |
| sean-k-mooney | gibi: but yes to your original point this proably does not need to be fix in the RC and can be "fixed" what ever way we decided in train and backported if needed | 10:33 |
| aspiers | kashyap: that would give just enough excuse to split it out, but whatever I parsed would not be consumed by anything, at least until you start using it | 10:34 |
| aspiers | kashyap: haha sure :) | 10:34 |
| gibi | sean-k-mooney: yeah. I don't want to rush this fix as honetly I don't see how can this be easily fixed | 10:34 |
| sean-k-mooney | gibi: if we block it in the neutron api that wont be backportable but the osc-placement plugin can always be used to fix it by hand if needed | 10:35 |
| cdent | "how can this be easily fixed" the story of nova's life | 10:36 |
| sean-k-mooney | cdent: well you can always just delete the vm and that will make everything better :) | 10:36 |
| gibi | cdent: especially that we "need to fix" even if we don't know what is the use case | 10:37 |
| cdent | that's the other half of the story: "we're using this cloud as a home for our pets" | 10:37 |
| sean-k-mooney | cdent: of cource clouds and nice and fluffy places for our pets to live with totally no sharp edges | 10:38 |
| cdent | made by magical open source bunnies! | 10:38 |
| gibi | sean-k-mooney: no edges?! :) we want EDGE in the cloud :D | 10:38 |
| sean-k-mooney | gibi: hehe ture but hopefully said edges wont be sharp :) | 10:39 |
| gibi | we need fluffy edges! | 10:39 |
| *** dpawlik has quit IRC | 10:40 | |
| sean-k-mooney | new goal for summit. get "fluffy edges" into someones marketing deck | 10:40 |
| cdent | fluffy edges sounds like a pretty good term for dynamically scalable edge nodes | 10:44 |
| *** dpawlik has joined #openstack-nova | 10:44 | |
| *** janki has quit IRC | 10:45 | |
| *** derekh has quit IRC | 10:45 | |
| *** nicolasbock has joined #openstack-nova | 10:45 | |
| *** derekh has joined #openstack-nova | 10:48 | |
| kashyap | aspiers: "Not consumed by anything" is okay so long as it (a) helps with clarity; (b) makes way for future patches to cherry-pick it for a particular use case | 10:48 |
| *** dpawlik has quit IRC | 10:49 | |
| *** dpawlik has joined #openstack-nova | 10:53 | |
| *** udesale has quit IRC | 10:53 | |
| aspiers | kashyap: yeah but needs to be testable still! | 10:55 |
| kashyap | aspiers: Agreed :-) Don't let this block you. | 10:56 |
| *** nicolasbock has quit IRC | 10:59 | |
| *** mrch_ has joined #openstack-nova | 10:59 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Eventlet monkey patching should be as early as possible https://review.openstack.org/626952 | 11:03 |
| *** psachin has joined #openstack-nova | 11:07 | |
| *** priteau has quit IRC | 11:08 | |
| *** ttsiouts has quit IRC | 11:09 | |
| kashyap | mdbooth_: When you get a moment, since we always return a non-None 'disk_cachemode' here: https://review.openstack.org/#/c/641981/10/nova/virt/libvirt/driver.py@443 | 11:09 |
| kashyap | mdbooth_: Are you implying to remove the check there? | 11:10 |
| *** mdbooth_ is now known as mdbooth | 11:10 | |
| mdbooth | kashyap: No, that comment was wrt the release note | 11:10 |
| mdbooth | Because we never return non-None there, we never use the qemu default | 11:10 |
| mdbooth | So to the qemu default is irrelevant | 11:10 |
| kashyap | Ah, guessed as much. | 11:11 |
| kashyap | You wrote it in reference to the help text | 11:11 |
| kashyap | Okay, elided the irrelevant bit. | 11:11 |
| kashyap | Thanks! | 11:11 |
| mdbooth | s/release note/docs/ | 11:11 |
| openstackgerrit | Gorka Eguileor proposed openstack/nova master: Fix unnecessary locking when connecting volumes https://review.openstack.org/614190 | 11:16 |
| *** tetsuro has quit IRC | 11:21 | |
| *** mrch_ has quit IRC | 11:21 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Use 'writeback' QEMU cache mode when 'none' is not viable https://review.openstack.org/641981 | 11:22 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: vzstorage: Use 'writeback' QEMU cache mode https://review.openstack.org/643376 | 11:22 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: smbfs: Use 'writeback' QEMU cache mode https://review.openstack.org/643377 | 11:22 |
| sean-k-mooney | kashyap: you are not enableing writeback by defualt when none is not available are you? | 11:23 |
| *** mrch_ has joined #openstack-nova | 11:23 | |
| kashyap | sean-k-mooney: That's exactly what it's doing; please read the commit message. I spent 3 hours writing it. | 11:24 |
| kashyap | https://review.openstack.org/#/c/641981/ | 11:24 |
| *** nicolasbock has joined #openstack-nova | 11:24 | |
| *** rcernin has quit IRC | 11:24 | |
| sean-k-mooney | kashyap: if none is not an option we shoudl default to the safet alternitive either directsync or writethrough and only use writeback if there is no other option | 11:24 |
| kashyap | FWIW, : I ran the patch by QEMU Block Layer maintainer over an e-mail thread. | 11:24 |
| kashyap | sean-k-mooney: No, that is wrong. See the commit message and the comment in the code, again. | 11:24 |
| sean-k-mooney | i just saw the ping. i had not opened it yet | 11:24 |
| sean-k-mooney | kashyap: i had under stood that writback did not honer guest fush commands. is that not the case? | 11:28 |
| sean-k-mooney | if it does the ok | 11:28 |
| sean-k-mooney | *then | 11:28 |
| kashyap | sean-k-mooney: It does honour it | 11:32 |
| kashyap | sean-k-mooney: Also see: http://git.openstack.org/cgit/openstack/nova/commit/?h=master&id=e7b64eaad8 | 11:33 |
| sean-k-mooney | ok ya looking at the docs it does | 11:33 |
| kashyap | Nod | 11:33 |
| * kashyap --> AFK; bbiab | 11:33 | |
| sean-k-mooney | i had tought there was still an edgecase with virtio scsi not informing the guest os that it was in writeback mode and that this only worked with virtio-block but maybe not | 11:34 |
| sean-k-mooney | i could be tinking of discard instead of flush | 11:34 |
| sean-k-mooney | kashyap: anywya i trust your and the qemu maintiners knoladge of this better then my own. its been a while since i dug into the cahce modes in any detail | 11:36 |
| *** pawel-baclawski has joined #openstack-nova | 11:36 | |
| *** wolverineav has joined #openstack-nova | 11:37 | |
| *** adrianc has quit IRC | 11:41 | |
| sean-k-mooney | kashyap: ah i found the ceph note i was missremembering. when using caching with RBD you expcitly have to use writeback cache mode not that you sould not use cach=writeback. http://docs.ceph.com/docs/master/rbd/qemu-rbd/#running-qemu-with-rbd the last time i looked into this was determing the correct way to use qemu safely with ceph storage with caching which at the time was virio-scsi for | 11:44 |
| sean-k-mooney | discard/trim support + writeback cache if i wanted safe caching | 11:44 |
| *** janki has joined #openstack-nova | 11:49 | |
| *** ratailor has quit IRC | 11:49 | |
| *** klindgren_ has quit IRC | 11:51 | |
| *** psachin has quit IRC | 11:54 | |
| *** dave-mccowan has joined #openstack-nova | 11:58 | |
| *** brinzhang has quit IRC | 11:58 | |
| *** klindgren has joined #openstack-nova | 12:03 | |
| *** finucannot is now known as stephenfin | 12:05 | |
| *** luksky has quit IRC | 12:07 | |
| *** panda is now known as panda|lunch | 12:07 | |
| *** ccamacho has quit IRC | 12:08 | |
| *** adrianc has joined #openstack-nova | 12:10 | |
| *** ttsiouts has joined #openstack-nova | 12:10 | |
| *** wolverineav has quit IRC | 12:12 | |
| *** dustinc has quit IRC | 12:20 | |
| *** luksky has joined #openstack-nova | 12:20 | |
| *** adrianc has quit IRC | 12:24 | |
| *** markvoelker has quit IRC | 12:24 | |
| *** luksky has quit IRC | 12:29 | |
| kashyap | sean-k-mooney: Just back from lunch | 12:31 |
| kashyap | sean-k-mooney: Yep, I also suggest you think about the cache modes in terms of the three booleans each mode toggles. | 12:32 |
| kashyap | It allows better reasoning of the concept | 12:32 |
| kashyap | sean-k-mooney: (Yeah, that Ceph thing is accounted for by our own NOTE in the Nova config help docs.) | 12:32 |
| *** mchlumsky has joined #openstack-nova | 12:33 | |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Move libvirt calculation of machine type to utils.py https://review.openstack.org/644554 | 12:35 |
| *** adrianc has joined #openstack-nova | 12:36 | |
| sean-k-mooney | kashyap: well i generally use boot from volume or rbd image driver so i generally have not needed to resonable about cachemodes for qemu/raw image backend | 12:39 |
| kashyap | Noted; was only saying as a future note :-) | 12:40 |
| sean-k-mooney | the only time i tend to have local images is devstack :) but yes i look this up about once a year when i need to but its one of those things that i generaly dont rememebr off the top of my head unfortunetly | 12:41 |
| *** luksky has joined #openstack-nova | 12:42 | |
| *** alex_xu has joined #openstack-nova | 12:44 | |
| kashyap | Nod; /me does a yearly "refresher course" on QEMU's caching behaviour...because damned if I could ever remember the details | 12:45 |
| mdbooth | There seems to be a problem with the openstack-tox-docs job | 12:48 |
| mdbooth | It's failed for me twice in a row now. | 12:48 |
| mdbooth | http://logs.openstack.org/52/626952/7/check/openstack-tox-docs/d4bea1d/job-output.txt.gz#_2019-03-19_11_13_38_088828 | 12:49 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Rework 'EBUSY' (SIGKILL) error handling code path https://review.openstack.org/639091 | 12:49 |
| *** adrianc has quit IRC | 12:51 | |
| aspiers | kashyap: trying to split getDomCaps out, but another issue is that the design decision to use a nested dict of dicts was heavily influenced by SEV https://review.openstack.org/#/c/633855/11/nova/virt/libvirt/host.py@706 | 12:52 |
| * mdbooth wants a 'kill this run and recheck' option in zuul | 12:52 | |
| aspiers | mdbooth: that would be nice :) | 12:52 |
| mdbooth | Or even better, a 'only recheck this single job' | 12:52 |
| kashyap | mdbooth: Sounds like a nice request for the fine folks on #openstack-infra | 12:52 |
| frickler | kashyap: seems this is your real error, probably related to your patch http://logs.openstack.org/52/626952/7/check/openstack-tox-docs/d4bea1d/job-output.txt.gz#_2019-03-19_11_13_31_488145 | 12:53 |
| kashyap | frickler: You mean mdbooth's ^ | 12:53 |
| aspiers | kashyap: so either the preliminary getDomCaps patch prematurely references SEV before introducing support for it, or it makes a design decision without explaining it | 12:53 |
| frickler | kashyap: yes, mangled up scrollback, sorry | 12:53 |
| kashyap | aspiers: Reading ... | 12:53 |
| mdbooth | frickler: Ah, thanks! | 12:54 |
| mdbooth | That was a super-unhelpful error message! | 12:54 |
| *** mriedem has joined #openstack-nova | 12:54 | |
| mdbooth | frickler: Out of curiosity, what grep-fu did you use to find that? | 12:55 |
| mdbooth | NM, I suppose it's just above the other error. | 12:56 |
| frickler | mdbooth: none, I just know that failure to copy "../doc/build" mostly results from the docs build having failed earlier. so I scrolled back a bit more to find the other error | 12:56 |
| *** mchlumsky has quit IRC | 12:56 | |
| frickler | mdbooth: another way to approach this is to look at the ara report, where you can easily spot that not only the docs-post job failed, but also the main run, which most of the time has the more interesting errors http://logs.openstack.org/52/626952/7/check/openstack-tox-docs/d4bea1d/ara-report/ | 12:57 |
| *** mchlumsky has joined #openstack-nova | 12:58 | |
| mriedem | dansmith: i'm assuming you want to just mark this as won't fix or something https://bugs.launchpad.net/nova/+bug/1820851 | 12:58 |
| openstack | Launchpad bug 1820851 in OpenStack Compute (nova) "Please do cell mapping without periodic task or active admin involvment" [Wishlist,Opinion] | 12:58 |
| *** adrianc has joined #openstack-nova | 12:59 | |
| mdbooth | frickler: That's awesome. Thanks. | 13:00 |
| * mdbooth needs a more surgical whack-a-mole hammer. | 13:00 | |
| *** liuyulong has joined #openstack-nova | 13:00 | |
| *** wolverineav has joined #openstack-nova | 13:01 | |
| *** panda|lunch is now known as panda | 13:01 | |
| *** udesale has joined #openstack-nova | 13:02 | |
| *** eharney has joined #openstack-nova | 13:03 | |
| kashyap | aspiers: Hmm. Still thinking on the nested dictionary of arch–machine type mapping ... | 13:04 |
| aspiers | ok | 13:04 |
| kashyap | aspiers: Can you put what you just wrote in the review, please? | 13:04 |
| kashyap | As a "consciousness strem" | 13:04 |
| kashyap | s/strem/stream/ | 13:04 |
| *** wolverineav has quit IRC | 13:12 | |
| openstackgerrit | Boris Bobrov proposed openstack/nova master: Extract SEV-specific bits on host detection https://review.openstack.org/636334 | 13:14 |
| openstackgerrit | Boris Bobrov proposed openstack/nova master: Fix memtune parameters according to libvirt docs https://review.openstack.org/636301 | 13:14 |
| openstackgerrit | Boris Bobrov proposed openstack/nova master: Add configs for AMD SEV https://review.openstack.org/636318 | 13:14 |
| openstackgerrit | Boris Bobrov proposed openstack/nova master: Add iommu driver to devices https://review.openstack.org/644564 | 13:14 |
| openstackgerrit | Boris Bobrov proposed openstack/nova master: [wip] Driver changes for sev https://review.openstack.org/644565 | 13:14 |
| *** mikal has joined #openstack-nova | 13:16 | |
| sean-k-mooney | kashyap: nested dict of arch machine type mappings? in the nova.conf or image. i dont think i makes sense in the image and we basically aready have that in the nova.conf | 13:16 |
| kashyap | sean-k-mooney: For more context, check out the "returns" here: https://review.openstack.org/#/c/633855/11/nova/virt/libvirt/host.py@706 | 13:17 |
| kashyap | sean-k-mooney: The context is to separate out the getDomainCapabilities() method from that file | 13:17 |
| mdbooth | Hmm. Unless I can differentiate between workloads which are simply loading Nova classes to gather metadata and workloads which actually plan to execute Nova code, I'm going to have to remove the monkey patching asserts :/ | 13:18 |
| mdbooth | Although the former really shouldn't be monkey patching at all | 13:18 |
| mdbooth | But they always have been, so I guess it doesn't matter in practise | 13:18 |
| *** tssurya has quit IRC | 13:18 | |
| sean-k-mooney | kashyap will a machine type ever be suppoted by two archs? i dont think so right. so it could jsut be a dict of machine_type to capablities | 13:19 |
| *** jmlowe has quit IRC | 13:19 | |
| sean-k-mooney | if your going with the nested structure however i would strongly think of make really OVO however instead of dicts of dicts of capablity dicts | 13:20 |
| *** mikal has quit IRC | 13:20 | |
| kashyap | sean-k-mooney: So far we have AArch64 that has 'virt' machine type -- now there is upstream QEMU talk to introduce something similar for x86_64 | 13:22 |
| kashyap | sean-k-mooney: Please note your point on the change pointed earlier. | 13:22 |
| sean-k-mooney | kashyap: ah and the capablitys fo the virt machine type would be different for aarch64 vs x86 | 13:22 |
| kashyap | Exactly | 13:23 |
| mriedem | yay this is old https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L1549-L1552 | 13:25 |
| sean-k-mooney | kashyap: when i +1'd it previously it retruned a config.LibvirtConfigDomainCaps object by the way | 13:26 |
| sean-k-mooney | kashyap: so i have not reviewed it since this changed to a dict of dicts of objectss | 13:26 |
| *** pawel-baclawski has quit IRC | 13:27 | |
| sean-k-mooney | im not sure how the change to detect sev suport which will only work on amd x86 some how morphed to include generic multi arch multi machiche type capablity discovery code | 13:27 |
| sean-k-mooney | that proably should be in a different patch or the commit needs to be updated as we have conflated two different change in this one patch | 13:28 |
| kashyap | sean-k-mooney: We've had some back and forth on the change. I still need to fully read the lengthy design doc myself (although I pointed out a potential blocker) | 13:28 |
| sean-k-mooney | well it would not be a blocker for sev | 13:29 |
| sean-k-mooney | as sev will only ever work on x86 amd cpus | 13:29 |
| kashyap | sean-k-mooney: This is not the blocker, a different topic | 13:29 |
| kashyap | sean-k-mooney: Check out the points from Dan Berrange and myself on the re-proposed spec: https://review.openstack.org/#/c/641994/ | 13:30 |
| sean-k-mooney | sure but my point is that adding the more complex logic to this patch mean we basicaly block sev support until we figure out this multi arch multi machine type proablem which is not germain to enabling sev suppot | 13:30 |
| kashyap | The machine types figuring out is a lesser problem than the more thorny blocker identified in the re-proposed spec. | 13:32 |
| sean-k-mooney | the memory hard limit was previously discussed. we determined that it woudl only be needed if you did not use hugepages i think. the sev limt is a new factoide but that can be handeled via a placment inventory. that is new work. | 13:32 |
| sean-k-mooney | kashyap: what is the issue you are refering to? | 13:33 |
| kashyap | Hard-coding a memory value. | 13:33 |
| sean-k-mooney | ya but we dont need to do that if we use hugepages which i suggested last cycle | 13:34 |
| kashyap | Also each AMD machine has some limit on the number of VMs. That is also something that needs to be worked into the spec | 13:34 |
| *** lbragstad has joined #openstack-nova | 13:34 | |
| sean-k-mooney | ya that is new. it was not raised previously but that is simple enough to fix with a inventory of sev context in placement | 13:35 |
| sean-k-mooney | assuming we can auto discover that limit form either libvirt or sysfs | 13:35 |
| kashyap | I don't think this is just limited to hugepages. The core point about protecting the host from getting DoSed needs to be addressed. | 13:35 |
| sean-k-mooney | if we really needed too we could have a config option for it. that shoudl be in the spec i agree but i still dont see a hard blocker here | 13:36 |
| kashyap | Please see the cgroups idea DanPB mentioned in his remark. | 13:36 |
| kashyap | I think this is hard blocker. I'm even more convinced of it (that's why I gave a "-1", FWIW) after reading DanPB's message and talking to a developer who implemented SEV bits in libvirt | 13:36 |
| kashyap | Let's work it out on the spec. IRC is too real-time for me sometimes :-) | 13:37 |
| sean-k-mooney | kashyap: well if you require the use of hugepage then you disable over commit and dont need to use a hard memory limit as the vm memory is preallcoated and fixed | 13:37 |
| kashyap | sean-k-mooney: So you're saying when using SEV one should "mandate" huge pages? | 13:37 |
| sean-k-mooney | yes | 13:37 |
| kashyap | Hmm, I don't know the full implications of it. Need to think more. | 13:38 |
| sean-k-mooney | i suggested it last cycle to not need to set the hardlimit | 13:38 |
| sean-k-mooney | the main implication is no over commit | 13:38 |
| sean-k-mooney | and an implict numa topology | 13:38 |
| sean-k-mooney | the memory for sev i think i memlocked so it cant be swapped anyway so you would have disabled over commit already | 13:39 |
| sean-k-mooney | so requireing hugepages would remove some of the complexity of this feature and give a performacne boost | 13:39 |
| kashyap | Okay, I see the pinning the memory by using `hw:mem_page_size=large` point in the spec. | 13:40 |
| mriedem | efried: cdent: edleafe: semi interesting but also for an older release bug about forced live migration failing during PUT /allocations https://bugs.launchpad.net/nova/+bug/1820612 | 13:41 |
| openstack | Launchpad bug 1820612 in OpenStack Compute (nova) "Logging is hard to read if there is a problem with resources during live migration" [Undecided,Incomplete] | 13:41 |
| kashyap | sean-k-mooney: Please respond to Dan on the review, for the discussion to be "sticky". | 13:41 |
| sean-k-mooney | ya i didnt push too hard because it would be nice to not require hugepages but it make this a hole lot eaiser as not useing htem has other accounting issues | 13:41 |
| mriedem | efried: cdent: edleafe: i've pieced together that they are forcing a live migration to a dest host before queens where we'd move the source node allocations from the instance to the migration record, | 13:41 |
| * cdent reads | 13:42 | |
| sean-k-mooney | kashyap: sure. | 13:42 |
| mriedem | so in this code, it's doubling up the allocations for the instance consumer by putting the same source node allocations to the source provider and the new allocations to the dest provider, | 13:42 |
| mriedem | but placement pukes on the source provider PUT | 13:42 |
| mriedem | which makes me wonder if placement is trying to make a new allocation rather than replace it | 13:42 |
| mdbooth | This monkey patching change is a hydra :( | 13:43 |
| *** marst has joined #openstack-nova | 13:44 | |
| mriedem | cdent: anyway if they were queens+ they wouldn't have this problem, but that isn't a very helpful response to someone at ocata or pike | 13:45 |
| mriedem | if we know about a bug fix in placement though i could say 'yup this is fixed here, backport it' | 13:45 |
| kashyap | mdbooth: Seems like it is turning into "cthulhu patching" | 13:45 |
| cdent | mriedem: I'm still digesting. | 13:45 |
| edleafe | mriedem: I don't remember ever doubling the resources on the source when migrating to a different host | 13:45 |
| mdbooth | kashyap: Eventlet is just awful. | 13:46 |
| kashyap | mdbooth: Nod, I only hear about it (I didn't have to faff with it much so far). Skimmed your e-mail; thanks for the write-up! | 13:46 |
| mdbooth | I still don't understand what problem it solves, tbh. Seems to me it just makes your multi-threading even jankier. | 13:46 |
| edleafe | mdbooth: One of my first tasks in OpenStack was to replace the Twisted code with eventlet, because people were having so much trouble tracking down bugs in Twisted :) | 13:47 |
| kashyap | mdbooth: I saw your second email about using "Python's APIs except when not possible" for multi-threading | 13:47 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Add detection of SEV support from QEMU/AMD-SP/libvirt on AMD hosts https://review.openstack.org/633855 | 13:48 |
| mdbooth | edleafe: Hah. There's always something worse :) | 13:48 |
| edleafe | mdbooth: Yup | 13:48 |
| mdbooth | edleafe: But at a top level, why not just use python's multi-threading? | 13:48 |
| cdent | mriedem: Is there any chance there's a race. Something else ate a bit more more RAM during the transaction. That shouldn't be possible (at least not in more recent allocation handling) but it's very hard to keep track of changes... | 13:49 |
| edleafe | mdbooth: I really don't remember. The original nova code used Twisted for async, and it caused a lot of problems, so someone suggested eventlet, and... | 13:49 |
| mdbooth | Python already had fake concurrency, so why implement fake concurrency on top of fake concurrency? | 13:49 |
| kashyap | aspiers: Thanks for addressing the docs bit. You saw what I "discovered" with my comment :D Writing that out helped me understand it better | 13:49 |
| mriedem | cdent: it's possible, this was pre-generations | 13:50 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup comments around claim_resources method https://review.openstack.org/644576 | 13:50 |
| mriedem | cdent: i guess one way to find out is write a functional regression test with at-capacity source compute and then try to live migrate off it | 13:51 |
| mriedem | tests against at-capacity computes is not our strong suit in nova | 13:51 |
| cdent | quite | 13:51 |
| mriedem | anyway, while in here i've cleaned up a bunch of old comments https://review.openstack.org/644576 | 13:51 |
| cdent | huzzah | 13:52 |
| * cdent sticks in queue | 13:52 | |
| *** BjoernT has joined #openstack-nova | 13:53 | |
| sean-k-mooney | aspiers: kashyap i replied in the spec reproposal | 13:54 |
| *** Luzi has joined #openstack-nova | 13:54 | |
| *** whoami-rajat has quit IRC | 13:55 | |
| aspiers | kashyap: yep, that extra clarity helped me too. Have just replied inline in the review | 13:55 |
| aspiers | sean-k-mooney: thanks | 13:55 |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Add new "supports_amd_sev" capability to libvirt driver https://review.openstack.org/638680 | 14:04 |
| mriedem | alex_xu: replied in https://review.openstack.org/#/c/644293/ - thanks for those questions | 14:05 |
| *** wolverineav has joined #openstack-nova | 14:05 | |
| efried | mriedem: This ready to go? https://review.openstack.org/#/c/644423/ I didn't want to rubber-stamp it if you wanted another look. | 14:05 |
| mriedem | efried: looking | 14:06 |
| mriedem | got it | 14:06 |
| efried | thx | 14:07 |
| *** needscoffee is now known as kmalloc | 14:09 | |
| *** wolverineav has quit IRC | 14:11 | |
| *** mlavalle has joined #openstack-nova | 14:11 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add docs for compute capabilities as traits https://review.openstack.org/644293 | 14:12 |
| mriedem | efried: aspiers: alex_xu: ^ updated to use one of aspiers' suggestions and document the resource_provider_association_refresh thing in the commit message | 14:12 |
| *** sapd1_x has joined #openstack-nova | 14:13 | |
| efried | mriedem: +2, still looks good | 14:14 |
| *** xek_ has joined #openstack-nova | 14:17 | |
| *** mvkr has quit IRC | 14:18 | |
| *** xek has quit IRC | 14:19 | |
| kashyap | aspiers: I updated the machine types blueprint name (to "gracefully-handle-qemu-machine-types"). When I originally wrote it, discussions were hot on QEMU / libvirt & related lists about making 'q35' default. But since then many things have changed. | 14:20 |
| kashyap | aspiers: When you next respin the patch, might want to update the reference to it. | 14:21 |
| *** BjoernT has quit IRC | 14:24 | |
| *** mikal has joined #openstack-nova | 14:24 | |
| *** hongbin has joined #openstack-nova | 14:29 | |
| *** mikal has quit IRC | 14:29 | |
| *** phasespace has quit IRC | 14:32 | |
| *** dustinc has joined #openstack-nova | 14:36 | |
| *** adrianc has quit IRC | 14:36 | |
| kashyap | aspiers: For later discussion, I'm still thinking if Nova should be in the business of deciding the default machine type fo 'x86'. I'm updating the problem description in the specification. Can discuss there | 14:38 |
| kashyap | (So feel free to disregard me here.) | 14:38 |
| *** adrianc has joined #openstack-nova | 14:39 | |
| sean-k-mooney | kashyap: right now you can set it in the config if you chooes to otherwise we leave it up to qemu | 14:41 |
| sean-k-mooney | kashyap: im not sure if we need to chage that or not as intaller tool like tripleO can always just set it in the config | 14:42 |
| kashyap | sean-k-mooney: Yes, I'm writing it up in the spec, if only for discussion, and that we considered it. | 14:42 |
| kashyap | (There was an elaborate discussion on the 'qemu-devel' and 'libvirt' lists some months ago to change the defaults.) | 14:42 |
| sean-k-mooney | yes i know :) | 14:43 |
| kashyap | People seem to be overly paranoid about "backwards compat". | 14:43 |
| sean-k-mooney | well it would break centos 6 vms | 14:43 |
| sean-k-mooney | which are apparently still a thing | 14:43 |
| sean-k-mooney | but ya i generally say use a sane default and if there isnt one leave it up to the operator | 14:44 |
| aspiers | kashyap: ah, got it thanks | 14:44 |
| sean-k-mooney | you could argue q35 is a sane default on security grounds but backwards comptiablity is also a good argument against it at least for a time | 14:45 |
| dansmith | mriedem: yeah, you marked it as wishlist...does it need further action to demote it? | 14:45 |
| sean-k-mooney | eventrualy security will win and we will want to change the default. not sure if its now or not but its good to discusse in anycase | 14:46 |
| *** BjoernT has joined #openstack-nova | 14:47 | |
| *** mvkr has joined #openstack-nova | 14:47 | |
| *** itlinux has joined #openstack-nova | 14:47 | |
| *** igordc has joined #openstack-nova | 14:49 | |
| mriedem | dansmith: if you think it's a "never going to happen" thing, then you could mark it as won't fix with further words | 14:52 |
| *** NostawRm has joined #openstack-nova | 14:52 | |
| mriedem | dansmith: also for your awareness, https://review.openstack.org/#/c/640688/ is being discussed by the tc | 14:52 |
| *** wolverineav has joined #openstack-nova | 14:53 | |
| *** cfriesen has joined #openstack-nova | 14:53 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Eventlet monkey patching should be as early as possible https://review.openstack.org/626952 | 14:54 |
| *** jmlowe has joined #openstack-nova | 14:54 | |
| dansmith | mriedem: I don't know that I have a strong opinion on that (nor do I know why the tc would be involved in a discussion about it, but...) | 14:54 |
| kashyap | Anyone with an Ubuntu machine, can you please post this output in a pastebin: `qemu-system-x86 -machine help` | 14:54 |
| dansmith | mriedem: AFAICT, that's nova publishing metrics for which it is the authority to an external metrics-gathering service | 14:56 |
| dansmith | mriedem: I've previously been opposed to abusing nova to collect metrics on non-nova things "since it's already deployed on the compute node" instead of installing a proper metrics-gathering daemon | 14:56 |
| dansmith | but this is not that thing | 14:56 |
| mriedem | kashyap: paste.openstack.org/show/748029/ | 14:56 |
| mriedem | http://paste.openstack.org/show/748029/ | 14:56 |
| kashyap | mriedem: Thank you! | 14:57 |
| *** zkzmmrc has quit IRC | 14:57 | |
| mriedem | dansmith: ack, see my comments about the smorgasbord of existing half baked metrics/usage/audit things we already expose | 14:58 |
| *** luksky has quit IRC | 14:58 | |
| mriedem | my main opposition is piling on to that pile | 14:58 |
| *** cfriesen has quit IRC | 14:58 | |
| dansmith | yeah, this is not that, AFAICT | 14:58 |
| *** cdent has quit IRC | 14:59 | |
| *** wolverineav has quit IRC | 15:00 | |
| *** diablo_rojo has joined #openstack-nova | 15:02 | |
| *** adrianc has quit IRC | 15:02 | |
| *** artom has joined #openstack-nova | 15:06 | |
| *** adrianc has joined #openstack-nova | 15:08 | |
| *** cdent has joined #openstack-nova | 15:10 | |
| *** awalende has quit IRC | 15:10 | |
| openstackgerrit | sahid proposed openstack/os-traits master: ci: re-enable openstack-python35-jobs https://review.openstack.org/644595 | 15:11 |
| openstackgerrit | Gorka Eguileor proposed openstack/nova master: Use os-brick locking for volume attach and detach https://review.openstack.org/614190 | 15:12 |
| *** Luzi has quit IRC | 15:12 | |
| *** eharney has quit IRC | 15:14 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Address old TODO in claim_resources_on_destination https://review.openstack.org/644596 | 15:15 |
| *** dpawlik is now known as dpawlik_ | 15:16 | |
| *** awalende has joined #openstack-nova | 15:16 | |
| *** awalende has quit IRC | 15:16 | |
| *** sapd1_x has quit IRC | 15:19 | |
| *** whoami-rajat has joined #openstack-nova | 15:23 | |
| *** adrianc has quit IRC | 15:24 | |
| *** adrianc has joined #openstack-nova | 15:27 | |
| *** jmlowe has quit IRC | 15:29 | |
| *** ttsiouts has quit IRC | 15:38 | |
| mdbooth | This is weird, and doesn't reproduce for me locally: http://logs.openstack.org/52/626952/8/check/openstack-tox-docs/0876468/ara-report/result/73a64a42-855e-4818-980a-0a106766e721/ | 15:39 |
| mdbooth | Does anybody know how I can kick off a 'openstack-tox-docs' job in check in isolation? | 15:40 |
| *** ttsiouts has joined #openstack-nova | 15:40 | |
| mdbooth | I don't see that job in our .zuul.yaml, so I assume it's defined elsewhere | 15:40 |
| *** yan0s has joined #openstack-nova | 15:41 | |
| artom | mdbooth, tox -e docs? Or am I being a simpleton? | 15:42 |
| mdbooth | artom: That's how I can run it locally. That works fine. | 15:42 |
| mdbooth | It failed in the check queue, though. I want to see if that happens again without having to wait 2 hours. | 15:42 |
| artom | mdbooth, oh, you mean in CI | 15:42 |
| mdbooth | Indeed. | 15:42 |
| artom | mdbooth, I think if you just touch the docs subtree it'll only run those jobs | 15:43 |
| mdbooth | Do you know where that's defined? | 15:43 |
| artom | ... and you mean on your existing patches, don't you? | 15:43 |
| artom | Without pushing a new docs-only DNM patch or w/e | 15:43 |
| mdbooth | Well if I can find a way to kick it off in isolation I can layer a patch on top or something | 15:43 |
| mdbooth | But I don't know what's running it. It's not in nova's .zuul.yaml | 15:43 |
| mriedem | gibi: question for you in bence's neutron docs patch https://review.openstack.org/#/c/640390/5/doc/source/admin/config-qos-min-bw.rst@98 | 15:44 |
| sean-k-mooney | mriedem: the check queue jsut use ansible to run the same tox env | 15:44 |
| sean-k-mooney | mriedem: sorry mdbooth ^ | 15:44 |
| mdbooth | sean-k-mooney: Where is it defined? | 15:45 |
| sean-k-mooney | mdbooth: ill get the link. its in either zull-jobs or openstack-zull-jobs | 15:45 |
| mdbooth | sean-k-mooney: I want a way to actually run it on CI infrastructure. | 15:45 |
| sean-k-mooney | it should be identically to how you run it locally as its using a tox env but you might need to recreate your tox env with -r to make sure its clean | 15:46 |
| mdbooth | It runs successfully in a local clean tox env | 15:47 |
| mdbooth | I'm looking for a way to run it on CI | 15:47 |
| sean-k-mooney | i think its this job https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/zuul.d/jobs.yaml#L293-L321 | 15:47 |
| mdbooth | It failed there for a reason which looks bizarre. | 15:47 |
| sean-k-mooney | oh wait thats the old one | 15:47 |
| sean-k-mooney | this one https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/zuul.d/jobs.yaml#L618-L632 | 15:47 |
| sean-k-mooney | mdbooth: do you have the failure so i can take a look | 15:48 |
| mdbooth | http://logs.openstack.org/52/626952/8/check/openstack-tox-docs/0876468/ara-report/result/73a64a42-855e-4818-980a-0a106766e721/ | 15:48 |
| mdbooth | AttributeError: module 'select' has no attribute 'poll' | 15:48 |
| mdbooth | Given that this is against a monkey patching change, that's obviously suspect. However, it works locally. | 15:49 |
| *** luksky has joined #openstack-nova | 15:51 | |
| sean-k-mooney | well you can see the command at the top it was "tox -edocs -vv" | 15:51 |
| mdbooth | Yep. That's how I know how to run it locally. That works ;) | 15:52 |
| mdbooth | It doesn't work in CI, though. I was looking for a lightweight way to run that in CI to see if it can be reproduced in CI without having to wait 2 hours. | 15:52 |
| sean-k-mooney | mdbooth: https://docs.python.org/3.6/library/select.html#select.poll | 15:53 |
| sean-k-mooney | so apparently that is not suppored on all operating systems... | 15:53 |
| mdbooth | sean-k-mooney: You think I might have been scheduled on a Windows CI worker? | 15:53 |
| sean-k-mooney | we might be missing a depencny on the gate job | 15:53 |
| mdbooth | That seems improbable ;) | 15:54 |
| artom | sean-k-mooney, I'd thunk that meant Linux and Windows, not F29 and Ubuntu | 15:54 |
| sean-k-mooney | it was on ubunut 18.04 | 15:54 |
| *** ttsiouts has quit IRC | 15:54 | |
| artom | mdbooth, actually wait, are *you* running in on Windows? | 15:54 |
| artom | You might be, knowing you | 15:54 |
| mdbooth | artom: Hmm.... | 15:55 |
| mdbooth | Yes I am. | 15:55 |
| sean-k-mooney | mdbooth: can i have the link to the gerrit review | 15:55 |
| sean-k-mooney | ill run it on my laptop | 15:55 |
| *** hamzy has quit IRC | 15:55 | |
| mdbooth | artom: Running in WSL. However, I'd expect to see that *I* don't have poll() | 15:56 |
| mdbooth | Linux definitely has poll. | 15:56 |
| * mdbooth will run it on native Linux | 15:56 | |
| artom | mdbooth, true. | 15:56 |
| jangutter | mdbooth: It's not a Windows CI worker - the pre jobs installed some bindep packages in Ubuntu: http://logs.openstack.org/52/626952/8/check/openstack-tox-docs/0876468/ara-report/result/9863769a-0d01-4a5a-b45a-345628504cbf/ | 15:56 |
| *** BjoernT has quit IRC | 15:57 | |
| mdbooth | jangutter: Surely we don't have any Windows CI workers! ? | 15:57 |
| sean-k-mooney | mdbooth: we proably do for hyperv | 15:57 |
| sean-k-mooney | but not for docs | 15:57 |
| mriedem | lyarwood: dansmith: can one of you fast approve https://review.openstack.org/#/c/643011/ ? | 15:57 |
| *** mrch_ has quit IRC | 15:58 | |
| *** igordc has quit IRC | 15:58 | |
| *** wolverineav has joined #openstack-nova | 15:59 | |
| mriedem | bauzas: you want to get this rocky backport https://review.openstack.org/#/c/643098/ ? | 15:59 |
| mdbooth | Linux is a bazillion times faster than WSL. | 15:59 |
| bauzas | mriedem: sure, looking | 15:59 |
| sean-k-mooney | mdbooth: cygwin was faster the WSL the last time i tried WSL | 15:59 |
| openstackgerrit | Merged openstack/nova master: Remove unnecessary default provider_tree when getting traits https://review.openstack.org/644452 | 15:59 |
| mriedem | once we get those in i think we can do a rocky release | 16:00 |
| mriedem | melwitt: ^ | 16:00 |
| jangutter | mdbooth, sean-k-mooney: do the tox docs jobs normally run without the "Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: oslo_context.context, urllib3." warning? | 16:02 |
| lyarwood | mriedem: done | 16:02 |
| *** ttsiouts has joined #openstack-nova | 16:02 | |
| mdbooth | jangutter: I just added that message | 16:02 |
| *** jaosorior has quit IRC | 16:02 | |
| sean-k-mooney | jangutter: we should not be monkeypatch for docs | 16:02 |
| *** BjoernT has joined #openstack-nova | 16:02 | |
| mdbooth | jangutter: However, we have always monkey patched for docs | 16:02 |
| sean-k-mooney | jangutter: that said we might be | 16:03 |
| mdbooth | That warning message simply points it out | 16:03 |
| mdbooth | Ok, I can't run tox -edocs on Linux with or without my patch | 16:03 |
| sean-k-mooney | mdbooth: so i get the same error locally | 16:03 |
| mdbooth | That's annoying | 16:03 |
| *** BjoernT has quit IRC | 16:04 | |
| mdbooth | Ah, that will be a pyc thing | 16:04 |
| sean-k-mooney | i had to recreate the env with "tox -e docs -vv -r" | 16:04 |
| sean-k-mooney | but i got the same | 16:04 |
| sean-k-mooney | Exception occurred: | 16:04 |
| sean-k-mooney | File "/usr/lib/python3.6/selectors.py", line 348, in __init__ | 16:04 |
| * mdbooth recreated it with rm -rf | 16:04 | |
| sean-k-mooney | self._poll = select.poll() | 16:04 |
| sean-k-mooney | AttributeError: module 'select' has no attribute 'poll' | 16:04 |
| sean-k-mooney | error | 16:04 |
| sean-k-mooney | that works too | 16:05 |
| sean-k-mooney | ill try it on a centos vm | 16:05 |
| mdbooth | So, the only thing my patching is changing is where we monkey patch | 16:05 |
| sean-k-mooney | mdbooth: ya so i dont think we used to monkey patch docs | 16:06 |
| *** cdent has quit IRC | 16:06 | |
| mdbooth | sean-k-mooney: Why do you think that? | 16:06 |
| sean-k-mooney | docs only installed stuff for the vrsion number and then config stuff | 16:06 |
| sean-k-mooney | mdbooth: i dont think shinix loaded an entrypoint that incldued a path that invokded monky pataching | 16:07 |
| mdbooth | Do you know what entrypoints it loads? | 16:07 |
| sean-k-mooney | it just opened the nova/conf subtree i think | 16:07 |
| sean-k-mooney | that is a stephenfin question but ill see if i can find out quickly | 16:07 |
| mdbooth | Ok, I can't reproduce locally on F29 | 16:07 |
| mdbooth | sean-k-mooney: What eventlet version do you have? | 16:08 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/doc/source/conf.py#L22 | 16:08 |
| mdbooth | Although that's tox installed, so should be the same | 16:08 |
| sean-k-mooney | eventlet==0.24.1 | 16:09 |
| mdbooth | Hmm, same | 16:09 |
| sean-k-mooney | am why do we do https://github.com/openstack/nova/blob/master/doc/source/conf.py#L201 | 16:09 |
| mdbooth | I don't know, but as that's not eventlet I assume that's ok | 16:10 |
| sean-k-mooney | anyway the only thing we import form nova is nova.version | 16:10 |
| mdbooth | sean-k-mooney: Where do you see that? | 16:10 |
| sean-k-mooney | so before that would not have monkeypatch anything | 16:10 |
| sean-k-mooney | where do i see what? | 16:10 |
| mdbooth | <sean-k-mooney> anyway the only thing we import form nova is nova.version | 16:11 |
| mdbooth | What's your basis for that? | 16:11 |
| sean-k-mooney | oh well i just looked at the conf.py file that sphinx uses i think that is the only pyton file it uses to build the docs | 16:11 |
| sean-k-mooney | im going to do a code search in a second | 16:12 |
| *** mikal has joined #openstack-nova | 16:12 | |
| *** BjoernT has joined #openstack-nova | 16:12 | |
| sean-k-mooney | mdbooth: so assuming my github foo hasnt failed me this is the only hit https://github.com/openstack/nova/search?l=Python&q=%22from+nova%22+path%3A%2Fdoc%2Fsource | 16:15 |
| mdbooth | sean-k-mooney: that link doesn't work. | 16:15 |
| mdbooth | What are you looking for? | 16:16 |
| sean-k-mooney | this was the search query in the nova github '"from nova" path:/doc/source' | 16:16 |
| stephenfin | sean-k-mooney: Huh? Context? | 16:16 |
| mdbooth | Ok, sphinx does load_extension() | 16:16 |
| * stephenfin is trying to parse log | 16:16 | |
| lyarwood | http://codesearch.openstack.org/ - might be easier to search with | 16:17 |
| mdbooth | mod = __import__(extname, None, None, ['setup']) | 16:17 |
| *** itlinux has quit IRC | 16:17 | |
| sean-k-mooney | stephenfin: we are trying to figur out what monkeypatch in the toplevel nova module would break the docs job | 16:17 |
| mdbooth | That ends up loading nova/tests/__init__.py, which in my patch does monkey patching | 16:17 |
| *** mikal has quit IRC | 16:17 | |
| mdbooth | Whereas previously it was lower down separately in both unit and functional | 16:17 |
| sean-k-mooney | lyarwood: tried that first but its harder to make look a a single directory | 16:17 |
| *** hamzy has joined #openstack-nova | 16:18 | |
| mdbooth | Although, why the hell would it load tests and not, for eg nova.cmd or nova.api.openstack? | 16:18 |
| stephenfin | Not an answer but various Sphinx extensions do load the nova source | 16:18 |
| lyarwood | sean-k-mooney: kk | 16:18 |
| *** eharney has joined #openstack-nova | 16:18 | |
| stephenfin | sphinx.autodoc and oslo_config.sphinxext jump to mind | 16:18 |
| mdbooth | stephenfin: I'd expect them to load the *whole* nova source though, right? | 16:18 |
| * mdbooth tries to work out if sphinx really was doing monkey patching before | 16:19 | |
| stephenfin | mdbooth: What are they loading? | 16:19 |
| * mdbooth assumed yes | 16:19 | |
| stephenfin | eventlet monkeypatching? | 16:19 |
| stephenfin | or monkeypatching of other things | 16:19 |
| sean-k-mooney | mdbooth: we dont generate docs for the code directly so ingeneral i would have assuemd it should not load the nova code. | 16:19 |
| mdbooth | stephenfin: eventlet. It's done just by loading certain modules. | 16:19 |
| mdbooth | stephenfin: If we import everything, we do monkey patching eventually. | 16:20 |
| stephenfin | We can monkeypatch the monkeypatch though... | 16:20 |
| stephenfin | if that was actually what was happening | 16:20 |
| stephenfin | stub out eventlet, so to speak | 16:20 |
| sean-k-mooney | stephenfin: ew | 16:20 |
| * mdbooth is going to try to work out if we loaded everything | 16:20 | |
| stephenfin | sean-k-mooney: That comment also applies to eventlet itself ;) | 16:21 |
| mdbooth | stephenfin: If it was acceptable we could define an environment variable which, if set, would disable monkey patching | 16:21 |
| mdbooth | However, I'd assumed that was not acceptable | 16:21 |
| jangutter | it would be really funny if rendering the docs accidentally caused an entire Nova deployment to spring up. | 16:21 |
| stephenfin | you'd have to load the os module to do that | 16:21 |
| stephenfin | which I assume eventlet needs to monkeypatch | 16:21 |
| sean-k-mooney | we should not be executing any of the code so we should not need to monkeypatch as monkeypatch should never change any of the api signiture and therefor should nto change the docs | 16:21 |
| sean-k-mooney | in the eventlet case | 16:21 |
| mdbooth | sean-k-mooney: coulda shoulda isn't relevant, though, as it's done at import time | 16:22 |
| sean-k-mooney | mdbooth: we do that for debug right | 16:22 |
| stephenfin | sean-k-mooney: Any code outside a function and not within a 'if __name__ == "__main__"' conditional is going to be executed tho | 16:22 |
| mdbooth | sean-k-mooney: that's done with config | 16:22 |
| mdbooth | sean-k-mooney: Actually by inspecting command line arguments | 16:22 |
| sean-k-mooney | yes but you can now set config values via the environment | 16:23 |
| sean-k-mooney | its new in a recent oslo config | 16:23 |
| mdbooth | def enabled(): | 16:23 |
| mdbooth | return ('--remote_debug-host' in sys.argv and | 16:23 |
| mdbooth | '--remote_debug-port' in sys.argv) | 16:23 |
| mdbooth | ^^^ is the guard | 16:23 |
| sean-k-mooney | yes for debug | 16:23 |
| sean-k-mooney | anyway stephenfin do you know what we use from nova beyond the version and config options for our docs. is that it or do we use other parts of the tree? | 16:24 |
| stephenfin | I'm trying to figure out if we still generate API docs. I don't think we do | 16:24 |
| mdbooth | It's safe to say that this interface is hairy AF, though | 16:25 |
| *** ttsiouts has quit IRC | 16:25 | |
| mdbooth | That is, even if we weren't doing it, somebody only had to sneeze and we were. | 16:25 |
| sean-k-mooney | ya that said i know the docs team wanted to get away from installing proejct deps and loading project in the doc env a while ago. i dont know if that effort did or not but its part of the reason why we created a seperate doc/requirement.txt | 16:27 |
| mdbooth | No, we weren't doing it before | 16:27 |
| sean-k-mooney | monkeypatching? | 16:27 |
| mdbooth | Previously we monkey patched in nova/test.py and nova/tests/functional/__init__.py and sphinx didn't hit that | 16:28 |
| sean-k-mooney | right | 16:28 |
| mdbooth | I moved the test monkey patching to nova/tests/__init__.py, and now it does hit that | 16:28 |
| mdbooth | Go figure | 16:28 |
| mdbooth | If anybody can explain that interface to me I'd be interested | 16:28 |
| sean-k-mooney | does that fix it our recreate teh breakage? | 16:28 |
| mdbooth | I can't reproduce the failure locally, even with monkey patching | 16:29 |
| mdbooth | Ok, I'm just going to move the monkey patching points and add comments not to move them again | 16:29 |
| mdbooth | I have no idea how you're supposed to understand what sphinx imports | 16:30 |
| stephenfin | mdbooth: The whole thing is basically a hack, but there's no easy way to do it | 16:30 |
| *** ivve has quit IRC | 16:31 | |
| *** sdunixgeek has joined #openstack-nova | 16:33 | |
| sean-k-mooney | stephenfin: the sane thing to do would be not import the nova source when building docs but that is an issue for generationg config docs if we want to do that with sphinx i guess | 16:33 |
| *** BjoernT has quit IRC | 16:33 | |
| stephenfin | sean-k-mooney: We either do it dynamically, like we do, or we have a tool to export config to a machine readable format (YAML) and parse that | 16:34 |
| stephenfin | and if we go with the latter, updating those definitions now becomes a manual step meaning we need to add tests to ensure it doesn't get out of date | 16:34 |
| mdbooth | stephenfin: But even then presumably the tool which generated the yaml would import the source with a python interpreter | 16:35 |
| jangutter | sean-k-mooney: http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html "Warning: autodoc imports the modules to be documented. If any modules have side effects on import, these will be executed by autodoc when sphinx-build is run." | 16:35 |
| sean-k-mooney | there is prably an option 3 | 16:35 |
| stephenfin | autodocing APIs is even more complicated since you do need a full environment to be able to do introspection | 16:35 |
| mdbooth | stephenfin: So we'd just be moving the problem elsewhere | 16:35 |
| stephenfin | mdbooth: Correct, but that's sometimes a reasonable thing to do | 16:36 |
| sean-k-mooney | jangutter: ya im aware of how autodocs works. | 16:36 |
| *** ttsiouts has joined #openstack-nova | 16:36 | |
| sean-k-mooney | jangutter: its both good and bad | 16:36 |
| sean-k-mooney | jangutter: we have some module level stuff we caluate on import that is slow | 16:37 |
| stephenfin | mdbooth: We can either store this stuff as code and deal with these dependency issues or we store it separately and then deal with keeping it up-to-date. Choose one | 16:37 |
| stephenfin | (I'm very much in favour of the former, as noted by my work on (cliff|oslo_config|oslo_policy).sphinxext | 16:38 |
| stephenfin | *) | 16:38 |
| sean-k-mooney | mdbooth: did you determin if the issue is with monkey patching twice or not monkey patch at all or didnt find the root cause | 16:38 |
| jangutter | sean-k-mooney: and the current theory is that autodoc and monkeypatching are fighting? | 16:38 |
| sean-k-mooney | jangutter: i think we are possibly monkeypatching twice | 16:38 |
| mdbooth | sean-k-mooney: I couldn't reproduce, so I have no idea. | 16:38 |
| sean-k-mooney | mdbooth: i can repoduce locally | 16:38 |
| mdbooth | sean-k-mooney: On Ubuntu? | 16:38 |
| stephenfin | jangutter: Not necessarily autodoc. Probably oslo_config.sphinxext | 16:38 |
| sean-k-mooney | yes.. | 16:39 |
| stephenfin | Don't think we do autodoc | 16:39 |
| jangutter | mdbooth: I reproduced on a CentOS 7 container. | 16:39 |
| sean-k-mooney | ok so it cross distro | 16:39 |
| jangutter | stephenfin: ah! | 16:39 |
| * mdbooth is just going to bypass the problem by not changing the patching point for tests. | 16:39 | |
| stephenfin | mdbooth: Good reading, fwiw http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html | 16:39 |
| sean-k-mooney | mdbooth: we should proably open a bug and fix it later | 16:39 |
| mdbooth | That seems to avoid whatever sphinx is importing | 16:39 |
| mdbooth | sean-k-mooney: For what? | 16:40 |
| stephenfin | mdbooth: It's loading the module. I suspect you'd get the same issue if you left it in 'nova/test.py' and added 'import .test' to 'nova/tests/__init__.py' | 16:41 |
| stephenfin | *'import nova.test' | 16:41 |
| mdbooth | stephenfin: Which module is it loading? | 16:42 |
| *** cdent has joined #openstack-nova | 16:42 | |
| stephenfin | It's loading 'nova', which is causing it to load 'nova.tests', I would imagine | 16:43 |
| sean-k-mooney | mdbooth: https://review.openstack.org/#/c/626952/8/nova/monkey_patch.py@55 | 16:43 |
| sean-k-mooney | mdbooth: does that comment make sense. | 16:43 |
| *** tstrul has joined #openstack-nova | 16:43 | |
| sean-k-mooney | actull i test it locally | 16:43 |
| *** itlinux has joined #openstack-nova | 16:44 | |
| stephenfin | and that's happening because of https://github.com/openstack/nova/blob/master/doc/source/conf.py#L27 | 16:45 |
| mdbooth | sean-k-mooney: Right, if the problem is importing twice that would work round it. However, if we previously weren't monkey patching at all by happy coincidence, I'm not sure if that's not just a dirtier hack. | 16:45 |
| stephenfin | though that's clearly not necessary https://github.com/openstack/nova/blob/master/doc/source/conf.py#L22 | 16:45 |
| mdbooth | stephenfin: “Never add a package directory, or any directory inside a package, directly to the Python path”. | 16:46 |
| *** mvkr has quit IRC | 16:47 | |
| mdbooth | From that traps doc you sent earlier | 16:47 |
| *** sdunixgeek has quit IRC | 16:47 | |
| *** udesale has quit IRC | 16:48 | |
| stephenfin | yuuuup | 16:48 |
| stephenfin | In general, messing with the Python path is a bad idea | 16:48 |
| stephenfin | but the alternative is making all these things installable | 16:48 |
| mdbooth | s/(the|path) // | 16:48 |
| mdbooth | g | 16:49 |
| stephenfin | :) | 16:49 |
| stephenfin | I like to live on the edge, personally | 16:49 |
| mdbooth | Ok, I'm going to do both | 16:50 |
| mdbooth | I'm going to move the monkey patching points back to where they were, *and* add a guard to explicitly avoid double monkey patching | 16:51 |
| stephenfin | mdbooth: I think you might also be interested in some of Armin Ronacher's most recent presentations on all things Python http://lucumr.pocoo.org/talks/ | 16:52 |
| sean-k-mooney | mdbooth: i tryed the guard locally and it did not seam to help | 16:52 |
| stephenfin | he of Flask fame | 16:52 |
| stephenfin | ...openstack/nova/doc/source/reference/notifications.rst:142:Unknown directive type "automodule". | 16:54 |
| stephenfin | So we are using autodoc | 16:54 |
| *** luksky has quit IRC | 16:54 | |
| *** dtantsur is now known as dtantsur|afk | 16:56 | |
| stephenfin | mdbooth: Try adding this to doc/source/conf.py | 16:58 |
| stephenfin | autodoc_mock_imports = ['nova.tests'] | 16:58 |
| *** amodi has quit IRC | 16:59 | |
| mdbooth | stephenfin: That's still really flaky, though, because it doesn't explain why we don't import the nova.cmd or nova.api.openstack.wsgi_app entry points | 16:59 |
| mdbooth | stephenfin: I'd prefer to either go from flaky to non-flaky, or just leave well alone | 16:59 |
| mdbooth | flaky->different flaky isn't worth the churn, I think | 17:00 |
| sean-k-mooney | mdbooth: im flipped the condition on my local gard to never monkey patch and that fixes it | 17:00 |
| sean-k-mooney | so the issue is the docs only build if we never monkeypatch | 17:00 |
| stephenfin | Oh, I didn't realize we did that there | 17:01 |
| sean-k-mooney | stephenfin: we monkey patch on the __init__.py in nova/cmd | 17:01 |
| stephenfin | Yup, see that now | 17:01 |
| stephenfin | weirdddd | 17:01 |
| *** jmlowe has joined #openstack-nova | 17:01 | |
| stephenfin | mdbooth: Can you share the debug logs from Sphinx? | 17:02 |
| sean-k-mooney | ya anywya as i said above the issue is the sphinx only works if we do not do eventlet.monkeypatch | 17:02 |
| mdbooth | stephenfin: In what configuration? | 17:02 |
| mdbooth | stephenfin: FWIW, this doesn't fail for me on F29, even when monkey patched | 17:02 |
| stephenfin | Oh, wherever it's failing | 17:03 |
| mdbooth | Although apparently it does fail on Centos7 and Ubuntu, so I'm guessing it's some library version | 17:03 |
| * mdbooth can't make it fail | 17:03 | |
| stephenfin | I assume that's only on Ubuntu so? | 17:03 |
| stephenfin | Ack | 17:03 |
| sean-k-mooney | stephenfin: jangutter mentioned its broken on centos 7 too | 17:03 |
| sean-k-mooney | mdbooth: are you running on py36 on fedora 29 or py37 | 17:03 |
| * mdbooth just hacked tracepoints into the code wherever we monkey patch to work out if we're hitting it and which one | 17:03 | |
| sean-k-mooney | ubuntu and centos 7 use py36 | 17:04 |
| *** wolverineav has quit IRC | 17:04 | |
| stephenfin | Yeah, we need debug logs for this stuff | 17:04 |
| mdbooth | 37 | 17:04 |
| sean-k-mooney | ok so it could be a 36 vs 37 thing | 17:04 |
| openstackgerrit | Lance Bragstad proposed openstack/nova master: Give the policy vision document a facelift https://review.openstack.org/644615 | 17:05 |
| sean-k-mooney | ill install 3.7 and check | 17:05 |
| lbragstad | mriedem lumped it all in there ^ lemme know if you want that broken up | 17:05 |
| lbragstad | cc efried ^ | 17:06 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Misc cleanups https://review.openstack.org/644616 | 17:07 |
| *** luksky has joined #openstack-nova | 17:07 | |
| sean-k-mooney | so looks like ti works fine on python 3.7 with the gauard | 17:10 |
| *** rpittau is now known as rpittau|afk | 17:10 | |
| sean-k-mooney | ill revert to a clean copy of the patch and confrim its fine without the guard too | 17:10 |
| mdbooth | Hehe, with vim's standard highlighting in nova.monkey_patch, I see: NOTE, NOTE, TODO, NOTE, NOTE, NOTE, NOTE | 17:13 |
| aspiers | sean-k-mooney: we got Brijesh from AMD to respond to your hugepages suggestion | 17:13 |
| sean-k-mooney | mdbooth: stephenfin so yes this only happens on python3.6 which is why it passes on fedora and not on ubuntu or centos which default to python 3.6 | 17:13 |
| *** jmlowe has quit IRC | 17:13 | |
| *** helenafm has quit IRC | 17:13 | |
| mdbooth | Well, at least they're all in the same place, now | 17:13 |
| sean-k-mooney | aspiers: oh ok i havent read it yet | 17:13 |
| sean-k-mooney | aspiers: was it favorable or not? | 17:14 |
| aspiers | sean-k-mooney: I don't think so. Bottom of https://review.openstack.org/#/c/641994/2 | 17:14 |
| aspiers | SEV by design prevents the kernel copying encrypted pages around | 17:15 |
| aspiers | he says using hugepages would pin it within the hugepages pool, but not pin each page to a single address | 17:15 |
| aspiers | I'm on a call with AMD now | 17:15 |
| *** jaypipes has joined #openstack-nova | 17:16 | |
| *** igordc has joined #openstack-nova | 17:16 | |
| sean-k-mooney | aspiers: ya dan berrange replied and said hugepages may not be sufficent too as the roms and uefi pflash would not be pinned | 17:16 |
| aspiers | right | 17:16 |
| aspiers | sean-k-mooney: how does it work for device passthrough? | 17:18 |
| aspiers | apparently that's a similar scenario in terms of pinning | 17:18 |
| *** zhubx has quit IRC | 17:19 | |
| *** zhubx has joined #openstack-nova | 17:19 | |
| sean-k-mooney | the iommu maps the device mmio region to a fixed set of addresses that are maped to the memory regions listed in the pci VF config space i think. but honestly i tought that a hugepage was similarly fixed and could not migrate when we use explcit not transparent hugepages | 17:20 |
| mdbooth | It seems there's no need for a double-import guard unless you borked the python path, because python doesn't appear to import a second time anyway | 17:21 |
| mdbooth | stephenfin: Is ^^^ your understanding? | 17:21 |
| sean-k-mooney | mdbooth: we dont need the import guard correct | 17:21 |
| stephenfin | mdbooth: That sounds reasonable, aye | 17:22 |
| stephenfin | Based on my understanding | 17:22 |
| sean-k-mooney | mdbooth: the issue is we never monkeypatched the docs before and under python 3.6 only it breaks things if we do. apparently | 17:22 |
| mdbooth | Incidentally, that likely only breaks because we didn't patch earlier... | 17:24 |
| mdbooth | Perhaps we should update sphinx to always monkey patch ;) | 17:24 |
| *** gyee has joined #openstack-nova | 17:24 | |
| *** priteau has joined #openstack-nova | 17:25 | |
| sean-k-mooney | mdbooth: you joke but we could proably write a tiny sphinx extenion to monky patch earlir. that said didnt you plane to patch in nova/__init__.py | 17:25 |
| sean-k-mooney | oh your not doing that in that patch | 17:26 |
| mdbooth | sean-k-mooney: Yeah, that breaks everything for different reasons. | 17:27 |
| aspiers | sean-k-mooney: yeah, I get the impression Brijesh (AMD) thinks otherwise but TBH I'm out of my depth here :) | 17:27 |
| sean-k-mooney | ah ok :) well it is eventlets | 17:27 |
| aspiers | definitely not my area of expertise | 17:28 |
| sean-k-mooney | aspiers: well we still lock the memory expiclitly when we use hugepages | 17:28 |
| aspiers | ah | 17:28 |
| sean-k-mooney | i.e. we set the locked element https://libvirt.org/formatdomain.html#elementsMemoryBacking | 17:28 |
| *** janki has quit IRC | 17:29 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Eventlet monkey patching should be as early as possible https://review.openstack.org/626952 | 17:29 |
| mdbooth | Thar she blows. | 17:29 |
| sean-k-mooney | or maybe we only do that for realtime i can check | 17:29 |
| mdbooth | Lets see if that passes muster. | 17:29 |
| aspiers | sean-k-mooney: ah OK, is that also done for device pass-through? | 17:30 |
| sean-k-mooney | no its not done for device pass-though | 17:30 |
| *** wolverineav has joined #openstack-nova | 17:31 | |
| *** BjoernT has joined #openstack-nova | 17:31 | |
| sean-k-mooney | aspiers: this is the only time we current lock the memory explcitly whic is for realtime guests | 17:32 |
| sean-k-mooney | http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/driver.py#n4810 | 17:32 |
| *** luksky has quit IRC | 17:32 | |
| *** yan0s has quit IRC | 17:33 | |
| *** jmlowe has joined #openstack-nova | 17:33 | |
| sean-k-mooney | the way dpdk work i am prtty sure the hugepage does not migrate to different phyical pages on the host as it is used for dma transfer for the vhost-user nics | 17:34 |
| sean-k-mooney | aspiers: in anycase if you have hardware you can test it on it would be worth testing with hugepages | 17:34 |
| aspiers | sean-k-mooney: AMD already tried hugepages in one of their earliest iterations | 17:36 |
| sean-k-mooney | aspiers: but if dan's and brijesh's assertion regarding the rom+pflash issue are correct it may not be enough | 17:36 |
| aspiers | that's what I just heard | 17:36 |
| aspiers | they are also saying that mlock() is just a hint not a guarantee | 17:36 |
| *** amodi has joined #openstack-nova | 17:36 | |
| sean-k-mooney | ok well in that case i dont know what the best path forword is on the hardlimit issue | 17:37 |
| aspiers | one suggestion was to expose the right hardlimit via QEMU / libvirt | 17:37 |
| aspiers | so nova could just query it | 17:38 |
| sean-k-mooney | aspiers: if qemu/libvirt can provide it that sounds resonable but it would make the relevent libvirt/qemu version that provides that the minium qemu/libivrt for that feature | 17:39 |
| aspiers | correct, there's already a minimum version requirement anyway | 17:39 |
| aspiers | I think everyone is aware of the issue now though, so hopefully Dan and Brijesh can figure something out :) | 17:40 |
| *** mrch_ has joined #openstack-nova | 17:40 | |
| *** ttsiouts has quit IRC | 17:40 | |
| sean-k-mooney | aspiers: so reading dan's comment again if we just remove the hard_limmit entirly then that may be suffienct to adress his orignial comment here https://review.openstack.org/#/c/641994/2/specs/train/approved/amd-sev-libvirt-support.rst@167 | 17:45 |
| sean-k-mooney | ah but that wont work as that is how you are pinning the memory | 17:46 |
| aspiers | right | 17:46 |
| *** bryan_stephenson has joined #openstack-nova | 17:48 | |
| aspiers | bryan_stephenson: catch up on the conversation via http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-03-19.log.html#t2019-03-19T17:13:27 | 17:49 |
| sean-k-mooney | aspiers: honestly looking at the documentation of hard_limit i think relying on it for the behavior you desire is undocumented and not part of the contract at least form the libvirt perspective | 17:50 |
| sean-k-mooney | aspiers: hard_limit does not guarntee teh meoeoy will not be swapped or that it will be preallocated | 17:50 |
| sean-k-mooney | aspiers: thos poperties are conntoeld via allocation and locked in the memory backing | 17:51 |
| bbobrov | lets just ask on the libvirt mailing list | 17:53 |
| bbobrov | (hi) | 17:53 |
| aspiers | I agree with bbobrov | 17:53 |
| aspiers | this probably requires libvirt changes | 17:53 |
| *** BjoernT has quit IRC | 17:54 | |
| aspiers | bbobrov: Do you want to kick off the thread there? | 17:54 |
| sean-k-mooney | bbobrov: aspiers sure let us know what they say | 17:54 |
| bbobrov | aspiers: yep, i will do it, but starting tomorrow | 17:55 |
| aspiers | we can just link to the spec review and say "please find us a solution" :-) | 17:55 |
| sean-k-mooney | bbobrov: aspiers but as i said the behavior that is being asserted for hard_limit. i.e. that it prevents the page form migrating is not documented in teh libvirt docs https://libvirt.org/formatdomain.html#elementsMemoryTuning | 17:55 |
| aspiers | bbobrov: great thanks | 17:55 |
| aspiers | sean-k-mooney: ack | 17:55 |
| aspiers | https://libvirt.org/formatdomain.html#elementsMemoryBacking seems to be the bit for pinning but if that just does an mlock() then apparently it's not enough | 17:56 |
| aspiers | from what I heard earlier | 17:56 |
| aspiers | although that seems strange to me, since then why would it be good enough for realtime? | 17:56 |
| aspiers | sean-k-mooney: real-time also needs a hard guarantee, right? | 17:56 |
| bbobrov | why have we decided to use memtune for pinning in the first place? | 17:57 |
| aspiers | because that's what the libvirt/QEMU guys recommended initially | 17:57 |
| bbobrov | if there is no direct indication that it actually pins memory | 17:57 |
| aspiers | they said it did | 17:57 |
| bbobrov | is there a mailing list thread about it? | 17:57 |
| aspiers | yes, on the internal list | 17:57 |
| sean-k-mooney | aspiers: not entirly. for minium latency variance yes but it wont break the feature if there was host page migration in the background but it may break SLAs | 17:58 |
| bbobrov | eh, internal lists | 17:58 |
| aspiers | bbobrov: our sev list | 17:58 |
| bbobrov | yeah, i got it, still ba | 17:58 |
| bbobrov | *bad | 17:58 |
| sean-k-mooney | bbobrov: well this is what the amdese repo does https://github.com/AMDESE/AMDSEV/blob/master/xmls/sample.xml#L218 | 17:59 |
| sean-k-mooney | it jsue set hard_limit | 17:59 |
| sean-k-mooney | there is no other memory tuning or pinning but i would not have assumed that pinned any memory | 17:59 |
| *** derekh has quit IRC | 18:00 | |
| aspiers | "The value of the domain/memtune/hard_limit element will be used to setrlimit(RLIMIT_MEMLOCK, hard_limit) on the qemu process and in memory.limit_in_bytes setting of the processes memory controller (/sys/fs/cgroup/memory/machine.slice/machine-<vmid><vmname>.scope/memory.limit_in_bytes)" | 18:00 |
| *** BjoernT has joined #openstack-nova | 18:00 | |
| aspiers | that's from our libvirt guy | 18:00 |
| aspiers | also https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/qemu/qemu_domain.c;h=ba3fff607a93533b9b47956cc2cfa70237e7c041;hb=HEAD#l10134 | 18:01 |
| *** itlinux has quit IRC | 18:02 | |
| gibi | mriedem: I've replied in https://review.openstack.org/#/c/640390/5/doc/source/admin/config-qos-min-bw.rst@98 | 18:02 |
| *** jmlowe has quit IRC | 18:04 | |
| bryan_stephenson | Accoring to https://libvirt.org/formatdomain.html#elementsMemoryBacking setting "locked" reserves/locks the physical memory. One use is for DMA, which would only be useful if the memory did not move. Do we think that memory might move locations even if it is locked? | 18:07 |
| *** priteau has quit IRC | 18:07 | |
| *** ivve has joined #openstack-nova | 18:07 | |
| *** gerrykopec has joined #openstack-nova | 18:07 | |
| aspiers | bryan_stephenson: that's what Brijesh said on the call. he said mlock(2) was just a hint | 18:07 |
| aspiers | the man page makes it sound like more than that, but I don't know | 18:08 |
| bryan_stephenson | Then how does DMA not get goofed up? | 18:08 |
| sean-k-mooney | bryan_stephenson we use hugepage memory for dma trasfer with dpdk so i think either hugepage or setting locked expclitly would be suffient | 18:08 |
| aspiers | Honestly it's beyond my paygrade ;-) | 18:08 |
| bryan_stephenson | So the "locked" just calls mlock() which may not be sufficient. Is that a correct understanding? | 18:09 |
| aspiers | That's what I think I heard from Brijesh | 18:09 |
| sean-k-mooney | aspiers: so looking at that funtion it runtrun Unlimited if the memory is locked to this would not caluatle a valide limit for realtime guests | 18:12 |
| sean-k-mooney | https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/qemu/qemu_domain.c;h=ba3fff607a93533b9b47956cc2cfa70237e7c041;hb=HEAD#l10049 | 18:12 |
| *** wolverineav has quit IRC | 18:12 | |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add request-filter-image-types spec https://review.openstack.org/644625 | 18:12 |
| efried | aspiers: Want me to throw a procedural -2 on the bottom SEV patch so you don't have to keep chasing -Ws? | 18:14 |
| openstackgerrit | Merged openstack/nova master: Remove additional policy configuration details from policy doc https://review.openstack.org/644423 | 18:14 |
| openstackgerrit | Merged openstack/nova master: Clarify policy shortcomings in policy enforcement doc https://review.openstack.org/643960 | 18:14 |
| sean-k-mooney | aspiers: so based on https://github.com/libvirt/libvirt/blob/0ec6343a069b21178d4580688a8380dbb6d76620/docs/news-2013.html.in#L1526 | 18:14 |
| sean-k-mooney | aspiers: libvirt used to set RLIMIT_MEMLOCK when you set the locked value | 18:14 |
| aspiers | efried: that would be nice thanks, but depends on which you think is the bottom patch ;-) | 18:14 |
| efried | aspiers: https://review.openstack.org/#/c/633855/ ? | 18:15 |
| sean-k-mooney | aspiers: and it still does https://github.com/libvirt/libvirt/blob/b6aacfc435ce3b7e2665ddf5a422d2153bca88b8/src/util/virprocess.c#L749 | 18:15 |
| efried | aspiers: Done, lmk if there are any stragglers not in the series that you'd like similarly tagged. | 18:15 |
| aspiers | efried: yeah, I think 644554 could maybe merge pre-Stein without doing harm | 18:15 |
| aspiers | sean-k-mooney: reading | 18:16 |
| efried | agree | 18:16 |
| efried | aspiers: if you can get that efried guy to approve it. | 18:16 |
| aspiers | :) | 18:16 |
| aspiers | efried: sounds like a tall order | 18:16 |
| efried | aspiers: I almost did a full rewrite of that method, but I'll settle for nixing backslashes. | 18:17 |
| aspiers | hrm, nix them how? | 18:18 |
| efried | aspiers: Use parens, or use an if, or... | 18:18 |
| aspiers | OK | 18:19 |
| efried | it's just a style rule that nova likes to follow. I'm pretty sure there was a time I didn't care, but now I'm brainwashed. | 18:20 |
| aspiers | sure | 18:20 |
| *** wolverineav has joined #openstack-nova | 18:20 | |
| *** tstrul has quit IRC | 18:22 | |
| aspiers | sean-k-mooney: seems like a key question is whether mlock(2) really is enough for SEV or not | 18:22 |
| *** wolverineav has quit IRC | 18:25 | |
| aspiers | sean-k-mooney, bbobrov: I guess there's also #libvirt | 18:26 |
| sean-k-mooney | aspiers: i think its calling https://linux.die.net/man/2/setrlimit but yes let use know what they determin | 18:30 |
| *** amodi has quit IRC | 18:32 | |
| sean-k-mooney | but as i said before im pretry sure hard_limit is not actully locking the memory its just limiting how much memory can be locked by an unprivalaged process | 18:33 |
| aspiers | sean-k-mooney: yes, I think it needs the combination of setrlimit and mlock | 18:33 |
| sean-k-mooney | which https://github.com/AMDESE/AMDSEV/blob/master/xmls/sample.xml does not do :) | 18:35 |
| sean-k-mooney | anyway im going to grab something to eat | 18:35 |
| sean-k-mooney | be back in a bit | 18:35 |
| aspiers | OK, me too. Cheers for the help! | 18:35 |
| *** sridharg has quit IRC | 18:46 | |
| *** jmlowe has joined #openstack-nova | 18:47 | |
| *** IvensZambrano has quit IRC | 18:47 | |
| openstackgerrit | Lance Bragstad proposed openstack/nova master: Give the policy vision document a facelift https://review.openstack.org/644615 | 18:49 |
| *** wolverineav has joined #openstack-nova | 18:55 | |
| *** jmlowe has quit IRC | 18:58 | |
| *** diablo_rojo has quit IRC | 19:03 | |
| *** wolverineav has quit IRC | 19:04 | |
| *** awalende has joined #openstack-nova | 19:11 | |
| *** tstrul has joined #openstack-nova | 19:13 | |
| *** spbarbieri has joined #openstack-nova | 19:15 | |
| *** awalende has quit IRC | 19:16 | |
| *** jmlowe has joined #openstack-nova | 19:16 | |
| *** phasespace has joined #openstack-nova | 19:18 | |
| *** ttsiouts has joined #openstack-nova | 19:20 | |
| sean-k-mooney | /away until noon gmt | 19:30 |
| sean-k-mooney | extra space .... | 19:30 |
| *** tstrul has quit IRC | 19:31 | |
| *** wolverineav has joined #openstack-nova | 19:33 | |
| *** gouthamr_ has joined #openstack-nova | 19:34 | |
| *** gouthamr_ has quit IRC | 19:38 | |
| *** imacdonn has quit IRC | 19:39 | |
| *** imacdonn has joined #openstack-nova | 19:40 | |
| spbarbieri | I provision a server with a block_device that specifies a volume to be used as the boot volume. I'd like to use the REST API to determine if a block_device was used to boot the instance, and if so, which attached volume is the boot volume for the server. The "server details" api returns an empty "image" so I guess I can assume it used a volume as the boot device, but I don't know how to determine which attached volume is t | 19:45 |
| *** jmlowe has quit IRC | 19:47 | |
| *** mikal has joined #openstack-nova | 19:48 | |
| artom | spbarbieri, you can list volume attachments with https://developer.openstack.org/api-ref/compute/?expanded=list-volume-attachments-for-an-instance-detail,show-a-detail-of-a-volume-attachment-detail#list-volume-attachments-for-an-instance, but that wouldn't tell you whether one was used for boot | 19:50 |
| *** mikal has quit IRC | 19:53 | |
| spbarbieri | Thanks for the pointer. When creating the server I can specify "block_device_mapping_v2.delete_on_termination". This is saved somewhere because my volume is deleted on server termination. I was hoping that the "this is the boot device" was saved there as well. But, I couldn't find it in any of the server/volume/volume_attach apis. | 19:58 |
| *** ttsiouts has quit IRC | 20:01 | |
| *** cdent has quit IRC | 20:01 | |
| artom | spbarbieri, I *think* boot_index is what you're looking for, but it doesn't look like we expose it anywhere in the API | 20:05 |
| artom | spbarbieri, you can also tag with 'boot' as a workaround | 20:06 |
| *** jmlowe has joined #openstack-nova | 20:07 | |
| *** dustinc is now known as dustinc|afk | 20:09 | |
| *** tstrul has joined #openstack-nova | 20:10 | |
| *** itlinux has joined #openstack-nova | 20:13 | |
| spbarbieri | yeah, boot_index would be perfect. I couldn't find it anywhere either. I'm actually trying to do this after the fact. I have a running server and I want to know which volume was used as the boot drive. I only have access to the REST API...sounds like a MacGyver episode...find the boot drive using a paperclip and some duct tape | 20:14 |
| *** cdent has joined #openstack-nova | 20:19 | |
| *** artom has quit IRC | 20:20 | |
| openstackgerrit | Merged openstack/nova master: Use assertXmlEqual() helper for all XML comparison tests https://review.openstack.org/641852 | 20:23 |
| openstackgerrit | Merged openstack/nova stable/rocky: Allow utime call to fail on qcow2 image base file https://review.openstack.org/643011 | 20:23 |
| mriedem | spbarbieri: that's eventually going to show up in the API here https://review.openstack.org/#/c/623981/24 | 20:24 |
| mriedem | e.g. https://review.openstack.org/#/c/623981/24/doc/api_samples/os-volumes/v2.72/volume-attachment-detail-resp.json | 20:24 |
| *** diablo_rojo has joined #openstack-nova | 20:25 | |
| mriedem | if you're an admin, you could also see the OS-EXT-SRV-ATTR:root_device_name parameter in the response from the GET /servers/{server_id} API | 20:25 |
| mriedem | and match that to the volume with the same device name, although those aren't guaranteed to be where the root volume is mounted in the guest | 20:26 |
| *** cdent has quit IRC | 20:26 | |
| spbarbieri | Thanks for the pointer to the volume-attach details. That will be helpful. I'm running against my local devstack, as admin, and I don't get that returned. I'm calling compute/v2.1/servers/{id}. | 20:27 |
| *** tosky has quit IRC | 20:30 | |
| *** tstrul has quit IRC | 20:37 | |
| *** wolverineav has quit IRC | 20:43 | |
| *** hamzy has quit IRC | 20:46 | |
| *** tesseract has quit IRC | 20:52 | |
| *** wolverineav has joined #openstack-nova | 20:53 | |
| *** luksky has joined #openstack-nova | 20:54 | |
| *** ttsiouts has joined #openstack-nova | 20:54 | |
| melwitt | spbarbieri: are you using novaclient or openstackclient? if the latter, you need to pass the necessary compute API microversion to see the volume-attach info. i.e. --os-compute-api-version 2.3 (see https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#maximum-in-kilo) | 20:55 |
| melwitt | s/volume-attach info/root_device_name/ | 20:55 |
| spbarbieri | melwitt: I'm using Postman with direct REST API calls | 20:57 |
| melwitt | spbarbieri: OK, see this doc on how to specify compute API microversion with the HTTP header https://developer.openstack.org/api-guide/compute/microversions.html#client-interaction | 21:00 |
| spbarbieri | melwitt: Yes! Thanks. Adding that header works. I guess now I can match the root_device_name against all of the attached volumes to find a match. Does that seem reasonable? | 21:04 |
| *** amodi_ has joined #openstack-nova | 21:04 | |
| *** amodi_ is now known as amodi | 21:04 | |
| melwitt | spbarbieri: I think so, based on what mriedem said earlier | 21:06 |
| spbarbieri | melwitt: Thank you for the help. | 21:07 |
| melwitt | np, good luck | 21:07 |
| *** priteau has joined #openstack-nova | 21:07 | |
| *** eharney has quit IRC | 21:07 | |
| *** xek_ has quit IRC | 21:11 | |
| *** dustinc|afk is now known as dustinc | 21:21 | |
| *** BjoernT has quit IRC | 21:21 | |
| *** spbarbieri has quit IRC | 21:22 | |
| *** BjoernT has joined #openstack-nova | 21:23 | |
| *** cdent has joined #openstack-nova | 21:26 | |
| *** igordc has quit IRC | 21:31 | |
| *** cfriesen has joined #openstack-nova | 21:35 | |
| *** bryan_stephenson has quit IRC | 21:38 | |
| *** itlinux has quit IRC | 21:40 | |
| *** whoami-rajat has quit IRC | 21:52 | |
| *** diablo_rojo has quit IRC | 21:57 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add a prelude release note for the 19.0.0 Stein GA https://review.openstack.org/644412 | 21:58 |
| efried | melwitt: it's like VOIP in the '90s. | 22:05 |
| efried | or, you know, skype today. | 22:05 |
| melwitt | haha, yeah | 22:05 |
| *** pcaruana has quit IRC | 22:08 | |
| *** mvkr has joined #openstack-nova | 22:09 | |
| *** itlinux has joined #openstack-nova | 22:16 | |
| *** rcernin has joined #openstack-nova | 22:16 | |
| mriedem | melwitt: efried: cdent: dansmith: also dumped comments in the prelude https://review.openstack.org/#/c/644412/ about vgpu | 22:20 |
| mriedem | i'd either remove that or really dumb it down | 22:20 |
| melwitt | mriedem: thanks | 22:21 |
| efried | mriedem: The part I was concerned about: | 22:21 |
| efried | admins will sometimes list resource providers | 22:21 |
| mriedem | the vcenter driver also supports live migration, that's a long-standing thing that might be worth mentioning | 22:21 |
| efried | Now they're gonna see two instead of one for some hosts | 22:21 |
| efried | that's a user-facing manifestation of the nrp-for-vgpus thing. | 22:22 |
| efried | that's the only reason I think it's worth mentioning | 22:22 |
| melwitt | mriedem: ack | 22:22 |
| efried | otherwise I agree there would be no reason to talk about it. | 22:22 |
| efried | "We did this thing, so don't be surprised if you see two resource providers for a libvirt host with VGPUs." | 22:22 |
| melwitt | I see, yeah | 22:22 |
| mriedem | there is an upgrade release note for it right? so as long as we can really distill the prelude comment on that to something short and sweet | 22:23 |
| efried | sorry I've been entirely ineffectual phrasing that :( I've been trying to nail down the zillion UT failures that happen when you change how ksa is mocked. Different headspace. | 22:24 |
| melwitt | there is an upgrade note, but it doesn't really spell out what an admin would see as a change when they list RPs | 22:24 |
| mriedem | "Upon restarting nova-compute services using the libvirt driver which expose VGPU inventory, the inventory will be moved from the root compute node resource provider to a child resource provider. See the upgrade notes for details." | 22:24 |
| efried | wfm | 22:24 |
| melwitt | yeah, that sounds good | 22:24 |
| mriedem | how about, ""Upon restarting nova-compute services using the libvirt driver which expose VGPU inventory, the inventory will be moved from the root compute node resource provider to a child resource provider. As a result, when listing resource providers, you may see two for the same compute node where there was one before. See the upgrade notes for details." | 22:25 |
| efried | even better | 22:25 |
| melwitt | thumbs up | 22:25 |
| *** itlinux has quit IRC | 22:27 | |
| openstackgerrit | Adam Spiers proposed openstack/nova master: Move libvirt calculation of machine type to utils.py https://review.openstack.org/644554 | 22:29 |
| *** diablo_rojo has joined #openstack-nova | 22:30 | |
| *** slaweq has quit IRC | 22:32 | |
| *** wolverineav has quit IRC | 22:32 | |
| *** wolverineav has joined #openstack-nova | 22:33 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Address old TODO in claim_resources_on_destination https://review.openstack.org/644596 | 22:34 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add a prelude release note for the 19.0.0 Stein GA https://review.openstack.org/644412 | 22:40 |
| *** wolverineav has quit IRC | 22:40 | |
| *** wolverineav has joined #openstack-nova | 22:41 | |
| *** igordc has joined #openstack-nova | 22:42 | |
| *** wolverineav has quit IRC | 22:45 | |
| *** mikal has joined #openstack-nova | 22:47 | |
| *** ttsiouts has quit IRC | 22:51 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add known issue for minimum bandwidth resource leak https://review.openstack.org/644694 | 22:53 |
| *** priteau has quit IRC | 22:54 | |
| melwitt | mriedem: gibi recommended the above is not an RC blocker, but I thought it might help to add a known issue reno about it ^ | 22:55 |
| *** tkajinam has joined #openstack-nova | 22:57 | |
| *** wolverineav has joined #openstack-nova | 23:00 | |
| *** marst has quit IRC | 23:00 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Use openstacksdk for placement https://review.openstack.org/643664 | 23:04 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Use SDK instead of ironicclient for node.get https://review.openstack.org/642899 | 23:04 |
| efried | dustinc: Warning: update here ^ | 23:04 |
| dustinc | thanks, I'll check it | 23:04 |
| efried | you will need to refetch and put your changes onto the new commit | 23:04 |
| dustinc | was just working on sdk_wrapper.py | 23:04 |
| efried | easiest way if you haven't already committed locally would be to stash, review -d, and restore | 23:04 |
| efried | dustinc: But we don't need sdk_wrapper.py | 23:05 |
| efried | do we? | 23:05 |
| dustinc | I thought we do...maybe we need to sync up on CONF tomorrow...I thought we needed to read everything and pass it to the SDK? | 23:06 |
| efried | dustinc: Oh, that's done already :) | 23:06 |
| dustinc | ok, I will stash my work and check yours out then I am going offline shortly | 23:07 |
| dustinc | let's sync back up tomorrow | 23:07 |
| efried | dustinc: See the predecessor patch at https://review.openstack.org/#/c/643664/5/nova/utils.py@1239 | 23:07 |
| *** mriedem is now known as mriedem_away | 23:07 | |
| *** mchlumsky has quit IRC | 23:08 | |
| efried | dustinc: So the only change needed to have ironic make use of that was here: https://review.openstack.org/#/c/642899/16/nova/virt/ironic/driver.py@188 | 23:08 |
| efried | dustinc: And tbc, this works, which you can tell by looking at the CI results, if you know what you're looking for: | 23:08 |
| efried | The job called ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa only succeeds if ^ is copacetic | 23:08 |
| efried | Here's the last run (before the latest rebase): http://logs.openstack.org/99/642899/15/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/c032d49/ | 23:09 |
| efried | http://logs.openstack.org/99/642899/15/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/c032d49/testr_results.html.gz | 23:09 |
| efried | The other red stuff in the CI report is the unit and functional tests - for those, it's the actual tests that need to be fixed, not the code. | 23:09 |
| *** tesseract has joined #openstack-nova | 23:09 | |
| efried | and most of the test fixage needs to happen in the predecessor patch - which is what I've been working on. | 23:09 |
| efried | made a little progress today, but definitely didn't finish. | 23:10 |
| efried | a couple of serious mysteries in there still. | 23:10 |
| *** tesseract has quit IRC | 23:10 | |
| *** alex_xu has quit IRC | 23:14 | |
| *** yaawang has quit IRC | 23:17 | |
| *** yaawang has joined #openstack-nova | 23:18 | |
| *** luksky has quit IRC | 23:21 | |
| *** hongbin has quit IRC | 23:25 | |
| *** mikal has quit IRC | 23:26 | |
| *** mlavalle has quit IRC | 23:36 | |
| *** diablo_rojo has quit IRC | 23:46 | |
| *** penick has joined #openstack-nova | 23:56 | |
| *** diablo_rojo has joined #openstack-nova | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!