| *** wolverineav has quit IRC | 00:02 | |
| melwitt | gmann: I'll get it. thank you for all your work on this | 00:03 |
|---|---|---|
| gmann | thanks. | 00:04 |
| *** tetsuro has joined #openstack-nova | 00:08 | |
| *** slaweq has joined #openstack-nova | 00:11 | |
| fungi | thanks melwitt! | 00:13 |
| *** slaweq has quit IRC | 00:15 | |
| *** marst has joined #openstack-nova | 00:19 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | 00:19 |
| *** xek has joined #openstack-nova | 00:23 | |
| *** igordc has quit IRC | 00:24 | |
| *** igordc has joined #openstack-nova | 00:25 | |
| *** Swami has quit IRC | 00:26 | |
| *** wolverineav has joined #openstack-nova | 00:28 | |
| *** dklyle has quit IRC | 00:31 | |
| *** dklyle has joined #openstack-nova | 00:31 | |
| *** igordc has quit IRC | 00:31 | |
| *** wolverineav has quit IRC | 00:35 | |
| *** gyee has quit IRC | 00:39 | |
| *** marst has quit IRC | 00:42 | |
| *** takashin has joined #openstack-nova | 00:42 | |
| *** mikal has joined #openstack-nova | 00:43 | |
| *** dklyle has quit IRC | 00:46 | |
| *** mikal has quit IRC | 00:47 | |
| *** erlon has quit IRC | 00:54 | |
| *** igordc has joined #openstack-nova | 00:58 | |
| *** liuyulong has joined #openstack-nova | 01:10 | |
| *** slaweq has joined #openstack-nova | 01:11 | |
| *** mriedem_afk has quit IRC | 01:11 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test for bug 1819963 https://review.openstack.org/643151 | 01:12 |
| openstack | bug 1819963 in OpenStack Compute (nova) "Reverting a resize does not update the instance.availability_zone value to the source az" [Medium,In progress] https://launchpad.net/bugs/1819963 - Assigned to Matt Riedemann (mriedem) | 01:12 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Update instance.availability_zone on revertResize https://review.openstack.org/643155 | 01:12 |
| *** slaweq has quit IRC | 01:16 | |
| *** snevi has quit IRC | 01:33 | |
| *** zhubx has quit IRC | 01:51 | |
| *** zhubx007 has joined #openstack-nova | 01:51 | |
| *** whoami-rajat has joined #openstack-nova | 02:02 | |
| *** wolverineav has joined #openstack-nova | 02:10 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova stable/rocky: Fix resetting non-persistent fields when saving obj https://review.openstack.org/643215 | 02:14 |
| *** hongbin has joined #openstack-nova | 02:15 | |
| *** wolverineav has quit IRC | 02:15 | |
| openstackgerrit | huanhongda proposed openstack/nova stable/queens: Fix resource tracker updates during instance evacuation https://review.openstack.org/643219 | 02:31 |
| *** BjoernT has joined #openstack-nova | 02:40 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271 | 02:44 |
| *** BjoernT_ has joined #openstack-nova | 02:45 | |
| *** BjoernT has quit IRC | 02:48 | |
| *** dave-mccowan has joined #openstack-nova | 02:49 | |
| *** dave-mccowan has quit IRC | 02:54 | |
| *** hoonetorg has quit IRC | 02:59 | |
| *** psachin has joined #openstack-nova | 03:01 | |
| *** slaweq has joined #openstack-nova | 03:11 | |
| *** hoonetorg has joined #openstack-nova | 03:12 | |
| *** slaweq has quit IRC | 03:15 | |
| *** alex_xu has quit IRC | 03:32 | |
| *** hongbin has quit IRC | 03:36 | |
| *** alex_xu has joined #openstack-nova | 03:36 | |
| openstackgerrit | Merged openstack/nova master: Disable the tls-proxy in nova-next & fix nova-tox-functional-py35 parent https://review.openstack.org/643129 | 04:06 |
| *** slaweq has joined #openstack-nova | 04:11 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Override the 'get' method in DriverBlockDevice class https://review.openstack.org/638821 | 04:14 |
| *** slaweq has quit IRC | 04:15 | |
| *** udesale has joined #openstack-nova | 04:22 | |
| *** janki has joined #openstack-nova | 04:35 | |
| *** lpetrut has joined #openstack-nova | 04:55 | |
| *** lbragstad has quit IRC | 05:00 | |
| *** lpetrut has quit IRC | 05:19 | |
| *** cfriesen has quit IRC | 05:19 | |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: object: Add pmem_namespaces field to the NUMACell obj https://review.openstack.org/634547 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: object: Add virtual_pmems fields to the InstanceNUMACell obj https://review.openstack.org/634548 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Enable driver configures PMEM namespace when initiating libvirt driver https://review.openstack.org/634549 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: Update PMEM namespaces info and usage https://review.openstack.org/634550 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: report pmem namespaces resources by provider tree https://review.openstack.org/634551 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: API: parse pmem related flavor extra spec https://review.openstack.org/634552 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: scheduler: translate virtual pmems request to placement request group https://review.openstack.org/634553 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: update _numa_fit_instance_cell for the support of virtual_pmems https://review.openstack.org/634554 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: libvirt: boot VM with vpmems and cleanup vpmems when destroying https://review.openstack.org/634555 | 05:20 |
| openstackgerrit | Luyao Zhong proposed openstack/nova master: support VM resizing with vpmem data migration https://review.openstack.org/634556 | 05:20 |
| *** sridharg has joined #openstack-nova | 05:56 | |
| *** alex_xu has quit IRC | 06:06 | |
| *** alex_xu has joined #openstack-nova | 06:07 | |
| *** slaweq has joined #openstack-nova | 06:11 | |
| *** slaweq has quit IRC | 06:16 | |
| *** wolverineav has joined #openstack-nova | 06:18 | |
| *** slaweq has joined #openstack-nova | 06:19 | |
| *** igordc has quit IRC | 06:26 | |
| *** ivve has joined #openstack-nova | 06:37 | |
| *** wolverin_ has joined #openstack-nova | 06:42 | |
| *** wolverineav has quit IRC | 06:45 | |
| *** Luzi has joined #openstack-nova | 06:46 | |
| *** wolverin_ has quit IRC | 06:49 | |
| *** BjoernT has joined #openstack-nova | 07:00 | |
| *** BjoernT_ has quit IRC | 07:02 | |
| *** lpetrut has joined #openstack-nova | 07:07 | |
| *** markvoelker has quit IRC | 07:19 | |
| *** dpawlik has joined #openstack-nova | 07:22 | |
| *** tosky has joined #openstack-nova | 07:23 | |
| *** mikal has joined #openstack-nova | 07:25 | |
| *** wolverineav has joined #openstack-nova | 07:27 | |
| *** dpawlik has quit IRC | 07:27 | |
| *** dpawlik has joined #openstack-nova | 07:29 | |
| *** mikal has quit IRC | 07:30 | |
| *** wolverineav has quit IRC | 07:31 | |
| *** lpetrut has quit IRC | 07:35 | |
| openstackgerrit | zhaixiaojun proposed openstack/nova master: Fix bug: non-admin user can not filter instance by instance-uuid https://review.openstack.org/642403 | 07:35 |
| *** lpetrut has joined #openstack-nova | 07:42 | |
| *** yaawang has quit IRC | 07:45 | |
| *** yaawang has joined #openstack-nova | 07:47 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271 | 07:47 |
| *** luksky has joined #openstack-nova | 07:54 | |
| *** takashin has left #openstack-nova | 08:01 | |
| *** tesseract has joined #openstack-nova | 08:07 | |
| *** pcaruana has joined #openstack-nova | 08:07 | |
| *** awalende has joined #openstack-nova | 08:08 | |
| *** dpawlik has quit IRC | 08:09 | |
| *** dpawlik has joined #openstack-nova | 08:10 | |
| *** pcaruana has quit IRC | 08:11 | |
| *** tssurya has joined #openstack-nova | 08:12 | |
| *** tkajinam__ has quit IRC | 08:13 | |
| *** ccamacho has joined #openstack-nova | 08:20 | |
| *** BjoernT has quit IRC | 08:23 | |
| *** pcaruana has joined #openstack-nova | 08:23 | |
| *** hoonetorg has quit IRC | 08:32 | |
| openstackgerrit | Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765 | 08:34 |
| *** ttsiouts has joined #openstack-nova | 08:37 | |
| *** helenafm has joined #openstack-nova | 08:38 | |
| openstackgerrit | Merged openstack/nova master: Update compute rpc version alias for stein https://review.openstack.org/642599 | 08:44 |
| *** hoonetorg has joined #openstack-nova | 08:45 | |
| *** slaweq has quit IRC | 08:53 | |
| *** slaweq has joined #openstack-nova | 08:55 | |
| *** dtantsur|afk is now known as dtantsur | 08:59 | |
| *** kashyap` has joined #openstack-nova | 09:02 | |
| *** kashyap` is now known as kashyap | 09:04 | |
| openstackgerrit | Merged openstack/nova master: Migrate legacy jobs to Ubuntu Bionic https://review.openstack.org/639017 | 09:04 |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Remove unsafe convert of lvm imagebackend https://review.openstack.org/642667 | 09:05 |
| *** panda|rover|off has quit IRC | 09:07 | |
| kashyap | gibi: Morning, mind having a quick gander at this (simple change, but brings non-trivial perf benefit): https://review.openstack.org/#/c/640781/ | 09:08 |
| openstackgerrit | zhaixiaojun proposed openstack/nova master: Fix bug: non-admin user can not filter instance by instance-uuid https://review.openstack.org/642403 | 09:12 |
| *** IvensZambrano has joined #openstack-nova | 09:23 | |
| *** IvensZambrano has quit IRC | 09:24 | |
| *** mikal has joined #openstack-nova | 09:41 | |
| *** mikal has quit IRC | 09:46 | |
| *** pcaruana has quit IRC | 09:48 | |
| *** cdent has joined #openstack-nova | 10:11 | |
| *** rpittau|afk is now known as rpittau | 10:13 | |
| *** sapd1 has joined #openstack-nova | 10:34 | |
| *** tbachman has quit IRC | 10:46 | |
| jangutter | kashyap: 15x --- there's that story that nearly every programming task reduces to either a caching or string manipulation problem. | 10:49 |
| *** wolverineav has joined #openstack-nova | 10:49 | |
| kashyap | jangutter: Yeah... | 10:49 |
| kashyap | It was a "one-worder" change, but I guess you read the full commit message | 10:50 |
| kashyap | It took 2 days to dig into QEMU and talk to Block Layer folks | 10:50 |
| jangutter | and the persistent nagging feeling that it can't be this simple. | 10:51 |
| *** wolverineav has quit IRC | 10:54 | |
| *** liuyulong has quit IRC | 10:54 | |
| *** udesale has quit IRC | 10:56 | |
| *** udesale has joined #openstack-nova | 10:56 | |
| *** mdbooth has joined #openstack-nova | 11:14 | |
| *** udesale has quit IRC | 11:15 | |
| *** udesale has joined #openstack-nova | 11:15 | |
| *** zbr has quit IRC | 11:16 | |
| kashyap | jangutter: Yeah, need to consider impact on multiple use cases | 11:22 |
| kashyap | mdbooth: Morning, if you get a moment: https://review.openstack.org/#/c/641981/ and https://review.openstack.org/#/c/640781/ | 11:23 |
| kashyap | (You were already +1 on those before.) | 11:23 |
| mdbooth | "But that's not correct, because of our misunderstanding of how cache modes work." | 11:24 |
| mdbooth | I'll say again: we never understood how cache modes work, only what they're called ;) | 11:24 |
| mdbooth | ...misunderstood... | 11:24 |
| mdbooth | Don't change that, though. | 11:24 |
| kashyap | Hehe, noted. | 11:25 |
| kashyap | Now I think not in terms of just named cache modes, but the three booleans each one impacts | 11:25 |
| kashyap | mdbooth: FWIW, the little blob of text written in driver.py -- I got it reviewed by Kevin (of QEMU) over a few emails. So it's not me just riffing on it :-) | 11:27 |
| *** udesale has quit IRC | 11:27 | |
| *** udesale has joined #openstack-nova | 11:28 | |
| *** derekh has joined #openstack-nova | 11:29 | |
| *** pcaruana has joined #openstack-nova | 11:32 | |
| mdbooth | ack | 11:37 |
| *** ttsiouts has quit IRC | 11:44 | |
| *** dave-mccowan has joined #openstack-nova | 11:45 | |
| *** luksky has quit IRC | 11:50 | |
| openstackgerrit | Fan Zhang proposed openstack/nova master: Retry after hitting libvirt error VIR_ERR_OPERATION_INVALID in live migration. https://review.openstack.org/612272 | 11:52 |
| *** gcerami_ has joined #openstack-nova | 11:57 | |
| *** zbr has joined #openstack-nova | 11:59 | |
| *** tbachman_ has joined #openstack-nova | 12:09 | |
| *** tbachman_ is now known as tbachman | 12:10 | |
| awalende | I'am trying to use hugepages with nova and qemu/libvirt. Hugepages is enabled and I have reserved a bunch of hugepages in standard 2048kb size. I added flavor properties with "flavor set hugePagesTest --property hw:mem_page_size=2Mb". However starting instances fails with "Exceeded maximum number of retries". Nova-compute log does not show any exceptions. Any Idea how to debug this? | 12:10 |
| *** ttsiouts has joined #openstack-nova | 12:17 | |
| *** luksky has joined #openstack-nova | 12:18 | |
| *** helenafm has quit IRC | 12:30 | |
| *** tetsuro has quit IRC | 12:32 | |
| *** eharney has quit IRC | 12:32 | |
| efried | awalende: Have you looked at the scheduler log? | 12:34 |
| awalende | Yes, I could narrow it down. It has something to do with NUMA-Topology: | 12:36 |
| awalende | Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology. | 12:36 |
| awalende | Which is weird, I have setup hw:numa_nodes=4 in the flavor, since my Intel Xeon has 4 Nodes. | 12:36 |
| awalende | beforeheand there is a bunch of repeating logs like: Attempting to fit instance cell InstanceNUMACell(cpu_pinning_raw=None,cpu_policy=None,cpu_thread_policy=None,cpu_topology=<?>,cpuset=set([0,1,2,3,4,5,6,7,8,9,10,11,12,13]),cpuset_reserved=None,id=0,memory=512,pagesize=256) on host_cell NUMACell(cpu_usage=0,cpuset=set([3,51,35,7,15,11,47,43,19,23,39,27,55,31]),id=3,memory=193532,memory_usage=0,mempages=[NUMAP | 12:37 |
| awalende | agesTopology,NUMAPagesTopology,NUMAPagesTopology],pinned_cpus=set([]),siblings=[]) | 12:37 |
| *** gcerami_ is now known as panda | 12:40 | |
| efried | mdbooth: are you a NUMA guy? | 12:43 |
| efried | awalende: (looking for someone who can spell NUMA) | 12:43 |
| awalende | :D | 12:43 |
| *** zhubx007 has quit IRC | 12:44 | |
| *** zhubx007 has joined #openstack-nova | 12:45 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | 12:46 |
| openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: WIP: Use ubuntu-xenial for sphinx jobs https://review.openstack.org/643320 | 12:55 |
| mdbooth | efried: I am not, unfortunately. | 12:57 |
| efried | kashyap: what about you? :) | 12:58 |
| mdbooth | efried: I can spell it, though. Our 2 best numa guys are travelling today. | 12:58 |
| efried | okay. | 12:58 |
| efried | eventually here I'm gonna need to learn something about it... | 12:58 |
| kashyap | efried: Hi, I have a NUMA-capable machine, and occasionally do testing of NUMA+SR-IOV stuff; besides that I am not an expert | 12:59 |
| * kashyap --> goes to cook late lunch | 12:59 | |
| *** breton has quit IRC | 12:59 | |
| kashyap | efried: What's the question, it might make curious enough to learn the answer :-) | 12:59 |
| *** jchhatbar has joined #openstack-nova | 13:00 | |
| efried | kashyap: awalende is having trouble scheduling an instance requesting 4 numa nodes and hugepages, see logs starting around :10 | 13:00 |
| *** janki has quit IRC | 13:00 | |
| efried | I'm guessing it's a simple thing - for somebody with a clue how NUMA-related scheduling works | 13:00 |
| efried | http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-03-14.log.html#t2019-03-14T12:10:17 | 13:01 |
| *** dklyle has joined #openstack-nova | 13:01 | |
| *** awaugama has joined #openstack-nova | 13:02 | |
| kashyap | Hmm; the error "Requested instance NUMA topology cannot fit the given host NUMA topology" is indeed weird *if* you have told the scheduler to use more than one NUMA node | 13:04 |
| kashyap | ... which you did. | 13:04 |
| * kashyap --> kitchen; bbiab | 13:05 | |
| mdbooth | kashyap: Did you rebase that writeback patch for some reason? | 13:06 |
| *** dpawlik has quit IRC | 13:07 | |
| mdbooth | Incidentally, ^^^ is a super-obvious change in light of the informative comment kashyap has added to disk_cachemode() | 13:08 |
| *** mriedem has joined #openstack-nova | 13:08 | |
| mdbooth | Erm... https://review.openstack.org/#/c/641981/ | 13:08 |
| kashyap | mdbooth: Had to rebase because "add the compress flag" thingie to 'qemu-img' got merged (you'll recall it, as you reviewed it) | 13:09 |
| mdbooth | kashyap: Ah, ok. | 13:09 |
| *** jchhatbar has quit IRC | 13:10 | |
| *** janki has joined #openstack-nova | 13:10 | |
| kashyap | (It caused a conflict, so I was forced to fix it and rebase :-)) | 13:12 |
| * kashyap back to kitchen | 13:12 | |
| kashyap | mdbooth: What about also doing: s/writethrough/writeback/ for SMBFS and virtuozzo? | 13:15 |
| kashyap | See: nova/virt/libvirt/volume/smbfs.py and nova/virt/libvirt/volume/vzstorage.py. | 13:15 |
| mdbooth | kashyap: We should do that. | 13:16 |
| mdbooth | kashyap: Separate patches though, please. | 13:16 |
| kashyap | Yes! | 13:16 |
| kashyap | That's what I noted in the TODO | 13:16 |
| mdbooth | Also super-low priority, because likely nobody cares. | 13:16 |
| kashyap | (In the commit) | 13:16 |
| kashyap | [quote] | 13:16 |
| kashyap | TODO: Investigate if this is okay for Cinder volume drivers such as SMBFS and Virtuozzo. If so, then change them to use 'writeback' as well (probably in separate patches). | 13:16 |
| kashyap | [/quote] | 13:16 |
| kashyap | mdbooth: Yes, nobody cares, except when it comes to bite you. | 13:16 |
| kashyap | Might as well do it when I remember it; or else it goes into /dev/null of my brain | 13:17 |
| mdbooth | kashyap: It would only bite you if you used those things. | 13:17 |
| kashyap | (Yep.) | 13:17 |
| *** eharney has joined #openstack-nova | 13:24 | |
| *** lbragstad has joined #openstack-nova | 13:26 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible https://review.openstack.org/640271 | 13:28 |
| *** marst has joined #openstack-nova | 13:31 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Remove unsafe convert of lvm imagebackend https://review.openstack.org/642667 | 13:34 |
| *** erlon has joined #openstack-nova | 13:36 | |
| zhubx007 | mdbooth: looking forward to your review on two patches ^ thanks :) | 13:37 |
| mriedem | dansmith: can i get your +2 back on https://review.openstack.org/#/c/643151/ - i had to fix a test otherwise it's the same | 13:45 |
| dansmith | mriedem: well, the test failed in a weird way in functional, | 13:45 |
| dansmith | so I was waiting to see how the recheck turned out | 13:45 |
| *** marst has quit IRC | 13:45 | |
| mriedem | that's the infra changes | 13:45 |
| mriedem | but ok | 13:45 |
| mriedem | this https://review.openstack.org/#/c/643129/ | 13:45 |
| dansmith | I looked in the log and couldn't see anything actually broken | 13:45 |
| mriedem | it was this https://review.openstack.org/#/c/643129/4/.zuul.yaml | 13:46 |
| mriedem | broke everything over night | 13:46 |
| dansmith | okay... and the result is it just doesn't even run? | 13:46 |
| mriedem | the old func py35 job couldn't find the py35 interpreter | 13:46 |
| mriedem | with the new job parent it should be setup for that | 13:46 |
| mriedem | uses a xenial rather than bionic node | 13:46 |
| dansmith | okay I didn't see anything about that in the log, but that's fine, just sayin' I was looking at it already this morning and that's why I didn't just slap back on there | 13:47 |
| dansmith | but if it's known | 13:47 |
| dansmith | .. | 13:47 |
| mriedem | thanks | 13:47 |
| mriedem | frickler: looks like you fixed that ceph thing https://review.openstack.org/#/c/643150/ | 13:52 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 13:53 |
| *** janki has quit IRC | 13:58 | |
| *** marst has joined #openstack-nova | 14:10 | |
| *** erlon has quit IRC | 14:10 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | 14:10 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Customize irrelevant-files for nova-live-migration job https://review.openstack.org/643346 | 14:10 |
| *** helenafm has joined #openstack-nova | 14:16 | |
| *** rcernin has quit IRC | 14:17 | |
| openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: Use ubuntu-xenial for sphinx jobs https://review.openstack.org/643320 | 14:18 |
| *** mlavalle has joined #openstack-nova | 14:19 | |
| *** _pewp_ has quit IRC | 14:30 | |
| *** zhubx007 has quit IRC | 14:30 | |
| *** zhubx007 has joined #openstack-nova | 14:30 | |
| *** _pewp_ has joined #openstack-nova | 14:30 | |
| *** lpetrut has quit IRC | 14:38 | |
| *** tbachman has quit IRC | 14:38 | |
| *** lpetrut has joined #openstack-nova | 14:39 | |
| *** tbachman has joined #openstack-nova | 14:39 | |
| dansmith | is jaypipes just never around anymore? | 14:40 |
| cdent | dansmith: I _think_ he's travelling this week. Not sure if it was business or pleasure. | 14:46 |
| dansmith | oh okay | 14:46 |
| cdent | he was at least on a plane on monday | 14:47 |
| *** yaawang has quit IRC | 14:47 | |
| *** Swami has joined #openstack-nova | 14:47 | |
| *** mlavalle has quit IRC | 14:48 | |
| *** yaawang has joined #openstack-nova | 14:48 | |
| *** wolverineav has joined #openstack-nova | 14:50 | |
| *** ttsiouts has quit IRC | 14:50 | |
| *** ttsiouts has joined #openstack-nova | 14:51 | |
| *** mordred has quit IRC | 14:51 | |
| *** ttsiouts has quit IRC | 14:51 | |
| *** ttsiouts has joined #openstack-nova | 14:52 | |
| *** burt has joined #openstack-nova | 14:54 | |
| *** wolverineav has quit IRC | 14:54 | |
| *** mordred has joined #openstack-nova | 14:57 | |
| *** dpawlik has joined #openstack-nova | 14:58 | |
| *** awalende has quit IRC | 14:59 | |
| *** awalende has joined #openstack-nova | 14:59 | |
| *** BjoernT has joined #openstack-nova | 15:02 | |
| *** cfriesen has joined #openstack-nova | 15:03 | |
| *** awalende has quit IRC | 15:04 | |
| *** awalende has joined #openstack-nova | 15:04 | |
| *** awalende has quit IRC | 15:09 | |
| *** lpetrut has quit IRC | 15:12 | |
| *** ccamacho has quit IRC | 15:16 | |
| *** mordred has quit IRC | 15:23 | |
| *** mriedem is now known as mriedem_away | 15:24 | |
| *** zhubx007 has quit IRC | 15:24 | |
| *** dpawlik has quit IRC | 15:25 | |
| *** mordred has joined #openstack-nova | 15:29 | |
| *** mikal has joined #openstack-nova | 15:32 | |
| *** agopi has quit IRC | 15:34 | |
| *** ccamacho has joined #openstack-nova | 15:36 | |
| *** mlavalle has joined #openstack-nova | 15:36 | |
| *** mikal has quit IRC | 15:37 | |
| *** awaugama has quit IRC | 15:39 | |
| kashyap | aspiers: When you're about, refer to comment#11 here: https://review.openstack.org/#/c/633855/11 | 15:41 |
| *** hamzy has quit IRC | 15:43 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: qemu: Make disk image conversion dramatically faster https://review.openstack.org/640781 | 15:47 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Use 'writeback' QEMU cache mode for Nova instances https://review.openstack.org/641981 | 15:47 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: vzstorage: Use 'writeback' QEMU cache mode https://review.openstack.org/643376 | 15:47 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: smbfs: Use 'writeback' QEMU cache mode https://review.openstack.org/643377 | 15:47 |
| *** awaugama has joined #openstack-nova | 15:48 | |
| cdent | I | 15:54 |
| *** zhubx has joined #openstack-nova | 15:54 | |
| *** sapd1 has quit IRC | 15:55 | |
| edleafe | cdent: Roman numerals, anyone? | 15:57 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: vzstorage: Use 'writeback' QEMU cache mode https://review.openstack.org/643376 | 16:00 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: smbfs: Use 'writeback' QEMU cache mode https://review.openstack.org/643377 | 16:00 |
| kashyap | Some simple catch above, if anyone wants a change of scenery. | 16:00 |
| kashyap | (The core change and documentation is in: libvirt: Use 'writeback' QEMU cache mode for Nova instances https://review.openstack.org/641981) | 16:00 |
| cdent | edleafe: α ω | 16:02 |
| *** jangutter has quit IRC | 16:02 | |
| edleafe | cdent: make up your mind: greek or roman?? | 16:02 |
| *** sridharg has quit IRC | 16:03 | |
| *** jangutter has joined #openstack-nova | 16:03 | |
| *** mlavalle has quit IRC | 16:03 | |
| *** Luzi has quit IRC | 16:03 | |
| *** jding1_ has quit IRC | 16:03 | |
| *** jackding has joined #openstack-nova | 16:04 | |
| cfriesen | kashyap: quick question. did you try running a "sync" after the writeback image conversion test to ensure the data had actually hit the disk? | 16:04 |
| cfriesen | or do we even care if it did or not? | 16:05 |
| kashyap | cfriesen: No, I didn't do. But, 'sync' is not required at all in this case -- why? | 16:05 |
| cfriesen | well, your numbers look a lot like it just wrote to the cache. If the host crashes at that point, it might think it has valid data on-disk but actually have a corrupt image. | 16:06 |
| *** jmlowe has quit IRC | 16:07 | |
| *** dpawlik has joined #openstack-nova | 16:09 | |
| cfriesen | now this may not actually be a problem in practice, just thought I'd raise it. | 16:09 |
| kashyap | cfriesen: I got dropped off my IRC proxy; what was the last message you got from me? | 16:11 |
| kashyap | (And if you wrote anything else after your "quick question ...", please re-post) | 16:11 |
| cfriesen | I just said... your numbers look a lot like it just wrote to the cache. If the host crashes at that point, it might think it has valid data on-disk but actually have a corrupt image. Now this might not be a problem in practice, but I thought I'd mention it. | 16:12 |
| *** dpawlik has quit IRC | 16:13 | |
| kashyap | cfriesen: Noted, but ... I spent careful 4 days talking to QEMU Block Layer maint folks. | 16:14 |
| kashyap | And not sure if my messages got through: | 16:14 |
| kashyap | 17:05 < kashyap> cfriesen: No, I didn't do. But, 'sync' is not required at all in this case -- why? | 16:14 |
| *** agopi has joined #openstack-nova | 16:14 | |
| kashyap | 17:06 < kashyap> cfriesen: After image conversion completes successfully, QEMU issues fsync(), thus flushing the content down to disk. | 16:14 |
| kashyap | cfriesen: So, the numbers are correct, FWIW. The data _is_ on disk. | 16:14 |
| kashyap | cfriesen: I even audited QEMU's code (with call trac) to track down 'qemu-img convert' ends up calling fsync(). | 16:15 |
| kashyap | cfriesen: There is the call trace (since I sometimes see you pop into #qemu on OFTC :-)) -- https://kashyapc.fedorapeople.org/qemu-img-call-trace-to-fsync.txt | 16:15 |
| kashyap | dansmith: Hehe, saw your "church retreat" remark on the channel archives (my IRC proxy kicked me, thanks to the IDENTIFY) | 16:16 |
| kashyap | (I commented on that weshay asked) | 16:16 |
| kashyap | s/on that/on that review/ | 16:16 |
| cfriesen | kashyap: I just did an strace of qemu-img convert, and I see it calling fdatasync() rather than fsync(). But that should still be sufficient. | 16:20 |
| kashyap | Yes, if fdatasync() is available it is used, otherwise its cousin fsync() | 16:21 |
| kashyap | See the QEMU function call trace I posted earlier :-) | 16:21 |
| kashyap | cfriesen: Shall I respin for the s/multiple orders/multiple times/? | 16:21 |
| * kashyap has OCD (the positive kind) to get "all sentences right" :D | 16:22 | |
| cfriesen | I did make it a nit. :) | 16:23 |
| kashyap | cfriesen: Saw that; that's what I was referring to | 16:24 |
| cfriesen | I mean, if I really thought it required a respin I'd have done -1 | 16:25 |
| cfriesen | but if you're respinning for any other reason you could fix that up too | 16:25 |
| *** ivve has quit IRC | 16:25 | |
| cfriesen | do we need four separate commits for this? I mean, if we make the change one place it makes sense to make it everywhere, right? | 16:27 |
| *** jangutter_ has joined #openstack-nova | 16:27 | |
| cfriesen | I've gotten flack for too-big patches though, so maybe my opinion is biased. :) | 16:28 |
| *** jangutter has quit IRC | 16:30 | |
| kashyap | cfriesen: See the discussion in the scrollback here in the channel with mdbooth (who asked for separte patches) | 16:31 |
| kashyap | Because: each one is touching a completely different driver (SMBFS vs. Virtuozzo, etc) | 16:32 |
| *** tbachman has quit IRC | 16:32 | |
| kashyap | (It is the "one logical change per patch" rule.) | 16:32 |
| *** tbachman has joined #openstack-nova | 16:33 | |
| mdbooth | cfriesen: Just MHO, but the potential for fallout and therefore reverts seems slightly different in each case. | 16:33 |
| cfriesen | okay, works for me. | 16:33 |
| mdbooth | But they all definitely have the same theme. | 16:33 |
| mdbooth | cfriesen: And we can't test virtuozzo | 16:34 |
| kashyap | Exactly, the "potential for reverts" is the thing | 16:34 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: qemu: Make disk image conversion dramatically faster https://review.openstack.org/640781 | 16:35 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Use 'writeback' QEMU cache mode for Nova instances https://review.openstack.org/641981 | 16:35 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: vzstorage: Use 'writeback' QEMU cache mode https://review.openstack.org/643376 | 16:35 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: smbfs: Use 'writeback' QEMU cache mode https://review.openstack.org/643377 | 16:35 |
| kashyap | cfriesen: ^ Mind re-acking? | 16:35 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 16:36 |
| cfriesen | I suspect the biggest impact will be I/O usage patterns on the host, since now we'll be queueing up a bunch of writes and flushing them all at once. | 16:37 |
| cfriesen | kashyap: done | 16:37 |
| *** gibi has quit IRC | 16:38 | |
| *** gibi has joined #openstack-nova | 16:38 | |
| kashyap | cfriesen: Thank you | 16:38 |
| *** agopi has left #openstack-nova | 16:40 | |
| *** luksky has quit IRC | 16:41 | |
| *** jangutter_ is now known as jangutter | 16:43 | |
| *** Swami has quit IRC | 16:45 | |
| *** tssurya has quit IRC | 16:45 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 16:45 |
| *** dpawlik has joined #openstack-nova | 16:48 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 16:50 |
| *** dpawlik has quit IRC | 16:52 | |
| *** mriedem_away is now known as mriedem | 16:54 | |
| *** tbachman has quit IRC | 16:55 | |
| *** jaosorior has quit IRC | 16:57 | |
| *** jaosorior has joined #openstack-nova | 16:58 | |
| *** dpawlik has joined #openstack-nova | 16:59 | |
| *** hongbin has joined #openstack-nova | 16:59 | |
| *** wolverineav has joined #openstack-nova | 16:59 | |
| *** wolverineav has quit IRC | 17:02 | |
| *** wolverin_ has joined #openstack-nova | 17:02 | |
| *** wolverin_ has quit IRC | 17:02 | |
| *** wolverineav has joined #openstack-nova | 17:02 | |
| *** dpawlik has quit IRC | 17:03 | |
| jackding | Could cores look at https://review.openstack.org/#/c/621646/? It's been too long. | 17:03 |
| *** dakshina-ilangov has joined #openstack-nova | 17:05 | |
| *** rpittau is now known as rpittau|afk | 17:05 | |
| *** IvensZambrano has joined #openstack-nova | 17:05 | |
| *** udesale has quit IRC | 17:06 | |
| *** gyee has joined #openstack-nova | 17:07 | |
| *** tesseract has quit IRC | 17:08 | |
| *** hamzy has joined #openstack-nova | 17:09 | |
| *** dpawlik has joined #openstack-nova | 17:11 | |
| *** tbachman has joined #openstack-nova | 17:14 | |
| *** dpawlik has quit IRC | 17:15 | |
| kashyap | cfriesen: Fancy having a gander at: https://review.openstack.org/#/c/631154/ (WIP: Gracefully handle QEMU machine types for guests) | 17:18 |
| *** dtantsur is now known as dtantsur|afk | 17:18 | |
| *** psachin has quit IRC | 17:21 | |
| dansmith | mriedem: who was asking about the infinite recursion ssl thing yesterday? | 17:27 |
| dansmith | oh EmilienM | 17:28 |
| dansmith | https://review.openstack.org/#/c/626952/ | 17:28 |
| mdbooth | dansmith: Yeah, so I was previously going on faith, but it didn't appear to fix the test. | 17:29 |
| mdbooth | But it *did* just fix the reproducer system I had direct access to. | 17:29 |
| dansmith | hrm, this looks weird, I'll have to do some poking | 17:30 |
| mdbooth | It's very weird. If you look in the BZ I have a minimal non-nova python reproducer. | 17:30 |
| mdbooth | Based on my understanding of the eventlet bug, the issue is that we need to do monkey patching really early before importing most things. | 17:31 |
| mdbooth | ...as some code runs at import time. | 17:31 |
| dansmith | yeah | 17:31 |
| mdbooth | So that was the goal of my patch. | 17:31 |
| cfriesen | kashyap: I think the blueprint name is supposed to match the spec filename | 17:31 |
| mdbooth | Have one thing that you import first that does the monkey patching without importing anything else. | 17:31 |
| *** cdent has quit IRC | 17:32 | |
| mdbooth | OSP15 bz is here: https://bugzilla.redhat.com/show_bug.cgi?id=1686817 | 17:32 |
| openstack | bugzilla.redhat.com bug 1686817 in openstack-nova "[osp15] nova is broken on the undercloud with ssl" [Urgent,Assigned] - Assigned to mbooth | 17:32 |
| dansmith | mdbooth: if you're patching now before importing oslo.context, I imagine that reload is no longer necessary | 17:33 |
| kashyap | cfriesen: Yeah, noticed it. Can I still update the BP URL? | 17:33 |
| kashyap | cfriesen: I wrote that BP while still the discussions were still hot on QEMU and libvirt lists on the default | 17:33 |
| cfriesen | kashyap: no idea | 17:34 |
| kashyap | cfriesen: Where they were on the verge to change the default to 'q35', but backed out due to "maybe we're breaking backwards compat" | 17:34 |
| * kashyap tries | 17:34 | |
| *** panda is now known as panda|off | 17:34 | |
| mdbooth | dansmith: Quite possibly, but I just moved it. | 17:34 |
| dansmith | yeah I know | 17:34 |
| dansmith | just answering your TODO | 17:34 |
| dansmith | er, note | 17:35 |
| kashyap | cfriesen: Yep, I could change it. | 17:35 |
| dansmith | yikun: FYI https://review.openstack.org/#/c/592285/ | 17:37 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova-specs master: WIP: Gracefully handle QEMU machine types for guests https://review.openstack.org/631154 | 17:37 |
| *** derekh has quit IRC | 17:38 | |
| dansmith | mdbooth: why are there two identical LP bugs? | 17:39 |
| dansmith | or am I confusing something? | 17:39 |
| kashyap | cfriesen: Fixed. | 17:40 |
| mdbooth | dansmith: Unclear. | 17:40 |
| mdbooth | dansmith: Is the other one a tripleo tracker? | 17:41 |
| dansmith | shouldn't be necessary with LP | 17:41 |
| dansmith | mdbooth: so, the cmd/__init__.py one.. does that get run by the entrypoint before anything in something like cmd/compute ? because at first glance, it seems to not move it for those commands, if it comes in the first time they import cmd.common | 17:42 |
| dansmith | I guess it must.. | 17:42 |
| mdbooth | dansmith: Yes, it does. The BZ is against nova compute in the undercloud, which isn't wsgi. | 17:42 |
| dansmith | right, okay so this ends up actually running sooner in the compute case because we've done the patching before we import utils, which imported oslo stuff? | 17:43 |
| mdbooth | Yes. | 17:44 |
| dansmith | because you changed things to run at import time, it's maybe a little more obscure, but really the fix here is detaching the monkeypatch from the utils module (or anything else that has a bunch of stuff in it) | 17:44 |
| *** igordc has joined #openstack-nova | 17:45 | |
| *** priteau has joined #openstack-nova | 17:45 | |
| mdbooth | Yeah. I don't think there's any way to avoid running anything at import time, tbh, because that's the fundamental underlying issue | 17:45 |
| mdbooth | you need to run monkey_patch() before importing stuff | 17:45 |
| *** cdent has joined #openstack-nova | 17:46 | |
| dansmith | no, I know.. I get that part | 17:46 |
| mdbooth | Yes. The purpose of the monkey_patch module is to do monkey patching with minimal dependencies. | 17:47 |
| *** dpawlik has joined #openstack-nova | 17:51 | |
| *** eharney has quit IRC | 17:54 | |
| *** dpawlik has quit IRC | 17:55 | |
| mdbooth | dansmith: Thanks | 17:55 |
| dansmith | aye | 17:55 |
| *** ttsiouts has quit IRC | 17:55 | |
| *** ttsiouts has joined #openstack-nova | 17:56 | |
| kashyap | dansmith: If you want some fresh change of scenery, a simple catch: https://review.openstack.org/#/q/topic:bug/1818847+(status:open+OR+status:merged) | 17:56 |
| dansmith | I really don't | 17:57 |
| kashyap | Hehe | 17:57 |
| kashyap | dansmith: It brings 15 times faster performance for image conversion :D -- https://review.openstack.org/#/c/640781/ | 17:58 |
| kashyap | I ran it through the QEMU Block Layer maintainer, who also reviewed the patches over e-mail. And gave their: "Looks-good-to-me'd-by", FWIW | 17:58 |
| *** ttsiouts has quit IRC | 18:00 | |
| *** panda|off has quit IRC | 18:01 | |
| *** panda has joined #openstack-nova | 18:04 | |
| *** awaugama has quit IRC | 18:13 | |
| *** wolverineav has quit IRC | 18:16 | |
| *** ivve has joined #openstack-nova | 18:17 | |
| *** wolverineav has joined #openstack-nova | 18:19 | |
| *** wolverineav has quit IRC | 18:21 | |
| *** wolverineav has joined #openstack-nova | 18:22 | |
| *** awaugama has joined #openstack-nova | 18:26 | |
| *** wolverineav has quit IRC | 18:28 | |
| *** wolverineav has joined #openstack-nova | 18:29 | |
| *** IvensZambrano has quit IRC | 18:34 | |
| *** eharney has joined #openstack-nova | 18:43 | |
| *** awaugama has quit IRC | 18:43 | |
| mriedem | someone want to approve these for gate goodness? https://review.openstack.org/#/q/topic:bug/1819944+(status:open+OR+status:merged)+project:openstack/nova | 18:46 |
| mriedem | gets the ceph portion of our live migration (+grenade) jobs running again | 18:46 |
| mriedem | efried: maybe you want to hit https://review.openstack.org/#/c/643150/ and melwitt can get the one below | 18:46 |
| efried | mriedem: is that reverting something? | 18:47 |
| mriedem | yeah in part | 18:47 |
| mriedem | revert + fix | 18:47 |
| mriedem | it's reverting this https://review.openstack.org/#/c/643122/ | 18:48 |
| efried | k, Ima update commit message accordingly | 18:48 |
| *** cdent has quit IRC | 18:48 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | 18:48 |
| efried | mriedem: done | 18:50 |
| *** gmann is now known as gmann_afk | 18:51 | |
| mriedem | thanks | 18:52 |
| *** priteau has quit IRC | 18:56 | |
| *** awalende has joined #openstack-nova | 19:00 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 19:01 |
| *** awalende has quit IRC | 19:04 | |
| *** wolverineav has quit IRC | 19:07 | |
| *** tssurya has joined #openstack-nova | 19:08 | |
| *** mikal has joined #openstack-nova | 19:08 | |
| *** mvkr has quit IRC | 19:10 | |
| *** mikal has quit IRC | 19:15 | |
| *** dakshina-ilangov has quit IRC | 19:15 | |
| *** awaugama has joined #openstack-nova | 19:16 | |
| *** priteau has joined #openstack-nova | 19:17 | |
| *** tbachman has quit IRC | 19:20 | |
| *** priteau has quit IRC | 19:21 | |
| edleafe | python | 19:36 |
| edleafe | doh! | 19:37 |
| *** wolverineav has joined #openstack-nova | 19:42 | |
| *** luksky has joined #openstack-nova | 19:43 | |
| *** wolverineav has quit IRC | 19:43 | |
| *** wolverineav has joined #openstack-nova | 19:44 | |
| *** wolverineav has quit IRC | 19:47 | |
| *** dpawlik has joined #openstack-nova | 19:51 | |
| *** wolverineav has joined #openstack-nova | 19:51 | |
| *** wolverineav has quit IRC | 19:52 | |
| *** wolverineav has joined #openstack-nova | 19:53 | |
| *** dpawlik has quit IRC | 19:56 | |
| *** martinkennelly has joined #openstack-nova | 19:59 | |
| mriedem | your secret is out | 19:59 |
| martinkennelly | Hey all, I just inherited an openstack deployment that I do not think is consuming the whole node. It is a single node deployment of OS. How can I help it consume the whole node? | 20:00 |
| *** IvensZambrano has joined #openstack-nova | 20:00 | |
| mriedem | drop reserved host config values for cpu/ram/disk, create finer grained flavors | 20:01 |
| mriedem | really pack it tight | 20:01 |
| mriedem | and then not be able to resize anything.... | 20:02 |
| *** dpawlik has joined #openstack-nova | 20:02 | |
| *** dpawlik has quit IRC | 20:07 | |
| *** IvensZambrano has quit IRC | 20:09 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize https://review.openstack.org/636253 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional test for cross-cell migrate with target host https://review.openstack.org/642591 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Validate image/create during cross-cell resize functional testing https://review.openstack.org/642592 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source https://review.openstack.org/637058 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask https://review.openstack.org/637070 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize conductor RPC method https://review.openstack.org/637075 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize from the API https://review.openstack.org/637316 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add revert_snapshot_based_resize_at_dest compute method https://review.openstack.org/637630 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deal with cross-cell resize in _remove_deleted_instances_allocations https://review.openstack.org/639453 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add finish_revert_snapshot_based_resize_at_source compute method https://review.openstack.org/637647 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask https://review.openstack.org/638046 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.openstack.org/638047 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.openstack.org/638048 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.openstack.org/638268 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.openstack.org/638269 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add zones wrinkle to TestMultiCellMigrate https://review.openstack.org/643450 | 20:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add negative test for cross-cell finish_resize failing https://review.openstack.org/643451 | 20:23 |
| *** awaugama has quit IRC | 20:25 | |
| *** priteau has joined #openstack-nova | 20:26 | |
| mriedem | sob, time to rebase | 20:30 |
| mriedem | 44 changes | 20:30 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test for bug 1818914 https://review.openstack.org/641521 | 20:34 |
| openstack | bug 1818914 in OpenStack Compute (nova) "Hypervisor resource usage on source still shows old flavor usage after resize confirm until update_available_resource periodic runs" [Low,In progress] https://launchpad.net/bugs/1818914 - Assigned to Matt Riedemann (mriedem) | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove unused context parameter from RT._get_instance_type https://review.openstack.org/641792 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Update usage in RT.drop_move_claim during confirm resize https://review.openstack.org/641806 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix ProviderUsageBaseTestCase._run_periodics for multi-cell https://review.openstack.org/641179 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Improve CinderFixtureNewAttachFlow https://review.openstack.org/639382 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.openstack.org/631123 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.openstack.org/627890 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832 | 20:34 |
| *** marst has quit IRC | 20:34 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image https://review.openstack.org/637605 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.openstack.org/627891 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor ComputeManager.remove_volume_connection https://review.openstack.org/642183 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deal with cross-cell resize in _remove_deleted_instances_allocations https://review.openstack.org/639453 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add finish_revert_snapshot_based_resize_at_source compute method https://review.openstack.org/637647 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask https://review.openstack.org/638046 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.openstack.org/638047 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.openstack.org/638048 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.openstack.org/638268 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | 20:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.openstack.org/638269 | 20:34 |
| mgoddard | hi nova. Hitting an issue on ubuntu bionic where libvirt has a guest arch (armv6l) unknown to nova, causing nova-compute to error | 20:37 |
| mgoddard | InvalidArchitectureName: Architecture name 'armv6l' is not recognised | 20:38 |
| mgoddard | http://logs.openstack.org/08/643308/4/check/kolla-ansible-ubuntu-source/eee856d/primary/logs/kolla/nova/nova-compute.txt.gz#_2019-03-14_19_11_31_709 | 20:38 |
| mgoddard | Is this a known issue? | 20:38 |
| mgoddard | qemu 2.11.1 | 20:42 |
| mgoddard | libvirt 4.0.0 | 20:42 |
| melwitt | looks like it, if the arch isn't in this list, calls to canonicalize() will raise that exception https://github.com/openstack/nova/blob/master/nova/objects/fields.py#L102 | 20:43 |
| *** priteau has quit IRC | 20:43 | |
| mgoddard | melwitt: better hope no one invents any new arches after a nova release then :) | 20:44 |
| melwitt | haha, yeah | 20:45 |
| *** hamzy has quit IRC | 20:45 | |
| melwitt | I don't know the story behind it, dansmith or someone might know | 20:45 |
| dansmith | the story behind why we have enums? | 20:45 |
| mgoddard | melwitt: I'll push a patch for armv6l, but maybe an item for the nova team TODO list to make it a bit more resilient to the unknown? | 20:45 |
| melwitt | dansmith: yeah, why it's a limited set that has to be added to when new arch arrive | 20:46 |
| dansmith | melwitt: kindof the definition of an enum right? | 20:46 |
| melwitt | dansmith: well, ok, then yes, why we have enum. you know what I mean | 20:46 |
| dansmith | we have enums for known image properties so one we don't support doesn't sneak into place, and we have enums for virt drivers for ... | 20:46 |
| *** takashin has joined #openstack-nova | 20:47 | |
| dansmith | I mean, everything on the wire could just be strings and dicts, but, the whole point of this stuff was to nail it down so we could tell when something changed | 20:47 |
| melwitt | mgoddard: in what way, a better error maybe? like ArchitectureUnknown? | 20:47 |
| mgoddard | the problem here is that a new arch in libvirt capabilities causes nova-compute to fail | 20:47 |
| mgoddard | it doesn't have to support the arch, but it shouldn't break everything else | 20:48 |
| dansmith | sure, so the driver maybe should catch the failure to set the enum, log an error and exit on startup | 20:49 |
| mgoddard | exit? | 20:50 |
| melwitt | nova meeting in 10 min | 20:50 |
| mgoddard | I don't think that libvirt adding an obscure new arch should break nova | 20:51 |
| dansmith | mgoddard: I haven't clicked on your link so I don't know what's happening specifically | 20:51 |
| dansmith | mgoddard: obviously if it's just a new supported emulated arch, then the driver just needs to not fail when it can't set that on an object, for sure | 20:51 |
| mgoddard | dansmith: +1 | 20:51 |
| melwitt | ok, looks like update_available_resource is raising because of the unknown arch | 20:51 |
| dansmith | mgoddard: if it's something more fundamental like it's the native system arch or something, it would make sense for it to not start if it isn't going to function | 20:52 |
| melwitt | I know we've talked before about whether we should except-continue for things that raise during update_available_resource. maybe this would be an acceptable one. for anything else, we let it halt | 20:52 |
| *** helenafm has quit IRC | 20:53 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 20:53 |
| dansmith | it of course depends on the context of the issue | 20:53 |
| dansmith | if it's unable to do something fundamental, it should abort | 20:53 |
| dansmith | if it just can't expose support for some thing because of a new libvirt (which is sounds like in this case) then of course it should just not break on forward compat | 20:54 |
| melwitt | yeah, I mean in update_available_resource, if InvalidArchitectureName is raised | 20:54 |
| dansmith | it's some virt specific place it's crashing I imagine, so it would be in there, not directly in update_available_resource() | 20:55 |
| dansmith | you surely don't want the resource tracker ignoring something like ArchNotSupported | 20:55 |
| dansmith | but the libvirt driver can know if it's okay to skip that | 20:55 |
| melwitt | oh | 20:55 |
| dansmith | _get_instance_capabilities() | 20:56 |
| dansmith | just needs to catch and ignore that | 20:56 |
| openstackgerrit | Mark Goddard proposed openstack/nova master: Add armv6l to Architecture enum https://review.openstack.org/643455 | 20:56 |
| mriedem | this is why it breaks https://review.openstack.org/#/c/356067/ | 20:56 |
| mriedem | i mean, there is no major reason for that change except "use the enums we have" | 20:56 |
| mriedem | w/o considering side effects | 20:56 |
| mgoddard | I'll leave this one in your capable hands - it's long past home time here. Thanks for your input | 20:57 |
| dansmith | mriedem: one reason (even if presumptive) so that each driver doesn't call the arches something slightly different causing the api to return inconsistent results | 20:57 |
| mriedem | it would blow up before that as well https://review.openstack.org/#/c/356067/12/nova/compute/arch.py | 20:57 |
| mriedem | so this is super latent behavior | 20:57 |
| mriedem | dansmith: like the 50 different ways to describe power | 20:58 |
| dansmith | point being, the bug (aside from the missing arch) is just in the libvirt driver being stupid | 20:58 |
| dansmith | we need a catch and ignore right here, is all: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5942-L5945 | 20:59 |
| melwitt | mgoddard: ok, thanks for letting us know. I can open a bug for that and propose a patch, if no one else wants to | 20:59 |
| * melwitt switches to nova meeting | 21:00 | |
| *** wolverineav has quit IRC | 21:00 | |
| dansmith | melwitt: I'm already rage typing it | 21:00 |
| mriedem | i would not be surprised if that value is used elsewhere though | 21:00 |
| melwitt | dansmith: cool | 21:00 |
| mriedem | and could blow up elsewhere | 21:00 |
| dansmith | mriedem: could be, but it's clearly wrong for that loop to not continue | 21:01 |
| *** wolverineav has joined #openstack-nova | 21:01 | |
| mriedem | yeah sure | 21:01 |
| mriedem | just exclude that one with a warning i guess | 21:01 |
| *** marst has joined #openstack-nova | 21:01 | |
| dansmith | I don't thin it should even be that, otherwise we'll log that over and over for every instance and every run of the periodic | 21:02 |
| mriedem | oh right this is the periodic | 21:02 |
| mriedem | well you could log once and set a flag in the driver instance, or are you just going to exit? | 21:02 |
| dansmith | no, not exit that makes no sense, just continue the loop | 21:03 |
| dansmith | nova should expose whatever arches libvirt reports that it supports, that's really all | 21:03 |
| dansmith | this code is intending to do that, assuming it knows about everything | 21:03 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Avoid crashing while getting libvirt capabilities with unknown arch names https://review.openstack.org/643458 | 21:11 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Avoid crashing while getting libvirt capabilities with unknown arch names https://review.openstack.org/643458 | 21:14 |
| *** itlinux has joined #openstack-nova | 21:17 | |
| *** hrw has joined #openstack-nova | 21:21 | |
| *** dpawlik has joined #openstack-nova | 21:23 | |
| mriedem | jroll: TheJulia: fyi https://bugs.launchpad.net/nova/+bug/1815799 | 21:26 |
| openstack | Launchpad bug 1815799 in OpenStack Compute (nova) "Attempting to delete a baremetal server places server into ERROR state" [Low,Confirmed] | 21:26 |
| mriedem | maybe a simple retry loop in the ironic driver on node destroy would solve that | 21:27 |
| *** mriedem is now known as mriedem_away | 21:27 | |
| *** dpawlik has quit IRC | 21:27 | |
| TheJulia | mriedem_away: we should be able to just pass on it if if we've already started the process of deleting the node in ironic | 21:28 |
| * TheJulia thought someone was working on this | 21:28 | |
| *** whoami-rajat has quit IRC | 21:31 | |
| *** BjoernT has quit IRC | 21:34 | |
| *** dpawlik has joined #openstack-nova | 21:34 | |
| *** wolverineav has quit IRC | 21:36 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.openstack.org/566153 | 21:36 |
| *** wolverineav has joined #openstack-nova | 21:37 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix cleaning up console tokens https://review.openstack.org/637716 | 21:37 |
| *** dpawlik has quit IRC | 21:38 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: virt/arch: use 'armv6l' for ARMV6 architecture https://review.openstack.org/643467 | 21:44 |
| hrw | that ^^ change may fix Kolla CI error | 21:46 |
| *** tbachman has joined #openstack-nova | 21:51 | |
| *** tssurya has quit IRC | 21:55 | |
| melwitt | dansmith ^ | 21:57 |
| hrw | and sorry for writing 3rd patch to what looks the same | 22:02 |
| hrw | dansmith: after reading https://review.openstack.org/#/c/643458 and https://review.openstack.org/#/c/643455 I think that 643355 is wrong, your patch is good (left comment) and mine https://review.openstack.org/643467 just fixes to comply with capabilities contents | 22:07 |
| hrw | but that's me - random guy from the internet D: | 22:07 |
| *** dpawlik has joined #openstack-nova | 22:09 | |
| openstackgerrit | Merged openstack/nova master: Customize irrelevant-files for nova-live-migration job https://review.openstack.org/643346 | 22:13 |
| openstackgerrit | Merged openstack/nova master: Re-enable Ceph in live migration testing https://review.openstack.org/643150 | 22:13 |
| *** dpawlik has quit IRC | 22:13 | |
| *** pcaruana has quit IRC | 22:21 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP/PoC: Bypass ironicclient for node.get https://review.openstack.org/642899 | 22:21 |
| *** marst has quit IRC | 22:25 | |
| dansmith | melwitt: what are you pointing me to? | 22:26 |
| melwitt | dansmith: hrw's patch https://review.openstack.org/643467 | 22:27 |
| melwitt | it's related to the unknown arch blowup, sort of | 22:28 |
| dansmith | yeah, I'm not sure which is the official arch name and/or if they're both legit as I'm not an arm specialist | 22:28 |
| dansmith | but my patch is still something we should do as (as hrw said) | 22:28 |
| hrw | dansmith: armv6 armv6l depends on kernel. | 22:29 |
| melwitt | hrw: if you think https://review.openstack.org/643455 is wrong, it would be helpful to put your comment on it about why. I'm curious anyway | 22:29 |
| dansmith | hrw: my concern would be it might also depend on libvirt behavior | 22:29 |
| melwitt | dansmith: yeah, not saying your patch isn't something we should do. just that along that line, if you knew anything about changing that enum, hrw's patch was there | 22:29 |
| dansmith | hrw: because if the l just means long or something, then I would expect both or either could be emulatable | 22:29 |
| dansmith | melwitt: hopefully it's obvious that hrw's patch to change the value of the enum is not okay :) | 22:30 |
| melwitt | dansmith: gah, I thought maybe you could comment on the review | 22:31 |
| *** wolverineav has quit IRC | 22:31 | |
| melwitt | nevermind me | 22:31 |
| hrw | dansmith: libvirt mentions only armv6l and armv7l. | 22:31 |
| dansmith | melwitt: I just did | 22:31 |
| melwitt | thanks | 22:32 |
| dansmith | hrw: on your system you mean right? | 22:32 |
| hrw | https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=&search=VIR_ARCH_ARMV6L&project_id=192693#L30 | 22:32 |
| hrw | in libvirt code | 22:32 |
| dansmith | hrw: okay, like I said, I'm not really able to say whether armv6 or armv6l (or both) are right | 22:33 |
| dansmith | but we can't just make the change you've proposed because it has effects to versioned objects and will break communication between services | 22:33 |
| hrw | ok. | 22:33 |
| hrw | perfectly fine for me. | 22:34 |
| dansmith | hrw: I will say, if you're right that it's always armv6l, then we definitely need a variant of your patch over mgoddard's | 22:34 |
| hrw | dansmith: let me discuss it tomorrow with libvirt guys, ok? | 22:34 |
| hrw | 23:34 here | 22:34 |
| dansmith | sure | 22:35 |
| hrw | as nova/virt/libvirt gets capabilities from libvirt so is it armv6 or armv6l is more or less their decision. if they say 'no, it was always armv6z' then we both are wrong | 22:36 |
| *** ivve has quit IRC | 22:36 | |
| *** wolverineav has joined #openstack-nova | 22:36 | |
| dansmith | yeah | 22:36 |
| hrw | the good point is that armv6 is not able to be a host so it is only guest issue | 22:36 |
| openstackgerrit | Kevin Rasmussen proposed openstack/nova master: created AggregateSubnetAvailabilityFilter https://review.openstack.org/633178 | 22:38 |
| dansmith | I'm not sure that's relevant to the thing that nova has to do in this case, but.. okay :) | 22:38 |
| hrw | nothing in this case ;D | 22:38 |
| *** gmann_afk is now known as gmann | 22:40 | |
| hrw | ok, went, asked, go to sleep. wake up in ~6h | 22:42 |
| dansmith | go :) | 22:42 |
| hrw | and I promised myself to not play with nova for at least a year... ;d | 22:44 |
| hrw | have a nice day | 22:44 |
| *** rcernin has joined #openstack-nova | 23:00 | |
| *** tkajinam has joined #openstack-nova | 23:02 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Remove [ironic]api_endpoint option https://review.openstack.org/643483 | 23:03 |
| openstackgerrit | Julia Kreger proposed openstack/nova master: [ironic] ignore conflict upon cleaning up node https://review.openstack.org/643484 | 23:03 |
| TheJulia | mriedem_away: melwitt: ^^^ | 23:03 |
| *** wolverineav has quit IRC | 23:04 | |
| *** hongbin has quit IRC | 23:04 | |
| melwitt | TheJulia: commented. I had thought jroll or someone had said we shouldn't do that in _cleanup_deploy unilaterally | 23:06 |
| TheJulia | I really don't know what to do then | 23:07 |
| * TheJulia thinks through it for a few | 23:08 | |
| *** wolverineav has joined #openstack-nova | 23:08 | |
| *** martinkennelly has quit IRC | 23:09 | |
| melwitt | TheJulia: well, I just mean that try-except would move to the destroy() method | 23:10 |
| TheJulia | I'm checking to see if we can do that | 23:11 |
| *** luksky has quit IRC | 23:11 | |
| melwitt | if I'm remembering correctly that it should be done for all clean deploys. I might be wrong | 23:11 |
| melwitt | *shouldn't | 23:11 |
| *** wolverineav has quit IRC | 23:11 | |
| *** wolverineav has joined #openstack-nova | 23:12 | |
| *** _pewp_ has quit IRC | 23:13 | |
| *** _pewp_ has joined #openstack-nova | 23:14 | |
| TheJulia | well, making sure we nuke everything and if we can just tolerate it on a destroy | 23:14 |
| *** mvkr has joined #openstack-nova | 23:18 | |
| TheJulia | melwitt: looks like we can tolerate a conflict anywhere along there because ironic will do cleanup. Some of the things more recently, but yeah... we should be good to just move it | 23:19 |
| melwitt | TheJulia: ok. maybe we'll double check with jroll first before you change anything. bc I'm not 100% sure I remembered that right and don't remember what date to look for #openstack-ironic logs | 23:20 |
| TheJulia | melwitt: I just double checked the paths and coductor behavior | 23:20 |
| melwitt | TheJulia: oh, sorry, I misunderstood. you confirmed it should be only for destroy() and not all cleanup deploy | 23:20 |
| TheJulia | yeah | 23:21 |
| melwitt | ok, cool | 23:21 |
| *** mriedem_away has quit IRC | 23:23 | |
| *** slaweq has quit IRC | 23:25 | |
| *** tosky has quit IRC | 23:26 | |
| openstackgerrit | Julia Kreger proposed openstack/nova master: [ironic] ignore conflict upon cleaning up node https://review.openstack.org/643484 | 23:27 |
| TheJulia | melwitt: ^^ | 23:27 |
| melwitt | coolness | 23:28 |
| melwitt | TheJulia: there's comments on PS1 about whether it's ok to ignore any and all conflict (only one way that conflict can occur) | 23:29 |
| melwitt | if you could reply and confirm that | 23:30 |
| *** gyee has quit IRC | 23:34 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add description about sort order in API ref guideline https://review.openstack.org/627282 | 23:36 |
| *** wolverineav has quit IRC | 23:41 | |
| *** wolverineav has joined #openstack-nova | 23:42 | |
| *** wolverineav has quit IRC | 23:46 | |
| *** wolverineav has joined #openstack-nova | 23:49 | |
| *** wolverineav has quit IRC | 23:53 | |
| *** wolverineav has joined #openstack-nova | 23:55 | |
| *** wolverineav has quit IRC | 23:56 | |
| *** wolverineav has joined #openstack-nova | 23:56 | |
| *** wolverineav has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!