*** hamalq_ has quit IRC | 00:05 | |
*** hongbin has quit IRC | 00:16 | |
*** jmlowe has joined #openstack-nova | 00:24 | |
*** ociuhandu has joined #openstack-nova | 00:24 | |
*** ociuhandu has quit IRC | 00:29 | |
*** hemna has quit IRC | 00:30 | |
*** hemna has joined #openstack-nova | 00:30 | |
*** jmlowe has quit IRC | 00:33 | |
*** hongbin has joined #openstack-nova | 00:34 | |
*** hemna has quit IRC | 00:38 | |
*** hemna has joined #openstack-nova | 00:39 | |
*** sapd1 has joined #openstack-nova | 00:51 | |
*** sapd1 has quit IRC | 00:59 | |
*** sapd1 has joined #openstack-nova | 01:04 | |
*** tbachman has quit IRC | 01:14 | |
*** tbachman has joined #openstack-nova | 01:15 | |
*** lbragstad has joined #openstack-nova | 01:18 | |
*** Liang__ has joined #openstack-nova | 01:18 | |
*** sapd1 has quit IRC | 01:23 | |
*** hongbin has quit IRC | 01:42 | |
*** hemna has quit IRC | 01:52 | |
*** hemna has joined #openstack-nova | 01:53 | |
*** hemna has quit IRC | 01:53 | |
*** hemna has joined #openstack-nova | 01:54 | |
*** Liang__ has quit IRC | 02:06 | |
*** Liang__ has joined #openstack-nova | 02:10 | |
*** Liang__ has quit IRC | 02:15 | |
*** Liang__ has joined #openstack-nova | 02:21 | |
*** xinranwang has joined #openstack-nova | 02:28 | |
*** jmlowe has joined #openstack-nova | 02:41 | |
*** hongbin has joined #openstack-nova | 02:43 | |
*** Liang__ has quit IRC | 02:44 | |
openstackgerrit | sunhao proposed openstack/nova master: Add checks for volume status when rebuilding https://review.opendev.org/734734 | 02:46 |
---|---|---|
*** spatel has joined #openstack-nova | 03:07 | |
*** spatel has quit IRC | 03:08 | |
*** rcernin has quit IRC | 03:08 | |
*** tetsuro has quit IRC | 03:16 | |
*** xiaolin has joined #openstack-nova | 03:31 | |
*** lbragstad has quit IRC | 03:36 | |
*** rcernin has joined #openstack-nova | 03:54 | |
*** tetsuro has joined #openstack-nova | 03:54 | |
*** rcernin has quit IRC | 03:55 | |
*** rcernin has joined #openstack-nova | 03:55 | |
*** yaawang has quit IRC | 04:11 | |
*** yaawang has joined #openstack-nova | 04:11 | |
*** evrardjp has quit IRC | 04:33 | |
*** hongbin has quit IRC | 04:33 | |
*** evrardjp has joined #openstack-nova | 04:33 | |
openstackgerrit | sunhao proposed openstack/nova master: Add checks for volume status when rebuilding https://review.opendev.org/734734 | 04:44 |
*** links has joined #openstack-nova | 04:50 | |
*** boxiang has quit IRC | 04:55 | |
*** boxiang has joined #openstack-nova | 04:55 | |
*** Liang__ has joined #openstack-nova | 04:56 | |
*** gyee has quit IRC | 05:11 | |
*** links has quit IRC | 05:12 | |
*** links has joined #openstack-nova | 05:21 | |
*** jaosorior has quit IRC | 05:21 | |
*** jaosorior has joined #openstack-nova | 05:23 | |
*** vishalmanchanda has joined #openstack-nova | 05:35 | |
*** arne_wiebalck has quit IRC | 05:38 | |
*** arne_wiebalck has joined #openstack-nova | 05:38 | |
*** Liang__ has quit IRC | 05:38 | |
*** Liang__ has joined #openstack-nova | 05:39 | |
*** Liang__ has quit IRC | 05:43 | |
*** Liang__ has joined #openstack-nova | 05:44 | |
*** ratailor has joined #openstack-nova | 05:44 | |
*** tetsuro_ has joined #openstack-nova | 05:45 | |
*** jsuchome has joined #openstack-nova | 05:46 | |
*** tetsuro__ has joined #openstack-nova | 05:47 | |
*** tetsuro has quit IRC | 05:47 | |
*** tetsuro_ has quit IRC | 05:50 | |
*** yaawang has quit IRC | 05:55 | |
*** yaawang has joined #openstack-nova | 05:56 | |
*** udesale has joined #openstack-nova | 06:02 | |
*** dklyle has quit IRC | 06:04 | |
*** udesale has quit IRC | 06:17 | |
*** udesale has joined #openstack-nova | 06:18 | |
frickler | sean-k-mooney: implemented a first draft of your idea regarding pip3 in https://review.opendev.org/735058 , tested only on focal so far, waiting for test results on other distros. ideally all of them would no longer need this whole manual installation thing anymore | 06:22 |
*** dustinc has quit IRC | 06:35 | |
*** belmoreira has joined #openstack-nova | 06:38 | |
*** jmlowe has quit IRC | 06:45 | |
*** xinranwang has quit IRC | 06:54 | |
*** yaawang has quit IRC | 06:55 | |
*** yaawang has joined #openstack-nova | 06:56 | |
*** tesseract has joined #openstack-nova | 07:05 | |
*** yaawang has quit IRC | 07:08 | |
*** yaawang has joined #openstack-nova | 07:09 | |
*** rpittau|afk is now known as rpittau | 07:18 | |
openstackgerrit | Harshavardhan Metla proposed openstack/nova master: Raising a failure in case of 403 https://review.opendev.org/735068 | 07:18 |
*** ttsiouts has joined #openstack-nova | 07:19 | |
*** ralonsoh has joined #openstack-nova | 07:21 | |
*** jawad_axd has joined #openstack-nova | 07:21 | |
*** ttsiouts has quit IRC | 07:22 | |
*** ttsiouts has joined #openstack-nova | 07:24 | |
openstackgerrit | Marek Lyčka proposed openstack/nova-specs master: Enable REST quiescing https://review.opendev.org/702810 | 07:25 |
*** boxiang_ has joined #openstack-nova | 07:30 | |
*** avolkov has joined #openstack-nova | 07:31 | |
*** boxiang has quit IRC | 07:34 | |
*** tetsuro__ has quit IRC | 07:37 | |
*** tosky has joined #openstack-nova | 07:38 | |
*** tetsuro has joined #openstack-nova | 07:40 | |
*** ociuhandu has joined #openstack-nova | 07:41 | |
*** factor has quit IRC | 07:46 | |
*** factor has joined #openstack-nova | 07:47 | |
*** ociuhandu has quit IRC | 07:50 | |
*** ociuhandu has joined #openstack-nova | 07:52 | |
*** rcernin has quit IRC | 08:00 | |
*** tesseract has quit IRC | 08:02 | |
*** tesseract has joined #openstack-nova | 08:03 | |
*** yaawang has quit IRC | 08:11 | |
*** yaawang has joined #openstack-nova | 08:11 | |
*** tesseract has quit IRC | 08:11 | |
*** martinkennelly has joined #openstack-nova | 08:11 | |
*** slaweq has quit IRC | 08:15 | |
*** jamesdenton has quit IRC | 08:16 | |
openstackgerrit | Marek Lyčka proposed openstack/nova-specs master: Enable REST quiescing https://review.opendev.org/702810 | 08:18 |
*** slaweq has joined #openstack-nova | 08:20 | |
*** boxiang has joined #openstack-nova | 08:31 | |
*** ociuhandu has quit IRC | 08:31 | |
*** boxiang_ has quit IRC | 08:31 | |
*** ociuhandu has joined #openstack-nova | 08:34 | |
*** ociuhandu has quit IRC | 08:36 | |
*** ociuhandu has joined #openstack-nova | 08:42 | |
*** ociuhandu has quit IRC | 08:46 | |
*** ociuhandu has joined #openstack-nova | 08:47 | |
*** tesseract has joined #openstack-nova | 08:49 | |
*** rcernin has joined #openstack-nova | 08:50 | |
*** ociuhandu has quit IRC | 08:53 | |
*** tesseract-RH has joined #openstack-nova | 08:56 | |
openstackgerrit | Alexandre Arents proposed openstack/nova stable/stein: Reserve DISK_GB resource for the image cache https://review.opendev.org/732529 | 08:56 |
*** tesseract-RH has quit IRC | 08:56 | |
*** tesseract has quit IRC | 08:56 | |
*** tesseract has joined #openstack-nova | 08:56 | |
*** tetsuro has quit IRC | 08:57 | |
*** martinkennelly has quit IRC | 09:00 | |
*** tetsuro has joined #openstack-nova | 09:00 | |
*** sapd1_x has quit IRC | 09:02 | |
*** sapd1 has joined #openstack-nova | 09:06 | |
bauzas | late morning hello world | 09:08 |
bauzas | was on teaching mode for my kid... | 09:09 |
*** rpittau is now known as rpittau|bbl | 09:15 | |
*** factor has quit IRC | 09:16 | |
*** factor has joined #openstack-nova | 09:16 | |
*** rcernin has quit IRC | 09:17 | |
gibi | bauzas: o/ | 09:17 |
bauzas | specs writing and reviewing day FTW (hopefully) | 09:18 |
bauzas | gibi: if you have some of them you want me to look... | 09:18 |
bauzas | gibi: btw. tomorrow will be a Red Hat worlwide PTO | 09:18 |
bauzas | gibi: so you won't see all of us I think | 09:18 |
gibi | bauzas: I looket throught the open specs, the list is short so I suggest to look there too | 09:19 |
bauzas | kk | 09:19 |
gibi | bauzas: thanks for the heads up. Have a nice PTO tomorrow | 09:19 |
*** slaweq has quit IRC | 09:19 | |
bauzas | it's actually a company holiday | 09:19 |
gibi | interesting concept | 09:19 |
gibi | do you have company events on this day or it is simply a day off | 09:20 |
gibi | ? | 09:20 |
*** ociuhandu has joined #openstack-nova | 09:20 | |
*** tetsuro has quit IRC | 09:28 | |
*** ociuhandu has quit IRC | 09:28 | |
bauzas | gibi: my company calls it 'Recharge Day' | 09:29 |
bauzas | gibi: US folks had company holidays called 'shutdown' between Xmas and New Year | 09:29 |
bauzas | now it's a bit changing for sales reasons, so they created 'Recharge days' | 09:30 |
bauzas | and for the first time, they offered recharge days | 09:30 |
bauzas | for this quarter, the first holiday will be tomorrow but we'll have a Q3 and Q4 holiday too | 09:30 |
*** Liang__ has quit IRC | 09:31 | |
*** ociuhandu has joined #openstack-nova | 09:32 | |
gibi | sounds good. Local Ericsson has a yearly recharge week but thats a normal work week with some exta perks like healthy food and free medical checkups | 09:36 |
*** ratailor has quit IRC | 09:38 | |
*** ratailor has joined #openstack-nova | 09:38 | |
bauzas | gibi: are you still working remotely or back at the office ? | 09:43 |
gibi | bauzas: still at home full time. I think from next week we can slowly return to the office but it is not mandatory to go back at this time | 09:46 |
bauzas | ack | 09:56 |
*** slaweq has joined #openstack-nova | 09:58 | |
*** slaweq has quit IRC | 10:20 | |
*** ociuhandu has quit IRC | 10:25 | |
*** efried has quit IRC | 10:32 | |
openstackgerrit | Harshavardhan Metla proposed openstack/nova master: Raising a failure in case of 403 https://review.opendev.org/735068 | 10:35 |
*** jraju__ has joined #openstack-nova | 10:35 | |
*** links has quit IRC | 10:36 | |
*** ttsiouts has quit IRC | 10:39 | |
*** martinkennelly has joined #openstack-nova | 10:42 | |
*** ociuhandu has joined #openstack-nova | 10:42 | |
*** derekh has joined #openstack-nova | 10:43 | |
gibi | stephenfin: hi! as I see the first part of the impl of bp use-pcpu-and-vcpu-in-one-instance is from you and it seems ready for a Runway slot. Will the next two weeks good for you to iterate on those patches? | 10:46 |
*** sapd1 has quit IRC | 10:46 | |
*** ociuhandu has quit IRC | 10:52 | |
*** ociuhandu has joined #openstack-nova | 10:53 | |
*** purplerbot has joined #openstack-nova | 10:58 | |
*** ociuhandu has quit IRC | 10:58 | |
*** ttsiouts has joined #openstack-nova | 11:03 | |
stephenfin | gibi: yup, sure thing | 11:05 |
*** martinkennelly has quit IRC | 11:05 | |
*** martinkennelly has joined #openstack-nova | 11:05 | |
gibi | stephenfin: cool. I will bring it up on the weekly meeting so if there are cores who willing to spend time on reviewing it | 11:05 |
stephenfin | \o/ sounds good to me | 11:06 |
*** yaawang has quit IRC | 11:08 | |
*** yaawang has joined #openstack-nova | 11:09 | |
*** threestrands has quit IRC | 11:10 | |
*** ttsiouts has quit IRC | 11:18 | |
*** nightmare_unreal has joined #openstack-nova | 11:23 | |
*** slaweq has joined #openstack-nova | 11:24 | |
*** ociuhandu has joined #openstack-nova | 11:27 | |
*** ttsiouts has joined #openstack-nova | 11:31 | |
*** ociuhandu has quit IRC | 11:32 | |
*** rcernin has joined #openstack-nova | 11:38 | |
*** rcernin has quit IRC | 11:43 | |
*** ttsiouts has quit IRC | 11:46 | |
*** rpittau|bbl is now known as rpittau | 11:54 | |
*** ratailor has quit IRC | 12:02 | |
*** slaweq has quit IRC | 12:13 | |
*** ttsiouts has joined #openstack-nova | 12:17 | |
*** nweinber has joined #openstack-nova | 12:18 | |
*** tkajinam has quit IRC | 12:25 | |
*** ociuhandu has joined #openstack-nova | 12:32 | |
*** ttsiouts has quit IRC | 12:32 | |
*** fyx has quit IRC | 12:32 | |
*** fyx has joined #openstack-nova | 12:32 | |
*** bcm has quit IRC | 12:32 | |
*** mnaser has quit IRC | 12:32 | |
*** jungleboyj has quit IRC | 12:33 | |
*** gmann has quit IRC | 12:33 | |
*** lseki has quit IRC | 12:33 | |
*** nicolasbock has quit IRC | 12:33 | |
*** jungleboyj has joined #openstack-nova | 12:33 | |
*** bcm has joined #openstack-nova | 12:33 | |
*** lseki has joined #openstack-nova | 12:33 | |
*** gmann has joined #openstack-nova | 12:33 | |
*** nicolasbock has joined #openstack-nova | 12:33 | |
*** mnaser has joined #openstack-nova | 12:33 | |
*** ttsiouts has joined #openstack-nova | 12:35 | |
*** ociuhandu has quit IRC | 12:36 | |
*** slaweq has joined #openstack-nova | 12:43 | |
*** jamesdenton has joined #openstack-nova | 12:46 | |
*** slaweq has quit IRC | 12:47 | |
artom | stephenfin, don't you have the hackfest? | 12:48 |
artom | Is that in the next two weeks? | 12:48 |
stephenfin | end of the month | 12:48 |
stephenfin | so all good | 12:49 |
stephenfin | Anyone tried deploying devstack on an RPi? | 12:49 |
artom | stephenfin, ah, sorry, in my head they were back to back | 12:51 |
*** mriedem has joined #openstack-nova | 12:55 | |
*** bbowen has joined #openstack-nova | 13:15 | |
*** owalsh_ is now known as owalsh | 13:20 | |
*** efried has joined #openstack-nova | 13:20 | |
*** boxiang has quit IRC | 13:22 | |
*** boxiang has joined #openstack-nova | 13:22 | |
*** ttsiouts has quit IRC | 13:31 | |
*** lbragstad has joined #openstack-nova | 13:34 | |
*** jmlowe has joined #openstack-nova | 13:38 | |
*** jaosorior has quit IRC | 13:44 | |
*** sangeet has joined #openstack-nova | 13:52 | |
openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job https://review.opendev.org/734184 | 13:53 |
dansmith | sean-k-mooney: yeah, default is uwsgi and apparently none of the magic glance stuff works without mod_wsgi.. the image import, image conversion, etc | 13:55 |
sean-k-mooney | ah i see | 13:56 |
sean-k-mooney | that is likely a glance bug... | 13:56 |
dansmith | I needed it for the copy-to-store, but forgot to copy that from my local config, but hadn't gotten far enough to notice and it's required for the conversion too | 13:56 |
dansmith | I dunno, don't we have problems with uwsgi and background tasks? | 13:56 |
dansmith | it's because those things happen in threads | 13:56 |
sean-k-mooney | oh so its the same issue as we have with the heartbeats | 13:57 |
dansmith | right | 13:57 |
sean-k-mooney | i guess you could argure that they are relying on behavior of the implemeation that is not requried | 13:57 |
sean-k-mooney | but ya fair point | 13:57 |
sean-k-mooney | e.g. the lifetime smeantics of mod_wsgi or the python one | 13:58 |
dansmith | well, it has been fine for multiple things (pre-wsgi, mod_wsgi) until uwsgi, but yeah they could use a conductor like service | 13:58 |
dansmith | but that kinda sucks | 13:58 |
dansmith | and needs to be on the same machine as the api I think because of the file uploads it needs to access | 13:58 |
sean-k-mooney | ya fair enough | 13:59 |
dansmith | anyway, it'll be a dependency for using this stuff | 13:59 |
*** ociuhandu has joined #openstack-nova | 14:05 | |
dansmith | sean-k-mooney: so, I'm tweaking my spec to remove the "functional tests should be easy" section, which I've realized is actually going to be super difficult | 14:13 |
dansmith | because we stub out ALL of the image backend stuff in functional tests for libvirt, | 14:13 |
sean-k-mooney | i assume we are just going to keep your new multistore ceph job ya | 14:13 |
dansmith | and would require me to write at least 10x the actual code to just have functional support | 14:13 |
dansmith | instead, | 14:13 |
sean-k-mooney | if so i think that is more then enough | 14:13 |
dansmith | I'm thinking we just change the existing job to do what I have here, which means the first tempest test kicks a copy-to-store from the file backend, and then the rest of the tests just have it set as it would | 14:14 |
dansmith | yeah, cool | 14:14 |
dansmith | my only concern is what happens if/when we test snapshot with the file backend as the default, | 14:15 |
sean-k-mooney | if we have a regression in the future i think we can corss the function test bridge then | 14:15 |
dansmith | but if that does something stupid I might need to do some devstack surgery to make the rbd still the default, but upload the cirros image to the file backend | 14:15 |
sean-k-mooney | well i think we can specify the store on upload | 14:15 |
dansmith | yeah, we can, | 14:16 |
dansmith | just requires devstack surgery | 14:16 |
dansmith | but I want to see if it just does the smarter thing | 14:16 |
sean-k-mooney | you are setting the default to cheap | 14:16 |
sean-k-mooney | whcih is the file backend right | 14:16 |
sean-k-mooney | but novas to robust | 14:16 |
sean-k-mooney | so the glacce upload shoudl be to file | 14:16 |
sean-k-mooney | and then we will use ceph when doing normal snapshots right | 14:17 |
dansmith | right, nova doesn't specify one when it does the snapshot.. however, since it's rbd-backed, I think snapshot does the ceph stuff under the covers, so it may work just by virtue of the fact that it doesn't actually upload, but just tells glance about it | 14:17 |
dansmith | the cirros upload will go to file currently, yeah | 14:17 |
sean-k-mooney | is this job using the rbd image backend | 14:18 |
dansmith | yeah, that's the whole point :) | 14:18 |
sean-k-mooney | or are we only using ceph for bfv | 14:18 |
sean-k-mooney | ya that is what i assuemed just checking | 14:18 |
dansmith | no, this is rbd on the image backend | 14:18 |
*** iurygregory_ has joined #openstack-nova | 14:19 | |
*** iurygregory has quit IRC | 14:19 | |
*** iurygregory_ is now known as iurygregory | 14:19 | |
sean-k-mooney | ya so im wondering if we want to either add a post playbook to test the file snapshot or if we want to add multi store to another job for that case | 14:19 |
*** udesale_ has joined #openstack-nova | 14:19 | |
sean-k-mooney | for example we have a multi cell job right | 14:20 |
sean-k-mooney | we could configure multisore on that too and check it uploads to the cell local one | 14:20 |
*** jawad_axd has quit IRC | 14:20 | |
openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job https://review.opendev.org/734184 | 14:21 |
*** udesale has quit IRC | 14:22 | |
*** ttsiouts has joined #openstack-nova | 14:23 | |
sean-k-mooney | dansmith: by the way before aarents upload a new version were you expecting to reuse this config option or add a new one https://review.opendev.org/#/c/734776/2/nova/conf/compute.py@912 | 14:26 |
dansmith | sean-k-mooney: tbh, I didn't know we had that one | 14:26 |
dansmith | I'm not really sure why we have that, as image downloads would be gated by the build limit | 14:27 |
sean-k-mooney | so when i asked about it in the ptg i tought we siad it was only for the filter schdulier btu i guess there is a seperate one for compute | 14:27 |
*** sapd1 has joined #openstack-nova | 14:28 | |
dansmith | that config you linked isn't for the filter/weigher, but for compute presumably yeah? | 14:28 |
sean-k-mooney | correct its in compute | 14:28 |
sean-k-mooney | but do we use it anywhere https://github.com/openstack/nova/search?q=max_concurrent_disk_ops&unscoped_q=max_concurrent_disk_ops | 14:29 |
sean-k-mooney | it was added by https://github.com/openstack/nova/commit/728f20e8f4ac2e3d4b893b7169b81d20471d0be9 | 14:29 |
dansmith | in libvirt | 14:29 |
sean-k-mooney | ah we do with compute_utils.disk_ops_semaphore | 14:30 |
dansmith | I mean... I was expecting there to be one for snapshotting as a whole, so we didn't even start a high-level snapshot operation until we could get to it, | 14:31 |
dansmith | this would presumably lock much further down the chain, so you're already paused and waiting for room | 14:31 |
*** mlavalle has joined #openstack-nova | 14:31 | |
dansmith | which seems less than ideal to me | 14:32 |
sean-k-mooney | ya it looks like its only locking on the actull io operation i.e. invoking qemu-image or glance | 14:32 |
*** sapd1 has quit IRC | 14:33 | |
*** ttsiouts has quit IRC | 14:34 | |
dansmith | right | 14:34 |
dansmith | I commented | 14:34 |
*** _mlavalle_1 has joined #openstack-nova | 14:34 | |
*** sapd1 has joined #openstack-nova | 14:34 | |
*** ociuhandu has quit IRC | 14:36 | |
sean-k-mooney | dansmith: that current semaphore is used in snapshot but only at the end https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driver.py#L2465 | 14:36 |
*** mlavalle has quit IRC | 14:37 | |
dansmith | ah for the actual upload I guess? | 14:37 |
*** _mlavalle_1 has quit IRC | 14:37 | |
dansmith | seems like that semaphore was added kinda willy-nilly in a bunch of places without a systemic view of "never call this code without the semaphore held" | 14:37 |
dansmith | because I assume if we grab it too high we'll double acquire | 14:37 |
stephenfin | sean-k-mooney: In brief, what would happen if you configured disk_allocation_ratio too high? Would you see outright corruption of existing images or simply failure to e.g. download images for a new instance? | 14:37 |
stephenfin | I'm not actually sure what happens with sparse image if you run out of disk space | 14:38 |
stephenfin | *a | 14:38 |
dansmith | stephenfin: you'll get disk errors in the guest when you run out of space | 14:38 |
dansmith | stephenfin: like your disk was failing if it was a real disk | 14:38 |
stephenfin | and if I tried to schedule a new guest to that host? | 14:39 |
sean-k-mooney | dansmith: ya probably | 14:39 |
dansmith | which generally means "you asked me to write this to disk, disk said no" which if it's in the journal flush could mean whole fs corruption, but likely a remount-readonly at minimum and application data loss | 14:39 |
sean-k-mooney | you can allso get into cases where you cant log in or exectue commands | 14:40 |
sean-k-mooney | in the guest | 14:40 |
dansmith | for sure | 14:40 |
stephenfin | dansmith++ sweet, thanks | 14:40 |
sean-k-mooney | i guess on the host too if the nova instnace directoy is in / | 14:40 |
stephenfin | I'm writing an overcommit doc atm and trying to tease out implications of these things | 14:40 |
sean-k-mooney | or under it on the same partion | 14:40 |
dansmith | sean-k-mooney: yes, it could be quite bad | 14:41 |
sean-k-mooney | stephenfin: honestly disk allocation ratio shoudl have a max of 1 | 14:41 |
dansmith | sean-k-mooney: presumably the host fs has some reserved-for-root set and qemu can't exhaust that though | 14:41 |
sean-k-mooney | dansmith: you would hope. | 14:41 |
dansmith | sean-k-mooney: agree, overcommit of disk is just asking for problems | 14:41 |
sean-k-mooney | peopel get a way with it with lvm thin providiouning by carfully monitoring it | 14:42 |
sean-k-mooney | but it makes me uncomfortable if i care about the data | 14:42 |
dansmith | it's the worst thing you can overcommit | 14:42 |
sean-k-mooney | dansmith: the host_reserved_disk_mb helps but if you allow suspend you can exceed that as we need to snapshot the guest ram and save it to disk | 14:43 |
sean-k-mooney | when kolla was young it did nto have logrotate and i made the mistake of running our internal intel dev cluserter with debug logging on | 14:44 |
sean-k-mooney | fixing out of disk was fun... | 14:44 |
sean-k-mooney | because it caused db curruption | 14:44 |
dansmith | I think stephenfin gets the picture.. "it's bad, yo" | 14:44 |
sean-k-mooney | stephenfin: are you going to write doc to cover the interaction betweeen file backed memory and reserved disk by the way | 14:45 |
sean-k-mooney | if not i can proably try to write something | 14:46 |
stephenfin | you mean above what I put in the config opt docs here https://review.opendev.org/#/c/734658/ ? | 14:46 |
stephenfin | oh, reserved _disk_ | 14:46 |
sean-k-mooney | yes | 14:46 |
sean-k-mooney | disk | 14:46 |
sean-k-mooney | and i guess we shoudl menetion the reserved hugepages option too | 14:47 |
stephenfin | hmm, I hadn't thought of that. Only seeing your comments now | 14:47 |
stephenfin | Sure, I can do that early next week | 14:47 |
sean-k-mooney | ok if you dont get to it remind me and i can try to do it | 14:48 |
sean-k-mooney | basically if its the same file system they shoudl add the value of the file backed memory option to reserved disk | 14:48 |
*** slaweq has joined #openstack-nova | 14:52 | |
*** factor has quit IRC | 14:52 | |
*** jraju__ has quit IRC | 14:57 | |
*** boxiang has quit IRC | 15:06 | |
*** boxiang has joined #openstack-nova | 15:07 | |
*** jdillaman has joined #openstack-nova | 15:13 | |
aarents | stephenfin: if host disk is full qemu may PAUSE himself (by default I think) see man qemu werror=action,rerror=action) | 15:15 |
*** mlavalle has joined #openstack-nova | 15:15 | |
sean-k-mooney | dansmith: looks like glance started listening on http https://zuul.opendev.org/t/openstack/build/8afb850f7844450389823d7f84b90c61/log/controller/logs/screen-g-api.txt#345 but devstack seams to be checking https https://zuul.opendev.org/t/openstack/build/8afb850f7844450389823d7f84b90c61/log/controller/logs/devstacklog.txt#19995 | 15:16 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add a guide on overcommit https://review.opendev.org/735175 | 15:16 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: tox: Stop linting as part of docs target https://review.opendev.org/735176 | 15:16 |
dansmith | that wsgi thing works for me locally | 15:17 |
stephenfin | sean-k-mooney, dansmith, aarents: that's the doc anyway | 15:17 |
sean-k-mooney | dansmith: i can deploy with the local.conf that was generated and see what happens | 15:17 |
*** dklyle has joined #openstack-nova | 15:17 | |
dansmith | sean-k-mooney: it's getting a 503, which means it's hitting apache or something which should be proxying | 15:17 |
sean-k-mooney | im wondering if it enabling the tls code | 15:17 |
sean-k-mooney | e.g. the tls proxy | 15:18 |
sean-k-mooney | but not actully configuring it for some reason | 15:18 |
dansmith | idk | 15:18 |
*** hamalq has joined #openstack-nova | 15:18 | |
sean-k-mooney | the tls proxy is enabled | 15:18 |
sean-k-mooney | https://94b38a054383d32ffbe4-eecb7ff79b5c6e121d6c20fabad8af79.ssl.cf1.rackcdn.com/734184/10/check/nova-ceph-multistore/8afb850/controller/logs/local_conf.txt | 15:18 |
dansmith | unless something changed recently | 15:19 |
dansmith | maybe the ceph plugin job I'm inheriting from does that? | 15:19 |
*** hamalq has quit IRC | 15:19 | |
dansmith | because at least a couple weeks ago, devstack didn't with just that wsgi switch in the localrc | 15:19 |
sean-k-mooney | dansmith: i think its enabeld by default | 15:19 |
dansmith | in d-g? because it wasn't for me, unless it's changed very recently | 15:20 |
sean-k-mooney | in the devstack or tempest job | 15:20 |
*** hamalq has joined #openstack-nova | 15:20 | |
sean-k-mooney | not d-g but the base devstack zuulv3 job i think | 15:20 |
*** slaweq has quit IRC | 15:20 | |
dansmith | yeah, that's what I mean | 15:20 |
dansmith | something other than devstack defaults | 15:20 |
dansmith | well, I'd not think this would change it, unless the base devstack job breaks with wsgi | 15:21 |
dansmith | maybe glance has a mod_wsgi job I can look at | 15:21 |
dansmith | doesn't look like it | 15:21 |
sean-k-mooney | which raiase the question how do they test this? | 15:22 |
aarents | sean-k-mooney: dansmith Should I need a reno for a new config parameter ? | 15:22 |
sean-k-mooney | looking at there zuul jobs they only use the normal tepest jobs | 15:22 |
dansmith | 2020-06-11 14:59:39.532624 [proxy:error] [pid 15204:tid 140023944947456] (111)Connection refused: AH00957: HTTP: attempt to connect to 10.0.1.13:19292 (10.0.1.13) failed | 15:22 |
dansmith | sean-k-mooney: yeah .... | 15:23 |
sean-k-mooney | dansmith: am its runing on a different port | 15:23 |
dansmith | yep | 15:23 |
sean-k-mooney | wsgi starting up on http://0.0.0.0:19191 | 15:23 |
sean-k-mooney | so close | 15:24 |
dansmith | GLANCE_URL is different for uwsgi | 15:24 |
dansmith | so maybe I need to set that | 15:24 |
sean-k-mooney | proably arent you having fun working on glance things... | 15:24 |
dansmith | or just change the service port | 15:24 |
dansmith | no :) | 15:24 |
dansmith | hmm, | 15:25 |
dansmith | 19191 is the registry port supposedly | 15:25 |
dansmith | and 19292 is the glance_service_port | 15:25 |
sangeet | I am trying to get some help on turning on TLS for nova. I was able to but nova-api is not interpreting the messages correctly. I had posed this opensatck channel as well but now response. Any help will be much apprecaited. Nova.conf is http://paste.openstack.org/show/794645 | 15:26 |
sangeet | 192.168.243.242 - - [11/Jun/2020 14:00:22] code 400, message Bad request syntax ('\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03O\xafjh.\xe4\xeb\x88\xfa\x9d\xff\x18\x813\x7f0\xa2\xe0\xb9J`\x15f\xb7=)\xf2}td~\x0e\x00\x00n\xc00\xc0,\xc02\xc0.\xc0/\xc0+\xc01\xc0-\x00\xa5\x00\xa3\x00\xa1\x00\x9f\x00\xa4\x00\xa2\x00\xa0\x00\x9e\xc0(\xc0$\xc0\x14\xc0') | 15:26 |
sangeet | Above log is from nova-api | 15:27 |
sean-k-mooney | sangeet: so nova itslef does not support tls you have to use a webserver to terminate the tls session before it hits the nova api | 15:28 |
sean-k-mooney | that kind of looks like the api is getting the encyped payload | 15:29 |
sangeet | Thank you sean-k-mooney | 15:45 |
gibi | nova meeting starts in 10 minutes on #openstack-meeting-3 | 15:49 |
*** ircuser-1 has joined #openstack-nova | 15:50 | |
*** markvoelker has joined #openstack-nova | 15:54 | |
*** markvoelker has quit IRC | 15:56 | |
*** belmoreira has quit IRC | 15:57 | |
*** rpittau is now known as rpittau|afk | 15:58 | |
jsuchome | dansmith: hi, thanks for your latest review, do you want me to reduce that comment about cyclic import or is there anything else? | 16:00 |
*** vishalmanchanda has quit IRC | 16:00 | |
*** gyee has joined #openstack-nova | 16:03 | |
openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job https://review.opendev.org/734184 | 16:05 |
*** ociuhandu has joined #openstack-nova | 16:06 | |
openstackgerrit | Dan Smith proposed openstack/nova-specs master: Update the testing section of the rbd-glance-multistore spec https://review.opendev.org/735192 | 16:13 |
dansmith | jsuchome: personally I think it should be reduced in line with the other examples I quoted | 16:14 |
dansmith | jsuchome: I don't remember if there was anything else, but I'll have to circle back next week.. trying to get some stuff done before I disappear tomorrow | 16:15 |
*** iurygregory has quit IRC | 16:15 | |
*** ociuhandu has quit IRC | 16:15 | |
*** sapd1 has quit IRC | 16:19 | |
*** boxiang has quit IRC | 16:19 | |
*** boxiang has joined #openstack-nova | 16:19 | |
sean-k-mooney | dansmith: this is the cyborg shelve path fyi https://review.opendev.org/#/c/729563/2 | 16:45 |
bauzas | gibi: fwiw, I'd have loved to volunteer for helping you during your holidays, but I'll be off at the same time | 16:46 |
gibi | bauzas: ack, thanks | 16:46 |
gibi | and don't worry | 16:47 |
dansmith | sean-k-mooney: ack, will try to keep that in mind in the coming weeks | 16:57 |
sean-k-mooney | i would like to see ci test it first so ill update this https://review.opendev.org/#/c/709641/ | 17:01 |
*** derekh has quit IRC | 17:01 | |
gibi | 19:08 -ChanServ(ChanServ@services.)- You are not authorized to (de)op gibi on | 17:08 |
gibi | #openstack-nova. | 17:08 |
gibi | so I need help to update the challen topic to point to the victori runway etherpad | 17:09 |
*** udesale_ has quit IRC | 17:19 | |
*** bbowen has quit IRC | 17:22 | |
*** artom has quit IRC | 17:26 | |
*** artom has joined #openstack-nova | 17:26 | |
sean-k-mooney | gibi: i think dansmith has op on this channel? | 17:30 |
*** ChanServ sets mode: +o dansmith | 17:31 | |
* dansmith speaks with a booming voice | 17:31 | |
*** dansmith changes topic to "Current runways: https://etherpad.openstack.org/p/nova-runways-victoria -- This channel is for Nova development. For support of Nova deployments, please use #openstack." | 17:31 | |
sean-k-mooney | hehe | 17:31 |
dansmith | good? | 17:31 |
sean-k-mooney | seams to work for me | 17:32 |
sean-k-mooney | and /topic works so yep | 17:32 |
gibi | dansmith: thanks | 17:34 |
*** ChanServ sets mode: -o dansmith | 17:34 | |
openstackgerrit | Dan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735205 | 17:44 |
dansmith | sean-k-mooney: did you have a stable commit with broken hashes in gerrit that is still yet to be fixed? | 17:45 |
dansmith | would like to use it to test | 17:45 |
*** mlavalle has quit IRC | 17:50 | |
*** nightmare_unreal has quit IRC | 17:51 | |
dansmith | sean-k-mooney: ^ ? | 18:04 |
sean-k-mooney | am im not sure if stephen fixed it | 18:05 |
sean-k-mooney | ill check | 18:05 |
dansmith | okay I wasn't sure if you had proposed it or someone else.. I don't see any recent -1ed patches on stable from you | 18:05 |
sean-k-mooney | https://review.opendev.org/#/c/728057/ was the one i think | 18:06 |
dansmith | okay that's not the one I was thiking of but yeah I see | 18:06 |
sean-k-mooney | https://review.opendev.org/#/c/728059/1 will be broken too | 18:06 |
sean-k-mooney | since it has not been update to pick up stephnes fix | 18:07 |
sean-k-mooney | also https://review.opendev.org/#/c/703115/ | 18:08 |
sean-k-mooney | again stephen updated it yesterday | 18:08 |
dansmith | I guess any patch with unmerged cherry picks would technically fail my check | 18:08 |
dansmith | so maybe that's enough | 18:08 |
*** factor has joined #openstack-nova | 18:08 | |
*** eharney has quit IRC | 18:09 | |
sean-k-mooney | ya basically if you dont wait for the preveious one to merge then it will be broken more then likely | 18:09 |
sean-k-mooney | well i guess only if you are asked to modify things | 18:09 |
dansmith | or you don't realize it fails for some other reason | 18:09 |
sean-k-mooney | ya | 18:09 |
dansmith | most people don't run tests on their backports, they wait until zuul tells them something is broken | 18:10 |
sean-k-mooney | ya if i have conflict i run the tests locally i often dont if it cleanly applied | 18:11 |
openstackgerrit | Dan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735208 | 18:11 |
dansmith | yup | 18:11 |
sean-k-mooney | it depens on the patch i guess and how much i think its likely to break | 18:11 |
dansmith | so the backport of my check should fail if it hasn't landed on master yet | 18:12 |
sean-k-mooney | do you plan to have each stable job just check the new branch | 18:13 |
sean-k-mooney | e.g. stien checks it merged on train which in trun check ussuri whcih checks master | 18:13 |
dansmith | I'm just checking that all referenced hashes exist on either master or stable/* which I think is fine | 18:14 |
sean-k-mooney | ya ok that should work | 18:14 |
*** tesseract has quit IRC | 18:15 | |
dansmith | if you backport something to stable before it's merged on master, zuul will fail it, which would also help us notice that the backport shouldn't be merged yet | 18:15 |
sean-k-mooney | so https://review.opendev.org/#/c/735208/1 should fail because https://review.opendev.org/#/c/735205/1 is not merged | 18:15 |
dansmith | might be too much for some people, but seems like a good idea to me | 18:15 |
dansmith | right | 18:15 |
sean-k-mooney | ya i kind of feel like that should be a standard zuul check | 18:15 |
dansmith | yeah | 18:16 |
dansmith | we'll see if it works | 18:16 |
sean-k-mooney | if it does maybe we could extend the merge-check job | 18:16 |
sean-k-mooney | ok im going to sign off soon. im takeing monday and tuseday off too so ill be back online on wednesday | 18:17 |
dansmith | cool, enjoy! | 18:17 |
openstackgerrit | Dan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735205 | 18:25 |
openstackgerrit | Dan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735208 | 18:27 |
*** bbowen has joined #openstack-nova | 18:30 | |
*** sean-k-mooney has quit IRC | 18:33 | |
*** jsuchome has quit IRC | 19:02 | |
*** ralonsoh has quit IRC | 19:15 | |
*** avolkov has quit IRC | 19:24 | |
gmann | elod: i was just reading log for cherry-pick things and the line you mentioned that "if patch on previous branch is not merged then it does not add cherry-pick from". but it does in both case either backported to all branch at once or one by one once merge in previous branch. | 19:39 |
elod | gmann: so if a patch has merged on master, and then i cherry-pick it to ussuri, and before it gets merged in ussuri i cherry pick it to stable stein *from* ussuri, you say that the patch in stein will have the 'cherry-picked' lines both from master and ussuri? | 19:43 |
gmann | elod: i think so but what i did previously was. master patch not merged yet and cherry picked to train, stein and all had 'cherry-piked from'.. | 19:44 |
*** hemna has quit IRC | 19:44 | |
elod | * s/stein/train/ | 19:46 |
elod | hmmm, I'll check it in the coming days, however I remember that it worked as I wrote | 19:47 |
elod | but I usually cherry-pick in cli, not via gerrit, so maybe I'm wrong | 19:48 |
*** slaweq has joined #openstack-nova | 19:49 | |
gmann | elod: i remember it does via gerrit where i end up having old commit id in 'cheery picked from' when i updated master patch. | 19:52 |
elod | gmann: I've cherry-picked a not merged patch from rocky to queens and it did not add the cherry-picked line: https://review.opendev.org/#/c/735222/ | 19:53 |
elod | did it via gerrit ui cherry-pick button | 19:54 |
gmann | elod: may be doing it from master it does? i did from master. | 19:56 |
*** hemna has joined #openstack-nova | 19:57 | |
elod | gmann: yes, that's possible. do you have an example from the past days maybe? | 19:57 |
gmann | let me check | 19:57 |
gmann | elod: this one, commit id in cherry pick from are wrong there :) - https://review.opendev.org/#/q/I96bb9fcb55f2a4fc3aefa01db70b45740db166c8 | 19:58 |
gmann | dansmith: how did you test your pep8 check? should we backport it and test with https://review.opendev.org/#/c/728059/ ? or any DNM patch on master and backport to ussuri | 20:00 |
dansmith | gmann: check the backport of the check -- it's self-checking :) | 20:01 |
gmann | ohk, just saw that | 20:02 |
elod | gmann: hmm, yes, it seems, if we backport from master, then it adds the cherry-picked line, even if that is not yet merged :S you are right | 20:02 |
elod | so to avoid this, we have to wait until patch on master gets merged ;) | 20:04 |
gmann | yeah that is best way and with dansmith proposed pep8 check will be helpful to wait or at least make checky pick from commit id upto date | 20:05 |
gmann | seems like pdf doc broken again? | 20:05 |
*** nweinber has quit IRC | 20:10 | |
dansmith | gmann: yeah seems like it | 20:11 |
gmann | i think we need to stop -vv option. | 20:11 |
openstackgerrit | Jiri Suchomel proposed openstack/nova master: Add ability to download Glance images into the libvirt image cache via RBD https://review.opendev.org/574301 | 20:13 |
dansmith | gmann: so are you cool with that pep8 backport check? | 20:19 |
gmann | dansmith: yeah, i am +2. i agree that is helpful. | 20:20 |
dansmith | sweet, I think so too | 20:21 |
dansmith | I'm always worried I'm going to approve something on N-2 before N-1 | 20:22 |
gmann | i wanted to move that to hacking :) if elod agree from stable backport perspective as general. | 20:22 |
dansmith | so this helps that, in addition to the hash change problem | 20:22 |
gmann | true and i recently forgot to update the commit id when i updated master one later | 20:22 |
dansmith | gmann: feel free to move it if you want it somewhere other than where I'm calling it from | 20:22 |
dansmith | gmann: ack, I'm sure I've done that in the past, despite trying to be good about it | 20:22 |
melwitt | gmann: why do you think -vv is causing the doc build failure? I'm looking at the log and don't yet see why it failed | 20:23 |
gmann | melwitt: not that, i was confused with echo message of too verbose :). | 20:24 |
melwitt | https://zuul.opendev.org/t/openstack/build/a678c8c3717d48a2aee415b1eea54a8e/log/sphinx-build-pdf.log#7604 | 20:24 |
gmann | its different problem. i can see lof of LaTeX warning but not sure that leads to error | 20:24 |
melwitt | yeah same | 20:24 |
*** slaweq has quit IRC | 20:26 | |
elod | gmann: yes, I think it would be good :) | 20:28 |
dansmith | elod: I guess we could, on stable, fail if we find no cherry-pick lines, unless the "StableOnly" tag is in the commit message | 20:30 |
elod | dansmith: sounds good! | 20:31 |
elod | is see e.g. train-only, stein-only, etc tags in commit messages, too, however. maybe those can be accepted next to stable-only as well | 20:36 |
gmann | +1, with 'StableOnly' tag it can work fine | 20:36 |
gmann | just stable-only is easy | 20:36 |
*** gyee has quit IRC | 20:36 | |
dansmith | maybe StableOnly: train ? | 20:36 |
elod | I'm OK with that | 20:36 |
dansmith | I'll just make it grep for StableOnly and we can decide | 20:37 |
gmann | but that may be backported to stein and so. | 20:37 |
gmann | like i start fixing from train till rocky | 20:37 |
gmann | devstack has such cases very often | 20:38 |
elod | StableOnly: train is a bit superfluous | 20:38 |
elod | let just be it simply stable-only or [stable-only] :) | 20:38 |
gmann | we can say 'StableOnly' means not coming from master but it can be backported to older stable | 20:39 |
openstackgerrit | Dan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735205 | 20:40 |
dansmith | gmann: yeah I think that's the way to do it | 20:40 |
dansmith | "StableOnly means it originated on some stable branch" | 20:40 |
gmann | yeah | 20:40 |
openstackgerrit | Dan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735208 | 20:41 |
openstackgerrit | Dan Smith proposed openstack/nova stable/ussuri: DNM: Testing a thing https://review.opendev.org/735241 | 20:41 |
*** cz3 is now known as WujaszekKrypta | 20:43 | |
*** mgariepy has quit IRC | 20:44 | |
*** WujaszekKrypta is now known as cz3 | 20:45 | |
*** hemna has quit IRC | 20:45 | |
*** mgariepy has joined #openstack-nova | 20:48 | |
*** artom has quit IRC | 21:13 | |
*** martinkennelly has quit IRC | 21:27 | |
*** martinkennelly has joined #openstack-nova | 21:27 | |
*** JamesBenson has quit IRC | 21:38 | |
openstackgerrit | Merged openstack/nova stable/train: Add functional test for bug 1550919 https://review.opendev.org/731272 | 21:48 |
openstack | bug 1550919 in OpenStack Compute (nova) train "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood) | 21:48 |
*** rcernin has joined #openstack-nova | 21:49 | |
*** markvoelker has joined #openstack-nova | 21:49 | |
*** markvoelker has quit IRC | 21:54 | |
*** dustinc has joined #openstack-nova | 21:55 | |
*** martinkennelly has quit IRC | 21:57 | |
*** martinkennelly has joined #openstack-nova | 21:58 | |
*** mriedem has left #openstack-nova | 21:59 | |
*** artom has joined #openstack-nova | 22:13 | |
openstackgerrit | Dan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735205 | 22:19 |
openstackgerrit | Dan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target https://review.opendev.org/735208 | 22:19 |
dansmith | elod: gmann: ^ I'm out tomorrow FYI | 22:19 |
*** threestrands has joined #openstack-nova | 22:20 | |
*** tkajinam has joined #openstack-nova | 22:46 | |
*** martinkennelly has quit IRC | 22:48 | |
*** markvoelker has joined #openstack-nova | 22:49 | |
*** markvoelker has quit IRC | 22:54 | |
*** sangeet has quit IRC | 22:56 | |
*** tosky has quit IRC | 23:10 | |
*** hamalq has quit IRC | 23:20 | |
*** hamalq has joined #openstack-nova | 23:20 | |
openstackgerrit | Merged openstack/nova stable/train: libvirt: Don't delete disks on shared storage during evacuate https://review.opendev.org/731273 | 23:33 |
*** rcernin has quit IRC | 23:48 | |
*** hamalq has quit IRC | 23:50 | |
*** rcernin has joined #openstack-nova | 23:51 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!