Monday, 2018-10-01

*** hshiina has joined #openstack-nova00:09
*** slaweq has joined #openstack-nova00:11
*** moshele has quit IRC00:12
*** slaweq has quit IRC00:16
*** tbachman has joined #openstack-nova00:31
*** efried has quit IRC00:38
*** efried has joined #openstack-nova00:38
openstackgerritTakashi NATSUME proposed openstack/nova master: Add API ref guideline for body text  https://review.openstack.org/60562800:43
*** hoangcx has joined #openstack-nova00:47
*** bhagyashris has joined #openstack-nova01:02
*** mhen has quit IRC01:08
*** mhen has joined #openstack-nova01:09
*** slaweq has joined #openstack-nova01:11
*** slaweq has quit IRC01:16
*** med_ has joined #openstack-nova01:18
*** mrsoul has joined #openstack-nova01:21
*** tetsuro has joined #openstack-nova01:44
*** med_ has quit IRC01:51
*** med_ has joined #openstack-nova02:10
*** tbachman has quit IRC02:16
*** tbachman has joined #openstack-nova02:37
*** markvoelker has joined #openstack-nova02:40
*** markvoelker has quit IRC02:45
*** markvoelker has joined #openstack-nova02:50
*** psachin has joined #openstack-nova02:56
*** bhagyashris has quit IRC03:09
*** Dinesh_Bhor has joined #openstack-nova03:31
*** pooja_jadhav has joined #openstack-nova03:55
*** Dinesh_Bhor has quit IRC03:56
*** med_ has quit IRC03:59
*** kevinbenton has quit IRC04:00
*** kevinbenton has joined #openstack-nova04:00
*** tetsuro has quit IRC04:03
*** pcaruana has joined #openstack-nova04:06
*** trungnv has joined #openstack-nova04:12
*** med_ has joined #openstack-nova04:14
*** pcaruana has quit IRC04:23
*** bhagyashris has joined #openstack-nova05:26
*** Dinesh_Bhor has joined #openstack-nova05:29
*** jaosorior has joined #openstack-nova05:37
*** pcaruana has joined #openstack-nova05:51
*** brinzhang has joined #openstack-nova05:53
*** med_ has quit IRC06:08
*** ratailor has joined #openstack-nova06:16
*** maciejjozefczyk has joined #openstack-nova06:18
*** ShilpaSD has joined #openstack-nova06:22
*** ShilpaSD has quit IRC06:22
openstackgerritBrin Zhang proposed openstack/nova master: Add microversion 2.67 to support volume_type  https://review.openstack.org/60639806:24
*** ShilpaSD has joined #openstack-nova06:26
*** skatsaounis has quit IRC06:33
*** markvoelker has quit IRC06:33
*** skatsaounis has joined #openstack-nova06:34
*** markvoelker has joined #openstack-nova06:34
*** markvoelker has quit IRC06:38
openstackgerritSam Morrison proposed openstack/python-novaclient master: Fix up userdata argument to rebuild.  https://review.openstack.org/60534106:45
*** slaweq has joined #openstack-nova06:52
bauzasgood morning Nova07:01
*** ralonsoh has joined #openstack-nova07:02
*** takashin has quit IRC07:07
*** takashin has joined #openstack-nova07:15
*** helenafm has joined #openstack-nova07:22
*** sapd1 has quit IRC07:26
*** Dinesh_Bhor has quit IRC07:33
*** markvoelker has joined #openstack-nova07:34
*** Dinesh_Bhor has joined #openstack-nova07:35
*** jpena|off is now known as jpena07:40
*** alexchadin has joined #openstack-nova07:43
*** Dinesh_Bhor has quit IRC07:44
*** zigo has joined #openstack-nova07:46
openstackgerritChristoph Manns proposed openstack/nova stable/rocky: Fix stacktraces with redis caching backend  https://review.openstack.org/60689507:50
openstackgerritChristoph Manns proposed openstack/nova stable/queens: Fix stacktraces with redis caching backend  https://review.openstack.org/60689607:52
openstackgerritBrin Zhang proposed openstack/nova master: Add microversion 2.67 to support volume_type  https://review.openstack.org/60639807:53
*** alexchadin has quit IRC08:03
*** mdbooth has joined #openstack-nova08:06
*** TuanDA has joined #openstack-nova08:11
*** tssurya has joined #openstack-nova08:11
*** sapd1 has joined #openstack-nova08:17
*** mdbooth has quit IRC08:18
*** giblet is now known as gibi08:19
*** lpetrut has joined #openstack-nova08:28
*** pooja-jadhav has joined #openstack-nova08:38
*** pooja_jadhav has quit IRC08:40
*** finucannot is now known as stephenfin08:43
*** pooja_jadhav has joined #openstack-nova08:44
*** pooja-jadhav has quit IRC08:44
*** pooja-jadhav has joined #openstack-nova08:44
*** pooja_jadhav has quit IRC08:46
*** mgoddard has joined #openstack-nova08:46
*** Shilpa has joined #openstack-nova08:47
*** pooja_jadhav has joined #openstack-nova08:49
*** ShilpaSD has quit IRC08:50
*** pooja-jadhav has quit IRC08:52
*** trungnv has quit IRC08:54
*** ttsiouts has joined #openstack-nova08:55
*** ttsiouts has quit IRC08:56
*** ttsiouts has joined #openstack-nova08:56
*** Dinesh_Bhor has joined #openstack-nova08:57
*** derekh has joined #openstack-nova09:00
*** Dinesh_Bhor has quit IRC09:05
*** sapd1_ has joined #openstack-nova09:06
*** sapd1 has quit IRC09:06
*** Dinesh_Bhor has joined #openstack-nova09:06
openstackgerritMatthew Booth proposed openstack/nova master: Raise error on timeout in wait_for_versioned_notifications  https://review.openstack.org/60485909:07
openstackgerritMatthew Booth proposed openstack/nova master: Add regression test for bug 1550919  https://review.openstack.org/59173309:08
openstackbug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9)09:08
openstackgerritMatthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate  https://review.openstack.org/57884609:08
*** dpawlik has joined #openstack-nova09:11
kashyaptestone nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_next_min_qemu_version_ok09:11
kashyapErr, disregard that09:12
openstackgerritTheodoros Tsioutsias proposed openstack/nova-specs master: Add PENDING vm state  https://review.openstack.org/55421209:13
ttsioutsjohnthetubaguy: I tried to refresh the spec ^09:13
*** priteau has joined #openstack-nova09:16
*** psachin has quit IRC09:17
*** Dinesh_Bhor has quit IRC09:22
*** alex_xu has quit IRC09:38
*** jmccarthy has joined #openstack-nova09:48
*** dtantsur|afk is now known as dtantsur09:48
*** jmccarthy has quit IRC09:49
*** tetsuro has joined #openstack-nova09:52
*** bhagyashris has quit IRC09:52
*** Dinesh_Bhor has joined #openstack-nova09:53
*** brinzhang has quit IRC09:59
*** mgoddard has quit IRC09:59
*** TuanDA has quit IRC10:00
*** hshiina has quit IRC10:01
sean-k-mooneyo/10:03
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Update the contributor guide  https://review.openstack.org/60692610:04
sean-k-mooneystephenfin: when you get a chance can you run your eyes over https://review.openstack.org/#/c/604992/3. looks like an easy addtion to me.10:05
stephenfinack10:05
sean-k-mooneystephenfin: i think we need to port other vif types from kuyr-kubernetes too but they can be in different patches10:06
*** Shilpa has quit IRC10:23
*** ttsiouts has quit IRC10:28
sean-k-mooneyralonsoh: hi are you around today?10:35
ralonsohsean-k-mooney: very well, fighting with DVR and FIP (and SRIOV)10:35
*** Luzi has joined #openstack-nova10:36
stephenfinsean-k-mooney: Also, "i prefer explcit else clauses"10:36
stephenfinsean-k-mooney: Just wanted you to know you're wrong10:37
stephenfinsean-k-mooney: That is all.10:37
stephenfin:)10:37
sean-k-mooneyralonsoh: if you bussy i can respin https://review.openstack.org/#/c/605422/210:37
ralonsohsean-k-mooney: sure, I'll do this now10:38
sean-k-mooneystephenfin: hehe well we can agree to disagree on that. it can help prevent some types of errors when you are maintaing the fucntion in the future but also consumes indentation space so is not allways desireable10:38
*** tbachman has quit IRC10:48
openstackgerritStephen Finucane proposed openstack/nova master: doc: Rewrite the console doc  https://review.openstack.org/60614810:48
openstackgerritStephen Finucane proposed openstack/nova master: conf: Allow 'nova-xvpvncproxy' to be called with CLI args  https://review.openstack.org/60692910:48
*** Dinesh_Bhor has quit IRC10:49
openstackgerritRodolfo Alonso Hernandez proposed openstack/os-vif master: Remove IPTools deprecated implementation  https://review.openstack.org/60542210:51
ralonsohsean-k-mooney: done10:51
sean-k-mooneyralonsoh: thanks :)10:55
*** dave-mccowan has joined #openstack-nova10:56
*** ttsiouts has joined #openstack-nova11:04
openstackgerritTheodoros Tsioutsias proposed openstack/nova-specs master: Enable rebuild for instances in cell0  https://review.openstack.org/55421811:11
*** erlon has joined #openstack-nova11:14
openstackgerritTheodoros Tsioutsias proposed openstack/nova-specs master: Add PENDING vm state  https://review.openstack.org/55421211:15
*** mgoddard has joined #openstack-nova11:23
*** jaosorior has quit IRC11:27
openstackgerritMerged openstack/nova stable/rocky: Use INFO for logging no allocation candidates  https://review.openstack.org/60577111:28
*** ratailor has quit IRC11:29
*** jpena is now known as jpena|lunch11:30
*** tetsuro has quit IRC11:30
openstackgerritMerged openstack/os-vif master: Add support for Windows network commands  https://review.openstack.org/48740511:48
*** phillu has joined #openstack-nova11:53
*** markvoelker has quit IRC11:56
kashyapstephenfin: On that version bump thing for libvirt, I just looked at "what's involved" to clean up12:05
kashyapstephenfin: I need to address at least 10 unit tests.  Will get to it this week.  Currently buried in something a bit more urgent12:06
*** jaosorior has joined #openstack-nova12:10
*** dave-mccowan has quit IRC12:22
*** tbachman has joined #openstack-nova12:25
*** ttsiouts has quit IRC12:26
*** mdbooth has joined #openstack-nova12:26
*** ttsiouts has joined #openstack-nova12:30
stephenfinkashyap: Yup, it's low priority and can be done anytime this cycle. No rush!12:30
*** jpena|lunch is now known as jpena12:31
kashyapYep.12:31
kashyapThen, I might even wait until I'm through all the CPU modelling stuff that I'm looking into (until 27 Oct)12:32
*** med_ has joined #openstack-nova12:33
*** lbragstad has joined #openstack-nova12:39
mdboothbauzas: LM failure on https://review.openstack.org/#/c/604400/12:40
*** lbragstad has quit IRC12:40
mdboothSo, the exact failure is unlikely to be relevant. Chances are that this is just the first thing which touched that host, and the host was broken by the evacuate test we just ran.12:41
bauzasmdbooth: so, you got a failure for req-dd88c142-4710-49d0-b6e9-594c6a1e89f6 (live-migrate action) because of http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/screen-n-sch.txt.gz#_Sep_28_20_39_55_71205812:41
mdboothbauzas: Yes12:41
bauzasrather http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/screen-n-sch.txt.gz#_Sep_28_20_39_55_71174012:42
mdboothSo, the question is: why does the computefilter think the host is down?12:42
bauzasmdbooth: because the db record is old12:42
bauzasmdbooth: I mean the updated_at filed12:42
bauzasfield*12:42
mdboothbauzas: Right, but in the compute logs I see the host joining the service group within the last 60 secs12:42
bauzashttps://github.com/openstack/nova/blob/master/nova/scheduler/filters/compute_filter.py#L4412:42
mdboothHowever, I note that it doesn't update the record at that point, just schedules it to be updated in 5 seconds12:43
mdboothSo I then got to wondering why it doesn't seem to have been updated 5 seconds later.12:43
mdboothI was wondering if perhaps a long running init_host might be related?12:44
mdboothBecause that's feasible following the evacuate test12:44
bauzashttps://github.com/openstack/nova/blob/master/nova/servicegroup/drivers/db.py#L7312:45
mdboothHowever, threadgroup.add_timer seems to be unconditional12:45
bauzasand https://github.com/openstack/nova/blob/master/nova/servicegroup/drivers/db.py#L6012:45
*** mchlumsky has joined #openstack-nova12:45
mdboothbauzas: Right, so it hasn't updated its record within the last 60 seconds12:45
bauzasjust checking, it's 60sec http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/screen-n-sch.txt.gz#_Sep_28_20_25_02_43964212:46
*** dave-mccowan has joined #openstack-nova12:46
bauzasyup12:46
mdboothBut given that it joined the servicegroup http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/subnode-2/screen-n-cpu.txt.gz?#_Sep_28_20_39_12_27735512:46
bauzasnow, I'm checking the compute logs12:46
mdboothI can't see why not.12:46
bauzasit's a periodic task IIRC12:47
mdboothLooks like it's a threadgroup timer12:47
mdboothWhich would be eventlet-fu12:47
mdboothBut I wonder if its execution could be blocked by, eg, a long-running periodic task12:48
sean-k-mooneymdbooth: we had a downstream bugs(3) related to osp7 upgrades that saw similar behavior. compute filter elimated host becase host was still down after upgrade. in the downstream case it was causeing tempset test to fail as the heartbeat had not been recived yet and the host was down. this seams similar.12:49
mdboothsean-k-mooney: The immediate cause would certainly be the same. I wonder if the underlying cause is, though.12:50
mdboothDid you get to the bottom of it?12:50
sean-k-mooneymdbooth: no in the down stream case we saw that subsequent schduling attempts suceeded as the heatbeat had been processed so we closed as not a but as the cloud had not stablised after upgrade before running the tests12:51
sean-k-mooneye.g. it corrected it self a few seconds after teh tempest test was started12:52
*** jroll has quit IRC12:54
*** jroll has joined #openstack-nova12:55
*** mrjk_ has joined #openstack-nova12:57
bauzasmdbooth: honestly, I don't see the problem with the service group .join() call12:57
*** mrjk__ has quit IRC12:58
*** artom has quit IRC12:58
bauzasmdbooth: is this only a transient issue ?12:58
bauzasmdbooth: if that's more than that, try to have a DNM patch adding some log like herehttps://github.com/openstack/nova/blob/master/nova/servicegroup/drivers/db.py#L9112:59
*** mdbooth has quit IRC12:59
*** mdbooth has joined #openstack-nova13:01
*** beagles is now known as beagles_mtg13:07
gryfis anyone know what is happening with this blueprint: https://blueprints.launchpad.net/nova/+spec/numa-aware-live-migration ?13:11
mnasera little idle, simple clean backport if someone has a second or two: https://review.openstack.org/#/q/I811e84af46d678c3fdbf94ee400eabe659fc3d4e13:12
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: WIP: Fix test_instance_action functional test failure  https://review.openstack.org/60694713:14
mdboothFolks, anybody else hitting the bug in parallel evacuate in CI I have a fix up for it here: https://review.openstack.org/#/c/605436/13:17
*** dave-mccowan has quit IRC13:17
mdboothIt's not a one line fix, though, so you'll need to block out some time to review it. However, probably not as much time as we're losing to this test failing continuously.13:18
*** ttsiouts has quit IRC13:21
*** dave-mccowan has joined #openstack-nova13:21
*** awaugama has joined #openstack-nova13:25
*** ttsiouts has joined #openstack-nova13:27
*** cdent has joined #openstack-nova13:28
mdboothHmm, I just had to recheck all 3 commits from this morning for that bug :/13:29
*** erlon has quit IRC13:35
*** erlon has joined #openstack-nova13:36
*** mchlumsky has quit IRC13:37
*** mchlumsky has joined #openstack-nova13:39
*** mdbooth has quit IRC13:41
*** dklyle has joined #openstack-nova13:47
*** jaosorior has quit IRC13:48
efriedn-sch/placement meeting in 7 minutes in #openstack-meeting-alt13:53
*** tetsuro has joined #openstack-nova13:55
*** artom has joined #openstack-nova13:57
*** markvoelker has joined #openstack-nova14:09
*** markvoelker has quit IRC14:15
*** tbachman has quit IRC14:15
*** leakypipes is now known as jaypipes14:15
*** dpawlik has quit IRC14:15
*** maciejjozefczyk has quit IRC14:15
*** maciejjozefczyk has joined #openstack-nova14:16
*** dpawlik has joined #openstack-nova14:16
*** markvoelker has joined #openstack-nova14:17
*** dpawlik has quit IRC14:20
*** beekneemech is now known as bnemec14:21
*** itlinux has quit IRC14:25
*** markvoelker has quit IRC14:26
*** lpetrut has quit IRC14:27
*** ttsiouts has quit IRC14:28
*** ttsiouts has joined #openstack-nova14:28
*** Luzi has quit IRC14:29
*** mlavalle has joined #openstack-nova14:33
*** tetsuro has quit IRC14:42
*** hshiina has joined #openstack-nova14:44
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Fix test_instance_action functional test failure  https://review.openstack.org/60694714:45
*** cfriesen has joined #openstack-nova14:57
openstackgerritChris Dent proposed openstack/nova master: Do not encode server create location reponse header  https://review.openstack.org/60696714:57
*** kukacz_ is now known as kukacz14:59
*** takashin has left #openstack-nova15:01
*** jmlowe has joined #openstack-nova15:01
*** ttsiouts has quit IRC15:02
*** lpetrut has joined #openstack-nova15:02
*** med_ has quit IRC15:03
*** dpawlik has joined #openstack-nova15:03
*** macza has joined #openstack-nova15:07
*** mriedem has joined #openstack-nova15:07
*** dpawlik has quit IRC15:08
*** munimeha1 has joined #openstack-nova15:11
*** lpetrut has quit IRC15:18
*** lpetrut has joined #openstack-nova15:18
*** dpawlik has joined #openstack-nova15:19
*** itlinux has joined #openstack-nova15:21
*** dpawlik has quit IRC15:24
*** mriedem has left #openstack-nova15:24
*** mriedem has joined #openstack-nova15:24
mriedembauzas: let's not run the linuxbridge job on all nova changes https://review.openstack.org/#/c/606616/15:25
mriedem^ fixes a regression15:25
bauzasmriedem: /me looks15:25
bauzasmriedem: oh shit, reviewing the py3-first changes was horribly difficult, my bad if I made a regression :(15:26
mriedemit's ok, one line within hundreds, hard to spot15:26
bauzasmriedem: https://review.openstack.org/#/c/606616/1/.zuul.yaml@186 why this line couldn't be only needed ?15:27
bauzas^(?!nova/network/.*)(?!nova/virt/libvirt/vif.py).*$ means 'all but nova/network or nova/virt/libvirt/vif.py AFAIU15:28
bauzasbut anyway, a nit15:28
bauzas+Wd15:29
mriedemyeah could do that in a follow up15:30
*** pcaruana has quit IRC15:30
*** ivve has joined #openstack-nova15:33
openstackgerritMatt Riedemann proposed openstack/nova master: Drop nova-multiattach job  https://review.openstack.org/60698115:39
openstackgerritChris Dent proposed openstack/nova master: Do not encode server create location response header  https://review.openstack.org/60696715:41
*** lpetrut has quit IRC15:42
openstackgerritMatt Riedemann proposed openstack/nova master: Drop nova-multiattach job  https://review.openstack.org/60698115:47
*** hshiina has quit IRC15:49
*** med_ has joined #openstack-nova15:50
openstackgerritMatt Riedemann proposed openstack/nova master: Drop nova-multiattach job  https://review.openstack.org/60698115:52
openstackgerritMatt Riedemann proposed openstack/nova master: Remove redundant irrelevant-files from neutron-tempest-linuxbridge  https://review.openstack.org/60698915:55
openstackgerritBalazs Gibizer proposed openstack/nova master: Ignore forcing of live migration for nested instance  https://review.openstack.org/60578515:55
openstackgerritBalazs Gibizer proposed openstack/nova master: Consider nested allocations during allocation cleanup  https://review.openstack.org/60605015:55
openstackgerritBalazs Gibizer proposed openstack/nova master: Ignore forcing of evacuation for nested instance  https://review.openstack.org/60611115:55
openstackgerritBalazs Gibizer proposed openstack/nova master: Run negative server moving tests with nested RPs  https://review.openstack.org/60412515:55
*** ttsiouts has joined #openstack-nova15:56
*** ttsiouts has quit IRC15:56
*** ttsiouts has joined #openstack-nova15:56
*** dave-mccowan has quit IRC15:58
*** ttsiouts has quit IRC16:01
*** beagles_mtg is now known as beagle16:02
openstackgerritStephen Finucane proposed openstack/nova master: doc: Rewrite the console doc  https://review.openstack.org/60614816:07
openstackgerritStephen Finucane proposed openstack/nova master: conf: Allow 'nova-xvpvncproxy' to be called with CLI args  https://review.openstack.org/60692916:07
openstackgerritStephen Finucane proposed openstack/nova master: doc: Add minimal documentation for RDP consoles  https://review.openstack.org/60699216:07
openstackgerritStephen Finucane proposed openstack/nova master: doc: Add minimal documentation for MKS consoles  https://review.openstack.org/60699316:07
openstackgerritSurya Seetharaman proposed openstack/nova master: Update --max-rows parameter description for archive_deleted_rows  https://review.openstack.org/60699516:10
*** helenafm has quit IRC16:12
*** lpetrut has joined #openstack-nova16:14
*** dpawlik has joined #openstack-nova16:17
stephenfinThis console documentation stuff is a damn rabbit hole :/16:21
*** dpawlik has quit IRC16:22
*** tbachman has joined #openstack-nova16:22
*** tbachman has quit IRC16:27
*** gyee has joined #openstack-nova16:30
*** moshele has joined #openstack-nova16:32
*** dpawlik has joined #openstack-nova16:33
*** tbachman has joined #openstack-nova16:36
*** dpawlik has quit IRC16:38
tssuryadansmith: thanks for the review on nova list patch; would you happen to have some ideas for https://review.openstack.org/#/c/591658/7/nova/compute/api.py@2327 ? melwitt and I briefly discussed the possibility to add the "type of exception" instead of the sentinel being returned..16:39
openstackgerritStephen Finucane proposed openstack/nova master: doc: Rewrite the console doc  https://review.openstack.org/60614816:39
openstackgerritStephen Finucane proposed openstack/nova master: doc: Add minimal documentation for RDP consoles  https://review.openstack.org/60699216:39
openstackgerritStephen Finucane proposed openstack/nova master: doc: Add minimal documentation for MKS consoles  https://review.openstack.org/60699316:39
openstackgerritStephen Finucane proposed openstack/nova master: conf: Allow 'nova-xvpvncproxy' to be called with CLI args  https://review.openstack.org/60692916:39
tssuryadansmith: sorry for the wrong timing of the ping, I have to leave office now, but I will update the nova list patch asap and see if I can just catch the InstanceNotFound exception seperately for the above and the then mabe we can disuss a generic solution.16:44
tssuryamaybe*16:44
*** med_ has quit IRC16:45
*** moshele has quit IRC16:47
*** dpawlik has joined #openstack-nova16:49
*** zzzeek_ has joined #openstack-nova16:53
*** dpawlik has quit IRC16:53
*** dave-mccowan has joined #openstack-nova16:54
*** tssurya has quit IRC16:54
*** pcaruana has joined #openstack-nova16:56
*** derekh has quit IRC17:00
*** dtantsur is now known as dtantsur|afk17:03
*** mriedem has quit IRC17:04
*** mgoddard has quit IRC17:04
*** jpena is now known as jpena|off17:11
*** moshele has joined #openstack-nova17:17
*** ralonsoh has quit IRC17:27
*** sapd1 has joined #openstack-nova17:30
*** macza has quit IRC17:41
*** macza has joined #openstack-nova17:41
*** moshele has quit IRC17:42
*** mgoddard has joined #openstack-nova17:42
*** lpetrut has quit IRC17:49
*** imacdonn has quit IRC17:51
*** moshele has joined #openstack-nova17:52
*** imacdonn has joined #openstack-nova17:52
*** jmlowe has quit IRC17:55
*** jmlowe has joined #openstack-nova18:06
*** mgoddard has quit IRC18:06
*** moshele has quit IRC18:07
*** imacdonn has quit IRC18:08
*** dpawlik has joined #openstack-nova18:10
*** markvoelker has joined #openstack-nova18:15
*** dpawlik has quit IRC18:15
*** mgoddard has joined #openstack-nova18:16
*** mriedem has joined #openstack-nova18:20
*** imacdonn has joined #openstack-nova18:21
*** markvoelker has quit IRC18:24
cdentthis very minor bug fix is just about ready to pass all its tests, if anyone is looking for something easy to review: https://review.openstack.org/#/c/606967/18:24
*** dpawlik has joined #openstack-nova18:26
*** jmlowe has quit IRC18:27
efriedcdent: Your comments is garbledes18:28
cdentagain?18:29
cdentah, misplaced )18:30
efriedis there be a tests somewhere that demonstrate the bug?18:30
cdentthe tests I changed?18:31
*** dpawlik has quit IRC18:31
efriedno, I mean where were you seeing the concatenation errors you mention in the bug report?18:31
efriedthe changed tests just prove that you changed the code in the way that you changed the code :)18:31
mriedemcdent: question inlline18:32
mriedem*inline18:32
cdentefried: exactly, it changed from being bytes to being not bytes. the code violated the spec, I referenced the spec18:33
efriedyeah, cool, but the bug report says you actually encountered a real problem as a result of the code being wrong. Something about concatenating the str url with the unicode header under wsgi intercept?18:34
cdentefried: in one of the iterations of https://review.openstack.org/#/c/601614/18:36
cdenthttp://logs.openstack.org/14/601614/25/check/placement-tempest-gabbi/d9a37a4/controller/logs/screen-n-api.txt.gz#_Sep_29_09_57_29_46301218:36
mriedemdansmith: want to this the queens version of that add host to aggregate case revert change (and the one after it): https://review.openstack.org/#/c/605266/118:37
cdentefried: but mriedem raises a good question. It appears like things may be more broken than I hoped (I guess I never learn)18:37
mriedems/this/hit/18:37
dansmithmriedem: omg I so want18:37
*** sapd1 has quit IRC18:38
dansmithheh18:41
dansmithmriedem: lee was +2 then removed.. I went to +W but then figured I must be missing something?18:41
mriedemi think melwitt asked the same thing last week18:41
mriedemi can only assume b/c the rocky change hadn't merged yet18:42
mriedemyou might as well re-apply that +W b/c it's in the gate18:42
dansmithgah okay18:43
mriedemlet's use the rule of stable core backport == +218:43
mriedemdansmith:  in case you haven't seen my email yet, i'm trying to flush the open ocata backports so we can EM https://etherpad.openstack.org/p/nova-ocata-em18:43
dansmithokay18:44
mriedemif you want to do some mindless stable branch reviews on your first day back18:44
*** jmlowe has joined #openstack-nova18:44
cdentefried: sigh. you know what you were saying about "senile" last week? Those gabbits are not using wsgi intercept at all that's a live devstack. I guess I will spin one up18:44
*** Sukhdev has joined #openstack-nova18:45
efriedglad I could help in not at all a way I intended or even understand.18:45
cdentefried: what it means in this case it is that it is likely the problem is present in "real" situations18:48
cdentso now I'm going to check18:48
efriedcool18:48
cfriesenmriedem: once you're done the ocata stuff, any chance you could look at https://review.openstack.org/#/c/605164/ ?  since you triaged the bug :)18:52
*** jitek4 has left #openstack-nova18:52
mriedemthat was the old me18:53
mriedemi also tried fixing that and couldn't recreate it in the functional test...18:54
cfriesenso, need to reproduce for real first on master?18:57
mriedemnot necessarily19:04
mriedemthe fix is the same thing i was doing,19:05
mriedemi just couldn't get a recreate in the functional test,19:05
mriedembut was also doing it while at one of my kids activity things19:05
mriedemlike a champion dad19:05
cfriesenmriedem: you were actually doing the delete/purge API calls, right?  (as opposed to this patch which is doing a direct DB operation)19:07
mriedemi deleted the marker record from the instances table but probably forgot to archive it19:09
mriedemcfriesen: i just need to wrap up what i'm doing locally first19:20
cfriesenmriedem: no worries19:21
*** tbachman has quit IRC19:22
*** erlon has quit IRC19:33
openstackgerritJack Ding proposed openstack/nova master: Add HPET timer support for x86 guests  https://review.openstack.org/60590219:39
*** Sukhdev has quit IRC19:42
cdentmriedem, efried : the location header issue is weirder/worse than I feared: https://bugs.launchpad.net/nova/+bug/179542519:42
openstackLaunchpad bug 1795425 in OpenStack Compute (nova) "create server api sends location header as bytestring in py3" [Undecided,In progress] - Assigned to Chris Dent (cdent)19:42
gryfhm. what is the status of live migration of numa nodes? I saw a really old change set started by Nikola, than a BP which trying to pick of the solution description in one place, but implementation didn't make it to Rocky19:43
openstackgerritJack Ding proposed openstack/nova master: Add HPET timer support for x86 guests  https://review.openstack.org/60590219:43
mriedemgryf: ask artom19:44
gryfalso I get the reading of etherpad from ptg, and there was an agreement for modeling numa nodes as RPs19:44
dansmithgryf: artom would love to talk to you about it at length over a candle-lit dinner19:44
sean-k-mooneycdent: im a little confused by that bug. what spec are you refering to that is violated pep 3333?19:44
gryfas far as I can tell, bauzas was the one who is dealing with that modeling, right?19:45
openstackgerritJack Ding proposed openstack/nova master: Correct instance port binding for rebuilds/reboots  https://review.openstack.org/60384419:45
cdentsean-k-mooney: initially I thought the code is violating pep333319:45
gryfdansmith, I'd like to skip romantic dinners ;)19:45
cdentsean-k-mooney: so when I "the spec" I'm referring to pep333319:45
openstackgerritJack Ding proposed openstack/nova master: Handle missing marker during online data migration  https://review.openstack.org/60516419:45
cdentbut the problem is more messy than just that19:45
melwittmriedem: two patches to un-deprecate nova-consoleauth start here https://review.openstack.org/60525019:46
gryfso, artom, you around?19:46
sean-k-mooneycdent: so " the content of native strings must still be translatable to bytes via the Latin-1 encoding" woudl seam to be in conflict with the http spec which allow unicode in URL such as the location header19:46
mriedemmelwitt: so you plan on undeprecating in stein, and backporting that change to stable/rocky, then re-deprecating in stein?19:49
cdentit's the source of much argument19:49
mriedemor only re-deprecate once cells v1 is gone?19:49
melwittmriedem: yes19:49
cdentsean-k-mooney: my understanding, however, was that you couldn't put unicode codepoints in headers, but you could have urls that encode unicode, which is not quite the same thing19:49
melwittmriedem: and I'll stack the patches that remove use of nova-consoleauth by non-cells v1 code on top of the re-deprecation patches19:50
sean-k-mooneycdent: that definetly used to be the case but i was under the impression that restiction was relaxed in the last few years19:51
cdentsean-k-mooney: that may be, but wsgi itself has not caught up/changed and has its own contract19:51
mriedemmelwitt: have you cherry picked those changes to stable/rocky and built the release notes to see if they are going to show up weird at all?19:56
sean-k-mooneycdent: ok so https://www.rfc-editor.org/rfc/rfc3986.txt relaxes the constraitn but not entirely. "on-ASCII19:56
sean-k-mooney   characters must first be encoded according to UTF-8 [STD63], and then19:56
sean-k-mooney   each octet of the corresponding UTF-8 sequence must be percent-19:56
sean-k-mooney   encoded to be represented as URI characters19:56
sean-k-mooney"19:56
mriedemrewriting release note history makes me uneasy19:56
sean-k-mooneycdent: so basically if you want to transport utf-8/unicode charaters in a url you must htmlencode the underlying utf-8 byte syequence rather then use the raw utf-8 charater stream19:58
cdentsean-k-mooney: urlencode, not htmlencode, but yes, pretty much20:00
melwittmriedem: no -- but let me do that and verify it looks ok20:00
sean-k-mooneycdent: sorry yes, i could not remember the normal name for the % encoding20:01
melwittmriedem: I guess, did you have something else in mind when you were thinking of un-deprecation? i.e. just modifying the code side and not touching the release note side?20:02
sean-k-mooneymelwitt: if its going to "remain" deprecated in  stein after the modification is there any reason to un deprecate it?20:04
melwittsean-k-mooney: just a bad signal to folks moving to rocky. it will make them think they can stop running nova-consoleauth in rocky, but I found the code still requires it20:05
sean-k-mooneymelwitt: oh  ok i was missing that context. the console auth service is technically optional so i dont neccicarly think that is a problem20:08
openstackgerritmelanie witt proposed openstack/nova stable/rocky: Revert "Deprecate the nova-consoleauth service"  https://review.openstack.org/60703720:08
openstackgerritmelanie witt proposed openstack/nova stable/rocky: Un-deprecate the nova-consoleauth service  https://review.openstack.org/60703820:08
sean-k-mooneyfor example i belive you can configure nova/libvirt dirver to have no console correct?20:09
melwittsean-k-mooney: a couple of users already tripped over it, caused confusion and wasted time20:09
sean-k-mooneymelwitt: right, then other than the possible weirdness with the release notes i guess undeprecating and deprecating in stein is proablly a better user experice.20:10
melwittyeah. I didn't consider the possibility of un-depcrecating it in the code only, and leaving the release notes alone. I had been thinking I should back out every mention of it20:11
artomgryf, hey, so, I can't really talk now, but I'll be around later tonight, or tomorrow20:14
openstackgerritBalazs Gibizer proposed openstack/nova master: Ignore forcing of evacuation for nested instance  https://review.openstack.org/60611120:14
openstackgerritBalazs Gibizer proposed openstack/nova master: Run negative server moving tests with nested RPs  https://review.openstack.org/60412520:14
gryfartom, tomorrow will be fine20:15
*** MasterofJOKers has quit IRC20:16
mriedemmelwitt: i didn't really have something specific in mind, because it's a weird issue20:19
mriedemif nova-compute in stein is expecting stuff in the consoleauth service, it shouldn't be deprecated20:19
mriedembut is that only true if the token isn't in the cell db?20:19
mriedemor you're using cells v1 - which is itself deprecated20:20
mriedemas an aside, it seems we should have something in grenade for consoles....create a console on a server on the old side and verify the console token is still valid on the new side...20:20
mriedemi'm not sure how easy that is though20:21
melwittwell, in rocky the console proxy by default will _not_ fall back to nova-consoleauth -- you have to use the [workarounds]enable_consoleauth to make it look there. the snag was in the fact that compute/api stores the token in both nova-consoleauth and the database, and that's an artifact that I didn't understand from the original patches from paulmurray20:26
melwittthat's the non-cells v1 case. for cells v1, it will always need nova-consoleauth20:26
*** dpawlik has joined #openstack-nova20:27
*** Sukhdev has joined #openstack-nova20:30
*** dpawlik has quit IRC20:32
sean-k-mooneymelwitt: so the only case where you always need console auth is cells v1 which is deprecated and for non cells v1 you only need it if you enable a workaround which is off by default20:38
melwittsean-k-mooney: non cells v1 you need it in rocky even if you didn't enable the workaround because compute/api is still storing a token auth in nova-consoleauth regardless. the console proxy doesn't have to have nova-consoleauth20:39
sean-k-mooneymelwitt: oh ok i see thank i  missed that bit20:40
mriedemi'm about to get kicked out of this coffee shop20:40
melwittit's confusing20:40
mriedemmaybe write something up to the ML to help digest and sort it out?20:41
mriedemso others are on the same page?20:41
dansmithmelwitt: would changing the default just be a better route?20:41
mriedemand give operators a heads up before they get to rocky20:41
dansmithwe _want_ it deprecated and you shouldn't need it right?20:41
dansmithor is there something we actually missed in the conversion to service-less?20:42
dansmithif the latter, why not just fix and backport that?20:42
*** mriedem has quit IRC20:42
melwittdansmith: yeah, but to "fix" that, we'd have to remove the calls in compute/api that regardlessly store the token auth in both the consoleauth service _and_ the database. me missing that fact is why it got deprecated in the first place20:42
melwittthe proxy code was optionally using nova-consoleauth, the compute/api code was unconditionally using nova-consoleauth20:43
dansmithis it just not making an rpc call unless the workaround is on?20:43
*** pcaruana has quit IRC20:43
melwittyeah, the proxy won't fall back on nova-consoleauth if the workaround option isn't enabled20:44
dansmithwhat I mean is,20:44
dansmithcan we just not make the rpc calls from the compute api code unless the workaround is enabled?20:44
dansmithwill that fix the problem such that they don't have to run the service?20:44
melwittyeah, I think we could do that20:44
dansmiththat seems massively better to me than confusing everyone (including some of us here) with a backported un-deprecation dance20:45
dansmithsince that was the original goal, we just missed some of it (right?)20:45
melwittI think that should do it, obviously after this fiasco I will have to test it and make 100% sure it would work20:45
melwittbefore I claim that's all we need20:46
dansmithyeah, that'd be good :)20:46
dansmithdid we deprecate and not also change devstack to not deploy?20:46
dansmithdeploy consoleauth I mean20:46
melwittyeah20:46
dansmithI would think that would have caught this20:46
dansmithack20:46
melwittaye20:46
melwittyeah, it'd have to do a "if cells v1" before deploy/not deploy but I think that would be fine20:47
melwittand maybe I could rig up a DNM test for the workaround being enabled to test that in the gate too20:47
dansmithwe have cellsv1 conditions20:48
sean-k-mooneymelwitt: you could make the rpc conditional and then make a devstack change that depends on it that disables deploying the consoleauth service to test it20:52
melwittyeah, that's what we're talking about doing20:53
sean-k-mooneycool and then the DNM test change is to reassert that if consoleauth is deploy the workaround still well works20:54
melwittyeah, that's what I was thinking20:56
*** priteau has quit IRC21:13
*** ivve has quit IRC21:14
*** priteau has joined #openstack-nova21:16
*** priteau has quit IRC21:18
*** macza has quit IRC21:20
*** phillu has quit IRC21:24
*** macza has joined #openstack-nova21:37
openstackgerritChris Dent proposed openstack/nova master: Clean up header encoding handling in compute API  https://review.openstack.org/60696721:39
*** itlinux has quit IRC21:39
*** munimeha1 has quit IRC21:41
*** macza has quit IRC21:42
openstackgerritMerged openstack/nova stable/queens: Revert "Make host_aggregate_map dictionary case-insensitive"  https://review.openstack.org/60526621:44
openstackgerritMerged openstack/nova stable/queens: Enforce case-sensitive hostnames in aggregate host add  https://review.openstack.org/60526721:44
*** cdent has quit IRC21:51
*** macza has joined #openstack-nova21:58
openstackgerritMerged openstack/nova stable/pike: Fix instance evacuation with PCI devices  https://review.openstack.org/59005921:59
*** macza has quit IRC22:02
openstackgerritMerged openstack/nova stable/pike: Update nova network info when doing rebuild for evacuate operation  https://review.openstack.org/59007022:04
*** slaweq has quit IRC22:15
*** macza has joined #openstack-nova22:17
*** ianw is now known as ianw_pto22:17
*** macza has quit IRC22:21
*** tbachman has joined #openstack-nova22:27
*** dpawlik has joined #openstack-nova22:28
*** dpawlik has quit IRC22:33
*** mriedem has joined #openstack-nova22:36
mriedemso has the great consoleauth undeprecafacation been sorted out?22:37
melwittmriedem: yeah, I'm working on an alternative approach dan suggested, to add the [workarounds] option condition to the compute/api calls, and see if that will make it possible not to run nova-consoleauth if backported to rocky. I can test it with a devstack change stacked on top. and I think I can use a DNM devstack change to verify that things also work with the [workarounds] option enabled22:39
mriedemso default to workaround enabled and people can disable that when they have drained their existing tokens in the consoleauth service?22:40
mriedemwonder if there is a nova-status check we can add for this...22:40
melwittI was thinking not to change the default, because only people with long TTL will potentially care about avoiding an instant reset of consoles22:41
melwittand the workaround is for those peeps22:41
melwittdefault TTL is 10 min22:41
*** Sukhdev has quit IRC22:41
*** threestrands has joined #openstack-nova22:42
*** dpawlik has joined #openstack-nova22:44
* sean-k-mooney why is networking never easy ...22:46
*** macza has joined #openstack-nova22:46
sean-k-mooneyi think 3 hours of debuging the neutron sriov nic agent is enough for tonight22:47
*** dpawlik has quit IRC22:48
*** rcernin has joined #openstack-nova22:49
*** macza_ has joined #openstack-nova22:51
*** macza has quit IRC22:51
*** macza_ has quit IRC22:55
*** efried has quit IRC22:57
sean-k-mooney... so i decided to restack and the thing i was debugging for the last 3 hours is now working and the only codecages i have done is add logging23:00
* sean-k-mooney is going to choose to pretend they spend the last 3 hour watching youtube and go to bed.23:01
*** macza has joined #openstack-nova23:08
*** macza has quit IRC23:08
*** macza has joined #openstack-nova23:11
*** med_ has joined #openstack-nova23:20
*** gyee has quit IRC23:47
openstackgerritmelanie witt proposed openstack/nova master: WIP Use nova-consoleauth only if workaround enabled  https://review.openstack.org/60706823:48
mriedemmelwitt: dansmith: thanks for hitting those pike backports in https://etherpad.openstack.org/p/nova-ocata-em23:49
melwittthanks for etherpadding them23:50
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Cross-cell resize  https://review.openstack.org/60393023:50
*** med_ has quit IRC23:51
*** moshele has joined #openstack-nova23:51
*** MasterofJOKers has joined #openstack-nova23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!