Friday, 2020-05-29

*** born2bake has quit IRC00:11
*** hamalq has joined #openstack-nova00:14
*** JamesBenson has joined #openstack-nova00:25
*** JamesBenson has quit IRC00:30
*** mlavalle has quit IRC00:46
*** hamalq has quit IRC00:54
*** eharney has quit IRC01:01
*** ociuhandu has joined #openstack-nova01:02
*** Liang__ has joined #openstack-nova01:03
*** ociuhandu has quit IRC01:08
*** sean-k-mooney has quit IRC02:07
*** sapd1 has joined #openstack-nova02:10
*** threestrands_ has joined #openstack-nova02:47
*** threestrands_ has quit IRC02:48
*** threestrands_ has joined #openstack-nova02:49
*** threestrands_ has quit IRC02:50
*** threestrands has quit IRC02:50
*** threestrands_ has joined #openstack-nova02:50
*** threestrands_ has quit IRC02:51
*** threestrands_ has joined #openstack-nova02:52
*** threestrands_ has quit IRC02:53
*** threestrands_ has joined #openstack-nova02:53
*** rcernin has quit IRC02:54
*** threestrands_ has quit IRC02:54
*** threestrands_ has joined #openstack-nova02:55
*** threestrands_ has quit IRC02:56
*** threestrands_ has joined #openstack-nova02:56
*** threestrands_ has quit IRC02:57
*** threestrands_ has joined #openstack-nova02:58
*** threestrands_ has quit IRC02:59
*** threestrands_ has joined #openstack-nova02:59
*** threestrands_ has quit IRC03:00
*** threestrands_ has joined #openstack-nova03:01
*** threestrands_ has quit IRC03:02
*** threestrands_ has joined #openstack-nova03:02
*** threestrands_ has quit IRC03:03
*** threestrands_ has joined #openstack-nova03:04
*** threestrands_ has quit IRC03:05
*** threestrands_ has joined #openstack-nova03:05
openstackgerritLiang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch  https://review.opendev.org/73138803:06
*** threestrands_ has quit IRC03:06
*** threestrands_ has joined #openstack-nova03:07
*** threestrands_ has quit IRC03:08
*** threestrands_ has joined #openstack-nova03:08
*** threestrands_ has quit IRC03:09
*** threestrands_ has joined #openstack-nova03:10
*** threestrands_ has quit IRC03:11
*** threestrands_ has joined #openstack-nova03:11
*** threestrands_ has quit IRC03:12
*** threestrands_ has joined #openstack-nova03:13
*** threestrands_ has quit IRC03:14
*** threestrands_ has joined #openstack-nova03:14
*** rcernin has joined #openstack-nova03:15
*** threestrands_ has quit IRC03:15
*** threestrands_ has joined #openstack-nova03:16
*** threestrands_ has quit IRC03:17
*** threestrands_ has joined #openstack-nova03:17
*** rcernin has quit IRC03:18
*** rcernin has joined #openstack-nova03:18
*** threestrands_ has quit IRC03:18
*** threestrands_ has joined #openstack-nova03:19
*** threestrands_ has quit IRC03:20
*** threestrands_ has joined #openstack-nova03:20
*** threestrands_ has quit IRC03:21
*** threestrands_ has joined #openstack-nova03:22
*** threestrands_ has quit IRC03:23
*** threestrands_ has joined #openstack-nova03:23
*** threestrands_ has quit IRC03:24
*** threestrands_ has joined #openstack-nova03:25
*** threestrands_ has quit IRC03:26
*** threestrands_ has joined #openstack-nova03:26
*** threestrands_ has quit IRC03:27
*** psachin has joined #openstack-nova03:28
*** threestrands_ has joined #openstack-nova03:28
*** threestrands_ has quit IRC03:29
*** threestrands_ has joined #openstack-nova03:29
*** threestrands_ has quit IRC03:30
*** threestrands_ has joined #openstack-nova03:31
*** threestrands_ has quit IRC03:32
*** threestrands_ has joined #openstack-nova03:32
*** threestrands_ has quit IRC03:33
*** threestrands_ has joined #openstack-nova03:34
*** threestrands_ has quit IRC03:35
*** threestrands_ has joined #openstack-nova03:35
*** threestrands_ has quit IRC03:36
*** threestrands_ has joined #openstack-nova03:37
*** threestrands_ has quit IRC03:38
*** threestrands_ has joined #openstack-nova03:38
*** threestrands_ has quit IRC03:39
*** threestrands_ has joined #openstack-nova03:40
*** threestrands_ has quit IRC03:41
*** threestrands_ has joined #openstack-nova03:41
*** threestrands_ has quit IRC03:42
*** threestrands_ has joined #openstack-nova03:43
*** threestrands_ has quit IRC03:44
*** threestrands_ has joined #openstack-nova03:44
*** threestrands_ has quit IRC03:45
*** threestrands_ has joined #openstack-nova03:46
*** threestrands_ has quit IRC03:47
*** threestrands_ has joined #openstack-nova03:47
*** threestrands_ has quit IRC03:48
*** threestrands_ has joined #openstack-nova03:49
*** threestrands_ has quit IRC03:50
*** threestrands_ has joined #openstack-nova03:50
*** threestrands_ has quit IRC03:51
*** threestrands_ has joined #openstack-nova03:52
*** threestrands_ has quit IRC03:53
*** threestrands_ has joined #openstack-nova03:53
*** threestrands_ has quit IRC03:54
*** threestrands_ has joined #openstack-nova03:55
*** threestrands_ has quit IRC03:56
*** threestrands_ has joined #openstack-nova03:56
*** threestrands_ has quit IRC03:57
*** threestrands_ has joined #openstack-nova03:58
*** threestrands_ has quit IRC03:59
*** threestrands_ has joined #openstack-nova03:59
*** threestrands_ has quit IRC04:00
*** threestrands_ has joined #openstack-nova04:01
*** threestrands_ has quit IRC04:02
*** threestrands_ has joined #openstack-nova04:02
*** threestrands_ has quit IRC04:03
*** threestrands_ has joined #openstack-nova04:04
*** threestrands_ has quit IRC04:05
*** threestrands_ has joined #openstack-nova04:05
*** threestrands_ has quit IRC04:06
*** threestrands_ has joined #openstack-nova04:07
*** threestrands_ has quit IRC04:08
*** threestrands_ has joined #openstack-nova04:08
*** threestrands_ has quit IRC04:09
*** threestrands_ has joined #openstack-nova04:10
*** benj_ has joined #openstack-nova04:10
*** threestrands_ has quit IRC04:11
*** threestrands_ has joined #openstack-nova04:11
*** threestrands_ has quit IRC04:12
*** threestrands_ has joined #openstack-nova04:13
*** threestrands_ has quit IRC04:14
*** threestrands_ has joined #openstack-nova04:14
*** threestrands_ has quit IRC04:15
*** threestrands_ has joined #openstack-nova04:16
*** threestrands_ has quit IRC04:17
*** threestrands_ has joined #openstack-nova04:17
*** threestrands_ has quit IRC04:18
*** lbragstad has quit IRC04:19
*** threestrands_ has joined #openstack-nova04:19
*** threestrands_ has quit IRC04:20
*** threestrands_ has joined #openstack-nova04:20
*** threestrands_ has quit IRC04:21
*** threestrands_ has joined #openstack-nova04:22
*** sapd1 has quit IRC04:23
*** JamesBenson has joined #openstack-nova04:26
*** JamesBenson has quit IRC04:32
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-nova04:33
*** vishalmanchanda has joined #openstack-nova04:36
*** sapd1 has joined #openstack-nova04:36
*** rambo_li has joined #openstack-nova04:52
*** hamalq has joined #openstack-nova04:54
*** rambo_li has quit IRC04:57
openstackgerritLiang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch  https://review.opendev.org/73138804:58
*** rambo_li has joined #openstack-nova04:58
*** hamalq has quit IRC05:04
*** ociuhandu has joined #openstack-nova05:04
*** jamesdenton has quit IRC05:05
*** JamesBenson has joined #openstack-nova05:05
*** hamalq has joined #openstack-nova05:06
*** JamesBenson has quit IRC05:09
*** ociuhandu has quit IRC05:10
*** jamesdenton has joined #openstack-nova05:22
*** udesale has joined #openstack-nova05:25
*** rambo_li has quit IRC05:28
*** efried has quit IRC05:32
*** dklyle has quit IRC05:32
*** jdillaman has quit IRC05:33
*** jdillaman has joined #openstack-nova05:34
*** hamalq has quit IRC05:38
*** ratailor has joined #openstack-nova05:40
*** udesale_ has joined #openstack-nova05:45
*** udesale has quit IRC05:45
*** dpawlik has joined #openstack-nova05:55
*** links has joined #openstack-nova05:58
*** links has joined #openstack-nova05:59
*** hamalq has joined #openstack-nova05:59
*** rambo_li has joined #openstack-nova05:59
*** hamalq has quit IRC06:04
*** ttsiouts has joined #openstack-nova06:10
*** ociuhandu has joined #openstack-nova06:15
*** nightmare_unreal has joined #openstack-nova06:23
*** jsuchome has joined #openstack-nova06:33
*** hamalq has joined #openstack-nova06:35
*** hamalq has quit IRC06:40
*** ociuhandu has quit IRC06:40
*** jaosorior has joined #openstack-nova06:48
*** ttsiouts has quit IRC06:48
*** ksk_ has joined #openstack-nova06:49
*** ralonsoh has joined #openstack-nova06:54
*** belmoreira has joined #openstack-nova06:54
*** tesseract has joined #openstack-nova07:01
*** damien_r has joined #openstack-nova07:04
*** rpittau|afk is now known as rpittau07:04
*** slaweq has joined #openstack-nova07:07
*** damien_r has quit IRC07:09
gibigmann: we have the whole 15:00 - 17:00 slot on Monday for oslo crossproject so within that timeframe any slot is OK07:09
gibigmann, bnemec: let me know if you have a more finegrained start time than somewhere between 15:00 - 17:0007:10
*** tosky has joined #openstack-nova07:12
*** ikla has quit IRC07:14
*** ttsiouts has joined #openstack-nova07:14
*** ociuhandu has joined #openstack-nova07:15
*** factor has quit IRC07:21
*** rcernin has quit IRC07:26
*** dtantsur|afk is now known as dtantsur07:30
*** ociuhandu has quit IRC07:33
*** ociuhandu has joined #openstack-nova07:34
*** xiaolin has quit IRC07:34
bauzasgood morning Nova07:37
*** ociuhandu has quit IRC07:39
gibigood morning bauzas!07:41
*** JamesBenson has joined #openstack-nova07:43
*** JamesBenson has quit IRC07:48
*** links has quit IRC08:01
*** links has joined #openstack-nova08:07
*** ociuhandu has joined #openstack-nova08:07
fricklerhello nova, I seem to be getting persistent tempest failures while detaching volumes when trying to run on focal (ubuntu 20.04), may this does ring a bell for someone? see e.g. https://zuul.opendev.org/t/openstack/build/a5aca94014224f1fa1acbb64e3f72f54/log/controller/logs/screen-n-cpu.txt?severity=4#3951408:12
fricklerlyarwood: ^^ that also seems to affect your recent server_rescue tests08:13
*** links has quit IRC08:13
*** ociuhandu has quit IRC08:14
*** yaawang has quit IRC08:16
*** JamesBenson has joined #openstack-nova08:17
*** JamesBenson has quit IRC08:22
*** ccamacho has joined #openstack-nova08:25
*** salmankhan has joined #openstack-nova08:32
*** JamesBenson has joined #openstack-nova08:38
*** salmankhan has quit IRC08:40
*** salmankhan has joined #openstack-nova08:41
*** derekh has joined #openstack-nova08:42
*** JamesBenson has quit IRC08:42
*** ttsiouts has quit IRC08:47
*** ttsiouts has joined #openstack-nova08:48
*** ociuhandu has joined #openstack-nova08:54
*** tetsuro has quit IRC08:57
*** ociuhandu has quit IRC09:07
lyarwoodfrickler: morning, looking09:09
*** JamesBenson has joined #openstack-nova09:09
*** JamesBenson has quit IRC09:14
lyarwoodI'm seeing this logged alot in there, this isn't the issue but still something we should fix09:20
lyarwood50459 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: Exception ignored in: <function _after_fork at 0x7f5217bc6b80>09:20
lyarwood50460 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: Traceback (most recent call last):09:20
lyarwood50461 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]:   File "/usr/lib/python3.8/threading.py", line 1454, in _after_fork09:20
lyarwood50462 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]:     assert len(_active) == 109:20
lyarwood50463 May 28 17:49:19.906197 ubuntu-focal-rax-ord-0016842009 nova-compute[139822]: AssertionError:09:20
fricklerlyarwood: didn't coreycb have some patches up for that? saw that at least in a couple of other projects09:26
fricklerhmm, that has long been merged in nova https://review.opendev.org/707474 , maybe it needs some more work09:28
*** Liang__ is now known as LiangFang09:28
*** hamalq has joined #openstack-nova09:33
*** hamalq has quit IRC09:37
*** JamesBenson has joined #openstack-nova09:45
*** threestrands_ has quit IRC09:48
fricklercoreycb: seems oslo_concurrency/processutils.py would need to get that fix, too09:49
*** JamesBenson has quit IRC09:50
*** ociuhandu has joined #openstack-nova10:00
openstackgerritLiang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch  https://review.opendev.org/73138810:01
*** rpittau is now known as rpittau|bbl10:07
lyarwoodfrickler: sorry had a downstream ping I had to jump on10:09
lyarwoodfrickler: it looks like we've missed the detach here somehow10:09
lyarwoodfrickler: does this reproduce easily for you?10:10
*** derekh has quit IRC10:11
*** udesale_ has quit IRC10:16
*** udesale has joined #openstack-nova10:18
fricklerlyarwood: well I failed to reproduce it locally so far, but on that patch it seems consistent through the last rechecks10:20
fricklerI'm still not 100% convinced that this is not related to the oslo_concurrency issue, trying to patch that one now10:21
lyarwoodfrickler: yeah I don't think it is10:25
lyarwoodfrickler: it looks like libvirt is raising a new exception when we can't find the device here https://github.com/openstack/nova/blob/323339236517b00ee2495572cfb2540b9e72ce98/nova/virt/libvirt/guest.py#L393-L41410:26
lyarwoodfrickler: I can see the error in libvirtd but n-cpu just continues trying to detach the device10:27
lyarwoodfrickler: I'll add some LOG.exception lines to prove it now10:27
*** udesale has quit IRC10:28
*** ttsiouts has quit IRC10:29
*** ttsiouts has joined #openstack-nova10:31
*** udesale has joined #openstack-nova10:32
*** sean-k-mooney has joined #openstack-nova10:33
openstackgerritLee Yarwood proposed openstack/nova master: DNM libvirt: Add exception logging when attempting to detach device  https://review.opendev.org/73172110:34
openstackgerritLiang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch  https://review.opendev.org/73138810:43
*** rambo_li has quit IRC10:46
*** LiangFang has quit IRC10:47
*** vishalmanchanda has quit IRC11:05
*** ttsiouts has quit IRC11:14
*** ttsiouts has joined #openstack-nova11:21
*** rcernin has joined #openstack-nova11:21
*** jaosorior has quit IRC11:37
*** rcernin has quit IRC11:38
*** JamesBenson has joined #openstack-nova11:43
*** brinzhang has quit IRC11:44
*** ttsiouts has quit IRC11:50
*** yaawang has joined #openstack-nova11:51
*** ttsiouts has joined #openstack-nova12:07
*** raildo has joined #openstack-nova12:07
*** elod has quit IRC12:10
*** lbragstad has joined #openstack-nova12:16
*** elod has joined #openstack-nova12:17
*** sapd1 has quit IRC12:24
*** mriedem has joined #openstack-nova12:30
*** ratailor has quit IRC12:31
*** rpittau|bbl is now known as rpittau12:35
*** tkajinam has quit IRC12:39
*** mdbooth has quit IRC12:48
gmanngibi: thanks, let's wait for bnemec when is best fit time for him. I am ok anytime after 15 UTC12:58
gibigmann: Ok12:58
*** elod has quit IRC13:06
*** elod has joined #openstack-nova13:13
*** eharney has joined #openstack-nova13:17
*** ociuhandu has quit IRC13:17
*** ociuhandu has joined #openstack-nova13:18
openstackgerritLee Yarwood proposed openstack/nova master: DNM libvirt: Add exception logging when attempting to detach device  https://review.opendev.org/73172113:21
*** ociuhandu has quit IRC13:23
*** vishalmanchanda has joined #openstack-nova13:28
*** ociuhandu has joined #openstack-nova13:30
*** nweinber has joined #openstack-nova13:33
*** ociuhandu has quit IRC13:35
*** mlavalle has joined #openstack-nova13:59
*** ociuhandu has joined #openstack-nova14:02
*** efried has joined #openstack-nova14:02
*** rambo_li has joined #openstack-nova14:04
*** ttsiouts has quit IRC14:05
fricklerlyarwood: o.k., I managed to trigger some of your logging code, anything in particular you would want to see?14:09
*** ociuhandu has quit IRC14:09
lyarwoodfrickler: `Unable to find device` or `Found device` would be a useful start14:10
*** rambo_li has quit IRC14:10
lyarwoodfrickler: this smells like two issues btw, a latent bug in Nova within our device lookup code for disks and a new issue in libvirt where it isn't raising the exception.14:12
*** dklyle has joined #openstack-nova14:12
*** ociuhandu has joined #openstack-nova14:17
fricklerlyarwood: http://paste.openstack.org/show/794150/ , the first one didn't give a tempest failure, the second did14:19
spotzHey all, this still the right place for placement questions? Might have a docs bug but not sure14:39
*** Liang__ has joined #openstack-nova14:49
*** ttsiouts has joined #openstack-nova14:53
*** belmoreira has quit IRC15:01
*** ebbex has quit IRC15:03
*** ociuhandu has quit IRC15:13
*** ociuhandu has joined #openstack-nova15:14
*** ociuhandu has quit IRC15:16
*** ociuhandu has joined #openstack-nova15:17
openstackgerritLiang Fang proposed openstack/nova master: DNM - test Nova UT that depends on os-brick patch  https://review.opendev.org/73138815:24
*** bnemec is now known as beekneemech15:25
*** Liang__ has quit IRC15:35
beekneemechgmann: gibi: I've tentatively scheduled the Oslo-specific topics for our first hour and left the second hour (starting 1600) for the Nova cross-project discussions.15:41
beekneemechDoes that work for you?15:41
*** jsuchome has quit IRC15:42
*** rambo_li has joined #openstack-nova15:46
*** rambo_li has quit IRC15:51
*** slaweq has quit IRC15:53
*** rambo_li has joined #openstack-nova16:04
*** rambo_li has quit IRC16:08
*** ttsiouts has quit IRC16:11
*** dpawlik has quit IRC16:15
*** hamalq has joined #openstack-nova16:20
*** rpittau is now known as rpittau|afk16:21
*** psachin has quit IRC16:21
*** hamalq has quit IRC16:25
*** ociuhandu has quit IRC16:28
*** bbowen has quit IRC16:28
*** ociuhandu has joined #openstack-nova16:28
*** bbowen has joined #openstack-nova16:28
*** dtantsur is now known as dtantsur|afk16:29
*** udesale has quit IRC16:30
*** hamalq has joined #openstack-nova16:30
openstackgerritLee Yarwood proposed openstack/nova master: WIP compute: Handle MessagingTimeout during reserve_block_device_name  https://review.opendev.org/73180416:30
*** ociuhandu has quit IRC16:34
*** ccamacho has quit IRC16:59
*** lyarwood has quit IRC17:05
*** hamalq has quit IRC17:19
*** hamalq has joined #openstack-nova17:19
*** hamalq has quit IRC17:22
*** hamalq has joined #openstack-nova17:22
*** nightmare_unreal has quit IRC17:23
*** tesseract has quit IRC17:42
*** vishalmanchanda has quit IRC17:51
*** salmankhan1 has joined #openstack-nova17:53
*** salmankhan has quit IRC17:57
*** salmankhan1 has quit IRC17:57
*** rambo_li has joined #openstack-nova18:05
*** rambo_li has quit IRC18:10
*** elod has quit IRC18:10
*** ociuhandu has joined #openstack-nova18:16
*** elod has joined #openstack-nova18:17
*** ralonsoh has quit IRC18:21
*** damien_r has joined #openstack-nova18:42
*** elod has quit IRC18:45
*** elod has joined #openstack-nova18:45
*** ociuhandu has quit IRC18:53
*** ociuhandu has joined #openstack-nova18:53
*** ociuhandu has quit IRC18:59
*** ociuhandu has joined #openstack-nova19:16
*** elod has quit IRC19:20
*** ociuhandu has quit IRC19:30
*** ociuhandu has joined #openstack-nova19:31
*** ociuhandu has quit IRC19:37
*** mgoddard has quit IRC19:44
*** nweinber has quit IRC19:45
*** rambo_li has joined #openstack-nova19:45
*** rambo_li has quit IRC19:50
*** mgoddard has joined #openstack-nova19:50
*** elod has joined #openstack-nova19:51
*** gary_perkins has quit IRC20:06
*** gary_perkins has joined #openstack-nova20:09
*** ociuhandu has joined #openstack-nova20:33
mriedemspotz: #openstack-placement20:43
spotzThanks mriedem wasn't sure if they'd made a channel or not20:43
*** ociuhandu has quit IRC20:45
*** raildo has quit IRC20:48
*** mgariepy has quit IRC21:01
*** rambo_li has joined #openstack-nova21:20
*** rambo_li has quit IRC21:25
*** ociuhandu has joined #openstack-nova21:36
*** xek has joined #openstack-nova21:36
*** xek has quit IRC21:37
*** xek has joined #openstack-nova21:42
*** xek has quit IRC21:42
*** ociuhandu has quit IRC21:44
*** fumesover6 is now known as fumesover21:44
*** rambo_li has joined #openstack-nova21:49
*** KeithMnemonic has quit IRC21:50
*** rambo_li has quit IRC21:53
*** mriedem has quit IRC21:56
*** maciejjozefczyk has quit IRC22:08
*** JamesBenson has quit IRC22:08
*** rambo_li has joined #openstack-nova22:19
*** rambo_li has quit IRC22:24
*** mlavalle has quit IRC22:58
*** rambo_li has joined #openstack-nova23:35
*** tosky has quit IRC23:38
*** rambo_li has quit IRC23:39
*** rambo_li has joined #openstack-nova23:53
*** rambo_li has quit IRC23:57

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!