Friday, 2022-05-06

opendevreviewMerged openstack/networking-generic-switch master: CI: Alternate logic for multinode key replacement  https://review.opendev.org/c/openstack/networking-generic-switch/+/84051901:39
opendevreviewJulia Kreger proposed openstack/networking-generic-switch stable/yoga: CI: Alternate logic for multinode key replacement  https://review.opendev.org/c/openstack/networking-generic-switch/+/84073102:37
opendevreviewSteve Baker proposed openstack/sushy-tools master: Fix per-request memoization  https://review.opendev.org/c/openstack/sushy-tools/+/84078003:08
opendevreviewSteve Baker proposed openstack/sushy-tools master: Add an allowed list to filter managed virtual machines  https://review.opendev.org/c/openstack/sushy-tools/+/84080403:08
opendevreviewMerged openstack/networking-generic-switch stable/yoga: CI: Alternate logic for multinode key replacement  https://review.opendev.org/c/openstack/networking-generic-switch/+/84073105:38
*** mat_fechner is now known as matfechner06:21
arne_wiebalckGood morning, Ironic!06:24
jandershey arne_wiebalck o/11:18
ajyaHi ftarasenko , cc dtantsur  looking at the latest logs these are the differences between succeeded node and failed node - https://paste.opendev.org/show/btgQTCvsi59oyvWaFifn/11:23
ajyathe one that succeeded don't have heartbeats, for the failed one the heartbeat seem to complete the cleaning while cleaning still running causing error later11:24
ajyaftarasenko: do you know why the successful node does not have heartbeat? 11:25
ajyaDoes anyone have idea why heartbeats make cleaning complete when it shouldn't? Something to configure in the step? 11:26
dtantsurajya: generally, heartbeats are supposed to cause progress in cleaning/deploy11:26
dtantsurwe have a special flag (polling?) to prevent that11:26
dtantsurthe question why it does not work is a good one11:26
dtantsurif you have a way to reproduce it, you can insert logging everywhere to see how the decision is made and what the node's status is11:27
ajyaI can't reproduce it, but I have more logs from ftarasenko 11:28
ajyawill check what happens with polling flag, otherwise this is the difference - it works for some nodes because no heartbeats happening11:29
ajyaalso will check what happens with heartbeats in my environment11:29
ftarasenkoHi! I can reproduce failed cleaning with more debug output if required.11:33
ajyaok, I'll check if and where more logging should be added. So far cleaning_polling is set to True in both cases.11:59
iurygregorygood morning Ironic12:04
iurygregorycan I get a +W in https://review.opendev.org/c/openstack/ironic/+/840671 so I can work on the release? :D12:10
TheJuliaI can in a little bit12:16
TheJuliaJust starting to wake up12:16
iurygregorysure o/12:19
iurygregoryrpittau, in the ironic-inspector-tempest-managed-non-standalone I see a lot of  No space left on device  in the bm-logs and after that ironic_python_agent.errors.HeartbeatError: Error heartbeating to agent API: Error 400: Invalid or missing agent token received.12:20
TheJuliasweet12:22
dtantsur>_<12:26
TheJuliaso... does this mean the weekend drinking starts... early?12:36
* dtantsur already had coffee with baileys12:36
TheJulia+2+A12:40
iurygregoryI'm going to attempt to increase the disk size just for testing...12:41
dtantsuriurygregory: we also need to check if we expect the raw image streaming to be used12:42
TheJuliawe should ask infra if they reduced the disk sizes again12:43
TheJuliajust to be on the safe side12:43
dtantsurI think we worked around the cloud disk sizes by using /opt12:44
iurygregorydtantsur, the stream_raw_images option you mean?12:44
TheJuliaYeah, we did that though...12:44
dtantsuriurygregory: it's not just the option, we need to be able to do that12:45
iurygregorybut we are only using opt in some scenarios (I think), let me double check12:45
iurygregoryohhh seems like we don't have LIBVIRT_STORAGE_POOL_PATH: /opt/libvirt/images in some inspector jobs12:47
dtantsur\o/12:48
iurygregoryI will attempt to add that to see 12:48
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-inspector master: [DNM] Testing CI with LIBVIRT_STORAGE_POOL_PATH  https://review.opendev.org/c/openstack/ironic-inspector/+/84089112:52
opendevreviewDmitry Tantsur proposed openstack/ironic-python-agent stable/xena: Do not try to guess EFI partition path by its number  https://review.opendev.org/c/openstack/ironic-python-agent/+/84089212:57
dtantsurthis was a crazy backport, please review carefully ^^^12:57
iurygregoryoh I can imagine12:57
opendevreviewDmitry Tantsur proposed openstack/ironic-python-agent stable/wallaby: Do not try to guess EFI partition path by its number  https://review.opendev.org/c/openstack/ironic-python-agent/+/84017013:04
dtantsursame ^^13:04
TheJuliaoh joy, at least we found it quickly13:04
* TheJulia reads absolutely batshit crazy laws being proposed and wishes there was an easy way to depart her country13:05
dtantsurTheJulia: yeah, I'm sorry to read the recent news :(13:06
dtantsurEasy it is not, but I wonder if some, ehemm, calmer country is an option13:06
TheJuliaits a nice fancy but if we do it, we have to walk away from ever returning13:08
TheJuliawell, fancy to think about if we were to do it13:08
iurygregoryTheJulia, I'm sure you will find a solution, fingers crossed \o/13:08
TheJuliaand that also means we have to walk away from promise of some funds for retirement13:08
TheJuliaIt is really only feasible if your younger13:08
iurygregory=(13:10
TheJuliaMy wife had someone super interested in recruiting her to work for a company in the EU about a year or two ago, and we looked at it and the long term impacts and yeah. It was soboring to do the math and look at the long term impact...13:13
dtantsuryeah, I cannot comment on the financial situation13:16
dtantsurmy own approach was to run without looking back, but that's definitely not for everyone13:16
TheJuliayeah13:16
* dtantsur has just read that > 3 mln people have left Russia THIS YEAR13:16
TheJuliawow13:16
iurygregoryWOW O.o 3mln ?13:17
dtantsurI can hardly believe myself13:17
iurygregoryhttps://review.opendev.org/c/openstack/releases/+/840896 IPA release \o/13:17
dtantsura lot of people just noped out into nowhere on short notice13:17
iurygregoryI'm wondering how they are able to do that on short notice...13:18
TheJulialeaving and never looking back perhaps?13:18
dtantsursurprisingly, there are still countries that accept Russians without a visa13:18
iurygregorymy point was related to visa =)13:19
dtantsurticket prises went through the roof (10x increase wasn't uncommon)13:19
iurygregorywow13:19
TheJuliaWow, their population growth rate based on just deaths/births is in hard contraction already13:19
dtantsurso, there are neighbouring countries like Armenia and Georgia, other former USSR countries as well13:19
dtantsurthe fact that Russia has a very old population is a part of the problem13:20
dtantsurthe old population in Russia is extremely conservative13:20
dtantsur(politely speaking)13:21
TheJuliaWe have a very similar problem here13:21
iurygregoryI do see some resemblance with Brazil...13:22
TheJuliaThe problem we also have is people are not really retiring13:22
dtantsurI think this scale is even worse13:22
TheJuliain part... because acquisition of wealth being the most important thing... and there is not really any sort of safety net13:22
TheJuliaYeah13:23
dtantsurRussia had a huge drop in births from ~ mid 80th to mid or even late 90th13:23
TheJuliaYeah, the graph is super sobering to look at13:23
dtantsurwhile there was a huge spike after the war (our variant of boomers, more or less)13:24
TheJuliaOn the subject of happier graphs... https://usercontent.irccloud-cdn.com/file/Uoncb8SK/muahahahaha.png13:25
dtantsur50th were years of recovery from both the war and stalinism. 70th were the years of (perceived) prosperity13:25
dtantsurin the 80th it went southwards, the (perceived) prosperity only returned in 2000th13:25
* dtantsur should probably stop distracting people with history lessons :)13:26
TheJuliaAnyone remember the bar discussion in Atlanta how weirdly similar propaganda was? :)13:26
dtantsurnot me, unfortunately13:26
iurygregorynope .-. I wasn't there13:27
TheJuliaIt was... amusing13:27
TheJuliabut basically same basic messaging, same idolization of a golden period of prosperity13:27
TheJuliaand same basic painting as everyone else as the enemy13:27
TheJuliaAnd the internet kind of enabled us to start breaking down some of those walls of understanding13:28
* TheJulia should go eat breakfast before her 7 am meeting13:28
dtantsurenjoy!13:28
iurygregory++ for breakfast13:28
* dtantsur got a questionable habit of eating korean ramen for breakfast13:28
iurygregorynow I'm jealous13:29
iurygregoryI miss ramen <313:29
dtantsurgive it a try :) just keep it mind - it's pretty hot. even the "mild" version13:29
dtantsurgot no ramen around you?13:29
iurygregoryI never had korean ramen, only the japanese13:30
dtantsurit's pretty different, although the base idea is the same13:30
dtantsurfirst and foremost, korean is SPICY13:30
iurygregorydtantsur, in Brazil only in some very big cities you will be able to find korean food I would say13:30
iurygregoryso I will keep eating japanese ramen <3 since is not spicy 13:31
iurygregoryI can't handle spicy food 13:31
dtantsurthen forget it :)13:31
dtantsurmy wife can also eat only japanese ramen13:31
* iurygregory is wondering how he can travel to South Korea now...13:31
dtantsurso we have two separate stashes :)13:31
dtantsuriurygregory: by a looooong flight13:31
iurygregorydtantsur, well I'm thinking about the food now... 13:32
iurygregoryif everything is spicy13:32
dtantsurwell13:32
dtantsurright13:32
dtantsuryou'll have to stick to drinking :)13:32
iurygregoryI will need to do a lot of research :D 13:32
iurygregorySoju <313:32
dtantsurI prefer Makgeoli (oh how is it spelled)13:32
iurygregorysoju + beer = amazing combination13:32
dtantsurneed to try13:33
* iurygregory googles13:33
dtantsurI even have a bottle of soju at home13:33
iurygregoryoh Makgeoli is the drink that is made with rice13:33
dtantsuryep. pretty milky and sweet.13:34
dtantsurcan be mixed with juice13:34
iurygregorydtantsur, I've tested soju + Hoegaarden beer 13:34
dtantsuran interesting idea13:34
dtantsurI don't think we have hoegaarden here13:34
iurygregorylike 200ml of beer + 30-40ml of soju =)13:34
iurygregoryin germany you can probably find I think (at least in CZ it was pretty easy)13:35
dtantsurthe markets are saturated by local beer. we do have belgian beer as well, but mostly strong ones13:35
iurygregoryyeah, I wouldn't try with strong ones since soju is already 17% (at least the one I have)13:36
dtantsur(we have Delirium in the local supermarket!)13:36
iurygregorytry with some light beers :D13:36
iurygregoryDelirium <3 we have here but is VERY expensive13:36
dtantsurin Bavaria they have extra-light beers, that's really cool13:37
dtantsur2-3% of alcohol, perfect for summer13:37
iurygregorynice! 13:37
iurygregoryoh good, it will be summer next month right?13:38
iurygregory<O>13:38
* iurygregory forgot seasons are different in EU...13:38
dtantsuryep, it's getting really warm. +20 today13:38
dtantsurI'm looking forward towards camping without the temperature dropping to around or even below zero in the morning :)13:39
TheJuliaI was not expecting a ramen discussion13:39
iurygregoryTheJulia, well, it's friday13:39
TheJuliaTrue, true13:40
TheJuliaoh! which reminds me13:40
* TheJulia loads up grendade for a recheck13:40
dtantsurthe combination of the news and reading Remarque make me shudder when reading "grenade"13:42
TheJuliasorry13:42
TheJulia:(13:42
TheJuliamaybe we should propose to rename it13:42
dtantsurI sometimes wonder that13:42
TheJuliaor... just... send it to the bit bucket13:42
dtantsurI'm not bothered too much, but I've seen quite a few people proposing to get rid of military terminology13:43
TheJuliaanyway, I *think* the multinode mixed state upgrade job should be fine moving forward13:43
TheJuliabut, the current recheck will let us know for sure13:43
TheJulia++13:43
iurygregory++13:43
TheJuliaMaybe a worthwhile inclusion in D&I efforts13:44
iurygregory++ ^13:45
* iurygregory can't find spotz in this channel13:46
TheJuliaemail her :)13:53
* TheJulia has reached the point where she believes the sock gnomes are conspiring with the hairbrush thieves...13:54
TheJuliaPossible sign I need a hair cut13:54
iurygregory sock gnomes LOL13:55
TheJuliahttps://twitter.com/ashinclouds/status/152257586655476940813:56
TheJuliafor the record...13:56
* TheJulia goes back to her regularly scheduled meeting programming.13:57
opendevreviewMerged openstack/ironic master: Ironic Release 20.2  https://review.opendev.org/c/openstack/ironic/+/84067113:57
iurygregoryI don't know any ... 13:57
iurygregorysince this will be my first time in Berlin :D13:57
iurygregoryI've heard about Brewdog's (I think dtantsur mentioned last year in the irc :D )13:58
iurygregoryok, ironic release now \o/13:59
iurygregoryyay...14:39
iurygregoryopenstack-tox-validate is broken14:39
iurygregoryLIBVIRT_STORAGE_POOL_PATH didn't help14:41
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-inspector master: [DNM] Testing CI with LIBVIRT_STORAGE_POOL_PATH  https://review.opendev.org/c/openstack/ironic-inspector/+/84089114:45
iurygregoryfacepalm, forgot to change the top commit to the one I pushed with the fix15:00
iurygregoryhttps://review.opendev.org/c/openstack/releases/+/84092115:00
iurygregoryironic release15:00
iurygregorywe should be good for the releases once https://review.opendev.org/c/openstack/releases/+/840919 merges15:01
iurygregoryfeel free to ignore the validate failures =D15:01
* TheJulia glares at CI15:05
TheJuliawhy do you hate me networking-generic-switch15:48
dtantsurhave a nice weekend folks o/16:20
opendevreviewJulia Kreger proposed openstack/networking-generic-switch master: CI: Swap legacy DSVM job for current grenade job  https://review.opendev.org/c/openstack/networking-generic-switch/+/84096416:33
iurygregoryz`+ 16:36
TheJuliaokay, figured out why grenade is not happy in multinode18:14
iurygregoryironic-inspector-tempest-managed-non-standalone https://zuul.opendev.org/t/openstack/build/77de77ef50cb471e971f355f512320bc : SUCCESS in 1h 00m 20s18:41
iurygregory\o/18:41
iurygregoryyay18:41
iurygregoryohhhh the grenade failure looks like the same from the non-standalone...18:59
opendevreviewJulia Kreger proposed openstack/ironic master: DNM: Multinode Grenade  https://review.opendev.org/c/openstack/ironic/+/84053019:21
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-inspector master: [DNM] Testing CI with LIBVIRT_STORAGE_POOL_PATH  https://review.opendev.org/c/openstack/ironic-inspector/+/84089119:21
TheJuliaiurygregory: the last two lines on my multinode grenade change commit message should shed some light w/r/t multindoe19:21
iurygregoryTheJulia, ack o/19:22
* TheJulia goes and lays down... cold is winning this afternoon19:23
iurygregorytake care19:25
opendevreviewAlexander Lingo proposed openstack/ironic master: Cleanup submitted SNMP driver code for additional PDUs  https://review.opendev.org/c/openstack/ironic/+/84098620:34
* TheJulia does the grenade ci job seems to be working dance20:49
iurygregoryinteresting, https://review.opendev.org/c/openstack/ironic-inspector/+/840891 is green (except for l-c)20:56
iurygregoryTheJulia, should I just drop l-c in the same patch or put as non-voting?20:57
TheJuliait is a half dozen either way20:59
TheJuliado what makes the most sense, and we can just remove the l-c job in the next patch20:59
TheJuliawe can't backport it as-is with the l-c change though20:59
TheJuliaso maybe make it n-v20:59
TheJuliaand then nuke l-c20:59
TheJuliaand then undo n-v20:59
opendevreviewIury Gregory Melo Ferreira proposed openstack/ironic-inspector master: Fix CI jobs  https://review.opendev.org/c/openstack/ironic-inspector/+/84089121:01
iurygregoryhummm21:01
iurygregoryI only removed from the zuul config21:02
iurygregorydo we need to backport to yoga?21:02
TheJuliado we>21:03
TheJulia?21:03
TheJuliadunno21:03
TheJuliamost likey... to be honest21:04
iurygregorydunno also, you mentioned backport so I was wondering ...21:04
* TheJulia needs new sinues and a cure for the common cold21:04
iurygregoryI have common cold also, I'm thinking about skipping the exercises today21:06
TheJuliaI'm waiting for groceries to be delivered... then likely going to bed early21:06
iurygregory++21:06
TheJuliaThat and the wife's requested ice cream will be in the groceries and she is far more sick than I today21:07
iurygregorywoot?!21:12
iurygregoryice cream?!21:12
iurygregorynormally I try to avoid cold things when I have a cold21:13
TheJuliawell, the grenade tests failed, but appeared to get far furhter today21:17
TheJuliaI suspect we've got a race with neutron networking someplace21:17
opendevreviewJulia Kreger proposed openstack/ironic master: DNM: Multinode Grenade  https://review.opendev.org/c/openstack/ironic/+/84053021:37
opendevreviewAlexander Lingo proposed openstack/ironic master: Cleanup submitted SNMP driver code for additional PDUs  https://review.opendev.org/c/openstack/ironic/+/84098621:40
opendevreviewAlexander Lingo proposed openstack/ironic master: Cleanup submitted SNMP driver code for additional PDUs  https://review.opendev.org/c/openstack/ironic/+/84098621:58

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!