Monday, 2020-10-05

*** k_mouza has quit IRC00:01
*** k_mouza has joined #openstack-ironic00:10
*** k_mouza has quit IRC00:15
*** k_mouza has joined #openstack-ironic00:18
*** k_mouza has quit IRC00:23
*** bfournie has joined #openstack-ironic00:26
*** k_mouza has joined #openstack-ironic00:43
*** k_mouza has quit IRC00:47
*** k_mouza has joined #openstack-ironic00:53
*** k_mouza has quit IRC00:58
*** k_mouza has joined #openstack-ironic01:05
*** k_mouza has quit IRC01:10
*** k_mouza has joined #openstack-ironic01:21
*** k_mouza has quit IRC01:26
*** k_mouza has joined #openstack-ironic01:29
*** Goneri has quit IRC01:32
*** k_mouza has quit IRC01:33
*** k_mouza has joined #openstack-ironic01:56
*** k_mouza has quit IRC02:00
*** k_mouza has joined #openstack-ironic02:21
*** cp- has quit IRC02:25
*** k_mouza has quit IRC02:26
*** cp- has joined #openstack-ironic02:28
*** mkrai has joined #openstack-ironic03:31
*** mkrai has quit IRC03:33
*** mkrai has joined #openstack-ironic04:05
*** zzzeek has quit IRC04:21
*** zzzeek has joined #openstack-ironic04:24
*** uzumaki has joined #openstack-ironic04:52
uzumakigood morning ironic! o/04:53
uzumakiDoes anybody know when the PTGs are this summit? The summit dates are 19-23, is the PTG before or afterwards?05:25
uzumakiMore specifically, I want to know when the Ironic PTG is05:26
*** ricolin_ has joined #openstack-ironic05:56
*** tkajinam has quit IRC06:13
*** tkajinam has joined #openstack-ironic06:13
*** rcernin has joined #openstack-ironic06:17
*** rcernin has quit IRC06:22
*** ricolin_ has quit IRC06:23
*** ricolin_ has joined #openstack-ironic06:23
arne_wiebalckGood morning, ironic!06:31
arne_wiebalckuzumaki: https://ethercalc.openstack.org/7xp2pcbh1ncb06:32
iurygregorygood morning uzumaki arne_wiebalck and Ironic!06:33
uzumakiBonjour arne_wiebalck iurygregory \o/06:34
uzumakiarne_wiebalck, is this the PTG schedule? or the whole summit?06:34
iurygregorybonjur is for arne_wiebalck =P06:34
iurygregoryuzumaki, PTG06:34
arne_wiebalckuzumaki: this is for the PTG06:34
uzumakiWell. buenos dias iurygregory xD06:34
iurygregoryI don't understand spanish LOL =P kidding06:34
uzumakiarne_wiebalck, merci beaucoup! arne_wiebalck  mucha gracias iurygregory :D06:35
uzumakiiurygregory, of course you do =D06:35
iurygregoryonly the very very basics06:35
uzumakiiurygregory, don't worry, I'm not a spaniard either, I love the fluency of the language though06:35
iurygregorylol06:36
*** jtomasek has joined #openstack-ironic06:53
iurygregoryFYI stable/victoria maybe ok to recheck, I saw they reverted the patch on zuul06:56
*** Qianbiao has joined #openstack-ironic07:05
*** yolanda has quit IRC07:21
*** rpittau|afk is now known as rpittau07:29
rpittaugood morning ironic! o/07:29
iurygregorywelcome back rpittau o/07:32
iurygregorygood morning =)07:32
rpittauhey iurygregory, thanks! :)07:32
iurygregoryI hope you had a good time!07:33
* iurygregory is sure he did07:33
*** yolanda has joined #openstack-ironic07:37
rpittauI did, I'm still in Sardegna at the moment :)07:41
rpittautrying to learn again how to type on a keyboard :P07:41
iurygregorynice =D07:41
*** tosky has joined #openstack-ironic07:42
*** dougsz has joined #openstack-ironic07:42
iurygregorydid you become a manager?07:42
rpittauiurygregory: not that I'm aware of, but I'm not sure I get your question07:43
iurygregoryaccording to shadower when your are an engineer and you become a manager you lose some skills07:43
rpittauah lol07:44
iurygregoryXD07:44
shadoweryea I don't even remember when they did the lobomomy, but the effect is real07:49
shadowerwelcome back rpittau!07:49
rpittauthanks shadower :)07:50
*** lucasagomes has joined #openstack-ironic07:56
*** ebagakis has joined #openstack-ironic07:56
*** ociuhandu has joined #openstack-ironic07:57
ebagakisGood Morning Ironic!08:01
arne_wiebalckebagakis has just joined the CERN Ironic team, welcome again o/08:02
rpittauwelcome ebagakis :)08:02
arne_wiebalckHey rpittau, welcome back!08:02
rpittauhey arne_wiebalck, thanks :)08:03
openstackgerritMark Goddard proposed openstack/bifrost master: Install DIB dependencies in a separate role  https://review.opendev.org/75600508:13
iurygregorywelcome ebagakis o/08:14
*** ricolin_ has quit IRC08:26
*** derekh has joined #openstack-ironic08:27
*** dtantsur|afk is now known as dtantsur08:32
dtantsurmorning ironic08:32
*** ebagakis has quit IRC08:37
rpittauhey dtantsur :)08:40
*** QianbiaoNG has joined #openstack-ironic08:40
*** rcernin has joined #openstack-ironic08:42
dtantsurwelcome back rpittau :) how was it?08:42
*** dougsz has quit IRC08:43
*** Qianbiao has quit IRC08:43
iurygregorygood morning dtantsur08:43
rpittaugreat, despite some stress because of the pandemic08:43
rpittaudecided to stay in Sardegna for the moment08:43
*** dougsz has joined #openstack-ironic08:44
*** mjturek has joined #openstack-ironic08:45
*** cp- has quit IRC08:45
*** rcernin has quit IRC08:46
*** mjturek1 has quit IRC08:46
dtantsurprobably a wise decision08:47
*** cp- has joined #openstack-ironic08:49
iurygregoryyay py36 jobs seems to be back to normal08:50
iurygregory\o/08:50
*** lmcgann_ has joined #openstack-ironic08:51
dtantsur\o/08:53
dtantsuranything requiring my attention right now?08:53
iurygregoryI'm trying to figure out the failures in metalsmith job on stable/victoria08:56
iurygregoryso we can merge the fix in case they change py36 to run on CentOS8 again08:56
iurygregorywithout an email... "hey we will switch the OS for this job"08:56
rpittauI see entropy is still high :)08:58
iurygregoryand we found a funny bug for idrac on friday XD08:59
*** k_mouza has joined #openstack-ironic08:59
rpittaufun-tastic :)09:00
iurygregoryhttps://storyboard.openstack.org/#!/story/200822109:02
openstackgerritDmitry Tantsur proposed openstack/metalsmith stable/victoria: Disable dstat in the CI  https://review.opendev.org/75601609:04
dtantsurcould you folks please apprve ^^^09:04
dtantsurre idrac: yeah, I've read this already09:04
dtantsuriurygregory: re metalsmith, see the patch above09:04
iurygregory+209:05
rpittauis dstat broken because of the new version? It was broken in ubuntu 20.0409:06
iurygregorythe fix for master was just so we could backport to victoria, I'm going to update master to get correct values from the idrac section09:06
*** Qianbiao has joined #openstack-ironic09:06
dtantsurrpittau: I suspect they switched devstack to 20.04 by default09:07
*** QianbiaoNG has quit IRC09:08
iurygregorydevstack was switched (ironic-base jobs are still bionic I think)09:09
rpittauok, good to know09:09
iurygregoryhttps://review.opendev.org/#/c/739448/09:09
patchbotpatch 739448 - ironic - migrate testing to ubuntu focal - 19 patch sets09:09
iurygregoryonly multinode job seems mad09:10
iurygregoryand ipv609:10
rpittauoh ok they included some changes of that patch in devstack, and there is a reference to the dstat issue as well09:12
openstackgerritAija Jaunteva proposed openstack/ironic master: Fix redfish BIOS apply config error handling  https://review.opendev.org/75577609:18
openstackgerritMerged openstack/ironic master: Imported Translations from Zanata  https://review.opendev.org/75571309:18
iurygregoryI'm wondering why the redfish tests became ilo specific or maybe I'm thinking too much https://github.com/openstack/ironic/blob/master/ironic/tests/unit/drivers/modules/test_image_utils.py#L420-L42209:26
openstackgerritAija Jaunteva proposed openstack/ironic master: Fix idrac-wsman BIOS step async error handling  https://review.opendev.org/75582109:28
*** sshnaidm|afk is now known as sshnaidm09:47
openstackgerritMerged openstack/ironic-ui stable/victoria: Update .gitreview for stable/victoria  https://review.opendev.org/75561709:51
*** dougsz has quit IRC09:57
*** pmannidi has joined #openstack-ironic09:58
*** dougsz has joined #openstack-ironic10:12
*** lmcgann_ has quit IRC10:17
*** Qianbiao has quit IRC10:19
openstackgerritMerged openstack/ironic-ui stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/75561810:37
arne_wiebalckSeems like our IPA images are getting to big and the nodes run OOM when downloading them from the PXE/tftp server ... the images are ~1.5GB uncompressed, created from the centos DIB with only a few additional elements. Anyone seen this?10:42
dtantsurarne_wiebalck: distributions are become overall larger10:43
arne_wiebalckdtantsur: yes, even consecutive builds over a few days let the image increase ...10:44
dtantsurI think only linux-firmware is hundreds of megabytes10:44
dtantsurlast time I researched this topic, I literally just unpacked the image, ran 'du' and looked what to remove10:44
dtantsurmaybe we should repeat this exercise10:44
*** priteau has joined #openstack-ironic10:44
arne_wiebalckI removed a few elements and that seemed to help.10:45
arne_wiebalckProblem is I need these elements :-D10:45
arne_wiebalckThis is on brand new nodes, so I guess that everyone will run into this very soon.10:46
openstackgerritDmitry Tantsur proposed openstack/metalsmith stable/victoria: Handle out-of-tree openstack module_utils  https://review.opendev.org/75590210:46
dtantsuriurygregory, rpittau, another attempt ^^^10:46
arne_wiebalckGiven that older nodes may even have less capabilities.10:46
dtantsurarne_wiebalck: what are the elements in question?10:46
arne_wiebalckdtantsur: the ones I removed?10:46
dtantsuryeah10:46
arne_wiebalckmellanox and docker-ce10:47
rpittauarne_wiebalck: any chance you can replace docker-ce with podman ? :)10:48
arne_wiebalckrpittau: I think gudrutis tried.10:48
dtantsurarne_wiebalck: docker in IPA? O___o10:53
arne_wiebalckdtantsur: yes10:54
rpittauthat sounds interesting and scary at the same time10:54
* dtantsur closes laptop and goes drinking10:54
arne_wiebalckdtantsur: this is used to do all the benchmarking, it downloads a container which encapsulates everything10:54
rpittauwow10:55
dtantsurarne_wiebalck: does it really have any benefits over chroot? have you considered systemd-nspawn?10:55
arne_wiebalckdtantsur: the IPA just gets a link what to download and an ES endpoint to send the data to10:55
arne_wiebalckdtantsur: the container is sth that is used in other environments, it is not ironic specific10:56
dtantsurat the very least, podman should have a good compatibility, and it's included in CentOS 8 (and likely 7)10:56
arne_wiebalckdtantsur: I'd like to avoid baking some special things for ironic10:56
arne_wiebalckdtantsur: ok, need to give this another go then10:56
arne_wiebalckdtantsur: but in the end, this is just symptoms10:56
dtantsurpodman is a swiftly developing project, if it sucked a year ago, it may be fine today10:57
arne_wiebalckdtantsur: it won't take long we will hit this without docler10:57
arne_wiebalck*docker10:57
arne_wiebalckdtantsur: ok10:57
dtantsurI don't disagree with that10:57
arne_wiebalckdtantsur: do we have a minimum list of DIBs the IPA needs?10:57
arne_wiebalckdtantsur: maybe I can remove sth else10:57
dtantsurDIBs?10:58
arne_wiebalckdtantsur: sorry, elements10:58
* arne_wiebalck goes and gets our list ...10:58
dtantsurironic-python-agent-ramdisk is the minimum10:58
arne_wiebalck    devuser \10:59
arne_wiebalck    disable-selinux \10:59
arne_wiebalck    centos \10:59
arne_wiebalck    ironic-python-agent-ramdisk \10:59
arne_wiebalck    ironic-hwmanager-cern \10:59
arne_wiebalck    local-config \10:59
arne_wiebalck    cleanup-kernel-initrd \10:59
arne_wiebalck    docker-ce10:59
arne_wiebalckoops, sorry ... but this is what we have rn10:59
arne_wiebalckplus mellanox10:59
arne_wiebalckthis has built up over years, so probably won't be able to explain the why for each of them ...11:00
openstackgerritMerged openstack/ironic-ui master: Imported Translations from Zanata  https://review.opendev.org/75594911:01
rpittauI highly advise to give podman another try, unless you do something exotic docker is overkill11:01
arne_wiebalckrpittau: ++11:02
arne_wiebalckdtantsur: sth else ... for the issue with the ESP missing when doing s/w RAID, I updated https://storyboard.openstack.org/#!/story/200816411:05
arne_wiebalckdtantsur: the ESP is not there since one disk is missing, and I thought it is since we switch off the node, but this is not the reason11:05
arne_wiebalckdtantsur: it seems that "sometimes" a disk gets kicked out11:06
arne_wiebalckdtantsur: I wrote some code to check and re-add, that seems to work and the failure rate goes down significantly (from 5/25 to 1/100)11:07
arne_wiebalckdtantsur: the remaining one seems to be udev trying to assemble the RAID while we're creating it11:08
arne_wiebalckdtantsur: the udev rules for mdadm make an exception for anaconda: https://github.com/neilbrown/mdadm/blob/master/udev-md-raid-assembly.rules#L511:08
arne_wiebalckdtantsur: we may need to disable event handling during raid creation as well11:09
arne_wiebalckdtantsur: I have some code which I am testing rn ... any thoughts?11:09
*** k_mouza has quit IRC11:12
*** k_mouza has joined #openstack-ironic11:30
dtantsurspecial casing \o/11:33
*** priteau has quit IRC11:34
dtantsurarne_wiebalck: can we somehow set this ENV[ANACONDA]?11:34
arne_wiebalckdtantsur: no idea, I don't know where this comes from tbh11:34
arne_wiebalckdtantsur: I was thinking sth like:11:35
arne_wiebalck"udevadm control--stop-exec-queue" while we create the md devices11:35
*** k_mouza has quit IRC11:35
arne_wiebalckbasically pause udevd11:35
dtantsuryep11:35
arne_wiebalckand11:36
dtantsurI wonder if the 1st problem is also caused by udev11:36
arne_wiebalck"udevadm control --start-exec-queue" after, ofc11:36
arne_wiebalckdtantsur: yes!11:36
arne_wiebalckdtantsur: I was thinking this as well.11:36
dtantsurokay, I have no problems with this11:36
arne_wiebalckdtantsur: atm, I do both: disable and re-add11:37
dtantsur(except that computers were clearly a mistake)11:37
dtantsurarne_wiebalck: did you try only disabling udev?11:37
arne_wiebalckdtantsur: I have not tried to only disable udevd11:37
dtantsuraha11:37
arne_wiebalckok, let me do try this as well, then we have all combinations11:38
*** priteau has joined #openstack-ironic11:38
arne_wiebalckre-adding alone fixes the problem almost11:38
arne_wiebalckboth 100% it seems11:38
arne_wiebalcklet's see if without udevd we do not have this issue in the first place ...11:38
dtantsurI wonder if we could split away a python software RAID library out of our code :D11:38
arne_wiebalck:-D11:44
* arne_wiebalck hopes there is none we missed when we started ... 11:44
*** k_mouza has joined #openstack-ironic11:46
*** k_mouza has quit IRC11:47
*** k_mouza has joined #openstack-ironic11:48
openstackgerritMerged openstack/python-ironic-inspector-client stable/victoria: Fix l-c job  https://review.opendev.org/75585311:51
iurygregorydtantsur, should we do the minor release now?11:58
*** Qianbiao has joined #openstack-ironic12:02
dtantsuriurygregory: I'm not sure about right now, we seem to be in a requirements freeze. You can ask the release team, but they'll likely ask to wait until the final release.12:08
iurygregorydtantsur, ack12:09
*** mkrai has quit IRC12:11
*** thiagop has joined #openstack-ironic12:17
openstackgerritYogesh Ramachandra proposed openstack/ironic master: Fix idrac-wsman bios factory-reset setting  https://review.opendev.org/74869612:30
*** jj123 has joined #openstack-ironic12:37
jj123Is it possible to use the ironic neutron network interface for a baremetal node that has a point to point cable directly to the NIC of an ironic-only compute node (rather than an external switch)? I tried all day yesterday and I couldn't get the baremetal port not to show 'binding failed'.12:43
jj123The documentation seems only to discuss: 1. Flat networking (I got this to work, but it's not what I need) 2. Neutron networking via an external switch.12:47
jj123I tried:1. Using openvswitch on the ironic-only compute node as the local_link_connection.2. Configuring a bridge on the NIC as the baremetal port, without local_link_connection. 3. Configuring a local flat network for that port alone, on the bridge.12:51
jj123To more closely replicate the documented setup for the neutron network interface, I'm considering trying: Adding a linux dev board as an external switch, running openvswitch, and using the networking-generic-switch driver... And once that works, then containerizing the external switch so it can run alongside the local openvswitch on the ironic-only12:58
jj123compute node. This idea sounds a bit convoluted.12:58
dtantsurjj123: all I know is that one person were researching the same idea, but he left the community long ago13:00
jj123Ultimately, I'm trying to achieve DVR multitenant netorking among a number of nodes that each have one or more baremetal devices on dedicated local NICs.13:01
dtantsurstarting with n-g-s may be the way to go, once it works you can see what it takes to remove it13:01
TheJuliadtantsur: could you run the meeting today? I'm feeling under the weather13:01
dtantsurTheJulia: absolutely! get better, take some rest13:01
dtantsurI'll #chair you in case you decide to jump in13:01
TheJuliaK13:01
* TheJulia goes back to sleep13:02
TheJuliaThanks13:02
jj123dtantsur: Thanks. I haven't yet tried networking-generic-switch. Seemed like it should work without it. Although now I'm questioning if the baremetal VNIC type requires the baremetal node to be behind a switch.13:02
dtantsurjj123: at the very least you need an ML2 driver that supports baremetal VNIC13:03
dtantsurI don't know if our networking-baremetal will help you. hjensas_ if you're around?13:03
jj123dtantsur: I tried with the networking-baremetal ML2 plugin. I'm using Ubuntu 18 with Ussuri on an ARM device, and Ubuntu 18 didn't have the networking-baremetal package, and it failed to build via pip, so I installed it from the Ubuntu 19 eoan package.13:05
*** jj123 has quit IRC13:08
*** jj123 has joined #openstack-ironic13:09
dtantsurrpioso: hi! I'm adding requests for details to your items on https://etherpad.opendev.org/p/ironic-wallaby-ptg please expand them with some context13:15
*** uzumaki has quit IRC13:18
*** k_mouza has quit IRC13:18
openstackgerritMerged openstack/metalsmith stable/victoria: Handle out-of-tree openstack module_utils  https://review.opendev.org/75590213:19
*** k_mouza has joined #openstack-ironic13:19
*** Qianbiao has quit IRC13:29
*** Qianbiao has joined #openstack-ironic13:30
openstackgerritDmitry Tantsur proposed openstack/ironic-prometheus-exporter stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/75561013:30
iurygregorydtantsur, release team said it's ok to do a minor version13:34
iurygregoryI'm going to push the patches13:34
dtantsuriurygregory: okay, cool. let's just wait for the gitreview changes13:34
iurygregoryack13:34
dtantsurI've realized we haven't merge them13:34
dtantsurand thanks for checking13:34
iurygregorycrazy CI \o/13:35
openstackgerritDmitry Tantsur proposed openstack/bifrost stable/ussuri: Ensure that repositories are owned by the running users  https://review.opendev.org/75397413:40
*** tzumainn has joined #openstack-ironic13:42
arne_wiebalckQianbiao: ftarasenko: you're still good to chat about your (network) setup tmrw?13:42
Qianbiaoarne_wiebalck what should i prepare btw.13:43
Qianbiaois it a zoom or chat room13:43
Qianbiaoif zoom, i am afraid i can not quite understand.13:44
arne_wiebalckyou don't have to explicitly prepare I think, but if you have something already you could certainly use it13:45
QianbiaoHi dtantsur, if you got a minute, may u help checking whether the IPA CI is good.13:45
arne_wiebalckQianbiao: it will be on zoom13:46
dtantsurQianbiao: which branch?13:46
Qianbiaothis patch fails always, i donot know what's the reason from log. https://review.opendev.org/#/c/752065/713:46
patchbotpatch 752065 - ironic-python-agent (stable/train) - Fix: make Intel CNA hardware manager none generic - 7 patch sets13:46
Qianbiaodtantsur train13:46
dtantsurtrain may be broken still. iurygregory do you know anything?13:46
*** mkrai has joined #openstack-ironic13:47
*** jj123 has quit IRC13:47
iurygregoryhaven't look stable/train13:47
iurygregorylet me take a look at the patch13:47
dtantsurthe failures seem to repeat a recent pattern13:47
Qianbiaoiurygregory thanks.13:47
dtantsurI just don't remember what fixed it13:47
dtantsur(on later branches)13:47
iurygregoryomg too many tabs open XD13:48
*** rloo has joined #openstack-ironic13:48
*** tkajinam has quit IRC13:49
*** tkajinam has joined #openstack-ironic13:49
Qianbiaoarne_wiebalck i will attand, but i am afraid I can not share, (I can not speak english fluently)13:50
arne_wiebalckQianbiao: ok, no problem!13:51
Qianbiaothanks for understanding. :)13:51
arne_wiebalckQianbiao: :)13:52
*** priteau has quit IRC13:52
iurygregoryI think I've found the problem for train13:52
iurygregorylet me give a try backporting13:53
*** jj123 has joined #openstack-ironic13:53
iurygregoryKernel Panic is our best friend <313:53
Qianbiaoiurygregory nice, update me if fix.13:53
Qianbiao:)13:53
iurygregoryofc the cherry-pick is not clean...13:55
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Documentation: make it clearer that Ussuri does not have bifrost-cli  https://review.opendev.org/75607813:55
openstackgerritMerged openstack/ironic-prometheus-exporter stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/75561013:57
*** k_mouza has quit IRC13:58
*** priteau has joined #openstack-ironic13:59
*** Goneri has joined #openstack-ironic13:59
*** uzumaki has joined #openstack-ironic14:02
*** k_mouza has joined #openstack-ironic14:03
dtantsurbfournie: hi, do you intend to provide any further patches for https://storyboard.openstack.org/#!/story/2008168 ?14:04
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-python-agent stable/train: Increase memory for tinyipa jobs  https://review.opendev.org/75608214:06
iurygregoryQianbiao, let's see how it goes ^14:06
Qianbiaoiurygregory thanks, should this backport to like rocky/queens?14:07
iurygregoryI don't think so14:07
iurygregorymaybe stein if it's broken also14:08
iurygregorybut rocky nope14:08
iurygregoryqueens... maybe14:08
dtantsurajya: hi, do you have any clue about https://storyboard.openstack.org/#!/story/2008094 ?14:10
Qianbiaook, after train is merged, i will test stein/rokcy/queens too.14:10
Qianbiaoiurygregory ^^^14:10
Qianbiaoto see if there is a problem in those branches.14:11
iurygregorygood luck on things older than stein =)14:11
rpiosodtantsur: Ack re: deets.14:11
Qianbiao:)14:11
iurygregoryrocky is not worth since it's EOL14:12
iurygregoryops Extended Maintenance14:12
Qianbiaook14:12
Qianbiaowhy dib packaged image has 1002:1002 group permission under /bin14:13
Qianbiaothere are no user has id 100214:13
iurygregorymaybe we will even move to Unmaintained during Wallaby...14:13
*** derekh has quit IRC14:24
*** dougsz has quit IRC14:26
*** mkrai has quit IRC14:27
*** mkrai_ has joined #openstack-ironic14:27
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent-builder master: Use tinycore 11 to build tinyipa  https://review.opendev.org/71164714:34
ajyadtantsur: haven't seen it, but if possible, can try to update iDRAC version to see if it's the case for newer versions. I14:36
ajyaI'll take a closer look later14:36
bfourniedtantsur: no we don't plan on more patches for this as it appears to be hardware specific14:41
dtantsurok, thx14:42
*** cdearborn has joined #openstack-ironic14:45
openstackgerritAnkit Kumar proposed openstack/ironic master: Adds config parameter kernel_append_param for iLO  https://review.opendev.org/75518914:50
*** tonyb has quit IRC14:50
*** stendulker has joined #openstack-ironic14:50
*** rloo has quit IRC14:56
*** rloo has joined #openstack-ironic14:56
*** ebagakis has joined #openstack-ironic14:58
dtantsur#startmeeting ironic15:00
openstackMeeting started Mon Oct  5 15:00:07 2020 UTC and is due to finish in 60 minutes.  The chair is dtantsur. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: ironic)"15:00
openstackThe meeting name has been set to 'ironic'15:00
rpittauo/15:00
dtantsurhey-hey15:00
erbarro/15:00
dtantsur#chair TheJulia15:00
openstackCurrent chairs: TheJulia dtantsur15:00
dtantsur#link https://wiki.openstack.org/wiki/Meetings/Ironic our agenda15:00
iurygregoryo/15:00
stendulkero/15:00
bdoddo/15:00
ebagakiso/15:00
arne_wiebalcko/15:00
ajyao/15:00
rpioso\o15:00
cdearborn\o15:01
rlooo/15:01
Qianbiaoo115:01
Qianbiaoo/15:01
dtantsur#topic Announcements / Reminder15:02
*** openstack changes topic to "Announcements / Reminder (Meeting topic: ironic)"15:02
bfournieo/15:02
dtantsur#info Victoria releases have been done, please watch out for potential backports15:02
dtantsurmeaning, the master branch is now progressing towards Wallaby15:02
dtantsur#info Wallaby PTG etherpad - https://etherpad.opendev.org/p/ironic-wallaby-ptg15:02
dtantsurPlease keep the ideas coming. But! Please make sure to include some details, links, etc. And your name.15:03
iurygregoryI would add that now it's safe to recheck masterand stable/victoria (the problem with the python36 job was solved)15:03
rlooyay, congrats all15:03
dtantsurthanks iurygregory15:03
JayFo/15:04
dtantsurand yay, congrats. it has been a great release cycle!15:04
iurygregoryyeah ++ congrats everyone!15:04
stendulkercongrats everyone !!15:04
dtantsurnote that OpenStack Victoria will be officially released and announced next week (if I understand it right)15:04
dtantsuranything else to announce or remind of?15:05
cdearbornwow, that's great - congrats all, and many thanks to all who helped turn the release crank15:05
openstackgerritMerged openstack/bifrost master: Another case for skip_package_install  https://review.opendev.org/75598115:05
openstackgerritMerged openstack/bifrost stable/ussuri: Ensure that repositories are owned by the running users  https://review.opendev.org/75397415:05
iurygregoryI think we can move dtantsur =)15:05
rlooCongrats TheJulia for being Wallaby PTL!15:05
dtantsurah, right. TheJulia++15:05
iurygregoryTheJulia++15:06
dtantsurokay, we had no actions items from the previous meeting15:06
cdearborn+215:06
rlooand a huge thanks to TheJulia for offering to be Wallaby PTL :)15:06
dtantsurindeed :)15:06
dtantsurI guess status updates don't make much sense before we plan Wallaby?15:06
* dtantsur hears no objections15:07
* iurygregory agrees with dtantsur 15:07
dtantsur#topic Deciding on priorities for the coming week15:07
*** openstack changes topic to "Deciding on priorities for the coming week (Meeting topic: ironic)"15:07
dtantsurWe did not have many items, and we've finished most of them15:07
dtantsurI'd like to propose that we concentrate on specs and finishing any remaining items from V?15:08
iurygregorymakes sense to me15:08
*** ebagakis has quit IRC15:10
dtantsurany patches or specs that should be there?15:10
dtantsurthe list is pretty light so far15:10
*** priteau has quit IRC15:11
*** ebagakis has joined #openstack-ironic15:11
dtantsurajya: could you add your idrac bug fixes to the list?15:11
iurygregoryI think the fixes we had for CI are already merged15:12
dtantsurI think so too15:12
iurygregorywe need to probably try to make focal working to switch our jobs...15:12
ajyadtantsur: ok15:12
iurygregoryI think the only "blocker" is our multinode job if I do remember15:12
*** ociuhandu_ has joined #openstack-ironic15:13
Qianbiaodtantsur may we facilitate the dib for building ramdisk deploy image15:13
dtantsuryou mean, grenade?15:13
iurygregoryno no15:13
dtantsurQianbiao: not sure what exactly you propose15:13
iurygregoryironic-tempest-ipa-wholedisk-direct-tinyipa-multinode15:13
iurygregoryhttps://review.opendev.org/#/c/739448/15:13
patchbotpatch 739448 - ironic - migrate testing to ubuntu focal - 19 patch sets15:13
Qianbiaoramdisk deploy interface does not have an official tool to build image for now15:14
Qianbiaoand has no document talking about that too. may we improve this part in W version.15:14
dtantsurQianbiao: yep, that's a problem. I'd really love to see that improved, but it depends on somebody allocating time to work on it.15:14
dtantsuriurygregory: aren't we already on focal by default?15:15
rpittaunot for all the tests15:15
rpittauthe dstat issue is related to https://bugs.launchpad.net/ubuntu/+source/dstat/+bug/186661915:15
openstackLaunchpad bug 1866619 in dstat (Ubuntu) "OverflowError when machine suspends and resumes after a longer while" [Undecided,Confirmed]15:15
iurygregory^ this15:15
*** ociuhandu has quit IRC15:15
iurygregorydtantsur, https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L615:16
iurygregory=)15:16
rpittauunfortunately dstat is bugged in focal15:16
dtantsurdo we need this dstat thing?15:16
dtantsurmaybe just disable it?15:16
dtantsuriurygregory: ah, I see. well, okay, we need to make migrating to focal a very high priority15:16
iurygregorydtantsur, yeah15:16
rpittauI'll push the rebase now15:16
iurygregorymaybe even put the multinode novoting sicne the other jobs seem happy15:17
openstackgerritRiccardo Pittau proposed openstack/ironic master: migrate testing to ubuntu focal  https://review.opendev.org/73944815:17
iurygregoryand we can backport to stable/victoria since we should be testing focal15:17
*** ociuhandu_ has quit IRC15:17
dtantsurI also suspect that the default versions of opensuse and fedora have changed, breaking the corresponding bifrost jobs15:17
iurygregorythanks rpittau !15:17
dtantsuriurygregory: s/can/have to/ yes15:17
iurygregorydtantsur, yeah sorry =)15:17
rpittaudtantsur: they have indeed, checked today15:18
dtantsuroh of course15:18
dtantsurany ideas for a fix?15:18
dtantsurokay, we should probably discuss it later15:18
iurygregoryyeah15:18
dtantsurhow is the weekly priority list looking?15:18
openstackgerritLeo McGann proposed openstack/ironic master: [WIP] Adds Security Interface to Node  https://review.opendev.org/75583615:18
iurygregoryI haven't look but I will try to look at it15:19
* dtantsur hears crickets15:20
rpittaulooks good15:20
stendulker looks good15:20
iurygregorylgtm15:20
dtantsur#topic Baremetal SIG15:21
*** openstack changes topic to "Baremetal SIG (Meeting topic: ironic)"15:21
dtantsur#info First meeting planned for next Tue, Oct 6 at 2pm UTC15:21
dtantsurarne_wiebalck: anything else?15:21
arne_wiebalckagenda/etherpad is here https://etherpad.opendev.org/p/bare-metal-sig15:21
arne_wiebalckno, that's it15:22
dtantsur#link https://etherpad.opendev.org/p/bare-metal-sig15:22
dtantsurthank you!15:22
dtantsur#topic Open discussion15:22
*** openstack changes topic to "Open discussion (Meeting topic: ironic)"15:22
dtantsuranything?15:22
openstackgerritMerged openstack/python-ironic-inspector-client stable/victoria: Update .gitreview for stable/victoria  https://review.opendev.org/75112615:23
openstackgerritMerged openstack/python-ironic-inspector-client stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/75112715:23
dtantsurokay, thank you folks!15:24
Qianbiaothank you, dtantsur15:24
dtantsur#endmeeting15:24
*** openstack changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/ironic/ | Bugs: https://storyboard.openstack.org/#!/project_group/75 | Contributors are generally present between 6 AM and 12 AM UTC, If we do not answer, please feel free to pose questions to openstack-discuss mailing list."15:24
openstackMeeting ended Mon Oct  5 15:24:28 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:24
rpiosodtantsur: ty15:24
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-10-05-15.00.html15:24
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-10-05-15.00.txt15:24
openstackLog:            http://eavesdrop.openstack.org/meetings/ironic/2020/ironic.2020-10-05-15.00.log.html15:24
rpittauthanks!15:24
arne_wiebalckthanks dtantsur15:24
rpittaudtantsur: for bifrost, I just realized that the CI is already running with Fedora 32, and it looks ok, so I think I did something wrong on my machine15:26
cdearborntx dtantsur!15:26
iurygregoryrpittau, we had the bug for the DB that was fixed =)15:26
rpittauyep15:26
iurygregoryif I do remember that was the only problem when I was testing15:27
iurygregorythey added F32 to CI?15:27
rpittauI probably have some old stuff, I will redeploy tomorrow to be 100% sure15:27
rpittauthe cI looks good anyway15:27
rpittauiurygregory: yeah15:27
rpittaufedora-latest is F3215:27
iurygregory\o/15:27
clarkbthat was forced by f31's systemd being broken by locale things (something like that anyway, it broke ansible)15:31
clarkbbasically figured since f31 is broken across the baord that was a good time to roll forward15:31
rpittauah!15:31
iurygregorydtantsur, release for our clients are up15:33
dtantsuriurygregory: thanks!15:33
*** gyee has joined #openstack-ironic15:34
openstackgerritMerged openstack/sushy master: Log only fields set in redfish response, not entire json  https://review.opendev.org/75506815:35
*** ebagakis has quit IRC15:37
openstackgerritDmitry Tantsur proposed openstack/ironic master: Wiping agent tokens on reboot via API - take 2  https://review.opendev.org/75611515:37
dtantsurI'll never have enough facepalms for ^^^15:37
dtantsurbfournie: ^^^15:37
dtantsurbfournie: also FYI "baremetal node inspect" plays with ironic states somewhat nicer than "baremetal introspection start"15:40
openstackgerritDmitry Tantsur proposed openstack/bifrost stable/victoria: Another case for skip_package_install  https://review.opendev.org/75611615:45
openstackgerritDmitry Tantsur proposed openstack/bifrost master: Explicitly unset OS_CLOUD when verifying openrc content  https://review.opendev.org/75611715:47
iurygregorybfournie, do you want to backport https://review.opendev.org/755068 ?15:49
patchbotpatch 755068 - sushy - Log only fields set in redfish response, not entir... (MERGED) - 1 patch set15:49
bfournieiurygregory: sure, I'll backport that15:50
iurygregorybfournie, ack o/15:50
bfourniedtantsur: nice catch15:50
dtantsurwould be nicer if I had caught it last time :D15:50
*** lucasagomes has quit IRC15:56
*** Qianbiao has quit IRC15:56
openstackgerritVerification of a change to openstack/metalsmith failed: Rework jobs so failed devstack does not retry the job  https://review.opendev.org/75268015:58
iurygregorydtantsur, my bad we want 4.4.0 right?16:03
dtantsuriurygregory: no worries, it's confusing. year, bump the 2nd number.16:05
rpittaugoodnight! o/16:06
*** rpittau is now known as rpittau|afk16:06
iurygregorygn rpittau|afk o/16:07
iurygregorydtantsur, fixed16:07
openstackgerritBob Fournier proposed openstack/sushy stable/victoria: Log only fields set in redfish response, not entire json  https://review.opendev.org/75612616:21
*** dtantsur is now known as dtantsur|afk16:21
dtantsur|afko/16:21
arne_wiebalckbye everyone o/16:21
openstackgerritAmeya Raut proposed openstack/ironic stable/train: DRAC: Fix RAID create_config clean step  https://review.opendev.org/75060516:22
*** tosky has quit IRC16:22
*** jtomasek has quit IRC16:24
mjturekTheJulia: I wanted to reach out about the powerkvm ci job - I know it's been down unreasonably long, but I intend to bring it back. The CI team has moved all of our jobs to zuulv3 after a ton of work in the last year and the job has some special sauce that they need to better understand. I have to find some cycles to bring them up to speed and restore the molteniron service for them16:26
*** mkrai_ has quit IRC16:30
*** mkrai__ has joined #openstack-ironic16:30
*** stendulker has quit IRC16:57
*** HardCase has joined #openstack-ironic16:57
*** k_mouza has quit IRC17:15
iurygregorymjturek, thanks for the information, we know a bit about zuulv3 since we did the migration from zuulv2 - feel free to ask questions and we can try to help =). About molteniron project is a bit inactive afaik (since I joined in 2018)...17:16
mjturekiurygregory: thanks! Yeaaah - we had the initial commit but not many improvements after that. I think it'll still do what we need17:18
iurygregorymjturek, you probably have this link already https://docs.openstack.org/project-team-guide/zuulv3.html but in any case here it's =)17:20
mjturekyep! thank you iurygregory :)17:22
*** ociuhandu has joined #openstack-ironic17:25
*** ociuhandu has quit IRC17:29
NobodyCamHappy Monday Ironic folks17:53
*** mkrai__ has quit IRC17:56
iurygregoryhappy monday NobodyCam o/18:11
*** k_mouza has joined #openstack-ironic18:26
*** k_mouza has quit IRC18:30
*** k_mouza has joined #openstack-ironic18:37
*** k_mouza has quit IRC18:42
*** k_mouza has joined #openstack-ironic18:42
openstackgerritMerged openstack/metalsmith stable/victoria: Rework jobs so failed devstack does not retry the job  https://review.opendev.org/75268018:47
*** k_mouza has quit IRC18:47
*** k_mouza has joined #openstack-ironic18:51
*** k_mouza has quit IRC18:56
*** k_mouza has joined #openstack-ironic19:10
*** k_mouza has quit IRC19:14
*** tosky has joined #openstack-ironic19:29
*** k_mouza has joined #openstack-ironic19:41
openstackgerritMerged openstack/ironic stable/victoria: Update .gitreview for stable/victoria  https://review.opendev.org/75563019:41
openstackgerritMerged openstack/ironic stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/75563119:41
*** k_mouza has quit IRC19:45
*** lmcgann_ has joined #openstack-ironic19:59
*** hjensas__ has joined #openstack-ironic20:01
dkingWhat's the best way to add in custom Python packages, like custom Hardware Managers, to the IPA image build? Would it be best to create a DIB element to add it?20:02
*** hjensas_ has quit IRC20:02
arne_wiebalckdking: this is what we do, yes: a simple element with a package-installs.yaml is  enough20:05
dkingarne_wiebalck: Thank you very much. That sounds simple enough. I'll give it a shot.20:06
*** jungleboyj_ has joined #openstack-ironic20:20
*** jungleboyj has quit IRC20:25
*** mnasiadka has quit IRC20:25
*** jroll has quit IRC20:25
*** jungleboyj_ is now known as jungleboyj20:25
*** jroll has joined #openstack-ironic20:26
*** k_mouza has joined #openstack-ironic20:33
*** k_mouza has quit IRC20:38
*** uzumaki has quit IRC20:49
*** k_mouza has joined #openstack-ironic20:54
*** k_mouza has quit IRC20:58
*** lmcgann_ has quit IRC21:01
*** tonyb has joined #openstack-ironic21:22
jj123Is the baremetal vnic-type similar to or based on another vnic-type that doesn't require a baremetal instance to troubleshoot? Maybe the direct or direct-physical vnic-types are relevant, but I haven't found any documentation about using them. I'm trying to use a regular NIC on a compute node for neutron (routed) networking to a baremetal instance.21:28
jj123Or, is there a way to use a baremetal vnic-type without creating an instance? I keep getting binding-failed.21:50
*** thiagop has quit IRC22:17
*** rcernin has joined #openstack-ironic22:19
*** thiagop has joined #openstack-ironic22:21
*** PrinzElvis has quit IRC22:24
*** dsneddon has joined #openstack-ironic22:24
*** PrinzElvis has joined #openstack-ironic22:27
*** zzzeek has quit IRC22:46
*** rloo has quit IRC22:46
*** k_mouza has joined #openstack-ironic22:50
*** zzzeek has joined #openstack-ironic22:50
*** k_mouza has quit IRC22:55
*** tosky has quit IRC23:04
rpiosodtantsur|afk: I added details to my proposed PTG agenda items. "Secure erase" seems to be a duplicate.23:22
*** k_mouza has joined #openstack-ironic23:24
*** k_mouza has quit IRC23:29
*** thiagop has quit IRC23:36

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!