15:00:55 #startmeeting ironic 15:00:55 Meeting started Mon Jul 27 15:00:55 2020 UTC and is due to finish in 60 minutes. The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:56 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:58 Good morning everyone! 15:00:59 The meeting name has been set to 'ironic' 15:01:02 o/ 15:01:03 \o 15:01:05 o/ 15:01:08 o/ 15:01:10 o/ 15:01:17 o/ 15:01:18 o/ 15:01:23 o/ 15:01:40 Our agenda this week is quite full, so hopefully we'll get through it quickly! 15:01:43 #link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting 15:02:03 o/ 15:02:14 #topic Announcements / Reminders 15:02:23 A few different items! 15:03:03 o/ 15:03:27 First off as everyone may have noticed we've been having a lot of issues with CI. Largely these have revolved around seemingly random failures that are race conditions due to substrate clouds or totally random issues. Please remember to log these on the whiteboard so everyone can keep track of them. We all need to remember to do it and update it as fixes merge. 15:04:01 #link https://etherpad.opendev.org/p/IronicWhiteBoard 15:04:08 Next up! 15:04:22 #info Ironic now has a website! (That is if you didn't catch this during OpenDev!) 15:04:36 \o/ 15:04:45 #link https://ironicbaremetal.org/ 15:05:03 YAY!!! 15:05:10 #info The whitepaper has also been published \o/ 15:05:12 #link https://www.openstack.org/bare-metal/how-ironic-delivers-abstraction-and-automation-using-open-source-infrastructure 15:05:23 A special thanks to everyone that contributed and helped make that a success! 15:05:59 Including OSF staffers who helped review and provide feedback in the very end! 15:06:26 Next up looks like arne_wiebalck has a call scheduled for the Redfish InterOp profiles work 15:06:31 arne_wiebalck: is there a link? 15:06:35 Yes, there is a zoom call tomorrow to discuss the Redfish interop profile status, everyone interested is welcome to join at 2pm UTC: https://cern.zoom.us/j/97073886481 15:06:41 #info Call to discuss Redfish Interop Profile status, Tue 28 at 2pm UTC 15:06:51 #link https://cern.zoom.us/j/97073886481 15:08:18 Two reminders! 15:08:27 Well 3 15:08:30 First! 15:08:57 #info The CFP for the Open Infrastucture Summit is presently open through August 4th 15:09:48 #link https://cfp.openstack.org/ 15:09:52 Second! 15:10:41 #info We're quickly coming up on August 10th which is around when we committed to making our next intermediate release. 15:10:45 Third! 15:11:30 #info In order for us to be able to be able to make that date, it would be extremely helpful for people who are not presently core reviewers to spend a little time and review 1-2 patches a day. It doesn't take long, it would just help us in the long run. 15:11:33 Wow, that was a lot 15:11:38 Does anyone have anything else? 15:11:50 We have not done this during the past summits, but for the summit we may want to get a slot for the bare metal SIG, to promote the new site, the paper and attract/organise people to work on items ... what do you think? 15:12:08 arne_wiebalck: I think it is a wonderful idea 15:12:16 I would encourage everyone to submit something! 15:12:18 arne_wiebalck ++ 15:12:20 Increase visibility is the whole point. 15:12:27 not just would, I do encourage everyone to submit something! 15:12:35 I'm even working on something in the back of my brain! 15:12:40 * iurygregory will \o/ 15:12:59 We plan submit sth on scaling Ironic. 15:13:18 ++ 15:14:16 Also, if anyone has more than than the CFP submission limit, please let me know and we can likely figure something out. 15:14:33 Anyway, if nobody else has anything to remind us of, we should move on! 15:15:26 Looks like we have no action items from the last meeting 15:15:35 So we will move directly to subteam status reports 15:15:43 #topic Review subteam status reports 15:15:54 #link https://etherpad.opendev.org/p/IronicWhiteBoard 15:16:16 Starting at line 341 15:17:57 Is there any update on the changes for bugfix branches? 15:18:24 I gave a try to override checkout but didn't work 15:18:31 not sure if dtantsur found something 15:18:49 I haven't done anything around it 15:18:55 iurygregory: do you have a patch? 15:19:02 dtantsur, I do 15:19:06 I feel like part of the idea originally was to try and use bifrost on it since we don't have stable hardcoded into its logic really 15:19:20 it was https://review.opendev.org/#/c/741825/ 15:19:20 patch 741825 - ironic-python-agent (bugfix/6.2) - [DNM] Testing setting IRONIC_PYTHON_AGENT_BRANCH - 2 patch sets 15:19:24 TheJulia: Mind taking a look at https://review.opendev.org/#/c/739174/ 15:19:25 patch 739174 - ironic - Decouple the ISO creation logic from redfish - 7 patch sets 15:19:40 I saw you recently made some changes in redfish/boot.py 15:19:42 vmud213: we're in our weekly meeting right now, we're not to the section on priorities for the week yet 15:19:54 oh.ok 15:19:57 sorry 15:19:58 vmud213: worth noting, that change is also in that list already 15:20:00 no worries 15:20:10 iurygregory: still fighting grenade? 15:20:23 dtantsur, patch1 I went trying override-checkout: bugfix/6.2 and patch 2 IRONIC_PYTHON_AGENT_BRANCH: bugfix/6.2 15:20:24 No data found for patch 2 15:20:38 TheJulia, yeah =( 15:20:59 haven't find a clear path on how to move forward 15:21:02 :( 15:21:15 Would it help to have a small meeting to try and walk through issues and maybe brainstorm ideas? 15:21:40 iurygregory: you need override-checkout and you need it per project 15:21:55 i.e. don't try to use bugfix/6.2 on ironic, ironic has bugfix/15.1 15:21:56 yeah, I will send and email to openstack-discuss so we can find a time 15:22:40 dtantsur, I will give a try overriding the ironic 15:22:54 we need bugfix branches for ironic and master for everything else 15:23:00 * for ironic and inspector 15:23:07 Re scale issues, no update other than patch for OOMs is still in review. Dtantsur did make a note on an issue in our db logic for ports that we should likely take a look at if someone has database spoons handy 15:25:16 Looks like stevebaker has been making good progress on cleaning up wsme stuffs \o/ 15:26:12 dtantsur: any update on ramdisk tls? 15:26:45 iurygregory: Looks like your working through reviews on job changes to simplify jobs? 15:27:24 TheJulia, yeah, need to first figure out a few things related to the flavor that we need to use partition test 15:27:36 since nova override the capabilites 15:27:37 iurygregory: interesting 15:27:44 ugh 15:27:49 NOVAAAAA!!! 15:28:01 so I was testing https://review.opendev.org/742381 as depends on for https://review.opendev.org/741445 15:28:02 patch 742381 - ironic - Create altbaremetal flavor - 1 patch set 15:28:03 patch 741445 - ironic-tempest-plugin - Update basic_ops - 12 patch sets 15:28:09 dtantsur: mgoddard: Anything new on in-band deploy steps? 15:28:14 and the tempest as depends on for https://review.opendev.org/741886 15:28:14 patch 741886 - ironic - Add tempest default_boot_option - 2 patch sets 15:28:19 TheJulia: trying to make in-band RAID work in the CI 15:28:28 staring at the logs as we speak 15:28:31 tempest is happy the the other ironic patches are unhappy so I'm debugging 15:28:34 dtantsur: ++ 15:28:37 the patch is https://review.opendev.org/741227 15:28:38 patch 741227 - ironic-tempest-plugin - POC deploy-time RAID - 5 patch sets 15:29:32 Looks like the refish interop profile one has been updated 15:29:49 arne_wiebalck: shall we discuss the baremetal program/sig item during the specific window for that topic area? 15:29:55 yes 15:30:00 Okay then! 15:30:11 Is everyone good to proceed forward? 15:30:15 ++ 15:30:16 +1 15:30:31 let's 15:30:39 #topic Deciding on priorities for the coming week 15:30:49 #link https://etherpad.opendev.org/p/IronicWhiteBoard 15:31:00 Starting at line 142 15:31:16 First I'll delete the merged items from the list... and almost everything on that list HAS merged 15:31:18 Thanks everyone! 15:31:55 iurygregory: is there a next step for dhcp-less? 15:32:20 TheJulia, so we just need to sync if we want to have the element on diskimage-builder or on ipa-builder I think 15:33:55 we won't need to use dhcp-all-interfaces element on the image 15:34:26 Dmitry Tantsur proposed openstack/ironic-tempest-plugin master: POC deploy-time RAID https://review.opendev.org/741227 15:34:26 https://specs.openstack.org/openstack/ironic-specs/specs/not-implemented/L3-based-deployment.html#ramdisk-impact 15:34:31 So as for items to add this week, I went through the projects this mroning and made a list starting at 231. Looks like ?dmitry? proposed some items starting at 224 15:35:02 Any objections to adding them all to the list? 15:36:41 * TheJulia takes silence as none 15:36:43 nope 15:38:35 TheJulia: We would appreciate reviews of https://review.opendev.org/#/c/740721/ 15:38:35 patch 740721 - ironic-specs - [WIP] Operationally efficient & fast-track provisi... - 6 patch sets 15:38:42 rpioso: it is already on the list 15:38:50 TheJulia: +1 :-) 15:39:02 Okay, aside from a little clean up, is there anything else to add? 15:39:29 looks good, I guess 15:40:37 Awesome 15:40:41 Then I guess we should proceed? 15:40:54 ++ 15:40:58 #topic Baremetal SIG 15:41:08 arne_wiebalck: do you want to do the typing ? :) 15:41:23 Sure :) Given the input from the opendev event, I was considering to introduce regular SIG meetings, to organise/tackle the work for the next item(s) after the white paper. 15:41:56 Does that sound sensible or would you rather despise more meetings? 15:42:06 I was thinking monthly maybe. 15:42:23 Any thoughts? 15:42:56 There are various topics that were raised during the event, I listed them on the white board as well. 15:43:13 I think monthly should be fine 15:43:15 line 424 15:43:18 monthly sounds good 15:43:44 maybe written updates from time to time? in the etherpad 15:43:52 Ok, let's see how it goes and if there is some attendance/interest. 15:44:05 I think monthly sig meetings woul dbe good 15:44:08 the weekly Ironic meeting should still get updates I think 15:44:10 +1 15:44:13 ++ 15:44:19 I think that is reasonable 15:44:23 but short, no discussion 15:44:25 absolutely 15:44:54 preferences for zoom or irc for the monthly meeting? 15:45:35 I kind of feel like actual video call might garner a little more engagement 15:45:49 yeah, I think so, too 15:45:59 as much as many of us might dislike putting on pants 15:46:07 :-D 15:46:31 just keep the camera always pointed to the top body part :) 15:46:43 rpittau: ++ 15:46:45 Finding a time suitable for everyone will be challenging. 15:46:51 Yeah, timing is always hard 15:47:21 Maybe the same slots we used for the white paper, like Tue 2pm UTC 15:47:31 this seemed to fit most 15:47:33 An open doodle for 1-2 weeks might yield a common window especially if the mailing list is reminded 15:47:37 That did kind of fit 15:47:59 Ok, we can also doodle for it, yes. 15:48:14 Is there anything else baremetal sig related? 15:48:20 I don't think so. 15:48:33 Okay then, well then next up is RFE Review 15:48:36 #topic RFE Review 15:48:48 kaifeng asked for us to quickly revisit the smart consoles RFE 15:48:54 #link https://storyboard.openstack.org/#!/story/2007876 15:49:36 it sounds good to me 15:51:41 Any comments/questions? 15:51:51 If there is none, I'll mark it rfe-approved and we can move on? 15:51:59 lgtm 15:52:18 Dmitry Tantsur proposed openstack/ironic master: Make the final deploy step validation actually fail deploy https://review.opendev.org/743266 15:52:25 ohhhh ahhh 15:52:36 Okay, time for Open Discussion with 8 minutes left 15:52:39 #topic Open Discussion 15:53:00 Does anyone have anything to discuss? Or is everyone busy working on proposals for the Open Infrastructure Summit? 15:53:46 TheJulia: I did run `openstack baremetal volume target create `. I'll get all steps down and then share it here. Thanks so much for your help! 15:54:16 just keep an eye on https://review.opendev.org/743248 as it should fix the ipa ussuri CI 15:54:16 patch 743248 - ironic-python-agent (stable/ussuri) - Update jobs - 1 patch set 15:54:27 rpittau: awesome 15:55:19 naved001: Hmm... Any chance you could provide us a list of the commands your executing with arguments? redacted of course. Also what openstack baremetal node show indicates when you tell it to deploy. Seems like something is not quite right somewhere... :( 15:56:17 John Garbutt proposed openstack/networking-generic-switch master: Add support for Cumulus 4.x NCLU driver https://review.opendev.org/743269 15:56:37 email sent to openstack-discuss http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016102.html =) 15:56:53 there is an issue with deploy steps that have higher priority than `deploy` step and that require rebooting. They execute successfully but they reset PXE config and deployment can't proceed. Any comments? 15:57:03 iurygregory: awesome 15:57:23 Should PXE/boot config happen closer to `deploy` step? Or make boot settings persistent and remove afterwards when no longer needed? 15:57:39 ajya: I was wondering the same thing yesterday because I was wondering how we could wire in a kexec step that is optional but can still be chosen 15:58:03 Dmitry Tantsur proposed openstack/ironic-python-agent master: Allow erase_devices_metadata to be used as a deploy step https://review.opendev.org/743272 15:58:06 ajya: pxe/boot config is in the preparation step 15:58:18 * TheJulia has this feeling many things are going to become steps soon 15:58:23 TheJulia: yes, at the very beginning of deploying 15:58:34 yep, DeployInterface.prepare should probably become a step 15:58:35 but before actual deploy steps are started 15:58:43 Verification of a change to openstack/ironic failed: Change non-tinyipa jobs to use multiple cores https://review.opendev.org/741577 15:59:02 ajya: what you describe is probably an issue in the reboot_requested flag handling 16:00:03 dtantsur: Could be. A reboot consumes the PXE boot once. 16:00:06 dtantsur: maybe, but there are things where iDRAC needs rebooting to complete a task 16:00:24 oh, so it reboots itself, not via ironic? 16:00:37 then it's probably up to the driver to restore the boot configuration 16:00:40 dtantsur: Via ironic. 16:00:48 Merged openstack/sushy master: Update version of hacking https://review.opendev.org/742712 16:00:55 okay, then it's probably an issue in reboot_requested 16:00:59 dtantsur: The idrac driver generates a reboot. 16:01:00 that would be a work around, but that means that every step that needs rebooting need to restore it 16:01:17 rpioso: so, not via reboot_requested? 16:01:30 Not sure off hand. 16:01:42 then you're probably still in charge for restoring the boot configuration 16:02:02 We found our bios apply_configuration deploy step doesn't work. 16:02:05 Ironic has no way to understand that we've requested "one-time boot", but "not THIS one time" :) 16:02:41 need to add some more logic if the step is configured to execute after deployment, then don't need to restore anything 16:02:53 Well, we're past time and it seems like this discussion is going to run for a little while 16:03:08 so maybe it's more configurable if can control when boot device is configured 16:03:22 ajya: I was kind of thinking some sort of "conditional awareness to invoke the step or not" but then again the step could be self aware and in the chain 16:03:39 If the effective action is a noop, I suspect that is fine on the default path 16:03:52 Keep in mind these steps are often clean steps, too. 16:03:59 Indeed 16:04:14 yes, that means adding more logic to determine if it's deploy/clean step 16:04:18 Any objection to ending the meeting? 16:04:25 no 16:04:34 +1 16:04:55 none 16:05:22 Thanks everyone! 16:05:24 #endmeeting