15:01:12 #startmeeting ironic 15:01:12 Meeting started Mon Jan 23 15:01:12 2023 UTC and is due to finish in 60 minutes. The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:12 The meeting name has been set to 'ironic' 15:01:14 o/ 15:01:16 o/ 15:01:16 Who all is here for the meeting? 15:01:18 o/ 15:01:24 o/ 15:01:49 #topic Announcements/Reminder 15:01:58 o/ 15:02:06 As always; a reminder to be sure to review patches tagged ironic-week-prio and please tag your patches if they need review 15:02:37 #topic Actions from previous meetings 15:02:52 My action is being fulfilled later in this meeting; only other one was rpittau promising to contact VirtualPDU folks 15:02:56 since we have patches for 'em 15:02:58 o/ 15:03:09 JayF: I did contacted them but haven't got any answers 15:03:21 you should be in CC btw 15:03:38 * rpittau is in 2 meetings at the same time 15:04:20 rpittau: might wanna check to make sure you have a good email for me then; I can't find that email CC and don't remember seeing it 15:04:29 either way; moving on 15:04:38 JayF: ack, I'll check later 15:04:42 #topic Review Ironic CI status & Update whiteboard if needed 15:04:53 I think we have a decent handle on CI for the first time in a while, yeah? 15:05:09 I know TheJulia was working on some kind of flakey unit test, that's landed which should help too 15:06:32 no other input there so I'm moving on 15:06:38 #topic 2023.1 Work in Progress 15:06:47 I also wanted us to look today not only at the status of items in progress 15:06:59 but also look at the full list of things we were considering for Antelope 15:07:05 possibly as an input into the PTG for bobcat 15:07:19 #link https://specs.openstack.org/openstack/ironic-specs/priorities/2023-1-workitems.html 15:07:20 * iurygregory is late o/ 15:07:27 #link https://etherpad.opendev.org/p/IronicWorkstreams2023.1 15:07:41 #link https://etherpad.opendev.org/p/ironic-bobcat-ptg 15:08:12 We had 7 items in the original list; only about three have ever been represented in the progress etherpad 15:08:27 and even if that was outta whack; I haven't seen much patch-movement on the other 4 either 15:08:32 are we going to get to those? 15:09:44 o/ 15:09:59 o/ sick today 15:10:38 I think Steve’s stuff already merged, I’ve not heard nor seen anything on active steps. 15:10:42 kubajj: do you think you could keep our inspector merging project updates in https://etherpad.opendev.org/p/IronicWorkstreams2023.1 ? 15:10:50 TheJulia: the conductor and scaling locking stuff? 15:10:53 Md5 is at the low end of my priority list 15:11:04 neither active steps nor RAID clean-up is moving to my best knowledge 15:11:08 JayF: yeah, I can check when I’m feeling human 15:11:16 ack; I didn't realize that 15:11:38 In the general topic of things-for-this-cycle-which-haven't-happened, I also owe some research on a bugtracker flip over 15:11:47 * TheJulia pulled a back muscle and now has a head cold 15:11:54 I'll take an action to write something up and have it for next meeting so we can do the flipover before B 15:12:09 #action JayF to write up a migration plan to launchpad to present next meeting 15:12:11 dtantsur: I guess so. Is there some template for it? 15:12:28 kubajj: I'll populate the today's version, you can keep it similar 15:12:44 I'll also migrate the items that are unstarted to the bobcat etherpad for re-consideration 15:13:00 it's tough for me to imagine them getting started this late (although it'd be awesome if they did :D) 15:13:18 going to move on now if there's no other comments on workstreams 15:13:36 Often stuff does get started late and just doesn’t merge in time to release, fwiw. It was a driving factor behind releasing more often. 15:14:15 I don't have any investment in getting these in "B" release in particular; I just am trying to know what's getting done and what's not :D 15:14:22 s/B/A/ 15:14:52 Speaking of releases... 15:14:56 #topic Future of Bugfix Releases 15:15:07 I think we have as much of a quorum as we'll ever get for this discussion 15:15:18 We (I?) have not cut any bugfix releases this cycle. 15:15:28 AFAICT there hasn't been much call for them so far. 15:15:40 I'd like to 1) Amend the policy to specify we only cut a release when there's an interested party 15:16:02 and 2) See if we should cut a bugfix release now-ish (if not nowish, it's awful close to A release) 15:16:43 and 3) Try to establish a cadence on the far side for support of bugfix branches (so they aren't ad-hoc retired like I did with several of them last week) 15:17:01 JayF: I was planning a bugfix cut this week or early next week 15:17:30 lets make sure to cut zed with a minor version bump (so there will be room lol) before we do that rpittau 15:17:48 we landed a backport which was a bit of a stretch for iRMC and we wanted to get a minor version bump in zed before it was too late to account for that 15:17:50 JayF: right, I'll make sure of that 15:18:00 I can try to make sure that happens today if you want; I think we're back in a good spot to release 15:18:09 that would be great 15:18:34 thanks 15:18:50 whatever we decide, we need a written policy that we should try to follow 15:19:02 what we have now says "a release and a branch roughly every 2 months" 15:19:07 we're not doing it any more apparently :) 15:19:30 I didn't proactively cut a release that I was fairly sure there were no customers for and none of the release liasons chose to either 15:19:53 I'd say it wasn't an explicit decision not to do it so much as I thought it more important to cleanup the branches we had left behind first 15:20:30 I'm not blaming you, only saying that whatever we decide to actually do should be documented :) 15:20:36 I agree our docs should match our reality; but I think maintaining releases that have little/no users is not as good of a path as updating the docs :D 15:21:02 I'll push an update to that spec, that reflects my statement in #1 (we'll only cut a release if there's a downstream sponsor/consumer who requests it) 15:21:02 I assume we don't observe much usage of bugfix branches outside of OCP? 15:21:04 do we know how many users are using the bugfix releases? if any? 15:21:12 I checked the pypi stats for it once 15:21:16 it was a couple thousand 15:21:28 was hard to tell if it was "mirroring noise" vs actual users 15:21:32 JayF: updating spec is good, but I'd rather see something in the officail "releasing" docs 15:21:46 dtantsur: our releasing docs are the openstack ones + the spec that modifies it for ironic 15:21:53 dtantsur: if you're talking about a third doc, I may not know it exists :) 15:22:13 I mean https://docs.openstack.org/ironic/latest/contributor/releasing.html 15:22:45 going to leave that open and make sure it gets updated too 15:23:00 #action JayF to propose updates to release policy to make bugfix releases explicitly optional 15:23:25 So we've covered the first two items; I'll update the policy, rpittau is cutting a release soon 15:23:31 so the idea is to create the bugfix branches and do any backports to them, but NOT release unless there is a request? 15:24:08 I'm going to propose a doc update that says we evaluate whether or not to cut a bugfix branch at the 2nd and 4th month of the cycle 15:24:14 if there are sufficient changes and interested users; we cut one 15:24:16 otherwise we do not 15:24:31 and for the most part, we maintain bugfix branches to a similar standard of quality to stable branches 15:25:08 ok. so 1. we might not cut a bugfix branch. IF we cut a bugfix branch, will we always do one or more releases off of that bugfix branch? 15:25:14 yep 15:25:29 The other item I'd like to pull on tin this discussion -> ironic bugfix/18.1 / IPA bugfix/8.1 / inspector bugfix/10.7 are all some of the oldest branches in our CI at this point 15:25:38 Is downstream still consuming these? 15:25:50 If yes, we should keep em up, if not, maybe we should retire them out 15:26:01 JayF: unfortunately downstream we're still consuming them 15:26:15 It's fortunate that we aren't maintining the branch for nobody :D 15:26:21 lol 15:26:30 as long as it's in use, that's what my concern is 15:26:46 (I can't recall why we'd want the same bugfix branch to have more than one release if eg we have branches bugfix 10.1 & 10.2. why do we still want 10.1 releases after 10.2 is released? 15:26:57 JayF: I can tell you that we're going to consume those for at least other 5 months 15:27:24 rpittau: it'd be neat if we could capture that info somewhere, e.g. change https://etherpad.opendev.org/p/IronicBugfixBranchCleanup into a tracking etherpad for it 15:27:28 does "consuming" imply that they need to keep being updated? 15:27:31 because I know that info is around and I wish it was written down 15:27:38 JayF: I'll take care of that 15:27:43 rloo: correct 15:27:46 rloo: so basically; the primary consumer for these bugfix releases are downstream RH releases 15:28:11 rloo: so as long as RH is consuming them, we'll keep backporting to them; and if/when it makes sense (and there's version-number-room) we might occassionally cut a release 15:28:36 so RH is consuming what is on the branch, not the actual releases from those branches? 15:28:48 rloo: historically, my bigger concern with these is that we had ~18 total branches that were not being consumed and were configured in CI still 15:28:51 rloo: yes 15:29:15 rloo: so we only cut releases when there's something egregiously broken enough that we don't wanna risk even a single user getting it :) 15:29:24 that doesn't make sense (sorry). too much overhead etc. there's got to be a simpler way. 15:29:46 right now a nontrivial % of that overhead is that releasing tools don't work on those branches 15:29:53 so it's manual work to cut one, to release one, and to retire one 15:30:07 (yeah, like the two PRs i backported to two bugfix branches, which i'll bring up later) 15:30:43 So are we to the end of bugfix branch chat? If so we can move on, otherwise happy to entertain more questions/discussion 15:30:59 summarize please 15:31:15 there were 2 issues you brought up JayF 15:31:28 1) I will put in a PR for bugfix branch policy updates; it'll be easier to review the actual text instead of talking about it in chat. 15:31:39 2) rpittau is cutting a bugfix release late this week or early next 15:31:52 3) The oldest of our bugfix branches need to stay maintained for ~5 more months 15:32:02 'a bugfix release' --> releases for all the bugfix branches we have open? 15:32:16 no, meaning we cut a release from master 15:32:19 bugfix/[version] 15:32:40 I don't know what version that would be, but we're talking a new branch/release line that'd be supported 15:33:12 ah, ok, so this would be the last bugfix branch/release we'll cut 'as per policy', before you propose that we only do ti if someone asks for it. 15:33:15 And that gives us a point to be able to release from again off that branch or someplace to put patches for intermediate releases, which the need has come up in the past 15:33:48 i get for intermediate releases, but i don't get why we still keep them around AFTER the major release goes out 15:33:51 it's going to be bugfix/21.2 15:33:54 for iornic 15:33:56 ironic* 15:34:50 rloo: the easiest mental model for them is "extra stable releases" 15:35:19 selective extra stable releases :-( 15:35:21 Because some folks don’t or are not able to jump to the next stable release, but can pull in a minor update. 15:35:51 since RH is using it, i'm sure there is a valid need! 15:36:44 ack; I'm going to move on njjow 15:36:56 There are no RFEs to review; skipping topic. 15:37:00 #topic Open Discussions 15:37:12 vanou had an item on the agenda about Vulnerability Management 15:37:22 vanou: I see you wrote quite a bit; are you are of the OpenStack VMT policy? 15:37:36 https://security.openstack.org/vmt-process.html 15:38:00 Baptiste Jonglez proposed openstack/networking-generic-switch master: Add ngs_ssh_disabled_algorithms setting https://review.opendev.org/c/openstack/networking-generic-switch/+/868316 15:38:02 No. When I consult it on storyboard, community member says Ironic doesn't follow VMT 15:38:26 We are not a VMT managed project. 15:38:50 We do consult with them though. 15:38:53 It's a bit weird, we're following the processes but we're not tracked by the team (for some reason that probably no longer holds today) 15:39:10 Yeah; I was about to say; I follow VMT policy as written usually for security things w/Ironic 15:39:18 even if we aren't listed as following their policy 15:39:25 should I pull on that string and see why and "fix" it ? 15:39:37 The answer could be "our team is small" 15:39:47 but we should at least agree on an escalation path 15:40:05 so that the team does not say "we don't know about Ironic", but rather "Ironic is handled by a separate team and your contact persons are $this" 15:40:19 I believe that is mostly what happens in practice, alreayd 15:40:35 well, apparently we give impression that Ironic does not have a security process 15:40:45 vanou: is this providing clarity? 15:40:46 Yeah; which I appreciate vanou bringing to our attention 15:41:01 I think the issue is vanou didn’t find an explicit policy in our docs 15:41:05 is the question: why aren't we, and should we/ironic be included in that vulnerability management process? 15:41:22 I think the answer to that is "I don't know" and "Yes" from my perpsective rloo :D 15:41:42 fungi: If you're around; you happen to know the historical reason why Ironic isn't security-managed by VMT? 15:41:56 if no one disagrees wrt ironic being part of that process, then i think someone could volunteer to see what is needed to get ironic added? :) 15:42:10 ingesting context, please wait 15:42:12 It predates my time, goes back to the time of jroll or Aeva 15:42:35 (maybe it was when only the core openstack services were included) 15:43:02 rloo: possible 15:43:35 TheJulia: yes. but I think it's better to have vulnerability handlig policy. Especially like in case of Fujitsu vulnerability, there are 2 domain of code responsibility. 15:43:54 JayF: i don't know if it was simply because nobody did it, or because of some other reason, bit we have a process for inclusion here: 15:44:06 #link https://security.openstack.org/repos-overseen.html 15:44:32 Ironic would add quite a few to this list 15:44:59 well, maybe. you'd need to make sure each repository met the criteria 15:45:12 That is a point, we would need review the vmt polcity because a library we did not control needed to be fixed and an option had to be added to the method call with ironic 15:45:33 We, speaking in terms of ironic the project 15:45:42 Reading those VMT requirements; we should wait until we migrate back to launchpad. 15:45:51 i'm happy to be counted in "we" for purposes of helping check things 15:46:14 just let me know if you need assistance 15:46:17 But in the meantime; I'd be extremely +2 to a change to Ironic docs stating that we prefer OpenStack's VMT policy to be followed for Ironic items when it can make sense 15:46:33 +2 as well 15:46:43 in cases like vanou mentions; I think we're better off being a big tent 15:47:06 I'm also in favor 15:47:10 if a library that we primarily use is vulnerable, impacting ironic, it only makes sense to treat it like an ironic vuln if the library maintainers are on board 15:47:16 yup, agreed. (I think we've been handling security issues already but good to make it explicit/consistent) 15:47:29 Does someone wanna take that doc update action? 15:47:39 I think I'm already like 3 action items deep :D 15:47:49 o/ 15:48:14 #action vanou to update Ironic docs to indicate we generally follow OpenStack policies around security and disclosure 15:48:27 I want to update. When I get stuck I'll ask you help 15:48:30 Thanks vanou! 15:48:35 please do; we're all here to help 15:48:36 Thanks too all! 15:48:37 thanks! 15:48:40 Anything else for Open Discussion? 15:48:45 so quickly 15:48:54 i am moving on to non-openstack stuff 15:49:12 this is probably the last meeting for me. i'll post something later. maybe. 15:49:28 Oh 15:49:32 Thank you for all the years and years of stacking opens rloo 15:49:39 rloo: :( 15:49:53 Yeah. Thanks rloo 15:49:57 i have 2 PRs open still. both are for bugfix branches, 20.2 & 19.0. CI fails for them. I'd normally just abandon them, but the backport merged to an 18.x branch. wanted to know what you think i ought to do 15:50:04 thanks for everything rloo 15:50:07 rloo: it has been great working with you! Thank you for all your contributions! 15:50:20 rloo: you wanna #link those here? those two changes? 15:50:21 rloo: I'll check them, probably issues on CI 15:50:29 i'm sorry to be leaving the community but it is time I think. I've met a lot of wonderful people -- am so happy you are still working on ironic! 15:50:40 and we need those for that https://review.opendev.org/q/topic:pin-tox-bugfix-ironic 15:50:43 https://review.opendev.org/c/openstack/ironic/+/868026 & https://review.opendev.org/c/openstack/ironic/+/868027 15:50:46 rloo: you'll be missed! you've been a fixture of the community for as long as i can remember 15:50:58 rloo: if you're going to be gone-gone, would you like your core access migrated to "core emeritus"; so we don't have to worry about your account being compromised? 15:51:08 heh, fungi -- i hope you will be there forever! 15:51:13 (that's just a nice way of saying pulling your access but we'll give it back if you come back :D ) 15:51:19 JayF: yes please. 15:51:39 #action JayF to remove rloo from core list as she is not actively working on OpenStack anymore :( 15:51:40 rloo: oh :( this community won't be the same without you. I hope we cross the roads again 15:52:10 dtantsur: sigh. it has been awesome. so glad to have met many of you in person. many great memories!!! 15:52:16 indeed! 15:52:37 rloo: Thanks for all the work and the contributions! (I think in all these past years, we actually never met in person :)) 15:52:45 will it make TheJulia or myself the oldest Ironicer still here? :) 15:52:55 oh arne_wiebalck. yes, so sorry we never met! 15:53:07 dtantsur: longest standing 15:53:07 will make dtantsur the oldest... 15:53:29 well, the invitation for a CERN tour stands, even for ex-Ironicers, so whenever you are in the area ... :) 15:53:34 or would it be jay... he was 'out' for a bit so not sure. 15:53:51 I'm pretty sure I'd say it was dtantsur 15:53:53 thx arne_wiebalck! 15:53:54 :D 15:54:13 (has been so long i can't remember if IPA or dtansur was first) 15:54:36 I definitely was reviewing the IPA addition, which makes me think I was already a core :D 15:54:43 at least when agent_ipmitool was proposed 15:54:59 oh, memories :) 15:55:20 * dtantsur remembers "zapping" and sheds a tear 15:55:21 Heh, the giant horrible patch of doom which had like 100+ patchsets before we gave up and broke it down into pieces lol 15:55:28 yeah! 15:55:30 dtantsur: I'm still sour we dropped that naming 15:55:46 there must be some ex-openstack/alumni community ;) 15:55:55 I think it's here lol 15:55:58 if you want to count the discussions at the bar where lifeless and devananda were debating the original design as early... 15:56:15 sorry, aeva 15:56:32 Memories are memories :) 15:56:32 ha ha. before my time. long live ironic (and openstack!) 15:56:35 I remember the mid-cycle at Yahoo sunnyvale (nudge nudge rloo) where we were like "how about an agent" 15:56:46 and Aeva looked at us like we were a child asking for a trip to disney 15:56:51 until we showed them the working agent lol 15:56:59 #endmeeting