15:00:23 #startmeeting ironic 15:00:23 Meeting started Mon Oct 2 15:00:23 2023 UTC and is due to finish in 60 minutes. The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:24 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:24 The meeting name has been set to 'ironic' 15:00:32 Welcome to the weekly Ironic meeting. 15:00:32 o/ 15:00:39 o/ 15:00:40 A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct. 15:00:47 #topic Announcements/Reminder 15:00:51 o/ 15:01:11 #info As always; please add hashtag:ironic-week-prio to patches ready for review; and prioritize your reviews on such patches. 15:01:31 #topic Review Action Items 15:02:19 I had an action to cut another release of NGS; I have not yet as it never made it onto my todo list; my bad. 15:02:24 I'll get that done as soon as we're done 15:02:47 #action JayF Carryover: release another NGS 15:03:00 #topic Bobcat release: October 4th, 2023 15:03:13 Other than that refreshed action; is there anything else we need to do/consider/etc? 15:03:55 Thank you all for another awesome Ironic release! \o/ I'll be on OpenInfra Live on Thursday talking about it; if you want me to highlight something specific please say so. 15:04:22 \o/ 15:04:28 Not that I’m aware of, likely a reminder wrt the PTG and etherpad 15:04:37 ack; good callout 15:04:54 That means I need to have a time for the BM SIG by then 15:04:57 which leads into... 15:05:02 #topic PTG Planning for PTG October 23-27, 2023 15:05:29 I'll be honest, the time I had set aside to take care of this before the meeting was eaten by an emergency Friday; but for now my plan is the following: 15:05:42 I'm working off the assumption of similar/identical availability to last PTG 15:05:49 1300-1700 UTC main window, 2200-2400 APAC friendly if needed. Any objections or additional input? 15:05:56 I will be sending this to the mailing list as well for feedback. 15:06:13 I think that is reasonable 15:06:40 My intention will be to start labelling topics in PTG etherpad with times (or 'do we need to talk about this?' in a couple of cases I suspect) 15:07:07 I'll make noise in here when I do so if someone wants to async feedback with me; the goal will be to have a schedule nailed down for time slots before Thursday, and hopefully have topics<->timeslots nailed down by EOW 15:07:32 I advise against aggressive compression, some of the topics can be split as well 15:07:50 ack 15:08:07 Also, operator feedback being good, we should be prepared for emergent items 15:08:08 maybe like, try to keep the top level slots to a larger theme 15:08:22 ++, and just be flexible 15:08:27 I think we always are 15:08:30 ++ 15:08:39 PTG schedules for Ironic are ... suggestive at best :D 15:08:48 Going to move on? 15:08:57 #topic Review Ironic CI status & update whiteboard if needed 15:09:05 We also need to schedule a 3rd party ci session during the PTG 15:09:10 That is on the list :) 15:09:17 Cool cool 15:09:20 I resume driving then 15:09:22 the list is ... huge 15:09:35 If you've (not Julia-you, anyone-listening-you) not looked at it, please do 15:09:47 I have no knowledge about CI status and the like 15:09:56 I think we're doing OK? Julia fixed some metalsmith pain around SDK/Ansible shenanigans 15:11:19 Looks like folks have similar opinions to CI as me :) outta sight outta mind 15:11:30 #info No RFEs to review, skipping agenda item 15:11:34 #topic Open Discussion 15:11:34 yeah, I'm not aware of issues 15:11:39 any open discussion items? 15:12:05 Does anything speak against increasing heartbeat interval? 15:12:50 Hmm. 15:12:58 we used to document it loudly 15:13:00 dtantsur: eh, mixed feelings. I’d teach agent to handle the too busy case first 15:13:21 oh, you're asking a different question than I thought you were? 15:13:29 heartbeat interval will not help you for performance when doing 500+ deploys 15:13:35 because agent will check in as soon as it's done with an item 15:13:42 regardless of hb interval, as an optimization 15:13:50 yeah, I know 15:13:57 but at this scale, even periodic check-ups can be an issu3e 15:14:40 This seems like a combination of edge cases? Extreme scale + fast track + simultaneous deploys? Just making sure I understand 15:14:42 ? 15:14:55 I guess extreme scale isn't really a good term, it's more about pushing so many deploys at once 15:14:56 I don't think fast track is even relevant; just huge scale 15:15:13 I think it's: huge scale *on one conductor* 15:15:21 which traditionally isn't even something we've optimized for, yeah? 15:15:23 Guys, we’re in meeting… 15:15:29 this is open discussion 15:15:31 I think this counts :) 15:15:39 Oh, didn’t realize we moved to open 15:15:42 Sorry! 15:15:52 it happens =) 15:16:23 I'm just trying to understand the problem space/edges here... if it's single conductor/single process I'm not super surprised there are edges here, and incresing perf in that case is good (and handling being on the edges of our scale) 15:18:44 dtantsur: I suggest writing this up in an LP bug, with details about the edges, and potentially if we can brainstorm specific "scaling up simultaneous deploys w/single-process ironic" maybe have a PTG session about it 15:18:55 hmm, good point 15:19:52 I know for a fact you can do 500 simul deploys with Ironic with a scaled up cluster, conductor groups, all the trimmings :D 15:20:16 doing it on a single-process will be an extremely valuable exercise in seeing how Ironic handles queuing tasks and being patient under heavy load 15:20:39 it'll be fun and will be a nice robustness exercise, similar to how the sqlite locking fixes were, except hopefully these will be 500x less painful :D 15:20:48 I'm going to close the meeting, this can just be chat in channel 15:20:49 ty all o/ 15:20:52 #endmeeting