15:00:23 <JayF> #startmeeting ironic
15:00:23 <opendevmeet> Meeting started Mon Oct  2 15:00:23 2023 UTC and is due to finish in 60 minutes.  The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:24 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:24 <opendevmeet> The meeting name has been set to 'ironic'
15:00:32 <JayF> Welcome to the weekly Ironic meeting.
15:00:32 <iurygregory> o/
15:00:39 <dtantsur> o/
15:00:40 <JayF> A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.
15:00:47 <JayF> #topic Announcements/Reminder
15:00:51 <TheJulia> o/
15:01:11 <JayF> #info As always; please add hashtag:ironic-week-prio to patches ready for review; and prioritize your reviews on such patches.
15:01:31 <JayF> #topic Review Action Items
15:02:19 <JayF> I had an action to cut another release of NGS; I have not yet as it never made it onto my todo list; my bad.
15:02:24 <JayF> I'll get that done as soon as we're done
15:02:47 <JayF> #action JayF Carryover: release another NGS
15:03:00 <JayF> #topic Bobcat release: October 4th, 2023
15:03:13 <JayF> Other than that refreshed action; is there anything else we need to do/consider/etc?
15:03:55 <JayF> Thank you all for another awesome Ironic release! \o/ I'll be on OpenInfra Live on Thursday talking about it; if you want me to highlight something specific please say so.
15:04:22 <iurygregory> \o/
15:04:28 <TheJulia> Not that I’m aware of, likely a reminder wrt the PTG and etherpad
15:04:37 <JayF> ack; good callout
15:04:54 <JayF> That means I need to have a time for the BM SIG by then
15:04:57 <JayF> which leads into...
15:05:02 <JayF> #topic PTG Planning for PTG October 23-27, 2023
15:05:29 <JayF> I'll be honest, the time I had set aside to take care of this before the meeting was eaten by an emergency Friday; but for now my plan is the following:
15:05:42 <JayF> I'm working off the assumption of similar/identical availability to last PTG
15:05:49 <JayF> 1300-1700 UTC main window, 2200-2400 APAC friendly if needed. Any objections or additional input?
15:05:56 <JayF> I will be sending this to the mailing list as well for feedback.
15:06:13 <TheJulia> I think that is reasonable
15:06:40 <JayF> My intention will be to start labelling topics in PTG etherpad with times (or 'do we need to talk about this?' in a couple of cases I suspect)
15:07:07 <JayF> I'll make noise in here when I do so if someone wants to async feedback with me; the goal will be to have a schedule nailed down for time slots before Thursday, and hopefully have topics<->timeslots nailed down by EOW
15:07:32 <TheJulia> I advise against aggressive compression, some of the topics can be split as well
15:07:50 <JayF> ack
15:08:07 <TheJulia> Also, operator feedback being good, we should be prepared for emergent items
15:08:08 <JayF> maybe like, try to keep the top level slots to a larger theme
15:08:22 <TheJulia> ++, and just be flexible
15:08:27 <JayF> I think we always are
15:08:30 <dtantsur> ++
15:08:39 <JayF> PTG schedules for Ironic are ... suggestive at best :D
15:08:48 <JayF> Going to move on?
15:08:57 <JayF> #topic Review Ironic CI status & update whiteboard if needed
15:09:05 <TheJulia> We also need to schedule a 3rd party ci session during the PTG
15:09:10 <JayF> That is on the list :)
15:09:17 <TheJulia> Cool cool
15:09:20 <TheJulia> I resume driving then
15:09:22 <JayF> the list is ... huge
15:09:35 <JayF> If you've (not Julia-you, anyone-listening-you) not looked at it, please do
15:09:47 <JayF> I have no knowledge about CI status and the like
15:09:56 <JayF> I think we're doing OK? Julia fixed some metalsmith pain around SDK/Ansible shenanigans
15:11:19 <JayF> Looks like folks have similar opinions to CI as me :) outta sight outta mind
15:11:30 <JayF> #info No RFEs to review, skipping agenda item
15:11:34 <JayF> #topic Open Discussion
15:11:34 <dtantsur> yeah, I'm not aware of issues
15:11:39 <JayF> any open discussion items?
15:12:05 <dtantsur> Does anything speak against increasing heartbeat interval?
15:12:50 <JayF> Hmm.
15:12:58 <JayF> we used to document it loudly
15:13:00 <TheJulia> dtantsur: eh, mixed feelings. I’d teach agent to handle the too busy case first
15:13:21 <JayF> oh, you're asking a different question than I thought you were?
15:13:29 <JayF> heartbeat interval will not help you for performance when doing 500+ deploys
15:13:35 <JayF> because agent will check in as soon as it's done with an item
15:13:42 <JayF> regardless of hb interval, as an optimization
15:13:50 <dtantsur> yeah, I know
15:13:57 <dtantsur> but at this scale, even periodic check-ups can be an issu3e
15:14:40 <JayF> This seems like a combination of edge cases? Extreme scale + fast track + simultaneous deploys? Just making sure I understand
15:14:42 <JayF> ?
15:14:55 <JayF> I guess extreme scale isn't really a good term, it's more about pushing so many deploys at once
15:14:56 <dtantsur> I don't think fast track is even relevant; just huge scale
15:15:13 <JayF> I think it's: huge scale *on one conductor*
15:15:21 <JayF> which traditionally isn't even something we've optimized for, yeah?
15:15:23 <TheJulia> Guys, we’re in meeting…
15:15:29 <JayF> this is open discussion
15:15:31 <JayF> I think this counts :)
15:15:39 <TheJulia> Oh, didn’t realize we moved to open
15:15:42 <TheJulia> Sorry!
15:15:52 <iurygregory> it happens =)
15:16:23 <JayF> I'm just trying to understand the problem space/edges here... if it's single conductor/single process I'm not super surprised there are edges here, and incresing perf in that case is good (and handling being on the edges of our scale)
15:18:44 <JayF> dtantsur: I suggest writing this up in an LP bug, with details about the edges, and potentially if we can brainstorm specific "scaling up simultaneous deploys w/single-process ironic" maybe have a PTG session about it
15:18:55 <dtantsur> hmm, good point
15:19:52 <JayF> I know for a fact you can do 500 simul deploys with Ironic with a scaled up cluster, conductor groups, all the trimmings :D
15:20:16 <JayF> doing it on a single-process will be an extremely valuable exercise in seeing how Ironic handles queuing tasks and being patient under heavy load
15:20:39 <JayF> it'll be fun and will be a nice robustness exercise, similar to how the sqlite locking fixes were, except hopefully these will be 500x less painful :D
15:20:48 <JayF> I'm going to close the meeting, this can just be chat in channel
15:20:49 <JayF> ty all o/
15:20:52 <JayF> #endmeeting