15:00:22 <JayF> #startmeeting ironic
15:00:22 <opendevmeet> Meeting started Mon Jul  3 15:00:22 2023 UTC and is due to finish in 60 minutes.  The chair is JayF. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:22 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:22 <opendevmeet> The meeting name has been set to 'ironic'
15:00:39 <dtantsur> o/
15:00:40 <JayF> I would anticipate an ill-attended, short meeting as tomorrow is a US federal holiday and today is a popular day to take off :D
15:00:40 <TheJulia> o/
15:00:54 <TheJulia> I was going to take today off
15:00:57 <JayF> #topic Announcements/Reminder
15:01:16 <JayF> #note Standing reminder: review patches tagged #ironic-week-prio, and tag your patches for priority review
15:01:25 <JayF> #topic Review previous action items
15:01:28 <iurygregory> o/
15:01:37 <dtantsur> I may add the next inspection patch to ironic-week-prio once I test it
15:01:53 <JayF> Just a reminder that rpittau has an action to moderate our meeting the next two weeks. Next week (starting a week from today) I will be out of office and out of country for a week, so don't expect me around :D
15:02:07 <JayF> #topic Review Ironic CI Status
15:02:08 <rpittau> o/
15:02:10 <JayF> Where are we?
15:02:20 <rpittau> that's very phylosophical
15:02:33 <dtantsur> Well, my patch has seen the first green for a long time. So not too bad at least?
15:02:40 <rpittau> JayF: I think we fixed/workarounded most of the issues
15:03:04 <JayF> Applying extremely cautious optimism lol
15:03:14 <JayF> Are there any outstanding CI related patches we need to review or land?
15:03:47 <rpittau> mmm not on ironic AFAICS
15:03:48 <TheJulia> There is one for sqlalchemy 2.0, to fix migrations
15:03:53 <rpittau> ah yeah, that one
15:04:01 <TheJulia> and there is the mysql split out patch
15:04:05 <iurygregory> yeah
15:04:16 <TheJulia> I'd split it and possibly merge additional troubleshooting
15:04:16 <JayF> Both of those sound good to me, you wanna link them or just we'll find and land them afterwards in any event
15:04:38 <iurygregory> some of them we need in stable / bugfix branches
15:04:47 <rpittau> I rechecked the sqla one
15:05:02 <TheJulia> ++
15:05:10 <JayF> ++ lets backport whatever is needed to stable branches; but I don't feel like that should be a rush until/unless we have patches to backport there tbh
15:05:18 <JayF> I trust us all to sanely prioritize
15:05:29 <JayF> Aight, going to move on
15:05:33 <JayF> #topic 2023.2 Workstream
15:05:40 <JayF> #link https://etherpad.opendev.org/p/IronicWorkstreams2023.2
15:05:59 <JayF> I will note that many things seem to be pending review; I'll be taking time to review today
15:06:59 <JayF> Thanks for updating that dtantsur
15:07:19 <dtantsur> btw, have we had a chance to say welcome (back) to masghar?
15:07:40 <dtantsur> Mahnoor is helping me with the inspector merger work and will take over more tasks as we go
15:07:55 <iurygregory> I don't think we did
15:08:00 <JayF> masghar: welcome (back?) I'm not sure we've ever met but any friend of ironic+dtantsur gets adopted by me :D
15:08:09 <iurygregory> welcome masghar =)
15:08:20 <dtantsur> Mahnoor participated in outreachy, I think TheJulia was her mentor
15:08:25 <JayF> oh, wonderful!
15:08:32 <JayF> heck yeah
15:08:41 <JayF> There is another former member of our community coming back, for at least a small stint
15:08:52 <JayF> but I'll let them make the announcement to that larger group when time comes
15:09:11 <JayF> Moving on
15:09:18 <JayF> #topic Open Discussion
15:09:48 <JayF> I had a note here on PTL availability, I will not be here next week as noted before and am planning to miss the next two-ish meetings due to travel.
15:09:54 <JayF> Anything else for open discussion?
15:10:07 <iurygregory> I probably have something (sorry didn't add to the agenda)
15:10:19 <JayF> it's open discsusion :D
15:10:23 <iurygregory> :D
15:10:42 <TheJulia> \o/
15:10:58 <iurygregory> ok, some of you probably remember a problem we had related to multipah and we added a lot of logic on IPA to be able to handle things
15:11:27 <opendevreview> Merged openstack/ironic master: Use jammy for base jobs  https://review.opendev.org/c/openstack/ironic/+/869052
15:11:31 <rpittau> \o/
15:11:37 * dtantsur hears multipath and runs away screaming
15:11:54 <rpittau> I actually tried hard to forget about that mpath stuff
15:12:20 <dtantsur> no amount of alcohol can wash this out of memory
15:12:22 <iurygregory> we have an interesting bug downstream, where inspection is timing out (takes more than 30min), because the machine has a loooooot of disks and we check all of them I think
15:12:22 <rpittau> but I guess we'll hear more about it :/
15:12:40 <iurygregory> +80 disks if I recall
15:12:45 <dtantsur> iurygregory: define "check" please. or is it unclear yet?
15:13:19 * iurygregory looks for the tab with the information
15:13:21 <zorun> is the issue about multipath in Linux + some NVMe disks?
15:13:25 <zorun> (hi there)
15:13:29 <TheJulia> who said mpath?!/
15:13:31 * TheJulia hides
15:13:57 * TheJulia builds a bunker
15:14:19 <JayF> We should proabbly ensure the behavior is documented in a launchpad bug
15:14:21 <JayF> then go from there?
15:14:44 <JayF> sounds like one in a long line of "ridiculously large hardware causes edge case" bugs we've been squashing for a decade :D
15:14:56 <TheJulia> iurygregory: your really going to need to be specific on what is being encountered
15:15:08 <TheJulia> because what JayF said :)
15:16:14 <JayF> Is there any further specifics on this or something else for open discussion?
15:16:25 <iurygregory> is not an error, it's a timeout issue because ipa doesn't report all info in 30 min, because they have a lot of disks and we will do all the checks from _get_multipath_parent_device etc
15:16:36 <iurygregory> so we take a lot of time and fails
15:16:42 <JayF> yeah, in that case I'd probably adjust timeouts to reflect the reality of that environment
15:16:55 <JayF> but we should likely have a way to turn off some of that if it's taking forever, too
15:17:19 <iurygregory> so I'm wondering if we have some ideas on how to avoid this taking a lot of time
15:17:22 <JayF> I suspect it's probably reproducable in a unit test; most of what takes a long time is probably in the python parsing, yeah?
15:17:39 <iurygregory> instead of just increasing timeout
15:18:29 <iurygregory> maybe the logic we adde to not clean some devices can be used? like "I don't want IPA to check things on /dev/sda, /dev/sdb...."
15:18:49 <JayF> I suspect there may be a more straightforward fix
15:19:00 <JayF> but until there's a bug with details we can look at alongside the code we're just guessing :D
15:19:00 * TheJulia is on a call so trying to digest
15:19:01 <iurygregory> (currently I don't think we are using the feature to skip downstream...)
15:19:04 <rpittau> you really need to know in advance very well ytour disks, that is not always trivial
15:19:42 <iurygregory> things are not trivial when they have like +80 disks :D
15:19:49 <JayF> lets continue this talk outside of the logged meeting?
15:19:53 <JayF> #endmeeting