15:00:00 <iurygregory> #startmeeting ironic
15:00:00 <opendevmeet> Meeting started Mon Aug 30 15:00:00 2021 UTC and is due to finish in 60 minutes.  The chair is iurygregory. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:00 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:00 <opendevmeet> The meeting name has been set to 'ironic'
15:00:05 <iurygregory> o/
15:00:08 <erbarr> o/
15:00:18 <iurygregory> Hello everyone, welcome to our weekly meeting!
15:00:18 <arne_wiebalck> o/
15:00:23 <ajya> o/
15:00:28 <dtantsur> o/
15:00:41 <rloo> o/
15:00:42 <iurygregory> You can find our agenda in the wiki
15:00:44 <iurygregory> #link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting
15:00:55 <rpioso> \o
15:00:56 <TheJulia> o/
15:01:04 <iurygregory> #topic Announcements / Reminder
15:01:20 <iurygregory> #info We have topics for the Review Jam tomorrow =)
15:01:52 <iurygregory> Anyone has something to announce or remind us?
15:02:13 <TheJulia> Iury will be our glorius PTL for Yoga!
15:02:36 <dtantsur> \o/
15:02:38 <iurygregory> \o/
15:02:43 <rloo> yay, many thanks iurygregory!!!
15:03:11 <iurygregory> np, I will do my best \o/
15:03:38 <TheJulia> :)
15:03:46 <iurygregory> #topic Review action items from previous meeting
15:04:01 <iurygregory> since we don't have any action items from the previous meeting, moving on
15:04:21 <iurygregory> #topic Review subteam status reports
15:04:28 <iurygregory> #link https://etherpad.opendev.org/p/IronicWhiteBoard
15:04:34 <iurygregory> starting on L60
15:06:34 <TheJulia> Looks like everyone is getting pulled downstream pretty hardcore
15:07:00 <iurygregory> yup, it happens from time to time =(
15:07:00 <dtantsur> hey, I've produced a ton of bifrost and sushy-tools patches :D
15:07:30 <iurygregory> I think dtantsur will add a lot patches  for the Deciding on priorities for the coming week :D
15:07:47 <dtantsur> with great pleasure!
15:08:26 <TheJulia> Sounds most excellent
15:09:11 <iurygregory> oh nice, we have a section for the Keylime work in the subteam status :D
15:10:30 <iurygregory> dtantsur, TheJulia Driver structure and model doesn't have any updates for like 1month, can I just add "no-updates"?
15:10:59 <TheJulia> Yeah, I've been pulled downstream so much I haven't really had time to think about any of it
15:11:01 <dtantsur> yes please
15:11:09 <iurygregory> ack =)
15:11:11 <dtantsur> I'll likely spend the rest of the cycle redfishing
15:11:28 <TheJulia> dtantsur: is that like going fishing on a boat?!
15:11:50 <dtantsur> I would love to try :)
15:12:04 <dtantsur> but my curse is figuring out TLS apparently
15:12:09 <iurygregory> I think dtantsur will become a chef :D
15:12:26 <dtantsur> Principal Fugu Engineer
15:12:34 <TheJulia> so next time we're all in Germany, dtantsur can cook us fish :)
15:12:38 <iurygregory> Fugu LOL
15:12:53 <TheJulia> lol
15:13:05 <dtantsur> TheJulia: we have very nice sushi fish here thanks to a few authentic japanese shops
15:13:16 <TheJulia> One day :)
15:13:23 <dtantsur> sigh
15:13:29 <TheJulia> sigh++
15:13:49 <iurygregory> almost dinner time and we are talking about food \o/
15:13:56 <iurygregory> sounds about right :D
15:14:16 <iurygregory> I think we are good on the subteam status, moving on
15:14:23 <iurygregory> #topic Deciding on priorities for the coming week
15:14:54 <ajya> can these be added - https://review.opendev.org/c/openstack/ironic/+/804112 https://review.opendev.org/c/openstack/ironic/+/804848 ?
15:15:38 <TheJulia> ajya: is there a story for background on those?
15:15:39 <iurygregory> makes sense to me
15:16:08 <iurygregory> yeah having a story RFE would be good =)
15:16:09 <dtantsur> can I just add everything non-WIP from https://review.opendev.org/q/owner:dtantsur%2540protonmail.com+status:open+(project:openstack/bifrost+OR+project:openstack/sushy-tools) ?
15:16:10 <ajya> TheJulia: no, it's to get it closer to idrac-wsman behavior
15:16:17 <dtantsur> I don't want to paste every patch here
15:16:27 <TheJulia> iurygregory: I'm not sure it is an rfe, but can't be sure without some more words somplace
15:16:52 <TheJulia> dtantsur: sounds reasonable
15:17:10 <iurygregory> TheJulia, well they are with feature in the releasenotes thats why I said rfe hehe
15:17:20 <TheJulia> iurygregory: ahh, yeah
15:17:55 <arne_wiebalck> I have added https://review.opendev.org/c/openstack/ironic/+/801425 (this is the fast track retry on failed BMC connections, seems to prevent fast track nodes from being switched off)
15:17:58 <iurygregory> dtantsur, I think you can =)
15:18:39 <TheJulia> arne_wiebalck: random bmc unhappiness?
15:18:56 <arne_wiebalck> TheJulia: I think UDP packet loss
15:19:06 <ajya> I can create story with tasks for these patches with some background info, otherwise patches should be self-contained.
15:19:10 <TheJulia> I guess that is bound to happen on conductors with lots of nodes
15:19:20 <arne_wiebalck> TheJulia: that is what I am thinking
15:19:30 <TheJulia> ajya: sounds good, and also lets explicitly state if there is a backport intent
15:19:35 <arne_wiebalck> TheJulia: and we do retries for power sync, but not on fast track
15:19:48 <TheJulia> arne_wiebalck: makes sense
15:20:14 * arne_wiebalck should try moving to redfish to fix these instead of retrying ...
15:20:40 <TheJulia> eh, we've still seem bmc's go out for lunch for a moment
15:20:44 <TheJulia> so makes sense to retry anwyay
15:20:45 <iurygregory> ajya, I think a story would be good
15:20:54 <arne_wiebalck> TheJulia: ++
15:22:19 <iurygregory> ajya, feel free to add the hashtag in the patches you mentioned o/
15:22:26 <ajya> ok, thanks
15:23:30 <iurygregory> anyone has more patches that needs review?
15:23:51 <TheJulia> Not on my end
15:24:40 <iurygregory> I think we are good with the amount of patches we added
15:24:50 <iurygregory> moving on
15:25:01 <iurygregory> #topic Discussion
15:25:25 <iurygregory> We don't have any topics for Discussion in the agenda, but I will give a few minutes to see if anyone has something
15:25:48 <TheJulia> Could/should we just move to Open Discussion then?
15:26:07 <iurygregory> maybe, arne_wiebalck do you have something for the SIG?
15:26:13 <arne_wiebalck> yes
15:26:17 <iurygregory> #topic Baremetal SIG
15:26:21 <iurygregory> go ahead =)
15:26:42 <arne_wiebalck> we said we would do an operator feedback session next time
15:26:58 <arne_wiebalck> this is what we have on the etherpad atm
15:27:25 <arne_wiebalck> IIRC, we said we would decide nearer the time if that would make sense
15:27:39 <arne_wiebalck> given that there was also the openinfra live event
15:27:40 <TheJulia> Yeah, we kind of didn't push that forward with holidays and the chance of getting something larger organized, yet that fell through
15:27:48 <arne_wiebalck> right
15:28:21 <arne_wiebalck> while I would like to have such a discussion I am not sure operators would show up by themselves
15:28:35 <arne_wiebalck> given the attendance we had over the last months
15:28:54 <TheJulia> No getting them to appear in a community setting seems to be one of the hardest things to do
15:29:16 <arne_wiebalck> so, we could either reach out directly or skip and have a topic of the day
15:29:27 <arne_wiebalck> thoughts?
15:29:56 <iurygregory> well, maybe we can try to reach out the osopsmeetup ?
15:30:21 <iurygregory> and if we know some operators we can try to ask if they are interested?
15:30:25 <arne_wiebalck> I think they face similar issues with participation
15:30:37 <arne_wiebalck> osops I mean
15:30:38 <jssfr> what's an operator feedback session?
15:31:02 <arne_wiebalck> jssfr: Ironic users come with their feedback, pain points, ideas, ...
15:31:16 <arne_wiebalck> jssfr: Ironic users/operators/admins
15:31:20 <jssfr> oh
15:31:23 <jssfr> never heard of it :)
15:31:30 <TheJulia> iurygregory: I think it might make sense to engage them
15:31:32 <arne_wiebalck> heh, that is the problem
15:31:46 <TheJulia> iurygregory: reach out and say "hey, we're willing to listen/chat" if interested
15:31:55 <TheJulia> but I don't know if that makes a baremetal SIG session
15:32:06 <iurygregory> right
15:32:08 <TheJulia> I'd <3 soemthing to helpf or planning with the next development cycle though
15:32:16 <arne_wiebalck> TheJulia: well, we can try and offer to listen
15:32:18 <iurygregory> we can probably come up with something for the PTG?
15:32:34 <arne_wiebalck> TheJulia: if noone comes, fine, at least we gave the chance
15:32:41 <iurygregory> or even a separate event
15:32:53 <iurygregory> arne_wiebalck, ++
15:33:00 <TheJulia> yeah, true
15:33:22 * TheJulia wonders if anyone has posted questions/comments on our existing videos
15:33:32 <arne_wiebalck> ok, so how about everyone reaches out and we see what happens?
15:34:11 <TheJulia> sounds reasonable
15:34:19 <iurygregory> yeah, I agree
15:34:37 <arne_wiebalck> like, we know some operators/users :)
15:34:51 <arne_wiebalck> ok, let's try this then
15:35:02 <iurygregory> #action everyone to reach out to know operators/users
15:35:11 <arne_wiebalck> this is for tomorrow in a week
15:35:17 <arne_wiebalck> Sep 7th
15:35:44 <arne_wiebalck> I will send out a mail s well.
15:35:51 <iurygregory> arne_wiebalck, ++
15:36:02 <arne_wiebalck> that is it from me for the SIG
15:36:08 <iurygregory> tweet also :D
15:36:16 <arne_wiebalck> yes, will do
15:36:18 <iurygregory> tks arne_wiebalck !
15:36:42 <iurygregory> skipping RFE review since we don't have any
15:36:43 <iurygregory> #topic Open Discussion
15:37:15 <TheJulia> I'm suspecting we should likely start discussing/planning the PTG
15:37:16 <arne_wiebalck> I have something :)
15:37:30 <iurygregory> go ahead arne_wiebalck
15:37:45 <iurygregory> TheJulia,++
15:37:54 <arne_wiebalck> For the burn-in we are interested in the log output in more or less real time.
15:38:13 <arne_wiebalck> We were thinking about having sth like fluentd on the image.
15:38:34 <arne_wiebalck> The other option would be to ship logs to the conductor and use the conductor logging.
15:38:56 <arne_wiebalck> The conductor gets the IPA logs eventually.
15:39:27 <arne_wiebalck> Now, for fast track this is more tricky, since it will only do this at the end.
15:39:30 <TheJulia> I'm not opposed to it, but how would it work when the conductor is just a process in a container
15:40:01 <arne_wiebalck> How does this work now for the IPA logs when the conductor is in a container?
15:40:35 <arne_wiebalck> My thinking is to leverage the logging that should be in place for the conductor anyway.
15:40:46 <TheJulia> arne_wiebalck: just lives on a filesystem, likely a bind mounted filesystem
15:41:02 <arne_wiebalck> Using fluentd on the IPA pushes the issue downstream and to image building.
15:41:12 <TheJulia> indeed
15:41:21 <TheJulia> so realistically, it could be anywhere configured by an operator
15:41:41 <arne_wiebalck> Having logs flow into the conductor more regularly offers operators to use them even in fast track.
15:42:11 <TheJulia> yeah
15:43:00 <arne_wiebalck> Imagine 100s of nodes in fast track, logs sent into the conductor, and  onwards to ES, for instance.
15:43:22 <arne_wiebalck> We have a mechanism to get the logs already.
15:43:41 <arne_wiebalck> It is just run only once and collects a lot of additional logsd.
15:43:43 <arne_wiebalck> logs
15:43:51 <TheJulia> I guess it is more an agent/image thing than anything else?
15:44:56 <arne_wiebalck> Yeah, you could see it that way. But configuring the IPA to push logs into a monitoring infra may be more difficult than the controllers which may do this anyway.
15:45:29 <arne_wiebalck> I am trying to leverage the existing pipeline.
15:45:56 <TheJulia> sounds good
15:46:11 <arne_wiebalck> I would imagine fast track users want the IPA logs, no?
15:46:41 <arne_wiebalck> And they may not want to wait until the IPA is fully done (which I think is when the logs are collected).
15:47:01 <arne_wiebalck> Anyway, if anyone has thoughts on this, let me know :)
15:47:22 <TheJulia> It makes sense to get them and if they are doing something like burn in it does also make sense
15:47:35 <TheJulia> And is really just more operator capable at that point
15:47:38 <arne_wiebalck> Yes, burn-in is the use case at hand.
15:48:06 <arne_wiebalck> The h/w colleagues would like to get the network speed results for their records for instance.
15:48:53 <arne_wiebalck> There is a logical difference between burn-in and benchmark, but it does not make sense to run a benchmark when the burn-in provides these values.
15:49:57 <TheJulia> indeed
15:50:20 <arne_wiebalck> No need to decide anything now, I was just thinking that sending logs more regularly may be sensible for fast track.
15:50:32 <arne_wiebalck> In general, not only burn-in.
15:50:46 * arne_wiebalck stops now :)
15:51:51 <iurygregory> it may be sensible I would say, just wondering the overhead that can cause
15:52:47 <arne_wiebalck> iurygregory: yes, atm, we create an archive for a clean "cycle"
15:53:21 <arne_wiebalck> iurygregory: we will need to avoid sending the same logs over and over again
15:53:52 <iurygregory> interesting, I do like the idea
15:54:40 <iurygregory> next week we can talk about the PTG topics I would say since we only have 5minutes now
15:54:51 <TheJulia> Sounds good
15:55:05 <iurygregory> we already have the etherpad https://etherpad.opendev.org/p/ironic-yoga-ptg
15:55:15 <iurygregory> feel free to add topics =)
15:55:29 <iurygregory> #topic Who is going to run the next meeting?
15:55:51 <iurygregory> do we have any volunteers?
15:56:07 <rloo> is next mon a holiday for folks? it is in canada and i think US.
15:56:35 <iurygregory> *-* 06 Sep?
15:56:48 <iurygregory> maybe we can skip if people agree
15:56:49 <rloo> labor day in US
15:57:39 <iurygregory> it's my bday so I give +1 to skip LOL
15:57:54 <rloo> that's a really good reason to celebrate!
15:58:21 <iurygregory> yeah =)
15:59:20 <dtantsur> ++
15:59:39 <iurygregory> so we good to skip next weekly meeting?
16:00:05 <rloo> crickets means... yes :D
16:00:12 <iurygregory> yeah :D
16:00:17 <iurygregory> I will send an email
16:00:21 <iurygregory> tks everyone!
16:00:26 <iurygregory> #endmeeting