15:01:41 <TheJulia> #startmeeting ironic
15:01:42 <openstack> Meeting started Mon Oct 29 15:01:41 2018 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:43 <TheJulia> o/
15:01:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:46 <openstack> The meeting name has been set to 'ironic'
15:01:48 <kaifeng> o/
15:01:50 <etingof> o/
15:01:56 <jiapei> o/
15:02:01 <rloo> o/
15:02:11 <hshiina> o/
15:02:19 <rpioso> \o
15:02:41 <cdearborn> o/
15:02:47 <iurygregory> o/
15:02:50 <stendulker> o/
15:02:53 * dtantsur is here but will have to go afk for a while really soon
15:02:55 <TheJulia> Good morning everyone! And welcome to another ironic meeting!
15:03:17 <rloo> dtantsur: then we should postpone the clean step discussion?
15:03:30 <TheJulia> We have a few things on our agenda today, one of which is the just mentioned discussion
15:03:32 <TheJulia> #link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting
15:03:43 <TheJulia> #topic Announcements/Reminders
15:03:47 <dtantsur> rloo: I nearly feel like it deserves a meeting of its own
15:04:06 <dtantsur> but feel free to go ahead without me. my head does not work properly after all these news anyway :)
15:04:22 <rloo> dtantsur: oh, right. go celebrate... ? :)
15:04:22 * TheJulia is in the same space as dtantsur at the moment
15:04:29 * iurygregory too
15:04:42 <TheJulia> I have no announcements or reminders for today. Does anyone have anything they wish to raise?
15:04:50 <rloo> sigh. maybe lets just do a quick meeting then.
15:04:55 <rajinir> o/
15:05:19 <TheJulia> #info Looks like we're going to try and do a team gathering in Berlin during the summit.
15:05:22 <TheJulia> #link https://doodle.com/poll/iw5992px765nthde
15:05:23 <hjensas> o/
15:06:02 <TheJulia> That just so happens to be the one action item from last week, so I believe we can jump directly to reviewing status?
15:06:19 <rloo> jump +
15:06:20 <dtantsur> yeah, please vote
15:06:33 <dtantsur> and yes, please vote before Friday
15:06:51 <TheJulia> Okay, if nobody has anything else, we shall jump to status review
15:07:05 <TheJulia> #topic Review subteam status reports
15:08:12 <TheJulia> I updated a few things on Friday that I'm aware of
15:08:41 <TheJulia> It looks like kaifeng and I have started proposing CI job updates to clean up our templates for zuulv3 and also get integration testing with python3 going
15:08:48 <dtantsur> #link https://github.com/dtantsur/ironic-proxy how a federating ironic API proxy could look (early WIP)
15:09:10 <TheJulia> Of which, I think it would be good to tag them all with a unique topic and put them up for review this week (which I can do after the meeting)
15:10:39 * kaifeng used zuul as topic, but actually contains python3-first
15:11:00 <TheJulia> Yeah, since we kind of merged the efforts together (which does actually make a l ot of sense)
15:11:37 <TheJulia> I'm intending to continue that this week, and I'm going to keep working on ipv6'ifying one of our CI jobs
15:11:45 <TheJulia> I'm close. So close.
15:12:27 <rloo> wrt deploy templates -- is the next step to review/approve the spec?
15:12:42 <TheJulia> It looks like a lot of our smaller items just need some patch reviews this week
15:12:47 <TheJulia> rloo: I believe so yes
15:13:41 <rloo> and i'm not sure about the comment about jay's proposal. is there more info as to his proposal. i thought he was going to write a spec. is it an email thread? is it captured in deploy templates spec?
15:14:12 <rloo> by 'not sure', i don't know what the action item is for that...
15:14:34 <TheJulia> rloo: I think we partially captured that in the spec, but we could likely write more. We should make that contingent upon the existing spec but I can do a follow-up
15:14:47 <rloo> TheJulia: ok, thx for explaining.
15:15:15 <TheJulia> rloo: I think my comments on the spec reflect that as well, so I think reviews are the more important thing right now
15:15:40 <TheJulia> etingof: Quick question, is it just the two patches your working with regarding ipmitool execution?
15:15:53 <TheJulia> etingof: I'm kind of feeling like larger discussion may be required so punting to next week may be best
15:15:59 <rloo> TheJulia: thx, i'll try to look at that spec.
15:15:59 <TheJulia> but we could encourage reviews for feedback?
15:16:04 <TheJulia> rloo: thanks
15:16:51 <rloo> dtantsur|brb: wrt federation capabilities, did we approve a spec about it?
15:17:11 <TheJulia> rloo: we're kind of doing Proofs of concept to see what actually sticks
15:17:30 <etingof> TheJulia, there are many related patches, but the one I currently propose to make ironic more predictable is this -- https://review.openstack.org/#/c/610007/\
15:17:30 <patchbot> patch 610007 - ironic - Kill misbehaving `ipmitool` process - 2 patch sets
15:17:33 <rloo> TheJulia: ok, i just realized I mixed that up with 'Conductor role splitting'...
15:18:04 <etingof> TheJulia, +1 for a large discussion
15:18:05 <TheJulia> rloo: I suspect that is going to take a back seat until later in the cycle. Depends on Jim's time I guess
15:18:29 <TheJulia> etingof: Okay, if there are no objections, I can put them in the list for feedback and we can discuss them next week as well.
15:18:43 <etingof> +1 from me
15:19:09 <rloo> TheJulia: I'm going to delete 'Zuul v3 playbook refactoring' (around L295). I think it is captured under the python3 work. OK?
15:20:10 <TheJulia> rloo: wfm
15:20:35 <rloo> done
15:20:57 <TheJulia> Okay, I'm good to proceed to priorities for the upcoming week. How about everyone else?
15:21:18 <rloo> ++proceed
15:21:58 <TheJulia> #topic Priorities for the coming week
15:22:24 <TheJulia> I need to do some quick clean-up of merged items on the list, and I think we have two additions based on the discussion so far. Is there anything else people feel should be on the list for this week?
15:23:54 <rloo> TheJulia: could we add the deploy template spec under 'specs needed for this cycle' ?
15:24:43 <TheJulia> rloo: it is already there
15:25:05 <rloo> TheJulia: i only see clean step retrieval
15:25:17 <rloo> TheJulia: oh, there are two specs for the cycle.
15:25:29 <TheJulia> yup, let me fix
15:25:31 <rloo> one is 'needed' the other isn't...
15:25:32 <rloo> :)
15:26:17 <TheJulia> Well, they are both kind of needed in the grand scheme of the universe
15:26:35 <TheJulia> just different points along the continuum of interaction
15:27:13 <TheJulia> I'm good with the priorities, I'm just curious if anyone has anything they wish to add.
15:27:55 <rloo> for cisco, it sez 'patches in works...'. so no actual patches they want reviewed just yet?
15:28:10 <cdearborn> bump: review https://review.openstack.org/#/c/602132/ for idrac vendor priority
15:28:11 <patchbot> patch 602132 - ironic-specs - Add firmware update spec for Dell EMC hardware - 8 patch sets
15:28:22 <TheJulia> I can hunt the patches down, they are somewhere in our list
15:28:54 <TheJulia> s/list/ironic review/
15:29:01 <rloo> thx. otherwise, I'm good with the list of stuff!
15:29:11 <TheJulia> cdearborn: already in the list under vendor stuffs.
15:29:17 <TheJulia> vendor driver stuffs at least
15:29:40 <TheJulia> Okay then, if there are no objections we can go to open discussion and go from there.
15:29:48 <cdearborn> TheJulia: y - not getting much attention, hence the bump
15:30:53 <rloo> cdearborn: what does 'bump' mean? that you're mentioning here? :)
15:30:56 <TheJulia> cdearborn: I can review again this week, I'll ask around :)
15:31:29 <cdearborn> rloo: just a request for more attention
15:31:53 <rloo> cdearborn: gotcha :)
15:32:28 <TheJulia> Moving to Open Discussion
15:32:30 <TheJulia> #topic Open Discussion
15:32:55 <kaifeng> TheJulia: wrt the job migration, do you know if there is any project has done?
15:33:09 <kaifeng> i saw this one https://review.openstack.org/#/c/588496/, but it seems doesn't work from the comment
15:33:10 <patchbot> patch 588496 - ironic - Switch ironic-grenade-dsvm too zuulv3 - 3 patch sets
15:33:29 <TheJulia> I has one discussion topic, and it was largely an observation... that many of our gate failures are related to the rescue test failing due to port plugging.
15:33:53 <TheJulia> kaifeng: Any project like in the openstack community, or do you mean CI job that has been migrated?
15:34:31 <TheJulia> Grenade is a special case because there is a lot of scripting that has to be invoked a certain way at this time. So migrating it doesn't seem entirely feasible.
15:34:34 <kaifeng> yeah, because it seems no project under ironic has completed grenade
15:35:38 <kaifeng> emm, if that so, we just hold that until there is a solution
15:39:46 <rloo> TheJulia: wrt rescue/port plugging, any idea why there are issues? is it something random that we have no control over?
15:39:48 <TheJulia> Yeah, I suspect it might be good to sync up with derekh about that and maybe try and start a larger discussion on the  mailing list regarding grenade
15:40:06 <TheJulia> rloo: it comes back to the lack of callbacks and timing I think.
15:40:43 <rloo> TheJulia: do we wait? Maybe increase the wait?
15:41:18 <TheJulia> rloo: we do wait, 15 seconds hard coded in the conductor and I think the job retries for like 5 minutes
15:41:39 <rloo> and it takes longer than 5 minutes for the port plugging?
15:42:03 <rloo> or err, 5 minutes in rescue_wait or whatever state?
15:42:05 <TheJulia> the job times out, there does seem to be more than one thing going on, I can sometimes see the port plug fail
15:43:11 <TheJulia> Maybe if we could keep an eye out and start making a list and possibly trying to corrilate the failures... we might be able to target the general area to work on
15:43:39 <TheJulia> it "feels" super sporadic to me... Just enough to get on my radar with rechecks.
15:45:37 <rloo> TheJulia: any way to use that elastic recheck or whatever thing, to track this?
15:46:11 <TheJulia> General timeout in remote jobs and I'm fairly sure the logs we would need to peer into are not indexed
15:47:27 <TheJulia> s/remote/rescue/
15:47:53 <TheJulia> I'll start keeping a list and maybe we'll have enough data in a couple weeks to begin to see a trend
15:48:11 <rloo> ok. not sure who else has been looking at ironic ci failures.
15:48:11 <TheJulia> Does anyone have anything else to discuss?
15:48:24 <openstackgerrit> Merged openstack/ironic master: Migration step to update objects to latest version  https://review.openstack.org/613181
15:48:36 <TheJulia> Ohh, well that should be in the priority list for stable cores
15:49:00 <TheJulia> gah, dtantsur|brb has WF-1 on them
15:49:01 <rloo> wrt rescue -- maybe put it under the Gate Status stuff...
15:49:04 <rloo> in our whiteboard
15:49:07 <TheJulia> rloo: will do now
15:49:11 <rloo> thx TheJulia
15:50:31 <TheJulia> done
15:50:37 <TheJulia> Do we have anything else to discuss?
15:51:29 <rloo> crickets
15:51:52 <TheJulia> Well then, Thanks everyone!
15:52:11 <iurygregory> tks =D
15:53:27 <TheJulia> Have a wonderful week!
15:53:35 <TheJulia> #endmeeting