17:00:25 <jlvillal> #startmeeting ironic_qa
17:00:26 <openstack> Meeting started Wed Jun 15 17:00:25 2016 UTC and is due to finish in 60 minutes.  The chair is jlvillal. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:29 <jroll> sup
17:00:30 <openstack> The meeting name has been set to 'ironic_qa'
17:00:31 <sambetts> o/
17:00:34 <vdrok> o/
17:00:34 <jlvillal> Hello all
17:00:39 <JayF> o/
17:00:44 <rpioso> o/
17:00:53 <jlvillal> As always the agenda is here: https://wiki.openstack.org/wiki/Meetings/Ironic-QA
17:00:53 <thiagop> o/
17:00:59 <jlvillal> #topic Announcements
17:01:17 <jlvillal> #info Grenade is a non-voting job for Ironic
17:01:17 <devananda> o/
17:01:39 <krtaylor> o/
17:01:48 <jlvillal> #info Grenade does not support 'partial' jobs anymore. If a project wants to test rolling upgrades they should use multinode grenade.
17:01:52 <thiagop> yeah! Congratz, guys!
17:01:57 <[2]cdearborn> o/
17:02:22 <vdrok> agent partition images job is in place but it's not doing what is expected  until this patch is merged - https://review.openstack.org/#/c/329625/
17:02:40 <sambetts> jlvillal: well thats going to be a fun one to solve for Ironic
17:02:58 <jlvillal> #info No meeting next week, due to virtual mid-cycle
17:03:11 <jlvillal> Any other announcements?
17:03:22 <devananda> jlvillal: oh, interesting!
17:03:34 <sambetts> I believe the network patches got a green from grenade
17:03:38 <vdrok> also please review https://review.openstack.org/#/c/330087/ I suspect that our gate is broken right now, at least locally it fails
17:04:00 <jlvillal> Okay, moving on
17:04:03 <jlvillal> #topic Grenade
17:04:07 <jroll> vdrok: fun, thanks
17:04:38 <jlvillal> So the good news is that Grenade is up and running. Seems to be working for the most part reliably. I have noticed a few timeout failures.
17:05:05 <jlvillal> #info jlvillal to research how to get job statistics and get data on pass/fail ratio of Grenade jobs
17:05:41 <JayF> pass/fail can be done with openstack health
17:05:47 <JayF> I can link you to it gimme 1s
17:05:55 <thiagop> somebody sent a link to JayF earlier this week with a tool that could help that
17:05:56 <jlvillal> JayF: Awesome :)
17:06:15 <jlvillal> So I learned yesterday that 'partial' has been removed from Grenade.
17:06:20 <thiagop> I think this is it:
17:06:23 <thiagop> #link http://status.openstack.org/openstack-health/#/?searchProject=ironic
17:06:25 <jlvillal> We need to use multinode testing instead.
17:06:42 <JayF> That doesn't appear to have our grenade jobs in it.
17:06:45 <jroll> openstack-health only collects gate queue data, not check. grenade isn't in the gate queue
17:06:47 <JayF> I wonder if check queue or -nv is excluded
17:06:50 <jroll> :)
17:06:56 <JayF> that's a major whomp
17:07:07 <thiagop> Can we path it? :)
17:07:08 <JayF> but the data is still in graphite.openstack.org, just gotta find the right metrics and graph 'em
17:07:11 <thiagop> patch*
17:07:24 <jroll> no, that's intentional
17:07:26 <krtaylor> #link ci-watch.tintri.com/project?project=ironic&time=7+days
17:07:29 <krtaylor> also this ^^^
17:07:31 <jroll> as check queue can fail due to bad code etc
17:07:33 <thiagop> :(
17:07:36 <jlvillal> Unbeknownst to me, it appears that vsaienko has been working on something along the lines of multinode Grenade. But I need to talk to him. Maybe vdrok knows more.
17:07:38 <jlvillal> https://review.openstack.org/#/c/296432/
17:08:08 <jlvillal> If so, that would be great! And vsaienko is a few steps ahead of us on this work :)
17:08:47 <jlvillal> Thanks for all the links.
17:09:07 <vdrok> jlvillal: not that I'm aware of
17:09:15 <jlvillal> #info Grenade status page: https://etherpad.openstack.org/p/ironic-newton-grenade-whiteboard
17:09:53 <jlvillal> vdrok: I saw this posted this morning:
17:09:58 <jlvillal> <vsaienko> Ironic'ers just a notice, who don't know multitenancy patches passed grenade and multitenancy gate tests https://review.openstack.org/#/c/296432/. I'm kindly asking to review them as it is highest priority for community :). Thanks in advance!
17:10:11 <jlvillal> And now that I read that, I understand.
17:10:24 <vdrok> correct, grenade seems to be pretty stable :)
17:10:24 <jlvillal> Not multi-node :( But multi-tenant.
17:10:25 <jlvillal> Duh
17:10:34 <jlvillal> And darn :(
17:10:42 <sambetts> multi-node is a whole other can of worms
17:10:51 <jroll> sambetts: imagine the networking setup :)
17:10:59 * sambetts crys
17:11:02 <vdrok> yeah, we'll need to significantly change our devstack plugin
17:11:32 <jlvillal> Maybe this should be a mid-cycle topic.
17:11:41 * rloo thinks rolling upgrades will not make it in newton
17:11:52 <JayF> I mean, I wonder if someone will be able to get deep enough into the tech to even make it a useful topic at midcycle
17:11:54 * JayF == rloo
17:12:20 <rloo> rloo is definitely not equivalent to JayF
17:12:36 * jlvillal adding a proposed topic for mid-cycle
17:12:46 <sambetts> ++
17:13:10 <JayF> rloo: just an indication of agreement with your last statement
17:13:16 <JayF> rloo: and ircism
17:13:19 <JayF> *an ircism
17:13:34 <rloo> JayF: OH :D
17:13:49 <jlvillal> #info Ironic Grenade multi-node testing added to proposed topics for Ironic mid-cycle
17:14:02 <jlvillal> So that is all I got for Grenade.
17:14:09 <jlvillal> Anything else?
17:14:52 <jroll> jlvillal: can you send an email by end of week with pass/fail stats?
17:15:00 <jlvillal> jroll: Will do
17:15:10 <jroll> (and preferably stats on why it failed, because that horizon bug or something else)
17:15:16 <jroll> I want to turn it voting soon
17:15:18 <jroll> thanks
17:15:33 <jlvillal> #topic 3rd Party CI, brought to you by krtaylor
17:15:42 <krtaylor> hehheh
17:15:46 <thiagop> lol
17:15:46 <jlvillal> krtaylor: Let me know when you are done :)
17:15:53 <krtaylor> will do
17:16:16 <krtaylor> so I updated the driver test system table to have the latest stackalytics info
17:16:33 <krtaylor> #link https://wiki.openstack.org/wiki/Ironic/Drivers#3rd_Party_CI_required_implementation_status
17:17:07 <krtaylor> the question is, do we want to add drivers that havent already?
17:17:35 <krtaylor> drivers in stackalytics show up in driver marketplace
17:17:59 <krtaylor> so, any that are marked "No" in that column don't appear there
17:18:01 <jroll> haven't already what?
17:18:02 <thiagop> I think it is good to have it there, but not community's responsibility
17:18:06 <jroll> oh
17:18:24 <krtaylor> haven't already added themselves
17:18:27 <jroll> if we feel they're going to be removed this cycle, I don't think it's worth it
17:18:36 <thiagop> maybe do a rolling call to remember the maintainers is a good ideia
17:19:03 <jroll> we do need to update the info in stackalytics for existing things, too
17:19:07 <krtaylor> well, it boils down to a larger question - what do we do with systems that are testing but have not met the infra requirements
17:19:10 <thiagop> (remember them that they should put information there, I mean)
17:19:23 <rloo> i am probably wrong, but i thought that jroll (and then krtaylor offered) was going to add the drivers to stackalytics
17:19:42 <krtaylor> I can, but is it really our job?
17:19:51 <krtaylor> isn't that a vendor thing?
17:19:52 <thiagop> krtaylor: ++
17:19:52 <jroll> rloo: indeed
17:19:57 <jroll> well
17:20:09 <JayF> I mean honestly, upstream pays the price in questions in channel, on ml, etc if the info isn't updated
17:20:12 <jroll> with my PTL hat on, I would like ironic's info in the marketplace to be accurate
17:20:18 <krtaylor> I think all the community drivers are represented
17:20:19 <JayF> so regardless whose responsibility it is, it's better for the info to be accurate
17:20:42 <jlvillal> +1 for accurate marketplace data. As keep having people ask me about it.
17:21:14 <krtaylor> ok, I can push a patch to make stackalytics current, but I don't want to maintain that for all the third party vendors
17:21:28 <jroll> it's our responsibility to ensure accuracy, whether we do it or push vendors to do it
17:21:31 <krtaylor> they should do that for themselves
17:21:59 <thiagop> sambetts: CIMC is different of pxe_iscsi_cimc?
17:22:16 <sambetts> thiagop: ?
17:22:17 <jroll> also, it isn't up to date currently for community drivers
17:22:25 <rloo> i seem to recall anita's concern about stackalytics is that any vendor can push whatever they want. so she wasn't sure how reliable that was.
17:22:29 <jroll> e.g. agent_ipmitool is in more than juno
17:22:35 <jroll> rloo: indeed
17:22:55 <thiagop> sambetts: on the wiki krtaylor sent, there is a CIMC and pxe_iscsi_cimc drivers. Are they different?
17:23:20 <jroll> anyway, glad to see the wiki updated
17:23:37 <jroll> I'll add to my todo list to verify it independently, and I can handle the stackalytics stuff
17:23:45 <sambetts> thiagop: no, I'm not sure why we're listed twice there
17:24:02 <rloo> is the info in stackalytics similar to the info tracked in the 3rd-party page?
17:24:16 <krtaylor> oops, my bad, listed them twice, I'll fix that thiagop
17:24:20 <jroll> rloo: yes
17:24:40 <thiagop> krtaylor: maybe UCSM and pxe_ucs too...?!
17:24:51 <sambetts> thiagop, krtaylor: yup
17:24:52 <rloo> jroll: so maybe someone could automate (yay, another bot) updating stackalytics with the data in 3rd party page.
17:25:09 <jroll> rloo: hrm
17:25:22 <rloo> "intern" comes to mind.
17:25:30 <thiagop> rloo: I think the 3rd party page has some information about "our" requirements that stackalytics don't
17:25:30 <krtaylor> well, maybe we can use that page as the master list, once all test systems have fulfilled all their infra requirements
17:25:39 <thiagop> rloo: lol
17:25:42 <krtaylor> thiagop, right
17:25:52 * sambetts personally thinks the interface provided by stackalytics is more useful than the wiki page
17:25:59 <krtaylor> ++
17:26:18 <thiagop> The good in stackalytics is that it updates the CI status automagically
17:26:31 <krtaylor> like I said a summit, I was just using the wiki table as a way to keep track of the systems and where they are in  meeting the requirements
17:26:45 <rloo> it just seems silly to have humans manually updating stackalytics if the info is already avail elsewhere.
17:26:56 <krtaylor> thiagop, but it is not a reliable status
17:27:09 <sambetts> this is the "offical" third party systems page https://wiki.openstack.org/wiki/ThirdPartySystems
17:27:18 <krtaylor> rloo, but the info in stackalytics is mined for marketplace
17:27:38 <thiagop> krtaylor: the wiki is still useful, until we move to the status were putting the CI online is requirement for merging new drivers (is that the intended future, right?)
17:27:40 <krtaylor> sambetts, exactly, that link is in the requirements
17:28:18 <krtaylor> thiagop, it is the requirement (see spec)
17:28:32 <rloo> krtaylor: i'm not questioning whether stackalytics should have data. i'm just wondering how we can be efficient about getting that data to stackalytics.
17:28:50 <jroll> I think the clear answer there is have vendors do it
17:28:52 <jroll> BUT
17:29:01 <jroll> it's really out of whack right now and needs someone to clean it up
17:29:05 <jroll> especially community drivers
17:29:09 <krtaylor> every system must satisfy infra requirements *and* ironic requirements (see spec)
17:29:43 <krtaylor> jroll, I understand that viewpoint, but it is also an indicator if the system is doing  the right thing in the community
17:29:50 <rloo> jroll: so *after* stackalytics is in sync (volunteer please), we ask vendors to keep it up to date?
17:30:10 <jroll> krtaylor: I don't understand what you mean
17:30:53 <krtaylor> jroll, what I mean is, the vendor should care enough about their driver to list it in stackalytics
17:31:11 <krtaylor> if they don't, should we?
17:31:14 <jroll> krtaylor: sure, what I'm actually referring to is ensuring the data is accurate
17:31:42 <jroll> also
17:31:50 <krtaylor> oh, not if it is listed in the first place, gotcha
17:32:02 <jroll> from a project marketing standpoint, we do want to say "hey this works ona bunch of hw"
17:32:17 <jroll> if someone comes to the marketplace wondering if we do iLO, and iLO isn't listed, then we lost a user
17:32:21 <jroll> right?
17:32:47 <jroll> so honestly we should hold ourselves responsible for ensuring the data is accurate and there in the first place
17:32:51 <cdearborn> is it documented anywhere how to make a driver show up in the marketplace?
17:32:53 <devananda> it also reflects poorly on us, as a project and a community, if that data is inaccurate
17:32:59 <jroll> maybe that means bugging $vendor to do it until they do
17:33:04 <jroll> maybe it means doing it ourselves
17:33:06 <krtaylor> cdearborn, yes, link is above the table
17:33:16 <devananda> jroll: ++
17:33:19 <jroll> I lean toward doing a first pass myself
17:33:27 <jroll> once it's accurate, it's easy to keep up to date
17:33:33 <thiagop> I think that information should be unified to a single source at some point in time
17:33:34 <cdearborn> can you post a link to the page with the table
17:33:38 * thiagop dislikes scripts
17:33:51 * devananda thinks we also need to document how to keep it up to date
17:33:56 <thiagop> not now, but in the future
17:34:03 <jroll> cdearborn: one better, https://wiki.openstack.org/wiki/DriverLog#How_To:_Add_a_new_driver_to_DriverLog
17:34:05 <thiagop> devananda: +1
17:34:05 <krtaylor> jroll, thats the problem with wikis, the info get stagnant quickly
17:34:19 <jroll> krtaylor: yeah, wikis are terrible, we should stop using them
17:34:28 <krtaylor> cdearborn, https://wiki.openstack.org/wiki/Ironic/Drivers#3rd_Party_CI_required_implementation_status
17:34:33 <jroll> krtaylor: but, drivers don't get added or removed often, so I disagree with that assertion
17:34:52 <cdearborn> jrol, krtaylor: thx much!
17:34:58 <jroll> np
17:35:00 <jlvillal> And last I recall new users had a difficult time getting an account on the Wiki, due to spam control.
17:35:23 <jroll> so anyway, I'll do a pass on stackalytics at some point, can't promise it will be soon though
17:35:29 * jroll volunteers
17:35:30 <krtaylor> jroll, fair enough, I'll push a patch and then we'll see how big  aload it is to keep it up to date, learn as we go
17:35:33 <thiagop> maybe we should start thinking of something that works with the driver composition reform, since it tends to increase the number of combinations
17:35:36 <krtaylor> ok, better yet!
17:35:39 <jroll> krtaylor: or that works :)
17:35:43 <krtaylor> hehheh
17:35:56 <jroll> krtaylor: context, I'm going afk for a week or two after midcycle, so
17:36:11 <krtaylor> I like rloo 's idea to automate it somehow
17:37:37 <krtaylor> jroll, imho, I think we should do this in a group then, to make sure what is there is accurate, because I don't know the state of everything
17:38:14 <rloo> maybe a midcycle topic/thing-to-do?
17:38:17 <jroll> krtaylor: yeah, let's start with a WIP gerrit review and we can chime in
17:38:19 <jroll> or that
17:38:42 <krtaylor> the info need to be complete in the table before we push a patch, stackalytics folks require a complete record
17:38:54 <jroll> uh
17:38:59 <jroll> what?
17:39:15 <jroll> we'll complete it before we ask them to merge it :)
17:39:29 <jroll> or are they against more than one patchset
17:39:35 <krtaylor> sure, np, it will just be -1ed by them until it is complete
17:39:52 <jroll> yeah, that's totally fine, hence the WIP
17:40:09 <krtaylor> ok, so I'll push a  WIP patch today
17:40:13 <jroll> that's really the goal, get ironic people to review it and suggest what needs to be added
17:40:30 <jroll> nice, thanks, maybe send it to the ML to ask people to jump in
17:40:32 <rloo> thx krtaylor
17:40:33 <krtaylor> then we can throw darts at it in channel or in the virtual midcycle
17:40:38 <jroll> ++
17:40:41 <jroll> thank you
17:40:47 * rloo saves her darts for other things
17:41:13 <krtaylor> so...anything else for third party CI topics?
17:41:20 <thiagop> I have an item
17:42:20 <thiagop> I was thinking about submitting a talk on 3rd party CI deployment for Barcelona's Summit and I'd like to herd some cats that would like to share their expertise
17:42:27 <thiagop> anyone interested?
17:42:36 <jroll> ooo a panel would be cool
17:42:57 <krtaylor> hm, panel
17:43:01 <jlvillal> A panel you can have 5 people. 4 for the panel and 1 moderator. A talk can only have two people. From when I looked.
17:43:12 <thiagop> or that :)
17:43:18 <sambetts> that would be cool!
17:43:56 <krtaylor> panel wouldn't be as good to show slides to clarify points
17:44:20 <thiagop> Then, can I write sambetts and krtaylor on my list of interested folks?
17:44:47 <krtaylor> thiagop, I am interested, or I would like to get one of my team members to look at it
17:44:55 <thiagop> jroll: do you do 3rd party or just community?
17:45:03 <jroll> thiagop: the latter
17:45:08 <rloo> would it be useful to have people that did 3rd party CI for other projects? are the issues the same regardless of ironic or neutron or ... ?
17:45:41 <sambetts> rloo: some issues are some issues aren't
17:46:09 <thiagop> rloo: I don't know many people from other projects. I was thinking on something more Ironic related, but if we can expand would be yet more awesome
17:46:25 <thiagop> *from other projects that do 3rd party CI*
17:46:40 <krtaylor> so, we have had third party BoF's before
17:46:50 <thiagop> Nobody from iLO CI here? :D
17:46:51 <sambetts> rloo: high level CI infra is normally the same accross third party CIs, but the bottom of the stack is where things get werid for Ironic because of talking to real hardware etc
17:47:30 <thiagop> sambetts: +1
17:47:31 <rloo> sambetts: ok.
17:47:37 <krtaylor> they do share a majority of the infrastructure, but ironic has new problems to conquer with machine pools (instead of VMs)
17:48:09 <vdrok> thiagop: it's Nisha or stendulker
17:48:22 <vdrok> from ilo
17:48:32 <thiagop> so, to move it on. If somebody else is interested, please ping me on the main channel.
17:48:45 <thiagop> I'll ping them, thanks vdrok
17:48:56 <krtaylor> thiagop, will do, thanks
17:49:02 <krtaylor> anything else?
17:49:18 <krtaylor> so, lets move on ... jlvillal ?
17:49:25 * jlvillal thinks this has been a new time record for 3rd party CI topic ;)
17:49:30 <krtaylor> hehheh
17:49:32 <jlvillal> Thanks!
17:49:46 <jlvillal> #topic Open Discussion
17:50:02 <jlvillal> If anyone has anything, speak up now
17:50:39 <jlvillal> Okay, I think we are done.
17:50:43 <jlvillal> Thanks everyone!
17:50:47 <jlvillal> #endmeeting