19:01:13 #startmeeting ironic 19:01:14 Meeting started Mon May 26 19:01:13 2014 UTC and is due to finish in 60 minutes. The chair is devananda. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:15 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:18 The meeting name has been set to 'ironic' 19:01:23 hi folks! who's here today? 19:01:27 o/ 19:01:31 * NobodyCam is here o/ 19:01:34 me :) 19:01:35 I know it's a holiday in the US, but we skipped last week due to post-summit travel ... 19:01:37 o/ 19:01:40 o/ 19:01:43 o/ 19:01:46 \o 19:01:48 so I wanted to still have a meeting today,e ven if not everyone makes it 19:02:12 as usual, our agenda is online (though it's short and wasn't edited much ahead of time) 19:02:16 #link https://wiki.openstack.org/wiki/Meetings/Ironic 19:02:22 :-p 19:02:28 #chair NobodyCam 19:02:29 Current chairs: NobodyCam devananda 19:02:37 #topic announcements 19:02:58 first off - was great to see everyone who made it to the summit 19:03:19 hope we're all recovered from the travel by now :) 19:03:30 I've been busy updating things based on our discussions 19:03:36 yes! sorry I had to leave mid summit :( 19:04:07 see the mailing list for recent nominations to add dtantsur to the core review team, and remove max_lobur 19:04:23 also see the ML for notices about the tempest-dsvm-virtual-ironic job 19:04:35 it's now gating on Ironic and diskimage-builder, and soon also gating our client 19:04:41 o/ 19:04:43 yay 19:04:51 :) woo hoo 19:04:52 lastly, we now have a specs repo 19:05:27 :) note name change from baremetal-specs to ironic-specs 19:05:48 yep. See the ML for full details (http://lists.openstack.org/pipermail/openstack-dev/2014-May/035885.html) 19:06:26 #topic status report 19:06:36 #topic status report: open reviews and release cycle 19:06:50 I added this section to the agenda,a nd would like to take a few minutes every meeting 19:07:00 for us to look at the overall review backlog 19:07:07 it's been steadily growing over the last few months 19:07:29 so I think we need a better way for cores to prioritize their review time than what we've been doing 19:07:38 has anyone seen a new dashboard: https://review.openstack.org/#/projects/openstack/ironic,dashboards/important-changes:review-inbox-dashboard ? 19:08:04 #link https://review.openstack.org/#/projects/openstack/ironic,dashboards/important-changes:review-inbox-dashboard 19:08:05 dtantsur, yeah, that's pretty neat! 19:08:28 dtantsur: I've seen it -- very helpful in finding what is "ready" t oreview 19:08:53 yeah, also patches that got no review yet 19:09:13 I'd also like to elicit suggestions on how to prioritize what we review -- in terms of alignment with our collective goals (eg, for integration with nova and graduation) 19:10:16 Should 1st priority be Nova driver? 19:10:40 yes, and anything related to helping that land in nova 19:10:47 eg, the instance_info refactoring 19:11:36 +1 for the nova driver and related stuff... the sooner we get the driver being reviewed and merged in nova the best 19:12:04 I wish the review ordering could take bug severity into account 19:12:12 lifeless: ++ 19:12:15 so we could review in this order - criticals, approved blueprints, anything else. 19:12:36 (or bikeshed variations on that) 19:12:38 that would be neat 19:12:39 i'd like the same ability for blueprints -- order by targeted release milestone, then by priority 19:12:48 fungi: ^ as an infra person around right now ... ^ wishlist ) 19:12:54 I wonder how hard it would be created this kind of dashboard? 19:13:21 this dashboard attempted to create a weight based on those things 19:13:22 #link http://status.openstack.org/reviews/ 19:13:32 but i have found it often not representative of what I actually want to see 19:13:32 where does all the factoring stuff come in? 19:13:56 devananda: lifeless: talking about a feature request for openstack-infra/reviewday? 19:14:28 i'm not too familiar with its internals, but dprince/pleia2 could probably better speak to the feasibility of that 19:14:39 rloo: code refactoring would be on my list of good-to-have by J1 milestone, so it doesn't slow down feature dev after that 19:14:53 rloo, they r important as well, idk if as important as the driver related stuff but def something that would be great to get in in J1 19:14:57 fungi: well for gerrit specifically, reviewday is not the One True Interface 19:15:14 fungi: but yes reviewday is a good workaround 19:15:33 whats its url again ? 19:15:36 lifeless: right, i think getting that capability into gerrit upstream is how that would happen 19:15:45 status.openstack.org/reviews 19:16:07 devananda: ^ code is in openstack-infra/reviewday, and patches obviously solicited 19:16:19 i dunno, NobodyCam's patch is last in that list... 19:16:33 perhaps we should make using that to focus reviews an explicit goal - with feedback and bugs about where it gets the order wrong ? 19:16:39 that needs a per-project UI to be useful, and some ability to customize it, IMO 19:17:10 I would prefer lower priority for what already got negative feedback 19:17:12 pleia2: bugs are on openstack-ci ? 19:17:16 rloo: I just pushed up a new rev so prob still in testing 19:17:22 lifeless: yes, please tag with reviewday 19:17:24 eg, reviewday puts the top ironic review as one that already has a -1 revie and a -1 from jenkins, on a bug that is importannce:undecided 19:17:26 it seems to be failing 19:17:32 dtantsur: thats tricky because non-core vs core negative feedback is different 19:17:56 lifeless, at least -1 from Jenkins :) 19:18:02 must just be me; I can't tell from status.openstack.org/reviews whether the top or bottom of the list of reviews is more important! 19:18:13 devananda: lifeless: probably the closest gerrit-centric solution which has been batted around is to get a review priority feature implemented upstream and then consume that in a later release 19:18:22 rloo: top => highest weight # 19:18:36 rloo: https://bugs.launchpad.net/ironic/+bug/1317647 is importance 'undecided' 19:18:37 Launchpad bug 1317647 in ironic "disk volume labels on conductor can conflict with nodes disk labels" [Undecided,In progress] 19:18:40 rloo: I think thats the problem :) 19:18:44 devananda: I find it hard to believe that some of the top ones are that important. 19:18:48 * lifeless fixes to critical 19:18:51 devananda: lifeless: then project review triagers could set review priorities via api based on whatever external sources/metrics they prefer 19:18:52 rloo: indeed 19:18:52 * dtantsur would also see setting weight as function, written say in Lua right in a browser :) 19:19:04 fungi: I liky 19:19:28 fungi: taht'd be handy 19:19:32 pleia2: how often does it regenerate ? 19:19:41 lifeless: devananda: know anybody who likes coding in java? ;) 19:20:02 fungi: depends how much you pay them :P 19:20:06 heh 19:20:42 ok - lest we spend the whole meeting on this, let's take this discussion to the ML and move on 19:21:19 lifeless: every 15 minutes, it takes about 10 minutes to run 19:21:46 just to recap this part, there were two parts? 1. decide on criteria by which to prioritize/rank reviews; 2. something that will list the reviews in our desired order. 19:22:03 #info reviewday's weighting isn't in line with core perception of what's important. enhancements to gerrit would be great, but need a java developer 19:22:21 * dtantsur did some Java long time ago 19:22:30 devananda: reviewday is an openstack thing though, so I would say 'needs fixing' not 'designed to be different' 19:23:24 it could be that since the gerrit upgrade some of the logic is off, so help getting priorities lined up is welcome 19:23:25 #info feedback/fixes to reviewday are welcome, while changes to gerrit will take longer / has external dependencies 19:24:14 thanks all. let's move on to specific sub teams for a bit 19:24:30 #topic subteam: CI testing 19:25:03 adam_g: hi! anything on the tempest front? 19:25:43 lifeless: aside from the disk label issue, which we've been reviewing with NobodyCam, anything else on the tripleo-ci front? 19:26:00 the flake issue with -virtual 19:26:07 NobodyCam: any updates // need anything on that bug? 19:26:15 and more broadly we still don't have the ironic tripleo test working 19:26:22 which is critical to prevent such regressions 19:26:47 lifeless: i replied about the ir-api bug, and dtantsur has suggested a potential fix 19:26:57 so bug https://bugs.launchpad.net/ironic/+bug/1323152 and check-tripleo-ironic-undercloud-precise failing 19:26:58 been poking folks for reviews on the patch, How we're getting close to landing 19:27:00 Launchpad bug 1323152 in ironic "CI failure "unable to enable dhcp for" (dup-of: 1321494)" [Undecided,New] 19:27:01 Launchpad bug 1321494 in ironic "NodeLocked causing random test failures" [High,In progress] 19:27:11 s/how/hope/ 19:27:18 which is https://bugs.launchpad.net/ironic/+bug/1300589 19:27:20 Launchpad bug 1300589 in ironic "check-tripleo-ironic-undercloud-precise failing" [Undecided,In progress] 19:28:04 lifeless: more descriptive bug labels are helpful ^^ :) 19:28:20 devananda: Ironic seems to not set bug importance, often 19:28:54 lifeless, my suggestion for https://bugs.launchpad.net/ironic/+bug/1321494 is https://review.openstack.org/#/c/95573/ (need to properly test it and write meaningful commit message) 19:28:55 Launchpad bug 1321494 in ironic "NodeLocked causing random test failures" [High,In progress] 19:29:00 devananda: so the problem is we don't know how many specific issues lie behind that symptom 19:29:13 devananda: picking the current obvious one may be misleading 19:30:09 lifeless: i'm noticing that we're not doing a good job prioritizing bugs as well. fwiw, i'm the only one regularly doing bug triage over the last few months -- and I already added that to the agenda to talk about today 19:31:03 ack 19:31:37 lifeless: fair point that there could be several bugs underlying tripleo-ironic-undercloud failing. i'd rather have a bug for each one than one bug for any and all of them over time that never gets closed 19:31:47 anyhow 19:32:10 let's dogpile on the known open bugs we have affecting both tempest and tripleo CI 19:32:17 and try to land those fixes tomorrow at the latest 19:32:25 +1 19:32:31 that would be good 19:32:43 #topic fedora support 19:33:01 devananda: closing that bug is super important 19:33:10 devananda: so I'm reallly scared by the idea it might not get closed. 19:34:37 dtantsur: any updates on fedora testing/support? 19:34:50 As to Fedora, I don't have anything new, except for I noticed DevStack failure related to SELinux inside diskimage-builder 19:35:17 Probably not related to Ironic, but worth investigating anyway 19:35:23 will do in 1-2 days 19:35:33 lifeless: bug 1300589 is hard to parse - it seems to be indiciating several things, some of which I think were already fixed in separate (more detailed) bugs 19:35:34 Launchpad bug 1300589 in ironic "check-tripleo-ironic-undercloud-precise failing" [Undecided,In progress] https://launchpad.net/bugs/1300589 19:35:52 dtantsur: ack, thanks for the update 19:35:56 adam_g, how possible would it be to add non-voting job on Fedora? Are we even close to this possibility? 19:36:11 devananda: ok, so run check experimental, it will fail, then update the bug to show the current failure ? 19:37:05 lifeless: i'd much rather have a separate bug per failure which can be closed when *that* failure is fixed -- and you're also welcome to use this bug as a catch-all for the current status 19:37:45 russell_h, jroll: either of you around? 19:38:04 devananda: sure, fundamentally I just want it fixed 19:38:11 devananda: I am, jroll isn't afaik 19:38:13 devananda: and AFAIK thats currently being driven by not-me. 19:38:33 devananda: e.g. NobodyCam and adam_g had volunteered to close the loop on this one. 19:38:42 devananda: so I'm going to direct your requests to them :) 19:38:44 lifeless: completely agree - I want to see it fixed as well. tracking the distinct failures distinctly will help us (or at least me) to see the progress 19:39:23 lifeless: and undersatnd what it means when there are multiple patches changing different things that reference the same bug # 19:39:43 russell_h: do you want to give any updates on IPA today? 19:40:56 devananda: not much change since the summit. I know jroll is working on some IPA-related specs, not sure how far he got. 19:41:37 russell_h: ack, thanks 19:41:46 skipping the IPA section then, and moving on 19:41:58 #topic discuss new subteams 19:42:11 So, I'd like to ask for ya'll's help on two things 19:42:15 1) reviewing specs 19:42:18 2) triaging bugs 19:42:28 questions? volunteers? 19:42:53 volunteering for 1) 19:42:58 I want to help with 1) 19:43:03 I'd help with both 19:43:24 awesome, thanks 19:43:29 want to help 2) 19:43:43 2) I can try give a hand, but urgh I often forget about the bugs there 19:43:48 #action devananda to add lifeless and dtantsur to spec review team 19:43:48 launchpad doesn't help much as well 19:43:55 I can help with 1 19:43:59 in theory, I'd like to help with 1) and 2), but then there's also reviews. I am out of time :-(. Will see how much I can get done. What are the priorities? (You'll say all.) 19:44:38 rloo: no worries. we need more code reviews too :) 19:44:51 #action devananda to add russell_h to specs review team 19:45:27 I can help with 2/ 19:45:29 ok, i'll stick with 3) :-) 19:45:35 i specifically want the specs review process to have the possibility to include non-core-reviewers whose opinions we value, eg. large scale operators who have domain architectural knowledge but less time to review code 19:45:43 so, thanks lifeless and russell_h ! 19:46:04 as for bug triage, there are some guidelines online that i'd like to make sure everyone has seen 19:46:13 #link https://wiki.openstack.org/wiki/Bugs 19:46:21 #link https://wiki.openstack.org/wiki/BugTriage 19:46:43 anyone want to be a point-of-contact for that subteam? 19:46:50 mostly that means giving a status report in the weekly meeting 19:47:10 mordred: please abandon https://review.openstack.org/#/c/95295/ 19:48:03 lifeless: actually no -- it got renamed BACK to ironic-specs 19:48:14 oh:) 19:48:25 I should have checked the details :/ 19:48:43 lifeless: there was a ML thread on the rename, and infra did some maintenance to move things around 19:48:51 yep 19:48:58 :) and it zuul seems to not be fixed yet to pick up those patches to merge :( 19:49:00 I thought Ironic had started out like the others 19:49:15 ok, ELOCAL, sorry 19:49:26 yuriyz, dtantsur, GheRivero -- any of you want to volunteer to lead the bug triage team? 19:49:53 lucasagomes: yea, fungi was looking into an issue with zuul. seems it still isn't processing events from ironic-specs repo :( 19:49:59 devananda, I can 19:50:07 dtantsur: great, thanks! 19:50:08 I'm just afraid I have not enough experience 19:50:15 but I'll do my best :) 19:50:47 dtantsur: I can help you with that :) 19:50:57 dtantsur: feel free to ask me all the questions that come up 19:51:00 devananda: yep, still digging deeper 19:51:02 GheRivero, devananda , \o/ 19:51:11 fungi: ack. thanks 19:51:12 ok, will do :) 19:51:51 thanks all - i think that actually wraps up the topics I had (we alraedy touched on review backlog) 19:51:59 #topic open discussion 19:52:20 I just wanted to draw attention to disk partitioning (what we with lucasagomes discussed) 19:52:37 of course /me is looking for reviews of 93133 19:52:44 part of it is actually https://bugs.launchpad.net/ironic/+bug/1322179 19:52:45 Launchpad bug 1322179 in ironic "Add check on total disk size before deploying" [Undecided,New] 19:52:58 NobodyCam, left one more comment btw 19:53:04 :) 19:53:17 will take a peek after the meeting 19:54:03 What I think I discovered is that 1. we need also 1 MiB spacing after the last partition; 2. don't know how these spacing play with rounding to GiB; 3. I don't think we have any checks on that 19:54:21 about diskimage-builder for Ironic agent image, I think CoreOS and Docker support should be removed for build with diskimage-builder 19:54:37 dtantsur, any updates on the local_gb == root_size thing? 19:55:02 yuriyz: I agree -- I do not want IPA to require docker. <-- russell_h 19:55:12 lucasagomes, I'm still looking into it, I am not sure it's the case 19:55:23 devananda, agree +2 19:55:43 devananda, yuriyz, wasn't it already mentioned that docker shouldn't be required. If it isn't, maybe it ought to be noted somewhere? 19:55:46 right yea, no hurry I'm just curious 19:55:53 yuriyz: based on my conversations with IPA team, they also agreed to remove it over time. docker was just the shortest initial path for them 19:56:01 +1 19:56:28 lucasagomes, well, judging my debstack source code you're right 19:56:55 today there is no runtime requirement on docker, but the only built-in packaging mechanism does use it 19:56:57 yeah I saw it awhile ago but didn't go deeper into the problem 19:57:08 but! if we have X GiB of hard drive, we need (X-1) local_gb 19:57:10 but it seems that there's a big miscalculation of the partitions and whole disk size 19:57:20 lucasagomes, dtantsur, NobodyCam: on the image size, it sounds like there are a few problems entagled together 19:57:20 because of these 1 MiB in front and in the end 19:57:29 devananda, yeah 19:57:35 sure, I'm still trying to sort things out 19:57:52 would anyone mind summarizing the whole thing? perhaps an etherpad to sort it out, then an email with proposed solution 19:58:02 We'd love to have someone add support to diskimage-builder for IPA. If no one else does it we might get around to it eventually, although I don't expect to actually use it. 19:58:21 russell_h: i plan to use it in the gate, fwiw 19:58:32 devananda, we have that bug ticket, so we can put all the finds there 19:58:39 ya I need dig in and see whats up there with the disk size stuff 19:58:46 russell_h: since DIB is alraedy in the gate to build the deploy ramdisk for PXE driver 19:58:51 devananda: yes, thats the biggest driving factor for us 19:59:02 devananda, yeah, I'll be updating the ticket 19:59:02 russell_h: my assumption is that we'll continue to use it to build the deploy ramdisk for IPA driver too 19:59:08 tl;dr... seems that nova get's the properties/local_gb and use it as the root_gb, it's not counting the ephemeral size and/or swap size to calculate the size of the root partition 19:59:12 *one* minute warning beep 19:59:43 yeah, it's one part of the problem 19:59:50 spacing is another 20:00:00 yeah 20:00:07 lucasagomes: sounds good 20:00:19 ok, time's up -- let's continue in channel if needed 20:00:23 thanks all! 20:00:28 great meeting all 20:00:30 thanks 20:00:31 thank you 20:00:32 #endmeeting