21:01:55 #startmeeting project 21:01:56 Meeting started Tue Feb 25 21:01:55 2014 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:01:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:02:00 The meeting name has been set to 'project' 21:02:03 #link http://wiki.openstack.org/Meetings/ProjectMeeting 21:02:12 #topic Swift 1.13.0 RC 21:02:23 Early this morning I cut the milestone-proposed branch from the SHA that notmyname gave me for 1.13.0. 21:02:24 o/ 21:02:31 Unless a regression is found in testing that RC, it shall be released as Swift 1.13.0 later this week 21:02:43 The current plan is that the next release of Swift shall be the Icehouse-synchronized release, with RCs expected early April 21:03:14 #topic icehouse-3 progress 21:03:39 After Feature Proposal Freeze last week we had a lot of cleanups going on 21:03:52 Now the current plans are all reasonable, but there is a lot left to do this week ! 21:04:05 Remember the feature code needs to land before EOB Tuesday March 4, after that it will need an exception 21:04:17 Review and land early, as always... winter is coming 21:04:35 One never knows how the gate will respond as we get closer and start piling up stuff on it 21:04:54 but it's unlikely to get a lot faster 21:05:22 Questions / comments on that ? 21:05:43 we're going to be requesting some exceptions 21:05:55 there are key heat pieces we need for proper upgrades 21:05:59 that aren't landed yet 21:06:06 it is likely that Ironic will request an exception for the nova "ironic" driver as well 21:06:08 lifeless: I think stevebaker mentioned them to me 21:06:11 lifeless: what heat changes? 21:06:20 without which TripleO (and Trove, and others) have issues 21:06:28 stevebaker: graceful node changes 21:06:42 stevebaker: and we need to solve the 'nova 500's -> everything stops' bug. 21:06:49 stevebaker: because honestly, APIs are not bulletproof. 21:07:08 ok 21:07:20 stevebaker: and the idea that a user should have to intervene after a DB or network glitch, or keystone being overloaded for a few seconds... is really hard to understand 21:07:25 fwiw Heat and Horizon are naturally more liekly to get exceptions, as they need to catch up with the craziness upstream 21:07:36 lifeless: at least that is a bug, not a feature 21:08:04 stevebaker: are graceful node changes in your "high" list already ? 21:08:25 stevebaker: sure, but SpamapS tells me its been getting pushback, so I'm expecting that we need yet more discussion around it. So... may be last minute when we finally get consensus 21:09:01 ttx: I'm not sure what specific blueprints are needed for graceful node changes 21:09:15 I'll sort it out with SpamapS and lifeless 21:09:22 sounds good 21:09:35 #topic Red Flag District / Blocked blueprints 21:09:53 stevebaker: while we are at it, you mentioned a potential conflict we may have to solve ? 21:10:16 blueprint instance-users needs this devstack change to allow its gating to pass https://review.openstack.org/#/c/76036/2 21:10:22 that is all 21:10:33 #info blueprint instance-users needs this devstack change to allow its gating to pass https://review.openstack.org/#/c/76036/2 21:10:56 dtroyer: can we get some express love on that one ? ^ 21:11:14 In other news, horizon/neutron-subnet-mode-support is blocked on neutron/ipv6-two-attributes 21:11:16 ttx: +2 21:11:27 dtroyer: thanks 21:11:49 just +Aed 21:11:50 ttx: I'd like to raise a question about nova BP deprecate-baremetal-driver, which was untargeted and unapproved last week 21:11:53 markmcclain: what's the rough ETA for ipv6-two-attributes ? 21:12:01 stevebaker: magic! 21:12:15 stevebaker: so does that close issues with the heat-slow tests? 21:12:22 dtroyer, sdague, thanks 21:12:24 devananda: cool, just a sec 21:12:55 sdague: I've seen this failure a few times, I need to look into it http://logs.openstack.org/36/76036/2/check/check-tempest-dsvm-neutron-heat-slow/1adf53f/console.html 21:13:13 ttx: hopefully later this week 21:13:34 ttx: both have gotten core review attention 21:13:40 sdague: some help in extracting pass/fail stats from logstash would be great 21:13:57 stevebaker: ok, getting that voting should definitely be high priority before the mad rush, otherwise I'm sure neutron and nova will break heat again during i3 21:14:14 sdague: we'll need to add the job to neutron too 21:14:22 yes 21:14:23 david-lyle: So.. if it lands then you shall get a FFE if needed to get it in 21:14:29 well, to all the jobs actually 21:14:37 ttx: ack 21:14:43 david-lyle: if it doesn't land and is deferred... then I guess your change doesn't make sense anyway 21:15:00 ttx: correct, will push to Juno at that point 21:15:16 david-lyle: like I said above, horizon (and Heat) are naturally given more FFEs to catch up with the latest 21:15:34 devananda: ok go 21:16:25 ttx: so, that blueprint was untargeted and the relevant patch -2'd last week 21:16:37 ttx: and, as i undersatnd it, ironic's graduation potential is pinned on that work 21:16:52 russellb: around? 21:16:55 yes 21:17:02 driver is blocked on driver CI 21:17:03 ttx: also, the code hasn't gotten any meaningful feedback from nova reviewers yet, even though we started the work months ago 21:17:22 devananda: was WIP for most of that time right? 21:17:39 russellb: it was, yes, but WIP was removed ~ a month ago, I think 21:17:58 i3 is a dangerous time to rely on, heh 21:18:05 sure 21:18:11 but the main blocker is CI 21:18:20 probably discouraged folks from looking 21:18:32 we're not merging drivers with no CI because we don't know if it works, 21:18:38 so if we don't have CI, what's the point of looking at it? 21:18:49 hmm, what options do we have here (trying to wrap my head around this) 21:18:51 ttx: so I'd like to be clear on whether that BP and the related work, nova "ironic" driver and CI for it, is a blocker for Ironic itself, or not 21:19:04 and if it is, what we can do to unblock it for Icehouse -- if anything 21:19:19 IMO, it is a blocker 21:19:32 ttx: it's blocked unless we give ironic a pass on the driver CI requirement 21:19:43 ttx: and when i socialized that around nova, i found about zero support :-/ 21:20:05 I don't want to fork the user base any more than anyone else -- but my mind is currently split on this 21:20:22 devananda: how far away are we to have proper driver CI ? 21:20:47 is it a "few more days" thing or a "never anyway" one ? 21:20:51 I would like to think we're close to getting devstack to set up the environment 21:20:55 * devananda fins the link 21:21:04 #link https://review.openstack.org/#/c/70348/ 21:21:18 it definitely needs more work -- it is breakign on some of the netron integration last I tried it 21:21:25 russellb: what happens to the current baremetal driver ? 21:21:38 it stays for now 21:21:50 russellb: but it doesn't have proper Ci either, right ? 21:21:52 can I ask 21:21:53 we're giving that a pass on the requirement to let ironic folks focus on ironic 21:21:57 from a project perspective 21:22:02 since that's where baremetal interest went 21:22:10 can TripleO switch to using Ironic before Ironic's driver is merged to Nova? 21:22:35 lifeless: i don't know, can it? 21:22:38 lifeless: I guess you could use an out-of-tree driver 21:22:49 russellb: socially, culturally. Not technically. 21:23:01 technical stuff is straight forward. 21:23:03 ttx: my concern with keeping baremetal in nova another cycle: IMO, that _will_ create a split. fokls are already pushing strongly to get additional features in ironic 21:23:03 i think supporting both would be acceptable 21:23:08 supporting only ironic seems kinda bad 21:23:43 devananda: how about parity and migration path from nova-baremetal? any issues there? 21:23:46 russellb: nova baremetal is extremely tricky do deploy properly. 21:23:49 just wondering if there's any other potential blockers 21:23:50 russellb: scale issues; HA issue. 21:23:57 russellb: parity isn't an issue - we alraedy have more features than nova-bm 21:24:01 russellb: I have zero interest in solving those problems in nova baremetal. 21:24:10 not just more, but at least all of the same, yes? 21:24:33 like, it's not missing some key feature nova-bm has, for example, even though it has all these other things 21:24:40 no 21:24:42 ok 21:25:04 to be extra clear 21:25:05 i ask because, you know, neutron has more features too :-) 21:25:16 devananda: so that devstack patch looks like it just has a bad format in one bit and hit a race in one of the tests 21:25:16 there are two features in noav-bm that are not in tree fo rironic yet -- but patches are up and nearly ready to land 21:25:19 ie, this week 21:25:31 *ie, i expect console and ephemeral support to land this week 21:25:35 so then, not parity? :) 21:25:47 heh, but parity expected by feature freeze 21:25:51 right 21:25:57 russellb: we could also consider that nova ironic driver is not a prerequisite for graduation. that would be "integration", as an Horizon panel. 21:26:05 so post-graduation 21:26:17 ttx: well that's not what our graduation requirements say 21:26:27 http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements#n69 21:26:29 ttx: which were specifically written to avoid a bad situation from happening again 21:26:37 ttx: so i'm -1 on that 21:26:39 that section landed last week, which caught me a bit by surprise 21:26:48 devananda: you saw it way before it landed, come on ... 21:27:19 russellb: not that long before ... 21:27:24 ok, but not last week :) 21:27:27 you commented jan 31 21:27:39 ack 21:27:41 but really, we can't allow another situation like we have with neutron 21:27:42 either way, approving another neutron (bomb) seems irresponsible 21:27:53 agreed on avoiding the neutron issue 21:27:53 and that's what we're allowing if we graduate ironic before we can deprecate the old thing 21:28:04 i agree -- i definitely do not want this to become a similar fiasco 21:28:10 ok cool 21:28:16 russellb: so what are the specific requirements for deprecating nova-bm ? 21:28:26 lifeless: parity and a migration plan 21:28:37 plan or implementation ? 21:28:48 lifeless: prototype :) 21:28:50 russellb: which is why i'm trying to figure out the shortest path to avoid another cycle of everyone's effort being split between two projects :) 21:28:53 implementation, if there's code involved 21:28:56 could just be docs 21:29:05 like, for migration, as a deployer I'd be entirely happy with use the API to copy settings from nova and push them to Ironic 21:29:09 i think keeping nova-bm is very little effort on our part 21:29:28 so I'm fine with granting a FFE to get the ironic driver in if the drievr CI is a bit late 21:29:39 sure, that's an option 21:29:44 if the driver is entirely self contained 21:29:47 seems like a big FFE 21:29:52 review bandwidth is an issue 21:29:59 but theoretically possible 21:30:07 nova-core spread incredibly thin as it is 21:30:29 so need a heads up if we expect that to happen 21:30:36 BUT it won't work unless driver CI is up to snuff 21:30:38 so i can figure out who could review it 21:30:41 right 21:31:15 russellb: lifeless: what do you see as the cost / impact of ironic not graduating // the nova-ironic driver not landing in Icehouse? 21:31:16 ironic has been running after the clock all cycle 21:31:33 So, TripleO /needs/ HA and scale 21:31:53 if it's not ready, it's just not ready 21:31:58 Like I say, I have 0 interest in the work needed to do that with nova-bm [as to why - thats a separate conversation] 21:32:01 i'm not willing to break rules and create another bad situation 21:32:11 russellb: sure, I'm not asking you to. Gimme a second here :) 21:32:23 russellb: that's fair. but i am concerned that both paths risk creating (different) bad situations 21:32:45 and it's also not fair to drivers who have busted their tails to get CI up 21:32:51 or the driver that may get removed next week over it 21:32:54 lifeless: could tripleO still use ironic (and an out-of-tree ironic driver) if it misses the Icehouse boat ? 21:33:00 So, if Ironic isn't fully integrated, TripleO will have to choose between wrapping nova-bm in external tools (corosync, pacemaker) and the fugly that ensues to get HA. 21:33:03 OR 21:33:09 Using Ironic with an out of tree Nova driver. 21:33:22 lifeless: assuming you're pinned to Icehouse 21:33:28 I think we'd choose the out of tree driver today, as the lesser of evils. 21:33:32 lifeless: this requirement has been in place for a long time 21:33:33 lifeless: presumably you could use an in-tree driver if it landed early juno right? 21:33:36 lifeless: like, a year 21:33:41 russellb: RH are -very- much intending to ship a product :) 21:33:41 lifeless: since TripleO isn't bound to releases, you could use an in-tree ironic early in Juno 21:33:56 lifeless: i'm aware, but i have my upstream hat on 21:34:04 russellb: and other vendors are planning to ship product too 21:34:15 dansmith: which requirement has ben in place for a year? 21:34:16 vendor product desires are the least of my worries 21:34:19 russellb: right, and with my upstream hat on, I'm looking at the constituency of tripleo 21:34:24 devananda: the CI requirement 21:34:25 i'm certainly not going to bend rules because of someone wanting to ship a product 21:34:27 russellb: which is all deployers 21:34:32 russellb: again, not asking you to 21:34:40 ok, not sure why you brought it up then 21:34:50 russellb: I'm answering deva's quesetion about consequences of Ironi failing to integrate/graduate 21:34:55 ok. 21:35:07 but anyway, you could use the in tree driver that lands early juno right? 21:35:21 and drivers are easy to backport, if some downstream wants to 21:35:23 Yes, and tell product folk to backport that to I 21:35:24 (since you're not bound to releases anyway) 21:35:26 yep 21:35:41 that sounds like plan C 21:36:04 so given that, I'm much more worried about Ironic API stability and driver fit-for-purpose than the integrated bit being set... *but* 21:36:06 dansmith: yes, but integration testing was always communicated to me as a post-graduation requirement. not a pre-graduation. but that's a bikeshed at this point. 21:36:16 the integrated bit being set is a great proxy for those things. 21:36:24 devananda: yes (re: bikeshed) 21:36:29 so.. plan A is we get driver CI up to snuff, and the driver in nova (potentially using a FFE) 21:36:43 what was B, heh 21:36:54 heh, yea, i think i missed B too ? 21:37:04 run around screaming with hands in the air ? 21:37:09 :) 21:37:10 ok okok ok plan B 21:37:17 devananda: so you said your merge got -2'd 21:37:30 lifeless: yes, and the BP was un-approved and un-targeted 21:37:35 because of driver CI, yes 21:37:39 devananda: which BP - CI? or driver? 21:37:43 russellb: i'm surprised that it isn't even approved now 21:37:44 plan C would be to miss the Juno integration boat and land the driver early in Juno. TripleO uses an out of tree driver in the interim between the two 21:37:45 https://blueprints.launchpad.net/nova/+spec/deprecate-baremetal-driver 21:37:51 lifeless: driver in noav 21:37:55 devananda: because everything has to be re-reviewed for juno 21:37:56 *nova 21:38:07 just what was done for everything 21:38:08 russellb: ahhh gotcha. 21:38:24 process artifact 21:38:31 and i have to say, the FFE thing worries me 21:38:40 so question 21:38:41 because i'm afraid it will be incredibly difficult to get folks to review it 21:38:47 how does the driver get CI without the driver being in tree? 21:38:55 and if they do, they'll feel immense pressure to just approve asap, instead of giving it the review it deserves 21:39:00 russellb: that's fair. a driver AND all the CI for it is not a small set of changes 21:39:04 and I won't approve it unless people are lined up to +2/APRV it 21:39:10 lifeless: that's a technical question, that's easy :-p 21:39:18 lifeless: it's possible, technically 21:39:18 russellb: sure, but whats the answer ;) 21:39:19 something like this 21:39:20 tight schedule 21:39:45 lifeless: land changes in devstack and tempest. add experimental check to nova pipeline. trigger it only on the patch sets that add the ironic driver 21:39:46 a custom devstack-gate job that installs the driver first 21:39:48 or whatever 21:40:02 I guess what I'm asking 21:40:05 since we're all here 21:40:28 is - are the devstack and tempest folk ok with how this will all fit together? 21:40:35 or are we inventing as we go along? 21:40:57 on the devstack side, i think it's fine with the way it supports plugins 21:41:11 there's some precedent - we did something like this already to land our Ironic tempest changes 21:41:13 it looks good to me once it passes gate 21:41:14 you can add whatever the heck you want to devstack by dropping in a couple new files 21:41:15 how much of tempest runs against ironic under nova, by the way? 21:41:31 dansmith: can you rephrase that? 21:41:52 devananda: how much of tempest runs against nova with ironic as the virt driver? 21:41:57 you could stage the driver in a stackforge repo while you work on CI, or just install it from the right review, i guess 21:41:58 dansmith: none today 21:42:06 devananda: ? 21:42:17 you mean, because it's not set up 21:42:24 i think he means, how much of the test suite do you expect to pass 21:42:30 ...yeah 21:42:33 dansmith: because the driver isn't in nova, and the devstack setup stuff is still being figured out 21:42:39 basic deploy/undeploy/stop/start 21:42:40 see patch I linked ~15 min ago 21:42:41 devananda: so you don't know? 21:42:43 oh 21:42:45 nothing with virtual networking 21:42:49 nothing ceilometer 21:42:58 nothing trove 21:42:59 yea, i expect start/stop/SSH into node to all work 21:43:17 if this isn't known at this point, I can't imagine that this is doable 21:43:22 CI will not just go green, 21:43:29 it will find tons of issues that have to be fixed, 21:43:34 then integrated into the proposal, 21:43:36 re-reviewed, etc 21:43:47 dansmith: I think its known but not written down. 21:43:56 lifeless: it doesn't sound like that 21:44:05 "I expect $foo to work" 21:44:10 dansmith: anything that depends on virtualised infrastructure is not offered by nova-bm, nor by Ironic 21:44:14 it's certainly conceivable that tempest hammering nova + the nova-ironic driver will uncover a ton of issues 21:44:17 but then 21:44:29 if you did that to nova-bm you'd have a lot of problems too, I bet ;) 21:44:44 devananda: that's not really the point, as we're already giving a free pass for it 21:44:47 devananda: we've certainly found issues with the docker driver since running tempest against it. Surprisingly few so far, but issues nonetheless. 21:44:53 devananda: if we weren't, nova-bm and ironic would be out 21:45:08 dansmith: sure 21:45:17 also - whether ironic will pass CI or not is a bikeshed 21:45:28 timeboxing this to 5 more minutes, we need to move on 21:45:28 what? 21:45:47 you can't just "have CI" and have it all fail and expect to get in... :) 21:46:28 dansmith: my point is, if there's no review bandwidth // too much doubt from other parties // etc 21:46:37 dansmith: I think devananda means that that is a matter for the folk pushing the work. Whether it passes on first push of the review, or 30th, is orthogonal to the requirements. 21:46:39 dansmith: then we won't devote what time is left in icehouse to that requirement 21:46:51 or not :P 21:46:55 dansmith: and we'll focus on the other important things 21:47:02 lifeless: I'm saying that it plays into the "can we possibly review this before icehouse" equation 21:47:10 dansmith: ah yes 21:47:26 ttx: I'll shut up now 21:47:33 Summary is: this has run unfortunately late, plan B looks more unlikely as each hour passes, so we might need to consider plan C 21:47:35 right, and I agree, if there aren't preliminary tempest results now, I don't think it's doable 21:47:48 ttx: thanks for giving this discussion some time 21:47:53 It's sad because it all probably just needs one more month 21:48:14 but tripleO at least shouldn't be taht much affected if it can consume early Juno-landed stuff 21:48:38 okn moving on 21:48:38 tripleo will figure something out 21:48:41 #topic Incubated projects 21:49:00 SergeyLukjanov, kgriffs; around ? 21:49:08 It's a bit of the same topic actually 21:50:39 ttx, o/ 21:50:39 re savanna i3 - https://launchpad.net/savanna/+milestone/icehouse-3 21:50:49 everything is ok, all features and mostly all bug fixes under review 21:50:50 icehouse - targeted client released 21:50:51 bunch of API tests merged to tempest, several more on review 21:50:52 the same with CLI tests 21:50:59 and I hope to be able to land some simple scenarios test 21:51:03 SergeyLukjanov: you plan to follow feature freeze next week ? 21:51:57 ttx, it sounds ok, we could probably not land some features patches that are under review 21:52:13 ttx, but FFE will be ok for it 21:52:24 SergeyLukjanov: ack 21:52:53 cool 21:53:08 ttx, I have two more questions :) 21:53:18 SergeyLukjanov: go for them 21:53:24 ttx, is it ok to hack docs after i3 while we're incubated? 21:53:46 sure! Doc (and test) fixes are not affected by feature freeze 21:53:53 ttx, awesome 21:54:02 so you can also increase test coverage 21:54:05 ttx, minus several FFEs :) 21:54:14 and where is the best place for start discussion about renaming? 21:54:24 honestly, I'm scared to rename after FF 21:54:58 SergeyLukjanov: we have the option to delay FF (icehouse-3) for savanna a bit too 21:55:35 it's better if you can follow the regular one, but i agree that renaming would better happen BEFORE FF 21:55:57 ttx, I see two options - delay renaming to the time when Juno dev will be opened vs. delay FF 21:56:26 SergeyLukjanov: I think it's better if the icehouse version carries the future name 21:56:30 for the first look #1 looks more consistent - to have savanna i1, i2, it and I release 21:57:18 SergeyLukjanov: but the message would be confusing. Savanna will be integrated in Juno under the name X ? 21:57:37 i1, i2 i3 are just intermediary milestones 21:57:43 they don't matter that much 21:57:46 heh 21:57:48 #topic Open discussion 21:58:00 SergeyLukjanov: other questions ? 21:58:18 ttx, I'm thinking about the ETA for renaming 21:58:37 SergeyLukjanov: i would prepare a change, just to be ready 21:58:50 SergeyLukjanov: and harass lauren to get early answers 21:58:56 :) 21:59:00 :) 21:59:02 PSA: we shall open the design summit session suggestion site on Friday or Monday 21:59:16 we should receive results of names validation at the end of this week + 1w to choose the new name + 1w to rename 21:59:34 #info Design summit session suggestion site shall open on Friday or Monday 21:59:37 ttx: that seems early 21:59:49 dhellman_: we usually do it around FF 22:00:12 ~ 2 months before summit 22:00:15 can we wait until Thursday? 22:00:20 ttx: ok 22:00:22 ttx, so, looks like we theoretically able to do renaming before the first rc1 22:00:33 markmcclain: icehouse-3 day ? 22:00:40 yes 22:00:51 markmcclain: to reduce the distraction ? 22:01:00 markmcclain: +1 22:01:11 yeah… I'd be ok with after Thursday too 22:01:18 I guess I could procrastinate and make that happen 22:01:27 on those good words... 22:01:29 ttx: don't push yourself too hard 22:01:32 #endmeeting