21:00:36 #startmeeting 21:00:37 Meeting started Thu Oct 13 21:00:36 2011 UTC. The chair is vishy. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:38 Useful Commands: #action #agreed #help #info #idea #link #topic. 21:00:49 #topic Blueprint list 21:01:14 #link http://etherpad.openstack.org/essex 21:01:33 i've gotten most of the blueprints added there 21:01:50 please everyone look and see if there is anything I missed 21:02:30 looks good Vish. 21:02:38 I think there is a lot of work here for Essex 21:03:17 i probably should create a XenServer ConfigDrive BP 21:03:33 comstud: pls do 21:03:37 no feature parity there yet 21:03:47 vishy: do you want to assign any of these today? 21:03:50 note that the admin apis has nothing yet 21:04:19 bcwaldon: first I want to make sure that we haven't missed any major ones 21:04:28 #info you missed the two blueprints I created yesterdat for VM State Machine and Image Cache Management 21:04:41 PhilDay: can you cpy them in pls 21:04:55 #action comstud to create xenserver config drive BP 21:05:13 comstud: I think there is an old one for that 21:05:19 maybe 21:05:27 might have been marked done with the libvirt part 21:05:29 I'll take a look 21:06:12 yeah 21:06:18 #link https://blueprints.launchpad.net/nova/+spec/configuration-drive 21:06:20 says implemented 21:06:33 comstud: gotcha 21:06:51 i'm supposed to create the story for our sprints anyway :) 21:06:54 i'm slacking 21:07:08 can someone add a blueprint for vnc console cleanup? Basically just combining the two vnc paths somehow 21:07:09 did the research, haven't written it yet 21:07:10 <_0x44> It was implemented with the understanding/expectation that someone with access to XenServer who cared about config-drive would ensure that it worked. 21:07:22 <_0x44> I think the only thing needed is to attach it 21:07:26 <_0x44> _think_ 21:07:47 _0x44: I think there was a bit more to do. Create the VDIs and such. 21:07:55 there's more than that 21:08:05 but yeah 21:09:00 vishy - i can add one for vnc 21:09:08 sleepsonthefloor: thx 21:09:47 vishy - added the VM state machine under ""Workflow State Machine" but not sure if that's quite the right place for it 21:10:18 PhilDay: that is fine 21:10:34 so we are still missing a few blueprints related to volumes 21:10:56 other than that i think we have everything in 21:12:13 well we are missing a few in the security session as well 21:13:09 <_0x44> vishy: Are the BP's bengrue worked on in Diablo already listed? 21:14:19 vishy: are you going to prioritize these from your perspective as PTL? 21:14:20 which are those? 21:14:30 pvo: that is what I want input on 21:14:36 ok 21:14:43 pvo: I need to know which ones are required by teams 21:14:48 <_0x44> vishy: Bringing bengrue in here to answer that. 21:15:06 there are definitely some missing 21:15:31 is westmaas here? 21:15:50 westmaaaaaaaaaaaaaaaaaaasssssssss 21:15:52 <_0x44> bengrue: What are the blueprints you were working on in Diablo around volumes? 21:16:10 I was just chatting with him. 21:16:19 so we have a few very big ones 21:16:31 and we need to decide if we want to tackle them in diablo 21:16:34 * essex 21:16:34 Sec. That was a bit ago. 21:16:38 or push to f 21:16:55 vishy: what do you think are the biggest? 21:16:58 top 5 21:17:03 so the big ones as I see it are: 21:17:04 1) DB Cleanup and testing 21:17:07 Do we have an overall goal for Essex? 21:17:16 2) Zone Aggregation 21:17:23 stablility, performance and scalability 21:17:27 3) Workflow State Machine 21:17:36 pvo: perfect 21:17:43 4) Secure Databases 21:17:49 pvo: is that all? :) 21:18:00 bhall_: over features, I would think so 21:18:06 I would prefer those 3, over any new features 21:18:13 agreed 21:18:36 vishy: under Workflow state machine do you mean "orchestration, jobs, retries, etc..." 21:18:40 the problem is i think all of those are required 21:18:56 #agree with pvo 21:19:00 vladimir3p_: I'm referring to sandy's proposal of actually having workflow management 21:19:10 vishy: yep 21:19:16 I don't thing we can hit all 4 of those 21:19:28 and they are listed in priority as i see them 21:19:40 #topic large blueprints 21:20:11 #info Largest blueprints: db cleanup and testing, zone aggregation, workflow state machine, secure databases 21:20:16 vishy: we have 4 milestones. One goal per milestone? 21:20:40 some of them can be worked concurrently 21:20:47 but I think all 4 of those require a team 21:21:00 i don't think one person can handle any of them 21:21:08 pvo: I'd be worried that we'd still have big changes comming through late in Essex if we do one per milestone 21:21:13 I'd like to push secure db / queue 21:21:25 #info upgrades is also large 21:21:33 _0x44: the blueprint I was working on was https://blueprints.launchpad.net/nova/+spec/volume-cleanup 21:21:43 bengure: that is implemented and merged 21:21:47 * bengrue 21:22:04 <_0x44> I wasn't sure if it was merged 21:22:14 Yeah, Vlad I think drove it home? 21:22:17 Upgrades is a v important now there are systems deployed 21:22:25 <_0x44> Ok, sorry for the confusion. 21:22:29 np. 21:22:47 Yes we need a team for all 5 of those, but if we don't hit secure dbs in essex does that kill anyone 21:22:57 PhilDay: can we push that one from your perspective? 21:23:12 vishy: secure db + zones might need to coordinate 21:23:24 yep 21:23:25 PhilDay: That one in particular requires massive changes 21:23:45 anotherjesse: I'd like to tackle db cleanup + zones first, i think it makes secure db easier 21:24:00 working on it concurrently will be a huge issue 21:24:14 vishy: also we might want to think of using a smaller component like the volume component for testing out the secure db 21:24:22 vishy: ++ 21:24:23 Agreed, and the session in Boston showed there are more questions than answers - but if we don't hit it soon it'll just get worse 21:24:31 #idea create subteams for the major work sections 21:24:43 vishy: teams mean launchpad teams? 21:25:19 anotherjesse: I don't know if it needs to be that formal 21:25:41 I think its just working groups that Vish keeps tabs on 21:25:49 zone aggregation, i'd like to see anotherjesse, comstud, sandywalsh collaborate 21:26:19 don't know if there is anyone else who should be involved there 21:26:50 yeah, myself and sandy are probably a given 21:26:56 We have some views, although we're not heavy into zones - yet 21:27:08 vishy: wiki page of major themes for essex with people on it (so people who aren't on it originally can join in) 21:27:24 does anyone want to tackle the db cleanup and testing stuff 21:27:28 anotherjesse: definitely 21:27:41 vishy: blamar would be a good one there, I'm also interested 21:27:58 bcwaldon: funny, I was just going to nominate both of you 21:27:59 how about upgrades? 21:28:01 :) 21:28:01 sleepsonthefloor: do you want to help on db, statemachine or ? 21:28:08 bcwaldon: thanks there buddy 21:28:11 that one is a lot of theory and prototyping 21:28:14 blamar: there you are! 21:28:14 blamar: np :) 21:28:16 <_0x44> vishy: Am interested in db=-cleanup at piston 21:28:25 once again, we need to have a group to define what upgrades are all about 21:28:33 I'm adding this to the etherpad 21:28:42 We are definitely interested 21:28:42 feel free to add yourself to a section 21:29:16 rjh: who are you with again? 21:29:26 I'm going to throw Dietz on the upgrades. He and I have been talking about it and whiteboarding 21:29:31 _0x44: are you guys interested in building a db backend that isn't sqlalchemy? 21:29:36 <_0x44> anotherjesse: Yes. 21:29:39 _0x44: zk? 21:29:52 <_0x44> anotherjesse: Likely 21:30:14 rjh=Ray Hookway (HP) 21:30:31 _0x44: I think it is important that whoever leads the db cleanup does in order to be able to ship an alternative backend 21:30:35 thanks PhilDay, was wondering ;) 21:30:39 <_0x44> pvo: 21:30:55 At least I think its Ray :-) 21:31:14 It is 21:31:16 if not, he was just volunteered ;) 21:31:16 _0x44: 21:31:22 <_0x44> pvo/vishy: I'd like to add novas0x2a to upgrades also 21:31:24 oh hai 21:31:40 #topic subteams for major sections 21:31:50 <_0x44> Sorry, am on a terrible network right now 21:31:57 _0x44: something didn't convert there 21:32:02 0x44: you still in boston? 21:32:03 Citrix would be more than happy to contribute on upgrades as well. I think the name is johngarbutt but I'm not totally sure 21:32:04 :) 21:32:06 <_0x44> pvo: novas0x2a is mike lundy 21:32:11 #info breaking out subteams for particular sections of the code 21:32:16 gotcha 21:32:23 <_0x44> pvo: The nordic bearded guy at Pisotn 21:32:29 ok we have one more subteam 21:32:34 feature parity 21:32:47 vishy - you mean hypervisors? 21:32:49 <_0x44> I have to run now 21:32:50 or apis? 21:32:59 anotherjesse: hypervisors 21:33:07 all hypervisors? 21:33:23 we focused just on KVM and XenServer at the summit 21:33:27 is that still the majority focus? 21:33:33 pvo: probably some things will result in documenting "not supported" - for things that don't make sense 21:33:35 no just the main two 21:33:39 ok 21:33:45 i don't want to get crazy 21:33:49 just wanted to make sure 21:33:57 so quick once over from the interested parties 21:34:17 vishy: does this include the fact that xenserver at rax vs. kvm openstack is very different from user perspective (image formats, userdata/metdata, how to auth to servers, ...) 21:34:34 anotherjesse: it doesn't 21:34:47 How about a security team to deal with message signing and db access? 21:34:53 anotherjesse: we have no blueprints for those things, but they sound important 21:35:14 rjh: yes security team is good. I'm concerned about them being able to make much progress 21:35:16 #info jesse write up a blueprint on ux for openstack 21:35:26 but they can at least start planning 21:35:51 pvo, is there someone on your team who is more focused on feature parity? 21:36:23 vishy: I volunteer pvo and sleepsonthefloor 21:36:29 :) 21:36:34 more focused? 21:36:54 it would be nice if someone from ubuntu (kvm/lxc) & citrix (xs) could help 21:37:08 #info db and rabbit security should be planned but the aren't targeted to be complete by essex 21:37:08 yeah, I'm definitely interested in parity 21:37:23 more or less the whole Citrix team is focused on parity 21:37:24 anotherjesse: agreed, I wonder if we could add salv-orlando 21:37:53 I'd say ewanmellor would be happy to be on this team as well 21:38:17 it would be nice if someone else who has thought about nova-network and security groups with xen could be on it 21:38:31 so does anything else seem to dangerous to be in scope for essex? 21:39:07 api 2.0 - but before zones is done … that seems scary to talk about 21:39:19 anotherjesse: agree 21:39:22 yeah 21:39:26 #agree 21:39:26 bcwaldon disappeared 21:39:49 I don't think we need 2.0 for essex 21:39:49 #info wait on zones to be completed before nova api 2.0 21:39:52 vishy: rbac is medium sized - mostly busy work 21:40:08 anotherjesse: good point we need to give special attention to that one 21:40:17 (arguing over what the proper verbs are) 21:40:24 anotherjesse: do you think we should have a team for that. Should rcb take lead on it? 21:40:31 maybe todd? 21:40:32 xtoddx: 21:40:42 he needs that for nasa 21:40:54 i don't mind heading that one up 21:41:06 vishy: having you on volumes would probably be better 21:41:11 vishy: re: the top5, are there BPs for each of those? 21:41:13 since you have done a lot of that work already 21:41:34 anotherjesse: I'm trying to offload volumes onto third parties 21:41:48 anotherjesse: so hopefully i can move into more of a supporting role there 21:42:36 There is a blueprint for upgrades, but it needs to be made more specific 21:43:00 rjh: i think that one needs to be broken out into specific blueprints 21:43:13 Secure databases is mentioned in the nova-security-updates blueprint 21:43:20 #action upgrades team to meet and come up with specific action items and blueprints 21:43:30 I agree - that goes for both of them 21:44:12 #info volumes team met today and started discussing the needed pieces 21:44:26 #info we don't have specific blueprints for the parts yet 21:45:06 blamar: do you want to lead the db cleanup team? 21:45:37 vishy: nothing against blamar but I really think having the lead caring about an alternative backend is important 21:45:43 caring = require 21:45:48 blamar: there is one blueprint so far, which is to make the db layer return dictionaries 21:45:58 anotherjesse: so you think 0x44 should head it up? 21:46:04 vishy: apologies, was on the phone..however not currently interested in leading a team; busy life 21:46:06 does anyone here care about zookeeper? 21:46:24 vishy: supposedly _0x44 does 21:46:40 can we pencil it in - with a note that we need to verify 21:46:53 #action vishy to verify with _0x44 that he can lead the db cleanup efforts 21:47:16 vishy, anotherjesse: I'm an advocate for getting a couchdb backend (as per my email to the mailing list a while back on this subject) and I've used ZK in the past 21:47:23 anotherjesse: are you heading up the zone scaling coordination? 21:47:32 so I'd love to assist whomever leads 21:47:35 blamar, so you're volunteering again :) 21:47:53 negatory, I'll be a fine assistant 21:47:54 vishy: i am from our team - if sandywalsh or comstud is a better lead overall, I have no objections 21:48:00 vishy: comstud/sandy can as well. 21:48:06 comstud: ? 21:48:07 Both have a pretty firm grasp of the issue 21:48:11 hmm 21:48:29 not sure i'm the best person 21:48:34 we also have that big nasty Recovery/Workflow section 21:48:43 esp. if i'll be tied up with zones 21:48:47 which sandy seems to be very excited about 21:48:47 comstud: how about you - since you guys have to have zones to ship? 21:48:53 I'm up for zones or orchestration ... two hot topics for me 21:48:55 oh 21:48:59 are we talking about zones? 21:49:00 comstud: we're talking about zones. : ) 21:49:01 comstud: zone scaling is what we're talking about 21:49:03 lol 21:49:10 i was chatting with sandy and not paying attention 21:49:11 :) 21:49:14 sure 21:49:20 #info comstud to lead zones with help from sandy and anotherjesse 21:49:21 I can take that 21:49:35 #info sandy to lead orchestration with help from ... 21:49:56 vishy: where is donabe in relation to orchestration? 21:50:22 I assumed donabe more operations oriented, not nova internal? no? 21:50:40 dendrobates? 21:50:48 pvo, sandywalsh: It is more related to creating groups of resources i think 21:50:49 I would think there would be some of the same retrying/requeueing in tehre as well 21:50:59 where as this is more concerned with the state of individual resources 21:51:28 does anyone here care particularly about Operational Support 21:51:30 resources and the workflows around them 21:51:42 including admin apis, cli tools, etc? 21:52:01 vishy: we do... I'm repping for our ops guys today 21:52:11 vishy: we should volunteer chmouel or others from deployment team 21:52:34 sleepsonthefloor: are you able to lead the feature parity section? 21:52:48 vishy - for sure 21:53:27 salv-orlando - you interested in helping then? 21:53:33 ok, so i need someone to take the lead on Upgrades, Operations, Security 21:53:56 sleepsonthefloor: on feature parity? Yes. 21:53:59 vishy: let me get back to you on the ops part. 21:54:07 pvo: ok 21:54:14 I'm recruiting some of our ops guys to take a more active role in the open. 21:54:16 pvo: are you leading upgrades? 21:54:38 me or Dietz. 21:54:38 rjh: shall I put you on the lead for security planning? 21:54:41 vishy: I'm not sure if pvo is a good fit - as they will run an internal version - upgrades is more about shipped software? 21:55:00 ok 21:55:04 anotherjesse: good point 21:55:17 vishy: someone in canonical or citrix or others who ship software based on milestones / releases 21:55:25 crowbar? 21:55:28 its more how to do upgrades is what were thinking. procedures and best practices. 21:55:41 anotherjesse: perhaps someone from canonical will step in here 21:55:43 you're meaning make sure upgrades work from Diablo to pre-Essex? 21:55:50 diablo to essex 21:55:54 pvo: it is diablo to essex 21:56:11 essex-pre release... sure. Yes, probably not the right guy there 21:56:19 vishy: we at Citrix have definitely interest in upgrades. Unfortunately our 'upgrade' men are not here tonight 21:56:26 we won't be doing massive upgrades every 6 months 21:56:35 ok 21:56:36 pvo: right - not that you won't be able to contribute and have lots of good feedback - but when you run a service you can do 100 upgrades between diablo and essex in stages (gradual db migrations, ...) 21:56:55 guys, I need some suggestions for how these groups should keep in touch 21:56:56 anotherjesse: yep. Totally valid point 21:57:07 walkie talkies 21:57:15 pvo: you should definitely have folks on the team 21:57:18 should we start little mailing lists for all of them like we did for the volume group? 21:57:23 anotherjesse: absolutely will 21:57:30 vishy: does lp teams give you a mailing list? 21:57:50 anotherjesse: I believe it does 21:57:52 anotherjesse: yeah 21:57:58 that is how we did it for voluems 21:58:16 teams with open enrollment (or somethign else?) seems valuable 21:58:31 anotherjesse: ok 21:58:49 #action vishy to create subteams for mailing list communication 21:58:56 vishy: melange integration into nova - ? is that an essex thing? 21:59:08 #info subteams will be open enrollment 21:59:26 #info each team will have a poc so that vishy has someone to bug 22:00:11 #info subteams are responsible for creating blueprints for needed work and helping vishy target them to specific milestones 22:01:14 #action vishy to go through the registered blueprints and accept them and assign them to the subteam. 22:01:40 #info subteam leader can reassign the blueprints to specific individuals or real teams to do the actual work 22:01:53 Ok guys, I don't want to make this too heavyweight 22:02:12 But there are way too many people working on the code for me to keep tabs on all of this at once, so I appreciate the help 22:03:07 I will create open launchpad groups for all of these teams so that they can communicate 22:03:32 i think there is already an operations team, so I might just use that one. Although I specifically want dev help for tool and admin api createion 22:03:55 Any other subteam that we're missing? 22:04:28 melange? 22:04:44 (integration) 22:04:44 I think there is alread a melange team 22:04:49 ah ok 22:05:40 #topic What do we do with melange 22:05:51 lets take a minute and plan that 22:06:20 any input on melange? 22:06:36 right now nova-network doesn't use it - if we integrate it then we should update nova-network? 22:06:50 (melange=ipam) 22:06:50 anotherjesse: perhaps 22:07:01 I think it has a long ways to go before it can merge into trunk 22:07:06 anotherjesse: why not just keep it separate 22:07:12 in QuantumManager we allow users to either use traditional nova IPAM or Melange 22:07:23 anotherjesse: if the merge was an easy refactor inside was better 22:07:24 we could do the same in Flat & VlanManager 22:07:37 +1 on keeping it seprate 22:07:38 anotherjesse: just let melange support quantum 22:07:47 has anyone investigated the level of effort to make nova-net use melange? 22:07:51 vishy: +1 22:08:29 anotherjesse: no, but I'm worried about the two integration points 22:08:42 quantum is already integrating, so we save work by just using that integration 22:09:06 vishy: +1 22:09:30 salv-orlando: my goal would be to kill the nova-ipam to make things simplier - instead of making both quantum and nova-net use both 22:09:41 but I'm good with waiting - just want to make sure 22:09:45 are any of the key melange devs here? 22:10:02 anotherjesse: does quantum use both? 22:10:04 they're mostly/all in India 22:10:06 I know only Troy and Rajaram 22:10:14 vishy: it can, currently 22:10:45 (by quantum I'm asuming you mean quantummanager) 22:11:08 just to be precise, quantummanager is nova-net manager for quantum 22:11:40 salv-orlando: so there is no integration in quantum itself? 22:11:54 vishy: that's correct 22:12:00 vishy: no, we actually do it in nova-network! 22:12:27 anotherjesse: in that case it sounds like the integration is fairly easy 22:12:49 I don't see it as a major hurdle. bhall_ might have more details 22:12:58 yeah, I don't think the integration would be that bad 22:13:10 bhall_: but you think it should be separate? 22:13:11 vishy: it is a 10k line diff - just worry about it living in the edge too long and not getting attention 22:13:24 anotherjesse: does it need attention? 22:13:27 (integrating, or replacing nova's IPAM as anotherjesse suggested) 22:13:52 vishy: given that we haven't looked at it outside of the quantum use case … maybe? 22:14:00 anotherjesse: someone is going to have to spike this it sounds like 22:14:04 i'm suggesting a spike - ya 22:14:15 we were just talking about this today 22:14:34 I think they're going to try to figureo ut how to break it apart 22:14:39 unknown if it can 22:15:04 ok who can spike it? 22:15:24 I think this is Trey or Koelker 22:15:29 tr3buchet might be a good choice 22:15:29 someone on my team 22:15:41 pvo can you have the two of them look at it and come up with a plan 22:15:56 considering a) our existing ip management sucks, what is the best path forward 22:16:09 #action will coordinate a plan to get the network diff broken up 22:16:18 pvo thx 22:16:21 ok anything else 22:16:26 pvo: I feel that if we don't move to melange we will end up implementing most of the api in an os-extension 22:16:34 #topic open discussion 22:17:52 so 22:18:00 reviews 22:18:13 I've been cracking down on HACKING violations when reviewing 22:18:20 ++ 22:18:34 I dunno if this is the case or not... but it seems maybe there hasn't been a lot of attention paid to them 22:18:42 hopefully these teams will help with the reviews 22:18:50 Some of the stuff that annoys me in the code is general inconsistency 22:19:04 comstud: be strict but helpful? 22:19:07 so I'm feeling like we could crack down a little more on that 22:19:08 haha 22:19:12 yeah 22:19:19 I kinda want a termie bot again - for the simple stuff 22:19:34 there are a lot of cleanup related tasks needed that don't have blueprints 22:19:40 true 22:19:44 there's a lot of stuff that isn't covered in HACKING 22:19:48 termie bot? 22:20:10 #idea jesse suggested dedicated a couple of milestones to stabilization and cleanup 22:20:14 So it wasn't the real termie putting needs fixing for a missing linefeed on a docstring in my branch? 22:20:37 we could probably dedicate a release to it 22:22:18 I hate making people fix little stuff like this.. for fear I'll come off like an asshole. :) But.. 22:22:33 That's the main complaint I hear when someone new goes to look at the code 22:22:45 A number of style differences... and then people not being sure which way they should do something 22:22:57 comstud: +++ 22:22:58 HACKING is pretty simple to follow.. there's clear instructions there. 22:23:12 maybe we need to add a link to hacking to how to contribute 22:23:18 and the gerrit instructions 22:23:28 HACKING is good to enforce, sadly "pythonic" is much harder 22:23:38 we can't really enforce something that isn't in HACKING. should we consider adding more to it? 22:23:38 sandywalsh: Yeah, that's the thing. 22:23:39 before typing git review: check HACKING, run tests, etc. 22:24:05 or enforce something like the termiebot as we enforce pep8 22:24:10 anyone not think it is reasonable to mark a patch "plz don't merge - needs fixing" if it has consistency issues? 22:24:26 that is the point of code review, right? 22:24:33 one point of it 22:24:38 I think the question is which consistency 22:24:44 whose consistency 22:24:46 but 'consistency' right now doesn't mean a lot 22:24:48 yeah 22:24:58 what else could be the point 22:25:02 that's why I propose adding more things to HACKING 22:25:10 tr3buchet: obvious bugs that tests don't catch 22:25:24