14:00:38 #startmeeting nfv 14:00:39 hi 14:00:39 Meeting started Wed Jun 25 14:00:38 2014 UTC and is due to finish in 60 minutes. The chair is russellb. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:40 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:43 The meeting name has been set to 'nfv' 14:00:44 hi 14:00:46 o/ 14:00:53 bonour 14:01:07 #chair sgordon 14:01:07 bonjour too (sorry for my bad french) 14:01:07 Current chairs: russellb sgordon 14:01:09 #chair nijaba 14:01:10 Current chairs: nijaba russellb sgordon 14:01:16 so a couple more people can do meetbot commands 14:01:20 hi 14:01:24 #link https://etherpad.openstack.org/p/nfv-meeting-agenda 14:01:30 hi 14:01:36 #topic review actions 14:01:37 needs more shades of pink/purple 14:01:44 sgordon: ha, i know .. 14:01:57 so bauzas appears to have made some progress on the nfv dash 14:02:06 #link https://github.com/sbauza/gerrit-dashboard-nfv 14:02:07 yey 14:02:08 yeah want to share what you have? 14:02:13 #link http://bit.ly/1iFdldx 14:02:13 sure 14:02:31 awesome progress :) 14:02:37 so, I just wrote a quick project for gathering blueprints from the wikipage 14:02:45 that's going to be very helpful to me when i go to review 14:02:48 asking launchpad to get the gerrit urls 14:03:01 bauzas: awsome! 14:03:02 and then doing some fancy stuff around it 14:03:16 +1 14:03:44 so i suppose the next step is some hosted instance of the script that keeps it updated + a redirect? 14:03:48 so, another feature is nice to have, which is the shortening service to request 14:03:49 so the main thing now is making it run on a regular basis and update a tiny url right? 14:03:59 and host it indeed 14:04:09 sgordon: russellb: +1 14:04:09 v. handy given today is the nova review day though 14:04:14 have a place to put it? 14:04:23 frankly not 14:04:26 ok 14:04:31 i can give you a small vm on rackspace if you want 14:04:39 would be awesome 14:04:41 OK 14:04:55 #action russellb to spin up a cloud VM for bauzas to host NFV dashboard script 14:05:11 anyone can get the project and test it 14:05:12 bauzas: i'll get with you about this outside of the meeting 14:05:15 bugs are welcome 14:05:22 russellb: sure 14:05:29 a few people drifting in late, here is the output we are discussing: http://bit.ly/1iFdldx 14:05:40 and it's awesome :) 14:05:49 bauzas: where is the code? 14:05:56 i havent dug into it enough to comment on the arrangement of the dash, but looks pretty good to me 14:06:05 nijaba, link to github up there somewhere ^^ 14:06:08 nijaba: https://github.com/sbauza/gerrit-dashboard-nfv 14:06:08 https://github.com/sbauza/gerrit-dashboard-nfv 14:06:12 +1 14:06:20 sorry, missed it 14:06:25 thnaks 14:06:25 no pb 14:06:28 bauzas: https://xkcd.com/208/ ? 14:06:29 :) 14:06:40 that basically the summary of the code? heh 14:06:46 except hopefully not perl. 14:06:56 russellb: ^^ 14:07:09 Python, of course :) 14:07:12 ok 14:07:14 next item! 14:07:16 russellb: and some DOM parser :) 14:07:19 so that is probably that covered for now? 14:07:31 cloudon posted another use case to the ML 14:07:36 the next action from last week was cloudon to provide a use case 14:07:36 yes 14:07:41 #link http://lists.openstack.org/pipermail/openstack-dev/2014-June/038556.html 14:07:45 Any feedback? 14:08:24 cloudon, just looking at the HA requirements 14:08:30 seems to be the main gap? 14:08:31 so the affinity requirement 14:08:36 i think that's already covered? 14:08:49 you can boot VMs into an (anti-)affinity group 14:08:52 sort of 14:08:57 this looks like having a second layer 14:09:03 where you would have two groups with affinity 14:09:10 but then placement of those groups with anti-affinity 14:09:14 ohhh.. 14:09:16 so all VMs from group A on one host 14:09:22 and all VMs from group B on another 14:09:27 but anti-affinity between A and B groups 14:09:28 and have that be enforced by the scheduler 14:09:35 right. 14:09:38 Not quite - gropus wouldn't necessarily have affinitty within them 14:09:49 cloudon, hmm ok 14:09:52 i see 14:09:54 Key thing is to be able to say a single host failure won't knock out more than x% of my VMs 14:09:54 cloudon, so just not allowed to overlap? 14:09:58 but at least anti-affinity between 2 groups ... 14:10:18 assume it's not jsut % 14:10:19 Don't want to go as far as saying all VMs must be on different hosts - limits scale for one thing... 14:10:25 right 14:10:31 but i understand now 14:10:37 that's good input 14:10:43 Think it's a fairly geenric req for N+k pool architectures 14:10:50 cloudon, right - so the soft anti-affinity proposal helps here i think 14:10:50 cloudon: Do you include any latency considerations for this application? 14:10:51 but you have within one "tier" the need to anti affinity? 14:10:56 * russellb also got sidetracked by project clearwater ... heh 14:10:59 cloudon, doesnt handle the buckets approach though 14:11:08 interesting stuff (i like voip stuff, too) 14:11:24 Latency not vital so long as not silly 14:11:39 cloudon: I think you can make use of aggregates isolation with tenants 14:12:01 V open to other approahces, particularly if can be done with existing function 14:12:08 should the scheduler understand the group anti afinity semantic, or should it just be possible to say for each VM start that it should not be on same host as this list of VM? 14:12:10 cloudon: for control plane application you mean, right? 14:12:29 russelb: yes, for control plane; clearly not for data plane... 14:12:33 cloudon: I think the CPU pinning/NUMA type work helps to ensure the latency is predictable 14:12:33 right :) 14:12:46 cloudon: not sure how we'd implement, but it's definitely a gap 14:12:58 IMHO, what we call group anti-affinity is managed thru aggregates, ie not spin that VM on that aggregate 14:13:07 adrian-hoban, right - it's not QoS as such but optimizing everything in the infrastructure to provide deterministic performance 14:13:18 bauzas: makes sense 14:13:27 We had some thoughts on a blueprint but were waiting for scheduler re-factoring... 14:13:44 yeah that is a good use case 14:14:09 when do we expect scheduler re-factoring to be completed 14:14:14 that's true, could do something with aggregates 14:14:16 sgordon: Exactly, so even if scale out is working well, we typically need deterministic behaviours. 14:14:20 a bit more static than i'd like though 14:14:43 well, I was saying that because notion of host grouping is the aggregates 14:14:45 on a related note, I shared our wiki page in the nova meeting last week, seemed to get a good response 14:14:57 bauzas, re. smazziotta's question? ^^ 14:14:59 devs really appreciated the organized info -- overview, usecases, and list of work 14:15:09 so this stuff is helping get stuff done 14:15:11 well, good question, no clear response there 14:15:16 the sooner is the better 14:15:38 I mean, we have some blueprints in progress, we expect to do good progress by end of Juno 14:15:57 Any suggestions for good nova folk to discuss this particular HA use case with? 14:16:00 but I'm unsure if we can just spin out Gantt by beginning of K 14:16:09 cloudon: bauzas :) 14:16:15 cool 14:16:22 what do people think about server groups, could be considered for anti-affinity support - needs work 14:16:33 ian_ott: yeah, works for some basic cases 14:16:36 we have a lot of BP that have a dependency to scheduler re-factoring completed. What is NFV sub-team recommendation ? we wait for refactoring or do something else ? 14:16:50 #topic blueprints 14:16:56 we're moving to that topic anyway 14:16:58 ian_ott, the problem is there is a significant jump required to get to what people what on top of the basic functionality 14:16:59 smazziotta: that's a good question 14:17:09 IMO, the breakout has to come first 14:17:17 ian_ott, which is to be able to change the group membership on a running vm and have the policy apply straight away 14:17:18 smazziotta: reviews on scheduler breakout is worth to do the :) 14:17:19 so, asking how you can help speed it up is the best next step 14:17:20 then 14:17:22 ian_ott, using migration etc 14:17:33 bauzas <-- best contact to the status of that work 14:17:43 ian_ott, also some concerns about the way the API is exposed 14:18:02 scheduler team is doing weekly meetings each Tues @3pm UTC 14:18:16 (jay started a thread on this a while back but we never really got to it in the relevant summit sessions) 14:18:44 if we consider here that sched breakout is high priority (which I agrees), then I would love seeing people helping us reviewing patches :) 14:18:56 yeah, that's my opinion at least 14:19:00 or contributing if people are willing :) 14:19:00 it's not the sexy feature work 14:19:06 but a necessary pre-requisite 14:19:18 so other blueprint news ... 14:19:22 some specs have been approved! 14:19:26 SR-IOV for Nova was approved 14:19:34 a couple of the NUMA related blueprints were approved 14:19:47 \o/ 14:19:52 today is also a nova spec review day, so should be a lot of movement on others today 14:20:01 hop in #openstack-nova if you'd like to participate / discuss 14:20:31 some organization on this etherpad ... https://etherpad.openstack.org/p/nova-juno-spec-priorities 14:20:37 i added links to NFV stuff on there 14:20:44 bauzas: let's discuss off-line 14:20:45 might break that out in more detail 14:21:19 any specific blueprints people here would like to discuss? 14:21:20 russellb: oh thanks for the link, totally missed it 14:21:35 really my priority is to dive into more of the nova ones and review ... 14:21:47 ah yes that reminds me 14:21:50 if you look at the NFV dashboard, most open specs have been reviewed actually 14:21:53 and need revision 14:22:07 there are some proposed dates to be aware of in Nova 14:22:16 Jun 25 (-10): Spec review day - TODAY! (https://etherpad.openstack.org/p/nova-juno-spec-priorities) 14:22:16 Jul 3 (-9): Spec proposal freeze 14:22:16 Jul 10 (-8): Spec approval freeze 14:22:16 Jul 24 (-6): Juno-2 14:22:43 have any of the other relevant proposals started laying out similar dates? 14:22:48 *projects 14:22:53 smazziotta: sure 14:23:00 #link http://lists.openstack.org/pipermail/openstack-dev/2014-June/038475.html 14:23:04 smazziotta: (about discussing off-meeting) 14:23:14 russellb, that's a good point 14:23:17 sgordon: i would expect similar dates for the other projects 14:23:22 there's an attempt to coordinate on dates like this 14:23:24 russellb, maybe we should go through some of those if the owners are here 14:23:27 was discussed last night in cross project meeting 14:23:31 russellb, and see if they need assistance revising 14:23:35 sgordon: sure 14:24:23 where to start is the question... 14:24:33 #link https://review.openstack.org/97716 14:24:39 2 interfaces, 1 net 14:24:46 sgordon: will let you drive the list :) 14:24:54 looks like ijw is not here today unfortunately 14:25:06 yeah, but at least got some review from john 14:25:08 i picked this one as it was one of the 3-4 on that top list 14:25:10 and he seems good with it in general 14:25:13 right 14:25:19 just needs a new iteration 14:25:23 yep 14:25:28 i see ijw responded to the comments .. 14:25:51 yes 14:26:04 he has a couple of others that are in similar state, i think we can skip those 14:26:13 ok 14:26:18 really, most are in that state ... 14:26:25 let's talk about the -2s... 14:26:28 so in general, if you own a spec, check the feedback and iterate :) 14:26:30 sgordon: ah, good idea 14:26:50 * bauzas so loves iterating on -1s... :) 14:26:56 so https://review.openstack.org/#/c/87978/ 14:27:01 heh, that's how we work 14:27:14 nic state aware scheduling, there was an attempt at reworking this based on the initial rejection 14:27:19 which is still the -2 on it 14:27:35 bauzas, looks like you had some comments/questions about it 14:27:41 since it was last uploaded 14:27:59 no updates from the author in response though 14:28:04 sgordon: not sure the update addresses the fundamental rejection though 14:28:05 indeed 14:28:28 alanm doesn't appear to be here, not sure what balazs nick is 14:28:43 wasn't the rejection ... "we don't want this in nova at all" ? 14:29:01 Not sure I understand why up/down state is not appropriate for Nova 14:29:04 sgordon: no clear disagreement with my review, just minor details to explain more 14:29:24 ...up/down NIC state... 14:29:25 to be clear, understand the importance of the use case, just think it should be handled outside of nova by your system monitoring tool of choice (nagios or whatever) 14:29:32 dansmith: around? 14:29:32 adrian-hoban, the suggestion on mailing list was that this is the domain of existing monitoring tools 14:29:44 russellb: yep 14:29:52 dansmith: discussing https://review.openstack.org/#/c/87978/3/specs/juno/nic-state-aware-scheduling.rst 14:29:59 adrian-hoban, i personally am not convinced that openstack couldnt/shouldnt handle this but it may not necessarily be *within* nova 14:30:03 so that's a little unclear 14:30:16 the other thing is there are differing preferences on what to actually do if the NIC is down 14:30:22 sgordon: the ML suggestion you refer to is my preference 14:30:30 disable the host, or just remove ports from the pool for passthrough 14:30:35 depending on which NIC it is of course 14:30:43 IMHO, if the monitoring is out of nova, 14:30:52 then it's easier for us to allow the deployer to do what they want if the nic link goes down 14:30:55 dansmith: +1 14:30:57 right, and having it be logic outside of nova gives you that flexibility, would rather not build all this policy into nova 14:31:01 from a libvirt perspective there are patches posed upstream in libvirt to at least expose this information there 14:31:06 but you still need to collect and act on it 14:31:15 if we provide tools to extract it from the network pool, then you can have the response script do what you want 14:31:18 or just report it 14:31:48 right 14:31:53 or disable the host entirely if you want 14:31:55 or whatever. 14:31:57 right 14:32:00 or make it blue 14:32:10 or go ahead and STONITH that sucker 14:32:18 I can see us reporting the link state in some host details blob or something 14:32:26 if we want that to avoid the need for a nagios infra, 14:32:29 dansmith, so that may be the question i have actually 14:32:37 but in reality, something like nagios is required anyway for a real deployment 14:32:41 dansmith: that's what the spec is proposing now 14:32:44 dansmith, because i dont think there currently is a plan to easily be able to remove them from the pool 14:32:50 russellb: reporting it but not doing anything? 14:32:56 let me check ... 14:33:03 I haven't read the updated one yet 14:33:10 doing something 14:33:14 scheduiler filter 14:33:32 so, report it via servicegroup junk 14:33:34 I don't really think we should even report it 14:33:37 right 14:33:48 i think we're crossing into system monitoring territory 14:33:49 but it's less offensive than encoding policy based on the status to me 14:33:52 yeah 14:34:03 nagios has flapping detection and all kinds of stuff 14:34:12 yeah.. 14:34:16 that are necessary for hysteresis that we don't want to have to implement, IMHO 14:34:18 so i think i'm going to add a -2 here as well 14:34:25 and if we don't, we'll suck compared to it 14:34:53 sgordon: not sure that's the outcome you wanted, heh ... 14:35:13 clear direction forward should be top priority :) 14:35:20 well, I'm thinking about this usecase as a possible good fit for a big Scheduler like Gantt 14:35:25 but not Nova 14:35:32 bauzas: maybe ... 14:35:39 I mean, notifications have to be done maybe thru Nagios indeed 14:35:41 russellb, not exactly but i really said lets look at the -2s 14:35:47 russellb, and then that was the only one 14:36:20 russellb, i did try to bump the thread about this a week or two ago to discuss nagios- based approaches to try and flesh out what we would need from the API but didnt get much interest 14:36:57 (again mainly talking about managing the pool of devices for passthrough) 14:38:14 yeah 14:38:17 that's all config based 14:38:19 so kind of messy 14:38:42 but we need an API for it anyway 14:38:50 this would be another use case for the API 14:38:56 hence Gantt :) 14:38:58 not just initial provisioning, but ongoing management 14:39:18 let's move on, can discuss more blueprints in open discussion if someone wants to 14:39:23 #topic Paris mud-cycle spring next week 14:39:38 looks like there is a small group of people interested in NFV attending this meetup 14:39:45 mud cycle? 14:39:50 sounds cool 14:39:51 lol 14:39:53 yeah! 14:39:57 #undo 14:39:58 Removing item from minutes: 14:40:05 #topic Paris mid-cycle spring next week 14:40:13 spring ? :) 14:40:15 I’m planning to be in Paris for this and it would be neat to talk with more OpenStack/NFV hackers there :-) 14:40:15 #link https://wiki.openstack.org/wiki/Sprints/ParisJuno2014 14:40:25 Looks like 5 people planning to go so far (not all confirmed) 14:40:29 bauzas: dangit 14:40:30 SPRINT 14:40:32 :) 14:40:48 #topic open discussion 14:40:51 anything else today? 14:41:01 as usual, please help review specs and code 14:41:08 and if you own active specs or code, please iterate based on feedback 14:41:14 most specs are waiting for updates from submitter right now 14:41:18 right 14:41:23 v. important to get that done asap 14:41:32 as we are quickly approaching crunch time for this cycle 14:41:43 as far as approval goes anyway 14:41:47 right 14:41:56 if specs aren't approved in the next week or two, they will be deferred to K 14:42:02 anything worthwhile to communicate to ETSI NFV folks? 14:42:08 Do we need more example use cases, or are the detailed control & data plane app examples we know have sufficiently representative? 14:42:15 Looks like we have two proposals for userspace vhost at the moment: vhost-user (the new QEMU feature) and also DPDK’s own solution. Got to haromnize these someow 14:42:24 MichaelB, there was a request previously for access to drafts of the ETSI gap analysis work 14:42:37 MichaelB, particularly around OpenStack gaps 14:42:49 MichaelB, obviously some of the members are here and filtering info through 14:42:50 gaps analysis is incomplet, not ready for publishing yet in ETSI 14:42:59 MichaelB, but would be good for everyone to be on the same page 14:43:05 MichaelB, not even via the draft documents page? 14:43:07 we are now in consistency review, followed by WG approval 14:43:18 then working on tightening the gap analysis 14:43:38 lukego, do you have the links handy 14:43:44 lukego, i think i only saw one of them 14:43:44 I have renamed VIF_SNABB to VIF_VHOSTUSER name. Hope it will suit everybody for userspace vhost going forward 14:43:53 so far, what we have is WG-level, partial/incomplete analysis 14:43:57 Multiple ETSI-NFV working groups will be looking at OpenStack from different perspectives. 14:44:13 there is something against OpenStack...but rudimentary 14:44:14 sgordon: QEMU vhost-user VIF (previously VIF_SNABB) is here: https://review.openstack.org/#/c/96138/ 14:44:33 #link https://review.openstack.org/#/c/96138/ 14:44:46 I can check with the MANO chairs if we want to share this in this stage, or wait till it is in better shape 14:44:59 MichaelB, that seemed to be the main request i am aware of 14:45:10 MichaelB, concern is that we're duplicating each others efforts on that front somewhat 14:45:51 sgordon: The DPDK one is here: https://blueprints.launchpad.net/nova/+spec/libvirt-ovs-use-usvhost 14:45:53 sgordon - what do u mean? Is the gap analysis happneing somewhere else other than ETSI NFV ? 14:45:59 #link https://blueprints.launchpad.net/nova/+spec/libvirt-ovs-use-usvhost 14:46:09 noted #link for future :) 14:46:27 MichaelB, much of the discussion in this group has been around identifying NFV use cases and the gaps they expose 14:46:41 MichaelB, driven in part by the existing ETSI NFV publications 14:46:52 My perspective is that vhost-user is the standard feature for the future (now upstream in QEMU) that everybody will use (?). but the DPDK-OVS people may want to support the other one anyway in the short term? need to get in sync 14:46:56 that is fine. I don't think that's a dupication with the gap analysis I am talking about 14:47:09 we should continue to do what u started HERE 14:47:12 From what I have seen in drafts, there are new items to be added to the list we are tracking 14:47:37 adrian-hoban, right that is my expectation 14:47:46 adrian-hoban ... yes, I think we should bring into here any deltas 14:47:55 lukego: The concern related to support for folks that are using earlier versions of qemu 14:48:10 adrian-hoban, obviously the earlier we get and iterate on deltas the sooner we can do something about them 14:48:20 adrian-hoban, but the plate for juno is getting full anyway :) 14:48:34 adrian-hoban: That makes sense. Question in my mind is how important that is? (I noticed that other OVS-DPDK related features, like security groups, were already deferred until the K-cycle — so do we still need this VIF and in Juno?) 14:50:25 adrian-hoban: Should we treat the two VIFs separately or would there be a practical way to combine them 14:52:26 adrian-hoban: more to the point — I want to push forward and get VIF_VHOSTUSER into Juno, because it will enable Deutsche Telekom to avoid deploying a fork. Does this effort step on you guys’ toes somehow or is this a positive thing for you too (for future use)? 14:52:39 lukego: It would be preferred to have it in Juno so that we can help support some time-to-market concerns of waiting on qemu 2.1 support to get into the distros etc... 14:52:43 lukego: we want to use your work to support vhost user for ovs 14:53:00 lukego: I think that is positive too 14:53:02 lukego: positive think 14:53:11 Go for it 14:53:24 ok good. I am happy to present both blueprints as best we can and ideally get both of them in. I don’t immediately see a way to combine them, but obviously that would be better. 14:53:35 Need to figure out how to support older versions of qemu 14:53:55 i think the 2 could reference each other 14:54:17 sounds like if there' separate VIFs, they should stay as separate specs 14:54:56 btw I have just run through and renamed VIF_SNABB to VIF_VHOSTUSER everywhere (based on sensible feedback) — please let me know if I missed some references somewhere 14:55:01 russellb: cool 14:55:42 will try to take a look in more detail though 14:55:53 adrian-hoban, i think there is a wider discussion there that we've never really nailed down 14:56:07 wrt support for a wide range of libvirt/qemu versions and what is actually tested in the gate 14:56:26 lukego: We will contribute some updates to extend your work to support the DPDK use case 14:56:47 for qemu 2.1 14:56:51 adrian-hoban: any chance you guys fancy a visit to Paris next week btw? :-) 14:57:06 adrian-hoban: (you’re based in the UK?) 14:57:30 adrian-hoban: awesome 14:57:34 Close, I'm in Ireland 14:58:00 Don't think I can make it 14:58:52 I’d love to understand more about how OVS-DPDK relates to OVS plugin and OpenDaylight and so on. (Like: do you plan to implement Neutron APIs directly in Python code in OpenStack or always use an external controller?) if there’s a link please send :) 14:59:35 adrian-hoban: I’m guessing external SDN controllers will be the big winners of the OVS-DPDK work? 14:59:46 (but I digress.) 14:59:48 looks like we're out of time 14:59:56 discussion can continue in #openstack-nfv if you'd like 15:00:00 thanks everyone! 15:00:01 Thanks for hosting russellb! 15:00:07 #endmeeting