21:03:50 <sc68cal> #startmeeting neutron_ipv6
21:03:51 <openstack> Meeting started Thu Dec  5 21:03:50 2013 UTC and is due to finish in 60 minutes.  The chair is sc68cal. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:03:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:03:54 <openstack> The meeting name has been set to 'neutron_ipv6'
21:04:36 <sc68cal> Hello everyone, I hope all are doing well
21:04:56 <shshang> Hey, Sean
21:05:08 * sc68cal pulls up agenda
21:05:08 <Randy__> hi sean
21:05:15 <aveiga> hello
21:05:24 <sc68cal> #topic review previous meeting items
21:06:10 <sc68cal> So our meeting 2 weeks ago, the first thing we talked about was how to move forward with the DHCP agent for V6
21:06:51 <sc68cal> we discussed that in depth on the brainstorm on tuesday night, and the consensus was that we're going to attempt to get all the work that everyone has done around this combined
21:07:21 <sc68cal> Randy__ and shshang posted their review, which contains the work they did to make dhcpv6 work with dnsmasq
21:07:57 <sc68cal> #link https://review.openstack.org/#/c/58186/ Review for IPv6 - Randy & shshang
21:08:14 <sc68cal> shshang: Do I have attribution right?
21:08:27 <shshang> Yup, that is correct, Sean!
21:08:32 <sc68cal> Great
21:08:50 <sc68cal> so let's go ahead and use that to transition into a recap about what we talked about tuesday
21:08:57 <sc68cal> #topic recap tuesday brainstorm
21:09:08 <sc68cal> shshang: want to summarize our discussion?
21:09:41 <shshang> Sure. So we had a brainstorming session on Tuesday
21:09:59 <shshang> We had teams from multiple placeholders, such as Comcast, IBM, HP, Cisco, and Nephos6
21:10:25 <shshang> Sean clarified several question regarding the blueprint he proposed to the community
21:10:40 <shshang> Regarding the blueprint, we focused on the short-term and mid-term goals
21:11:03 <sc68cal> #link https://blueprints.launchpad.net/neutron/+spec/ipv6-provider-nets-slaac Comcast short-term goal for IPv6
21:11:33 <shshang> In summary, for short-term goals, with the new patch in Neutron to modify the firewall rules, the system should be able to achieve the short-term goal
21:12:02 <shshang> In other words, VM should be able to learn RA from upstream router, and come up with dual-stack address
21:12:24 <shshang> W.r.t. the mid-term goal, a lot of work have been done already jointly between Comcast, IBM and HP team
21:12:38 <aveiga> something to note here is that patch applies to the hybrid VIF driver
21:13:16 <haleyb> i (hp) won't take credit for any work, just reviews :)
21:13:45 <jjmb> so what is next here on this front "Comcast short-term goal for IPv6", how do we advance this so we can deploy preliminary support  IPv6?
21:13:47 <shshang> There are several bug fixes already submitted . In addition, Randy and I did POC before, and we believe that we can merge the work we did to add support of the architecture with Neutron Router and L3 agent
21:14:15 <shshang> @aveiga, thanks for the note. Yup, that is correct
21:14:55 <sc68cal> that sounds like a good time to transition to bugs
21:15:01 <sc68cal> #topic bugs
21:15:16 <shshang> We will leverage the work Sean and IBM team already did, i.e. the framework to add dnsmasq mode from CLI to the code, and make sure dnsmasq has flexibility to achieve Sean's mid-term gold
21:15:19 <shshang> goal
21:15:50 <shshang> In addition, Sean encouraged us to create a new blueprint
21:15:59 <sc68cal> #undo
21:16:00 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x2b8ed10>
21:16:08 <shshang> make it as addition to the existing blueprint he submitted.
21:16:34 <shshang> Randy created a blueprint for that purpose
21:17:10 <sc68cal> perfect
21:17:13 <shshang> We will create at least one more blueprint as supplement to Sean's, in order to support router and l3 agent
21:17:22 <Randy__> #link https://blueprints.launchpad.net/neutron/+spec/dnsmasq-bind-into-qrouter-namespace
21:17:30 <aveiga> thanks Randy__
21:17:32 <shshang> Thanks, @Randy
21:17:43 <Randy__> np
21:18:06 <shshang> So eventually, as part of the Comcast mid-term goal, we should be able to allow user to define which mode dnsmasq will run at, and spin up dual-stack VM accordingly
21:18:17 <shshang> We didn't have time to discuss long-term goal yet.
21:18:47 <shshang> Anything I missed, Sean or Randy?
21:18:58 <aveiga> to be specific about the mid-term goal, there are 3 major ways to provision a network in ipv6
21:19:05 <aveiga> SLAAC, stateful, and stateless
21:19:23 <aveiga> it would be useful to extend neutron's network attributes to include provisioning mode
21:19:32 <aveiga> and basically set the v4 subnets to stateful
21:19:41 <sc68cal> aveiga: I believe the subnet_mode bp covers that
21:19:44 <aveiga> but allow the v6 networks t oselect one of these modes
21:19:55 <sc68cal> since it allows you to do static, ra-names, ra-stateless, etc
21:19:55 <aveiga> exactly, mind linking it?
21:20:03 <Randy__> that is our intent
21:20:15 <sc68cal> #link https://blueprints.launchpad.net/neutron/+spec/dnsmasq-mode-keyword dnsmasq v6 mode configuration
21:20:24 <Randy__> we will use the modes introduced with keywords blueprint
21:20:28 <Randy__> that is a dependency
21:20:29 <shshang> The framework Sean and IBM team defines, including a list of constants for all of these modes
21:20:40 <sc68cal> jjmb: I think the last peice for our short term is just getting the hairpin fix into nova
21:21:10 <shshang> we will use the definition of those constants for our next code submission.
21:21:43 <sc68cal> #link https://review.openstack.org/#/c/52983/ defines the modes available to use with dnsmasq
21:21:46 <shshang> We will also drop the source code, which are duplicated with Comcast and IBM teams' contribution.
21:22:21 <shshang> I believe that should be the summary of our Tuesday night's meeting
21:23:01 <Randy__> we will create another blueprint to address the external gateway of the L3 agent
21:23:05 <sc68cal> excellent - thanks for organizing that meeting, I think it was very helpful for the IBM people who are not able to make these meetings
21:23:31 <sc68cal> due to timezone differences
21:23:46 <shshang> Np, Sean!
21:23:51 <sc68cal> Randy__: excellent
21:24:42 <sc68cal> aveiga: You discovered a bug in dnsmasq with dualstack - I think shshang and Randy__ ran into the same issue
21:24:46 <sc68cal> #topic bugs
21:24:55 <sc68cal> want to give you guys time to discuss
21:24:58 <aveiga> sure
21:25:21 <aveiga> #link https://bugs.launchpad.net/neutron/+bug/1257446 neutron dual-stack bug
21:26:03 <aveiga> I ran into an issue that seems like a show stopper.  If you create a regular IPv4 provider network, it operates as expected.  As soon as you add an IPv6 subnet to that network, dnsmasq stops issuing leases
21:26:22 <aveiga> it turns out that neutron is trying to send both networks to dnsmasq as tags
21:26:34 <sc68cal> shshang: this is happens in the part of the code where you place 1.1.1.1 in as a workaround, if I recall correctly, from Tuesday's meeting
21:26:43 <aveiga> this is invalid syntax for dnsmasq, and causes it to stop issuing responses
21:26:53 <sc68cal> in your review
21:27:21 <aveiga> so in order to get dual-stack working, it might be necessary to fix a fw flaws in the v4 workflow as well
21:27:26 <aveiga> few*
21:27:40 <shshang> yes, you are right....I remember Havana release packaged 2.5.x release, which had a lot of issues of IPv6.
21:27:58 <shshang> I later on "upgraded" it to 2.6.5 release
21:28:28 <aveiga> yes, in the bug I mention that the installed dnsmasq (per the repo) is 2.59
21:29:15 <shshang> Do you plan to fix it from OpenStack side?
21:29:25 <aveiga> that's what I wanted to bring up
21:29:37 <aveiga> there are multiple ways to fix it, but which is correct?
21:30:00 <sc68cal> aveiga: since we are working closely with the maintainer of dnsmasq- perhaps we can discuss with him/her?
21:30:04 <aveiga> we could bump the minimum required version of dnsmasq, or the (imho) proper way would be to fix the command to not have both networks tagged
21:30:27 <aveiga> sc68cal: I'd rather work with what's already deployed
21:30:38 <aveiga> I'm not sure issuing both networks is correct
21:31:04 <jjmb> sean we (comcast) are happy to direct some of our resources to address issues that will advance openstack ipv6 support
21:31:20 <jjmb> we should remain open to upping the version of dnsmasq that is required
21:31:30 <jjmb> some fixes and features may be required
21:31:45 <aveiga> jjmb: The issue lies in that we are treating ipv4 and ipv6 on the same network as separate networks
21:31:45 <sc68cal> jjmb: agreed. I think like aveiga said this is neutron just constructing a bad cmd string
21:31:50 <shshang> @aveiga, I agree with you...I am afraid of if dnsmasq doesn't support it, then there is no way we can make it happen....unless the author changes it
21:32:10 <shshang> clearly it is supported in the newer release
21:32:33 <shshang> another bad thing on 2.5.x release is, it may not support --enable-ra
21:32:33 <aveiga> perhaps this is something we should put to the mailing list
21:32:43 <aveiga> to see what kind of operator pain it would cause to bump the version
21:33:06 <shshang> I remember at the time when I add --enable-ra, it complained. That was the main reason I switched to 2.6.x
21:33:06 <jjmb> we need to keep a running list of dnsmasq related bugs and rfe's so we can ensure they are addressed and accounted for...
21:33:22 <shshang> @jjmb, yup
21:33:50 <aveiga> jjmb: I think I'll go retag the bug dnsmasq in launchpad
21:33:59 <aveiga> we should consider doing that for future issues
21:34:35 <sc68cal> Anyone else have any other bugs to discuss?
21:35:00 <sc68cal> othewise we'll hit tempest, then do open discussion
21:35:52 <sc68cal> #topic tempest
21:36:18 <shshang> I see IBM team initiated several threads on tempest, right?
21:36:25 <sc68cal> shshang: exactly
21:36:41 <sc68cal> #link https://review.openstack.org/#/c/58721/ static ipv6 injection scenario
21:37:16 <sc68cal> I don't know if anyone has anything related to tempest, but I felt it would be good to bring it up anyway
21:37:26 <shshang> What they are doing is helpful for us too.
21:37:57 <aveiga> actually, I'm interested in setting up a separate meeting (one-ff) to review neutron tempest cases
21:38:10 <aveiga> we should look at them and write v6 cases for existing v4 ones
21:38:13 <shshang> Sooner than later, we need to leverage their tool to sanitize our code.
21:38:30 <sc68cal> haleyb: does tempest run on infrastructure that hp provides? i forget
21:38:37 <aveiga> eventually, security groups, fwaas, etc will need to be tested with v6 as well
21:39:24 <shshang> forgive my ignorance, who has the ownership of testing and test case development?
21:39:30 <haleyb> sc68cal: probably, i'd have to ask someone, it's nothing i do
21:40:07 <aveiga> tangentially related, I'm going to start leveraging the Third Party Testing tools to run code against our lab here at Comcast too
21:40:29 <aveiga> I can only cover certain scenarios, but every bit helps until Tempest is running against v6 code
21:40:45 <sc68cal> aveiga: I think that'd be great
21:41:25 <sc68cal> #action aveiga sc68cal jjmb investigate third party test integration for doing ipv6 tempest testing
21:41:28 <shshang> So clearly there is a need...I am eager to see what we can do in order to make IBM team's tool available
21:42:17 <aveiga> are they willing to assist? Forgive my ignorance, but I'm not familiar with the situation
21:42:55 <shshang> In the context of what we are trying to do here, I believe they are going to help
21:44:06 <aveiga> one of the action items I had from the first meeting was to gather a list of the things we're looking for in code review that are "IPv4-only-isms"
21:44:16 <aveiga> jjmb was kind enough to start the ehterpad for it
21:44:23 <aveiga> #link https://etherpad.openstack.org/p/ipv6-coding-considerations ipv4-isms
21:44:40 <aveiga> the list isn't complete by any means, but is a starting point
21:45:06 <sc68cal> #topic open discussion
21:45:15 <shshang> Sorry, Sean, what's the decision of tempest?
21:45:47 <sc68cal> shshang: I think we just keep an eye on what the IBM people are adding to tempest
21:46:40 <aveiga> shshang: I agree with sc68cal, and we should go over it when it's published
21:46:47 <sc68cal> and maybe our team inside comcast can investigate running tempest so we can formailize some of our testing that we've been doing
21:46:48 <aveiga> from there we can look for gaps and try to patch them up
21:47:13 <aveiga> sc68cal: can you give me that as an action item?
21:47:17 <aveiga> I'll put it in my backlog
21:47:37 <sc68cal> I think I got it already
21:47:46 <aveiga> no, the tempest integration part
21:47:50 <aveiga> we're good on theird party
21:47:51 <shshang> Agree, I feel like we should give their tool a try
21:47:57 <sc68cal> ah gotcha
21:48:11 <aveiga> shshang: is there a link to this tool, or are they still in the process?
21:49:04 <shshang> I am not 100% sure of the readiness, but based on my current reading, it adds some enhancement to the existing template
21:49:42 <aveiga> ok, would you mind providing a link (or encouraging them to) to the mailing list when it's available? I'm interested in doing an evaluation
21:50:30 <shshang> The same here. I will nudge them to provide more details. Hopefully it will become community level testing solution for IPv6
21:52:25 <sc68cal> Sounds good - we've got 8 mins left if anyone else has anything
21:54:13 <shshang> Nothing from my side...btw, Randy is going to create more blueprints and link to yours. We will share the link to the mailer when it is done.
21:54:56 <shshang> A quick question, for mid-term, are we going to consider running dnsmasq as dhcp6 server?
21:55:09 <aveiga> shshang: I think that's a good idea
21:55:36 <aveiga> it's already being used today in openwrt and similar projects, and it even handles prefix delegation, which I think needs to be on the long-term list
21:56:12 <aveiga> i.e. we should be able to pull a delegated prefix for use in software load-balancers or service VMs
21:56:45 <shshang> If I understand you correctly, the same dnsmasq will announce RA, and act as DHCPv6 server at the same time, in case of stateless DHCPv6 mode, right?
21:56:57 <sc68cal> aveiga: doesn't the current code attempt to do dhcp6 today?
21:57:09 <aveiga> so that's the question, there's a discussion to be had there
21:57:13 <sc68cal> obviously it breaks, but it tries to do that with the flags its passed
21:57:19 <aveiga> some modes you want RAs issued there
21:57:23 <aveiga> but in some modes you don't
21:57:41 <aveiga> for instance, my deployment with l3 provider networks, you want the RA coming from the upstream gateway
21:57:47 <ijw> aveiga: um, surely you want RAs from the router, not the DHCP server (says he jumping in late)
21:57:49 <aveiga> er, l2 provider
21:58:06 <aveiga> ijw: you want it from dhcp server IF it's the l3 gateway
21:58:15 <aveiga> otherwise you don't
21:58:20 <ijw> OK, told you I'd jumped in late ;)
21:58:24 <aveiga> np
21:58:27 <shshang> I see. That is a very good use case, aveiga
21:58:47 <aveiga> there are several bugs in many implementations wehre multiple RAs fail to work proerly
21:58:55 <aveiga> or where RA priorities are not respected
21:59:06 <aveiga> we have to work around them, unfortunately, as they are widespread
21:59:36 <shshang> aveiga, if you don't mind, can Randy and I take a look at this issue?
21:59:43 <aveiga> sure!
21:59:50 <aveiga> let's take it to the mailing list, since we're out of time
21:59:50 <shshang> i.e. run dnsmasq as DHCPv6 for various mode.
21:59:57 <shshang> absolutely
22:00:01 <aveiga> thanks
22:00:10 <shshang> Don't mention it! Any time!
22:00:22 <shshang> Will keep everybody posted
22:01:06 <sc68cal> OK everyone - let's keep things going on the ML
22:01:22 <shshang> Thanks, everyone
22:01:29 <sc68cal> lots of great stuff going on, very excited to see things coming together
22:01:51 <sc68cal> everyone have a great weekend!
22:01:51 <jjmb> same, thanks sean
22:02:04 <sc68cal> #endmeeting