19:59:56 <johnsom> #startmeeting Octavia
19:59:57 <openstack> Meeting started Wed Jul  6 19:59:56 2016 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:59:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:02 <openstack> The meeting name has been set to 'octavia'
20:00:02 <Frito> o/
20:00:11 <TrevorV> o/
20:00:12 <johnsom> I everyone
20:00:29 <johnsom> Well, those of you that aren't off enjoying a vacation week
20:00:32 <eezhova> hi
20:00:48 <johnsom> #topic Announcements
20:01:01 <johnsom> #linkhttps://etherpad.openstack.org/p/lbaas-octavia-newton-midcycle
20:01:10 <johnsom> #link https://etherpad.openstack.org/p/lbaas-octavia-newton-midcycle
20:01:13 <johnsom> Try that again
20:01:25 <johnsom> So, just a reminder, upcoming mid-cycle.
20:01:49 <johnsom> Please sign up if you plan to join us
20:02:09 <rm_mobile> o/
20:02:19 <johnsom> Also,  I will put a section in soon for topics we want to cover.
20:02:32 <johnsom> Act/Act and LBaaS merge come to mind
20:03:08 <xgerman> o/
20:03:29 <johnsom> Also, last call for summit LBaaS talk topics.  I will be entering our talk this week, so if you have a topic you would like to speak about, please drop me an e-mail.
20:03:39 <TrevorV> Also keep in mind we need to know if you plan to be remote.  That way we can set up for VC stuff
20:03:42 <johnsom> Right now I think it is just sbalukoff and I
20:04:11 <johnsom> Most likely covering the merge and act/act, with the normal roadmap-ish stuff
20:04:27 <johnsom> #topic Brief progress reports / bugs needing review
20:04:33 <johnsom> Ok, updates?
20:04:59 <rm_mobile> Are we doing a Demo again?
20:05:07 <bana_k> o/
20:05:19 <johnsom> I have been working on failover related bugs.  I should have a new spin of the DNS integration patch soon-ish.  Cleaning up unit test breakage from the new code
20:05:25 <TrevorV> OSIC bug smash is going on over the next 3 days (including today) so we might want to or already have people crushing some buggeronies
20:05:49 <Frito> Mostly internal stuff. OSIC bug smash here as well over the next couple days :-)
20:05:55 <johnsom> rm_mobile Maybe.  Not sure what we will have cooked for a demo.  act/act would be great but I think sbalukoff said he wasn't sure if they would have something ready
20:06:53 <johnsom> After the DNS patch is done, I am going to focus on the failover critical bug (which is blocked by the dns issue)
20:07:24 <johnsom> Other patches needing review?
20:07:29 <dougwig> would be great if we could do a more real-world demo this time, like sending the summit app through an octavia vip
20:07:45 <johnsom> We also got a fix in for the scenario gate to pull the right agent code.  That was good
20:07:51 <eezhova> yes,  I have some needing review :)
20:07:57 <eezhova> #link https://review.openstack.org/320999
20:08:10 <eezhova> that's a functional test for lbaas
20:08:35 <eezhova> #link https://review.openstack.org/#/c/332913/
20:08:36 <johnsom> dougwig Sure, we could even stream one of the summit YouTube videos.  Do you think that would be interesting enough?
20:08:46 <eezhova> #link https://review.openstack.org/#/c/337650/
20:09:35 <dougwig> johnsom: less of a demo, and more of a real-world proof point that it's production ready.
20:09:57 <johnsom> I guess I just have a "demoed basic functionality in Vancouver, need to amp it up...." mentality.  Which might be not so good
20:10:20 <johnsom> Ah, yeah, I follow you
20:10:44 <johnsom> That actually brings up an interesting question.  Do we plan to call newton 1.0?
20:10:56 <johnsom> Focusing on bugs would be good
20:11:21 <rm_work> yes, bugs + stability
20:11:30 <johnsom> Hmmm, food for thought.
20:11:32 <dougwig> i'd be fine with calling whatever release has a seamless amp upgrade as 1.0
20:12:08 <johnsom> Well, with Act/Stndby, you could do a rolling amp upgrade with Mitaka
20:12:16 <johnsom> We just didn't script it up or such
20:12:42 <dougwig> that's a tad "assembly required", even for openstack.
20:12:53 <johnsom> Ha
20:13:08 <xgerman> yep, and the packagers are not stepping up so we should do it
20:13:11 <johnsom> Is that a volunteer to write up the script?
20:14:00 <johnsom> Starting with getting that doc bug closed would be good too.
20:14:48 <johnsom> Ok.  Moving on....
20:15:09 <johnsom> #topic Should amphorae be rebootable?
20:15:31 <johnsom> I did get a chance to test this per my action item.  It does in fact work correctly.
20:15:38 <johnsom> Pretty fast too
20:15:40 <dougwig> i think this topic *might* have a toe-hold in the "should amphorae be distro tweakable?" discussion.
20:16:23 <johnsom> So, I have moved that bug to incomplete as it works for me
20:16:49 <johnsom> Any questions/comments on rebootable before we jump into distros?
20:16:55 <kobis> distro tweakable means that it should run on other linux flavors?
20:17:11 <dougwig> that's one possible consequence of that, yes
20:17:37 <johnsom> I suspect the reporter was running without nested virt, so boot took a very long time and/or triggered a failover
20:17:55 <rm_work> ah, cool
20:18:07 <johnsom> #topic amphorae distros
20:18:08 <xgerman> I think the limiting factor might be the tooling aka the disk image builder
20:18:42 <johnsom> I initially dropped this from the agenda as there was an e-mail chain going with folks that can't usually make the meeting.  But we can discuss more here
20:19:14 <dougwig> has folks been reading that email thread? i only see a few opinions on there, and i think that has some pretty profound long-term maintenance implications.
20:19:36 <xgerman> yeah, I would *not_ want to support multiple distros
20:19:47 <johnsom> Yeah, I have skimmed it, but haven't had a chance to reply
20:19:48 <rm_work> i'm not on the ML...
20:19:50 <kobis> can that be pluggable, as other stuff in octavia?
20:19:54 <rm_work> is there a link?
20:20:28 <xgerman> there is a set of distros the disk image builder supports and we only would need to change/adjust the agent/start-up scripts
20:20:33 <johnsom> #link http://lists.openstack.org/pipermail/openstack-dev/2016-July/thread.html
20:20:37 <rm_work> thx
20:20:43 <TrevorV> I've been having email list issues... I apparently keep getting email bounces... Might be something to talk to someone about
20:20:46 <johnsom> search forsuggestion for today's meeting agenda: How to make the Amphora-agent support additional Linux flavors
20:20:49 <xgerman> if we need to throw out the builder to support some exotic stuff, well...
20:20:54 <rm_work> well, we need to support different forms of init
20:20:55 <rm_work> yeah
20:20:57 <dougwig> http://lists.openstack.org/pipermail/openstack-dev/2016-July/098680.html
20:21:00 <johnsom> Ugh, macs and their strange paste function
20:21:04 <dougwig> that's in the middle; it must've started in june
20:21:13 <rm_work> which can be a pain >_> especially if we absolutely require they are updated in lockstep (see my stalled patch that fixes sysvinit)
20:21:22 <xgerman> so first we need to figure out if we want to/can keep disk image builder
20:21:34 <johnsom> To answer kobis's question, yes it is plugable.
20:21:40 <dougwig> i'm actually most concerned of when we switch to containers instead of nova. i'm not sure i want to maintain the nova stuff forever.
20:21:44 <rm_work> I'm not sure why DIB would be non-keepable?
20:22:00 <johnsom> The amp to us is a black box.  As long as it has the agent that works, we are fine with it.
20:22:01 <xgerman> if you like to go to a bistro which is not supported
20:22:03 <rm_work> yeah I've discussed the containers stuff with some infra peeps, DIB should be able to build container images with very little fuss
20:22:09 <dougwig> we can keep it all; the real question is how much we keep, since keep implies docs, tests, upkeep, etc.
20:22:19 <johnsom> We do ship a reference image build script, to facilitate gate tests, etc.
20:22:33 <rm_work> I'm not married to DIB, but it seems workable sans a better alternative
20:22:46 <johnsom> Currently our script can create Ubuntu, centos, fedora, and docker images
20:22:49 <rm_work> and other distros can be added to DIB if they aren't supported...
20:22:58 <johnsom> Though , we only really test the ubuntu image
20:23:05 * blogan stumbles in
20:23:08 <xgerman> rm_work +!
20:23:21 <xgerman> well, we would still test only one image
20:23:22 <dougwig> i don't care about dib.  if we had a script that downloaded stock ubuntu, booted it, tweaked it, and saved it, i'd be equally content.
20:23:27 <xgerman> 3rd party CI could test others
20:23:38 <kobis> well, if anyone needs a different builder and would like to test it, he can run his own CI
20:23:50 <xgerman> +!
20:24:09 <dougwig> ok, so xgerman has hit the point indirectly right there.  yes, 3rd party CI can test others.  and what happens when they break?  because we totally change how amps work?  do we have an obligation to support that, since we accepted it?
20:24:15 <johnsom> dougwig That is basically what DIB does....
20:24:45 <xgerman> well, what does Neutron do if the build breaks? Remove the offending 3rd party stuff?
20:25:10 <dougwig> neutron considers the ML2 and plugin interfaces as long-term public contracts.  are we saying that amps are now the same?
20:25:30 <dougwig> that is the maintenance obligation that i'm talking about.
20:25:36 <xgerman> nope - our contract is with the amp-drivere
20:26:00 <johnsom> Personally, I lean towards: we support one distro as needed for our gates.  If other contribute updates, that is fine, but we don't support/babysit those other distros.
20:26:11 <xgerman> +1
20:26:16 <kobis> +1
20:26:34 <dougwig> ihar is very knowledgable about distros and packaging, so i'd encourage everyone to discuss this with him on thread.
20:27:01 <xgerman> yeah, they asked for some refactoring to make it easier for other distress IHMO
20:27:21 <rm_work> i'm in favor of cleanup and fixing assumptions in code
20:27:22 <dougwig> johnsom: i agree with that statement, and further agree with blogan's statement that if we get a micro-image that switches distros, i'm totally fine dumping ubuntu.  :)
20:27:33 <rm_work> yeah fffffff ubuntu
20:27:38 <rm_work> micro-image please
20:27:39 <johnsom> Yep
20:27:43 <rm_work> I was working on that for a while but was retasked
20:27:58 <rm_work> my experience with distro-building was not enough to do it very quickly
20:28:13 <rm_work> I am sure someone with better knowledge could hammer it out in a week
20:28:14 <dougwig> xgerman, rm_work: hang on, if we accept refactoring that allows other distros, and they tool around that, it's not feasible to just yank that carpet away a few weeks later. that comes with implications.
20:28:50 <rm_work> dougwig: I'm happy to allow time for 3rd-party CIs to fix issues around CRs that change amphora contract
20:28:55 <rm_work> as a matter of policy
20:29:01 <rm_work> but, there just need to be limits on that
20:29:21 <rm_work> on the other hand, are we really going to seriously change much with the amp agent *contract* at this point?
20:29:34 <dougwig> what happens if we decide REST is really dumber than ssh and want to switch back?  or if a pub-sub model for amps is better?  do we have to maintain the rest agent forever?
20:29:34 <xgerman> mmh, the refactoring they wanted was that agent would start keepalived and haproxy without the help of the system
20:29:47 <dougwig> because rest is dumber, but that's a tangent.
20:29:48 <johnsom> In reality, our touch points with the distro are pretty small.  Just a few minor "we have to do it different to be different" things.
20:29:51 <rm_work> dougwig: well, some of us already decided that, because it is :P
20:29:57 <blogan> rpc
20:30:06 <xgerman> SOAP - keep its clean
20:30:14 * rm_work exiles xgerman
20:30:22 <TrevorV> you mention SOAP again I'll cut you...
20:30:25 <rm_work> -2
20:30:25 <rm_work> -2
20:30:27 <rm_work> -2
20:30:27 <kobis> :)
20:30:28 <xgerman> and while we are at it rewrite agent in Go
20:30:28 <TrevorV> SOAP is not a good thing.  EVER
20:30:41 <TrevorV> Unless its not an acronym and is for cleaning
20:30:41 <johnsom> SOAP is a great idea!
20:30:45 <johnsom> Corba too
20:30:46 <dougwig> i get that it's easy to support multi-distro amps. but it's setting a precedent for how our amps are built and communicate, and once we have a community around *that*, we will need to support it.
20:30:59 <rm_work> yeah, I like the contract we have now with the agent
20:31:04 <johnsom> People can write alternate amphora drivers
20:31:13 <rm_work> everything is just around "make the agent init, make the agent properly touch the machine"
20:31:22 <rm_work> the contract is around the API interaction, and should be solid
20:31:23 <blogan> is the problem here really whether or nto the amp agent interface is something we always maintain? or is it the actual distro its running on?
20:31:31 <dougwig> johnsom: and what happens if the current amp driver becomes the alternate?  i guess we can tell the 3rd party they have to take it over.
20:31:38 <rm_work> yep
20:31:40 <dougwig> though that has not gone well with the namespace driver.
20:31:45 <rm_work> there's deprecation cycles for a reason
20:31:47 <xgerman> nor V1
20:32:15 <rm_work> i'm just not super concerned that we'll break *contract*, which I define as the REST API contract with the agent
20:32:38 <johnsom> Yes, I would argue that if "we" move on for the reference amp (gates) then yes, if someone wants to keep the REST they need to maintain it.
20:33:03 * rm_work calls a vote to return to using the SSH driver
20:33:06 <johnsom> Even the REST API contract is inside the amp driver, so....
20:33:13 <blogan> i'd like to move on to an API on the host/hypervisor instead of on each amp
20:33:14 <xgerman> well, we need to be clear with people not that they write their own REST agents and get caught flat footed
20:33:20 <dougwig> and what, in our history  or openstack's history, would imply that we've ever been able to remove drivers we don't like?  :)
20:34:00 <rm_work> people SHOULD be able to write their own agent code, if they want -- as long as they follow the contract :P
20:34:05 <dougwig> i'm ok if the answer is being upfront about what's in flux and what they might have to take over.  as xgerman said, i have no interest in sowing the seeds of surprise later.
20:34:09 <rm_work> that's the whole point of a REST API contract
20:34:34 <dougwig> rm_work: note above blogan's statement about where the rest endpoint lives, e.g.
20:34:37 <johnsom> I think it comes down to going in and moving the driver/agent code out to a separate repo.  Probably as it should have been.  But only if it really comes down to moving it out
20:34:52 <rm_work> yeah I started on that, and it's a lot more work than I thought
20:34:57 <rm_work> there's a LOT of code-reuse in there
20:35:00 <rm_work> which is gooood, but
20:35:09 <rm_work> it means it touches a ton of the rest of the project
20:35:17 <xgerman> it’s a hair ball...
20:35:20 <rm_work> so isolating it means a LOT of copy/paste
20:35:37 <johnsom> I just want it to be clear, the contract is with the amp driver, not the REST API driver
20:35:37 <rm_work> we almost need octavia-lib
20:35:47 <dougwig> johnsom: +1
20:35:49 <rm_work> hmm, fair point
20:35:57 <xgerman> johnsom +1 — and that needs spelling out in CAPS
20:36:09 <rm_work> yeah because i didn't even think of it that way until you mentioned it
20:36:26 <blogan> thats how we initially thought of it
20:36:26 <xgerman> (was another one of the votes I lost)
20:36:32 <johnsom> It's all that indirection you guys love so much
20:36:37 <xgerman> +1
20:36:49 <rm_work> well, yeah, the indirection that'll let us go back to SSH once this whole REST thing blows over
20:36:53 <xgerman> and now you come around to just make the REST interface a contract ;-)
20:36:55 <johnsom> (well, I love it, but)
20:37:10 <xgerman> next time we will create the haproxy file on the amp as well
20:37:14 <dougwig> i suspect *strongly* that once we have internal support for custom distro amps, having that support disappear would be... loud.
20:37:59 <johnsom> Ok, so, what I hear is we need to put up a doc, stating our "contract" intentions as well as the amp image build scripts.
20:38:05 <blogan> so what are the implications of just saying the contract is with the amp driver and not the amp agent api? that we're not maintaining custom distros?
20:38:49 <xgerman> well, we limit our support if we decide that hypervisor things is better
20:39:02 <johnsom> I can take that, but it will likely be next week or later.....
20:39:03 <xgerman> or dougwig’s beloved SSH driver
20:39:17 <dougwig> i'm stackforging that someday.
20:39:38 <dougwig> johnsom: let's go for consensus on the ML before we write it up?
20:40:06 <johnsom> Ok, that buys time.  But we will need to build consensus around the patch too
20:40:09 <rm_work> dougwig: i'll contribute <_<
20:40:31 <rm_work> dougwig: I'll +2 it for main repo >_>
20:41:30 <johnsom> I really don't want to lock out other distros, if they want to support it, from the code, but having clear expectations around that are needed.
20:41:51 <johnsom> (thus why I put fedora/centos/docker support in our image builder script)
20:41:58 <blogan> we're not locking anyone out if we're using the best driver we have
20:42:25 <xgerman> well, they were concerned about us supporting upstart and they have something else
20:42:34 <dougwig> right, i don't want to lock anyone out. i also don't want to lock our group into a crushing maintenance burden, when we still lack basic features.
20:42:36 <xgerman> but that should be fixable
20:42:36 <blogan> systemd
20:42:42 <blogan> but everyone is moving to systemd
20:42:52 <dougwig> oh, systemd can go get fucked. but that's another tangent.
20:42:54 <xgerman> so it’s a non issue?
20:42:55 <johnsom> Yeah, we need to take that plunge
20:42:55 <rm_work> well
20:43:05 <rm_work> for one, can we vote on whether we have to update EVERY INIT SCRIPT in lockstep?
20:43:09 <rm_work> that holds back progress IMO
20:43:29 <blogan> pretty sure dougwig will be on the wrong end of systemd getting fucked
20:43:31 <johnsom> Ok, good discussion, let's all try to contribute to the ML discussion with an end goal of getting a doc into the repo
20:43:43 <rm_work> see: https://review.openstack.org/#/c/298424/
20:44:03 <rm_work> I ended up abandoning it because it's a fix to sysvinit only, and I'm not going to *uck around with upstart
20:44:07 <xgerman> I would think we leave updating init scripts to distress but provide interfaces
20:44:11 <johnsom> Well, I pushed back because it wasn't updating our reference init script for the gates
20:44:12 <rm_work> so i guess we don't get fixed sysvinit? >_>
20:44:36 <rm_work> i feel like a lot of people are going to feel similarly
20:44:41 <rm_work> so let people who want things fix them
20:44:51 <rm_work> don't force people to fix things they aren't using
20:45:16 <xgerman> well, we should not have more features in the non-reference implementation
20:45:21 <rm_work> that has a tendency to discourage any contribution, because "if you want to submit a fix for A, you'll have to submit a fix for B too" gets old
20:45:29 <rm_work> LESS BUGS, not more features
20:45:45 <rm_work> "can't fix the bug in A, because you didn't fix a similar bug in B which isn't used at the same time as A"
20:45:46 <dougwig> i don't find the lassiez-faire approach to merging vendor code to be very practical.  it's very non-deterministic, and still has global maintenance implications.
20:46:27 <dougwig> we just need guidelines, i think. and to not accept code that doesn't fit with our reference, unless it's shims or hooks to enable people.  IMO.
20:46:34 <johnsom> rm_work I'm sorry you abandoned that.  I don't think it was much work to fix the upstart side too and offered to help with that.
20:46:59 <rm_work> I mean, could go fix upstart, but at this point I'm not doing it as a matter of principle -- this is a dumb way to encourage sontributions
20:46:59 <rm_work> *contributions
20:47:04 <rm_work> I think it sets a bad precedent
20:47:10 <rm_work> I guess others disagree
20:47:17 <rm_work> but that's my $0.02
20:48:02 <johnsom> Let's talk about it offline
20:48:06 <rm_work> kk
20:48:23 <johnsom> #topic Open Discussion
20:48:42 <johnsom> Any other topics?
20:48:54 <dougwig> i don't see *any* edits to the midcycle etherpad since we finalized dates/location.  go edit yourself!
20:49:03 <fnaval> i'd like to request to tag along as a co-presenter for any talks being given
20:49:04 <blogan> cascade delete
20:49:28 <johnsom> cascade delete is a good one
20:49:35 <rm_work> I'm interested in helping with a demo, if we'll have one, but since it sounds like maybe we won't have a specific demo lab, I'm prolly good
20:49:36 <blogan> just want to make sure that we have consensus to do this in nlbaas still
20:49:39 <rm_work> not sure i'd be able to go anyway
20:49:43 <blogan> with the impending move away from nlbaas
20:49:49 <johnsom> If we don't get that API cooked I think the client update is going to be dead
20:50:01 <blogan> https://review.openstack.org/#/c/338048/
20:50:06 <fnaval> +1 rm_work
20:50:07 <blogan> i've started teh cascade delete stuff again
20:50:21 <blogan> and it shouldn't be too difficult with the graphs stuff already done
20:50:35 <xgerman> you know it’s almost N-2 — and only because of a naming issue...
20:50:48 * xgerman bangs head on wall
20:50:52 <johnsom> blogan Cool!  Ping me when the API works-ish and I will update the client code
20:51:31 <sbelous_> guys, can you please check this one:
20:51:33 <sbelous_> #link https://review.openstack.org/#/c/337117/
20:52:22 <dougwig> my vote is keeping cascade in octavia only and just accelerate the split
20:52:33 <xgerman> +1
20:52:52 <rm_work> would love to "accelerate the split", but what is the timeline we're looking at still?
20:52:58 <rm_work> maybe O-release?
20:53:02 <johnsom> That is an interesting one.  It could be implemented to allow overlapping IPs as long as the subnets were different.  It would be a lot of work however
20:53:34 <bana_k> johnsom thanks for the review on image create patch, was able to solve the problem and addressed couple of your comment. will address other comments in the follow up patch
20:53:39 <johnsom> The vendors might not support that
20:54:01 <johnsom> bana_k Ok, thanks
20:54:33 <johnsom> Anything else?
20:55:00 <johnsom> Thanks folks
20:55:03 <johnsom> #endmeeting