19:59:56 #startmeeting Octavia 19:59:57 Meeting started Wed Jul 6 19:59:56 2016 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:59:59 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:02 The meeting name has been set to 'octavia' 20:00:02 o/ 20:00:11 o/ 20:00:12 I everyone 20:00:29 Well, those of you that aren't off enjoying a vacation week 20:00:32 hi 20:00:48 #topic Announcements 20:01:01 #linkhttps://etherpad.openstack.org/p/lbaas-octavia-newton-midcycle 20:01:10 #link https://etherpad.openstack.org/p/lbaas-octavia-newton-midcycle 20:01:13 Try that again 20:01:25 So, just a reminder, upcoming mid-cycle. 20:01:49 Please sign up if you plan to join us 20:02:09 o/ 20:02:19 Also, I will put a section in soon for topics we want to cover. 20:02:32 Act/Act and LBaaS merge come to mind 20:03:08 o/ 20:03:29 Also, last call for summit LBaaS talk topics. I will be entering our talk this week, so if you have a topic you would like to speak about, please drop me an e-mail. 20:03:39 Also keep in mind we need to know if you plan to be remote. That way we can set up for VC stuff 20:03:42 Right now I think it is just sbalukoff and I 20:04:11 Most likely covering the merge and act/act, with the normal roadmap-ish stuff 20:04:27 #topic Brief progress reports / bugs needing review 20:04:33 Ok, updates? 20:04:59 Are we doing a Demo again? 20:05:07 o/ 20:05:19 I have been working on failover related bugs. I should have a new spin of the DNS integration patch soon-ish. Cleaning up unit test breakage from the new code 20:05:25 OSIC bug smash is going on over the next 3 days (including today) so we might want to or already have people crushing some buggeronies 20:05:49 Mostly internal stuff. OSIC bug smash here as well over the next couple days :-) 20:05:55 rm_mobile Maybe. Not sure what we will have cooked for a demo. act/act would be great but I think sbalukoff said he wasn't sure if they would have something ready 20:06:53 After the DNS patch is done, I am going to focus on the failover critical bug (which is blocked by the dns issue) 20:07:24 Other patches needing review? 20:07:29 would be great if we could do a more real-world demo this time, like sending the summit app through an octavia vip 20:07:45 We also got a fix in for the scenario gate to pull the right agent code. That was good 20:07:51 yes, I have some needing review :) 20:07:57 #link https://review.openstack.org/320999 20:08:10 that's a functional test for lbaas 20:08:35 #link https://review.openstack.org/#/c/332913/ 20:08:36 dougwig Sure, we could even stream one of the summit YouTube videos. Do you think that would be interesting enough? 20:08:46 #link https://review.openstack.org/#/c/337650/ 20:09:35 johnsom: less of a demo, and more of a real-world proof point that it's production ready. 20:09:57 I guess I just have a "demoed basic functionality in Vancouver, need to amp it up...." mentality. Which might be not so good 20:10:20 Ah, yeah, I follow you 20:10:44 That actually brings up an interesting question. Do we plan to call newton 1.0? 20:10:56 Focusing on bugs would be good 20:11:21 yes, bugs + stability 20:11:30 Hmmm, food for thought. 20:11:32 i'd be fine with calling whatever release has a seamless amp upgrade as 1.0 20:12:08 Well, with Act/Stndby, you could do a rolling amp upgrade with Mitaka 20:12:16 We just didn't script it up or such 20:12:42 that's a tad "assembly required", even for openstack. 20:12:53 Ha 20:13:08 yep, and the packagers are not stepping up so we should do it 20:13:11 Is that a volunteer to write up the script? 20:14:00 Starting with getting that doc bug closed would be good too. 20:14:48 Ok. Moving on.... 20:15:09 #topic Should amphorae be rebootable? 20:15:31 I did get a chance to test this per my action item. It does in fact work correctly. 20:15:38 Pretty fast too 20:15:40 i think this topic *might* have a toe-hold in the "should amphorae be distro tweakable?" discussion. 20:16:23 So, I have moved that bug to incomplete as it works for me 20:16:49 Any questions/comments on rebootable before we jump into distros? 20:16:55 distro tweakable means that it should run on other linux flavors? 20:17:11 that's one possible consequence of that, yes 20:17:37 I suspect the reporter was running without nested virt, so boot took a very long time and/or triggered a failover 20:17:55 ah, cool 20:18:07 #topic amphorae distros 20:18:08 I think the limiting factor might be the tooling aka the disk image builder 20:18:42 I initially dropped this from the agenda as there was an e-mail chain going with folks that can't usually make the meeting. But we can discuss more here 20:19:14 has folks been reading that email thread? i only see a few opinions on there, and i think that has some pretty profound long-term maintenance implications. 20:19:36 yeah, I would *not_ want to support multiple distros 20:19:47 Yeah, I have skimmed it, but haven't had a chance to reply 20:19:48 i'm not on the ML... 20:19:50 can that be pluggable, as other stuff in octavia? 20:19:54 is there a link? 20:20:28 there is a set of distros the disk image builder supports and we only would need to change/adjust the agent/start-up scripts 20:20:33 #link http://lists.openstack.org/pipermail/openstack-dev/2016-July/thread.html 20:20:37 thx 20:20:43 I've been having email list issues... I apparently keep getting email bounces... Might be something to talk to someone about 20:20:46 search forsuggestion for today's meeting agenda: How to make the Amphora-agent support additional Linux flavors 20:20:49 if we need to throw out the builder to support some exotic stuff, well... 20:20:54 well, we need to support different forms of init 20:20:55 yeah 20:20:57 http://lists.openstack.org/pipermail/openstack-dev/2016-July/098680.html 20:21:00 Ugh, macs and their strange paste function 20:21:04 that's in the middle; it must've started in june 20:21:13 which can be a pain >_> especially if we absolutely require they are updated in lockstep (see my stalled patch that fixes sysvinit) 20:21:22 so first we need to figure out if we want to/can keep disk image builder 20:21:34 To answer kobis's question, yes it is plugable. 20:21:40 i'm actually most concerned of when we switch to containers instead of nova. i'm not sure i want to maintain the nova stuff forever. 20:21:44 I'm not sure why DIB would be non-keepable? 20:22:00 The amp to us is a black box. As long as it has the agent that works, we are fine with it. 20:22:01 if you like to go to a bistro which is not supported 20:22:03 yeah I've discussed the containers stuff with some infra peeps, DIB should be able to build container images with very little fuss 20:22:09 we can keep it all; the real question is how much we keep, since keep implies docs, tests, upkeep, etc. 20:22:19 We do ship a reference image build script, to facilitate gate tests, etc. 20:22:33 I'm not married to DIB, but it seems workable sans a better alternative 20:22:46 Currently our script can create Ubuntu, centos, fedora, and docker images 20:22:49 and other distros can be added to DIB if they aren't supported... 20:22:58 Though , we only really test the ubuntu image 20:23:05 * blogan stumbles in 20:23:08 rm_work +! 20:23:21 well, we would still test only one image 20:23:22 i don't care about dib. if we had a script that downloaded stock ubuntu, booted it, tweaked it, and saved it, i'd be equally content. 20:23:27 3rd party CI could test others 20:23:38 well, if anyone needs a different builder and would like to test it, he can run his own CI 20:23:50 +! 20:24:09 ok, so xgerman has hit the point indirectly right there. yes, 3rd party CI can test others. and what happens when they break? because we totally change how amps work? do we have an obligation to support that, since we accepted it? 20:24:15 dougwig That is basically what DIB does.... 20:24:45 well, what does Neutron do if the build breaks? Remove the offending 3rd party stuff? 20:25:10 neutron considers the ML2 and plugin interfaces as long-term public contracts. are we saying that amps are now the same? 20:25:30 that is the maintenance obligation that i'm talking about. 20:25:36 nope - our contract is with the amp-drivere 20:26:00 Personally, I lean towards: we support one distro as needed for our gates. If other contribute updates, that is fine, but we don't support/babysit those other distros. 20:26:11 +1 20:26:16 +1 20:26:34 ihar is very knowledgable about distros and packaging, so i'd encourage everyone to discuss this with him on thread. 20:27:01 yeah, they asked for some refactoring to make it easier for other distress IHMO 20:27:21 i'm in favor of cleanup and fixing assumptions in code 20:27:22 johnsom: i agree with that statement, and further agree with blogan's statement that if we get a micro-image that switches distros, i'm totally fine dumping ubuntu. :) 20:27:33 yeah fffffff ubuntu 20:27:38 micro-image please 20:27:39 Yep 20:27:43 I was working on that for a while but was retasked 20:27:58 my experience with distro-building was not enough to do it very quickly 20:28:13 I am sure someone with better knowledge could hammer it out in a week 20:28:14 xgerman, rm_work: hang on, if we accept refactoring that allows other distros, and they tool around that, it's not feasible to just yank that carpet away a few weeks later. that comes with implications. 20:28:50 dougwig: I'm happy to allow time for 3rd-party CIs to fix issues around CRs that change amphora contract 20:28:55 as a matter of policy 20:29:01 but, there just need to be limits on that 20:29:21 on the other hand, are we really going to seriously change much with the amp agent *contract* at this point? 20:29:34 what happens if we decide REST is really dumber than ssh and want to switch back? or if a pub-sub model for amps is better? do we have to maintain the rest agent forever? 20:29:34 mmh, the refactoring they wanted was that agent would start keepalived and haproxy without the help of the system 20:29:47 because rest is dumber, but that's a tangent. 20:29:48 In reality, our touch points with the distro are pretty small. Just a few minor "we have to do it different to be different" things. 20:29:51 dougwig: well, some of us already decided that, because it is :P 20:29:57 rpc 20:30:06 SOAP - keep its clean 20:30:14 * rm_work exiles xgerman 20:30:22 you mention SOAP again I'll cut you... 20:30:25 -2 20:30:25 -2 20:30:27 -2 20:30:27 :) 20:30:28 and while we are at it rewrite agent in Go 20:30:28 SOAP is not a good thing. EVER 20:30:41 Unless its not an acronym and is for cleaning 20:30:41 SOAP is a great idea! 20:30:45 Corba too 20:30:46 i get that it's easy to support multi-distro amps. but it's setting a precedent for how our amps are built and communicate, and once we have a community around *that*, we will need to support it. 20:30:59 yeah, I like the contract we have now with the agent 20:31:04 People can write alternate amphora drivers 20:31:13 everything is just around "make the agent init, make the agent properly touch the machine" 20:31:22 the contract is around the API interaction, and should be solid 20:31:23 is the problem here really whether or nto the amp agent interface is something we always maintain? or is it the actual distro its running on? 20:31:31 johnsom: and what happens if the current amp driver becomes the alternate? i guess we can tell the 3rd party they have to take it over. 20:31:38 yep 20:31:40 though that has not gone well with the namespace driver. 20:31:45 there's deprecation cycles for a reason 20:31:47 nor V1 20:32:15 i'm just not super concerned that we'll break *contract*, which I define as the REST API contract with the agent 20:32:38 Yes, I would argue that if "we" move on for the reference amp (gates) then yes, if someone wants to keep the REST they need to maintain it. 20:33:03 * rm_work calls a vote to return to using the SSH driver 20:33:06 Even the REST API contract is inside the amp driver, so.... 20:33:13 i'd like to move on to an API on the host/hypervisor instead of on each amp 20:33:14 well, we need to be clear with people not that they write their own REST agents and get caught flat footed 20:33:20 and what, in our history or openstack's history, would imply that we've ever been able to remove drivers we don't like? :) 20:34:00 people SHOULD be able to write their own agent code, if they want -- as long as they follow the contract :P 20:34:05 i'm ok if the answer is being upfront about what's in flux and what they might have to take over. as xgerman said, i have no interest in sowing the seeds of surprise later. 20:34:09 that's the whole point of a REST API contract 20:34:34 rm_work: note above blogan's statement about where the rest endpoint lives, e.g. 20:34:37 I think it comes down to going in and moving the driver/agent code out to a separate repo. Probably as it should have been. But only if it really comes down to moving it out 20:34:52 yeah I started on that, and it's a lot more work than I thought 20:34:57 there's a LOT of code-reuse in there 20:35:00 which is gooood, but 20:35:09 it means it touches a ton of the rest of the project 20:35:17 it’s a hair ball... 20:35:20 so isolating it means a LOT of copy/paste 20:35:37 I just want it to be clear, the contract is with the amp driver, not the REST API driver 20:35:37 we almost need octavia-lib 20:35:47 johnsom: +1 20:35:49 hmm, fair point 20:35:57 johnsom +1 — and that needs spelling out in CAPS 20:36:09 yeah because i didn't even think of it that way until you mentioned it 20:36:26 thats how we initially thought of it 20:36:26 (was another one of the votes I lost) 20:36:32 It's all that indirection you guys love so much 20:36:37 +1 20:36:49 well, yeah, the indirection that'll let us go back to SSH once this whole REST thing blows over 20:36:53 and now you come around to just make the REST interface a contract ;-) 20:36:55 (well, I love it, but) 20:37:10 next time we will create the haproxy file on the amp as well 20:37:14 i suspect *strongly* that once we have internal support for custom distro amps, having that support disappear would be... loud. 20:37:59 Ok, so, what I hear is we need to put up a doc, stating our "contract" intentions as well as the amp image build scripts. 20:38:05 so what are the implications of just saying the contract is with the amp driver and not the amp agent api? that we're not maintaining custom distros? 20:38:49 well, we limit our support if we decide that hypervisor things is better 20:39:02 I can take that, but it will likely be next week or later..... 20:39:03 or dougwig’s beloved SSH driver 20:39:17 i'm stackforging that someday. 20:39:38 johnsom: let's go for consensus on the ML before we write it up? 20:40:06 Ok, that buys time. But we will need to build consensus around the patch too 20:40:09 dougwig: i'll contribute <_< 20:40:31 dougwig: I'll +2 it for main repo >_> 20:41:30 I really don't want to lock out other distros, if they want to support it, from the code, but having clear expectations around that are needed. 20:41:51 (thus why I put fedora/centos/docker support in our image builder script) 20:41:58 we're not locking anyone out if we're using the best driver we have 20:42:25 well, they were concerned about us supporting upstart and they have something else 20:42:34 right, i don't want to lock anyone out. i also don't want to lock our group into a crushing maintenance burden, when we still lack basic features. 20:42:36 but that should be fixable 20:42:36 systemd 20:42:42 but everyone is moving to systemd 20:42:52 oh, systemd can go get fucked. but that's another tangent. 20:42:54 so it’s a non issue? 20:42:55 Yeah, we need to take that plunge 20:42:55 well 20:43:05 for one, can we vote on whether we have to update EVERY INIT SCRIPT in lockstep? 20:43:09 that holds back progress IMO 20:43:29 pretty sure dougwig will be on the wrong end of systemd getting fucked 20:43:31 Ok, good discussion, let's all try to contribute to the ML discussion with an end goal of getting a doc into the repo 20:43:43 see: https://review.openstack.org/#/c/298424/ 20:44:03 I ended up abandoning it because it's a fix to sysvinit only, and I'm not going to *uck around with upstart 20:44:07 I would think we leave updating init scripts to distress but provide interfaces 20:44:11 Well, I pushed back because it wasn't updating our reference init script for the gates 20:44:12 so i guess we don't get fixed sysvinit? >_> 20:44:36 i feel like a lot of people are going to feel similarly 20:44:41 so let people who want things fix them 20:44:51 don't force people to fix things they aren't using 20:45:16 well, we should not have more features in the non-reference implementation 20:45:21 that has a tendency to discourage any contribution, because "if you want to submit a fix for A, you'll have to submit a fix for B too" gets old 20:45:29 LESS BUGS, not more features 20:45:45 "can't fix the bug in A, because you didn't fix a similar bug in B which isn't used at the same time as A" 20:45:46 i don't find the lassiez-faire approach to merging vendor code to be very practical. it's very non-deterministic, and still has global maintenance implications. 20:46:27 we just need guidelines, i think. and to not accept code that doesn't fit with our reference, unless it's shims or hooks to enable people. IMO. 20:46:34 rm_work I'm sorry you abandoned that. I don't think it was much work to fix the upstart side too and offered to help with that. 20:46:59 I mean, could go fix upstart, but at this point I'm not doing it as a matter of principle -- this is a dumb way to encourage sontributions 20:46:59 *contributions 20:47:04 I think it sets a bad precedent 20:47:10 I guess others disagree 20:47:17 but that's my $0.02 20:48:02 Let's talk about it offline 20:48:06 kk 20:48:23 #topic Open Discussion 20:48:42 Any other topics? 20:48:54 i don't see *any* edits to the midcycle etherpad since we finalized dates/location. go edit yourself! 20:49:03 i'd like to request to tag along as a co-presenter for any talks being given 20:49:04 cascade delete 20:49:28 cascade delete is a good one 20:49:35 I'm interested in helping with a demo, if we'll have one, but since it sounds like maybe we won't have a specific demo lab, I'm prolly good 20:49:36 just want to make sure that we have consensus to do this in nlbaas still 20:49:39 not sure i'd be able to go anyway 20:49:43 with the impending move away from nlbaas 20:49:49 If we don't get that API cooked I think the client update is going to be dead 20:50:01 https://review.openstack.org/#/c/338048/ 20:50:06 +1 rm_work 20:50:07 i've started teh cascade delete stuff again 20:50:21 and it shouldn't be too difficult with the graphs stuff already done 20:50:35 you know it’s almost N-2 — and only because of a naming issue... 20:50:48 * xgerman bangs head on wall 20:50:52 blogan Cool! Ping me when the API works-ish and I will update the client code 20:51:31 guys, can you please check this one: 20:51:33 #link https://review.openstack.org/#/c/337117/ 20:52:22 my vote is keeping cascade in octavia only and just accelerate the split 20:52:33 +1 20:52:52 would love to "accelerate the split", but what is the timeline we're looking at still? 20:52:58 maybe O-release? 20:53:02 That is an interesting one. It could be implemented to allow overlapping IPs as long as the subnets were different. It would be a lot of work however 20:53:34 johnsom thanks for the review on image create patch, was able to solve the problem and addressed couple of your comment. will address other comments in the follow up patch 20:53:39 The vendors might not support that 20:54:01 bana_k Ok, thanks 20:54:33 Anything else? 20:55:00 Thanks folks 20:55:03 #endmeeting