20:00:42 <johnsom> #startmeeting Octavia 20:00:43 <openstack> Meeting started Wed Aug 10 20:00:42 2016 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:46 <openstack> The meeting name has been set to 'octavia' 20:00:51 <johnsom> Hi there 20:00:52 <sbalukoff> Howdy, folks! 20:01:14 <diltram> hey 20:01:22 <johnsom> Geez, is it just the two of us? 20:01:27 <johnsom> Oh, hi diltram 20:01:44 <diltram> no like, you see there is 3 of us :P 20:01:47 <johnsom> #topic Announcements 20:01:56 <sbalukoff> Haha! Time to hold some critical votes! 20:02:17 <johnsom> #link https://etherpad.openstack.org/p/lbaas-octavia-newton-midcycle 20:02:28 <sbalukoff> That's a week and a half away. 20:02:31 <rm_work> perelman: o/ 20:02:33 <johnsom> #link https://wiki.openstack.org/wiki/Sprints/LbaasNewtonSprint 20:02:34 <rm_work> err 20:02:35 <rm_work> wat 20:02:46 <johnsom> mid-cycle links. It's coming up soon. 20:02:49 <rm_work> o/ 20:02:49 <sbalukoff> Looking forward to annoying y'all face-to-face then! 20:03:03 <johnsom> I have added a proposed agenda to the etherpad. Please comment, etc. 20:03:06 <johnsom> Hahaha 20:03:10 <sbalukoff> Ok. 20:03:21 <johnsom> You probably have the hat already picked out 20:03:22 <rm_work> should we talk about the stuff i mentioned last week that got pushed? 20:03:35 <sbalukoff> Oh, man, in that heat? 20:03:40 <johnsom> rm_work Sorry, remind me? 20:03:50 <rm_work> uhh 20:03:53 <rm_work> previous agenda 20:03:58 <rm_work> let me find the link... 20:04:01 <rm_work> there were two reviews 20:04:05 <sbalukoff> rm_work: Er... later in this meeting, I think so? 20:04:09 <rm_work> ok 20:04:13 <johnsom> The agenda I am talking about is for the mid-cycle 20:04:24 <rm_work> ah not meeting agenda 20:04:30 <johnsom> I'm in announcements, about the mid-cycle. 20:04:33 <rm_work> lol yeah missed that 20:04:48 <rm_work> was wrapping up another meeting simultaneously >_> 20:04:52 <rm_work> anywho, continue 20:05:16 <johnsom> Ok, so, please let me know what you think of the agenda, etc. 20:05:22 <sbalukoff> Ok. 20:05:30 <johnsom> #topic Brief progress reports / bugs needing review 20:05:30 <sbalukoff> Looks good to me. :) 20:05:48 <johnsom> I tried to make it a bit heavy on actually getting stuff done... 20:05:58 <sbalukoff> Ok, I've been doing a lot of reviews over the past few days. Please point me at urgent ones that have not gotten attention, eh! 20:06:21 <sbalukoff> johnsom: Always a good idea, especially given the N3-deadline. 20:06:22 <rm_work> i've been out the last couple days, so i need to start looking again (but i think a ton of stuff was sitting already with +2s from me) 20:06:44 <sbalukoff> rm_work: I've hit most of those, I think. 20:06:48 <rm_work> cool 20:07:06 <johnsom> So, I have been still working on failover. I'm down to dealing with nova changes impacting us. Right now we are moving the ports too quickly for the new nova code, so I need to "wait" for the ports to be dis-associated from the failed amp. 20:07:39 <rm_work> i hope that's actually deterministic / programatic 20:07:41 <sbalukoff> johnsom: Oh, good to know! I was just messing with some failover stuff I noticed this morning. But I suspect you already know about it. 20:07:42 <rm_work> and not "wait a bit" 20:07:48 <johnsom> They wanted me to do the detach a different way, but it looks like that way ends up deleting the ports, which isn't good. So, more frustrating testing/rework to do. 20:08:15 <johnsom> Yeah, I'm polling instead of waiting (poor word choice) to see when nova is done.... 20:08:25 <sbalukoff> johnsom: Please ping me when you've got stuff you'd like reviewed there, as this is obviously a high priority to fix, eh. 20:09:18 <johnsom> sbalukoff Feel free to look at this: https://review.openstack.org/#/c/312270/ Just note that it doesn't "work" right now 20:09:45 <sbalukoff> Ok. 20:09:46 <johnsom> It has ballooned into a large-ish patch 20:10:17 <sbalukoff> Not surprising, given what you're having to do there. :P 20:10:40 <johnsom> Ok, any other updates/priority patches? 20:10:50 <sbalukoff> FWIW, I'd love to see the IPv6 VIPs stuff merge in the relatively near future. 20:11:05 <johnsom> Yeah, me too. The host routes thing too 20:11:10 <sbalukoff> Oh yes! 20:11:19 <johnsom> Then failover. they are all conflict monsters 20:11:38 <sbalukoff> Again, these are features which will make Octavia / N-LBaaS more useful to people if we can get them in in Newton. 20:11:39 <sbalukoff> Yep. 20:12:05 <rm_work> yesplz 20:12:07 <sbalukoff> What order should we be shooting for on merging those? 20:12:13 <diltram> and my create lb flow is still waiting for merge 20:12:22 <rm_work> ipv6 maybe needs to be tested again by someone besides me in devstack 20:12:32 <johnsom> diltram I started reviewing that yesterday 20:12:36 <rm_work> it seems to work but i would like one more set of eyes functionally 20:12:41 <rm_work> i think most of those are already in a chain? 20:12:47 <sbalukoff> Yep, diltram's patch appears to work for me. I've been throwing various ad-hoc scenarios at it and so far it looks like it works. 20:13:01 <johnsom> #link https://review.openstack.org/#/c/349708/6 20:13:11 <diltram> ok, great johnsom, sbalukoff :) 20:13:11 <johnsom> That is the host route patch that finished up the DHCP thing 20:13:16 <sbalukoff> rm_work: Oh, I hadn't noticed. XD 20:13:38 <johnsom> #link https://review.openstack.org/#/c/339826 20:13:43 <johnsom> That is the IPv6 patch 20:14:16 <johnsom> #link https://review.openstack.org/#/c/345003/ 20:14:36 <johnsom> Is merging the flow. I started reviewing, it looks good, but there are a few tests I want to run 20:15:32 <johnsom> #topic Patches requiring discussion/consensus 20:15:44 <johnsom> rm_work I think this is the section you were talking about earlier? 20:15:55 <rm_work> yes 20:16:28 <johnsom> Is it the same list from last week? 20:16:31 <rm_work> yes 20:16:54 <johnsom> #link https://review.openstack.org/#/c/325624/ 20:17:03 <johnsom> Update member status to return real statuses 20:17:11 <johnsom> So what do we need to discuss here? 20:17:31 <rm_work> my concerns were posted on there 20:17:31 <Frito> o/ 20:17:39 <rm_work> I will copy/paste them here, sec 20:17:43 <sbalukoff> OK. 20:17:58 <rm_work> "My concern is that, as far as I can tell, this would make it possible for a user API call to be processed by making more API calls to Octavia synchronously, which seems ugly to me. I don't know what the right answer is though, maybe this is the best we can do? I just want to hear some other opinions before I make a decision." 20:18:28 <rm_work> it's a GET that triggers syncronous API calls to another backend (Octavia) 20:18:37 <rm_work> I didn't think that was a thing we'd ever want to do 20:18:43 <rm_work> for a variety of reasons 20:18:50 <rm_work> only one of which is "that's really slow" 20:19:19 <johnsom> Sigh, yeah. Wouldn't this "go away" with the merge? 20:19:34 <diltram> yes, it will :P 20:19:39 <sbalukoff> Well, the other option is to make Neutron-LBaaS be the source of information, and require various drivers to update stats as the see fit (via event-streamer?) 20:19:50 <johnsom> Isn't that more of an outcome of the way the octavia driver is implemented as opposed to this patch? 20:19:55 <xgerman> sbalukoff this is even more clunky 20:20:19 * TrevorV is really really really late 20:20:26 <sbalukoff> xgerman: Because it'd need to be done every few seconds or something? 20:20:39 * xgerman beat TrevorV by 10s 20:20:42 <sbalukoff> Haha 20:20:45 <rm_work> yewell, the event-streamer is Push, not Poll 20:20:49 <rm_work> *yeah well 20:20:57 <rm_work> so that was actually *the design* 20:21:04 <sbalukoff> Right. 20:21:05 <rm_work> i mean, that was a huge part of why we wrote the event streamer 20:21:11 <rm_work> it just never got hooked up 20:21:24 <xgerman> so once Octavia is and-alone this all we be a non issue 20:21:25 <rm_work> I don't like doing it this way *at all* 20:21:25 <xgerman> ? 20:21:30 <rm_work> yep that too 20:21:37 <rm_work> i kinda wanted to -2 this but i'm not that mean 20:21:57 <sbalukoff> Well, we're talking about it now. And if you sway others to your opinion, -2 it'll be... 20:22:04 <rm_work> and i wanted to make sure it wasn't just me that thinks this is horrible 20:22:16 <johnsom> So, my quick scan of the code, it looks like when someone does a status API request, it calls the driver to get updated status. Right? 20:22:34 <rm_work> yes, and the octavia driver does a REST call to octavia as part of that 20:22:37 <sbalukoff> So... it can be a driver decision. 20:22:38 <rm_work> IIRC 20:22:47 <sbalukoff> Or rather, it is. 20:22:58 <rm_work> hold on, trying to find where i saw the REST calls going out 20:22:59 <johnsom> So, Octavia could implement that status call on the driver to be, pull from local DB because event streamer updated it. Right? 20:23:08 <diltram> but like johnsom said merging api will fix that issue, Octavia will one source of the truth and there will be no more problem with that 20:23:37 <johnsom> So, I am ok with this code and leaving it up to the driver to "do the right thing" 20:23:45 <rm_work> ok so that is true 20:23:57 <rm_work> i swear i actually saw code doing requests 20:24:02 <rm_work> but i can't find it 20:24:11 <rm_work> the octavia driver isn't even touched here 20:24:24 <rm_work> so maybe I was too sleep deprived when i was looking at this 20:24:28 <johnsom> I think the way the Octavia driver is currently implemented it might do the requests calls as you mention 20:24:31 <rm_work> this is exactly why i wanted to discuss :P 20:24:53 <rm_work> ah, maybe that is it 20:25:06 <rm_work> it was like 3 weeks ago now that I reviewed this >_> 20:25:06 <johnsom> Maybe it was a previous version too. Ok, so we are good on this one? 20:25:19 <sbalukoff> Honestly, this gives us more incentive to merge sooner. But would it break someone to merge the code as is? 20:25:25 <rm_work> oh https://review.openstack.org/#/c/324197/ 20:25:41 <rm_work> yeah 20:25:44 <rm_work> so i +2'd that 20:25:53 <rm_work> not realizing it was about to be used in this mess 20:26:00 <sbalukoff> Heh! 20:26:00 <rm_work> that's why 20:26:01 <sbalukoff> Ok. 20:26:15 <rm_work> so as-is, if we merge that, this is going to "happen" 20:26:38 <johnsom> So what you are saying is you want to pull this one back? 20:26:44 <johnsom> Not the one we just looked at? 20:27:01 <johnsom> Buyer's remorse? 20:27:47 <sbalukoff> So yeah, it's not ideal and kind of makes a bunch of the event-streamer stuff pointless. But so will the eventual merge. 20:27:55 <sbalukoff> Eh... it's hard to keep track of this stuff sometimes. 20:27:59 <johnsom> Right 20:28:26 <johnsom> Ok, let's move on to the next one on the list: 20:28:31 <johnsom> #link https://review.openstack.org/#/c/255963/ 20:28:46 <johnsom> Add support for HTTP custom headers 20:28:58 <sbalukoff> Aah yes, this one: 20:29:01 <johnsom> Oye, ok, I remember this discussion from January. 20:29:25 <rm_work> yeah the problem i guess is that not everyone does remember the discussion from january 20:29:29 <rm_work> or remembers it differently :P 20:29:37 <sbalukoff> that's entirely possible. 20:29:41 <sbalukoff> What do you remember from that discussion? 20:29:44 <johnsom> Yeah, that happens 20:29:49 <rm_work> kinda what you do sbalukoff 20:30:03 <rm_work> that i don't think this implementation is quite what i thought 20:30:18 <johnsom> #link https://etherpad.openstack.org/p/lbaas-mitaka-midcycle 20:30:27 <johnsom> Those are the notes, such they are 20:31:26 <sbalukoff> Hmm... doesn't look like those notes definitively answer our discussion. 20:31:40 <johnsom> 5. Optional headers API 20:31:40 <johnsom> We will add a dictionary of optional headers to the pool 20:31:40 <johnsom> pool: { 20:31:40 <johnsom> id: xxxx 20:31:40 <johnsom> optional_headers: { x-forwarded-for: True, ...} 20:31:41 <johnsom> } 20:31:53 <sbalukoff> I'm just worried we're opening a huge window here for vendors to exploit to introduce non-portable features to LBaaS 20:31:54 <johnsom> That is what we captured 20:32:11 <johnsom> Yeah, interesting question 20:33:12 <sbalukoff> It's a relatively small concern. But it's brought up because I think the implication is that that's not a literal list... 20:33:15 <sbalukoff> haproxy is not going to literally insert "X-Forwarded-For: True" in the headers. 20:33:46 <johnsom> Yeah, well, I hope not as that isn't how that header is defined.... 20:34:05 <johnsom> I wish we had Doug here as a "vendor" 20:34:08 <sbalukoff> The haproxy config we generate is going to instruct haproxy to insert the x-forwarded-for header as it's understood to work (ie. it'll be the client IP address.) 20:34:20 <johnsom> Right 20:34:25 <sbalukoff> And we're setting a precedent that that's not a literal list. 20:34:43 <johnsom> It seems like since the drivers will need to implement code for these, it should be a defined list 20:35:10 <sbalukoff> That's my argument-- just do some sanity checking on what we allow there. 20:35:22 <rm_work> yeah we wanted it to be flexible 20:35:27 <sbalukoff> Kind of wish Kobi were here as well. 20:35:27 <rm_work> but still defined as a list 20:35:32 <rm_work> yeah :( timezones though 20:35:35 <rm_work> i am sensitive to that <_< 20:35:37 <sbalukoff> Heh! Indeed. 20:35:56 <johnsom> Yeah. 20:36:28 <johnsom> Ok, so, I will take a note and comment my concern on the patch as well. 20:36:34 <sbalukoff> Ok. 20:36:37 <sbalukoff> Thanks. 20:36:56 <sbalukoff> Feel free to link this conversation when the minutes get generated. :D 20:37:51 <johnsom> Ok, so the last one was the pending state timeout issue, which I jumped into the meeting for. (ignored some finance guy talking) 20:38:00 <johnsom> I think we wrapped that one up right? 20:38:13 <sbalukoff> I think so. 20:38:19 <johnsom> It is marked abandoned 20:38:24 <johnsom> Ok 20:38:26 <lera> hi 20:38:31 <sbalukoff> Yep, we decided there was a better way to do it. 20:38:45 <johnsom> Yes. 20:38:54 <rm_work> ok so wait, for the record, what was the outcome of our discussion of that first patch (the status thing) 20:39:09 <rm_work> we said we wrapped it up but it wasn't clear to me what the decision was :P 20:39:14 <sbalukoff> Oh! 20:39:32 <rm_work> we let this one through and then go back and try to fix the octavia driver, or live with it being really really bad until the merge? 20:39:33 <sbalukoff> Ok, what I was hearing was "we know this is ideal, but we're willing to accept it for now, until the merge happens." 20:39:39 <johnsom> rm_work My take was the patch linked in the agenda is good. We may want to revisit the octavia driver patch however 20:39:45 <rm_work> yeah 20:39:53 <rm_work> i really don't like the octavia bit 20:39:55 <sbalukoff> Aah, ok. 20:39:56 <rm_work> really really 20:40:08 <rm_work> i feel tricked, lol 20:40:14 <sbalukoff> Aah, sorry. 20:40:19 <rm_work> not by you :P 20:40:24 <johnsom> rm_work So, pull your +A. I think you can do that since it hasn't merged 20:40:31 <rm_work> oh, it hasn't 20:40:32 <rm_work> wat 20:40:34 <rm_work> k 20:40:43 <sbalukoff> It's dependent on the other patch you didn't like. :D 20:40:56 <rm_work> lol ok so i just did my review backwards 20:41:03 <sbalukoff> Me too. 20:41:25 <sbalukoff> If you -2 that, I'd give them the hint that we want to see that implemented using event-streamer. 20:41:53 <johnsom> Yeah, be kind and -1 with feedback on a better way 20:41:55 <rm_work> i just -1'd because they could make a new patchset with event streamer :P 20:42:13 <johnsom> I like to encourage people! grin 20:42:19 <sbalukoff> Haha! 20:42:21 <sbalukoff> Not me! 20:42:33 <rm_work> done 20:42:37 <johnsom> Well, yes, you have a special place in my heart sbalukoff 20:42:39 <sbalukoff> Get off my damn lawn, you whippersnappers! 20:42:49 <johnsom> Ok 20:42:55 <johnsom> #topic Open Discussion 20:42:56 <TrevorV> "young whippersnappers" 20:43:36 <johnsom> It seems like there was something I was trying to remember to comment on here from upstream.... 20:43:52 <johnsom> neutron or OpenStack level... 20:44:31 <sbalukoff> Oh? 20:44:59 <johnsom> Hmmm, yeah, don't remember now. Too many meetings. 20:45:08 <johnsom> Voting on summit sessions is done 20:45:10 <sbalukoff> I hear you. 20:45:58 <johnsom> Any other topics today? 20:45:59 <sbalukoff> Last week they had me internally reviewing some LBaaSv1 code fixes. I held my nose and did it, but not without making it abundantly clear they need to abandon that version of LBaaS. Just I like I told them 14 months ago. :P 20:46:15 <sbalukoff> *sigh* 20:46:21 <johnsom> Yeah, it's time to move on.... 20:46:50 <johnsom> We need to talk about our Newton release plans. It's on the agenda for the mid-cycle. 20:47:07 <sbalukoff> What makes it in, and what doesn't? 20:47:13 <johnsom> We need to cut a release. We need to decide what is in and what is out. 20:47:19 <johnsom> Yep. 20:47:20 <sbalukoff> Right. 20:47:24 <johnsom> Plus a version # 20:47:32 <johnsom> Mitaka was 0.8 20:47:46 <diltram> 0.8.1 20:47:49 <diltram> :P 20:47:51 <johnsom> So think about that 20:47:53 <TrevorV> Is mid-cycle next next Monday? 20:47:59 <diltram> yes 20:48:00 <sbalukoff> Well... all the major fixes in the pipes right now (failover flow, IPv6 VIPS, host routes for member subnets)... 20:48:05 <TrevorV> Sweet. 20:48:10 <sbalukoff> I'd also love to see that keystone auth patch land. 20:48:11 <TrevorV> Just making sure, cuz my memory is shot 20:48:18 <johnsom> Yeah, I might be bold and propose 0.9 20:48:26 <TrevorV> that's crazy talk johnsom 20:48:39 <diltram> create flow 20:48:57 <rm_work> errr 20:48:59 <sbalukoff> I think there's very little chance active-active will make it in: Those patches have been WIP for a long time and not usually passing CI. 20:49:00 <rm_work> midcycle is next next monday 20:49:06 <rm_work> "next monday" is the 15th 20:49:09 <rm_work> right? 20:49:09 <johnsom> Ok, well, if there isn't more to cover, let's get back to bug fixes and reviews!!!! 20:49:10 <sbalukoff> Also, again, I'm hoping to get serious docs done. 20:49:29 <sbalukoff> rm_work: Octavia / Neutron-LBaaS mid-cycle is a week after. 20:49:30 <diltram> rm_work: yes 20:49:32 <johnsom> Yeah. BTW, docs did fix the API reference for LBaaS 20:49:33 <rm_work> "this monday" was two days ago 20:49:57 <sbalukoff> johnsom: Rad! 20:50:44 <TrevorV> yes rm_work 20:51:10 <johnsom> Ok, thanks folks! 20:51:14 <rm_work> kk cool, just double-checking semantics :P 20:51:14 <sbalukoff> Thanks! 20:51:18 <johnsom> #endmeeting