20:00:34 #startmeeting Octavia 20:00:34 o/ 20:00:34 Meeting started Wed Mar 9 20:00:34 2016 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:35 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:37 The meeting name has been set to 'octavia' 20:00:38 Howdy folks! 20:00:50 Hi everyone 20:00:58 Hi o/ 20:01:00 o/ 20:01:02 * mhayden stumbles in 20:01:05 yo 20:01:22 hi 20:01:39 o/ 20:01:43 o/ 20:01:44 #topic Announcements 20:02:14 We got at least six LBaaS related talks accepted for Austin! Good job folks! 20:02:25 Yay! 20:02:29 I have included the links here: 20:02:31 #link https://wiki.openstack.org/wiki/Octavia/Weekly_Meeting_Agenda 20:03:06 Really pretty cool from our little corner of OpenStack 20:03:41 Any other announcements? 20:03:59 #topic 20:03:59 Mitaka blueprints/rfes/m-3 bugs for neutron-lbaas and octavia 20:04:22 Ok, so Mitaka is winding to a close. RC1 this week. 20:04:31 There are more bugfixes in the Octavia queue that need attention 20:04:36 #link https://etherpad.openstack.org/p/Mitaka_LBaaS_priority_reviews 20:04:41 We still have ~23 open bugs targeted for Mitaka 20:04:49 #link https://bugs.launchpad.net/octavia/+bugs?field.tag=target-mitaka 20:05:03 related note, this review modifies the governance of lbaas-dashboard to match octavia: https://review.openstack.org/#/c/290735/ 20:05:22 Handy! 20:05:29 dougwig Yeah, let's talk about that given dashboard is for LBaaSv2..... 20:05:47 ok sbalukoff i will look today if i can 20:05:55 maybe laaaaater or maybe nowish 20:06:04 dougwig Can we talk about that during open discussion? 20:06:09 in HAProxy training with bedis :P 20:06:19 yep 20:06:30 rm_work: Sounds good, thanks! 20:06:54 There are a number of bugs still open for TrevorV 20:07:04 Do we know if those are going to get addressed? 20:07:28 All of which are deferred bugs from the merge 20:07:49 o/ 20:07:51 Sorry I'm late 20:07:55 what a coincidence! 20:07:55 Speaking of TevorV 20:08:00 Speak of the devil! 20:08:01 its like someone told him... 20:08:04 *poof* here I am 20:08:09 Haha 20:08:24 TrevorV I was just asking about the deferred bugs that are still open for Mitaka 20:08:42 Can we get those done today/tomorrow? 20:09:01 o/ 20:09:09 That should be possible, if we can do a little talk about how shared pools should work 20:09:22 TrevorV: Didn't we already have that talk? 20:09:33 Did we decide? (scattered in the brain ATM) 20:09:37 Also, I don't *think* those bugs are dependent on shared-pools support in one-call-create. 20:09:50 I thought one of them was shared-pools specifically 20:09:55 Well, one of them is, yes. 20:09:56 I think they are all missing unit test issues 20:10:04 https://bugs.launchpad.net/octavia/+bugs?field.tag=target-mitaka 20:10:40 If you have time to fix the lack of shared-pools support in one-call-create, that would be great. But at this point I doubt you've got the time. 20:11:02 Other than that, there are a bunch of patches up that have +2, cores please review those so we can get them in 20:11:04 sbalukoff: to make it in for Mitaka? 20:11:16 blogan: Yes. 20:11:16 sbalukoff To recap, its utilizing the name as a unique identifier, right? 20:11:29 As well as adding id-verification 20:11:35 TrevorV: No... let's talk after this. 20:11:38 Alright 20:11:49 The last open bug(s) is around admin_state_up not working. bharathm and I are working on that today. 20:12:47 Also, FYI, the tracking page for FFE and postmortem is here: 20:12:50 #link https://review.openstack.org/#/c/286413/5/specs/mitaka/postmortem/postmortem.rst 20:12:58 I think we have updated all of our sections. 20:13:07 dougwig is there more we need to do there? 20:13:18 i don't think so. 20:13:36 Excellent 20:13:49 #topic Decide on Mitaka octavia release number 20:14:03 42 20:14:09 With our impending Mitaka release we need to decide on our release number. 20:14:12 dougwig if dashboard will release with octavia does it need to be added to the ffe list? 20:14:21 * johnsom slaps blogan's wrist. (Again...) 20:14:25 neelashah1: no. 20:14:32 johnsom actually I don't see the shared-pools bug we talked about. 20:14:44 9001? 20:14:52 Haha 20:14:53 TrevorV So it's not targeted for Mitaka and there is time on that one 20:14:57 Oh okay 20:14:57 what are the release number options? 20:14:58 got it 20:15:03 We are currently at 0.5 20:15:12 Do we think we are 1.0 worthy? 20:15:20 I'd put at 0.9. 20:15:21 hmmm 20:15:22 Or still 0.75-ish? 20:15:23 nope 20:15:23 Do we have scaling? 20:15:28 I thought scaling was 1.0 20:15:34 0.999a-1 20:15:36 our earlier 1.0 mvp included some kind of fault tolerance. 20:15:41 Yeah... maybe closer to 0.75 20:15:43 We have failover? 20:15:53 TrevorV: We have failover. 20:15:53 Well, on my roadmap active/active was after 1.0 20:15:53 api is missing features, i'd not be comfortable with it being 1.0 bc of that 20:15:58 if active/standby and ha controllers are good to go... 20:16:03 yeah < 1 20:16:10 1.0 was active/standby 20:16:12 maybe .9753 20:16:14 ish 20:16:21 or are we feeling more 0.8-ish 20:16:28 * rm_work stops 20:16:34 number shedding? 20:16:34 * rm_work is too caffeinated 20:16:40 good heavens, this is the most hippyish release numbering i've ever seen. 20:16:46 HA controllers are not... particularly ready to go. You can do it as long as only one runs at a time. But operators are beholden to "figure that out" right now without documentation. 20:16:47 bike numbering? 20:17:08 Just don't HA your HA and you're set. 20:17:11 I like 0.75 20:17:14 Yo dawg 20:17:17 HA controllers as a post 1.0 I think as well. Not sure 20:17:18 Haha 20:17:22 next release can be 0.875 20:17:32 rm_work you didn' 20:17:36 you didn't stop**** 20:17:37 Man... have to dig out those old roadmaps we put together a loooong time ago. 20:17:41 Should probably get those updated. 20:17:42 y'all are gonna vote, i can feel it. 20:17:45 .6 20:17:47 sbalukoff they're still in tree I think :P 20:17:50 just go one up and be sane 20:17:53 then then 0.9375 20:17:58 I think 0.5.9 20:18:00 just use constants, e, c 20:18:01 blogan: +1 20:18:08 we can just approach 1 20:18:09 Ok, so I'm hearing 1.0 is out, so I vote for 0.8 20:18:20 yeah i think we're more 0.8 than 0.6 20:18:22 * blogan chokes dougwig for being prophetic 20:18:25 Haha 20:18:37 I'm fine with 0.8 20:18:39 I'm okay with 0.8, all jokes aside 20:18:44 sure .8 20:18:49 Done! 20:18:55 Done 20:18:58 No vote necessary 20:19:03 ... that was a vote 20:19:03 it fits the requirement of .5 < x < 1.0 20:19:04 * sbalukoff sticks out his tongue at dougwig 20:19:21 That was a *discussion* 20:19:25 We didn't use the voting system. 20:19:29 #topic Open Discussion 20:19:36 Cores please review patches! 20:19:41 Yes, please! 20:19:43 * johnsom nags some more 20:19:44 please review this later today: https://review.openstack.org/#/c/172199 20:19:52 I'll jump into that for the patches that aren't mine later today, too. :D 20:19:53 * dougwig head is spinning. 20:19:59 i'm going to make 1 change to make it work with the shared pools stuff 20:20:03 then it should be rady 20:20:05 ready 20:20:10 I'd like for Newton to start filling in the gaps of the Octavia API to get parity with neutron-lbaas 20:20:23 are we still as a group on board with that? 20:20:24 fnaval: Ping me when it's ready. 20:20:29 sbalukoff: sure thanks! 20:20:31 dougwig, can you +1 on project-config patch: https://review.openstack.org/#/c/284875/ 20:20:35 what is newton 20:20:40 blogan: Yep. 20:20:41 N release 20:20:41 So! 20:20:42 did i miss a project starting? :P 20:20:48 madhu_ak: i can take a look at it, i can't promise a particular result. :) 20:20:48 oh we named that already, right 20:20:56 forgot they did two at once this time 20:21:03 dougwig The topic of dashboard under octavia project. 20:21:06 Next week, or whenever the RC is cut and newton is open: We need to have a design discussion around features targeted for Newton. 20:21:20 We could really use feedback on this: 20:21:21 https://review.openstack.org/#/c/234639/ Add spec for active-active 20:21:23 I have a question to gauge the group with 20:21:25 That seems, odd. Just temp? 20:21:35 So I'm supposed to write up docs for the single create 20:21:35 Because I know that IBM will be working hard to make that happen in Newton. 20:21:45 johnsom: really, that's a topic of me considering the "lbaas project" to be octavia, neutron-lbaas, and neutron-lbaas-dashboard. and if we're kicking -dashboard to release indepedent, let's keep it close to octavia. IMO. 20:22:05 Especially given I'm apparently signed up for a talk to showcase a proof-of-concept of the same at the Austin summit. XD 20:22:09 sbalukoff Let's focus on getting the Mitaka patches reviewed as we already put active/active out of scope for Mitaka 20:22:15 repeat of review link on that governance change: 20:22:15 https://review.openstack.org/#/c/290735/ 20:22:23 Is anyone against me detailing the request in a paragraph and then providing a full example of request/response? 20:22:35 dougwig Ok 20:22:36 dougwig: +1 20:22:37 I am still working on the launchpad ticket —File creation susceptible to TOCTOU type attack(https://bugs.launchpad.net/octavia/+bug/1548552) 20:22:37 Launchpad bug 1548552 in octavia "File creation susceptible to TOCTOU type attack" [High,In progress] - Assigned to min wang (swiftwangster) 20:22:38 Rather than some massively verbose table of all the elements and such? 20:22:55 hello guys :) 20:22:58 Oh! 20:23:08 For all things docs: 20:23:13 #link https://etherpad.openstack.org/p/lbaas-octavia-docs-needed 20:23:33 I have a call with IBM tech writers tomorrow afternoon. I'm hoping to get help fleshing out our docs. 20:23:45 That's the summary of what I would like to see eventually. 20:23:47 Please update that! 20:23:54 Yes, that etherpad for docs is great 20:23:56 TrevorV: Please add a note about single-create to that. 20:24:23 a question about amphora-agent: i noticed that the agent configures interfaces inside the service vm when it starts. my question is: why? won't it be preferable that the diskimage-create script will pre-configure the interfaces (which are dhcp anyways) when it cooks the image? 20:24:38 I created a launchpad ticket for lb's status is in PENDING_UPDATE for about 50-55 mins after creating a listener with TLS no SNI. https://bugs.launchpad.net/octavia/+bug/1555316 20:24:38 Launchpad bug 1555316 in octavia "loadbalancer status is stuck in PENDING_UPDATE for about 50-55 mins after creating listener with TLS no SNI" [Undecided,New] 20:24:48 nmagnezi No, we hot-plug interfaces 20:25:05 johnma, care to elaborate? 20:25:07 madhu_ak: i'll see if I can reproduce 20:25:19 So no one cares about how I'll write that up? If I follow the current structure of requests, the table will be insane. 20:25:34 nmagnezi: we have the option to have a standby pool of prebuilt amps, which are active but are not configured we will not know what networks they need to be hot plugged into upfront 20:25:42 nmagnezi We hot-plug interfaces into the amphora VM, so those interfaces are not necessarily present at boot time. 20:25:47 TrevorV: You will need to show a fully-fleshed-out example in your docs somewhere. :/ 20:26:05 sbalukoff that's what I'm talking about, I WANT the example, I DON'T want the table 20:26:12 nmagnezi: would work great for containers though! since they can't hot plug 20:26:19 whenever we get support for that in 20:26:26 TrevorV: Then maybe I don't understand what you mean by 'table' 20:26:44 TrevorV yeah, I am having a hard time visualizing as well 20:27:17 https://raw.githubusercontent.com/openstack/octavia/master/doc/source/api/octaviaapi.rst 20:27:20 blogan, johnsom, I understand the reasoning now. thanks! the reason i'm asking is because it is configuring the NICs ubuntu style. meaning it sets invalid configuration in Centos/Fedora/RHEL based images 20:27:33 If you look in there, the sections that are "tables" 20:27:46 All the objects of the LB are defined in tables, and then an example is provided 20:28:02 I'm talking about just saying "single create uses all the things above in one call, like so:" more or less 20:28:06 You know, more officially 20:28:14 But only including an example, and a reference to the other sections 20:28:23 blogan, johnsom, the bug https://bugs.launchpad.net/octavia/+bug/1548070 . I hope to submit a fix soon, but in general the agent should be aware of the OS it is running on. 20:28:23 Launchpad bug 1548070 in octavia "Amphora agent fails to start in a Centos based amphora" [Medium,New] 20:28:24 nmagnezi Yes. That could be fixed. If you set the interfaces file name option, it does work with RedHat style systems as it uses the interface file and not the .d directory 20:28:28 TrevorV: Oh! Ok, so... something does need to be added to the Octavia API reference about this. But I don't think you need a massive table under 'create a load balancer' which repeats all the stuff in later sections. 20:28:51 nmagnezi: indeed it should, we just got it working for ubuntu first 20:28:54 Right, if everyone can agree on that, then I'll have a review up tonight/tomorrow 20:29:08 i never tested it out with any redhat distros 20:29:13 TrevorV I think you can just reference the other sections. 20:29:17 Perfect 20:29:35 johnsom: are amphora images hosted on openstack infra? if so, can you point me to where? 20:29:35 johnsom, the filename is one thing. but it's configurable so it's ok. but it is missing the jinja2 template for valid configuration and also the logic of when to use which template 20:29:39 Then I'll work something into it all, and when its up for review you guys can provide some feedback, as per usual 20:29:42 as is tradition 20:29:46 Ok. 20:30:24 neelashah1 No, they are built by devstack plugin script or the user using the diskimage-create.sh script included 20:30:28 johnsom: Just to be clear on everyone's focus for the next week: We should be working on bug fixes or bug fix reviews, yes? 20:30:40 markvan ^ 20:30:44 johnsom: Newton stuff and stuff not necessary for the RC1 deadline should happen next week, yes? 20:31:06 I have a question about what sbalukoff pointed above. re: controller HA. is that not expected to work at all for Mitaka? for all services what consist the controller? worker, health-manager, housekeeping 20:31:32 sbalukoff Yes, the focus this week is bug fixes and bug reviews. We really need to get the fixes in for Mitaka 20:31:49 I have a top priority internally for neutron lbaas single-create 20:31:54 nmagnezi: It works so long as you have one instance of the controller-worker and health manager running at a time. We have not tested it with multiple instances, and it's unlikely to work, given what I know about the code. 20:32:08 So you can achieve HA if you code a way to detect a failure of one of these processes and deal with it yourself. 20:32:14 So I can't say I share that dedication for the bug fixes, but the two bugs I have for mitaka I'll get in tonight/tomorrow for sure 20:32:17 There are no pre-packaged scripts for this right now, though. 20:32:21 nmagnezi You can have multiple instances of the controller processes 20:32:33 sbalukoff You are wrong on single instance 20:32:40 I will try to wrap upthe bug that i am working ASAP 20:32:45 johnsom: I am? You guys have tested this? 20:32:53 sbalukoff: we can have multiple workers 20:32:56 TrevorV: i thought single-create for neutron-lbaas got -2'd for Mitaka, or is that old news and now it is RFE'd in or something? 20:33:01 johnsom, same node? multiple nodes? both cases? 20:33:09 We tested this on a multi-node env and it works 20:33:12 rm_work doesn't matter, reach needs it at Rax 20:33:12 or was that something else entirely 20:33:14 nmagnezi It has been very lightly tested and will not fail over requests that are in process yet 20:33:21 Huh! Ok. 20:33:23 ah :/ 20:33:26 Yeah 20:33:28 my same face 20:34:14 johnsom, got it. and by controller which octavia process do you mean? 20:34:40 nmagnezi All four and both cases 20:34:57 nmagnezi but same node might get strange with port issues 20:34:57 johnsom, got it, thank you for the answers :) 20:35:22 johnsom, make sense 20:35:44 nmagnezi I highly recommend doing extra testing if you plan to deploy this way, as we have lightly tested it. 20:35:55 i haven't tested it out either, but seems like it would work 20:35:57 johnsom: When did y'all test it? 20:36:03 It's not a feature I am highlighting as "we are controller HA" 20:36:17 I'm willing to bet there are race conditions... but I'm not sure I could reliably reproduce them. 20:36:26 After all, events that change things are relatively rare. 20:36:33 you're a race condition! 20:36:37 johnsom, is this expected to get test coverage (tempest?) at some point? is it in the works? 20:36:43 As those services were being written. Yes, there likely are bugs 20:37:14 Yes, at some point. We also plan to have recovery for in progress requests in a future release 20:37:31 (ie. job board) 20:37:47 Ok, any other Mitaka concerns. I think cascade delete is going to be on the out list. 20:37:54 dougwig Any comments on that? 20:37:56 sounds like it 20:38:11 nmagnezi: no tempest tests for it as far as I know; please create a blueprint/bug for it though 20:38:13 Really unfortunate given the series of events 20:38:21 johnsom: sorry, wasn't paying attention. scrolling 20:38:22 and if thats the case...should we just drop the current proposed way and put it on the lbtree resource? since that is being done for single create? 20:38:42 fnaval, will do 20:38:44 * sbalukoff gets out his paint again. 20:38:50 nmagnezi: thank you so much 20:38:55 blogan If that is the case, we can continue painting after Mitaka release 20:39:08 * blogan paints paint 20:39:52 dougwig Cascade delete mess. Not landing I take it? I hope it's not just because of the client release timing. 20:40:15 johnsom: oh, i hadn't seen the defer. gimme a few, not sure yet. 20:40:59 dougwig: https://review.openstack.org/#/c/287593/ 20:41:09 last comment 20:41:24 are both the client and server ready to go? 20:41:37 Yes 20:42:01 I think the issue was around neutron client being cut early 20:42:16 We argued about the API path too long 20:42:19 Why was it cut early? 20:42:42 So that somebody could create the CLI command reference using their non-documented procedure? :P 20:43:38 hmm.... OR, we could move the client commands into an in-repo extension and do an end-run around the cat's skin. 20:43:39 Don't get me started on the docs issues. No answers that actually work from those folks. Just deleted docs. 20:43:50 (Not actually looking for an answer to that, I guess, just pointing out what a mess that all is.) 20:44:32 doug-fish for the dashboard cascade delete are you using the neutron client or just going to the API? 20:45:04 neutron client 20:45:11 Damn. 20:45:16 posted an alternative in the review. 20:45:25 we'd need to shuffle some files around. 20:45:49 Ok, so, seems un-likely 20:46:46 Going out the door without cascading delete from the UI isn't my favorite, but it's not the end of the world. 20:47:31 this could be updated during the stable release cycle later on right? 20:47:40 doug-fish: so without cascading delete users will still be able to delete things? 20:47:55 doug-fish: you mean as a backport? 20:48:01 blogan: yes 20:48:21 doug-fish: likely not, features arent backported 20:48:35 blogan: yes, as a backport - also yes to users can delete things part by part 20:48:35 Right, only bug fixes get backported 20:48:40 doug-fish: sorry i asked back to back yes/no questions lol 20:49:22 I have to run but will be back later 20:49:25 ah well if users can delete things part by part, i'm less inclined to jump through hoops just to get the cascade delete in, it'll go in almost immediately when N opens up 20:49:41 not assuming bike shedding 20:49:44 Yeah, I agree, pushing rope at this point 20:49:58 phrasing 20:49:59 lol 20:50:23 if the client piece got done today, i'd be willing to push hard for M. if that's going to take awhile, then it might indeed be N. 20:50:42 well, i think it IS done? 20:50:49 but not with the alternate method? 20:50:53 as an extension, not in python-neutronclient 20:51:00 no way will i be able to budge the client proper. 20:51:03 dougwig Client is done as-is, your proposal I don't know how that works 20:51:39 dougwig: would that extension have to live on as an extension? 20:51:46 johnsom: you put the client code in neutron-lbaas, and add a stevedore extension that python-neutronclient finds automagically. http://docs.openstack.org/developer/python-neutronclient/devref/client_command_extensions.html 20:51:49 ah k 20:52:02 blogan: if you ask me, all the lbaas client commands should be in our repo as an extension. 20:52:06 huh, that doesn't seem to bad 20:52:11 ^^ actually I agree with that 20:52:17 So you have to install neutron-lbaas on the hosts that run the client? Not sure that would fly 20:52:27 ah THAT is interesting 20:52:39 johnsom: you could always put it in yet another separate repo, i guess. :) 20:52:50 T_T nevermind i take it back 20:52:55 Yep, that would sadly be the way to go 20:53:10 can we create an lbaas stadium? 20:53:18 :P 20:53:23 blogan I like the way you think! Grin 20:53:25 johnsom: right, because installing n-l pulls in n, which pulls in ovs and crap. ugh. 20:53:38 Right 20:53:45 we have to accept our fate 20:53:55 does -dashboard just need the api? 20:54:03 dougwig: it needs the client 20:54:06 I'm sure openstackclient will solve all of our problems.... 20:54:13 Haha 20:54:18 johnsom: yes...of course 20:54:20 really gross, but the client extension could go into -dashboard. oh god, i can't believe i typed that. 20:54:32 Eew. 20:54:33 okay with that meeting is almost done 20:54:41 hitler! 20:54:42 dougwig has lost it 20:54:45 * johnsom face palms 20:54:47 godwin has been invoked 20:54:49 and godwin 20:54:49 Haha 20:55:03 Ok, any other topics in the next six minutes? 20:55:39 Quick sanity check regarding dashboard.; 20:55:51 Dashboard sanity checks always fail 20:56:06 ajmiller yes? 20:56:16 LOL. I just want to have common understanding that it is OK to merge patches. Or not? 20:56:55 ajmiller Yes. They are proposing an independent release cadence, so please continue reviews. 20:57:02 dougwig Do you agree? 20:57:42 johnsom dougwig - its unclear to me who will be doing this independent release for the dashboard…just so we can stay in sync and get ready... 20:58:16 maybe we can continue in the lbaas channel 20:58:17 neelashah1 I can be your release point person. 20:58:29 ok, great…thanks, johnsom! 20:58:59 Ok, thanks folks! Please fix bugs and do reviews!!!! 20:59:04 #endmeeting