20:00:24 #startmeeting Octavia 20:00:25 Meeting started Wed Jun 7 20:00:24 2017 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:26 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:28 The meeting name has been set to 'octavia' 20:00:37 Hi folks! 20:00:40 o/ 20:00:44 o/ 20:00:54 o/ 20:01:05 Pretty full agenda today. 20:01:14 #topic Announcements 20:01:24 This week is Pike-2 milestone 20:01:30 #link https://releases.openstack.org/pike/schedule.html 20:01:47 I will be cutting our Pike-2 releases today 20:02:27 Also on releases, I had some stable branch releases queued, but the stable team did not like the DIB changes, so I need to work through that. I will probably split them up. 20:02:57 For reference DIB changes the package that has the command to execute, requiring a requirements.txt change which is not cool on stable, so... 20:03:18 umm 20:03:21 <_< 20:03:44 they can't make an exception? 20:03:49 Also of note, there is a lot of discussion on the mailing list about changes to the docs team. Basically a push to move more of the docs under the project repos. So I think that is a good thing. 20:04:01 aren't we already doing that? 20:04:04 rm_work They probably will, I just need to circle back and make the case 20:04:12 what more can we move under our project 20:04:20 AFAIK *all* of our docs are in our project already 20:04:26 Yes, we have been aggressively moving our docs into the repo 20:04:37 so, it's a NOP for us? 20:04:51 There is still some setup stuff in the networking guide that I hope to move into an install guide for us. 20:05:15 Yeah, no-op, but something to raise as all of the projects will need to start dealing with this 20:05:27 Plus, we have been pushing to own our own docs 20:05:58 Also, goals are a hot topic for the Queens series. Fun. 20:06:20 We have not closed our Pike goals (py3 and wsgi) yet. 20:06:35 Py3 we need to shift some voting around with n-lbaas 20:06:35 >_< 20:06:53 we were the first to close our our wsgi goal... technically 20:06:59 *close out 20:07:15 Also, they changed the wsgi goal to now be running standalone with a wsgi pass through in Apache, so yeah, start over on that fun 20:07:15 rm_work True! 20:07:15 of all projects AFAIK 20:08:16 They changed it the day we merged 20:08:49 Ah well. I'm worried about the impact to projects with these goals and them not being thought through at the start of the cycle. It burns a bunch of time. 20:09:19 yes 20:09:39 Final announcement, due to the great work from JudeC we are about to have an openstack client plugin that has almost all of the features! 20:09:51 We should be able to merge the rest of it today. 20:10:04 :) 20:10:10 I will probably create a new release of that tomorrow if all goes well. 20:10:21 Any other announcements? 20:10:52 #topic Brief progress reports / bugs needing review 20:11:01 Any progress reports to share or hot bugs? 20:11:28 I see some came in about failover and the public subnet that I plan it investigate later today. 20:12:09 Wow, ok, nobody jump in at once. 20:12:20 On to the agenda items... 20:12:39 #topic jsonschema blueprint 20:12:46 #link https://blueprints.launchpad.net/octavia/+spec/jsonschema-api-validation 20:13:36 So it was raised that WSME is not really being actively maintained (I have not researched this) and we should consider jsonschema as an alternative. 20:13:57 My quick glance at jsonshema is that it isn't a full replacement for WSME. 20:14:03 Anybody have thoughts on this? 20:14:25 sorry i'll have to read more carefully first, so can't comment 20:14:25 :) 20:15:09 Yeah, no problem. I just wanted to raise awareness and see if others had opinions on this 20:15:38 I did check, it is in global requirements. 20:15:59 I guess I will do some searches and see if other projects are using it with their API 20:16:18 well 20:16:29 I know internally we use voluptuous and I think it seems neat? 20:16:38 it was mentioned in one of the linked ML threads 20:16:43 as an alternative 20:17:14 Hmm, yeah, that is in global reqs as well. 20:17:27 I'm just not sure how pressing of a concern this is... 20:18:28 #topic Oslo versioned objects blueprint 20:18:36 #link https://blueprints.launchpad.net/octavia/+spec/oslo-versioned-object 20:18:41 This has come up before 20:19:04 My take on this is, yes, we probably should do it at some point 20:19:36 Again, not sure there is a pressing need until we need to contract a database schema. 20:19:44 right yeah 20:19:47 nothing is broken 20:19:50 when something breaks... 20:20:05 i mean, i don't know who happens to have the copious free time necessary to do this stuff 20:20:15 yep, low priority 20:20:19 I know most of the folks that worked on neutron's OVO implementation are not working on OpenStack anymore, so I'm not sure there is a lot of developer resource around to work on this. 20:20:41 and it’smore pressing for themn because of ROPC calls 20:21:18 Yeah, and rolling upgrades 20:21:42 Which again, so far we have only added to the schema, so... 20:22:07 Any reason not to approve this blueprint at a low priority in case someone wants to work on it? 20:22:15 nope 20:22:32 i'd say go for it, but with the recommendation that we still discuss voluptuous vs jsonschema 20:22:37 I will probably push back a bit just to have more details added 20:22:41 i'm not sold on jsonschema 20:23:01 Yeah, the jsonschema thing I'm going to hold on for more research time 20:23:41 #topic Proposal to move Octavia to storyboard\ 20:23:57 So, the storyboard folks reached out asking if we are interested in migrating. 20:24:12 They have run test migrations using the octavia launchpad data with success 20:24:20 #link https://storyboard.openstack.org/ 20:24:44 I have not looked at it in a few years when it was first proposed 20:24:51 Any thoughts here? 20:25:23 what are the benefits here? how is it better compare to launchpad? (i have not looked at it) 20:25:39 It's nice to have developer support for the move, but I'm also not hating launchpad at the moment. It seems to be ok for our small team needs 20:25:47 sbalukoff was a bof proponent of stiryboard 20:25:58 bof=big 20:26:12 There is some discussion here: 20:26:15 #link https://storyboard-blog.io/ 20:26:27 Really, I have not looked at it too much in a long time. 20:26:44 looks more modern ;-) 20:27:19 Maybe this is another one where we go research and come back next week ready to discuss. 20:27:37 yup 20:27:43 yeah 20:27:44 i hate to fix what's not broken 20:27:45 at this point 20:27:49 with such limited mental bandwidth 20:27:54 Yeah, I'm kind of in that mode too 20:28:13 #topic multibinder 20:28:22 #link https://review.openstack.org/#/c/466260/ 20:28:49 I wonder if the automation is there for CR auto-updates / etc 20:28:58 So, I have kind of said what I wanted to say about this on the patch. I -1'd it but the author put up another patch. 20:29:06 ah i thought he abandoned it 20:29:16 yea, I believe he abandoned it 20:29:36 Ah, he did yesterday. Ok! Solves that. 20:29:52 I tested reloads with 1 vCPU and didn't notice any impact 20:30:05 when members are added/removed is it called through systemctl? 20:30:09 the reload that is 20:30:17 Yeah, my analysis of the issue is our default with 1 CPU should not be impacted 20:30:28 Yes 20:31:06 johnsom, I can share some of the numbers with you that I did with benchmarks and reloading 4x per second 20:31:31 #topic Discuss asynchronous API and response handling (updates) (rm_work / johnsom) 20:31:44 jniesz Alway interested to hear folks experiences 20:32:01 We kind of ran out of time last week on this one. 20:32:23 so my take on this is that i'm ok with the method everyone else liked last time 20:32:25 which was that? 20:32:44 It was option 1 (or at least that is what I liked) 20:32:48 though, I've kinda run out of available time to hack at this, so this whole discussion may be a moot point again, unless other people have copious free time 20:32:51 yup. 20:32:54 1 20:33:56 1. We return the data in the pre-update form but with provisioning status "PENDING_UPDATE". i.e. If they update the name field, the data the API will return will still have the old name but reflect that a change is in progress. This means we return the user accurate data at all times, but can be confusing. 20:34:17 err wait 20:34:21 isn't that what we do NOW? 20:34:26 yep 20:34:41 can you just copy/paste the rest again 20:34:49 Well, for the most part yes. I think we should go through and clean up and make sure it is consistent, but yes 20:35:00 2. We return the data with "faked" updates and "PENDING_UPDATE". This way the user gets back what they just changed, but the changes may not yet actually be applied yet. Followup GET calls will reflect the accurate state and not the "faked" data. For example, setting the LB to admin_state_up=False would show false in the response, but the LB would still 20:35:09 3. We change our revert models such that we actually update the database with their requested changes. This means the result and future GET calls reflect the requested change, even though they may not yet be in effect. "PENDING_UPDATE" would still apply. To enable proper rollback to "actual state" we would need to pass the previous state data model over 20:35:15 4. Make the calls synchronous. Probably not a good idea. 20:35:24 5. Don't return anything on an Update call. This is how the new OSC commands behave. It would mean breaking the API though, so rolling the version, etc. 20:35:41 That is what came off the top of my head last week 20:37:24 hmmm 20:37:42 I would probably also add a blurb in our api-ref about this too 20:37:43 i think i liked 3, but i do admit that is weird in its own right 20:37:54 so maybe just leaving it alone is better 20:38:02 do we know of anywhere there are inconsistencies? 20:38:17 it’s confusing but that’s all 20:38:30 I just have a nag in the back of my head that it needs double checked. I'm happy to do that 20:38:43 k 20:39:17 It is a bit confusing, but gives the user an accurate view. Also confusing is getting back a change but not having the actual traffic flow reflect it yet.... 20:39:46 I.e. doing an admin-state-up=False, getting back that result, but traffic is still flowing. 20:39:56 It really comes down to the meaning of PENDING_UPDATE 20:40:02 yeah 20:40:25 well, I also kinda wanted a "SYNC" command 20:40:30 in the operator API 20:40:35 to "try again" 20:40:47 but that doesn't work without us saving the pending state somewhere 20:40:54 because it's lost once the worker gives up 20:41:08 is it? 20:41:08 if we save the state immediately, at least you can try a SYNC 20:41:28 Yeah. I wonder if we shouldn't change the way we pop things off the handler queue. Plus, job board should make that case better too 20:41:43 xgerman_: AFAIK yes, since it's not saved to the DB anywhere, just passed to rabbit, picked up, acknowledged, and now the only copy is in the worker's memory 20:41:45 yes, I think job-board could improve this 20:41:50 if it maintains a state of the rabbit data 20:41:58 or rather, the job data in-flight 20:42:08 Well, job board would maintain the state of the flow 20:42:13 johnsom: yeah i did kinda wonder why we did an immediate ack 20:42:24 is it because a LB create can take like 10m? :P 20:42:26 Yeah, I wondered the same 20:42:31 we just need to set the timeout a little higher on rabbit 20:42:34 and then it'd like... 20:42:35 work PROPERLY 20:42:46 ie, worker dies, another can actually pick up the task after the timeout period 20:42:54 instead of it being gone forever 20:43:06 but again, i think jobboard is supposed to solve for this? harlowja 20:43:12 The benifit of job board is you are less likely to run into resource issues where the queue option can blow up with ports in use and such 20:43:23 yes, true 20:43:43 i want to say most of our operations don't have that issue, but 20:43:44 i think actually the majority do 20:43:54 don’t summon him - he will push etcd again 20:43:56 LB obviously does, Listener plugs the VIP, member plugs subnets 20:44:05 xgerman_: i'm working on the etcd3 driver myself :/ 20:44:15 it's actually a pretty good idea for smaller scale <_< 20:44:23 Xgerman_: lol 20:44:28 the UDP HM driver is great for *hyperscale* but way overkill otherwise 20:44:38 Does job board use etcd? 20:44:45 not sure -- maybe? :P 20:44:51 hahaha 20:44:59 i mean it's an openstack base service now, so 20:45:20 Ok, so on the async thing. For now, return actual state with PENDING_UPDATE? 20:45:43 IE, no change 20:45:58 i'm really voting for "no change" rather than specifically that mode 20:46:01 Yeah, just document it and make it official-ish 20:46:24 How about the rest of everyone? 20:47:11 xgerman_ nmagnezi jniesz ??? 20:47:12 yes that makes sense to me 20:47:23 JudeC 20:47:52 We are a team, so I like feedback from the whole team... Grin 20:48:32 Ok, study hall for the rest of you... grin 20:48:36 #topic Batch updates of members (rm_work) 20:48:45 yeah so, PUT sucks 20:48:55 rm_work this was your topic.... 20:48:58 right now we are using PUT for members to do what PATCH really is supposed to be for 20:49:13 we have no way to just *set* a list of members 20:49:21 which means people have to do a ton of fiddly-work 20:49:43 this leads to the case of having a ton of very quick changes in succession 20:49:53 which exacerbates things like the reload issue 20:50:21 so, any opinions on how we might do a *replace* action for members 20:51:00 #link https://specs.openstack.org/openstack/api-wg/guidelines/http.html#http-methods 20:51:03 I don't know of any APIs that actually use PATCH (nor do I know if Pecan even supports it) 20:51:14 The API-WG is not super helpful here 20:51:18 well 20:51:24 PUT is supposed to be for a full update 20:51:43 Yeah, I agree with PUT is normally full updates 20:51:44 so a PUT to /v2.0/lbaas/pools/12345/members 20:51:51 should *set the list of members* 20:52:00 actually maybe I CAN implement it just like that? 20:52:06 without breaking existing functionality 20:52:19 as I don't think we use PUT directly on the /members at the moment 20:52:21 right? 20:52:33 Yes 20:52:35 So is part of this that we should accept lists of objects? I.e. create more than one LB in one call? (I think we should) 20:52:40 yes 20:52:41 well 20:52:45 not necessarily that low 20:52:46 but 20:52:54 it's kinda part of "expanding single-actions to all objects" 20:52:59 so I can treat it as such 20:53:03 but I think PUT /v2.0/lbaas/pools/12345/members 20:53:04 is the answer 20:53:06 Correct, PUT is not valid on /members without a member ID currently 20:53:16 any objections to enabling that? 20:53:40 No. 20:53:43 I can make a BP on launchpad 20:53:44 none here 20:53:44 Not from me 20:53:54 sounds vood 20:54:11 ya, jobboard can be used for stuff like that, though imho its a bigger arch change for octavia 20:54:24 but sureeee :-P 20:54:41 harlowja We will wait for you to finish... grin 20:54:44 ha 20:55:18 Ok rm_work any more on batch updates of members? 20:55:47 #topic Open Discussion 20:56:00 A few minutes left, any other topics for today? 20:56:11 https://blueprints.launchpad.net/octavia/+spec/member-put-list 20:56:16 ^^ made that BP 20:56:28 ah and 20:56:42 is everyone OK with *codifying* officially the requirement not to use backslashes for line continuation? 20:56:50 https://review.openstack.org/#/c/471503/ 20:57:18 it's in the style guide as "preferred" but 20:57:24 IMO we are already following it 20:57:28 That is so much cleaner 20:57:30 Yeah, comment on the patch. I think it is a good thing (frankly I thought it was already a restriction) 20:57:39 backslashes are evil 20:57:43 and we should just make it official 20:57:44 yes, I was really confused when I discovered it wasn't already in place 20:57:48 as I thought it always was 20:58:07 yeah, pep8 always criticized me for backsalsh and I had to do crazy bracktes 20:58:15 I THOUGHT SO! 20:58:19 But I can't find it anymore 20:58:25 T_T 20:58:40 if anyone can find it (maybe it's just disabled by default now?) we can use that version instead of the one I cooked up 20:58:41 It must have been pulled out 20:58:53 but, mine seems to work fine 20:59:13 if you want to verify, pull down the patch, add some backslash continuation somewhere, and run the pep8 checks 21:00:04 Out of time, thanks folks! 21:00:07 #endmeeting