20:00:24 <johnsom> #startmeeting Octavia
20:00:25 <openstack> Meeting started Wed Jun  7 20:00:24 2017 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:28 <openstack> The meeting name has been set to 'octavia'
20:00:37 <johnsom> Hi folks!
20:00:40 <nmagnezi> o/
20:00:44 <JudeC> o/
20:00:54 <xgerman_> o/
20:01:05 <johnsom> Pretty full agenda today.
20:01:14 <johnsom> #topic Announcements
20:01:24 <johnsom> This week is Pike-2 milestone
20:01:30 <johnsom> #link https://releases.openstack.org/pike/schedule.html
20:01:47 <johnsom> I will be cutting our Pike-2 releases today
20:02:27 <johnsom> Also on releases, I had some stable branch releases queued, but the stable team did not like the DIB changes, so I need to work through that.  I will probably split them up.
20:02:57 <johnsom> For reference DIB changes the package that has the command to execute, requiring  a requirements.txt change which is not cool on stable, so...
20:03:18 <rm_work> umm
20:03:21 <rm_work> <_<
20:03:44 <rm_work> they can't make an exception?
20:03:49 <johnsom> Also of note, there is a lot of discussion on the mailing list about changes to the docs team.  Basically a push to move more of the docs under the project repos.  So I think that is a good thing.
20:04:01 <rm_work> aren't we already doing that?
20:04:04 <johnsom> rm_work They probably will, I just need to circle back and make the case
20:04:12 <rm_work> what more can we move under our project
20:04:20 <rm_work> AFAIK *all* of our docs are in our project already
20:04:26 <johnsom> Yes, we have been aggressively moving our docs into the repo
20:04:37 <rm_work> so, it's a NOP for us?
20:04:51 <johnsom> There is still some setup stuff in the networking guide that I hope to move into an install guide for us.
20:05:15 <johnsom> Yeah, no-op, but something to raise as all of the projects will need to start dealing with this
20:05:27 <johnsom> Plus, we have been pushing to own our own docs
20:05:58 <johnsom> Also, goals are a hot topic for the Queens series.  Fun.
20:06:20 <johnsom> We have not closed our Pike goals (py3 and wsgi) yet.
20:06:35 <johnsom> Py3 we need to shift some voting around with n-lbaas
20:06:35 <rm_work> >_<
20:06:53 <rm_work> we were the first to close our our wsgi goal... technically
20:06:59 <rm_work> *close out
20:07:15 <johnsom> Also, they changed the wsgi goal to now be running standalone with a wsgi pass through in Apache, so yeah, start over on that fun
20:07:15 <johnsom> rm_work True!
20:07:15 <rm_work> of all projects AFAIK
20:08:16 <johnsom> They changed it the day we merged
20:08:49 <johnsom> Ah well.  I'm worried about the impact to projects with these goals and them not being thought through at the start of the cycle.  It burns a bunch of time.
20:09:19 <rm_work> yes
20:09:39 <johnsom> Final announcement, due to the great work from JudeC we are about to have an openstack client plugin that has almost all of the features!
20:09:51 <johnsom> We should be able to merge the rest of it today.
20:10:04 <JudeC> :)
20:10:10 <johnsom> I will probably create a new release of that tomorrow if all goes well.
20:10:21 <johnsom> Any other announcements?
20:10:52 <johnsom> #topic Brief progress reports / bugs needing review
20:11:01 <johnsom> Any progress reports to share or hot bugs?
20:11:28 <johnsom> I see some came in about failover and the public subnet that I plan it investigate later today.
20:12:09 <johnsom> Wow, ok, nobody jump in at once.
20:12:20 <johnsom> On to the agenda items...
20:12:39 <johnsom> #topic jsonschema blueprint
20:12:46 <johnsom> #link https://blueprints.launchpad.net/octavia/+spec/jsonschema-api-validation
20:13:36 <johnsom> So it was raised that WSME is not really being actively maintained (I have not researched this) and we should consider jsonschema as an alternative.
20:13:57 <johnsom> My quick glance at jsonshema is that it isn't a full replacement for WSME.
20:14:03 <johnsom> Anybody have thoughts on this?
20:14:25 <nmagnezi> sorry i'll have to read more carefully first, so can't comment
20:14:25 <nmagnezi> :)
20:15:09 <johnsom> Yeah, no problem.  I just wanted to raise awareness and see if others had opinions on this
20:15:38 <johnsom> I did check, it is in global requirements.
20:15:59 <johnsom> I guess I will do some searches and see if other projects are using it with their API
20:16:18 <rm_work> well
20:16:29 <rm_work> I know internally we use voluptuous and I think it seems neat?
20:16:38 <rm_work> it was mentioned in one of the linked ML threads
20:16:43 <rm_work> as an alternative
20:17:14 <johnsom> Hmm, yeah, that is in global reqs as well.
20:17:27 <johnsom> I'm just not sure how pressing of a concern this is...
20:18:28 <johnsom> #topic Oslo versioned objects blueprint
20:18:36 <johnsom> #link https://blueprints.launchpad.net/octavia/+spec/oslo-versioned-object
20:18:41 <johnsom> This has come up before
20:19:04 <johnsom> My take on this is, yes, we probably should do it at some point
20:19:36 <johnsom> Again, not sure there is a pressing need until we need to contract a database schema.
20:19:44 <rm_work> right yeah
20:19:47 <rm_work> nothing is broken
20:19:50 <rm_work> when something breaks...
20:20:05 <rm_work> i mean, i don't know who happens to have the copious free time necessary to do this stuff
20:20:15 <xgerman_> yep, low priority
20:20:19 <johnsom> I know most of the folks that worked on neutron's OVO implementation are not working on OpenStack anymore, so I'm not sure there is a lot of developer resource around to work on this.
20:20:41 <xgerman_> and it’smore pressing for themn because of ROPC calls
20:21:18 <johnsom> Yeah, and rolling upgrades
20:21:42 <johnsom> Which again, so far we have only added to the schema, so...
20:22:07 <johnsom> Any reason not to approve this blueprint at a low priority in case someone wants to work on it?
20:22:15 <xgerman_> nope
20:22:32 <rm_work> i'd say go for it, but with the recommendation that we still discuss voluptuous vs jsonschema
20:22:37 <johnsom> I will probably push back a bit just to have more details added
20:22:41 <rm_work> i'm not sold on jsonschema
20:23:01 <johnsom> Yeah, the jsonschema thing I'm going to hold on for more research time
20:23:41 <johnsom> #topic Proposal to move Octavia to storyboard\
20:23:57 <johnsom> So, the storyboard folks reached out asking if we are interested in migrating.
20:24:12 <johnsom> They have run test migrations using the octavia launchpad data with success
20:24:20 <johnsom> #link https://storyboard.openstack.org/
20:24:44 <johnsom> I have not looked at it in a few years when it was first proposed
20:24:51 <johnsom> Any thoughts here?
20:25:23 <nmagnezi> what are the benefits here? how is it better compare to launchpad? (i have not looked at it)
20:25:39 <johnsom> It's nice to have developer support for the move, but I'm also not hating launchpad at the moment.  It seems to be ok for our small team needs
20:25:47 <xgerman_> sbalukoff was a bof proponent of stiryboard
20:25:58 <xgerman_> bof=big
20:26:12 <johnsom> There is some discussion here:
20:26:15 <johnsom> #link https://storyboard-blog.io/
20:26:27 <johnsom> Really, I have not looked at it too much in a long time.
20:26:44 <xgerman_> looks more modern ;-)
20:27:19 <johnsom> Maybe this is another one where we go research and come back next week ready to discuss.
20:27:37 <nmagnezi> yup
20:27:43 <rm_work> yeah
20:27:44 <rm_work> i hate to fix what's not broken
20:27:45 <rm_work> at this point
20:27:49 <rm_work> with such limited mental bandwidth
20:27:54 <johnsom> Yeah, I'm kind of in that mode too
20:28:13 <johnsom> #topic multibinder
20:28:22 <johnsom> #link https://review.openstack.org/#/c/466260/
20:28:49 <rm_work> I wonder if the automation is there for CR auto-updates / etc
20:28:58 <johnsom> So, I have kind of said what I wanted to say about this on the patch.  I -1'd it but the author put up another patch.
20:29:06 <rm_work> ah i thought he abandoned it
20:29:16 <jniesz> yea, I believe he abandoned it
20:29:36 <johnsom> Ah, he did yesterday.  Ok!  Solves that.
20:29:52 <jniesz> I tested reloads with 1 vCPU and didn't notice any impact
20:30:05 <jniesz> when members are added/removed is it called through systemctl?
20:30:09 <jniesz> the reload that is
20:30:17 <johnsom> Yeah, my analysis of the issue is our default with 1 CPU should not be impacted
20:30:28 <johnsom> Yes
20:31:06 <jniesz> johnsom, I can share some of the numbers with you that I did with benchmarks and reloading 4x per second
20:31:31 <johnsom> #topic Discuss asynchronous API and response handling (updates) (rm_work / johnsom)
20:31:44 <johnsom> jniesz Alway interested to hear folks experiences
20:32:01 <johnsom> We kind of ran out of time last week on this one.
20:32:23 <rm_work> so my take on this is that i'm ok with the method everyone else liked last time
20:32:25 <rm_work> which was that?
20:32:44 <johnsom> It was option 1 (or at least that is what I liked)
20:32:48 <rm_work> though, I've kinda run out of available time to hack at this, so this whole discussion may be a moot point again, unless other people have copious free time
20:32:51 <nmagnezi> yup.
20:32:54 <nmagnezi> 1
20:33:56 <johnsom> 1. We return the data in the pre-update form but with provisioning status "PENDING_UPDATE".  i.e. If they update the name field, the data the API will return will still have the old name but reflect that a change is in progress.  This means we return the user accurate data at all times, but can be confusing.
20:34:17 <rm_work> err wait
20:34:21 <rm_work> isn't that what we do NOW?
20:34:26 <xgerman_> yep
20:34:41 <rm_work> can you just copy/paste the rest again
20:34:49 <johnsom> Well, for the most part yes.  I think we should go through and clean up and make sure it is consistent, but yes
20:35:00 <johnsom> 2. We return the data with "faked" updates and "PENDING_UPDATE".  This way the user gets back what they just changed, but the changes may not yet actually be applied yet.  Followup GET calls will reflect the accurate state and not the "faked" data.  For example, setting the LB to admin_state_up=False would show false in the response, but the LB would still
20:35:09 <johnsom> 3. We change our revert models such that we actually update the database with their requested changes.  This means the result and future GET calls reflect the requested change, even though they may not yet be in effect.  "PENDING_UPDATE" would still apply.  To enable proper rollback to "actual state" we would need to pass the previous state data model over
20:35:15 <johnsom> 4. Make the calls synchronous.  Probably not a good idea.
20:35:24 <johnsom> 5. Don't return anything on an Update call.  This is how the new OSC commands behave.  It would mean breaking the API though, so rolling the version, etc.
20:35:41 <johnsom> That is what came off the top of my head last week
20:37:24 <rm_work> hmmm
20:37:42 <johnsom> I would probably also add a blurb in our api-ref about this too
20:37:43 <rm_work> i think i liked 3, but i do admit that is weird in its own right
20:37:54 <rm_work> so maybe just leaving it alone is better
20:38:02 <rm_work> do we know of anywhere there are inconsistencies?
20:38:17 <xgerman_> it’s confusing but that’s all
20:38:30 <johnsom> I just have a nag in the back of my head that it needs double checked.  I'm happy to do that
20:38:43 <rm_work> k
20:39:17 <johnsom> It is a bit confusing, but gives the user an accurate view.  Also confusing is getting back a change but not having the actual traffic flow reflect it yet....
20:39:46 <johnsom> I.e. doing an admin-state-up=False, getting back that result, but traffic is still flowing.
20:39:56 <johnsom> It really comes down to the meaning of PENDING_UPDATE
20:40:02 <rm_work> yeah
20:40:25 <rm_work> well, I also kinda wanted a "SYNC" command
20:40:30 <rm_work> in the operator API
20:40:35 <rm_work> to "try again"
20:40:47 <rm_work> but that doesn't work without us saving the pending state somewhere
20:40:54 <rm_work> because it's lost once the worker gives up
20:41:08 <xgerman_> is it?
20:41:08 <rm_work> if we save the state immediately, at least you can try a SYNC
20:41:28 <johnsom> Yeah.  I wonder if we shouldn't change the way we pop things off the handler queue.  Plus, job board should make that case better too
20:41:43 <rm_work> xgerman_: AFAIK yes, since it's not saved to the DB anywhere, just passed to rabbit, picked up, acknowledged, and now the only copy is in the worker's memory
20:41:45 <rm_work> yes, I think job-board could improve this
20:41:50 <rm_work> if it maintains a state of the rabbit data
20:41:58 <rm_work> or rather, the job data in-flight
20:42:08 <johnsom> Well, job board would maintain the state of the flow
20:42:13 <rm_work> johnsom: yeah i did kinda wonder why we did an immediate ack
20:42:24 <rm_work> is it because a LB create can take like 10m? :P
20:42:26 <johnsom> Yeah, I wondered the same
20:42:31 <rm_work> we just need to set the timeout a little higher on rabbit
20:42:34 <rm_work> and then it'd like...
20:42:35 <rm_work> work PROPERLY
20:42:46 <rm_work> ie, worker dies, another can actually pick up the task after the timeout period
20:42:54 <rm_work> instead of it being gone forever
20:43:06 <rm_work> but again, i think jobboard is supposed to solve for this? harlowja
20:43:12 <johnsom> The benifit of job board is you are less likely to run into resource issues where the queue option can blow up with ports in use and such
20:43:23 <rm_work> yes, true
20:43:43 <rm_work> i want to say most of our operations don't have that issue, but
20:43:44 <rm_work> i think actually the majority do
20:43:54 <xgerman_> don’t summon him - he will push etcd again
20:43:56 <rm_work> LB obviously does, Listener plugs the VIP, member plugs subnets
20:44:05 <rm_work> xgerman_: i'm working on the etcd3 driver myself :/
20:44:15 <rm_work> it's actually a pretty good idea for smaller scale <_<
20:44:23 <JudeC> Xgerman_: lol
20:44:28 <rm_work> the UDP HM driver is great for *hyperscale* but way overkill otherwise
20:44:38 <johnsom> Does job board use etcd?
20:44:45 <rm_work> not sure -- maybe? :P
20:44:51 <johnsom> hahaha
20:44:59 <rm_work> i mean it's an openstack base service now, so
20:45:20 <johnsom> Ok, so on the async thing.  For now, return actual state with PENDING_UPDATE?
20:45:43 <rm_work> IE, no change
20:45:58 <rm_work> i'm really voting for "no change" rather than specifically that mode
20:46:01 <johnsom> Yeah, just document it and make it official-ish
20:46:24 <johnsom> How about the rest of everyone?
20:47:11 <johnsom> xgerman_ nmagnezi jniesz ???
20:47:12 <jniesz> yes that makes sense to me
20:47:23 <johnsom> JudeC
20:47:52 <johnsom> We are a team, so I like feedback from the whole team...  Grin
20:48:32 <johnsom> Ok, study hall for the rest of you...  grin
20:48:36 <johnsom> #topic Batch updates of members (rm_work)
20:48:45 <rm_work> yeah so, PUT sucks
20:48:55 <johnsom> rm_work this was your topic....
20:48:58 <rm_work> right now we are using PUT for members to do what PATCH really is supposed to be for
20:49:13 <rm_work> we have no way to just *set* a list of members
20:49:21 <rm_work> which means people have to do a ton of fiddly-work
20:49:43 <rm_work> this leads to the case of having a ton of very quick changes in succession
20:49:53 <rm_work> which exacerbates things like the reload issue
20:50:21 <rm_work> so, any opinions on how we might do a *replace* action for members
20:51:00 <johnsom> #link https://specs.openstack.org/openstack/api-wg/guidelines/http.html#http-methods
20:51:03 <rm_work> I don't know of any APIs that actually use PATCH (nor do I know if Pecan even supports it)
20:51:14 <johnsom> The API-WG is not super helpful here
20:51:18 <rm_work> well
20:51:24 <rm_work> PUT is supposed to be for a full update
20:51:43 <johnsom> Yeah, I agree with PUT is normally full updates
20:51:44 <rm_work> so a PUT to /v2.0/lbaas/pools/12345/members
20:51:51 <rm_work> should *set the list of members*
20:52:00 <rm_work> actually maybe I CAN implement it just like that?
20:52:06 <rm_work> without breaking existing functionality
20:52:19 <rm_work> as I don't think we use PUT directly on the /members at the moment
20:52:21 <rm_work> right?
20:52:33 <JudeC> Yes
20:52:35 <johnsom> So is part of this that we should accept lists of objects?  I.e. create more than one LB in one call?  (I think we should)
20:52:40 <rm_work> yes
20:52:41 <rm_work> well
20:52:45 <rm_work> not necessarily that low
20:52:46 <rm_work> but
20:52:54 <rm_work> it's kinda part of "expanding single-actions to all objects"
20:52:59 <rm_work> so I can treat it as such
20:53:03 <rm_work> but I think PUT /v2.0/lbaas/pools/12345/members
20:53:04 <rm_work> is the answer
20:53:06 <johnsom> Correct, PUT is not valid on /members without a member ID currently
20:53:16 <rm_work> any objections to enabling that?
20:53:40 <johnsom> No.
20:53:43 <rm_work> I can make a BP on launchpad
20:53:44 <jniesz> none here
20:53:44 <johnsom> Not from me
20:53:54 <xgerman_> sounds vood
20:54:11 <harlowja> ya, jobboard can be used for stuff like that, though imho its a bigger arch change for octavia
20:54:24 <harlowja> but sureeee :-P
20:54:41 <johnsom> harlowja We will wait for you to finish...   grin
20:54:44 <harlowja> ha
20:55:18 <johnsom> Ok rm_work any more on batch updates of members?
20:55:47 <johnsom> #topic Open Discussion
20:56:00 <johnsom> A few minutes left, any other topics for today?
20:56:11 <rm_work> https://blueprints.launchpad.net/octavia/+spec/member-put-list
20:56:16 <rm_work> ^^ made that BP
20:56:28 <rm_work> ah and
20:56:42 <rm_work> is everyone OK with *codifying* officially the requirement not to use backslashes for line continuation?
20:56:50 <rm_work> https://review.openstack.org/#/c/471503/
20:57:18 <rm_work> it's in the style guide as "preferred" but
20:57:24 <rm_work> IMO we are already following it
20:57:28 <eandersson> That is so much cleaner
20:57:30 <johnsom> Yeah, comment on the patch.  I think it is a good thing (frankly I thought it was already a restriction)
20:57:39 <eandersson> backslashes are evil
20:57:43 <rm_work> and we should just make it official
20:57:44 <rm_work> yes, I was really confused when I discovered it wasn't already in place
20:57:48 <rm_work> as I thought it always was
20:58:07 <xgerman_> yeah, pep8 always criticized me for backsalsh and I had to do crazy bracktes
20:58:15 <rm_work> I THOUGHT SO!
20:58:19 <rm_work> But I can't find it anymore
20:58:25 <rm_work> T_T
20:58:40 <rm_work> if anyone can find it (maybe it's just disabled by default now?) we can use that version instead of the one I cooked up
20:58:41 <johnsom> It must have been pulled out
20:58:53 <rm_work> but, mine seems to work fine
20:59:13 <rm_work> if you want to verify, pull down the patch, add some backslash continuation somewhere, and run the pep8 checks
21:00:04 <johnsom> Out of time, thanks folks!
21:00:07 <johnsom> #endmeeting