14:03:36 <dougwig> #startmeeting neutron lbaas
14:03:37 <openstack> Meeting started Thu Jul  3 14:03:36 2014 UTC and is due to finish in 60 minutes.  The chair is dougwig. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:38 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:03:40 <openstack> The meeting name has been set to 'neutron_lbaas'
14:03:41 <ptoohill_> o/
14:03:49 <jorgem> there we go
14:03:56 <enikanorov> dougwig: you'll need to #endmeeting afterwards
14:03:56 <sballe_> :)
14:04:00 <jorgem> I was looking for the proper topic name
14:04:08 <jorgem> lol
14:04:12 <dougwig> enikanorov: thanks
14:04:30 <dougwig> stealing, not steering.  weird autocorrect.
14:04:31 <jorgem> Alright so all i have on the agenda is what Brandon put on the ML
14:05:10 <xgerman> go blogan!
14:05:13 <jorgem> 1) Are shim layers really needed for Juno?
14:05:13 <jorgem> - If the old API and new API will coexist independently, why is a
14:05:13 <jorgem> shim layer needed?
14:05:13 <jorgem> - Has the caveat that the pools resource can exist
14:05:13 <jorgem> independently in both APIs.  This can be accomplished by
14:05:14 <jorgem> renaming new API pools to something different or by doing a
14:05:14 <jorgem> shim for the pool resource.
14:05:15 <jorgem> 2) Should the agent refactoring be included in the main object
14:05:15 <jorgem> object model refactor?  The plugin might be able to just call the
14:05:16 <jorgem> namespace driver directly, with some modification.
14:05:16 <jorgem> 3) Status of entities that only exist in the database and not in
14:05:17 <jorgem> a backend (i.e. they are not linked to an existing load balancer)
14:05:25 <dougwig> #topic shim layers
14:05:35 <jorgem> would anyone like to add anything to the agenda before we start?
14:06:11 <blogan> update on the nodepools, I've found a decent way to allow both API versions to have pools
14:06:26 <vjay> Yes. I want to add one more to it. what about enhancements to drivers
14:07:21 <jorgem> So looks like we are trying to decide on a direction here.
14:07:25 <samuelbercovici> would like to add exepctation for when the "core" code will land in gerrit
14:07:26 <markmcclain> blogan: cool
14:08:05 <jorgem> blogan: Would you like to enlighten us?
14:08:40 <blogan> jorgem: it'd be a long conversation and dont want to take up too much time
14:08:59 <blogan> the code will be up in gerrit soon to review
14:09:29 <blogan> markmcclain: do you think the shim layer is needed if both api version can live independently at the same time?
14:09:53 <blogan> at least is a shim layer needed until the old version is removed?
14:10:03 <dlundquist> So is the intention to land both persistence layers side by side in Juno?
14:10:04 <vjay> jorgem: i want to add one item to agenda. Can the existing drivers continue to be enhanced in Juno release?
14:10:12 <markmcclain> the idea of the shim is that the data model coverges now
14:10:14 <jorgem> vjay: added
14:10:21 <dougwig> blogan: how will that mesh with out plan to not update horizon, but instead let it hit the old entry points?
14:10:22 <markmcclain> s/coverges/converges/
14:10:28 <dougwig> s/out/our/
14:10:49 <blogan> markmcclain: you mean they converge on just name?
14:11:25 <markmcclain> no.. the actual internal representation
14:11:36 <blogan> dougwig: i dont see that affecting anything, the old api still serves requests like it always has and responds like it always has
14:11:48 <jorgem> dougwig: Wouldn't horizon just choose one of the versions? Preferably the new one.
14:12:02 <blogan> markmcclain: well there are separate models and separate tables for both versions
14:12:09 <dougwig> right, but which set of drivers?  no shim, old entry points, presumably it only ever uses old drivers?
14:12:19 <dougwig> (w.r.t. horizon)
14:12:21 <jorgem> dougwig: Or rather, horizon would update eventually and still use v1 which will still work?
14:12:54 <xgerman> separate data models worry me. We will end up instead of a migration with some merging
14:12:57 <samuelbercovici> was there a decision not to fix horizon to use the new api?
14:13:10 <markmcclain> blogan: so what's the upgrade path is we have separate versions?
14:13:28 <markmcclain> s/is/if/
14:14:13 <blogan> markmcclain: i would say no upgrade path is needed for juno, but once the old API is actually removed there will need to be a shim of some kind and a data migration script for existing deployments?
14:14:33 <blogan> markmcclain: i could be totaly missing something so please tell me if I am all wrong
14:14:59 <xgerman> so we won't allow running both versions in parallel?
14:15:02 <samuelbercovici> repeat: was there a decision not to fix horizon to use the new api?
14:15:10 <blogan> dougwig: the old drivers remain in v1 but the driver maintainers should know that they need to upgrade their drivers to the new version
14:15:30 <blogan> samuelbercovici: that hasn't been discussed yet, as I said in the email
14:15:31 <markmcclain> blogan: we could go that route, but I'd prefer us to not need to support 2 different internal paths for 12-15 mos
14:15:40 <markmcclain> hence the shim
14:15:45 <samuelbercovici> blogan:10x
14:16:01 <jorgem> xgerman: I was imagining both v1 and v2 would run in parallel with separate drivers. v1 would be marked as deprecated which would give anyone a cycle or two to move to v2
14:16:15 <dougwig> blogan: right, but the old horizon plan as i understood it was to let it call v1, the new model would figure out which driver to call (old or new), and then functionality wouldn't change until horizon was updated.  this plan means only old functionality, old drivers, until horizon is updated, which wasn't planned for juno?  so new drivers are just api only?
14:16:42 <blogan> markmcclain: wouldn't it be akin to having an actual v2 of the API that still works but isn't supported any longer?
14:16:42 <jorgem> xgerman: thus, no new dev work would be done on v1 during its deprecated life. Only major bug fixes.
14:17:45 <markmcclain> we can have side-by-side for now, but after we get working code I'd really like to see a v1 rest api shim
14:18:10 <xgerman> me, too, writing to two sets of DB tables worries me for the migration
14:18:16 <sballe_> markmcclain, +1
14:18:21 <blogan> markmcclain: ok that can be done, I'm just worried about thousands of lines of code hitting in one blueprint
14:18:33 <markmcclain> blogan: understood
14:18:47 <samuelbercovici> blogan:+1
14:19:20 <jorgem> xgerman: Could you elaborate? I thought we were trying to create interfaces that allowed for v2 to be separated easily?
14:19:35 <samuelbercovici> markmcclain: would you suggest trying to do both new api + shims in one or have it in two or more steps?
14:19:46 <TrevorV_> Just for understanding, an "api shim" is a translation layer between v1 and v2 calls right?
14:19:51 <blogan> dougwig: are you saying we agreed that horizon would be updated as well for the new API?
14:19:52 <xgerman> ok, if we have v1->v1 tables in the DB, and v2->v2 tables in the DB we eventually need to merge that
14:20:13 <xgerman> it's easier to merge into an empty table set
14:20:22 <blogan> TrevorV_: yes and another shim to translate new model to old model for old drivers to understand
14:20:29 <dougwig> blogan: i think your suggestion means that either we accelerate juno to use the new api, and we must get all the drivers updated, or new drivers will not be exposed in horizon in juno.
14:20:51 <markmcclain> samuelbercovici: two steps would be acceptable
14:20:53 <dougwig> /accelerate juno/accelerate juno horizon/
14:21:52 <samuelbercovici> so can we agree to split in two steps. this will release work dependent on availability of new model and hopefully reduce the review cycle
14:21:57 <blogan> dougwig: ok I understand now, that is something that should be discussed then
14:22:19 <samuelbercovici> fo the 1st step
14:22:35 <sballe_> dougwig, My experience with Horizon is that the merge into Horizon cannot be "accelerated". It rakes a lot of time ot get changes accpeted into Horizon :(
14:22:52 <sballe_> s/takes
14:22:57 <markmcclain> sballe_: amotoki is a core on both Neutron and Horizon
14:23:20 <samuelbercovici> markmcclain: besides amotoki, is there anyone else in horizon that can assist?
14:23:27 <sballe_> markmcclain, Understood that might help
14:23:28 <markmcclain> sballe_: once we have code ready for horizon he's usually helped us get the needed reviews
14:23:31 <samuelbercovici> assuming someone else implements
14:23:34 <jorgem> sballe_: I agree. As long as Horizon doesn't break by adding v2 I think we should be okay. From our dev with our GUI team here that is totally doable.
14:24:02 <jorgem> we release features via API all the time and our GUI eventually catches up.
14:24:10 <samuelbercovici> jorgem: do you have contirbuturs to horizon?
14:24:11 <sballe_> jorgem, +1
14:24:16 <dougwig> i'm fine with splitting the review, if the shim or horizon is still targeted for juno.  maybe there's precedent for the api/horizon being out of sync for a major release?  seems odd to me.
14:24:26 <markmcclain> jorgem: my guess is horizon will need to have both v1 and v2 support selectable by config, but we should engage the horizon team to understand their preference
14:24:29 <jorgem> samuelbercovici: I think we do. I can try to hunt some of them down.
14:24:51 <markmcclain> I think if we can get v2 by juno-2
14:24:59 <markmcclain> then we can work on the shim and horizon support in j3
14:25:01 <sballe_> I don't see horizon as crucial for juno. All the new features always start with cli only and then horizon catches up. It is not like v2 is going to be GA anyway by juno.
14:25:05 <samuelbercovici> jorgem: assuming the api will be very soon available, can you get someone from them to commit to do the ui?
14:25:20 <blogan> dougwig: you really are just talking about the shim layer that will translate new model to old model for all drivers correct? so I think that is something that can be worked on in a parallel blueprint and make it in juno
14:25:29 <jorgem> samuelbercovici: I can try. No guarantees though! :)
14:25:41 <samuelbercovici> jorgem: cool
14:25:43 <xgerman> bloga - that would be great
14:25:44 <dougwig> blogan: yes, the backend shim, not the frontend.
14:25:49 <jorgem> Does anyone else have Horizon devs?
14:26:02 <TrevorV_> worst case, jorgem, one of our team can focus some effort into Horizon-related code.
14:26:08 <sballe_> jorgem, we have some but they are working on other projects
14:26:08 <xgerman> same here
14:26:49 <jorgem> #action Engage with Horizon devs for feasibility of development
14:27:02 <jorgem> Shall we move on to next item?
14:27:04 <blogan> #action create separate blueprint from backend shim
14:27:13 <samuelbercovici> also should put as action split the work to core and shim
14:27:26 <xgerman> +1
14:27:28 <blogan> samuelbercovici: i read your mind
14:27:43 <dougwig> blogan: i'll get with you later today about the shim.
14:27:47 <samuelbercovici> blogan: yep..as demonstarted before and faster than light
14:27:48 <jorgem> okay lets move to next topic.
14:27:49 <dougwig> #topic Should the agent refactoring be included in the main object object model refactor
14:27:50 <blogan> dougwig: ok sounds good
14:27:54 <jorgem> 2) Should the agent refactoring be included in the main object
14:27:54 <jorgem> object model refactor?  The plugin might be able to just call the
14:27:54 <jorgem> namespace driver directly, with some modification.
14:28:22 <jorgem> blogan: What are your thoughts on this?
14:28:33 <blogan> so this is also related to keeping the review more manageable
14:28:54 <markmcclain> so I these can be a series of reviews
14:28:55 <samuelbercovici> well. looking at some vendor's implementation they look like using the agent so this might make sense to have a new agent for the new api
14:28:59 <markmcclain> s/I//
14:29:48 <blogan> markmcclain: do you mean the first review will be the core refactor, the second the shim, the third the agent
14:29:54 <blogan> or some other order
14:30:06 <dougwig> (or the second and third in parallel)
14:30:16 <markmcclain> I'm indifferent on the order of 2 & 3
14:30:53 <blogan> me too
14:30:58 <samuelbercovici> my only concern is that it will be difficult to get tempest tests working without a reference implementation
14:31:12 <markmcclain> I think core, agent, shim seems a little more natural given the current work in the pipeline
14:31:13 <blogan> okay so separate blueprint for agent refactor/new agent
14:31:28 <markmcclain> samuelbercovici: +1
14:31:35 <dougwig> #action separate blueprint for agent refactor/new agent
14:32:05 <dougwig> move on, or any other discussion on this point?
14:32:06 <blogan> samuelbercovici: I think the reference implementation can be modified fairly easily to not use the agent, but I would need to do some more research on that for sure
14:32:53 <dougwig> #topic Status of entities that only exist in the database and not in a backend
14:32:54 <samuelbercovici> blogan: idealy if we get the first commit to work end to end without major effort, it might be resonable to do so
14:33:08 <blogan> #action implement namespace_driver to work with new data model
14:33:19 <samuelbercovici> and then fix it with a "new" agent
14:33:31 <blogan> samuelbercovici: indeed
14:33:52 <blogan> so this next topic, I know we discussed at the midcycle meetup, but I don't know if we came to a definite decision
14:34:25 <blogan> its a question of should an entity that only exists int he database have an initial status of PENDING_CREATE or ACTIVE
14:34:47 <samuelbercovici> blogan: should such entity even have a status?
14:35:09 <xgerman> which entities do you have oin mind?
14:35:09 <blogan> samuelbercovici: once it is linked to a load balancer it probably does
14:35:16 <samuelbercovici> ic
14:35:34 <blogan> xgerman: for example, a user creates a pool, but since that pool is not linked to a listener yet, should that status be ACTIVE or PENDING_CREATE
14:35:36 <dougwig> if you have a listener with no LB, and mark it ACTIVE, and then pass  the LB and listener to a driver later, and the listener creation fails, it would then go from ACTIVE to ERROR?
14:35:55 <jorgem> blogan: We might have to make a new status
14:35:58 <blogan> dougwig: thats what I'm currently doing, but I wanted to make sure everyone agreed
14:36:06 <jorgem> blogan: Both of those are misleading
14:36:09 <samuelbercovici> well. the way the model is shaping is that there is no reuse so entities created should be linked to a living object hence not sure if you can get to such a state
14:36:17 <xgerman> jorgem +1
14:36:18 <sballe_> jorgem, +1 I agree it looks like this is a new status
14:36:30 <TrevorV_> PENDING_CREATE in the case that it has yet to be driver-created doesn't sound misleading to me...
14:36:52 <dougwig> if the status is whether it's in the db and backend, pending_update is appropriate.  what is the intent of the current state definitions?
14:36:56 <samuelbercovici> can you create a listener which is not connected to an lb?
14:37:14 <xgerman> What about "SCHEDULED"?
14:37:30 <blogan> samuelbercovici: yes you can
14:37:34 <dougwig> samuelbercovici: yes, as part of the eventual shift to many-to-many links (which i do not like, but still, if it's headed that way, it has to be a first level object.)
14:37:41 <sballe_> SCHEDULED is mis-leading.
14:37:46 <samuelbercovici> oh, ic
14:37:49 <TrevorV_> samuelbercovici: I was under the impression we were supporting the creation of every major object in an isolated fashion... Without the necessity to point it to another object
14:38:10 <jorgem> So one thing that relates to statuses is locking. We need to lock entities from multiple updates at the same time. The more statuses we have the more complicated the locking mechanism becomes.
14:38:17 <sballe_> I would suggest something more like PENDING_ACTIVE so it is pending further work but the part that is done is ACTIVE
14:38:30 <samuelbercovici> TrevorV_: theoreticaly, yes, but i thought there was a decision to not support many to many
14:38:41 <blogan> sballe_: what about QUEUED?
14:38:54 <sballe_> QUEUED would work.
14:38:57 <xgerman> +1
14:39:03 <xgerman> shorter is better
14:39:07 <blogan> sameulbercovici: there was a decision that enough people want it then implement it after
14:39:15 <TrevorV_> samuelbercovici: initially yes, but the desire for it shows that we'll probably have an update to support that in the future.  Don't quote me though :D
14:39:19 <sballe_> but it doesn't reflect the fact taht you already have somethign taht has completed
14:39:27 <dougwig> is pending_create already defined to mean this?  by that i mean, are we inferring too much from it's name, but the state really exists to be exactly what we're talking about here?
14:39:30 <jorgem> sballe_: I vote for QUEUED over SCHEDULED since that is the reality of it. It also tells the user that we actually aren't provisioning yet.
14:40:07 <TrevorV_> I think QUEUED is misleading, since its not in a queue... but its better than SCHEDULED
14:40:08 <samuelbercovici> can we do the "state" discusion in ML?
14:40:20 <dougwig> works for me.
14:40:32 <samuelbercovici> who will send a proposal?
14:40:45 <blogan> dougwig: I think its a matter of having two different states so its obvious when a driver is actually provisioning it
14:41:08 <samuelbercovici> to initiate the discussion
14:41:17 <markmcclain> yes please send this to the ML
14:41:17 <dougwig> blogan: I know that's the intent, but is that different from every other neutron object?  i.e. are we reinventing the wheel here, because we don't like the flow/name?
14:41:25 <dougwig> i'm actually fairly neutral on the state name.
14:41:41 <dougwig> #action move state transition discussion to ML
14:41:41 <sballe_> me too as lon gas it is not misleading
14:41:42 <xgerman> as long as it's explained in our glossary...
14:41:43 <markmcclain> as there needs to broad community agreement on these terms since it impacts FW, VPN too
14:41:47 <blogan> dougwig: but i don't think neutron has this same problem
14:42:08 <dougwig> #topic update on the nodepools
14:42:24 <jorgem> next item was "Can the existing drivers continue to be enhanced in Juno release?" I thought?
14:42:28 <jorgem> per vjay
14:42:41 <dougwig> that's two down on my list, but i can shift it
14:42:47 <dougwig> #topic Can the existing drivers continue to be enhanced in Juno release
14:42:49 <blogan> dougwig: we already discussed the nodepools
14:42:58 <jorgem> gotcha
14:43:37 <dougwig> vjay: want to get this one started?
14:44:01 <vjay> Yes.
14:44:18 <vjay> we want to do some enhancments to the current driver.
14:44:24 <vjay> like status updates and stats update
14:44:35 <vjay> can we post these enhancments to gerrit?
14:44:37 <jorgem> vjay: Current reference driver?
14:44:47 <vjay> current NS driver
14:45:28 <ptoohill> vjay, this is ontop of the BP to modify for updated objmodel right
14:45:32 <sballe_> vjay, NS?
14:45:42 <vjay> oops
14:45:45 <vjay> sorry NetScaler driver
14:45:52 <sballe_> vjay, thx :-)
14:46:14 <vjay> currently the pool status will be ACTIVE always, it does not reflect the status as the backend sees it.
14:46:59 <ptoohill> ah
14:47:02 <dougwig> this goes back to the shim v1/v2 simultaneous conversation.  if we don't support both versions of drivers in some fashion, the transition path gets trickier.  (and i'm fine moving this discussion to an offline chat about that new blueprint.)
14:47:57 <markmcclain> my preference is that we don't enhance v1 drivers
14:48:11 <sballe_> markmcclain, +1
14:48:18 <jorgem> markmcclain: +1
14:48:31 <xgerman> +1
14:49:05 <vjay2> or for ex. the calls to the backend are becoming Asynchronous. So we have to accomodate this in the current NetScaler driver
14:49:12 <vjay2> can we submit such changes?
14:49:12 <jorgem> Can drivers get deprecated in Neutron LBaaS?
14:49:28 <jorgem> As in, is there a formal way to do this?
14:49:35 <blogan> vjay2: would you be able to modify your driver to support the new object model and driver interface?
14:49:39 <dougwig> vjay2: i don't think it's getting any more async than it currently is.
14:50:01 <blogan> dougwig: i think he is talking about the netscaler backend becoming async
14:50:02 <vjay2> no, the call between the driver and backend is async
14:50:16 <vjay2> blogan: right
14:50:55 <markmcclain> jorgem: yes we can deprecate
14:51:09 <dougwig> ahh, got it.  that would imply a major driver change, which would imply being able to switch to the new interface?
14:51:10 <markmcclain> and the cycle is much shorter than REST
14:51:37 <vjay2> blogan: we are planning for the new object model, but we cannot say for sure.
14:51:53 <dougwig> vjay2: can we chat offline on this topic in #openstack-lbaas in about 2 hours?
14:52:02 <dougwig> you, me, blogan, and anyone else interested?
14:52:12 <vjay2> that should work
14:52:15 <blogan> i just got voluntold
14:52:23 <dougwig> ok
14:52:27 <dougwig> #topic when the "core" code will land in gerrit
14:53:05 <dougwig> i think this one was from sam
14:53:13 <blogan> this will land soon, we need to get the "db" unit tests finished
14:53:57 <blogan> and by soon I mean in the next couple of working days
14:53:57 <dougwig> samuelbercovici: if you need it sooner, it's in a github fork right now.
14:54:13 <blogan> of course we have a major holiday coming up here in the states so it may end up being next week
14:54:37 <dougwig> #topic free play
14:54:40 <blogan> but I will try to get it done sooner
14:54:41 <TrevorV_> couple of days >= 3 to 5
14:54:49 <samuelbercovici> I am asking, as we are trying to work on TLS and L7 in pralel
14:54:56 <sballe_> can somebody point me at the github repo
14:55:03 <jorgem> As devs always say "I'm about 90% complete" lol
14:55:29 <dougwig> #link https://github.com/orgs/oslbaas/dashboard
14:55:37 <sballe_> dougwig, thx
14:55:49 <samuelbercovici> so TLS which is simple to modify to the new model is alreadi as WIP in git. Evgeny is on vacation next week so it will be fine if sometime next week the code will land in gerrit
14:55:54 <ptoohill> would like to bring up the outstanding BP's : https://review.openstack.org/#/c/98640/ and https://review.openstack.org/#/c/100931/. This need reviewing
14:56:17 <TrevorV_> #link https://review.openstack.org/#/c/98640/
14:56:24 <blogan> samuelbercovici: ok that sounds good, but I'll still try to get it out by this weekend
14:56:31 <TrevorV_> #link https://review.openstack.org/#/c/100931/
14:56:59 <jorgem> Update from last weeks action items. I updated https://wiki.openstack.org/wiki/Neutron/LBaaS. Please add new links there.
14:57:02 <samuelbercovici> so on https://review.openstack.org/#/c/98640/ - mark can you please review?
14:57:55 <dlundquist> There are additional blueprints still pending listed here: https://etherpad.openstack.org/p/juno_lbaas_mid_cycle_meetup_reviews
14:57:56 <avishayb> and L7 rst is here: https://review.openstack.org/#/c/99709/10
14:58:18 <dougwig> #link https://etherpad.openstack.org/p/juno_lbaas_mid_cycle_meetup_reviews
14:58:27 <jorgem> I also added relevant bps to the wiki links section
14:58:29 <dougwig> #link https://review.openstack.org/#/c/99709
14:58:33 <jorgem> for juno that is
14:58:34 <dougwig> about a minute left in this meeting.
14:59:38 <jorgem> alright looks like we are done. Thanks everybody!
14:59:39 <dougwig> alright, bye folks.
14:59:40 <blogan> i like turtles
14:59:46 <xgerman> ?
14:59:57 <rm_work> o/
14:59:59 <samuelbercovici> bye! it was a good one!
15:00:03 <dougwig> #endmeeting