20:00:04 <johnsom> #startmeeting Octavia
20:00:05 <openstack> Meeting started Wed Apr 20 20:00:04 2016 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:09 <openstack> The meeting name has been set to 'octavia'
20:00:11 <bharathm> o/
20:00:15 <blogan> ^^^^^^^^^^^|||||||||||||||^^^^^^^^^^^^^^^
20:00:15 <johnsom> Hi folks
20:00:23 <fnaval> o/
20:00:24 <ajmiller> hi
20:00:30 <TrevorV> o/
20:00:43 <johnsom> blogan ???  It looks like we are playing an old text based game...
20:00:53 <minwang2> o/
20:00:53 <blogan> johnsom: its my new wave, its trademarked
20:00:58 <johnsom> Ah the good old days
20:01:08 <xgerman> blogan make sure to add TM to it
20:01:17 <johnsom> #topic Announcements
20:01:35 <johnsom> I don't have any announcements other than the summit is next week, no meeting
20:01:58 <xgerman> for those at home there will be videos to watch
20:02:01 <johnsom> Anyone else?
20:02:26 <johnsom> #topic OpenStack summit
20:02:29 <blogan> the summit is next week?!?!?!?!
20:02:37 <xgerman> in Austin!
20:02:42 <johnsom> I posted a list of session that may be of interest to us:
20:02:49 <johnsom> #link https://wiki.openstack.org/wiki/Octavia/Weekly_Meeting_Agenda
20:03:15 <TrevorV> Its my first summit guys, so I'm probably just going to follow people around like a lost puppy... :D
20:03:34 <xgerman> :-)
20:03:36 <johnsom> Hahaha
20:03:37 <blogan> good thing i'm going in disguise
20:03:56 <johnsom> Yeah, we have HPE areas to we can totally ditch you
20:04:01 <johnsom> JK
20:04:10 <blogan> just leave him face down in an alley
20:04:10 <Frito> sry for being late.
20:04:17 <blogan> Frito: you lied
20:04:18 <xgerman> also keep in mind for a lot of us this will be the last summit
20:04:23 <johnsom> Frito welcome!
20:04:33 <blogan> xgerman: what do you mean?
20:04:34 <xgerman> (since Europe travel is always tricky)
20:04:34 <TrevorV> the last summit xgerman ?
20:04:57 <blogan> oh i wont be able to go to barcelona bc of new babies
20:04:59 <TrevorV> You mean it'll be tougher to travel to future summits, or you're backing out of the project?
20:05:12 <Frito> I did lie. I joined it on the rackspace irc, not freenode
20:05:13 <Frito> oops
20:05:23 <xgerman> there will only be one more summit — then it’s dev only meetings
20:05:27 <Frito> mando is hopping on webchat now
20:05:36 <johnsom> Ah, the back channels where you guys mock us
20:05:37 <blogan> xgerman: oh thats final?
20:05:39 <TrevorV> Oh, I missed that comm, xgerman
20:05:47 <Frito> thx johnsom
20:05:50 <xgerman> blogan I think so
20:06:00 <johnsom> Yeah, it's final as far as I know
20:06:05 <xgerman> our management is treating it as a done deal
20:06:09 <fnaval> yeah, the split between dev sessions and summit
20:06:13 <blogan> your management is hoping probably lol
20:06:14 <xgerman> yep
20:06:15 <blogan> ours too
20:06:26 <blogan> honestly though, the biggest expense is hotel, not airfare
20:06:42 <xgerman> well, hence the new locations like Des Moines
20:06:51 <xgerman> or Boise
20:06:52 <blogan> but i guess if the hotels are in the touristy cities internationally then that does hurt
20:06:57 <blogan> yeah
20:07:08 <xgerman> Fargo in Winter...
20:07:08 <blogan> paris is cheap right?
20:07:29 <xgerman> I think so
20:07:30 <johnsom> HP has done Miami in July.  That was un-pleasant
20:07:49 <TrevorV> I say Hawaii
20:07:59 <TrevorV> One of the smaller islands
20:08:01 <blogan> I say we've diverged :)
20:08:02 <johnsom> I'm sure it was cheap, but wow, hot and humid
20:08:13 <blogan> digressed
20:08:14 <blogan> i mean
20:08:16 <blogan> :(
20:08:22 <johnsom> Yes.  Any other announcements/questions about the summit?
20:08:27 <xgerman> well, what I am saying we need to make Austin epic
20:08:47 <blogan> it needs to be so epic that i'm sober enough to drive back home everynight
20:09:07 <xgerman> well, if you stay long enough sessions start again
20:09:14 <johnsom> Hahaha
20:09:17 <TrevorV> If I'm riding with blogan then I'm okay to be trashed.
20:09:35 <xgerman> RAX isn’t running a shuttle?
20:09:38 <fnaval> i'm not gonna be the dd
20:09:39 <blogan> TrevorV: that will throw a wrench in plans to never see you at the summit
20:09:49 <johnsom> Yeah, but he will want to leave at like 5 so he doesn't miss Jeopardy or something
20:09:51 <blogan> xgerman: that was discussed, not sure if we are or not
20:09:51 <TrevorV> You right.
20:10:13 <blogan> i dont want to miss murder she wrote
20:10:21 <johnsom> Ok, moving on....
20:10:30 <johnsom> #topic Brief progress reports / bugs needing review
20:10:33 <fnaval> matlock
20:10:56 <xgerman> naval $1 Mio $ idea remake of Matlock as a movie
20:11:00 <TrevorV> Single create still needs someone to attack it with reviews!!!
20:11:02 <fnaval> i'm working on neutron-lbaas stuff; bugs written up; thanks for everyones help
20:11:02 <xgerman> fnaval
20:11:04 <TrevorV> #link https://review.openstack.org/#/c/257201/
20:11:21 <johnsom> Sorry folks, I have been heads down on internal papers, posters, and presentations and haven't had much time for reviews
20:11:26 <fnaval> interesting idea xgerman
20:11:28 <xgerman> finalized lab talk: https://docs.google.com/presentation/d/1AqmF3BnKLLu1W1n-XT5MSfVYrGue1JIwKSvjbWgGA6s/edit?usp=sharing
20:11:36 <fnaval> nice! xgerman
20:11:57 <xgerman> yeah, I discovered you need 50 GV HDD space on devstack rm_work
20:12:02 <fnaval> reviews needed for some simple short tests:
20:12:03 <johnsom> Yeah, we have made good progress on the LBaaS talk too.
20:12:06 <fnaval> https://review.openstack.org/164828
20:12:06 <fnaval> https://review.openstack.org/306669
20:12:06 <fnaval> https://review.openstack.org/304866
20:12:08 <fnaval> https://review.openstack.org/306083
20:12:10 <fnaval> https://review.openstack.org/306182
20:12:12 <fnaval> https://review.openstack.org/305525
20:12:15 <blogan> whoa!
20:12:18 <TrevorV> #link https://review.openstack.org/164828
20:12:22 <johnsom> Busy, that is great!
20:12:24 <TrevorV> #link https://review.openstack.org/306669
20:12:27 <Frito> I have been trying to get ramped up mostly. I've had some discussions with blogan and mando around billing and just trying to get some general headway there. It sounds like we still need some design stuff to flesh out which can be brought up later.
20:12:31 <TrevorV> #link https://review.openstack.org/304866
20:12:35 <TrevorV> #link https://review.openstack.org/306083
20:12:38 <blogan> i like how the first review # is almost half of the 2nd
20:12:40 <TrevorV> #link https://review.openstack.org/306182
20:12:42 <blogan> must be an old one
20:12:45 <TrevorV> #link https://review.openstack.org/305525
20:12:54 <xgerman> Fruit our people want to bill as well so we should sync up
20:12:57 <xgerman> Frito
20:13:01 <fnaval> #link https://review.openstack.org/164828
20:13:01 <fnaval> #link https://review.openstack.org/306669
20:13:01 <fnaval> #link https://review.openstack.org/304866
20:13:03 <fnaval> #link https://review.openstack.org/306083
20:13:05 <fnaval> #link https://review.openstack.org/306182
20:13:05 * xgerman curses at the spell checker
20:13:06 <Frito> lol @ fruit
20:13:07 <fnaval> #link https://review.openstack.org/305525
20:13:23 <fnaval> thanks anyway TrevorV
20:13:24 <TrevorV> Frito xgerman is notorious for renaming people
20:13:29 <Frito> sounds good xgerman :-D
20:13:30 <blogan> fruit, pothole, all great names xgerman has given people
20:13:38 <TrevorV> Who gave you towgan?
20:13:42 <johnsom> So true
20:13:42 <blogan> doug
20:13:46 <TrevorV> :D
20:14:13 <johnsom> #topic Open Discussion
20:14:22 <blogan> billing!
20:14:28 <TrevorV> bling!
20:14:28 <johnsom> So, the LBaaS gate is broken
20:14:34 <blogan> it is?
20:14:36 <johnsom> The current patch doesn't fix it
20:14:46 <johnsom> #link https://bugs.launchpad.net/neutron/+bug/1572592
20:14:48 <openstack> Launchpad bug 1572592 in neutron "gate-neutron-lbaasv1-dsvm-api gate broken on Neutron LBaaS: fail to find a quota" [Critical,In progress] - Assigned to Victor Stinner (victor-stinner)
20:15:01 <fnaval> ah!  cool - i thought it was my test
20:15:01 <blogan> v1?
20:15:04 <blogan> ugh
20:15:21 <johnsom> I just saw it, so don't have much to report on it
20:15:31 <fnaval> since it's down, folks can start reviewing the PRs that passed (like mine)
20:15:41 <johnsom> Yeah, v1
20:16:06 <blogan> fnaval: if your reviews got +A'ed they wouldn't be merged bc this would fail
20:16:18 <fnaval> ah ok
20:16:29 <johnsom> Yep, I'm a victim from this on a merge
20:17:51 <johnsom> Anyone want to jump on this?  Victor hasn't poked it in three hours, so I suspect he's in a different timezone
20:18:40 <johnsom> crickets
20:18:43 <johnsom> ok
20:18:48 <blogan> i'm looking
20:18:50 <blogan> can't guarantee it
20:18:58 <johnsom> Yeah, that is where I am too
20:19:00 <blogan> i think its something we can do in aparallel in discuss in channel
20:19:28 <johnsom> Ok, so, someone wanted to revive the billing discussion from a few weeks back.
20:19:40 <blogan> i must have missed that discussion
20:19:56 <johnsom> blogan You commented during it...
20:20:06 <xgerman> so I got our requirements which are basically when did the LB start; when did it end; subtraction
20:20:08 <blogan> ok i have some form of temporary amnesia?
20:20:34 <xgerman> that sort of squared with what TrevorV said
20:20:39 <johnsom> You advocated for runtime vs. uptime if I remember correctly
20:20:50 <blogan> oh yeah
20:21:00 <blogan> i guess right now i'm thinking about more implementation details
20:21:11 <Frito> Billing: blogan, mando and I were talking earlier. One thing we noticed is that when we save heartbeats from different amphorias it would overwrite the previous heart beat for a different amphoria since the listener_statistics table only has the pool_id PK
20:21:26 <blogan> listener_id
20:21:30 <Frito> Yea, that
20:21:31 <Frito> thx
20:21:42 <xgerman> yeah, heartbeats are sort of one-offish in our world
20:21:48 <blogan> xgerman: what do you mean subtraction?
20:22:00 <blogan> oh you just mean uptime
20:22:09 <blogan> yall dont need bandwidth or connection info?
20:22:12 <johnsom> Yeah, the stats stuff was kind of "implemented" without a "how it's used" plan
20:22:14 <xgerman> they want to bill for the time the system was used/up (need to nail that down better)
20:22:31 <Frito> blogan: I assume he meant listener_stopped_at - listener_created_at
20:22:58 <blogan> xgerman: my opinion is that once the loadbalancer goes active in the API thats when we started calculating uptime, if amps failover or whatever that shouldn't matter for uptime calculation
20:23:06 <xgerman> yep
20:23:11 <johnsom> Agree
20:23:21 <xgerman> and if there is a network outage we reimburse
20:23:22 <blogan> ok we need a vote
20:23:29 <blogan> i dont feel safe without a vote
20:23:33 <xgerman> lol
20:23:34 <johnsom> Well, uptime is a slightly different thing, but right general idea
20:23:35 <Frito> lol
20:24:01 <johnsom> What do you want to vote on?
20:24:06 <blogan> i'm joking
20:24:16 <blogan> sounded unanimous, at least from those present
20:24:16 <fnaval> launched_at vs created_at ?
20:24:16 <xgerman> the more interesting thing is that we need push notifications - so driver, yada, yada
20:24:33 <xgerman> let’s not split hairs on that one ;-)
20:24:36 <blogan> xgerman: can you expand on taht?
20:25:00 <xgerman> right now we push things into the labs V2 DB for the metering solution to pull with the API
20:25:14 <xgerman> what is wanted is a push into the metering solution
20:25:21 <blogan> ohhh yeah
20:25:25 <blogan> tahts what we discussed today too
20:25:46 <xgerman> and unless you guys use ceilometer we probably want a driver
20:25:46 <johnsom> launched_at should be pretty much created at I would expect.  At least database bits would be in use.
20:25:52 <blogan> yeah we want it pluggable
20:26:04 <johnsom> Yes, we need yet another driver
20:26:14 <Frito> +1 pluggable
20:26:29 <xgerman> (ideally we would get LBaaS V2 to push that so we can bill for 3rd party LBs as well)
20:26:42 <blogan> yeah that was the next question
20:27:02 <xgerman> yep, I see path of least resistance would be to just have Octavia do it
20:27:09 <blogan> is that an octavia responsibility or nlbaas, but good piont about it needing to support all nlbaas drivers
20:27:44 <johnsom> How is the "merge" spec coming?
20:27:59 <blogan> well, octavia is still goign to need to aggegrate the stats from each amphora's listener into one listener stat and push that to nlbaas, that needs to be there regardless
20:28:10 * blogan hides
20:28:25 <xgerman> yep, but should we push into metering from Octavia or LBaaS or both?
20:28:27 <blogan> johnsom: its coming this year!
20:29:05 <blogan> xgerman: i guess ideally operator's choice, octavia will need it eventually as johnsom just pointed at me about with the "merge" spec
20:29:11 <fnaval> launched_at - created_at = (time waiting for LB to become ACTIVE)  though, that's how servers does it
20:29:17 <johnsom> Probably lbaas, but write it in such a way it will copy into octavia nicely
20:29:45 <xgerman> labs would then be some metering extension… which leads us to microversioning
20:30:14 <blogan> octavia is still has more work to do for both efforts anyway, so that can start to be thought out and implemented
20:30:25 <johnsom> +1
20:30:44 <xgerman> unless we are cheap and just run a thread every second who queries the DB and puts that out on the queue
20:30:55 <xgerman> or triggers...
20:31:04 <Frito> ... don't say triggers
20:31:10 <johnsom> triggers are evil
20:31:27 <Frito> we can vote on triggers being evil if blogan still wanted a vote ;-)
20:31:45 <blogan> we need some kind of vote to complete the meeting
20:32:09 * johnsom puts his head on the table
20:32:25 <xgerman> is there a Neutron dinner this summit?
20:32:31 <blogan> anyway, would yall agree that the work on at least aggregating the metrics from amps into single listener/loadbalancer combos can get started now, and then we can just send taht off to nlbaas, bc thats gonig to have to happen anyway
20:32:36 <xgerman> or should we vote on an LBaaS one :-)
20:32:39 <johnsom> I haven't seen anything announced
20:32:41 <Frito> I think a background process that aggregates the amphorias and spits out whatever is needed for the listener is the more scalable soloution longterm. That's just my 2 cents though.
20:33:06 <johnsom> Yeah, I would personally spawn it from housekeeping manager
20:33:14 <xgerman> +1
20:33:21 <johnsom> It's all about the periodic jobs
20:33:22 <blogan> yeah that makes sense
20:33:24 <xgerman> and make it HA some way
20:33:47 <TrevorV> I agree
20:33:48 <blogan> xgerman: that might be an operator problem at that point
20:33:53 <blogan> like making the API HA
20:34:20 <xgerman> we probably need to support locking...
20:34:27 <johnsom> Well, aggregating anything makes HA harder, so, I agree we should have a plan
20:34:37 <xgerman> +1
20:34:45 <Frito> I brought that point up too xgerman :-)
20:35:00 <xgerman> great minds think alike :-)
20:35:07 <blogan> bwahaha
20:35:11 <blogan> i mean yeah
20:36:54 <Frito> I apologize for the ignorance here then. Is this something we'd then schedule a break out for at some point or just hash it out in the #openstack-lbaas channel or what? I don't want to eat up the rest of the meeting on this if there are other topics that need to be hit.
20:37:27 <johnsom> Well, this is the end of the agenda, so we can burn the time if it's a topic we need to discuss
20:37:32 <xgerman> yep
20:37:43 <Frito> Yea, I more wanted to toss that out there in case there was something else :-)
20:38:28 <Frito> I'm going to take silence as consensus and go back to billing then :-D
20:38:37 <xgerman> yep
20:38:46 <blogan> well sounds like we'll spawn a process to do the aggregation and it'll be the thign responsible for aggregating and shooting it off to nlbaas (and at some point a pluggable piece of code)
20:39:22 <johnsom> So, I'm guessing you are going to scrape the db on some interval.  HA could be done through locking and some timestamp column or something
20:39:38 <xgerman> that’s what I had in mind
20:40:14 <blogan> well one other thing to decide on, currently heartbeats just overwrite the old record, so we don't keep track of any records in history
20:40:15 <TrevorV> xgerman johnsom could that cause other API issues?  Probably not right, since the lock is per table and that table is only for heartbeats, right?
20:40:18 <blogan> do we want to continue doing that?
20:40:30 <Frito> Dumb question but we don't have a locking store? The only thing I've used in the past was Redis for it's atomic get / set operation. I'm not sure if that's suited here though.
20:40:49 <xgerman> we use mysql which locks
20:40:54 <johnsom> Yeah, depends on what you lock.  Could be rows(I think mysql can do that) or an aggregation table
20:41:04 <Frito> got ya.
20:41:32 <Frito> Okay. So the "shooting it off" bit from earlier. How do we go about defining the "it"
20:41:56 <Frito> Is that a pattern we already have somewhere (like v1) or would this be a new chunk of info?
20:42:01 <johnsom> Probably could borrow from the build rate-limiting patch that is up for review
20:42:35 <TrevorV> Frito honestly we can pull from the jinja stuff for the REST client for the amphora configuration.
20:42:57 <blogan> what?
20:42:58 * Frito note to self: find that patch
20:43:00 <xgerman> ??
20:43:08 <johnsom> Lost me Frito.  Are you asking about the nlbaas side sending to somewhere?
20:43:32 <johnsom> #link https://review.openstack.org/#/c/303304/
20:43:36 <TrevorV> That's the communication contract after rolling the stats up for a period of time, right Frito ?  That's what you're talking about
20:43:45 <blogan> yeah there's gonig to be 2 pieces that shoot, octavia to someplace, and nlbaas to some place
20:44:00 <johnsom> blogan right.
20:44:06 <TrevorV> He's talking about the ammunition, "what" is being shot.  Not where its going.
20:44:13 <TrevorV> I think.
20:44:15 <TrevorV> o_0
20:44:19 <blogan> octavia to someplace can be to nlbaas
20:44:22 <johnsom> octavia up to nlbaas, like event streamer?
20:44:42 <xgerman> likely exactly like event streamer
20:44:58 <blogan> johnsom: yeah thats it for now, but i can see that becoming just a plugin in octavia at some point, but don't need to worry about it now
20:45:02 <johnsom> nlbaas to celio or whatever, not sure on that one.  Probably something new
20:45:12 <blogan> however, it will need tobe modified now to aggregate, and be done in the housekeeping manager process
20:45:16 <blogan> spawned
20:45:53 <johnsom> blogan good idea, make it a plugin, current implementation is to nlbaas, but should be able to plug in the nlbaas code if needed/in future
20:46:00 <xgerman> +1
20:46:06 <johnsom> +1
20:46:15 <xgerman> and that allows people to only send a subset suited to their needs
20:46:31 <mando_d> Frito is having IRC issues
20:46:56 <xgerman> don’t tell ma about issues - our DC in Houston flooded
20:46:58 <johnsom> Ugh, we just totally solved his problem...  grin
20:47:29 <blogan> i think i've got a good idea on waht we're talkinga bout, the details will need to be worked out but thats probably good enough to start implementing and discussing those as they come up
20:47:30 <xgerman> hope the irc server is up on a hill :-)
20:47:55 <johnsom> Ok, cool
20:47:56 <xgerman> yep, we probably should write a spec to make sure we are all on the same page
20:47:56 <TrevorV> blogan yeah, we'll have to put those details in our tickets
20:48:09 <TrevorV> Yeah, spec would be good for upstream
20:48:22 <xgerman> +1
20:48:24 * blogan burns specs
20:48:27 <blogan> whoops
20:48:29 <fnaval> +1
20:48:38 <ajmiller> +1
20:48:48 * johnsom hands blogan a chisel and stone slab
20:48:50 <blogan> all your +1's are for burnign specs
20:49:00 <xgerman> #voye?
20:49:29 <xgerman> #startvote Should we burn the spec? Yes, No
20:49:30 <openstack> Only the meeting chair may start a vote.
20:49:39 <TrevorV> #vote yes,
20:49:43 * johnsom slaps xgermans hand
20:49:44 <TrevorV> :P
20:49:49 <Frito> back. sorry. The wireless at my desk decided to go retarded.
20:50:18 <johnsom> Frito we finished designing it.  blogan volunteered to write the design spec.
20:50:38 <Frito> Works for me
20:50:39 <Frito> Lol
20:50:46 <johnsom> grin
20:51:05 <Frito> Hey, if that's untrue that is between you guys ;-)
20:51:21 <blogan> johnsom, Frito: no!
20:51:26 <Frito> lmao
20:51:43 <blogan> i still have that other spec to write
20:51:45 <blogan> one day
20:51:48 <Frito> We're getting close to time anyways and that discussion felt like it could go on for way longer.
20:51:50 <blogan> that i said i'd ahve done like last month
20:52:18 <johnsom> Yeah.  Anyway, I think we have a good start.
20:52:29 <TrevorV> Frito you can check the meeting notes after for what was discussed, and the rest is "implementation details"
20:52:40 <TrevorV> Interest in saving time I mean
20:52:41 <johnsom> Any other topics?  (btw, Frito you can check the meeting log for what you missed)
20:53:15 <Frito> topic: have a great rest of your day ;-)
20:53:23 <johnsom> Ok, then I will see you in person next week.
20:53:26 <johnsom> #endmeeting