15:00:42 <flaper87> #startmeeting Zaqar
15:00:43 <openstack> Meeting started Mon Jun 15 15:00:42 2015 UTC and is due to finish in 60 minutes.  The chair is flaper87. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:47 <openstack> The meeting name has been set to 'zaqar'
15:00:57 <kragniz> o/
15:01:16 * flaper87 waits few more seconds for ryan
15:02:06 <ryansb> \o
15:02:13 <flaper87> ok ok
15:02:18 <flaper87> #link https://wiki.openstack.org/wiki/Meetings/Zaqar#Agenda
15:02:25 <flaper87> That's our agenda for today
15:02:38 <flaper87> I believe we don't have pending action items from last time
15:02:53 <flaper87> lets get to our agenda right away
15:02:55 <flaper87> #topic Specs
15:03:01 <flaper87> We still have some specs to review
15:03:04 <flaper87> #link https://review.openstack.org/#/c/188662/
15:03:24 <flaper87> flwang: is not around today (NZ) but I wanted to bring that spec to your attention
15:03:43 <flaper87> SpamapS left very good comments that should simplify that spec and the implementation, I hope
15:04:03 <flaper87> There are several deployers impacts that will need to be well documented
15:04:16 <flaper87> #link https://review.openstack.org/#/c/188770/
15:04:20 <flaper87> vkmc: ^
15:04:20 <flaper87> updates ?
15:04:59 <flaper87> vkmc: ?
15:05:03 * flaper87 shakes vkmc
15:05:03 <vkmc> wait!
15:05:07 * flaper87 waits
15:05:09 <flaper87> :D
15:05:16 <vkmc> so in reply to ryansb comments
15:05:28 <vkmc> well, only for the how to delimit messages
15:05:34 * ryansb listens intently
15:05:49 <vkmc> I honestly didn't check the pubsub for Autobahn
15:06:08 <vkmc> I was delimiting messages by having an array of messages
15:06:11 <vkmc> probably not the best
15:06:28 <dynarro> o/
15:06:34 <vkmc> I could check it out thoug h
15:06:37 <flaper87> ryansb:was that comment referring to pushing notifications back to users?
15:06:42 <flaper87> dynarro: loooook who's late
15:06:57 <dynarro> flaper87: Im sorry!!!
15:07:38 <vkmc> apart from that, the messages api is for review and the claims one is done, I just need to upload it
15:07:40 <flaper87> apparently ryansb is sleeping
15:07:49 <vkmc> and finish the unit tests
15:07:50 <diga__> o/
15:07:52 <ryansb> vkmc: the thing with that is users can't parse a partial array
15:07:58 <flaper87> vkmc: good, I have (still) half of the patch reviewed
15:08:02 * ryansb is not asleep
15:08:06 <flaper87> :P
15:08:11 * flaper87 hands coffee to ryansb
15:08:14 <vkmc> ryansb, I see your point
15:08:22 <flaper87> or should I hand you something for lunch
15:08:24 <ryansb> so if you send along an array of 100 messages, they can't use it
15:08:33 <ryansb> until all of them arrive
15:08:40 <vkmc> the only thing is that Autobahn can transfer bits or plain text only
15:09:00 <vkmc> so we were returning only one response with a set of messages
15:09:05 <ryansb> whereas with a pubsub through autobahn or a delimited wire protocol, messages can be used as they come in
15:09:10 <vkmc> maybe we could decouple that and create n responses
15:09:22 <flaper87> ryansb: by sending one message at a time, though.
15:09:41 <ryansb> yes, one at a time
15:10:07 <ryansb> one way I'd like would be to use netstrings
15:10:31 <ryansb> so the format would be
15:10:49 <vkmc> sweet
15:10:57 <vkmc> you would like to encode the message or all the response?
15:11:18 <ryansb> <number of bytes><JSON of a single message>
15:11:51 <flaper87> I like the idea
15:12:04 <ryansb> I don't see why we couldn't do the whole response like so
15:12:35 <flaper87> vkmc: mind investigating on that and add it to the spec?
15:12:35 <ryansb> <num bytes><JSON about the messages that will come in, e.g. how many, etc><num bytes><JSON of message>....
15:12:53 <vkmc> flaper87, sure
15:12:55 <ryansb> so the caller could still know that it's got N more messages coming in
15:13:09 <ryansb> but could start using them as they come, instead of waiting
15:13:19 <vkmc> sounds good
15:13:35 <ryansb> and since we tell them how many bytes each chunk is, there's no escaping headaches
15:13:40 <ryansb> they just read N bytes
15:14:33 <vkmc> I'll also check for the pubsub Autobahn mechanism
15:14:33 <flaper87> That sounds good to me
15:14:38 <flaper87> vkmc: cool
15:14:57 <flaper87> #action vkmc to check netstrings for websocket responses
15:15:09 <flaper87> #action vkmc to check Autobahn's pubsub
15:15:25 <ryansb> Ok. I've never used the autobahn one, just heard it's a thing
15:15:26 <flaper87> ok, that way I make sure vkmc won't lie to me next week
15:15:29 <flaper87> :P
15:15:38 <vkmc> damn
15:15:40 <vkmc> you are good
15:15:41 <vkmc> haha
15:15:44 <flaper87> yeah, lets read about it and see which one works best
15:15:55 <vkmc> thanks ryansb for the suggestion :)
15:16:03 <flaper87> ok, anything else?
15:16:04 <vkmc> it will make the implementation much better
15:16:10 <vkmc> flaper87, review my patch
15:16:13 <vkmc> nothing else
15:16:41 <flaper87> vkmc: what about flwang ?
15:16:42 <ryansb> np
15:16:44 <flaper87> ah ?
15:16:46 <flaper87> ah?
15:16:47 <flaper87> uisssh
15:17:06 <flaper87> ok, moving on
15:17:12 <vkmc> flwang, you know what to do after reading the log
15:17:13 <flaper87> #topic Client progress
15:17:19 <flaper87> dynarro: updates ?
15:17:29 <flaper87> where are you at on your spec?
15:17:56 <dynarro> I'm fixing all the things vkmc and you commented on
15:18:11 <dynarro> I'm now working on that
15:18:23 <flaper87> #link https://review.openstack.org/#/c/190706/
15:18:28 <flaper87> FYI ^
15:18:34 <flaper87> dynarro: any blockers?
15:18:39 <flaper87> I believe Zaqar's patch landed
15:18:47 <flaper87> (the one that fixes the bug you were blocked on)
15:19:23 <flaper87> diga__: updates from you?
15:19:34 <dynarro> well I was trying all tests and i's giving me server-errors
15:19:58 <dynarro> like poolNotFound
15:20:23 <dynarro> I'm trying to figure out what changed
15:20:40 <flaper87> dynarro: ok, let us know
15:20:51 <dynarro> flaper87: count on it ;)
15:20:58 <flaper87> diga__: ?
15:21:00 <flaper87> exploreshaifali: ?
15:21:04 <flaper87> updates on the client work?
15:21:41 <flaper87> ok, moving on
15:21:49 <exploreshaifali> flaper87, diga said he is sure he will work on CLI and will complete it by 25
15:21:49 <flaper87> #topic Server updates
15:22:00 <flaper87> #undo
15:22:01 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x9ba9090>
15:22:06 <flaper87> exploreshaifali: ah ok, thanks for the update
15:22:14 <exploreshaifali> he will also cover pools task
15:22:25 <flaper87> exploreshaifali: wait, weren't you going to work on that?
15:22:42 <exploreshaifali> that is what I was about to say
15:23:00 <exploreshaifali> I have no assigned work if he will complete it
15:23:07 <flaper87> exploreshaifali: mmh, I believe there are also other things in the CLI to work on. You should split the work
15:23:22 <exploreshaifali> sure
15:23:32 <flaper87> exploreshaifali: otherwise, I already have something else for you
15:23:34 <flaper87> :)
15:23:38 <exploreshaifali> So I will take my pools task back :)
15:23:38 <flaper87> on the server side, though.
15:23:43 <flaper87> exploreshaifali: go go go
15:23:57 <exploreshaifali> okay
15:24:21 <diga__> flaper87: Hi
15:24:25 <flaper87> diga__: there you are
15:24:38 <flaper87> Any chance you can split the CLI tasks with exploreshaifali ?
15:24:44 <flaper87> since she was already looking into it as well
15:24:53 <flaper87> There's plenty to do there, TBH.
15:25:06 <vkmc> yeah, the client certainly needs some love
15:25:32 <diga__> I am now working a CLI for queue CRUD openrations
15:25:42 <exploreshaifali> diga__, what about if I will add the pools support for CLI?
15:26:51 <diga__> but As I see here queue CLI patches are there
15:26:58 <vkmc> diga__, IIRC there is queue, messages and claims
15:27:02 <exploreshaifali> https://review.openstack.org/#/c/122611/
15:27:06 <vkmc> exploreshaifali, you could take pools and flavors?
15:27:11 <vkmc> I dunno if we want cli for flavors
15:27:14 <vkmc> flaper87, ^
15:27:22 <flaper87> yes, we do!
15:27:29 <vkmc> we do
15:27:30 <vkmc> :D
15:27:37 <flaper87> We need to provide tools for OPs to manage Zaqar
15:27:38 <exploreshaifali> :D
15:27:43 <vkmc> yeah
15:28:00 <vkmc> also, I dunno if we should also consider to get out client under openstackclient
15:28:07 <vkmc> but that is for another time I guess
15:28:25 <vkmc> we are already using openstackclient so it wouldn't be so hard
15:28:30 <ryansb> vkmc: FWIW heat is a client plugin
15:28:32 <diga__> vkmc, flaper87 : anything is fine for me,
15:28:44 <ryansb> which is (IMO) a good way to go for operator tools
15:28:49 <flaper87> ryansb: what do you mean?
15:29:02 <ryansb> instead of going in-tree of openstackclient
15:29:07 <flaper87> zaqarclient is a pure library, the CLI stuff is implemented as a plugin for openstackclient
15:29:13 <vkmc> ryansb, actually that should work better
15:29:14 <ryansb> use the hooks to show up as a plugin
15:29:22 <vkmc> yes yes
15:29:24 <flaper87> ryansb: ah yeah, that's exactly how zaqarclient works
15:29:31 <ryansb> oh, k, thought you were talking about moving in-tree
15:29:39 <flaper87> no no
15:29:43 <ryansb> kk
15:29:58 <flaper87> diga__: it'd be nice if you and exploreshaifali could split the work so we can complete support there
15:30:42 <exploreshaifali> flaper87, he said he is fine in splitting work, so I will work for pools and flavors
15:30:48 <flaper87> cool
15:30:53 * flaper87 loves collaboration
15:30:55 <exploreshaifali> :)
15:30:56 <flaper87> ok, moving on
15:31:00 <flaper87> #topic Server updates
15:31:05 <flaper87> kragniz: updates on the policy stuff ?
15:31:23 * flaper87 shakes kragniz
15:31:26 <kragniz> very little done :(
15:31:39 <kragniz> the weekend was not as zaqar filled as I'd hoped
15:31:52 <flaper87> kragniz: no worries, it's good to know that there's a little done
15:31:54 <flaper87> :)
15:31:59 <flaper87> will sync again next week
15:32:04 <kragniz> a small ray of hope!
15:32:20 <flaper87> #link https://review.openstack.org/#/c/190630/
15:32:23 <flaper87> ^ that landed
15:32:36 <flaper87> that means the minimum required version for pymongo is now 3.0.2
15:32:45 <flaper87> That's good because we can focus the implementation on 1 version
15:32:48 <vkmc> <3
15:32:50 <flaper87> instead of supporting 2
15:32:52 * flaper87 happy
15:33:01 <flaper87> I'll update the patch soonish
15:33:02 <vkmc> yeah!
15:33:18 <kragniz> woo!
15:33:22 <flaper87> I have little done for the pre-signed url but I'm expecting to boost it this week
15:33:27 <ryansb> neat
15:33:43 <flaper87> Other than that, dynarro has helped a lot in finding bugs in the server
15:33:49 <diga__> sure flaper87
15:34:12 <flaper87> And I while I was setting up a zaqar with mongo for data and sqlalchemy for management I found out our mongodb controllers assume the queue controller is on mongodb
15:34:14 <flaper87> that's bad
15:34:17 <flaper87> really bad
15:34:18 <dynarro> :)
15:34:18 <diga__> exploreshaifali: I will take a pools, you can work for flavors
15:34:34 <diga__> because there is less work for queue side
15:34:47 <flaper87> we need to fix that asap so, either one of you take it or I will (Which means I'll put pre-signed URL on hold a bit)
15:34:57 <flaper87> but I'd like to fix those issues sooner rather than later
15:34:58 <ryansb> yeah, they do :(
15:35:05 <flaper87> as they impact deployment
15:35:21 <flaper87> (and usability)
15:35:24 <flaper87> ryansb: :(
15:35:31 <flaper87> that's because queue's used to be part of the data plane
15:35:45 <flaper87> not necessarily a valid excuse but it was indeed expected to work before
15:35:59 <flaper87> something we didn't review well enough, I guess
15:36:06 <ryansb> also impacts my swift patch
15:36:14 <exploreshaifali> diga__, fine :)
15:36:18 <flaper87> Anyway, I'll take that as no and fix the issue myself
15:36:21 <flaper87> ryansb: LOL, you're right
15:36:23 <flaper87> sorry about that
15:36:25 <flaper87> :D
15:36:33 <diga__> exploreshaifali: thanks !
15:36:36 <vkmc> booo, we want the swift driver
15:36:54 <flaper87> ryansb: want to share something about the driver?
15:37:00 <flaper87> I saw a patch showing up
15:37:03 <flaper87> :)
15:37:06 <ryansb> how do you want those sorts of bugs reported, btw? launchpad?
15:37:15 <ryansb> Yeah, so I have claims sorta-ish working
15:37:24 <ryansb> batch ops work (bulk_get/delete)
15:37:29 <ryansb> and ID ops work
15:37:33 <ryansb> TTLs work
15:37:45 <ryansb> queue create works, but not queue delete AFAIK
15:38:00 <ryansb> and message indices are sharded, but queues are not (yet)
15:38:02 <flaper87> ryansb: yeah, launchpad sounds good.
15:38:12 <ryansb> so it's coming right along
15:38:37 <ryansb> I do need to pick someone's brain about getting your test suite to run against it
15:39:04 <ryansb> because it seems like you have a suite per driver, so I guess I'd need to add a swift-specific test suite
15:39:23 <ryansb> but that's something we can chat about after the meeting
15:39:51 <flaper87> ryansb: there's a base suite that should be generic for all drivers
15:40:04 <flaper87> (functional suite)
15:40:13 <flaper87> The unittest one I believe is specific
15:40:17 * flaper87 doesn't even remember
15:40:20 <flaper87> :P
15:40:21 <ryansb> k
15:40:24 <flaper87> we need to refactor those tests
15:40:36 <ryansb> summary: it's going ok, but I suspect it'll be higher latency than the redis/mongo drivers
15:40:56 <flaper87> gotcha
15:40:56 <ryansb> though I think it'll scale more linearly, since Swift is good at scaling out
15:41:07 <flaper87> anything you had to do differently than other drivers?
15:41:12 <flaper87> I remember you mentioning eventlet
15:41:21 <ryansb> yeah, haven't explored eventlet yet
15:41:25 <ryansb> so right now it's the same
15:41:44 <ryansb> but before I merge it I want to add eventlet b/c it has so many HTTP calls to swift
15:42:08 <flaper87> got it
15:42:20 <flaper87> thanks for the update, looking forward to see those gates green
15:42:28 <ryansb> and swift doesn't have batch operations for some stuff
15:42:35 <ryansb> so I'd like to parallelize some calls
15:42:48 <ryansb> so having an eventlet pool would help enormously
15:43:01 <flaper87> yeah, that makes sense.
15:43:36 <flaper87> ok, that's all we have
15:43:40 <flaper87> #topic Open Discussion
15:43:51 <flaper87> weekly reminder YOU ALL ROCK!
15:44:02 * flaper87 sends cake to the team
15:44:15 <exploreshaifali> \o/
15:44:41 <flaper87> if no one has anything to add, we can call it
15:45:15 <flaper87> ok, thanks folks!
15:45:16 <vkmc> nothing else from me
15:45:18 <vkmc> !
15:45:19 <flaper87> have an amazing week
15:45:30 <ryansb> \o
15:45:31 <vkmc> thanks all, have a great week o/
15:45:33 <flaper87> talk to you next week and remember, next week the meeting is at 21 UTC!
15:45:54 <vkmc> same bat channel
15:46:03 <flaper87> #endmeeting