21:07:56 <kgriffs> #startmeeting zaqar
21:07:57 <openstack> Meeting started Mon Sep 29 21:07:56 2014 UTC and is due to finish in 60 minutes.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:07:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:08:00 <openstack> The meeting name has been set to 'zaqar'
21:08:08 <zhiyan> o/
21:08:27 <kgriffs> o/
21:08:29 <zhiyan> kgriffs: ok, no problem.
21:08:38 <zhiyan> kgriffs: thanks to let me know it.
21:08:51 <kgriffs> so, i believe we skipped the meeting last week?
21:08:55 <kgriffs> so no action items?
21:09:33 <zhiyan> kgriffs: actually i'd like to know what's the idea on swift backend support ^^
21:10:01 <kgriffs> ok, just a sec
21:10:14 <zhiyan> oops. you first.
21:10:46 <kgriffs> ok, so we can chat about the swift thing. flwang1 did you have anything you would like to add to the agenda?
21:11:05 <flwang1> kgriffs: integration with other components
21:11:09 <kgriffs> ok
21:11:24 <kgriffs> #topic swift driver
21:11:32 <flwang1> kgriffs: for now, we're focusing on horizon and heat
21:11:42 <kgriffs> flwang1: kk, we'll get to that in a minute
21:11:49 <kgriffs> so the swift driver
21:12:30 <kgriffs> this is something Flavio is driving, but the idea has been floated a few times in the past; we just haven't had the bandwidth to look into it seriously yet.
21:12:47 <kgriffs> it sounds like Flavio is proposing doing so during the next cycle
21:13:05 <kgriffs> it won't be the fastest, but it will be the most durable/reliable
21:13:21 <zhiyan> kgriffs: ok, so seems team like this idea as one of goal in K?
21:13:46 <kgriffs> i think the first step is a POC to see how well it maps to the problem domain
21:14:01 <flwang1> kgriffs: +1
21:14:09 <kgriffs> it would be good to get that done early so we can have time to flesh it out if it looks good
21:14:40 <zhiyan> kgriffs: +1. so do you think it could effective improve durable/reliable for zaqar/message?
21:15:15 <zhiyan> kgriffs: ok, got your position. will talk with flaper87 on it more later.
21:15:59 <zhiyan> kgriffs: and probably he could talk more on this for his idea in then meeting he could join
21:16:00 <kgriffs> zhiyan: yeah, swift would be nice if we can get it to work and have reasonable performance, since it would handle replication and HA
21:16:18 <kgriffs> zhiyan: kk. Hopefully he will be able to make the meeting next monday
21:16:23 <zhiyan> kgriffs: yep, built-in replic with HA in cluster
21:16:30 <zhiyan> kgriffs: thanks.
21:16:33 <zhiyan> kgriffs: btw
21:16:36 <kgriffs> 1500 UTC next monday
21:16:42 <kgriffs> (we alternate times, FWIW)
21:16:50 <zhiyan> kgriffs: oh, nice to know it!
21:16:54 <kgriffs> i know there are tricks we can use if need be to make things work, like embedding some metadata in the object names so you can get it quickly without having to actually read each object.
21:17:00 <kgriffs> s/tricks/performance tricks
21:17:11 <kgriffs> anyway
21:17:21 <kgriffs> flwang1: did you have any other thoughts to add on the topic?
21:18:07 <flwang1> kgriffs: not really, but I'm really like to see it can happen in K, since it would fix many concerns from CT :)
21:18:09 <zhiyan> kgriffs: i think we need think more things on design level, e.g. how to use container and object to effective organize message store.
21:19:01 <kgriffs> yep. some careful thought up front could pay big dividends
21:19:39 <flwang1> kgriffs: you know what 'CT' means ;)
21:20:07 <kgriffs> #topic integration
21:20:26 <zhiyan> kgriffs: btw, may i ask two quick feedbacks on https://wiki.openstack.org/wiki/Zaqar/specs/api/v1.1#Changes_from_v1.0 ?
21:20:26 <zhiyan> kgriffs: oops
21:20:43 <kgriffs> zhiyan: sure, we can get that in a minute
21:21:15 <kgriffs> flwang1: you wanted to discuss heat, horizon?
21:21:53 <zhiyan> two items on message/stats i think
21:22:15 <flwang1> kgriffs: I think vmkc is working on horizon
21:22:37 <flwang1> so I'd like to know if there is any body working on heat, and see what i can hlep
21:23:56 <zhiyan> 1. do you think we can add a field in status to show show an average waiting % before a message get claim. 2. do you think we can add a filed to show how many claimed message get ttl timeout.
21:24:23 <zhiyan> and back to queue which means allow other client claim it.
21:24:25 <flwang1> zhiyan: they're most like PKI, right?
21:24:38 <flwang1> if so, you can do that in health endpoint
21:24:44 <kgriffs> KPI = Key Performance Indicators
21:24:52 <flwang1> KPI, sory
21:24:55 <zhiyan> flwang1: not sure PKI, i see it as a workload indicator.
21:25:05 <flwang1> typo
21:25:07 <zhiyan> ok, KPI
21:25:16 <flwang1> KPI as kgriffs said above
21:25:19 <zhiyan> sounds like that
21:25:33 <flwang1> then you can add it in /health, IIUC
21:25:46 <kgriffs> zhiyan: is this something for the app developer or the operator, or perhaps both?
21:26:23 <zhiyan> client/worker could easily to query those two to help them make how many worker/thread/process, and how short query time between each loop.
21:26:38 <zhiyan> kgriffs: in my draft idea, only for client/worker developer
21:26:43 <kgriffs> makes sense
21:27:01 <zhiyan> let me explain a little on #1
21:27:51 <zhiyan> imo, it could be like this: avg((ttl-age)/ttl *100)
21:28:13 <zhiyan> ttl value is what we provide when create message to queue
21:28:59 <zhiyan> for example. if we get this value as 80%, then probably client could raise up worker pool size
21:29:31 <flwang1> zhiyan: but you said it's for developer?
21:30:02 <flwang1> flwang1: IMHO, i'm more interested in the KPI for operators
21:30:10 <zhiyan> flwang1: yes, for client developer. code need to query it can take care logic on it
21:30:39 <kgriffs> kk
21:30:41 <zhiyan> flwang1: but seems operator could read it as well...
21:30:52 <kgriffs> one of the original ideas of exposing queue stats was for autoscale and such
21:30:58 <kgriffs> so I think this fits
21:31:13 <zhiyan> e.g, a high value for #1, also means producer need to increace ttl as well.
21:31:24 <zhiyan> kgriffs: nice to know
21:31:41 <zhiyan> for #2, i think it's very easy to understand
21:32:20 <zhiyan> just a count value for how many claimed msg get ttl timeout
21:32:51 <kgriffs> 2 may be tricky to implement if the storage driver is relying on a DB auto-expiration of messages, but it could count how many are about to expire
21:33:06 <zhiyan> kgriffs: flwang1 so frankly i'm not check the code very close for the implementation on those two ideas, do you think I can do it technically?
21:33:10 <kgriffs> but yeah, I think it would be good info
21:33:30 <kgriffs> zhiyan: yes, it can be done. the trick is always making it performant. :p
21:34:03 <zhiyan> kgriffs: great. thanks for input. flwang1 !
21:34:23 <flwang1> zhiyan: glad to know you start to contribute zaqar :)
21:34:43 <kgriffs> one thing I'm not sure about is whether we can sneak this into v1.1
21:34:52 <flwang1> kgriffs: see, there are lot of China developer are coming :D
21:35:00 <zhiyan> kgriffs: if so, btw, would you mind to update wiki page at https://wiki.openstack.org/wiki/Zaqar/specs/api/v1.1#Changes_from_v1.0 in NO_BREAKING section?
21:35:03 <kgriffs> I had proposed not freezing 1.1 until probably end of k-1
21:35:53 <zhiyan> flwang1: thanks. glade to join.
21:36:40 <zhiyan> kgriffs: sorry, i'm confused on this. so you mean you can add this two to there?
21:36:40 <zhiyan> or not?
21:36:44 <kgriffs> zhiyan: I think we need to follow up with a few others on the team to see what we want to do this in 1.1 or what. Since it is non-breaking, we can probably sneak it in, but i don't want to speak for everybody
21:38:15 <zhiyan> kgriffs: ohok
21:38:32 <flwang1> kgriffs: may I know why?
21:38:33 <flwang1> kgriffs: get a more stable v1.1?
21:39:11 <kgriffs> yeah, I guess with all the graduation stuff I'm not sure where we landed on our API plans.
21:39:15 <zhiyan> actually i think it's a sensible change on 1.1 when K open...why sneak =)
21:40:13 <kgriffs> hmmm
21:40:45 <zhiyan> kgriffs: no worries. i'm interested in tech more than process. so..next pls :)
21:40:51 <kgriffs> ok
21:41:31 <kgriffs> well, we haven't documented 1.1 yet so I suppose it isn't "official" anyway and we have some fudge room. I'll ping some folks and see what we want to do.
21:42:34 <zhiyan> kgriffs: do you think i can think about it on implementation level?
21:42:34 <kgriffs> flwang1: re heat integration, I haven't heard that anyone was working on that yet.
21:42:53 <flwang1> flwang1: ok
21:43:06 <flwang1> kgriffs: ok :)
21:43:09 <kgriffs> zhiyan: sure thing. I think it is a good idea, we just need to figure out what API version to put it in
21:43:24 <zhiyan> kgriffs: nice ^^
21:43:38 <kgriffs> flwang1: I just grep'd blueprints for heat and didn't see anything
21:43:39 <kgriffs> so...
21:44:05 <flwang1> kgriffs: ah, fair enough, I will catch flaper87|afk for that
21:44:06 <kgriffs> I think first step is to sync with the heat team and identify any work we need to do on our end to make their features work well
21:44:43 <flwang1> kgriffs: yep
21:44:55 <kgriffs> flwang1: you can ping randall burt
21:45:13 <flwang1> kgriffs: actually, I'm trying to figure out what's my focus in K for zaqar
21:45:13 <kgriffs> I think he was going to help implement some of the heat features that will rely on zaqar
21:45:23 <kgriffs> flwang1: ah, yes
21:46:12 <flwang1> or seems another racker :)
21:46:15 <kgriffs> generally speaking, I think K should focus on scalability and reliability, polishing up the API, and adding anything that heat and horizon need. For example, calling a web hook or something
21:46:58 <kgriffs> (AKA notifications)
21:47:02 <flwang1> kgriffs: maybe touchy
21:47:22 <flwang1> but do you think we should keep an eye on sqs/sns of aws
21:47:29 <kgriffs> I wouldn't mind seeing first-class support for javascript apps - I think horizon mentioned that CORS would be nice, for example.
21:48:45 <zhiyan> btw, is zaqar going to get graduation on this J release?
21:48:47 <kgriffs> flwang1: good question. we can get some ideas from aws but we need to be focused on getting the basics right and serving horizon and heat.
21:49:16 <kgriffs> zhiyan: unfortunately no - we missed it by a vote or two
21:49:19 <flwang1> zhiyan: good question, but I have no idea :)
21:49:30 <kgriffs> so the project will incubate one more cycle
21:49:34 <flwang1> zhiyan: ah  you mean J, no
21:49:59 <flwang1> we missed 1, iirc
21:50:04 <flwang1> 6:7
21:50:15 <kgriffs> to be completely honest, several committee members did not engage until the last minute and were confused about our mission and goals
21:50:19 <zhiyan> sadly, but fine, just means need more efforts on it.
21:50:44 <flwang1> zhiyan: maybe more effort from TC ;)
21:51:02 <flwang1> they should take more time to understand what zaqar is
21:51:31 <zhiyan> flwang1: or maybe more effort for proj on tech level :)
21:51:45 <flwang1> sure, you know I'm kidding
21:51:48 <zhiyan> flwang1: but, i see the value of your point . true
21:51:56 <kgriffs> heh, well, you need everyone to help from both ends. :p
21:52:19 <flwang1> kgriffs: yes, sir :)
21:52:20 <zhiyan> absolutely
21:52:41 <flwang1> anything else we need to discuss today?
21:53:12 <zhiyan> that's all for me.
21:53:23 <kgriffs> oh, I guess we better focus on reviews the next few days
21:53:25 <kgriffs> https://review.openstack.org/#/q/status:open+project:openstack/zaqar,n,z
21:53:37 <kgriffs> otherwise, that's all from me
21:54:26 <kgriffs> ok, thanks flwang1 and zhiyan!
21:54:31 <kgriffs> #endmeeting