Wednesday, 2016-02-10

openstackgerritEva Balycheva proposed openstack/zaqar: Add queue name to notifications  https://review.openstack.org/27503000:04
openstackgerritEva Balycheva proposed openstack/zaqar: Add queue name to notifications  https://review.openstack.org/27503000:19
flwangEva-i: ping00:20
Eva-iflwang: pong00:20
flwangEva-i: some questions about https://review.openstack.org/#/c/275030/8/zaqar/notification/notifier.py00:20
Eva-iflwang: yes?00:21
flwangcan you remind me why do we need the change from line #43 - #50?00:21
Eva-iflwang: Reasons: 1. We do deepcopy to not modify original messages which will be sent to storage controller. Though it would not harm, if we modify original list, because our storage controllers parse messages correctly and will just omit the new key that we added to each message's dict. If you want, we can not do deepcopy, which affects performance.00:27
Eva-i2. If there are no subscribers, let's not do any work (deepcopy and adding "queue_name" key to each message. For this reason we check the number of subscribers.00:29
*** flwang has quit IRC00:29
*** flwang has joined #openstack-zaqar00:43
*** achanda has joined #openstack-zaqar00:46
flwangEva-i: did you see my comments?00:51
Eva-iflwang: have you received my IRC messages with reasons?00:51
flwangi saw that, but i can't be convenienced00:51
flwangconvinced00:52
flwangi can't see any benefit to do the change at notifier level00:52
Eva-iflwang: did you see my answer in gerrit also?00:53
Eva-iflwang: the gerrit answer tells why I did it in notifier level.00:53
Eva-i*at00:54
flwangi can see the point, let me explain it a bit00:54
flwangmaybe i'm wrong00:55
Eva-iflwang: oki00:56
flwanggenerally, we assume the messages are million level or more, then it's 1/10 million level, that means there are many times loop to add the queue name00:56
flwangif we add the queue_name at the task driver level, firstly, we don't need any loop00:56
flwangwe can just add a new line after https://github.com/openstack/zaqar/blob/master/zaqar/notification/task/mailto.py#L43 for example00:57
flwangand as you know, we can benefit the power of sub thread instead of bugging the main thread00:57
flwangthoughts?00:58
* Eva-i thinking00:58
*** achanda has quit IRC00:59
Eva-iflwang: first of all, let's decide can we modify original list of messages or we must do deepcopy?01:02
flwangEva-i: i don't think we can touch the original messages01:05
flwangsince it's not the responsibility of notifications, even zaqar01:05
flwangthat's my opinion01:06
Eva-iflwang: the messages would be parsed and delivered to the database normally01:06
Eva-iflwang: I mean the messages will be stored in databases without the additional key we have added.01:07
openstackgerritThai Tran proposed openstack/zaqar-ui: WIP Queues create wizard  https://review.openstack.org/27759001:07
flwangEva-i: yep01:07
Eva-iflwang: and the user will not notice any change in messages, when it will retrieve the messages normally from Zaqar.01:08
Eva-i*he will01:08
flwangsorry, i can't follow01:08
openstackgerritThai Tran proposed openstack/zaqar-ui: Queues create wizard  https://review.openstack.org/27759001:09
Eva-iflwang: I mean even if the storage controller will receive messages(dicts) each one with additional key, they will store the messages normally in the database. Our added "queue_name" key will not be stored in any way in databases.01:11
Eva-iflwang: so maybe it's safe to modify it.01:11
*** amitgandhinz has joined #openstack-zaqar01:13
*** amitgandhinz has quit IRC01:13
Eva-iflwang: okay. If we decide to use deepcopy, where we will do it: in task driver level or in notifier level?01:13
flwangEva-i: yep, i can see your point now01:14
flwangi don't think we need the deepcopy either way01:14
Eva-iflwang: okay, without deepcopy things are easier. I'll benchmark tasks with adding new key and without. ;)01:18
flwangawesome, i just want to keep things simple and no performance impact01:19
openstackgerritwanghao proposed openstack/zaqar: Fix improperly LOG using in Zaqar  https://review.openstack.org/27781201:20
Eva-iflwang: things would be even easier if we were allowed to change notifications format in non-backward compatible way.01:20
Eva-iespecially easier on CPU01:21
flwangwhat do you mean non-backword compatible?01:21
*** pt_15 has quit IRC01:25
Eva-iI mean to change our notifications format from this: {'body': 'message1', 'ttl': 60}.01:25
Eva-iTo this: {'queue_name': 'somequeue', 'messages' : [{'body': 'message1', 'ttl': 60}, {'body': 'message2', 'ttl': 60}]}01:25
Eva-iflwang: ^^^01:25
*** itisha has quit IRC01:26
Eva-iFor example, with second format, we can notify the subscriber with more than one message in single request. And we don't need to append "queue_name" to each message dict.01:27
flwanginteresting01:30
flwangpersonally, i like the idea01:34
Eva-iPerhaps it's possible to do it in backward compatible way. We just need to add an option either to Zaqar or to subscription options. So the user would be able to choose in which format he would like to receive notifications.01:35
Eva-iAnd if there will be such option, there is no need in my patch https://review.openstack.org/#/c/27503001:39
*** shakamunyi has quit IRC01:40
flwanghmm... i don't want to introduce an option, but i would like to know the others' opinion01:40
Eva-iflwang: would you like to discuss it on the next meeting?01:41
Eva-iflwang: maybe for now we should abandon my patch01:41
flwangyou don't have to abandon it01:43
flwangyep, we can discuss it on next meeting01:44
Eva-iflwang: so, now I make these changes to my patch: 1. stop using deepcopy 2. Move to the task level adding "queue_name" key to each message dict.01:46
*** kgriffs is now known as kgriffs|afk01:46
flwangEva-i: good for me02:00
*** achanda has joined #openstack-zaqar02:00
openstackgerritEva Balycheva proposed openstack/zaqar: Add queue name to notifications  https://review.openstack.org/27503002:10
openstackgerritEva Balycheva proposed openstack/zaqar: Add queue name to notifications  https://review.openstack.org/27503002:13
openstackgerritEva Balycheva proposed openstack/zaqar: Add queue name to notifications  https://review.openstack.org/27503002:15
openstackgerritEva Balycheva proposed openstack/python-zaqarclient: Do not catch ConflictError on subscription create  https://review.openstack.org/27719502:59
flwangEva-i: around?03:03
Eva-iflwang: yes03:03
flwangdid you ever use the queue set metadata command?03:03
Eva-iflwang: in our client? or via curl?03:05
flwangclient, i'm trying to figure out how to set the metadata after creating queue03:06
flwangfor v203:06
Eva-iflwang: do we need it? Isn't queue metadata feature for v1 only?03:07
flwangwe need it03:08
flwangv1.1 and v2 supports metadata as well03:08
flwangthere is a bug unfortunately03:16
Eva-iflwang: when I fix API version number to float conversion (which our patch will fix soon), I'm able to set metadata like this: openstack --os-queues-api-version=1 queue set metadata my-lovely-queue '{"oh": "yeah"}'03:19
flwangi see. but for v1.1 and v2, after you created the queue, there is no way to update its metadata03:20
flwangyou can try03:20
Eva-iflwang: alright, you can fix this in your patch https://review.openstack.org/#/c/267923/.03:23
flwangi can't03:24
flwangsince it's a server side bug03:24
Eva-iflwang: aha, I see. Zaqar returns 404 error on these operations.03:28
flwang404?03:30
flwangdo you mean 204?03:30
flwangit returns 204 but didn't update the metadata03:30
Eva-iWhen I use curls to set metadata: 1. on API v1: 204 response code (and I see update in the database). 2. On API v1.1: 404 response code. 3. On API v2: 404 response code03:33
Eva-iflwang: ^^^03:34
flwangEva-i: that's correct03:35
flwangsince we don't support metadata endpoints for v1.1 and v203:35
flwangand unfortunately, we didn't provide any other way to let the user update metadata03:36
flwangthat's the problem03:36
*** wanghao_ has joined #openstack-zaqar03:36
Eva-iflwang: "since we don't support metadata endpoints for v1.1 and v2", but should we? Maybe it's unsupported intentionally?03:37
flwangit's designed intentionally, but i can't remember the reason03:40
flwangoh, i think i can03:40
flwangIIRC, we was going to remove the metadata for queue03:41
flwangand after that we decide to keep them03:41
wanghao_flwang: hi, happy Chinese year! I have committed the client patch(https://review.openstack.org/#/c/276603/) for bug '03:41
wanghao_"pool_group" word should be used instead "pool"',  if you have time, please check it, thanks.03:41
flwangwanghao_: happy chinese year03:42
flwangclicking...03:42
flwangwanghao_: interesting... there is no gate failure for the patch03:45
flwangi expect there is a failure since you're posting pool_group now instead of pool03:45
wanghao_flwang: yeah, it's also shocking me.03:46
flwangso obviously, we need a check at here https://github.com/openstack/python-zaqarclient/blob/master/zaqarclient/queues/v1/api.py#L20403:46
Eva-iflwang: wanghao: yes, happy Chinese year too you =)03:46
flwangi think on the server side, the pool_group is just skipped silently03:47
flwangEva-i: thank you:)03:47
flwangwanghao_: but i prefer to do that later03:47
wanghao_Eva-i: thanks Eva03:47
flwangwanghao_: for now, you can try to let your server patch depends on the client patch, and see if the gate will be happy03:47
flwangif all of them are good, then we can merge them and add the api param check for client03:48
Eva-iflwang: I found blueprint, but can't find a spec to read https://blueprints.launchpad.net/zaqar/+spec/api-v1.1-remove-queue-metadata03:48
wanghao_flwang: yes, I added dependent-on in server patch and will check the gate.03:49
Eva-iflwang: seems like Zaqarians decided to remove queue metadata, because it's not much compatible with the idea of lazy queues03:52
Eva-iflwang: I noticed a strange thing In our API reference in wiki I read: "If auth is not enabled, clients must provide the X-Project-Id header themselves.". But in my Zaqar database I see many created queues without xprojectid, so seems like it's possible to create resources without xprojectid. Is it a bug?04:01
*** flwang1 has quit IRC04:02
*** wanghao has quit IRC04:46
*** wanghao_ is now known as wanghao04:46
*** shu-mutou-AFK is now known as shu-mutou04:57
*** rcernin has joined #openstack-zaqar07:13
*** openstackgerrit has quit IRC07:47
*** openstackgerrit has joined #openstack-zaqar07:47
*** jtomasek_ has joined #openstack-zaqar07:50
*** rcernin has quit IRC07:52
*** pt_15 has joined #openstack-zaqar08:03
*** itisha has joined #openstack-zaqar08:03
*** mdnadeem has quit IRC08:13
*** mdnadeem has joined #openstack-zaqar08:13
*** achanda has quit IRC08:13
*** rcernin has joined #openstack-zaqar08:15
*** dynarro has joined #openstack-zaqar08:27
*** achanda has joined #openstack-zaqar08:37
*** mdnadeem has quit IRC08:43
*** mdnadeem has joined #openstack-zaqar08:44
*** shu-mutou is now known as shu-mutou-AFK09:56
*** achanda has quit IRC10:06
*** achanda has joined #openstack-zaqar10:08
*** mdnadeem has quit IRC10:20
*** mdnadeem has joined #openstack-zaqar10:20
*** achanda has quit IRC10:45
*** dynarro has quit IRC11:14
*** achanda has joined #openstack-zaqar11:45
*** achanda has quit IRC11:50
openstackgerritwanghao proposed openstack/python-zaqarclient: "pool_group" word should be used instead "pool"  https://review.openstack.org/27660312:43
*** achanda has joined #openstack-zaqar12:48
*** achanda has quit IRC12:54
*** pt_15 has quit IRC13:15
*** itisha has quit IRC13:16
*** amitgandhinz has joined #openstack-zaqar13:33
*** amitgandhinz has quit IRC13:47
*** amitgandhinz has joined #openstack-zaqar13:47
*** sriram has joined #openstack-zaqar14:11
*** shu-mutou-AFK has quit IRC14:34
*** dynarro has joined #openstack-zaqar14:37
openstackgerritMerged openstack/zaqar: Add the missing Client-ID check in wsgi v2  https://review.openstack.org/27497214:38
flaper87flwang: ryansb vkmc What do you think about moving from a release model based on cycles and milestones to an independent one?14:39
ryansbI wouldn't mind having shorter freezes :)14:40
flaper87I think Zaqar would benefit from smaller, isolated and more frequent releases14:40
vkmcme too14:40
therveflaper87, How so?14:42
ryansbAlso how would associating with cycles work?14:43
ryansbe.g. if someone deploys a Liberty (or Mitaka or N) cloud, how would they pick a zaqar version?14:43
flaper87therve: well, the project is still young and small, the 6month cycle is too long for Zaqar. There are plans and features that could be implemented and released faster14:44
flaper87ryansb: No need to associate it if there's backwards compatibility, right?14:44
therveflaper87, Really?14:44
flaper87A newer Zaqar (Mitaka) won't/shouldn't break Heat Liberty14:44
flaper87therve: you seem to know something I don't so please, share14:45
therveflaper87, I mean, which feature in Mitaka would have benefited from an earlier release?14:45
ryansbthere could be users that want, say, a new feature that doesn't exist yet in Zaqar 5, released at the start of Mitaka, but does in Zaqar 6, released in the middle of mitaka14:45
ryansbbut if they're that aware, they can make that choice themselves14:45
flaper87therve: I'm not saying I have one specific feature in mind. I'm saying more often releases might help Zaqar moving forward. This affects planning, of course.14:46
flaper87I can tell you releasing pre-signed urls earlier would have helped in liberty, for instance14:46
flaper87ryansb: they can always upgrade to Zaqar 6, I don't see the issue14:46
ryansbyeah, I answered my own question while writing it - nm14:47
therveflaper87, I'm not against the idea in principle, I don't see it making it a big difference in practice14:48
therveAlso releases take some effort, so more releases == more effort14:48
flaper87There's also 6month releases w/ intermediary releases14:48
therveEven if you can streamline it a bit14:48
flaper87Sure but they don't in Zaqar, let's be honest. Releasing zaqar would take the same amount of effort as cutting a milestone14:49
therveThat said, I won't work on the releases, so maybe I should just shut up14:49
flaper87I guess my main point is that I don't think Zaqar needs to align with other projects for releases and it could use some extra flexibility on that side14:49
flaper87I don't like the lack of prediction in not having a scheduled release14:50
*** achanda has joined #openstack-zaqar14:52
openstackgerritMerged openstack/zaqar: Make error response better for patching flavors  https://review.openstack.org/27449514:52
openstackgerritMerged openstack/zaqar: Don't return links if subscriptions are empty  https://review.openstack.org/27430414:52
therveflaper87, I don't get that last point. There is a scheduled release? Every six months?14:55
flaper87therve: yeah, I meant to say. What I don't like about the independent release model is that there's no schedule14:56
therveOh I see you're criticizing your own idea :)14:56
flaper87therve: yeah :D I'm known for contradicting myself14:56
flaper87:D14:56
*** achanda has quit IRC14:57
*** ametts has joined #openstack-zaqar14:58
*** vkmc is now known as vkmc|midcycle15:01
*** mpanetta has joined #openstack-zaqar15:13
*** mdnadeem has quit IRC15:14
*** mdnadeem has joined #openstack-zaqar15:14
*** kgriffs|afk is now known as kgriffs15:28
*** rcernin has quit IRC15:33
*** wanghao has quit IRC15:37
*** malini has joined #openstack-zaqar15:42
*** malini1 has joined #openstack-zaqar15:44
*** malini has quit IRC15:47
*** rcernin has joined #openstack-zaqar16:00
*** csoukup has joined #openstack-zaqar16:18
*** rcernin has quit IRC16:24
*** boris-42 has quit IRC16:33
*** jtomasek_ has quit IRC16:42
*** achanda has joined #openstack-zaqar16:55
*** achanda has quit IRC16:59
*** mpanetta has quit IRC16:59
therveI think zaqar devstack plugin is broken17:23
therveBecause of the client id check17:23
therveIf someone can have a look, otherwise I'll do it tomorrow17:23
ryansbtherve: I'll put it on my list & ping you if I get it solved17:27
*** davideagnello has quit IRC17:35
*** davideagnello has joined #openstack-zaqar17:36
*** boris-42 has joined #openstack-zaqar17:39
therveryansb, I believe it block Heat gate, though we don't really know because people are rechecking without looking17:43
*** jtomasek has joined #openstack-zaqar17:48
*** rcernin has joined #openstack-zaqar17:52
*** itisha has joined #openstack-zaqar17:55
*** achanda has joined #openstack-zaqar17:57
*** dynarro has quit IRC17:58
*** achanda has quit IRC18:01
*** mpanetta has joined #openstack-zaqar18:04
*** achanda has joined #openstack-zaqar18:12
*** achanda has quit IRC18:15
*** malini1 has quit IRC18:23
*** achanda has joined #openstack-zaqar19:00
*** malini has joined #openstack-zaqar19:03
*** malini1 has joined #openstack-zaqar19:05
ryansbtherve: yeah, looks like it's broken (just spun up a devstack)19:06
*** malini has quit IRC19:07
*** c_soukup has joined #openstack-zaqar19:09
*** csoukup has quit IRC19:13
*** c_soukup has quit IRC19:19
*** kgriffs is now known as kgriffs|afk19:20
* Eva-i looking at devstack issue too19:24
*** csoukup has joined #openstack-zaqar19:26
therveryansb, Eva-i: I belive the wget is simply missing the client-id19:27
*** kgriffs|afk is now known as kgriffs19:27
Eva-itherve: yes19:27
ryansbyep, I'm doing a patch19:28
Eva-iryansb: I made patch too, but go on =)19:30
openstackgerritRyan Brown proposed openstack/zaqar: Add Client-ID to devstack health check  https://review.openstack.org/27857719:33
* ryansb waits for the gate...19:36
*** achanda has quit IRC19:36
ryansbah good, only a billion changes ahead in the queue19:37
*** achanda has joined #openstack-zaqar19:45
*** pt_15 has joined #openstack-zaqar19:48
*** eliqiao_ has quit IRC19:59
*** eliqiao_ has joined #openstack-zaqar20:02
*** jtomasek has quit IRC20:10
*** achanda has quit IRC20:16
*** achanda has joined #openstack-zaqar20:23
flwangryansb: therve: Eva-i: thanks for the quick fix20:32
Eva-iflwang: good morning20:33
ryansbahh, so nice20:34
ryansbgerrit lets you edit commit message in-browser now20:34
openstackgerritRyan Brown proposed openstack/zaqar: Use Client-ID in devstack ping request to Zaqar  https://review.openstack.org/27857720:34
ryansbso neat20:34
Eva-iryansb: gerrit even lets edit python files in-browser20:36
ryansbyeah, but then I can't run tests20:36
ryansbcommit messages don't have test coverage ;)20:37
flwanganybody know        Neerja Narayan                  ?20:37
flwangshe/he registered a bp in zaqar for tempest test20:37
*** flwang1 has joined #openstack-zaqar20:40
flwangEva-i: as for your patch https://review.openstack.org/#/c/27503020:42
flwangas we discussed, we don't need the change in notifier.py to pass in the queue_name20:43
flwangsince we can easily get it from the subscription20:43
Eva-iflwang: I'm sorry I forgot about it, gonna fix it20:44
flwangEva-i: cool, thanks20:44
openstackgerritFei Long Wang proposed openstack/zaqar-ui: Queues create wizard  https://review.openstack.org/27759020:45
ryansbflwang: if you wouldn't mind ninja-approving something, it'd be great if you could peek at https://review.openstack.org/278577 once the CI passes20:50
ryansb;)20:50
openstackgerritEva Balycheva proposed openstack/zaqar: Add queue name to notifications  https://review.openstack.org/27503021:02
*** c_soukup has joined #openstack-zaqar21:03
Eva-iflwang: seems like Zaqarians decided to remove queue metadata in the past, because it's not much compatible with the idea of lazy queues21:04
*** csoukup has quit IRC21:06
flwangEva-i: yes21:07
flwangEva-i: but21:07
flwangi'm not a fan of lazy queues, especially now the subscriptions depends on that21:08
openstackgerritFei Long Wang proposed openstack/zaqar-ui: Queues create wizard  https://review.openstack.org/27759021:08
Eva-iflwang: maybe we can redesign subscriptions to make them not depend on queue existence, like messages do not depend.21:15
flwangEva-i: it sounds like subscribe a non-existing object(queue)?21:18
Eva-iflwang: right21:18
flwangit's weird for me :)21:18
Eva-iflwang: it's like subscribing to tag21:19
flwangwe can discuss it in Newton21:19
Eva-iflwang: what I think: we can't treat queues as lazy resources in one situations, and as non-lazy in other situations. We have to choose.21:22
flwangfor now, i don't think it's a yes or no problem21:23
flwanguser can post message without queue create21:23
flwanguser can't create subscription without queue create21:23
flwangfor me, it's not too bad21:23
flwangand i prefer to let user poke me instead of deciding by the end user21:24
Eva-iflwang: we have strange API =)21:25
flwangjust like you called it unstable API, right?21:25
flwangyou will see more strange thing in openstack21:25
flwangi call it 'surprise'21:26
flwangsoftware is a product of compromise21:27
Eva-iflwang: okay, that's sad to hear21:27
Eva-iIs X-PROJECT-ID header required in requests to Zaqar, when noauth is used?21:28
flwangEva-i: i'm sorry to show you the real world :)21:28
flwangEva-i: yes, IIRC21:30
flwangEva-i: anything wrong?21:30
Eva-iflwang: I feel I can make this world better. And in lazy resources problem I don't see where compromise is.21:30
Eva-iflwang: in my mongodb database I noticed a strange thing. Some of queues records don't contain project id. And is it okay that one of the queues have project id "{}", do we allow such symbols?. See http://paste.openstack.org/show/k9udhl7qqS4GOt2flTKM/21:33
flwangEva-i: hah, glad to hear you can make the world better, though i'm trying either21:35
flwangEva-i: i think you need to figure out why there is no project id21:36
flwangbased on your paste, i have no idea21:36
*** csoukup_ has joined #openstack-zaqar21:39
*** malini1 has quit IRC21:39
Eva-iflwang: because Zaqar allows it. I can create resources in Zaqar via curl without passing X-PROJECT-ID header.21:39
Eva-iflwang: if it allows it, it's a bug, right?21:40
*** achanda_ has joined #openstack-zaqar21:40
*** njohnston_ has joined #openstack-zaqar21:41
flwangEva-i: i need to check, wait a moment21:41
flwangEva-i: i think it only happens when you using noauth21:43
flwangright?21:43
Eva-iflwang: yes, as I said21:43
flwangEva-i: i think it can be improved21:44
Eva-iWiki API v1 reference tells: "If auth is not enabled, clients must provide the X-Project-Id header themselves."21:45
*** c_soukup has quit IRC21:48
*** achanda has quit IRC21:48
*** njohnston has quit IRC21:48
*** sriram has quit IRC21:48
flwangEva-i: if noauth enabled, can the request work without project id?21:54
Eva-iflwang: yes, I tried few types of requests to Zaqar without project id. They all seem to be working.22:00
flwangEva-i: if so, we may need to fix it22:05
*** rcernin has quit IRC22:07
Eva-iflwang: okay, I'm filling a bug report, so we will not forget about this problem22:11
*** mpanetta has quit IRC22:15
flwangEva-i: thanks :)22:15
*** boris-42_ has joined #openstack-zaqar22:17
*** boris-42 has quit IRC22:22
*** boris-42_ is now known as boris-4222:24
*** ametts has quit IRC22:29
therveflwang, Hey, we really need https://review.openstack.org/278577 it breaks heat gate22:36
flwangtherve: yep, i see. i will nijia approve it22:38
therveThanks!22:38
flwangtherve: did you test it?22:39
*** malini has joined #openstack-zaqar22:40
therveNo hopefully jenkins will22:46
*** achanda_ has quit IRC22:49
flwangtherve: is it possible to let a trival patch of heat depends on https://review.openstack.org/#/c/278577/ ?22:52
flwangthough i have approved it22:52
flwangtherve: anyway, let's see22:52
therveflwang, Not sure the depends would pick up the devstack config22:53
openstackgerritMerged openstack/zaqar: Use Client-ID in devstack ping request to Zaqar  https://review.openstack.org/27857722:57
flwangok, meged23:01
flwangtherve: can you recheck the heat patch?23:01
therveflwang, Yep thanks!23:01
flwangtherve: welcome :)23:02
*** malini has quit IRC23:17
*** achanda has joined #openstack-zaqar23:31
*** csoukup_ has quit IRC23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!