Thursday, 2014-06-05

*** flwang has joined #openstack-marconi00:17
*** mwagner_lap has joined #openstack-marconi00:40
*** balajiiyer has joined #openstack-marconi00:49
*** balajiiyer has quit IRC00:50
*** rwsu has quit IRC00:51
*** oz_akan_ has joined #openstack-marconi01:29
*** oz_akan_ has quit IRC01:33
*** oz_akan_ has joined #openstack-marconi01:36
*** oz_akan_ has quit IRC01:40
*** vkmc has joined #openstack-marconi01:42
*** nosnos has joined #openstack-marconi01:46
*** balajiiyer has joined #openstack-marconi01:48
*** oz_akan_ has joined #openstack-marconi01:49
*** mpanetta has quit IRC01:56
*** jmckind has quit IRC02:05
*** balajiiyer has quit IRC02:18
*** vkmc has quit IRC03:26
*** nosnos has quit IRC03:43
*** prashanthr_1 has joined #openstack-marconi03:43
*** reed has quit IRC03:47
*** prashanthr_1 has quit IRC03:51
*** prashanthr_ has joined #openstack-marconi03:57
*** whenry has quit IRC03:57
*** oz_akan_ has quit IRC04:04
*** renlt has joined #openstack-marconi04:15
*** prashanthr_ has quit IRC04:26
*** nosnos has joined #openstack-marconi04:41
*** renlt has quit IRC04:46
*** prashanthr_ has joined #openstack-marconi05:15
*** prashanthr_ has quit IRC05:36
*** oz_akan_ has joined #openstack-marconi05:37
*** oz_akan_ has quit IRC05:41
*** prashanthr_ has joined #openstack-marconi05:50
*** jraim has quit IRC06:05
*** jraim has joined #openstack-marconi06:06
*** flaper87|afk is now known as flaper8706:08
flaper87tjanczuk_: Alex_Gaynor Marconi doesn't use eventlet. It used to have an "indirect" dependency on it but it's now gone.06:11
flwangflaper87: wooooooooooooooooooow06:11
flaper87flwang: what's up buddy?06:12
flaper87how are you doing?06:12
flaper87long time no see06:12
flwangflaper87: yep, i'm working from New Zealand06:12
flwangeverything good until now :)06:12
flwangjust wanna say hi to let you know I'm still alive :D06:12
flwangand not far away from OpenStack06:12
flwangso far so good06:15
*** renlt has joined #openstack-marconi06:16
tjanczuk_flapper87: does it mean Marconi is doing blocking IO then (i.e. a separate thread is used for every active request)?06:23
flaper87tjanczuk_: it depends on what wsgi container you use06:33
flaper87tjanczuk_: Marconi just ships the wsgi app and users are free to choose whatever wsgi container they want06:34
flaper87you could use uwsgi (written in C and supports non-blocking IO) or gunicorn + eventlet06:34
*** prashanthr_1 has joined #openstack-marconi06:34
prashanthr_1hi flaper87 , flwang :)06:34
flaper87If you run `marconi-server` you're using wsgiref which is a "toy" container shipped with python06:34
flaper87and it's meant to be used just for development06:35
flaper87prashanthr_1: yo!06:35
flaper87flwang: man, so happy to hear that! Take good care of yourself and your family.06:35
flaper87flwang: are you going to spend more time on Marconi now/06:35
prashanthr_1flaper87: How u doing ?06:35
flwangflaper87: recently, now, but next month, I suppose yes06:35
flaper87prashanthr_1: doing great, thanks :D06:35
flaper87prashanthr_1: you?06:36
*** prashanthr_ has quit IRC06:36
prashanthr_1flaper87: doing great as well :) thanks !06:38
*** oz_akan_ has joined #openstack-marconi06:38
prashanthr_1i wanted 2 have a brief chat with you about supporting redis for marconi.06:38
prashanthr_1are u free for sometime now ?06:38
flaper87prashanthr_1: shoot :)06:38
prashanthr_1flaper87: Yesterday we were having a discussion about supporting redis on the main branch of marconi. kgriffs|afk felt that it would be could to get a collective opinion.06:40
prashanthr_1*could-> good06:40
prashanthr_1we also had a brief chat about the AGPL issue for mongodb.06:41
*** oz_akan_ has quit IRC06:42
flaper87prashanthr_1: mmh, TBH, I thought that was decided already :P06:45
prashanthr_1flaper87: ha ha :).06:45
flaper87prashanthr_1: it sounds good to me, FWIW. We've been looking for a store driver that would help us with the whole AGPL issue.06:46
flaper87those 2 drivers are redis' and amqp's06:46
prashanthr_1flaper87: awesome :). The AGPL issue was pretty serious i just noticed that yesterday.06:47
flaper87prashanthr_1: I still need to dig better into how serious it is. There are a whole bunch of different opinions around it and some seem to be very subjective.06:48
prashanthr_1flaper87: Ohh okay. I read a couple of posts yesterday. But the one's i read made it look draconian.06:49
flaper87prashanthr_1: did you read this one?
flaper87wait, not that one06:50
prashanthr_1This one's interesting as well :)06:51
flaper87prashanthr_1: this one:
flaper87that's the "legal discussion"06:52
prashanthr_1flaper87: I will go through it.06:56
flaper87prashanthr_1: btw, thanks for the heads up w.r.t the discussions and the comments06:59
*** jamie_h has joined #openstack-marconi07:01
prashanthr_1flaper87: No problems :). Phew these licenses i never understand them well enough. Reading  them is never fun :(.07:01
*** prashanthr_ has joined #openstack-marconi07:03
*** prashanthr_1 has quit IRC07:05
*** ykaplan has joined #openstack-marconi07:19
*** jamie_h has quit IRC07:36
*** jamie_h has joined #openstack-marconi07:37
prashanthr_flaper87: I just had a small doubt about the oslo cache we r using.07:38
prashanthr_when we append  a list to another one in the cache07:38
*** oz_akan_ has joined #openstack-marconi07:38
prashanthr_the one in the cache must already by of the same list type. Am i right ?07:38
*** flaper87 is now known as flaper87|afk07:39
prashanthr_flaper87: thanks.07:39
*** oz_akan_ has quit IRC07:43
*** prashanthr_ has quit IRC08:23
*** goodes has joined #openstack-marconi08:30
*** mkoderer has quit IRC08:32
*** oz_akan_ has joined #openstack-marconi08:39
*** oz_akan_ has quit IRC08:43
*** jamie_h has quit IRC08:48
*** prashanthr_ has joined #openstack-marconi08:52
*** haomaiwa_ has joined #openstack-marconi09:01
*** prashanthr_ has quit IRC09:04
*** haomai___ has joined #openstack-marconi09:04
*** haomaiwa_ has quit IRC09:07
*** flaper87|afk is now known as flaper8709:19
*** flaper87 is now known as flaper87|afk09:38
*** oz_akan_ has joined #openstack-marconi09:40
*** oz_akan_ has quit IRC09:44
*** mkoderer has joined #openstack-marconi09:53
*** ykaplan has quit IRC10:05
*** goodes has quit IRC10:22
*** renlt has quit IRC10:24
*** oz_akan_ has joined #openstack-marconi10:38
*** haomai___ has quit IRC10:40
*** haomaiwang has joined #openstack-marconi10:41
*** oz_akan_ has quit IRC10:42
*** haomaiw__ has joined #openstack-marconi10:47
*** haomaiwang has quit IRC10:50
*** vkmc has joined #openstack-marconi10:51
*** vkmc has quit IRC10:51
*** vkmc has joined #openstack-marconi10:51
*** jamie_h has joined #openstack-marconi10:56
*** goodes has joined #openstack-marconi11:15
*** nosnos has quit IRC11:20
*** jamie_h has quit IRC11:29
*** AAzza_afk is now known as AAzza11:30
*** oz_akan_ has joined #openstack-marconi11:39
*** oz_akan_ has quit IRC11:43
*** jamie_h has joined #openstack-marconi11:59
*** flaper87|afk is now known as flaper8712:02
*** jamie_h has quit IRC12:10
*** jamie_h has joined #openstack-marconi12:29
*** jchai has joined #openstack-marconi12:39
*** oz_akan_ has joined #openstack-marconi12:40
*** oz_akan_ has quit IRC12:44
*** sriram has joined #openstack-marconi12:45
*** sriram has quit IRC12:46
*** sriram has joined #openstack-marconi12:46
*** jamie_h has quit IRC12:47
*** AAzza is now known as AAzza_afk12:48
*** ykaplan has joined #openstack-marconi12:49
flaper87sriram: kgriffs|afk alcabrera|afk malini vkmc et all! Good morning! Just a heads up. The gate seems to be uner heavy load so lets not recheck, push, approve anything unless it helps fixing the gate.12:49
*** goodes has quit IRC12:50
sriramflaper87: thanks for the headup, I just read the email :)12:50
sriramand good morning :P12:50
flaper87sriram: awesome :)12:50
flaper87OH: "If you do a query on a database without indexes it will take a lot of time"12:51
flaper87(Captain obvious)12:51
* flaper87 in jerk mode12:51
*** jamie_h has joined #openstack-marconi12:51
vkmcmorning flaper87, sriram :)12:52
vkmcthanks for the update flaper8712:52
vkmcs/update/heads up12:52
srirammorning vkmc, I saw that there was heavy discussion on AMQP going on, I learnt quite a bit ;)12:53
vkmcsriram, oh yes... there are a lot to discuss in that area :)12:53
vkmcI'm learning a lot12:54
vkmcthe downside is that there are not enough docs :(12:55
sriramOh I see, I have been doing some reading on rabbitmq.12:56
* sriram has never used rabbitmq before :/12:56
vkmcrabbit folks did a great job with docs, unfortunately I'm with another version of the protocol and hence another library12:58
sriramoic, this is amqp 1.0?12:58
*** jchai has quit IRC12:59
vkmcprecisely, Apache's Proton13:00
*** jchai has joined #openstack-marconi13:00
*** flaper87 is now known as flaper87|afk13:02
*** jmckind has joined #openstack-marconi13:07
*** jergerber has joined #openstack-marconi13:07
*** prashanthr_ has joined #openstack-marconi13:10
*** oz_akan_ has joined #openstack-marconi13:11
*** sriram has left #openstack-marconi13:15
*** sriram has joined #openstack-marconi13:15
*** stannie has joined #openstack-marconi13:18
*** jergerber has quit IRC13:22
*** Obulpathi has joined #openstack-marconi13:31
*** balajiiyer has joined #openstack-marconi13:36
*** goodes has joined #openstack-marconi13:36
*** Obulpathi has quit IRC13:39
sriramkgriffs|afk: the client was smoke tested yesterday, and it looks good.14:04
*** alcabrera|afk is now known as alcabrera14:05
*** goodes has quit IRC14:06
*** goodes has joined #openstack-marconi14:07
alcabreragood morning. :)14:07
sriramalcabrera: good morning! :D14:07
sriramhow are you doing?14:07
*** malini has joined #openstack-marconi14:07
alcabrerahungry, happy, thoughtful. I've been reading and learning a lot lately. How about you, sriram? :)14:08
*** balajiiyer has quit IRC14:08
*** jay-atl has quit IRC14:08
sriramawesome!, I didnt get much sleep yesterday, a bit tired.14:08
alcabreraoh no. this requires a midday nap to rectify, sriram! :D14:08
vkmcmorning alcabrera!14:09
vkmc+1 alcabrera14:09
vkmcyou totally should get a power nap sriram14:09
sriramhaha, I'll try! :)14:10
alcabreravkmc: good morning! :)14:10
*** AAzza_afk is now known as AAzza14:12
*** balajiiyer has joined #openstack-marconi14:12
*** goodes has left #openstack-marconi14:14
*** Obulpathi has joined #openstack-marconi14:14
*** jamie_h has quit IRC14:17
*** haomaiw__ has quit IRC14:20
*** cpallares has joined #openstack-marconi14:26
alcabreracpallares: good morning!14:27
cpallaresalcabrera: hi! how are you?14:28
alcabrerahungry, happy, thoughtful. How about you, cpallares? :)14:28
vkmccpallares, morning :)14:29
prashanthr_hi good morning all ! :)14:32
cpallaresalcabrera: Doing great :)14:35
cpallareshi vkmc! hey prashanthr_ :) how's it going?14:36
vkmcgood evening prashanthr_ :)14:36
*** prashanthr_ has left #openstack-marconi14:38
*** prashanthr_ has joined #openstack-marconi14:38
prashanthr_thank you vkmc !14:38
*** haomaiwang has joined #openstack-marconi14:40
prashanthr_alcabrera: Do we have a meeting today ?14:41
*** goodes has joined #openstack-marconi14:42
alcabrerayes, prashanthr_! and also, good evening to you. :)14:43
alcabrerawe'll get started in about 15m.14:43
prashanthr_awesome :) Thank you ! :)14:43
*** haomaiwang has quit IRC14:45
*** oz_akan_ has quit IRC14:50
*** oz_akan_ has joined #openstack-marconi14:51
*** Obulpathi has quit IRC14:51
*** Obulpathi has joined #openstack-marconi14:52
*** haomaiwang has joined #openstack-marconi14:52
*** goodes has quit IRC14:54
*** kgriffs|afk is now known as kgriffs14:55
alcabreraalright! I managed to finish breakfast and make coffee in time for today's OPW/GSoC meeting. :D15:00
alcabreraAAzza, prashanthr_, vkmc: are you all ready to get started? :)15:00
prashanthr_yes :)15:01
*** goodes has joined #openstack-marconi15:01
alcabreraso let's get going: AAzza, could you start us off?15:01
alcabrerahow has progress been going since last week on py3-marconi? :)15:02
AAzzaalcabrera: hey, the commits were merged, and i'm ready to put next one with small fixes15:02
AAzzaalcabrera: but have some doubts)15:02
alcabrerasure thing, what doubts have you got, AAzza?15:03
AAzzaalcabrera: first of all is it okey to have one commit with small fixes, or better to have several with relevant grouped?15:03
alcabreraI prefer several, small commits grouped by relevance/category15:04
alcabreraif the small fixes in mind are less than say, 20-ish lines, and they all relate to the broad category of py3-compat, I think that's okay, too15:04
AAzzaalcabrera: okey, maybe split in max two commits to be smth in the middle)15:05
alcabrerathat'll work for me. :)15:05
AAzzaalcabrera: next one is a little bit specific, maybe will better discuss it in commit, but nevertheless15:05
alcabrerayes, what's coming up next?15:06
alcabreracould you summarize?15:06
AAzzaalcabrera: it is ugly to add encode/decode some specific encoding)15:06
*** jay-atl has joined #openstack-marconi15:07
AAzzaalcabrera: always hard to find what is particular expected encoding for this piece of text) is it utf-8 or plain ascii text)15:07
alcabrerathat is a tricky problem. :/15:07
AAzzaalcabrera: after this commit, i plan to have another one, also with strings15:08
*** jchai is now known as jchai_afk15:08
alcabrerathanks for that - the string problem is particularly bad between py2 <-> py315:08
AAzzaalcabrera: need some time to research the problem, since some parts of this fix is not working on py3
AAzzaalcabrera: you know py3 is even more explicit than py215:09
alcabrerayup, yup15:09
AAzzaalcabrera: you should always know where is text, and where is bytes and explicitly convert between them)15:09
AAzzaalcabrera: and when you have in py2 pure ascii string, then it is ugly piece15:10
alcabreraoh yeah. :/15:10
*** jamie_h has joined #openstack-marconi15:10
AAzzaalcabrera: because either to have it in utf-8, or do tricks and magic)15:10
AAzzaalcabrera: it is summary for now)15:10
alcabreraI suspect regex's don't handle the case of unicode strings very well. it's definitely something to dig into.15:10
alcabrerabut thanks, AAzza!15:11
alcabreraI had one thought on py315:11
alcabreraeven though we don't use memcached in marconi any more15:11
alcabrerawhen we do bring it back, it may be worthwhile to use this:
kgriffsI believe we decided a while back to try standardizing on wide unicode strings inside, while converting to utf-8 at I/O boundaries. Correct?15:11
alcabrerainstead of python-memcached15:11
alcabrerakgriffs: I believe that was the decision15:12
kgriffsI think this py3k effort is highlighting where we got lazy. :p15:12
alcabrerayup, yup. :P15:12
alcabreraAAzza: any other thoughts before we move on to prashanthr_? :)15:12
AAzzaalcabrera: oh... just want an advice15:13
alcabreraoh, sure. :)15:13
AAzzaalcabrera: i see this commit with decoupling tests15:13
AAzzaalcabrera: and my commit also changes the tests15:14
AAzzaalcabrera: not sure how to handle this, since first commit copying them15:14
alcabrerathere's likely going to be some conflict15:14
*** jchai_afk is now known as jchai15:14
alcabreramy suggestion15:14
alcabrerais to rebase your commit on top of
alcabrerawhat do you think, AAzza?15:14
AAzzaalcabrera: yes, good idea, and i will sure, that my changes will be in both tests groups, since i add them to both by myself) thanks :)15:15
AAzzaalcabrera: i' done then for now)15:16
*** haomaiwang has quit IRC15:16
alcabrerathanks, AAzza!15:16
*** haomaiwang has joined #openstack-marconi15:17
alcabreraprashanthr_: what news have you to share on redis? :)15:17
prashanthr_alcabrera: Redis yes i had a chat with flaper87|afk today in the morning as kgriffs had suggested yest15:17
prashanthr_he also was of the same opinion that redis must be a part of the main branch15:18
prashanthr_and we had a discussion of the AGPL issue of mongodb15:18
kgriffsquick thought on the topic15:18
kgriffsI like the idea of developign it first outside the main repo just to verify our test structure and plugin stuff works, but we can immediately pull it in after that15:19
kgriffshow close are we to a basic driver?15:19
kgriffsany chance we will have something by next tuesday?15:20
prashanthr_kgriffs: we are very close on my private forked repo15:20
prashanthr_i just need to fix a couple of stuffs15:20
prashanthr_and write some tests15:20
kgriffsgreat! You are making quick progress.15:20
prashanthr_next tuesday yes i must be able to finish it up15:20
vkmckudos prashanthr_!15:21
kgriffsI think we can call this blueprint "completed" without having to merge the driver into the main repo. Just need to see tests passing and a demo.15:21
prashanthr_kgriffs and alcabrera: the code of marconi is so well organized that working is seemless15:21
prashanthr_vkmc: he he thanks15:22
kgriffsrock on15:22
alcabreranice. I'm glad your experience working with it was a good one, prashanthr_. :)15:22
prashanthr_alcabrera: loving it :)15:22
alcabreraso then, it sounds like your next steps are to finish it up and get all the tests passing.15:23
alcabreraany thoughts/concerns, prashanthr_?15:23
prashanthr_alcabrera: yes. Just one doubt.15:23
prashanthr_while deleting a shard from the catalogue15:24
prashanthr_to clear all the queues is it acceptable to perform a flush of the db ?15:24
AAzzaprashanthr_: by the way, could i afterwards help with code review? not promise to find smth special (probably just neats in formatting, etc), but more people  reviewing - better code)15:25
prashanthr_AAzza: That would be really awesome thanks15:25
prashanthr_this is my private fork of marconi15:25
prashanthr_and my source code resides in   marconi-redis / marconi / queues / storage / redis15:26
alcabreraprashanthr_: so if a request to delete a shard comes along, it seems like the right thing to do would be to remove all queues associated with that shard, hmm...15:26
vkmcprashanthr_, are you also working in riak? neat!15:26
prashanthr_vkmc: ha ha just added support for queues and messages15:27
prashanthr_in my free time.15:27
alcabrerasuper nice15:27
prashanthr_vkmc: the rest of the implementation is just mock interfaces.15:28
alcabreraI think flushing the db will work well15:28
prashanthr_alcabrera: Sure. I will do that. Thank you. That's all i had to share for today.15:28
alcabrerakk, thanks prashanthr_!15:28
alcabreravkmc: what's the latest on amqp? :)15:29
vkmcalcabrera, nothing much to bring up on this meeting15:29
vkmcthe mapping is done I presume, and I was hoping to start working on the code for queues and messages today15:29
vkmcthe need for websockets for claims is still not clear yet15:30
alcabrerait sounded like that was related to the connection affinity problem with amqp < 1.015:30
vkmcyes it does15:30
alcabrerawhere one must issue an ACK on the same connection one started with15:30
vkmccurrent AMQP 1.0 docs are not enough15:31
alcabreraI saw that flaper87|afk is investigating this more thoroughly on our behalf. I super appreciate that he's checking that out for us!15:31
vkmcso... I'll poke some Proton devs today and ask them about the undocumented functions/attributes in the lib15:31
alcabrera sounds good. I hope we'll get some more answers there. :)15:32
alcabreraamqp support in marconi is turning out to be quite the collaborative effort, and a bit of a research project!15:32
vkmcI'm hoping that too15:32
vkmcof course!15:32
*** haomai___ has joined #openstack-marconi15:32
vkmcit's a different way of managing storage15:33
alcabreravery much so15:33
vkmcso there is a lot to discuss and there will be a lot do polish too15:33
vkmcI was hoping that flaper87 could provide some feedback about the server affinity issue15:34
alcabrerame, too15:34
vkmche knows Proton better and probably he thought about a workaround for that15:34
alcabrerathe latest I saw from flaper87|afk is that15:34
alcabrerawe might be able to work around it by putting claims on a separate amqp-topic15:34
alcabrerasomething like that15:34
vkmcsounds good15:34
vkmcin the mean time, it would be good to start seeing some progress and have the basic driver support15:35
*** haomaiwang has quit IRC15:35
alcabrerado you think you'll be able to simulate basic messaging support (-claims) where you're at now, vkmc?15:35
vkmcyes I think so :)15:36
vkmcI'm hoping to have it done for Monday15:36
vkmcso if something comes up we can discuss about it on Tuesday's meeting15:36
alcabreraI'm happy to hear we've made it to a point where we can begin playing with the code. :D15:36
alcabreraoh yeah!15:36
vkmcyes me too :)15:37
alcabreragood idea. we can leverage Tuesday's meeting to help cooordinate your effort and tjanczuk_'s15:37
alcabrerathough they are a bit different, given the 0.9 - 1.0 diff15:37
vkmcyes there are really different15:37
alcabreraother thoughts/concerns, vkmc? :)15:37
vkmcbut well, it's good that we are joining efforts and working on the support for both of them15:38
vkmcI have one tiny doubt about this bp
vkmcafaik we still don't need big changes for AMQP 1.015:39
vkmcwe can make it work without changing the API15:40
vkmcthat will change if it turns that we need websockets15:40
alcabreraoh, yeah. :/15:40
alcabrerawe'll need to watch that space.15:40
alcabreraif we determine that API changes are needed to support amqp, that'll be quite the discussion15:40
alcabrerait's already known that to support amqp+marconi would lead to partial API support15:41
vkmcso the thing is... I'll start working on this bp
alcabrerathat some operations aren't possible as is15:41
vkmcand if we discover that we have to change the API (and we all agree on that) I'll take the first one15:41
kgriffsspeaking of partial API support... we should discuss sometime soon how to approach that15:42
kgriffsbrb (meeting)15:42
vkmc+1 kgriffs15:42
alcabrerakgriffs: agreed. :)15:42
alcabreradid I address all your concerns, vkmc?15:43
vkmcyou did alcabrera, thanks15:43
kgriffsanybody know the status of this?
alcabrerakgriffs: in review, as far as I've seen15:43
vkmcnothing else to bring up, hope to have some code to peek for next meeting :)15:43
alcabrerasriram was working on it last I checked15:43
alcabreraAAzza, prashanthr_, vkmc: we're drawing to a close on the meeting for this week. Any parting thoughts/concerns? :)15:44
vkmcnot sriram, alex :)
alcabrerathanks for the correction, vkmc15:45
*** jchai is now known as jchai_afk15:45
alcabreraso that's it for the meeting this week15:45
alcabrerathanks all for joining!15:45
alcabrerafeel free to reach me if you all need any help15:45
vkmcthanks alcabrera, prashanthr_, AAzza :)15:45
*** jchai_afk is now known as jchai15:45
prashanthr_alcabrera: I will start writing the tests for redis specific drivers from tmrw. I have one more day b4 the weekend to ask q's :) thanks alcabrera, vkmc, AAzza :)15:46
vkmc+1 prashanthr_15:47
kgriffsAAzza: you are famous. I just bragged about your py3k work to Thierry, the openstack release manager. :D15:53
*** cpallares has quit IRC15:54
AAzzakgriffs: Oh... :-[15:55
kgriffskeep up the good work.15:56
prashanthr_Aazza: awesome :)15:58
prashanthr_kgriffs: I discussed with oslo.cache team today morning regarding adding support for redis16:00
maliniAAzza: woot !!16:00
prashanthr_malini : Hi good morning :)16:00
malinihello prashanthr_!16:01
*** oz_akan_ has quit IRC16:03
prashanthr_kgriffs: They have asked me to create a spec for it and have also given me some sample templates to follow.16:05
prashanthr_I will focus on that after completing the work on redis support for marconi16:06
*** goodes has quit IRC16:07
kgriffsprashanthr_: let us know how the spec process goes. A lot of projects are experimenting with that this cycle.16:08
prashanthr_kgriffs: Sure. the spec writing seems to be fairly straight forward. I will keep you updated on it.16:09
kgriffsvkmc, alcabrera: thanks for the update. I see why the status on the bp wasn't auto-updated; Alex misspelled the blueprint name in the commit message. :p16:10
kgriffsprashanthr_: is it just an RST template you fill out or something?16:11
*** prashanthr_ has left #openstack-marconi16:12
*** cpallares has joined #openstack-marconi16:14
*** rossk has joined #openstack-marconi16:14
*** goodes has joined #openstack-marconi16:17
*** openstackgerrit has quit IRC16:20
*** openstackgerrit has joined #openstack-marconi16:21
*** balajiiyer has quit IRC16:22
openstackgerritAlex Bettadapur proposed a change to openstack/marconi: Decoupled Unit Tests
*** jmckind has quit IRC16:26
*** ovorp has joined #openstack-marconi16:35
*** haomai___ has quit IRC16:58
*** ykaplan has quit IRC16:59
*** jamie_h has quit IRC17:10
*** reed has joined #openstack-marconi17:18
tjanczuk_vkmc, flapper87: what are your thoughts currently on implementing claims on top of AMQP 1.0? Did you find anything new since yesterday?17:31
*** oz_akan_ has joined #openstack-marconi17:35
tjanczuk_vkmc, flapper87: in particular, does the AMQP 1.0 library and any of the actual AMQP 1.0 backends allow acking messages using a different connection these messages were received on?17:35
tjanczuk_flapper87: could you expand on the idea of using AMQP topics to represent claims mentioned earlier?17:36
*** oz_akan_ has quit IRC17:39
*** goodes has quit IRC17:53
*** jmckind has joined #openstack-marconi17:54
*** jmckind has quit IRC18:07
*** jmckind has joined #openstack-marconi18:08
kgriffstjanczuk_: usually folks are more responsive in IRC, but I think flaper87 was on the road for a conference or something. You might try the mailing list.18:11
kgriffsgotta run for a bit (lunch)18:11
kgriffsvkmc: was in here earlier... not sure where she went. :p18:12
*** kgriffs is now known as kgriffs|afk18:12
*** reed has quit IRC18:15
*** reed has joined #openstack-marconi18:16
*** balajiiyer has joined #openstack-marconi18:26
*** oz_akan_ has joined #openstack-marconi18:35
*** oz_akan__ has joined #openstack-marconi18:38
*** flaper87|phone has joined #openstack-marconi18:39
*** oz_akan_ has quit IRC18:40
*** jmckind has quit IRC18:47
*** flaper87|phone has quit IRC18:58
*** Obulpathi has quit IRC18:59
openstackgerritNataliia Uvarova proposed a change to openstack/marconi: Small fixes in storage drivers for Python 3
vkmchere here!19:02
vkmctjanczuk_, I started working on adding queues and messages for AMQP 1.0, nothing new on claims since yesterday19:03
alcabreraAAzza: why doesn't map(str, collection) work, as compared to [str(x) for x in collection]?19:03
*** jchai is now known as jchai_afk19:04
AAzzabecause map returns iterator)19:05
tjanczuk_vkmc: so without claims, how do you propose to consume messages? I thought since the "get & consume" HTTP API is not in yet, there is really no way, is there?19:05
AAzzaalcabrera: in py3 I mean, and functions that use this storage api, kinda not happy to be able to see the values only once)19:06
alcabreraI see19:06
alcabreragood catch!19:06
*** Obulpathi has joined #openstack-marconi19:07
alcabreraAAzza: +2'd that patch19:07
AAzzaalcabrera: you are too fast) I' m just writing comments will the doubts about the patch) but thanks anyway)19:09
alcabreraI think utf8 is a safe default over ascii, AAzza19:11
vkmctjanczuk_, the only problem with claims is the ability to access the claimed messages, but the send and receive funcionalities don't have that problem since we can send a response on receive (is a feature on 1.0)19:12
vkmctjanczuk_, I heading on that direction for now... a basic driver with these functionalities19:13
vkmctjanczuk_, after that we would need to discuss the claims operation19:14
tjanczuk_vkmc: maybe I misunderstand something about current state of APIs. What is the HTTP API I call today to receive a mesaege?19:14
alcabreraAAzza: I think the .format solution for handling marker_encode is good enough.19:15
alcabrerait's consistent between versions and easy to understand19:15
AAzzaalcabrera: yes, then let it be as it is)19:16
AAzzagood night everybody19:17
alcabreraAAzza: good night!19:17
vkmcAAzza, nite :)19:17
tjanczuk_vkmc: so I was looking at the Mongo implementation of the Message.get, Message.first, and Message.bulk_get. None of them seem to *remove* messages from the database, just read them. It seems to me the only API that removes messages in Message.delete, but it requires both ClaimID and MessageID. What am I missing?19:19
vkmctjanczuk_, claimed messages can be deleted only for the consumer who claimed it19:21
vkmctjanczuk_, messages can be either active or claimed19:22
vkmctjanczuk_, if they are active, any consumer can get a copy of it or delete it19:22
vkmctjanczuk_, if it is claimed, just the consumer who previously claimed it can get a copy of it or delete it19:22
*** AAzza is now known as AAzza_afk19:22
tjanczuk_vkmc, I understand the model. All I am saying is the the *only* way to physically delete a message is to call Messages.delete. Is that correct?19:27
tjanczuk_vkmc, the Mongo implementation of Message.delete *requires* at least messageID to be provided. How do you plan to reconcile that with AMQP?19:28
vkmctjanczuk_, oh cool, sorry I wasn't sure if we were on the same page regarding the claims functionality19:30
vkmctjanczuk_, well Proton messages have an ID that can be used to filter them19:31
tjanczuk_vkmc, let's talk about a workflow of permanently consuming a single message off of the queue. What HTTP APIs would be called and how to they map onto AMQP 1.0?19:34
vkmctjanczuk_, for now we should focus on the simple get, this is without id, since we were talking about the remove that from the API
vkmcs/the remove/removing19:35
tjanczuk_vkmc, in your proposal, are you assuming the removal of IDs from these APIs, or are you trying to build on what's in the tree right now? I am referring to what's in the code right now.19:39
vkmctjanczuk_, I'm assuming the API as is, but I'm considering the possibility of those functionalities being removed19:40
vkmctjanczuk_, it's just a side note19:40
tjanczuk_vkmc, so I still don't understand how you plan to implement the "consume one message from a queue19:40
tjanczuk_using today's storage contract mapped onto AMQP. Can you walk me through it? What APIs are called and in which order? How do they map to AMQP?19:41
vkmctjanczuk_, I'm implementing those right now, would you like to discuss more about this tomorrow? Proton lib have many features that can be used to handle messages, I'm figuring out how to use them accordingly19:44
tjanczuk_OK, let's talk more about it tomorrow.19:46
vkmctjanczuk_, how are things going on your side with AMQP < 1.0?19:47
tjanczuk_I think supporting AMQP 0.9 will require changes in both HTTP APIs and storage driver abstraction. (I also suspect the same will turn out to be true for AMQP 1.0).19:49
tjanczuk_Some of the issues I encountered:19:49
tjanczuk_1) AMQP 0.9 requires message acknowledgement on the same instance of AMQP channel that the message was received on. This is a major constraint that I think can only be reasonably addressed by exposing a connection-oriented endpoint to the client (e.g. WebSockets) which will support server affinitity.19:52
tjanczuk_2) The concept of message ID does not exist in AMQP 0.9, so all operations that accept or return message ID would require changes.19:52
tjanczuk_3) The concept of a "claim" in Marconi is unlike the corresponding concept in AMQP (0.9, 1.0), SQS, or Azure Queues. In Marconi a claim maps to N messages, and (in the current implementation) must be used in conjunction with a message ID to delete the message. In AMQP (0.9 and 1.0), as well as SQS and Azure Queues, a "claim" is associated with a single message. Think about it as a transient message ID created by the server at the time message is19:57
tjanczuk_4) Of course all the APIs that list messages or get messages by ID cannot be supported by AMQP, but we knew this all along.19:58
tjanczuk_5) The Message.first API is strange in that it allows sorting the queue. No sorting can be done with AMQP. I am not quite sure I understand why that API is there (except sorting), since the same can be accomplished with Message.get19:59
*** jchai_afk is now known as jchai19:59
tjanczuk_It looks more and more likely to me there is no way AMQP can be accommodated with just cosmetic changes to V1.1 HTTP APIs or storage driver abstraction.20:00
tjanczuk_I think I am going to spend some time prototyping how a "V2" HTTP API that accommodates well both Mongo and AMQP would look like, and validate that HTTP API with AMQP 0.9 implementation. Thoughts?20:02
tjanczuk_This is what alcabrera referred yesterday as a "new transport" I think.20:03
*** goodes has joined #openstack-marconi20:07
*** flwang has quit IRC20:07
*** malini is now known as malini_afk20:12
*** goodes has left #openstack-marconi20:16
*** flwang has joined #openstack-marconi20:17
*** sriram has quit IRC20:33
*** balajiiyer has quit IRC20:39
*** balajiiyer has joined #openstack-marconi20:48
*** alcabrera is now known as alcabrera|afk20:54
*** balajiiyer has quit IRC20:58
*** malini_afk is now known as malini21:06
*** Obulpathi has quit IRC21:45
*** oz_akan__ has quit IRC21:46
*** jchai has quit IRC21:50
*** malini is now known as malini_afk21:54
*** malini_afk is now known as malini22:00
*** cpallares has quit IRC22:01
*** oz_akan_ has joined #openstack-marconi22:48
*** oz_akan_ has quit IRC22:55
*** malini is now known as malini_afk23:15
*** sriram has joined #openstack-marconi23:36
*** sriram has quit IRC23:48
*** oz_akan_ has joined #openstack-marconi23:56

Generated by 2.14.0 by Marius Gedminas - find it at!