Monday, 2014-06-09

*** nosnos has joined #openstack-marconi01:43
*** nosnos has quit IRC03:18
*** nosnos has joined #openstack-marconi04:06
*** haomaiwang has joined #openstack-marconi04:19
*** haomaiw__ has joined #openstack-marconi04:22
*** haomaiwang has quit IRC04:23
*** vkmc has quit IRC04:26
*** haomaiw__ has quit IRC04:32
*** haomaiwang has joined #openstack-marconi04:33
*** renlt has joined #openstack-marconi05:34
*** whenry has joined #openstack-marconi05:35
*** davidhadas has joined #openstack-marconi06:47
*** nosnos has quit IRC06:57
*** davidhadas has quit IRC07:00
*** davidhadas has joined #openstack-marconi07:00
*** mgibert has joined #openstack-marconi07:41
*** amalagon_ has joined #openstack-marconi07:56
*** amalagon has quit IRC07:58
*** flaper87|afk is now known as flaper8708:01
*** russell_h has quit IRC08:01
*** russell_h has joined #openstack-marconi08:02
*** openstackgerrit has quit IRC08:02
*** davidhadas_ has joined #openstack-marconi08:06
*** openstackgerrit has joined #openstack-marconi08:07
*** davidhadas has quit IRC08:08
*** russell_h has quit IRC08:13
*** russell_h has joined #openstack-marconi08:16
*** renlt has quit IRC08:17
*** russell_h has quit IRC08:21
*** russell_h has joined #openstack-marconi08:24
*** russell_h has quit IRC08:31
*** russell_h has joined #openstack-marconi08:33
*** russell_h has quit IRC08:38
*** russell_h has joined #openstack-marconi08:42
*** russell_h has quit IRC08:58
*** russell_h has joined #openstack-marconi08:59
*** ykaplan has joined #openstack-marconi09:00
*** russell_h has quit IRC09:02
*** russell_h has joined #openstack-marconi09:04
*** russell_h has quit IRC09:14
*** russell_h has joined #openstack-marconi09:14
*** russell_h has quit IRC09:18
*** russell_h has joined #openstack-marconi09:20
*** russell_h has quit IRC09:26
*** ykaplan has quit IRC09:58
*** russell_h has joined #openstack-marconi10:01
*** haomaiwang has quit IRC10:41
*** haomaiwang has joined #openstack-marconi10:41
*** haomaiw__ has joined #openstack-marconi10:45
*** haomaiwang has quit IRC10:46
*** ykaplan has joined #openstack-marconi10:58
*** davidhadas has joined #openstack-marconi11:20
*** davidhadas_ has quit IRC11:23
*** davidhadas has quit IRC11:25
*** haomaiw__ has quit IRC11:26
*** haomaiwang has joined #openstack-marconi11:26
*** vkmc has joined #openstack-marconi11:47
*** vkmc has quit IRC11:47
*** vkmc has joined #openstack-marconi11:47
flaper87russell_h: ping11:57
flaper87russell_h: I think you're IRC bouncer went nuts :P11:57
vkmc:o11:58
vkmcmorning flaper87! :)11:58
flaper87vkmc: gooooooooooooooooooooooooood morning :)11:58
flaper87vkmc: how are you doing?11:58
vkmcflaper87, all good and you? how was your weekend?11:59
flaper87vkmc: it was refreshing... I spent the whole weekend out, walking around the lake, meeting with friends and family.12:00
flaper87vkmc: yours ?12:00
vkmcflaper87, oh that's awesome :)12:01
vkmcflaper87, I visited my friends and family as well... but no walk outdoors, it's pretty cold here12:02
flaper87What? No visit to those poor pinguins ?12:02
flaper87penguins12:03
vkmcboo they are far away from where I live, so no penguins for me this weekend12:04
*** mwagner_lap has quit IRC12:08
vkmcflaper87, let me know if you have a moment later to discuss about amqp :)12:08
flaper87vkmc: yup, gimme 1h or 2. Would that work for you?12:09
*** ykaplan has quit IRC12:09
vkmcflaper87, of course, there is no rush12:09
*** ykaplan has joined #openstack-marconi12:20
openstackgerritFlavio Percoco proposed a change to openstack/marconi: Rename shards to pool  https://review.openstack.org/9646312:23
*** jchai has joined #openstack-marconi12:47
*** sriram has joined #openstack-marconi12:48
openstackgerritFlavio Percoco proposed a change to openstack/marconi: Rename shards to pool  https://review.openstack.org/9646312:48
openstackgerritFlavio Percoco proposed a change to openstack/marconi: Start scratching Queue Flavors api  https://review.openstack.org/9877712:48
sriramGood Morning! :)13:02
vkmcmorning sriram!13:05
flaper87sriram: good morning13:20
sriramgood morning flaper87, vkmc :)13:21
sriramflaper87: could you take a look at https://review.openstack.org/#/c/97875/? , its blocking marconi-bench. it had a +2 before, I added some tests afterwards and thats gone now :P13:22
*** Obulpathi has joined #openstack-marconi13:31
*** Obulpathi has quit IRC13:31
*** malini has joined #openstack-marconi13:32
*** Obulpathi has joined #openstack-marconi13:32
*** prashanthr_ has joined #openstack-marconi13:40
openstackgerritFlavio Percoco proposed a change to openstack/marconi: Add flavors support to mongodb  https://review.openstack.org/9879313:40
*** ykaplan has quit IRC13:41
* flaper87 clicks13:44
flaper87sriram: reviewed, -1 but easy fixes13:49
*** balajiiyer has joined #openstack-marconi13:54
sriramthanks flaper87 will address them :)13:55
*** mwagner_lap has joined #openstack-marconi14:00
*** amitgandhi has joined #openstack-marconi14:00
*** rossk has joined #openstack-marconi14:02
*** abettadapur has joined #openstack-marconi14:03
*** amitgandhi has quit IRC14:03
*** amitgandhi has joined #openstack-marconi14:04
*** rwsu has joined #openstack-marconi14:04
*** alcabrera|afk is now known as alcabrera14:06
alcabreragood morning, everyone. :)14:06
vkmcgood morning alcabrera!14:07
flaper87alcabrera: gooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooood morning14:07
alcabreravkmc: heya! How was your weekend?14:07
alcabreraflaper87: heeeeeeeeeeeeeeeyyyyyyyyyyyyyyy :D14:08
alcabreraI heard you had an awesome weekend, flaper87. Glad to hear it. <314:08
flaper87yeah, I needed it :D14:08
flaper87what about you?14:08
vkmcalcabrera, it was great thx! and yours? :)14:08
prashanthr_good morning alll :)14:08
vkmcprashanthr_, hi! :)14:08
prashanthr_vkmc : Hi :)14:08
alcabreraprashanthr_: hey! :)14:08
alcabreravkmc: my weekend was super comfy. I spent time with jess, hung out with a friend at home, bought some new clothes, played minecraft, and relaxed a whole lot. :)14:09
openstackgerritSriram Madapusi Vasudevan proposed a change to openstack/python-marconiclient: Throw exceptions on erroneous status codes  https://review.openstack.org/9787514:09
vkmcawesome! :)14:10
*** whenry has quit IRC14:11
malinihello all :)14:18
alcabreraheya, malini. :)14:19
prashanthr_malini : Hi , Good morning :)14:19
vkmcmalini, hi! :)14:19
maliniheyyy alcabrera, prashanthr_, vkmc !14:19
maliniI missed most of last week with my Texas trip14:20
maliniBut here I am, back again :)14:20
malini& As usual, I started my day with a tempest patch :)14:20
*** Alex_Gaynor has quit IRC14:20
sriramhah14:20
sriramhey alcabrera,prashanthr_ :)14:21
*** Alex_Gaynor has joined #openstack-marconi14:21
alcabrerasriram: good morning!14:21
prashanthr_sriram: Hi , Good mornin :)14:21
sriramalcabrera: good morning!, I see you had an awesome weekend! :)14:21
alcabreraoh yes, sriram. It was lovely. :)14:21
sriramprashanthr_: Good morning :)14:22
prashanthr_alcabrera: I started writing the test cases for redis today, so we write the tests for the private methods and the utility functions am i Right ?14:27
*** kgriffs|afk is now known as kgriffs14:28
alcabreraprashanthr_: yes. :)14:29
alcabreraespecially with a dynamically-typed language like python, you've gotta test everything you can.14:29
prashanthr_alcabrera: Awesome, sure I have quite some tests. Will look at some more scenarios.14:30
alcabrerathanks, prashanthr_!14:31
sriramflaper87: the patch is back :P https://review.openstack.org/#/c/97875/14:32
sriramalcabrera: ^14:32
sriramkgriffs: ^14:32
srirammalini: ^14:32
* sriram spams everyone! :-P14:32
* alcabrera watches sriram open a can of spam14:33
* alcabrera clicks the link anyway14:33
alcabrera;P14:33
alcabrerasriram: +2/A14:34
sriramwow, ninja +2's :P14:34
sriramalcabrera: thanks!14:34
openstackgerritA change was merged to openstack/python-marconiclient: Throw exceptions on erroneous status codes  https://review.openstack.org/9787514:36
alcabreraseems like the queue is swift this morning14:38
sriramok, kgriffs I'll submit a patch for marconi-bench now. should the files be under /marconi-bench outside the package?14:47
sriramdo we want to name it something else?14:47
kgriffssriram: I think you can just call the dir "bench"14:49
*** haomaiwang has quit IRC14:49
kgriffsas far as having it outside the package...14:49
kgriffsthe question is, do we want to make it pip installable from pypi?14:49
*** haomaiwang has joined #openstack-marconi14:49
*** cpallares has joined #openstack-marconi14:50
sriramhmm, good point. would that require a different repository in itself just for bench?14:50
*** amitgandhi has quit IRC14:52
kgriffswell, we could include the benchmark tools in the marconi package and register a "marconi-bench" command using a console_scripts entry point14:53
*** balajiiyer has quit IRC14:53
kgriffsor we could put things in the same repo, but not in the same package14:54
kgriffsor we could put things in the client repo14:54
kgriffsor in its own repo14:54
flaper87o/14:54
kgriffsI have some thoughts, but would like to see what everyone thinks first14:55
kgriffsflaper87:  \o14:55
alcabreraif the benchmark program addresses the behavior of the server, I believe it should live with the server14:55
alcabreraif it covers more systems than marconi, then it should be its own PyPI package (e.g., queue-bench)14:55
* kgriffs wonders who let R. Flavio in14:55
* flaper87 has no idea14:55
* flaper87 is a victim14:56
* kgriffs hopes nobody realizes my true name starts with an R. as well14:56
*** rongze has joined #openstack-marconi14:56
* kgriffs wonders how many people still read Asimov these days.14:56
malinikgriffs, flaper87: Can you take a look at this one plz https://review.openstack.org/#/c/95228/ ?14:57
* flaper87 says hi to Sir. Robert Kurt14:57
* flaper87 clicks14:57
kgriffsalcabrera: I think it is going to be specific to Marconi14:57
flaper87+214:58
* kgriffs What "R" stands for: http://goo.gl/7WhMRc14:58
maliniyayy!14:58
flaper87btw, I started doing some work on the queue flavors thing14:58
alcabrerakgriffs: cool. I'm very favorable towards -- mkdir benchmark; touch benchmark/marconi-bench; pip install .; marconi-bench14:58
alcabrera /cc sriram ^^14:58
sriramThat seems to make sense.14:59
kgriffsor we could just install it by default as part of the server package14:59
* flaper87 likes the benchmark package idea14:59
sriramkgriffs: the conductor could be our console entry point14:59
kgriffshmm15:00
alcabreravkmc, flaper87: I'm catching up on email, but I'd love to join the amqp discussion whenever you all have it. :)15:00
kgriffsif we make it in the same repo, but it's own package, what would be the process for someone to set this up?15:00
flaper87alcabrera: oh right, mmh, I'm ready15:00
kgriffsgrab the tarball, unzip, cd in there, pip install . ?15:00
sriramhmm15:01
kgriffssounds reasonable... but definitely needs some docs15:01
flaper87kgriffs: mmh, what about installing it with marconi ?15:01
flaper87not sure why it needs a separate install process15:01
alcabrera+1 for install w/ marconi15:01
kgriffswould make installation simpler15:01
sriramthat seems interesting. it could live in the same repo, but be a different package? It might complicate things a bit15:02
flaper87why?15:02
kgriffson the other hand, installs stuff on the benchmarking clients that isn't needed. Well, unless you are benchmarking everything on the same box. :p15:02
vkmcalcabrera, cool! :)15:02
sriramI like the idea of it being installed along with marconi15:02
flaper87I've seen this done in other parts too15:02
flaper87I mean, same repo, different package, same install script15:02
kgriffsno, I mean, it would be simpler to have it part of the same package15:02
kgriffs(as you suggested)15:02
kgriffsok15:02
kgriffsnow I am thoroughly confused.15:03
sriramkind of like redis benchmark, which is included with redis.15:03
* flaper87 is confused15:03
flaper87LOL15:03
alcabrerasriram: +1. Good example. :)15:03
kgriffsok, sure, let's do that15:03
flaper87so, now, wait15:03
kgriffsLOL15:03
flaper87are we talking about having marconi/benchmark ?15:03
flaper87or marconi/marconi/benchmark15:03
flaper87?15:04
alcabreragood thought, flaper8715:04
alcabreraI'm favorable towards marconi/benchmark15:04
kgriffsso that is same repo, standalone package, right?15:04
sriramhmm yes.15:04
flaper87the former is same repo, standalone package15:04
flaper87the later is same-everything15:04
flaper87even blood type15:04
maliniabettadapur: Can you uncomment these skips when you get a chance https://github.com/openstack/marconi/blob/master/tests/functional/wsgi/v1/test_claims.py#L69 . It is fixed now15:04
alcabrerahmmm15:04
flaper87the later, though, will force us to ship benchmark with marconi15:05
alcabrerawhy would they have to be a separate package given marconi/benchmark?15:05
sriramif its a different package, does it get installed with a pip install -e .15:05
flaper87(.dbs, targz, .rpms)15:05
flaper87not very useful but useful anyway15:05
flaper87sriram: it depends on your setup.py (or setup.cfg)15:05
abettadapurmalini: on it15:05
sriramflaper87: I see.15:05
flaper87but yeah, it can be installed by just doing: pip install -e .15:05
srirambrb15:07
flaper87sriram: command not found, stay there15:07
abettadapurmalini: as a new patch?15:08
abettadapur(can patches be added on to already merged things?)15:08
maliniabettadapur: a new patch15:08
abettadapurok15:08
sriramflaper87: LOL15:09
* sriram gets back silently15:10
sriramso marconi/benchmark standalone package?15:12
*** kgriffs is now known as kgriffs|afk15:14
*** ykaplan has joined #openstack-marconi15:15
sriramflaper87,kgriffs|afk,malini,alcabrera : https://review.openstack.org/#/c/97534/ , this also could also use a review.15:16
vkmcalcabrera, I'm working on https://bugs.launchpad.net/marconi/+bug/1257391, if you have a moment I'd like to ask you about the inheritance hierarchy we are using there15:16
* sriram has been quite spammy today15:16
alcabrerasriram: you're fine. :)15:16
sriramalcabrera: heh :P15:16
malinisriram: its at jenkins −1 :(15:17
srirammalini: the failing tests are tempest related.15:17
malinisriram: let me chk15:17
abettadapurmalini: i hvae the same errors on my unittest patch15:17
abettadapuran ssh connection timesout15:17
alcabreravkmc: I've a moment. let's talk about it. :)15:17
abettadapurmalini: cirros@172.24.4.515:18
malinisriram, abettadapur: the failure is no way related to marconi….grrrr15:18
srirammalini: yes :/15:18
malinilets try a recheck, if it has gone away now15:18
flaper87wait15:18
sriramshould we do recheck no bug?15:18
flaper87the gate is still under HEAVY load and moving very slow15:19
malinisriram: yes15:19
flaper87it'd be better not to recheck so frequently15:19
flaper87in order to let other patches that will help fixing the gate to move forward15:19
maliniok..we'll hold off on rechecks till the gate is happy again15:19
vkmcalcabrera, great, thanks :) I'm wondering why we are not inheriting directly from ControllerBaseTest instead of TestBase instead https://github.com/openstack/marconi/blob/master/tests/unit/queues/storage/test_impl_mongodb.py#L13215:20
vkmc(I know I'm not providing you enough context but I don't want to flood the channel with links)15:20
*** sriram1 has joined #openstack-marconi15:21
alcabreralet's see...15:22
alcabreravkmc: for that case, it doesn't inherit from the ControllerBaseTest because the driver-level code isn't a controller15:22
alcabrerait instead tests the sheer basics -- can it connect? can the data be purged?15:23
*** sriram has quit IRC15:23
alcabrerathe context was sufficient in this case. :)15:23
vkmcalcabrera, makes sense15:24
alcabrerafeel free to provide as many links as you need to express the context in the future, though!15:24
alcabrera:D15:24
alcabrerasince that context may also help others understand15:24
alcabreraand learn about marconi15:24
vkmcalcabrera, will do :)15:24
alcabrerayay!15:24
*** Obulpath_ has joined #openstack-marconi15:28
*** sriram has joined #openstack-marconi15:28
*** kgriffs|afk is now known as kgriffs15:29
kgriffssorry folks15:29
kgriffshad a network blip15:29
kgriffssriram, alcabrera, flaper87: where will /benchmark go?15:30
*** abettadapur has quit IRC15:30
flaper87kgriffs: I guess separate package, same repo15:31
*** Obulpath_ has quit IRC15:31
flaper87that'd be my first option. We should have benchmarks/queues there15:31
* kgriffs prefers "bench" for no good reason15:31
* flaper87 prefers bench too15:31
kgriffsok15:31
kgriffsso15:31
*** Obulpath_ has joined #openstack-marconi15:31
kgriffs /marconi/bench/queues15:31
*** Obulpathi has quit IRC15:32
*** sriram has quit IRC15:32
*** whenry has joined #openstack-marconi15:32
*** sriram1 has quit IRC15:32
*** sriram has joined #openstack-marconi15:32
kgriffswhich will benchmark15:32
kgriffs /marconi/marconi/queues15:32
sriramnetwork troubles. :/15:32
kgriffssriram: wifi seems to work for me15:32
*** russell_h has quit IRC15:32
*** russell_h has joined #openstack-marconi15:32
sriramI'm randomly getting kicked off.15:33
sriramok so marconi/bench, different package and same repo15:33
sriramI missed some parts of the conversation, could you guys fill me in?15:34
*** ykaplan has quit IRC15:34
* sriram needs to setup znc15:34
openstackgerritA change was merged to openstack/python-marconiclient: fix: add https support  https://review.openstack.org/9522815:35
sriramkgriffs: ^15:36
kgriffs<random>Why didn't anyone tell me about Charles? All this time I've been mourning the loss of Fiddler when I repented of my sins and stopped using Windows</random>15:36
* kgriffs drops a link. http://www.charlesproxy.com/15:36
sriramkgriffs: on another note, we need client 0.0.2, has it already been released?15:37
kgriffsum15:39
kgriffsnot to my knowledge15:39
sriramabettadapur smoke tested it, and it worked fine.15:39
kgriffsfortunately somebody wrote up a nice wiki page detailing how that is done15:40
kgriffs;)15:40
kgriffshttps://wiki.openstack.org/wiki/How_to_Release_(Marconi)15:40
sriram:)15:40
*** Obulpath_ has quit IRC15:41
kgriffshmmm15:41
*** Obulpathi has joined #openstack-marconi15:41
kgriffsnormally we want to track the server milestones15:41
kgriffshow about this15:41
kgriffswe release 0.0.215:42
kgriffsthen we release 1.0b1 later this week.15:42
kgriffsthey will basically be the same thing, but I am concerned that pip won't install 1.0b1 over 0.0.115:43
kgriffssince the former is a beta15:43
sriramI see, oh. Good point15:43
kgriffsam I wrong in that assumption?15:43
kgriffsflaper87: thoughts?15:43
kgriffsalternatively, we could change the version mapping15:44
* flaper87 reads the backlog15:44
flaper87do we need a new client release?15:44
kgriffsso that the PEP440 version doesn't have the beta flag15:44
kgriffsflaper87: yes, because claims are broken right now. you can't delete claimed messages15:44
flaper87oh, mmh, in that case we should release a minor version15:44
*** rongze has quit IRC15:44
flaper87I guess 0.0.215:45
flaper87no wai, did we release 1.0 already ?15:45
kgriffsno15:45
flaper87we released 0.0.115:45
flaper87right ?>15:45
kgriffsright15:45
*** sriram1 has joined #openstack-marconi15:45
kgriffshere is the proposed release schedule: https://wiki.openstack.org/wiki/How_to_Release_(Marconi)15:45
kgriffs(under "versionioning"15:45
kgriffsblah15:45
* kgriffs can't type15:45
kgriffsthat section doesn't address what to do with critical bug fixes15:46
kgriffsI assume you would do a minor bump on the latest non-beta/rc version15:46
*** Obulpath_ has joined #openstack-marconi15:46
flaper87I think we can keep the major versions aligned, doing so for the minor ones will be hard15:46
flaper87since bug fixes are minor worth it15:46
kgriffsbug fixes could go in the third version15:47
kgriffslike15:47
kgriffs1.0 is released15:47
kgriffsthen we do bug fixes15:47
kgriffsand release 1.0.115:48
*** sriram has quit IRC15:48
kgriffs1.0.215:48
kgriffsetc.15:48
*** sriram1 is now known as sriram_afk15:49
*** Obulpathi has quit IRC15:49
flaper87yup15:49
flaper87that sounds aboud right15:49
kgriffsok, we can continue to refine as we go. I'll make a note on the release wiki15:50
*** amitgandhi has joined #openstack-marconi15:53
*** balajiiyer has joined #openstack-marconi15:53
*** amitgandhi has quit IRC15:53
*** amitgandhi has joined #openstack-marconi15:54
*** jchai has quit IRC15:54
*** jamie_h has joined #openstack-marconi15:57
*** jamie_h has quit IRC15:57
*** malini is now known as malini_afk15:57
*** jamie_h has joined #openstack-marconi15:58
*** prashanthr_ has quit IRC15:58
*** jdprax has joined #openstack-marconi15:59
*** balajiiyer has quit IRC16:00
*** balajiiyer has joined #openstack-marconi16:00
*** ykaplan has joined #openstack-marconi16:07
kgriffseveryone: got word back from openstack legal on names16:17
kgriffswe will discuss some more tomorrow, but here is the gist16:17
kgriffs"Our trademark counsel did a search and recommended Zaqar or Naav as the best potential names with low-to-moderate risk. Tamtam came up as a moderate risk based on TamTamy, but we could still likely pursue it because the software is quite distinguishable. Both Raven and Copper pose a higher risk, because they are registered to companies in dev / cloud spaces (Ravenflow and Copper.io respectively)."16:17
kgriffsflaper87, flwang, alcabrera, malini_afk: ^^16:18
kgriffsspeaking of the meeting tomorrow, here is the proposed agenda. tweak as needed before tomorrow: https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda16:19
* flaper87 votes for Naav16:28
vkmc:)16:29
vkmchmm.. Naav makes me think about Avatar16:32
*** jay-atl has joined #openstack-marconi16:33
kgriffsany folks from India in the house? I think Naav is a river there. How is it pronounced?16:33
sriram_afkkgriffs: I think Naav means boat16:36
*** sriram_afk is now known as sriram16:37
openstackgerritA change was merged to openstack/marconi: Prepare marconi for localization  https://review.openstack.org/9791416:39
kgriffsoic16:40
*** reed has joined #openstack-marconi16:48
sriramwith regards to pronunication16:49
vkmcbrb16:59
sriramN and v are their phonetically equiavalents, the double 'aa' represents the sound the dentist asks you to make, on opening your mouth really wide.17:01
sriramok bad joke :P17:01
flaper87LOL17:01
flaper87you got me trying to pronounce it17:02
flaper87I was like :O (aaaaaa)17:02
kgriffsbrb. lunch17:08
*** kgriffs is now known as kgriffs|afk17:09
*** ovorp has quit IRC17:10
*** jamie_h has quit IRC17:16
*** amitgandhi has quit IRC17:17
sriramflaper87: its actually accurate :P17:28
*** haomaiwang has quit IRC17:30
vkmcback17:34
*** ykaplan has quit IRC17:35
*** russell_h has quit IRC17:36
flaper87vkmc: hey, so, amqp17:38
vkmcflaper87, !17:38
alcabrerayes, amqp. :D17:38
* alcabrera listens17:38
flaper87sorry, I know I said 2h :(17:39
*** russell_h has joined #openstack-marconi17:39
flaper87vkmc: shoot, what's in your mind? You've done a great work investigating things17:39
flaper87have you written that somewhere?17:39
flaper87Some kind of mapping between features and what we can support17:39
vkmchaha np17:40
vkmcflaper87, dunno if you saw the email I sent yesterday17:40
flaper87yup17:40
flaper87I read them all, I was going to reply when you said you wanted to have a chat17:40
flaper87so, I preferred to wait and chat :)17:40
vkmcsure, it's indeed better :)17:40
vkmcso I've been struggling with connection handling17:41
*** russell_h has quit IRC17:41
*** russell_h has joined #openstack-marconi17:41
flaper87vkmc: re pyngus, I think that's the way to go17:41
flaper87I mentioned I'll have a call with them tomorrow, didn't I?17:41
vkmcgreat! so I could start polishing things from there17:42
vkmcyes you did17:42
*** amitgandhi has joined #openstack-marconi17:42
vkmcwhat time are you having that call? I could try to join you just as an spectator (so you can discuss without waiting me to understand) if the times suites me17:43
peoplemergeg'morning all! Put in a few hours this weekend reading marconi's tests.17:43
vkmchey peoplemerge! o/17:44
flaper87peoplemerge: AWESOME! Any feedback so far?17:44
flaper87vkmc: so, just pinged one of the guys, it'll probably be on Thursday17:45
flaper87that said, lets go through your doubts17:45
peoplemergeflaper87: yes, I thought they were pretty easy to read17:45
vkmcflaper87, cool17:45
flaper87you made really good points there17:45
flaper87peoplemerge: glad to hear that :)17:45
peoplemergeflaper87: they also were a good intro to messages, queues, and claims17:46
peoplemergeClaims were something new to me.17:46
peoplemergeI have some basic product questions I'd like to understand as well17:46
flaper87peoplemerge: shoot17:47
peoplemergeFor example, is it true that when a client sends a claim, and another one does too, there can be an overlap of messages?17:47
flaper87vkmc: most of your concerns in that email are related to the lack of a brocker17:48
flaper87vkmc: broker17:48
*** malini_afk is now known as malini17:48
vkmcflaper87, yeah I had that feeling17:48
flaper87peoplemerge: nope, the first to claim wins the battle17:48
peoplemergegreat17:48
flaper87the second one will claim a different set of messages17:48
peoplemergeso no race conditions17:48
peoplemergeIt's also possible to perform GET on a message.  When would one use that?17:49
flaper87vkmc: ok, there are brokers supporting AMQP 1.0 which means we should probably just focus on those.17:49
flaper87peoplemerge: we've thought about removing the per-message GET. NO consensus has been reached, though.17:49
flaper87the use case is still foggy, TBH. One use case would be having temp-urls for messages17:50
vkmcflaper87, I saw in Proton forums that they are working on client side CRUD operations for brokers http://qpid.2158936.n2.nabble.com/Creating-queues-with-PROTON-messenger-td7593967.html17:50
vkmcflaper87, it's a bit old... not sure which is the current state of it17:50
peoplemergeflaper87: so as a client, I might send a message, and want to check back on it later?17:51
flaper87peoplemerge: that's one use case, yes.17:51
peoplemerge"hey message, are you still there waiting to be delivered?"17:51
peoplemergegot it17:52
flaper87vkmc: yeah, the store-forward message pattern will be needed here17:52
flaper87that is, having a broker seems to be mandatory17:52
vkmcflaper87, regarding the broker, I'm using ActiveMQ currently17:52
flaper87vkmc: sounds good. ActiveMQ and Apollo use the Java bindings for proton, AFAIK17:53
vkmcflaper87, It's possible yeah... I'm being able to interact with it with Python client though... and I haven't build proton-j17:53
vkmcflaper87, I also tried it out with qpid-cpp but I'm afraid that its support of AMQP 1.0 is not clear17:55
flaper87you've to enable it manually. I think it's there since 0.2217:55
vkmcoh I missed that part17:55
vkmcI'll check it out then17:56
vkmcone nit... if we will be implementing this driver with a broker in mind we are losing a lot of flexibility :/17:56
vkmcnot sure how we could make it work... will the broker be a requirement?17:57
peoplemergeflaper87: I had also wondered if there was a horizon plugin. Found a ticket that says it's on Juno's roadmap so that answered my question17:57
peoplemergeit did raise the question of multitenancy which I gather marconi supports17:57
flaper87vkmc: what flexibility are you referring to?17:58
peoplemergeBut what's the lifecycle of a marconi server?  When you create a new project, does another marconi server spin up?17:58
vkmcflaper87, I meant broker independence17:58
vkmcone of the beauties of AMQP 1.017:59
*** malini is now known as malini_afk17:59
flaper87peoplemerge: nope, you can use the same instance18:00
flaper87vkmc: that's true. However, you can also think as the brokers as intermediary peers. (bridge-peers)18:01
flaper87the benefit, though, is that we won't depend on a single broker and we'll be able to federate everything using qpid-dispatch18:02
*** mkoderer has quit IRC18:02
flaper87I can think of a future API that will allow us to have support for peer-to-peer messaging through Marconi. But that's far far far far away18:02
flaper87does that make sense?18:03
flaper87vkmc: ^18:03
vkmcflaper87, so... we would have to deal with AMQP routing mechanism as well?18:03
flaper87vkmc: you mean, qpid-dispatch ?18:03
peoplemergeflaper87: is it possible to scale instances other than scaling the backend?18:03
vkmcflaper87, yeah18:04
flaper87vkmc: not mandatory, qpid-dispatch is just an easy way to scale brokers horizontaly18:04
flaper87we could also use Marconi pools + many brokers18:04
flaper87peoplemerge: yup, you can add as many marconi instances as you wish18:04
vkmcflaper87, hmm I would need to take a look at some examples of it18:06
flaper87vkmc: marconi pools?18:06
vkmcflaper87, yeah, and the qpid-dispatch alternative18:06
flaper87vkmc: lets forget about qpid-dispatch for now18:07
flaper87we don't need it to work on the driver18:07
vkmc:D18:07
* vkmc lost 18:07
flaper87hahahahha18:07
flaper87sorry, that's probably my fault18:07
vkmcnah it's ok... I'm giving my first steps here so there are a lot of things I don't know yet18:08
flaper87vkmc: small recap. We need an AMQP 1.0 capable broker, proton, pyngus and coffee18:08
flaper87(note that I said coffee, not coffeescript)18:08
vkmcscaling brokers is one of those things18:08
vkmcyes, coffee and gummy bears18:08
flaper87vkmc: I said this in a talk last week: "People say that it's now possible to scale brokers and that you can simply read the docs and cluster it. Well, that's bullshit"18:09
*** kgriffs|afk is now known as kgriffs18:09
vkmclol18:09
peoplemergeflaper87: hurrah18:10
*** malini has joined #openstack-marconi18:10
vkmc(talking abt gummy bears... I saw this the other day and thought you would like it http://makezineblog.files.wordpress.com/2014/01/gummy-bear-popsicles-1.jpg?w=600&h=647)18:10
flaper87I also said that OpenStack is "software-stable" (someone asked whether openstack is stable or not and I couldn't help it)18:10
flaper87OMFG, I'm sooooo doing that18:11
vkmchahaha18:11
vkmcwell, so I already have an AMQP 1.0 capable broker, proton and pyngus18:12
flaper87vkmc: bring the code in :)18:12
vkmcregarding the connection handling... we want Marconi (an AMQP container) interact with the broker18:12
vkmca container can have 0...* nodes, being nodes queues, topics, producers, consumers (...)18:13
vkmcright now Proton allows us to create producers and consumers... but they behave as it were IPC... not like if we had an storage in between18:14
vkmcand that is my main concern18:14
flaper87vkmc: re marconi pools. Not sure if this helps: https://speakerdeck.com/flaper87/marconi-queuing-and-notification-service-for-openstack?slide=3018:15
* vkmc clicks18:15
vkmceverything helps18:15
* vkmc got it 18:15
vkmcoh I see what you meant... yes, qpid-dispatch would be a far more elegant implementation18:16
flaper87vkmc: is that code up-there already ?18:16
flaper87vkmc: we can do many fancy things there but lets talk about those later18:17
vkmcflaper87, yes https://github.com/vkmc/marconi-amqp1018:17
vkmcflaper87, of course, let's focus on the most basic use case... one broker + one client18:18
*** kgriffs is now known as kgriffs|afk18:19
flaper87ah right, I had it opened already :P18:19
vkmcthat code haven't been tested yet... I wanted to be sure about Marconi's role in AMQP scenario before going further with it18:19
flaper87vkmc: ok, do you want to do more experiments and get back with the results?18:20
peoplemergeflaper87: about that horizon plugin, I sat in on one of the horizon design sessions.  They're adding AngularJS for Juno, think that could be an option for marconi's dash?18:21
vkmcflaper87, first I would need to clear out how to manage connections18:21
flaper87vkmc: do you really need the parse function ? I thought pyngus did that already18:21
*** kgriffs|afk is now known as kgriffs18:21
flaper87peoplemerge: mmh, probably. Someone in RAX was working on the horizon plugin. Not sure what the state of that is18:21
flaper87vkmc: sure18:21
flaper87vkmc: OTOH, you need to create a consumer when you need a new message18:22
flaper87keeping a consumer alive means you'll consume a message as soon as it's posted in the broker18:22
vkmc:T18:22
vkmcmakes sense18:22
openstackgerritMalini Kamalambal proposed a change to openstack/marconi: Implement POP in v1.1 API  https://review.openstack.org/9020218:23
vkmcflaper87, in which line are you seeing the parse function?18:25
flaper87vkmc: https://github.com/vkmc/marconi-amqp10/blob/master/amqp/v1_0/utils.py18:25
vkmcflaper87, oh no, pyngus doesn't do that18:26
* flaper87 takes the bat he has under his table and pings pyngus dev18:26
vkmcheh :)18:28
vkmcwell, I'll fix the connection... I wasn't sure about creating a consumer/producer each time for efficiency reasons... but there is no other way to do it18:28
peoplemergeflaper87: lmk if or when AngularJS might make it to this dash, my wife uses it at her job and I'd be down to learn it from her.18:29
peoplemergeshe might be convinced to contribute. possibly18:29
flaper87vkmc: just the consumer. You can keep the producer alive18:29
flaper87I'm sure we'll find a better way to do it18:29
flaper87but for now, that seems to be the easiest18:29
vkmcpeoplemerge, IIRC the Horizon plugin has been delayed until Marconi's graduation18:29
flaper87peoplemerge: AWESOME!!!! Do you have kids ? We can get them to code too :P18:30
* flaper87 takes advantage of peoplemerge's family18:30
peoplemergeflaper87: not yet!  but soon :)18:30
flaper87peoplemerge: then congrats ;)18:30
peoplemergewe're having too much fun now18:30
vkmcpeoplemerge, here is the bp in Horizon's side https://blueprints.launchpad.net/horizon/+spec/marconi-queues-panel18:31
vkmcflaper87, great18:31
vkmcflaper87, now, regarding the queues and messages controller18:31
vkmcflaper87, the first approach we discussed with alcabrera (returning a 201/204 code) is not feasible because we would be changing the API behaviour, so we should decide another way to handle this18:33
vkmcqueues controller ^^18:34
flaper87vkmc: why do you need to change the status code?18:34
flaper87oh because there's no way to know if a queue exists18:34
flaper87mmhh18:34
peoplemergevkmc: thx, I also saw https://blueprints.launchpad.net/marconi/+spec/marconi-horizon-integration18:34
vkmcflaper87, apart from that, doing that goes against the interface defined in https://github.com/openstack/marconi/blob/master/marconi/queues/storage/base.py#L125-L21318:36
vkmcwe say we are returning a dictionary, and instead we return an error code18:36
vkmcs/error code/code18:37
vkmcpeoplemerge, np!18:37
flaper87this is exactly one of the reasons I wanted to get rid of queues18:39
flaper87besides their metadata, they are useless18:39
vkmclol18:39
maliniflaper87, peoplemerge : jay-atl was working on the horizon stuff18:39
flaper87malini: thanks!18:40
*** Obulpath_ has quit IRC18:40
peoplemergeflaper87: who ever heard of a queuing service that uses queues?18:40
flaper87peoplemerge: we wanted to rename it to Messaging Service ;)18:40
flaper87Fuck the system18:40
flaper87w000000000000000000000t18:40
vkmcno moar queues! D:<18:41
peoplemergespeaking of renaming, are we still looking for a name change away from marconi for copywrite or trademark reasons?18:41
maliniwe have it in our meeting agenda tomorrow18:41
vkmcpeoplemerge, yeah... Zaqar and Naav are the most viable options18:41
kgriffsfor the record, I like the idea of topics and if you want to associate metadata with a topic you can, but the topic isn't a real resource. metadata would just get "tagged" with a topic or something.18:42
kgriffsbut yeah, we have that pesky 1.0 version of the API to deal with...18:42
vkmcflaper87, last thing about AMQP... I'm a bit confused of how are we going to use the broker here... besides using it for testing18:44
flaper87kgriffs: YAY!18:44
vkmcflaper87, Proton abstracts you from it18:44
flaper87call me stubborn but I still kinda feel we should give up queues and become a queues-less service18:44
flaper87I'd like to keep metadata around, though18:44
flaper87vkmc: you need to pass the broker address to proton, it'll do the rest. The broker will act as if it were another peer18:45
vkmcflaper87, oh yes, I assumed that :o18:46
vkmcflaper87, I thought we were depending on the broker for something else18:46
peoplemergevkmc: malini: my wife has some creative background doing this kind of stuff, will try to attend, but 8am PST is my drive in.  Good to hear you have some options on the table18:47
*** Obulpathi has joined #openstack-marconi18:47
flaper87vkmc: nope, just that18:47
vkmcflaper87, great18:47
flaper87vkmc: you're doing a great job, I really mean it. Keep it up!18:48
*** Obulpath_ has joined #openstack-marconi18:48
vkmcflaper87, message controller details are pending til the meeting with kgiusti and gsim, or is there something I could tackle on that area?18:48
flaper87vkmc: lets wait until after the meeting. We could work on a list of pending questions18:49
flaper87that'd be really helpful18:49
*** wirehead_ has quit IRC18:49
*** wirehead_ has joined #openstack-marconi18:50
flaper87for those interested in MQTT, this is a really good read: http://vasters.com/clemensv/2014/06/02/MQTT+An+Implementers+Perspective.aspx18:50
vkmcflaper87, thanks F, I hope we can see it working soon18:50
*** Obulpathi has quit IRC18:51
kgriffsand now for something completely different18:51
vkmcI'll wrote down some questions for Thursday18:52
kgriffsimagine if you could write an API reference using a little bit of markup.18:52
vkmcbrb18:52
kgriffsthen you could run it through a tool and it would split out a nice API server skeleton that implements your document?18:52
kgriffsthis is not my idea, to be fair... a colleague of mine suggested that our industry is thinking about the relationship of APIs and documentation backwards18:56
kgriffsUX people would say you should design your interface (in our case, the API) before the implementation so you don't artificially constrict what is best for the user by your implementation bias18:57
kgriffsnow I'm thinking, perhaps we should be going one step further and writing the docs before anything else.18:57
kgriffs</food-for-thought>18:58
* kgriffs watches a tumbleweed roll by18:59
peoplemergeflaper87: regarding adding msgpack back as an option (the ticket I have assigned), I see the patchsets where it got removed in favor of json.  I'm having a hard time comparing that with what's in our current source tree.  What was sqlite is now done in sqlalchemy so we have a nice orm abstraction.19:00
peoplemergedon't really get why that was in the storage driver tho19:01
peoplemergeshould have been in transport, no?19:01
kgriffsI think it was serializing the message body so it could be stored in a single field in the table19:02
kgriffspeoplemerge: in other words, nothing to do with the transport iirc19:02
peoplemergekgriffs: AFAICT we're not storing pure json in our persistence layer, we parse it, right?19:03
*** Obulpath_ has quit IRC19:04
peoplemergewhat I'm asking is: which way should we do it in the msgpack case?19:04
peoplemergekgriffs: regarding doc or ui, I've always thought you start with the domain model.  DDD says 'let the bones show' to users19:05
peoplemerge(but that's coming from a marconoob)19:06
peoplemergeIn msgpack, is it safe to presume we want to have msgpack-over-REST?19:09
kgriffsre msgpack, the message comes in, we parse it out to a Python dict type, then we pass that to the storage driver and it decides how to persist the dict19:10
peoplemergekgriffs: thx19:10
kgriffsSeems like that would be the same regardless of msgpack or JSON19:10
kgriffspeoplemerge: fwiw, the "correct" media type seems to be application/x-msgpack19:11
peoplemergekgriffs: seems like a sane default19:11
kgriffspeoplemerge: also, one more random thing I discovered recently19:12
kgriffsbe sure to pass encoding='utf-8' when decoding with the python msgpack library19:12
peoplemergekgriffs: will do19:13
kgriffsotherwise it will screw up unicode strings19:13
kgriffsand in the client, when encoding, pass this flag19:13
kgriffsuse_bin_type=True19:13
peoplemergek19:13
peoplemergeshould we support msgpack in 1.0 client as well as 1.1?19:13
kgriffspeoplemerge: we can "unofficially" support it in 1.0 since it doesn't break anything19:14
peoplemergek19:14
kgriffswe will only document it in 1.1 in any case19:14
kgriffsI think flaper87 had some thoughts on that19:14
*** seiflotfy has quit IRC19:15
peoplemergewill see what he says.19:16
kgriffspeoplemerge: TBH, I would start with 1.1 and get that 100% done. then we can backport if needed19:16
peoplemergexlnt19:16
*** Obulpathi has joined #openstack-marconi19:16
peoplemergeThinking it makes sense to start with a RESTful test, though I'm looking for something like that and not finding much.19:19
peoplemergeKnow of a test that mimics a rest client, sends json without the python client lib?19:20
*** seiflotfy has joined #openstack-marconi19:20
peoplemergeif that exists, imo the natural place to start would be to write a test that sends msgpack-encoded19:21
peoplemergefeedback appreciated tho19:22
kgriffspeoplemerge: are you looking for a unit or functional test, or some general REST client library to use?19:24
peoplemergean existing marconi functional test is what I had in mind, though I guess a unit test would be equally good too.19:27
kgriffslast time I checked, the tests were not using python-marconiclient19:27
peoplemergewant to avoid reinventing wheels if there are one already19:28
kgriffsthey were simulating WSGI requests19:28
kgriffsnot sure if that has changed19:28
kgriffsmalini, Obulpathi, sriram: do you know? ^^^19:28
peoplemergeahh, found something19:29
kgriffsunder /marconi/tests/queues/transport/wsgi19:29
kgriffstest_messages.py19:29
peoplemergemarconi/marconi/tests/queues/transport/wsgi/*19:29
kgriffsyep19:29
peoplemergeyesh19:29
kgriffsthere you go19:29
kgriffsthe directory structure is a little convoluted19:30
kgriffsused to be simpler but we moved some things around when we were thinking about having a "notifications" service separate from a "queues" service19:30
kgriffsbut I digress19:30
*** malini has quit IRC19:32
kgriffs<random>http://xiki.org/screencasts/</random>19:35
peoplemergekgriffs: wow19:39
flaper87holy crap19:47
flaper87where did that come from?19:47
*** balajiiyer has quit IRC19:54
tjanczuk_kgriffs, others: looking at the agenda for tomorrow: https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda. What is the "unified API model"?19:58
*** boris-42 has quit IRC20:08
*** balajiiyer has joined #openstack-marconi20:10
*** balajiiyer has quit IRC20:10
*** balajiiyer has joined #openstack-marconi20:10
*** boris-42 has joined #openstack-marconi20:11
openstackgerritSriram Madapusi Vasudevan proposed a change to openstack/marconi: feat(benchmarking) : Producer-Consumer scenario  https://review.openstack.org/9887520:15
tjanczuk_kgriffs: never mind, just saw the mail in ML.20:16
openstackgerritSriram Madapusi Vasudevan proposed a change to openstack/marconi: feat(benchmarking) : Producer-Consumer scenario  https://review.openstack.org/9887520:20
kgriffsflaper87: any objection to releasing client 0.0.2 ?20:23
kgriffsabettadapur tested all operations and gave the thumbs-up20:23
*** Obulpathi has quit IRC20:29
*** kgriffs is now known as kgriffs|afk20:29
*** Obulpathi has joined #openstack-marconi20:30
*** alcabrera is now known as alcabrera|afk20:31
*** kgriffs|afk is now known as kgriffs20:33
*** sriram has quit IRC20:33
*** mwagner_lap has quit IRC20:34
*** fifieldt_ has quit IRC20:54
peoplemergejust played around with xiki.  It's not exactly easy to get started with it, vim support doesn't seem to work.  Unless you use emacs?21:03
peoplemergeemacs, dude...21:03
*** fifieldt_ has joined #openstack-marconi21:07
*** whenry has quit IRC21:07
openstackgerritKurt Griffiths proposed a change to openstack/python-marconiclient: chore: Bump version to 0.0.2 in preparation for bugfix release  https://review.openstack.org/9890121:09
*** whenry has joined #openstack-marconi21:14
vkmcflaper87, I'm working on https://bugs.launchpad.net/marconi/+bug/1257391, what did you mean in this review? https://review.openstack.org/#/c/59539/2/tests/unit/queues/storage/test_impl_mongodb.py21:16
vkmc(if you remember)21:16
*** jdprax has quit IRC21:18
*** flwang has quit IRC21:18
flaper87vkmc: IIRC, it was related to the difficulti of calling the right method of the super class when using mixins21:18
flaper87kgriffs: lets release it21:19
kgriffsok, I submitted a patch to bump the version21:20
vkmcflaper87, ty21:23
kgriffsbe back in a minute - gotta play with my ZNC box21:24
*** kgriffs is now known as kgriffs|afk21:28
*** Obulpathi has quit IRC21:33
*** balajiiyer has quit IRC21:36
*** flwang has joined #openstack-marconi21:37
*** balajiiyer has joined #openstack-marconi21:38
*** balajiiyer has left #openstack-marconi21:42
*** kgriffs|afk has quit IRC21:53
*** malini_afk has quit IRC21:53
*** megan_w|afk has quit IRC21:53
*** rossk has quit IRC21:57
*** amitgandhi has quit IRC22:00
*** malini_afk has joined #openstack-marconi22:11
*** malini_afk is now known as malini22:11
*** megan_w|afk has joined #openstack-marconi22:12
*** megan_w|afk is now known as megan_w22:12
*** kgriffs has joined #openstack-marconi22:12
*** ChanServ sets mode: +o kgriffs22:12
*** flaper87 is now known as flaper87|afk22:23
*** mwagner_lap has joined #openstack-marconi22:24
openstackgerritVictoria Martínez de la Cruz proposed a change to openstack/marconi: Refactor auxiliary methods for MongoDB unit tests  https://review.openstack.org/9891622:41
openstackgerritA change was merged to openstack/marconi: Small fixes in storage drivers for Python 3  https://review.openstack.org/9821423:01
*** kgriffs is now known as kgriffs|afk23:13
*** sriram has joined #openstack-marconi23:16
*** whenry has quit IRC23:28
*** sriram has quit IRC23:34
*** sriram has joined #openstack-marconi23:34
openstackgerritSriram Madapusi Vasudevan proposed a change to openstack/marconi: feat(benchmarking) : Producer-Consumer scenario  https://review.openstack.org/9887523:35
*** sriram has quit IRC23:36

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!