Monday, 2013-12-02

*** oz_akan_ has joined #openstack-marconi00:11
*** oz_akan_ has quit IRC00:16
*** rongze has joined #openstack-marconi00:18
*** rongze has quit IRC00:23
*** amitgandhi has joined #openstack-marconi00:36
*** amitgandhi has quit IRC00:40
*** amitgandhi has joined #openstack-marconi00:46
*** amitgandhi has quit IRC00:51
*** oz_akan_ has joined #openstack-marconi01:12
*** oz_akan_ has quit IRC01:17
*** nosnos has joined #openstack-marconi01:33
*** amitgandhi has joined #openstack-marconi01:36
*** amitgandhi has quit IRC01:41
*** rongze has joined #openstack-marconi02:01
*** oz_akan_ has joined #openstack-marconi02:04
*** oz_akan_ has quit IRC02:08
*** adrian_otto has joined #openstack-marconi02:14
*** flwang has quit IRC02:25
*** amitgandhi has joined #openstack-marconi02:33
*** amitgandhi has quit IRC03:04
*** amitgandhi has joined #openstack-marconi03:04
*** amitgandhi has quit IRC03:08
*** flwang has joined #openstack-marconi03:12
*** ay has quit IRC04:02
*** oz_akan_ has joined #openstack-marconi04:05
*** oz_akan_ has quit IRC04:10
*** amitgandhi has joined #openstack-marconi04:15
*** adrian_otto has quit IRC04:16
*** amitgandhi has quit IRC04:19
*** ayoung has joined #openstack-marconi04:43
*** rongze has quit IRC04:57
*** adrian_otto has joined #openstack-marconi04:59
*** amitgandhi has joined #openstack-marconi05:05
*** oz_akan_ has joined #openstack-marconi05:06
*** amitgandhi has quit IRC05:10
*** oz_akan_ has quit IRC05:10
*** rongze has joined #openstack-marconi05:27
*** nosnos_ has joined #openstack-marconi05:35
*** rongze has quit IRC05:36
*** nosnos has quit IRC05:37
*** nosnos_ has quit IRC05:37
*** nosnos has joined #openstack-marconi05:38
*** adrian_otto has quit IRC06:00
*** rongze has joined #openstack-marconi06:02
*** amitgandhi has joined #openstack-marconi06:06
*** oz_akan_ has joined #openstack-marconi06:07
*** amitgandhi has quit IRC06:10
*** rongze has quit IRC06:11
*** rongze has joined #openstack-marconi06:11
*** oz_akan_ has quit IRC06:12
*** amitgandhi has joined #openstack-marconi06:16
*** amitgandhi has quit IRC06:20
*** nosnos_ has joined #openstack-marconi06:57
*** nosnos has quit IRC06:57
*** amitgandhi has joined #openstack-marconi07:06
*** oz_akan_ has joined #openstack-marconi07:08
*** amitgandhi has quit IRC07:10
*** oz_akan_ has quit IRC07:12
*** amitgandhi has joined #openstack-marconi08:07
*** oz_akan_ has joined #openstack-marconi08:09
*** amitgandhi has quit IRC08:11
*** oz_akan_ has quit IRC08:13
*** flaper87|afk is now known as flaper8708:15
*** amitgandhi has joined #openstack-marconi08:17
*** amitgandhi has quit IRC08:21
*** nosnos_ has quit IRC08:27
*** nosnos has joined #openstack-marconi08:28
*** rongze has quit IRC08:44
*** yassine has joined #openstack-marconi09:00
*** amitgandhi has joined #openstack-marconi09:07
*** ykaplan has joined #openstack-marconi09:09
*** oz_akan_ has joined #openstack-marconi09:09
*** amitgandhi has quit IRC09:12
*** oz_akan_ has quit IRC09:13
*** flwang has quit IRC09:41
*** amitgandhi has joined #openstack-marconi10:08
*** oz_akan_ has joined #openstack-marconi10:10
*** oz_akan_ has quit IRC10:14
*** amitgandhi has quit IRC10:15
*** amitgandhi has joined #openstack-marconi10:18
*** amitgandhi has quit IRC10:23
*** rongze has joined #openstack-marconi10:55
*** ykaplan has quit IRC11:10
*** oz_akan_ has joined #openstack-marconi11:11
*** amitgandhi has joined #openstack-marconi11:11
*** oz_akan_ has quit IRC11:16
*** amitgandhi has quit IRC11:16
*** oz_akan_ has joined #openstack-marconi11:18
*** oz_akan_ has quit IRC11:23
*** flwang has joined #openstack-marconi11:28
*** ykaplan has joined #openstack-marconi11:34
*** yassine has quit IRC11:55
*** amitgandhi has joined #openstack-marconi12:21
*** nosnos has quit IRC12:22
*** rongze has quit IRC12:23
*** amitgandhi has quit IRC12:25
*** fifieldt has quit IRC12:47
*** oz_akan_ has joined #openstack-marconi12:52
*** flwang has quit IRC12:57
*** vkmc has joined #openstack-marconi13:09
*** amitgandhi has joined #openstack-marconi13:12
*** amitgandhi has quit IRC13:16
*** oz_akan_ has quit IRC13:23
*** oz_akan_ has joined #openstack-marconi13:23
*** rongze has joined #openstack-marconi13:25
*** mpanetta has joined #openstack-marconi13:43
*** flwang has joined #openstack-marconi13:43
*** jcru has joined #openstack-marconi14:04
*** jergerber has joined #openstack-marconi14:29
*** tedross has joined #openstack-marconi14:33
*** ametts has joined #openstack-marconi14:41
*** amitgandhi has joined #openstack-marconi14:42
*** amitgandhi has quit IRC14:46
*** amitgandhi has joined #openstack-marconi14:47
kgriffs0/14:48
kgriffsI'm baaaaaack.14:48
* kgriffs queues scary music14:48
mpanettaYo!14:49
kgriffsmpanetta: have a good turkey day?14:49
mpanettaI did, how about yourself?14:49
kgriffsdefinitely. Pie was served and there was much rejoicing!14:50
mpanettaMmmm pie, apple?  Pumpkin?14:50
kgriffspumpkin, berry, and banana cream14:50
flaper87helloooooooooooooooooooooooooooooooooooooooo14:50
mpanettaOh man, sounds yummy!14:50
mpanettaflaper87: Mornin!14:51
kgriffsflaper87: hey slacker. where have you been this past week?14:51
* kgriffs runs away14:51
mpanettakgriffs: I need to get with you about sharding in a bit.14:51
kgriffsmpanetta: kk14:51
mpanettaSo no running away :P14:51
kgriffswe have a mtg in 9 minutes14:52
kgriffsafter that we can dance14:52
mpanettaSame here actually, different meeting tho.14:52
flaper87kgriffs: eating your turkey, obviously14:52
flaper87:D14:52
kgriffslol14:52
*** adrian_otto has joined #openstack-marconi14:54
*** adrian_otto has quit IRC14:55
*** adrian_otto has joined #openstack-marconi14:56
flwangmorning, guys14:56
kgriffso/14:56
kgriffsflwang: how do you say good evening in your native language?14:56
flwangwaiting for the meeting O:-)14:57
flwang晚上好14:57
kgriffs(and/or good morning)14:57
kgriffswow14:57
*** jcru_ has joined #openstack-marconi14:57
flwangwan shang hao14:57
* kgriffs copies to notepad14:57
flwanghehe, thanks for the sweet :)14:57
flwanggood morning =  早上好14:57
flwang好 means good14:57
flwang晚上 = evening14:58
flwang早上 = morning14:58
kgriffsflwang: I'm jealous; your script looks so much cooler than my lame latin characters14:58
kgriffsarabic looks cool too14:59
flaper87kgriffs: agreed!14:59
kgriffs(farsi or whatever it is called)14:59
flaper87and way cooler than my morning-hand-writing14:59
flwangI can teach you guys mandarin, one sentence per day :D14:59
kgriffsw00t14:59
flaper87actually, my hand-writing in general14:59
flwanghello = 你好14:59
* kgriffs copies that15:00
flwanghello = 你好 the pronunciation is ‘ni hao'15:00
kgriffsexcellent.15:00
*** jcru has quit IRC15:00
kgriffsmeeting time!15:00
flwangmeeting time15:01
flaper87MEEEEEEEEEEEEEEEEEEEEEEETING15:01
kgriffssee you in #openstack-meeting-alt15:01
kgriffsamitgandhi: mtg?15:02
amitgandhion mmy way15:04
*** alcabrera has joined #openstack-marconi15:10
*** rwsu has joined #openstack-marconi15:14
kgriffsflwang: meet oz_akan_15:21
kgriffsoz_akan_: meet flwang15:21
kgriffsflwang: will be working on heat templates for Marconi15:21
kgriffsI told him you may be able to share some lessons learned15:21
flwangkgriffs: got it, Sir15:21
kgriffsmpanetta: you too. ^^^15:21
*** adrian_otto has quit IRC15:27
kgriffsflwang: re that bug, don't forget to add some unit tests to prove it is fixed - we wouldn't have had this problem if we'd had tests in the first place!15:45
flwangkgriffs: sure, i'm professional stacker :)15:46
*** ykaplan has quit IRC15:57
*** briancline has quit IRC15:58
kgriffsflwang: w00t!16:02
kgriffsthanks16:02
flwangkgriffs: sure16:02
flwanganybody can take a look at this https://review.openstack.org/#/c/58092/ and let the baby fly?16:03
flwangit's my first baby in Marconi :)16:03
alcabrerakgriffs: all set as far as the meeting minutes go.16:04
kgriffsflwang: yeah, I've been meaning to16:04
kgriffsalcabrera: thanks again!16:04
kgriffsflwang: I should be able to get to it today16:05
flwangkgriffs: thanks God :)16:05
flwangi'm looking the new bug you just assigned to me16:06
alcabrerabrb16:07
mpanettahi flwang :)16:08
flwangmpanetta: hi16:08
mpanettakgriffs: I'm back16:08
alcabreraback16:09
mpanettamornin alcabrera16:09
alcabrerampanetta: yo! :D16:09
mpanettaHow goes?16:09
alcabreraalright. I feel great after that 10 day break. I read so many great books while on vacation. :)16:10
alcabrerampanetta: you?16:10
mpanettaDoing pretty well :)16:10
*** oz_akan_ has quit IRC16:10
*** oz_akan_ has joined #openstack-marconi16:10
mpanettaHad far too much to eat heh16:12
alcabreraheh :)16:15
*** oz_akan_ has quit IRC16:15
*** ilgiz has quit IRC16:22
*** ilgiz has joined #openstack-marconi16:23
flwangalcabrera: ping16:24
alcabreraflwang: pong16:26
flwangalcabrera: against my patch https://review.openstack.org/#/c/58092/2/marconi/queues/transport/wsgi/messages.py16:26
flwangdo you think it's really deserved an unit test?16:26
alcabrerayeah, considering it's a subtle case, and the test would help document the expected behavior and weed out the bug in the future.16:27
flwangalcabrera: okay, make sense. at my initial thought, I don't think it's worth to do that. anyway, i will fix it in next PS. thanks for the reviewing,16:28
alcabreraflwang: if you don't mind explaining, why do you think a test is necessary? :)16:29
alcabreraermmm16:29
alcabrera** a test is not necessary16:29
alcabrera?16:29
flwangI just think it's not related to a logic branch, not a specific function, not a user scenario16:30
flwangjust a tiny and clear fix16:30
alcabreraI see what you're saying. I'm concerned that is a user-scenario, in the case where the user is using a CLI or a module to automate message retrieval, and then fails because of '/messages/messages...'16:32
flwangyep, it makes more sense16:34
*** cpallares has joined #openstack-marconi16:37
flwangalcabrera: thanks for the clarification16:37
alcabreraflwang: no problem! :)16:37
alcabreracpallares: o/16:38
cpallaresalcabrera: hi!16:43
openstackgerritA change was merged to openstack/marconi: feat(shard): queue listing from multiple sources  https://review.openstack.org/5312716:44
alcabreracpallares: how goes? I hope your thanksgiving break was great. :D16:45
cpallaresalcabrera: it was pretty good thank you :) how was yours?16:46
*** haomaiwang has quit IRC16:46
alcabreravery relaxing! I found a lot of time to read several books I'd queued up. :)16:47
*** haomaiwang has joined #openstack-marconi16:47
cpallaresalcabrera: Very nice. What did you read? :)16:47
alcabreraFour books; Good Calories, Bad Calories (Gary Taubes), Grain Brain (David Perlmutter), 40 Alternatives to College (James Altucher), and Choose Yourself (James Altucher).16:49
alcabrerathere's some pretty surprising insights in the first two, and the latter two were very motivating. I recommend the lot of them. :D16:50
flaper87cpallares: hellooooooo16:51
flaper87cpallares: how are you doing?16:51
cpallaresflaper87: hello!16:52
*** briancline has joined #openstack-marconi16:52
cpallaresflaper87: I'm doing good! How are you?!16:52
flaper87cpallares: I'm doing well :D16:52
flaper87cpallares: reviewed your patch last week16:52
flaper87cpallares: you missed marconi's meeting :(16:52
cpallaresflaper87: woooot!16:52
cpallaresflaper87: what I thought it was in 10 minutes?16:53
alcabrerawe moved it earlier. I almost missed it. >.>16:53
flaper87cpallares: erm we changed the time last week16:53
flaper87:(16:53
flaper87it's 15:00 UTC now16:53
alcabreraIt was almost two hours ago.16:53
cpallaresalcabrera, flaper87: ah shoot.16:54
*** adrian_otto has joined #openstack-marconi16:54
alcabrerawe talked about JSON homedoc, marconi documentation, and...16:54
alcabreraaction items16:54
alcabrerathat was the gist of it.16:54
alcabrerawe're having another meeting tomorrow to talk about notifications support in marconi16:55
alcabrerasame time16:55
*** rongze has quit IRC16:56
alcabreracpallares: ^16:56
cpallaresalcabrera, good thing to know I won't miss that one16:57
flaper87cpallares: soooo, API ?16:59
flaper87cpallares: how's that mapping going?16:59
cpallaresflaper87: I'm commiting it right now :)16:59
flaper87cpallares: w000000000000000000000000000000000t16:59
flaper87who wants to get some extra info about the websocket stuff?16:59
flwangflaper87: count me17:00
flwangI bet i need to know more than others17:00
*** adrian_otto has quit IRC17:01
alcabreraI'll pass on the websocket details, but I'm happy to listen and comment as you tell it. I must say...17:01
alcabreraI'm happy with the choice of third-party library.17:01
alcabreraws4py17:01
flaper87so, I was going to pick socket.io but after some research I found out that most of the browsers now support websocket and the need for long-polling (which everybody hates but everyone uses) is not that high17:01
flaper87also, socket.io has it's very own way to do tings and the python implementation is tight to gevent17:02
flaper87so, I dug into the protocols and libraries and found out ws4py17:02
flaper87which seems to be maintained and easy to maintain - worst case scenario17:02
*** torgomatic has joined #openstack-marconi17:02
flaper87also, some of the things currently implemented in ws4py could be ported to falcon17:02
flaper87at least the wsgi part17:03
flaper87that being said, it just proxies the wsgi public API for now17:03
flaper87but, the goal is to integrate it with the work cpallares is doing17:03
flaper87also, I took a look at zmq, nanomsg and raw tcp and thought a bit more about how those would've been implemented17:04
flaper87the result is that the implementation is pretty much the same we're using for websocket17:04
* alcabrera listens intently to this part17:04
alcabrerasweet17:04
flaper87and so will be the integration with the API layer17:04
kgriffsinteresting17:04
flaper87so, since websocket seems like more important right now, I bet on it17:05
kgriffswebsocket is basically just TCP plus a low-level framing protocol, right?17:05
flaper87also, websocket makes a lot of sense for the notification API17:05
flaper87kgriffs: yeah17:05
kgriffskewl17:05
kgriffsalso some semantics around push/pull17:05
kgriffsspeaking of which17:05
flaper87the cool thing is that ws4py has an implementation for gevent, tornado, wsgiref and cherrypy17:05
kgriffsthis will give us the push message stream we need for notifications17:05
flaper87so, it was a good fit for us17:05
flaper87kgriffs: exactly17:06
flaper87:)17:06
flaper87I put an example in the repo17:06
kgriffsflaper87: interesting. I have been hearing more about gevent on the street lately17:06
kgriffsI wonder if it is starting to gain some traction over eventlet17:06
flaper87kgriffs: and there's some work going on gevent+Py3K17:06
kgriffsoooooh17:06
alcabreragevent released 1.0 recently17:06
kgriffsthat would be the nail in eventlet's coffin17:06
kgriffsalcabrera: w00t. no more RC?17:07
alcabrerafantik has maintained a py3k-friendly branch of gevent 1.0 for ages.17:07
kgriffshave they updated their website docs yet?17:07
kgriffs(for 1.0)17:07
alcabrerakgriffs: no more RC17:07
flaper87I'm not going to say eventlet has been abandoned BUT it pretty much feels like it is17:07
alcabrerahttps://pypi.python.org/pypi/gevent17:07
alcabrerapip install gevent -> 1.017:07
flaper87the guy that maintains it is way to busy17:07
flaper87and AFAIK, there's no Py3K work going on for it17:07
alcabreraflaper87: I echo that sentiment about eventlet. :/17:07
kgriffsyeah, the issue thread where py3k was mentioned has been stalled for a while17:08
kgriffs(eventlet issue)17:08
flaper87and BTW, the 'external' transport support works like a charm17:08
flaper87I almost cried17:08
kgriffs:D17:08
alcabrera:)17:08
flaper87we need to improve a bit the whole public/admin implementation, but that will happen as part of the API layer implementation17:08
alcabreraflaper87: agreed17:09
flaper87it was sooo cool to just open marconi's conf and set marconi_ws as the enabled transport17:09
flaper87then marconi-server and BOOM!17:09
kgriffssexy17:09
flaper87there it was, running and ready to be used17:09
alcabrerahehe17:10
alcabrerathat's the way it should be17:10
alcabreraalways17:10
alcabreraforever17:10
kgriffsf****ing awesome!17:10
alcabrera(unless we make it even easier some how017:10
alcabrera)17:10
* kgriffs is practicing his startup buzzwords17:10
flaper87now, I think there's a way for us to deploy WS and the wsgi API in the same process17:10
flaper87but that's something we should: 1) Decided whether we want that, 2) Tackle once the API layer is done17:10
alcabreratransports = wsgi, marconi_ws, [queues:....:marconi_ws] port =, ... [queues:....:wsgi] port = ...  (??)17:11
flaper87alcabrera: you have marconi as a dependency in the requirements.txt of marconi-redis17:11
flaper87right?17:11
alcabreraflaper87: yup17:11
alcabreraI have it set as....17:12
alcabrera-e git+https://github.com/openstack/marconi#egg=marconi17:12
flaper87alcabrera: yeah, something like that. The question is more technical in this case. I mean, WS does something different with the socket and the wsgi request so, we may need to dig a bit more17:12
flaper87alcabrera: I've done that17:12
flaper87alcabrera: but that was using custom code17:12
flaper87alcabrera: so, can you pip install -r requirements.txt ?17:13
flaper87I've it as part of marconi-websocket but it fails to install marconi17:13
alcabreraflaper87: yeah, and I can also 'pip install .' (for marconi-redis)17:13
flaper87if I run it manually: $ pip install git+.... it works17:13
flaper87if I use the requirements file it just fails17:13
flaper87mmh, kk17:13
flaper87I'll double check that17:13
alcabrerahmm17:13
alcabreraweird that it fails17:14
alcabrerawe have the same requirements file style17:14
flaper87weird indeed17:14
alcabreraincluding the17:14
alcabrera-e git+https://github.com/openstack/marconi#egg=marconi17:14
flaper87yeah17:14
alcabreraeven the setup.py/cfg17:14
alcabreraah well17:14
alcabrerainvestigations into subtle pythonitites17:14
flaper87yeah17:15
cpallarescould someone be super awesome and review this https://review.openstack.org/#/c/57536/? I'm trying to commit something but git's complaining that I can't submit to a file to a package  that doesn't exist yet17:18
alcabreracpallares: I'll give it a look over17:19
cpallaresthank you alcabrera!17:19
alcabrera#link https://review.openstack.org/#/c/57536/ (relinked because that ? was 404-ing the URL. :P )17:19
flaper87based on that review:17:19
* flaper87 is super-awesome17:19
cpallaresalcabrera, oops it's this one https://review.openstack.org/#/c/57536/17:20
cpallareshahaha17:20
alcabreracpallares: approved17:22
alcabrera:D17:22
flaper87now alcabrera is more awesome than flwang17:22
flaper87erm, than flaper8717:22
flaper87:(17:22
alcabreralol17:22
flwang:(17:22
cpallaresalcabrera: yay, thanks :D17:22
alcabreraeveryone is awesome17:22
alcabrerahurray!17:22
cpallareslol17:22
*** rongze has joined #openstack-marconi17:28
*** reed has joined #openstack-marconi17:28
openstackgerritA change was merged to openstack/marconi: Added empty packages and a test for the API v1  https://review.openstack.org/5753617:28
flaper87cpallares: great work17:28
flaper87looking forward to the new patch17:29
flwangflaper87: how to run unit test against one case in Marconi?17:31
kgriffsflaper87: I can't find a bp for notifications17:32
alcabreraflwang: like this17:32
kgriffsam I missing something, or is it not there yet?17:32
kgriffs(i can create it, just making sure)17:32
alcabreratox -e py27 -- tests.unit.queues.transport.wsgi.test_file:TestClass.test_method17:32
flaper87flwang: tox -epy27 -- --tests tests.unit.queues....17:32
flaper87or what alcabrera said17:32
flaper87:D17:32
alcabrera:D17:32
flaper87kgriffs: mmh17:32
flwangoh, same as Glance17:32
flaper87weird17:32
* alcabrera high-fives flaper8717:32
flaper87flwang: yup17:32
* flaper87 can't wait to migrate to testr17:32
flaper87seriously17:32
* flaper87 high-fives back17:33
flwangflaper87: gotcha17:33
flaper87:D17:33
flaper87kgriffs: ah, we just have that etherpad from the design session17:33
alcabreraflaper87, amitgandhi, kgriffs: who wants to draft the notifications BP?17:35
alcabrerait's definitely a hot item, so it's surprising we don't have that written out yet.17:35
alcabreraIt's probably one of those that should come with an associated wiki entry page.17:35
alcabrerae.g.. notications/api/v117:35
kgriffsflaper87: kk, I'll create a bp17:36
alcabrerakgriffs: thanks!17:36
amitgandhii can contribute to the api schema (basically what i have in my hackday project + notification template semantics)17:37
kgriffsamitgandhi: cool17:39
kgriffsLet me register a bp and setup a wiki17:39
*** flwang has quit IRC17:49
*** alcabrera has quit IRC17:59
*** alcabrera has joined #openstack-marconi17:59
ilgizHello, I'm choosing theme for my master thesis and found openstack marconi interesting for me, but the problem is that I'm going to spend around 15-25 per week and project is pretty fast moving, so I'm not sure that it is possible to work on project with such effort. What do you think?18:05
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: fix: cache contract for memcached backend  https://review.openstack.org/5950718:08
openstackgerritCindy Pallares proposed a change to openstack/marconi: Added schema.py which maps out the HTTP request codes in jsonschema  https://review.openstack.org/5950818:11
alcabrerailgiz: hey!18:12
alcabreraI'm happy to hear you're considering marconi for your master's thesis - awesome!18:12
alcabrerailgiz: what do you hope to accomplish for your thesis, what is the thesis area, and how could marconi play a role? I'm happy to discuss.18:13
ilgizI could chose a random area, I just want to work on open source project (as for me it is the best way to working not just for diploma). As already mentioned I'm going to work on project occasionally and can not spend a lot of time on it, so I would like to chose obscure feature, with which everyone is ok if it's going to take a while to get it finish. But I'm not sure that it is possible with new projects like this18:17
*** openstackgerrit has quit IRC18:18
*** openstackgerrit has joined #openstack-marconi18:18
alcabrerailgiz: hmmm....18:19
alcabreraworking on an open-source project does have some very awesome possibilities, but it is very different that anything I did in academia (a year ago).18:20
alcabrerayou can certainly contribute something really cool working 15-25 hours a week18:20
alcabrerahowever, it's best if it's done in the open, and always talking with people on this channel18:21
alcabreraall the better if it isn't an obscure feature18:21
alcabrerafor example18:21
ilgizI'm from Russia, so it is possible to work on such projects for thesis here18:22
alcabreraone idea I've had in mind (and flaper87 has brought this one up, too)- what happens if you map the semantics of marconi to distributed storage, e.g., hooking up marconi to an openstack swift backend?18:22
alcabreracpallares: I'll check out the new patch set soon - thanks!18:23
alcabrerailgiz: cool! What kinds of projects do you find interesting? Also, what made you consider marconi? :)18:25
cpallaresthank you alcabrera :)18:25
kgriffsalcabrera, flaper87: crazy thought for you18:26
kgriffsflaper87: ping18:27
ilgizalcabrera: can you please explain a bit more?18:28
alcabrerakgriffs: listening. :)18:30
alcabrerailgiz: sure thing, but before I do, could you answer my previous two questions?18:30
ilgizI would like to work on distributed service,  I understand that is too wide area. I have talked with Tomaz Muraus (He was my mentor on GSOC 2012, I have worked on Apache Libcloud project) and he recommended to me to take a look on openstack projects18:33
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: opt: add hints to make delete/claim checks faster  https://review.openstack.org/5951118:34
alcabrerailgiz: awesome!18:34
alcabreraI'm aware of Tomaz Muraus and the libcloud project. That's a great combination, and I imagine that must've been a very fruitful GSOC.18:35
alcabrerailgiz: so regarding the idea I mentioned above, how much do you know about openstack swift and openstack marconi?18:37
* flaper87 back18:37
flaper87kgriffs: shoot18:37
flaper87ilgiz: there are a lot of thing syou can do on marconi that actually don't depend on the changes happening in the code base. For example, working on new transports or new backends for it18:38
ilgizoh, it was awesome experience, it was pleasure to work with him18:38
ilgizI know only some basics about openstack projects18:39
alcabreraas flaper87 said, a fun idea for working on marconi (likely a little simpler than my idea above) is to add support for a new storage backend or a new transport.18:40
alcabrerailgiz: so - openstack marconi is a queuing/messaging system. You create queues, post messages to them, and read them out later. It's pretty useful for distributed worker/master setups and even for publish/subscribe. :)18:41
*** rongze has quit IRC18:42
alcabreraopenstack swift is distributed storage - you create buckets and upload files/"objects" to those buckets, and swift takes care of storage placement and object lookup.18:42
ilgizthank you, got it18:43
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: cleanup: remove proxy cruft  https://review.openstack.org/5951318:43
flaper87swift would be a good experiment18:43
ilgizis idea implement queue on top of swift?18:44
alcabrerailgiz: that's correct18:45
ilgizlike queue is buckets and items are objects?18:46
alcabreraswift as the storage backend18:46
alcabrerayeah! :)18:46
flaper87alcabrera: https://review.openstack.org/#/c/59513/ <- I think there's more thing under marconi/common18:46
flaper87*I think*18:46
flaper87I saw it the other day18:46
alcabreraqueue == bucket, object == messages18:46
flaper87again, I THINK18:46
flaper87:D18:46
alcabreraflaper87: lol, I'll look again. :)18:46
ilgizhm, that is interesting, but I do not understand then it is useful18:47
alcabreraflaper87, kgriffs: one of the recent merges introduced a likely serious problem into marconi, particularly with mongodb.18:47
alcabreraload_storage_driver is giving me...18:47
flaper87alcabrera: mmh18:48
* alcabrera runs to utest suite again18:48
flaper87what patch?18:48
flaper87and why the (unit|functional)tests didn't get that18:48
alcabrerait's the test_shard_catalog18:49
alcabrerawhich runs only under a mongodb contextz18:49
alcabrera*context18:49
alcabrerailgiz: the usefulness is the possibility to support many, many queues and messages, with long TTLs, without worrying about overloading storage18:49
alcabreraflaper87: another issue I discovered is that test clean up seems to be failing. I've filled up my local storage 3 times in the past hour - 38GB gone in 5 test runs. :/18:50
flaper87mmh, could you try going back in the history and find which patch broke that?18:51
flaper87I bet it's all related to mongodb18:51
flaper87since it's not being tested in the gate18:51
alcabrerayup, yup18:51
flaper87gosh, I want the devstack gate18:51
alcabreraI'll check it out18:51
alcabreraflaper87: here's the load_storage_driver issue: paste.openstack.org/show/54305/18:52
flaper87alcabrera: ah, that sounds like the s/queues// patch18:52
flaper87that removes the config prefixes for queues and proxy18:53
flaper87devstack patch is almost there: https://review.openstack.org/#/c/47999/18:53
alcabreraflaper87: awesome - +1 for devstack18:53
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: cleanup: remove proxy cruft  https://review.openstack.org/5951318:55
alcabreraflaper87: there was one more ref to the proxy in marconi.common. ;)18:55
flaper87I KNEW IT!18:56
flaper87:D18:56
flaper87brb18:56
mpanettaHmm, I am going to have to watch for the conf name change...18:56
alcabrerampanetta: oh yes18:57
alcabrerait's changed quite a bit18:57
ilgizhm, looks like this task is require a good knowledge of marconi and not hard by itself, am I right?18:58
alcabrerailgiz: that's correct. With a decent knowledge of the concepts of swift, and a good understanding of how marconi works, it is very doable.19:00
ilgizgreat :)19:01
* kgriffs is back19:01
mpanettawb19:01
kgriffsflaper87, alcabrera: here is my crazy idea19:01
kgriffswhat if we just used webhooks to implement push semantics?19:02
ilgizI'm willing to learn about marconi more :)19:02
* alcabrera needs to learn about webhooks19:03
alcabrerathat word keeps coming up and I'm still not aware of how it's used as an implementation mechanism. :P19:03
kgriffsflaper87: for example, if i have a worker pool listing to a queue they could just as soon be little web apps that we push messages to19:03
kgriffshttps://webhooks.pbworks.com/w/page/13385128/RESTful%20WebHooks19:03
kgriffsalcabrera: ^^^19:03
ilgizIn my mind it is good task to get familiar with codebase19:03
alcabrerailgiz: feel free to ask questions here. We're all pretty helpful! also, a good starting place is the wiki: https://wiki.openstack.org/wiki/Marconi19:04
alcabrerakgriffs: thanks!19:04
ilgizthank you :)19:04
kgriffsilgiz: we would LOVE to have your help!19:04
kgriffsour team is pretty small and so we really appreciate your interest19:05
ilgizthis is great, thank you, I'm looking forward  to implement something useful for project19:07
mpanettakgriffs: So webhooks is just a RESTful API, not any sort of implementation?19:10
*** rongze has joined #openstack-marconi19:12
kgriffsmpanetta: it's just a concept19:13
kgriffsbasically, post data to this URL when stuff happens19:13
kgriffslike, github can post to a URL when code is pushed19:13
kgriffsso you can have jenkins listen and do stuff, for example19:14
mpanettaHmm ok thanks.19:14
*** rongze has quit IRC19:20
*** rongze has joined #openstack-marconi19:49
*** rongze has quit IRC19:53
*** rwsu has quit IRC19:55
*** rwsu has joined #openstack-marconi19:56
*** notmyname has quit IRC20:10
alcabrerakgriffs: ping20:11
*** notmyname has joined #openstack-marconi20:14
kgriffsalcabrera: pong20:24
alcabrerakgriffs: yo!20:24
alcabreraI was just pinging you about eom:bastion. ;)20:24
alcabrerathat's resolved.20:24
*** cpallares has quit IRC20:29
flaper87back20:46
flaper87kgriffs: re webhooks20:46
flaper87I like the idea, I'm not sure about *just* having webhooks but I think they'll play an important role in how the notifications will work in marconi20:48
*** rongze has joined #openstack-marconi20:49
flaper87I think we wrote some notes about this in the notification session20:49
*** cpallares has joined #openstack-marconi20:50
flaper87ah we just had 'make them secure'20:50
kgriffsflaper87: yeah, so I was just wanting to expand our thinking a bit20:51
kgriffswebhooks is a nice way to enable push without dealing with persistent connections20:51
flaper87kgriffs: totally, don't get me wrong, I just have a really bad memory and always need to go back and lookup things20:51
kgriffsi think a websocket transport is cool for latency reasons for clients, like a web app talking directly to marconi - but there we need to make it secure again20:52
flaper87yeah, I see websockets being useful for mobile integrations20:52
flaper87for example20:52
flaper87and easy browser integration20:52
flaper87but webhooks are way better for *real apps* integration20:53
kgriffskewl20:53
kgriffsI haven't thought about webhooks vs. websockets a lot so far, but I wanted to throw it out there for discussion20:53
flaper87IMHO, most of our efforts around the notification work will go on implementing a nice plugin API20:53
flaper87that will make it easy to 'listen' to queues and do something20:54
flaper87kgriffs: +1 that sounds like a great topic for tomorrows meeting. I'll make sure to put some thoughts on that!20:54
*** rongze has quit IRC20:54
kgriffsflaper87: thanks!20:54
flaper87kgriffs: great idea, d00d!20:55
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: fix: mongodb tests consume storage like crazy  https://review.openstack.org/5953920:56
* alcabrera catches up20:56
kgriffsflaper87: https://etherpad.openstack.org/p/marconi-notifications-brainstorm20:57
alcabreracool - I'm game for thinking more on how webhooks/websockets will play into our notifications plan.20:57
flaper87kgriffs: and TBH, I'd just call it *remote hooks*20:57
kgriffsalcabrera: ^^^ feel free to add ideas to that pad20:57
flaper87because they could be web but also rpc20:57
flaper87or some messaging technology20:58
flaper87I see them implemented as separate plugins though20:58
alcabreraI'm heading out. More brainstorming tomorrow! :D21:00
flaper87alcabrera: take care :)21:00
*** alcabrera has quit IRC21:01
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: fix: shard test failing  https://review.openstack.org/5954121:05
*** oz_akan_ has joined #openstack-marconi21:08
*** oz_akan_ has quit IRC21:09
*** oz_akan_ has joined #openstack-marconi21:10
flaper87kgriffs: there are 2 reviews here that need a small push to land: https://review.openstack.org/#/dashboard/694321:17
*** haomaiwang has quit IRC21:20
*** ilgiz has quit IRC21:20
*** ilgiz has joined #openstack-marconi21:21
kgriffsflaper87: thanks21:23
kgriffsI've been trying to catch up on email all day21:23
kgriffswill get to reviews shortly21:23
flaper87kgriffs: sure21:24
kgriffsflaper87: pong21:24
*** vkmc has quit IRC21:40
*** rongze has joined #openstack-marconi21:51
*** rongze has quit IRC21:57
*** jamieh has joined #openstack-marconi22:06
*** fifieldt has joined #openstack-marconi22:16
*** tedross has quit IRC22:21
*** rwsu has quit IRC22:27
*** mpanetta has quit IRC22:34
openstackgerritFlavio Percoco proposed a change to openstack/marconi: Make sqlite connection lazy  https://review.openstack.org/5956622:42
*** rwsu has joined #openstack-marconi22:43
openstackgerritA change was merged to openstack/marconi: feat(validation): verify project id length  https://review.openstack.org/5049622:46
openstackgerritA change was merged to openstack/marconi: feat(health): make health API storage-aware  https://review.openstack.org/5345722:50
* flaper87 is happy to see patches landing22:50
*** rongze has joined #openstack-marconi22:53
*** rongze has quit IRC22:58
*** flaper87 is now known as flaper87|afk23:00
openstackgerritA change was merged to openstack/python-marconiclient: Remove unused packages  https://review.openstack.org/5722323:09
*** jamieh has quit IRC23:12
*** jergerber has quit IRC23:18
*** amitgandhi has quit IRC23:21
*** jcru_ has quit IRC23:21
*** oz_akan_ has quit IRC23:32
*** oz_akan_ has joined #openstack-marconi23:32
openstackgerritKurt Griffiths proposed a change to openstack/marconi: chore: Pass cache into drivers  https://review.openstack.org/5444523:33
*** oz_akan_ has quit IRC23:33
*** oz_akan_ has joined #openstack-marconi23:34
*** oz_akan_ has quit IRC23:38
*** kgriffs is now known as kgriffs_afk23:43
*** whenry has quit IRC23:52
*** rongze has joined #openstack-marconi23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!