Wednesday, 2014-01-29

*** ayoung_ has joined #openstack-marconi00:11
*** amitgandhi has joined #openstack-marconi00:13
*** jmckind has quit IRC00:15
*** amitgandhi has quit IRC00:24
*** vkmc has quit IRC00:28
*** dev999 has joined #openstack-marconi00:29
*** oz_akan_ has joined #openstack-marconi00:30
*** dev999 has quit IRC00:30
*** malini is now known as malini_afk00:39
*** amitgandhi has joined #openstack-marconi01:20
*** amitgandhi has quit IRC01:21
*** amitgandhi has joined #openstack-marconi01:21
*** amitgandhi has quit IRC01:25
*** nosnos has joined #openstack-marconi01:37
*** rossk has quit IRC02:01
*** rossk has joined #openstack-marconi02:02
*** frankbutt has joined #openstack-marconi02:03
*** frankbutt has left #openstack-marconi02:03
*** mpanetta has quit IRC02:10
*** rossk has quit IRC02:11
*** reed has quit IRC02:11
*** amitgandhi has joined #openstack-marconi02:21
*** amitgandhi has quit IRC02:25
*** flwang has joined #openstack-marconi02:27
*** oz_akan_ has quit IRC02:32
*** chandankumar_ has joined #openstack-marconi02:36
*** chandankumar_ has quit IRC02:48
*** kgriffs_afk has quit IRC02:52
*** kgriffs_afk has joined #openstack-marconi02:52
*** kgriffs_afk is now known as kgriffs02:52
*** ChanServ sets mode: +o kgriffs02:52
*** chandankumar_ has joined #openstack-marconi03:10
*** chandankumar_ has quit IRC03:16
*** oz_akan_ has joined #openstack-marconi03:20
*** amitgandhi has joined #openstack-marconi03:22
*** oz_akan_ has quit IRC03:26
*** amitgandhi has quit IRC03:26
*** oz_akan_ has joined #openstack-marconi04:22
*** oz_akan_ has quit IRC04:23
*** oz_akan_ has joined #openstack-marconi04:23
*** balajiiyer1 has quit IRC05:09
*** amitgandhi has joined #openstack-marconi05:23
*** amitgandhi has quit IRC05:27
*** nosnos has quit IRC05:38
*** nosnos has joined #openstack-marconi05:41
*** amitgandhi has joined #openstack-marconi06:23
*** amitgandhi has quit IRC06:28
*** amitgandhi has joined #openstack-marconi07:24
*** amitgandhi has quit IRC07:28
*** rwsu has quit IRC07:46
*** flaper87|afk is now known as flaper8707:47
*** rwsu has joined #openstack-marconi07:50
*** oz_akan_ has quit IRC07:54
*** chandankumar_ has joined #openstack-marconi07:59
*** chandankumar_ has quit IRC08:03
*** chandankumar_ has joined #openstack-marconi08:04
*** ykaplan has joined #openstack-marconi08:12
*** chandankumar_ has quit IRC08:15
*** oz_akan_ has joined #openstack-marconi08:24
*** amitgandhi has joined #openstack-marconi08:25
*** oz_akan_ has quit IRC08:29
*** amitgandhi has quit IRC08:29
*** chandankumar_ has joined #openstack-marconi08:40
*** chandankumar_ has quit IRC08:46
*** ykaplan has quit IRC09:18
*** oz_akan_ has joined #openstack-marconi09:25
*** amitgandhi has joined #openstack-marconi09:26
*** oz_akan_ has quit IRC09:30
*** amitgandhi has quit IRC09:30
*** ykaplan has joined #openstack-marconi09:41
*** nosnos has quit IRC09:51
*** andreykurilin has left #openstack-marconi10:21
*** oz_akan_ has joined #openstack-marconi10:26
*** amitgandhi has joined #openstack-marconi10:26
*** oz_akan_ has quit IRC10:30
*** amitgandhi has quit IRC10:31
*** oz_akan_ has joined #openstack-marconi11:27
*** amitgandhi has joined #openstack-marconi11:27
*** oz_akan__ has joined #openstack-marconi11:28
*** oz_akan_ has quit IRC11:28
*** amitgandhi has quit IRC11:31
*** oz_akan__ has quit IRC11:33
*** ykaplan has quit IRC11:45
*** malini_afk is now known as malini12:17
*** haomaiwang has quit IRC12:18
*** haomaiwang has joined #openstack-marconi12:18
*** amitgandhi has joined #openstack-marconi12:28
*** oz_akan_ has joined #openstack-marconi12:29
*** amitgandhi has quit IRC12:32
*** oz_akan_ has quit IRC12:34
*** oz_akan_ has joined #openstack-marconi12:36
*** oz_akan_ has quit IRC12:38
*** oz_akan_ has joined #openstack-marconi12:39
*** ykaplan has joined #openstack-marconi12:51
*** tedross has joined #openstack-marconi12:51
*** chandankumar_ has joined #openstack-marconi13:01
*** chandankumar_ has quit IRC13:05
*** chandankumar_ has joined #openstack-marconi13:07
*** amitgandhi has joined #openstack-marconi13:28
*** chandankumar_ has quit IRC13:30
*** amitgandhi has quit IRC13:33
*** amitgandhi has joined #openstack-marconi13:39
*** chandankumar_ has joined #openstack-marconi13:51
*** chandankumar_ has quit IRC13:51
*** ayoung_ has quit IRC13:51
*** balajiiyer has joined #openstack-marconi13:54
*** balajiiyer has quit IRC13:58
*** ayoung_ has joined #openstack-marconi14:05
*** haomaiwang has quit IRC14:10
*** cpallares has joined #openstack-marconi14:13
*** haomaiwang has joined #openstack-marconi14:16
cpallaresGood morning o/14:23
*** balajiiyer1 has joined #openstack-marconi14:27
*** ayoung_ is now known as ayoung14:29
*** tedross has quit IRC14:33
*** mpanetta has joined #openstack-marconi14:35
*** mpanetta has quit IRC14:35
*** mpanetta has joined #openstack-marconi14:36
*** jmckind has joined #openstack-marconi14:36
*** alcabrera has joined #openstack-marconi14:37
alcabreragood morning! :)14:39
*** haomaiwang has quit IRC14:42
*** haomaiwang has joined #openstack-marconi14:43
*** oz_akan_ has quit IRC14:46
*** chandankumar_ has joined #openstack-marconi14:48
balajiiyer1alcabrera: morning14:49
*** tedross has joined #openstack-marconi14:49
alcabrerabalajiiyer1: :)14:50
*** jcru has joined #openstack-marconi14:50
*** oz_akan_ has joined #openstack-marconi14:51
*** chandankumar_ has quit IRC14:51
*** oz_akan_ has quit IRC14:52
*** chandankumar_ has joined #openstack-marconi14:53
*** zyuan has joined #openstack-marconi14:56
*** chandankumar_ has quit IRC14:57
*** chandankumar_ has joined #openstack-marconi14:58
*** chandankumar_ has quit IRC15:02
*** chandankumar_ has joined #openstack-marconi15:03
*** chandankumar_ has quit IRC15:07
*** flaper87 is now known as flaper87|afk15:22
*** oz_akan_ has joined #openstack-marconi15:23
*** oz_akan_ has quit IRC15:27
*** alcabrera is now known as alcabrera|afk15:29
*** chandankumar_ has joined #openstack-marconi15:34
*** flaper87|afk is now known as flaper8715:35
*** alcabrera|afk is now known as alcabrera15:35
*** chandankumar_ has quit IRC15:38
*** chandankumar_ has joined #openstack-marconi15:39
*** chandankumar_ has quit IRC15:45
*** chandankumar_ has joined #openstack-marconi15:46
*** chandankumar_ has quit IRC15:49
*** chandankumar_ has joined #openstack-marconi15:50
*** chandankumar_ has quit IRC15:52
*** ykaplan has quit IRC15:57
*** ykaplan_ has joined #openstack-marconi15:57
*** tedross has quit IRC16:03
*** ykaplan_ has quit IRC16:04
*** ykaplan has joined #openstack-marconi16:08
kgriffshi folks16:08
kgriffsalcabrera, flaper87: ttx mentioned yesterday that we should have all bp's assigned, and consider slipping or deprioritizing some of them since we have a short time frame16:09
kgriffsbalajiiyer1: ^^16:10
balajiiyer1kgriffs: I think we have all bp's assigned, right? Are there any unassigned bp's?16:13
kgriffsthere are a few16:14
kgriffsI am going to take a first pass getting things assigned where it is obvious16:14
alcabrerakgriffs: thanks16:15
kgriffsseems like a pretty big thing that I kinda doubt will get done in the next four weeks, but I could be wrong16:16
alcabrerakgriffs: you're right - it's too big for the next few weeks16:16
kgriffsok. I am going to move it out of icehouse; if we end up getting everything else done early, we can consider bringing it back16:17
kgriffsbalajiiyer1: looks like there was already a bp for this?
kgriffsalcabrera: ^^16:18
*** oz_akan_ has joined #openstack-marconi16:19
*** oz_akan__ has joined #openstack-marconi16:20
alcabrerakgriffs: I'd like that one brought up for general brainstorming. The first proposed solution is nice for deployers, but may be unfriendly to users depending on how it is presented.16:21
alcabreraso - I'd like to defer implementation for after icehouse16:21
kgriffsalcabrera: so, i think we have two things that need to be done16:22
kgriffsi mean, issues to address16:22
kgriffs1. noisy neighbor16:22
*** haomaiwa_ has joined #openstack-marconi16:22
kgriffs2. virtually unlimited queue length16:22
kgriffsWe've had some discussion in here and via email16:22
kgriffsI sent a summary to balajiiyer116:23
alcabreraby handling (2), we'd have the beginnings of enabling back pressure mechanisms for users16:23
kgriffsbalajiiyer1: did you already create any other bps?16:23
balajiiyer1kgriffs: I did not, now wondering should I?16:23
*** oz_akan_ has quit IRC16:24
kgriffswell, if we are able to migrate queues, will that address noisy neighbor?16:24
kgriffs(there's already a bp for that)16:24
balajiiyer1how easy it is to migrate queues without losing inflight messages?16:25
*** haomaiwang has quit IRC16:26
kgriffsbalajiiyer1: we have discussed some strategies for doing that16:26
kgriffsbasically you do something similar to what VMs do in a live migration16:26
balajiiyer1kgriffs: ok16:26
kgriffsthe idea is16:26
alcabrerasnapshot/freeze/catch-up/unfreeze kind of deal, iirc16:26
kgriffsyou get the old and new data stores synced as closely as possible16:27
kgriffsthen you briefly freeze, move over the remaining delta, and unfreeze16:27
kgriffsif we can do it fast enough, we can hold incoming message posts without them timing out16:27
kgriffsI think that this will be one of the flaship features of the Juno release16:28
kgriffsFrom my notes:16:30
kgriffsI think infinite storage can be addressed by using mongo's built-in sharding - you could hash on the marker, but this would require a marker proxy because we can no longer depend on the unique index.16:30
kgriffsIf you shard the queue across two different backend db nodes, no matter how you look at it, you will need to have a marker proxy.16:30
kgriffsAnyway, I could be wrong, but this is my current thinking. Flavio is in favor of not sharing a queue across multiple backends, period (FWIW).16:30
kgriffsmarker proxy is a thing that proxies message inserts16:31
balajiiyer1I see, ok. thanks for the detailed response16:31
kgriffsAn alternative to "infinite queue length" is to solve that in the filesystem layer using something like Ceph, Gluster, SolidFire, etc.16:32
kgriffsalso, let's not forget the interesting possibilities that docker gives us16:33
kgriffswe could essentially provision single-tenant micro-marconis very cheaply16:33
balajiiyer1'single tenant micro marconis' I like that16:33
kgriffsyep! we could say you have unlimited queue length, but please contact us to raise your quota16:34
balajiiyer1my only concern with moving queues is how often the queue will get moved. What if after moving the queue again experiences a noisy neighbor in the new shard?16:35
kgriffsbalajiiyer1: yes, that is possible. the placement algorithm would need to be pretty clever16:36
kgriffsfrom my notes:16:37
kgriffsNoisy neighbor can be addressed by being able to migrate/rebalance shards to progressively less-crowded shards. You can take this to an extreme by isolating someone in their own micro storage cluster using docker.16:37
kgriffsnoisy neighbor doesn't really apply to the web heads because we can just round-robin and autoscale the load16:37
*** tedross has joined #openstack-marconi16:38
balajiiyer1kgriffs: so the question, do you want me to create a new bp for this? or do you think existing bps will handle this use case?16:41
kgriffsI think the two that are there will work16:41
kgriffsI added some notes the whiteboards16:41
balajiiyer1ok cool16:42
kgriffsbalajiiyer1: I just assigned them to you16:42
kgriffsso you can take the lead on the design and organizing the tasks16:42
kgriffsthe marker-proxy thing would let us shard a single queue across multiple redis nodes, which may or may not be interesting16:43
kgriffsi think the redis flavor use cases may not overlap with the ones that require super long queues, though16:43
kgriffsI think people will want redis for super fast throughput, where things go in and out quickly, and that means you won't have very long queues16:44
kgriffsbalajiiyer1: final thought16:44
kgriffsif we can rebalance queues based on CPU and network usage16:45
kgriffswe may also consider rebalancing based on queue length16:45
kgriffsas queues get longer, we can move them to nodes that have more disk16:45
kgriffsI'd be willing to bet we could fit more than enough messages on a big shard for anyone to use16:46
kgriffsi mean, it isn't like we are storage megabytes per message16:46
balajiiyer1kgriffs: I have some questions about the documentation bp16:48
balajiiyer1I spoke to Catherine, she is ready to help us - she has asked me a few questions, which I will need your help with. here it goes -16:49
balajiiyer1"Is there a repository on github that you'd like for me to use for the docs source files?"16:49
kgriffsbalajiiyer1: p.s. - also consider that clients can only post so many messages/sec anyway16:50
kgriffsbalajiiyer1: ah, good question16:50
kgriffsi don't know whether other projects have a standalone repo or just use the same as the app code16:50
kgriffslooks like there is a precedent16:52
alcabrerathere;s also:
alcabrerait seems like developer facing docs live with a project, while things that are more applicable to openstack deploymeny lives in 'openstack-manuals', like:
kgriffsso, that begs the question16:54
kgriffshow do you write docs in order to graduate that are added to the official manuals before you graduate?16:54
kgriffsi mean, chicken-and-egg problem16:54
balajiiyer1ha,good question.16:55
kgriffsalcabrera: so, it looks like this is indeed the kind of docs that bp is talking about16:55
kgriffsmy guess is that we put them in our own repo now, and after graduation (during Juno, with luck) we move them over16:55
kgriffsso, I would try to make them compatible with
kgriffs(use maven and stuff)16:56
kgriffsflaper87: ^^^16:56
*** ykaplan has quit IRC16:57
kgriffsmalini: ping16:58
malinikgriffs: need to hop on a meeting now16:58
maliniwill ping u in 30 min16:59
kgriffsmalini: real quick, did you see chad's note about devstack?16:59
kgriffsthat's all - just wanted to make sure you were on top of that16:59
kgriffsbalajiiyer1: so, my current thinking wrt docs is this:17:00
kgriffsCreate a "doc" directory under
kgriffsand then mirror what the openstack-manuals project does, but only do the user and operator manuals17:01
kgriffslet's see17:01
kgriffsstart with those17:03
kgriffsbut yeah, we should run this by Anne and see what she thinks about the plan - see if we are on track, or way off on this17:03
kgriffsbalajiiyer1: ^^^17:03
kgriffsflaper87: ^^^17:03
balajiiyer1kgriffs: Ok. I will touch base with Catherine on what we spoke and keep Anne in loop17:04
balajiiyer1kgriffs:  so, install-guide, where should it come from? ops?17:04
balajiiyer1also, for user guide, do you think for initial draft, take RAX guide and remove RAX specfic components and branding?17:05
kgriffsinteresting --- looks like maven is like a Jenkins17:06
kgriffsbalajiiyer1: sounds good to me17:06
balajiiyer1what is the timeline for this to be complete? I dont think we can make it by Feb 617:07
kgriffsbalajiiyer1: The TC said that it would be OK to have docs after the i-3 milestone, but if we are putting this directly in our repo, then...17:08
kgriffswe would want to get it in before the freeze17:08
kgriffswhich is March 6th17:08
balajiiyer1ok sounds good, I will target end of Feb17:09
kgriffsthere is a voluntary "soft freeze" that just got added to the schedule17:09
kgriffsFeb 20th17:09
kgriffsto help avoid killing the gate17:09
balajiiyer1might be a little difficult to meet the date, but I will shoot for it.17:10
*** ykaplan has joined #openstack-marconi17:14
alcabreraykaplan: hey! I wanted to check with you - how goes the sqlalchemy backend for marconi? :)17:24
*** oz_akan__ has quit IRC17:32
*** balajiiyer1 is now known as balajiiyer1_afk17:40
*** ykaplan has quit IRC17:44
*** zyuan__ has joined #openstack-marconi17:46
*** zyuan has quit IRC17:50
kgriffsalcabrera: too slow. :(17:51
kgriffsalcabrera: I updated the i-3 bps17:51
kgriffseverything is now assigned and I moved a few out to Juno17:51
kgriffscan you take a look and see if we are missing anything critical?17:52
alcabrerasure thing, kgriffs18:04
alcabreralooks good to me - heat, tempest, docs, pecan evaluation, and storage driver are all there18:06
alcabreraI've updated the status on my part of the sqlalchemy driver. I started that today - sharding and catalogue.18:07
kgriffsgreat, thanks!18:07
kgriffs"ConnectionError: HTTPConnectionPool"18:08
kgriffsmalini: ^^^18:08
alcabrerawas that the same test we started skipping?18:11
malinino its not18:11
malinilooks like the marconi-server didnt start ?18:12
*** vkmc has joined #openstack-marconi18:13
malinibut tht shud have failed all the tests18:14
kgriffsthe error says "broken pipe"18:14
kgriffsI think that the server is crashing at some point18:14
kgriffsor getting killed by Jenkins or something (although, I don't know why it would)18:15
*** zyuan__ has quit IRC18:34
*** zyuan has joined #openstack-marconi18:34
kgriffsanybody see an error from python-subunit lately?18:35
kgriffs    from testtools.compat import all18:35
kgriffsImportError: cannot import name all18:35
alcabreraI saw something about this in the openstack ML this morning18:37
alcabrerathey linked it to testtools18:37
kgriffsif I didn't hate email so much I would have seen that. :p18:37
alcabreraI'm in a similar boat. I only ever glance over the subject heading for OS-dev ML posts, tbh, kgriffs.,18:39
alcabreraIf something seems *really* interesting, for large values of interesting, I Ctrl-F to that set of posts and review a little more. That's my approach, anyway. :P18:39
*** balajiiyer1_afk is now known as balajiiyer18:43
alcabrerakgriffs: here's the proposed solution to the testtools problem:
kgriffsthanks, just found it too18:48
openstackgerritKurt Griffiths proposed a change to openstack/marconi: fix(testtools): 0.9.35 is not compatible with subunit 0.0.17
alcabrera+2 kgriffs - let's get this fix in18:54
kgriffsalcabrera: can you ninjappove it once gate +1's?18:56
kgriffsflaper87: ^^^ or you can if you are around18:56
alcabrerakgriffs: sure thing18:56
*** kgriffs is now known as kgriffs_afk18:59
*** ametts has joined #openstack-marconi19:05
openstackgerritKurt Griffiths proposed a change to openstack/marconi: test(functional): Don't use a dead test server
*** kgriffs_afk is now known as kgriffs19:12
*** rossk has joined #openstack-marconi19:12
alcabrerakgriffs: blocked by pypi not available. :/19:33
*** jmckind has quit IRC19:49
openstackgerritA change was merged to openstack/marconi: fix(testtools): 0.9.35 is not compatible with subunit 0.0.17
*** kgriffs is now known as kgriffs_afk20:07
*** vkmc has quit IRC20:38
*** jmckind has joined #openstack-marconi20:42
*** alcabrera has quit IRC21:05
*** alcabrera has joined #openstack-marconi21:05
*** kgriffs_afk is now known as kgriffs21:06
*** alcabrera is now known as alcabrera|afk21:12
balajiiyerkgriffs:  for bps that should meet i3 date, is it the new soft freeze date of Feb 20?21:57
*** balajiiyer has quit IRC22:08
*** amit has joined #openstack-marconi22:11
*** amit is now known as Guest5291622:12
*** amitgandhi has quit IRC22:14
*** Guest52916 has quit IRC22:16
*** jmckind has quit IRC22:21
*** tedross has quit IRC22:24
*** zyuan has quit IRC22:31
*** jcru has quit IRC22:48
*** malini is now known as malini_afk23:13
*** alcabrera|afk has quit IRC23:15
*** amitgandhi has joined #openstack-marconi23:23
*** amitgandhi has quit IRC23:27

Generated by 2.14.0 by Marius Gedminas - find it at!