Friday, 2013-09-13

*** amitgandhi has joined #openstack-marconi00:49
*** malini_afk is now known as malini00:51
*** oz_akan_ has joined #openstack-marconi00:54
*** nosnos has joined #openstack-marconi01:06
*** fandikurnia01 has joined #openstack-marconi01:29
*** fandikurnia01 has quit IRC01:59
*** fandikurnia01 has joined #openstack-marconi02:04
*** oz_akan_ has quit IRC02:06
*** oz_akan_ has joined #openstack-marconi02:06
*** oz_akan_ has quit IRC02:09
*** fandikurnia01 has quit IRC02:25
*** oz_akan_ has joined #openstack-marconi02:37
*** amitgandhi has quit IRC03:23
*** malini is now known as malini_afk03:28
*** fandikurnia01 has joined #openstack-marconi03:58
*** fandikurnia01 has quit IRC04:15
*** fandikurnia01 has joined #openstack-marconi04:16
*** oz_akan_ has quit IRC04:18
*** oz_akan_ has joined #openstack-marconi04:18
*** torgomatic has quit IRC04:19
*** russellb has quit IRC04:19
*** fandikurnia01 has quit IRC04:21
*** russellb has joined #openstack-marconi04:22
*** torgomatic has joined #openstack-marconi04:23
*** oz_akan_ has quit IRC04:23
*** oz_akan_ has joined #openstack-marconi05:29
*** oz_akan_ has quit IRC05:34
*** flaper87|afk is now known as flaper8706:39
*** whenry has quit IRC06:41
*** key4 has quit IRC08:48
*** key4 has joined #openstack-marconi08:48
*** key4 has quit IRC10:11
*** key4 has joined #openstack-marconi10:12
*** nosnos has quit IRC10:26
*** nosnos has joined #openstack-marconi10:27
*** nosnos has quit IRC10:27
*** nosnos has joined #openstack-marconi10:28
*** nosnos has quit IRC10:35
*** nosnos has joined #openstack-marconi10:36
*** nosnos has quit IRC10:40
*** cthulhup has quit IRC10:47
*** tedross has joined #openstack-marconi11:39
*** vkmc has joined #openstack-marconi11:42
*** malini_afk is now known as malini11:48
*** malini is now known as malini_afk13:00
*** oz_akan_ has joined #openstack-marconi13:20
*** oz_akan_ has quit IRC13:23
*** oz_akan_ has joined #openstack-marconi13:24
*** amitgandhi has joined #openstack-marconi13:29
*** malini_afk is now known as malini13:58
*** whenry has joined #openstack-marconi14:05
*** acabrera has joined #openstack-marconi14:36
acabreraMorning!14:40
flaper87acabrera: GOOOD MORNING!14:40
* flaper87 is reviewing acabrera's proxy patch14:40
* acabrera cannot settle on a nick :P14:40
acabrerawoot, flaper87!14:40
acabreraIt's a hefty bunch of patches.14:40
flaper87acabrera: I'm reviewing this: https://review.openstack.org/#/c/44356/14:41
flaper87got there following dependency's chain14:41
*** JRow has joined #openstack-marconi14:42
flaper87acabrera: I meant to say, this: https://review.openstack.org/#/c/43909/14:42
flaper87:D14:42
acabreraahh, round 1 of the whole thing. :)14:42
flaper87acabrera: did you want me to review another one first?14:43
acabreraQuick note on that one - for the whole round-robin/select when creating a queue, I'm thinking of simplifying that portion and deferring the load balancing logic to the partition URL.14:43
acabreraHmm...14:43
flaper87I'm a bit confused because that one still uses redis14:44
flaper87and I thought you were working on multi-storage support14:44
acabreraYeah, the Redis logic is getting replaced by oslo.cache, and the multi-storage support - I worked on that yesterday.14:44
acabreraSo where to start...?14:45
flaper87yup, that's my question :P14:45
flaper87because I've the feeling I'm reviewing the wrong patch14:45
flaper87but the dependencies' chain led me there14:45
acabreraI'd say right where you are - it's the best place to get the big picture design of the thing - routes, resources.14:45
flaper87kk14:45
* flaper87 does that and STFU14:45
flaper87actually14:45
acabreraJust keep in mijnd that the redis details will get replaced soon.14:46
* flaper87 STFU and then does that14:46
acabrera:)14:46
flaper87kk, +1 for replacing that14:46
* flaper87 loves reviews that get simpler14:46
flaper87malini: helllooooooooooooo14:46
acabreraAsk me anything while you're reading - it's a hefty bunch of patches and if you're confused on any points, I'm happy to help guide you through how I got to the current design. :D14:46
flaper87malini: btw, I got an environment with marconi + keystone setup14:47
maliniflaper87: yayyy!!!14:47
flaper87I promisse to test against that from now on and don't screw your auth tests anymore14:47
flaper87:D14:47
malininow you can enjoy the weekend in peace!14:47
flaper87weekend ? what's that ?14:47
* flaper87 is now confused14:47
flaper87is there something called weekend ?14:48
maliniyou know, the thing that comes between friday & monday.14:48
flaper87why do people never tell me these things.14:48
flaper87?14:48
maliniyou shud have paid attention in kindergarten ;)14:48
flaper87aaahhh, I think I did, I just have really bad memory so, no idea what they taught me there14:49
malini:D14:49
malinime neither..I had a troubled KG life :D14:49
acabreralol14:51
acabreraweekends...14:51
flaper87btw, tomorrow marconi will be migrated under openstack/14:52
acabreraPython Cookbook 3e is on sale today: http://shop.oreilly.com/category/deals/programmers-day.do14:52
acabreraOohh, migration. :D14:53
flaper87you'll have to update your repo remote14:53
malininew home!!! <:o)14:53
openstackgerritZhihao Yuan proposed a change to stackforge/marconi: WPI: test(proxy): catalog tested  https://review.openstack.org/4636214:59
acabrerazyuan: I'll check that out after I catch up on morning news. :)15:00
*** kgriffs_afk is now known as kgriffs15:06
acabrerakgriffs: o/15:07
kgriffsyo yo yo15:08
kgriffsoooh, this is tempting: http://shop.oreilly.com/product/9781593272203.do?code=DEAL15:09
acabrerayeeess15:10
acabreraI have that one. It's totally the spiritual succeesor to Advanced Unix Programming.15:10
acabreraIt's so thorough it's scary.15:10
acabrera50+ glorious chapters on every nook and cranny of Linux system's programming.15:11
acabrera*60+15:11
kgriffshmm15:13
kgriffsno mention of ALSA15:13
kgriffs?15:13
flaper87kgriffs: gooood morning15:15
zyuanhehe15:16
acabrerakgriffs: nope - no mention of sound libraries at all. I suppose that's higher level than systems. :P15:16
acabreragit pissed: https://github.com/chrishunt/git-pissed15:16
kgriffsLOOOL15:17
acabreratests, queues, and storage come up a lot in marconi logs. :P15:18
kgriffsacabrera: this one is also tempting me - http://shop.oreilly.com/product/9781118612101.do?code=DEAL15:18
* kgriffs is such a geek15:18
acabreraThat one tempted me, too, kgriffs. :D15:19
acabreraI checked it out on amazon for "Peek Inside" and ultimately, I ended up comparing it to CLRS Algorithms: http://www.amazon.com/Introduction-Algorithms-Thomas-H-Cormen/dp/026203384415:20
acabreraI'd pick the latter over Essential Algorithms15:20
zyuanoh, that's the text book used by many schools15:21
zyuanintrodution algorithms15:21
acabrerazyuan: Yeah, I still haven't found anything better than that. :)15:21
acabreraI need to get a copy sometime. I don;'t have my 2e of it any more.15:22
zyuan(i still haven't read that)15:22
zyuan( i read some others...)15:22
zyuanabout CLRS, berkley has a series of videos15:23
zyuanwell... altually those lectures are widely available on internet during the recent years...15:25
acabreraI've made note of the algorithm courses on Coursera, but I hadn't noticed Berkeley's past offerings.15:27
acabreraLink?15:27
zyuancan't find... i saved them on my computer serveral years ago, and forgot where i got them...15:29
*** JRow has quit IRC15:29
zyuanChanServ: found. it's actually from MIT: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-046j-introduction-to-algorithms-sma-5503-fall-2005/video-lectures/15:33
zyuanacabrera: ^^15:35
acabreraahh, ocw content. classic.15:37
acabrerazyuan: Thanks!15:37
acabreraToo bad they start calling everything after lecture 19 "advanced topics" :P15:38
zyuanmy hope is to learn how to implement STL; i have no further pursue on algorithms...15:40
zyuanacabrera: check out this video http://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning the first 30mins are about algorithms15:47
zyuanmy favoriate part on goingnative15:47
flaper87btw guys, can I get your thoughts about the code structure I proposed the other day?15:50
zyuanwhere?15:50
flaper87If we agree it makes sense, I think that should be the first thing to do after marconi is migrated under openstack15:50
flaper87zyuan: some therpad, let me try to find it again15:50
flaper87https://etherpad.openstack.org/marconi-tree15:51
zyuanlooks deep15:52
flaper871 level deeper, yes15:52
flaper87I think it makes sense to separate the 3 main serivces, it's not that we HAVE to15:52
kgriffsI still think "marconi-queueing" is a little awkward15:52
kgriffscan anyone think of a better name?15:53
flaper87could be marconi-service15:53
kgriffsmarconi-api15:54
kgriffsother ideas?15:54
kgriffs(brainstorming)15:54
flaper87marconi-api sounds good to me15:54
* acabrera joins brainstorm15:54
flaper87but I would call the package queues (just to give it some context)15:55
kgriffsI'm adding them to the etherpad so we can see how the different options look in context15:55
flaper87(in case we change the package structure)15:55
kgriffsflaper87: do you think notifications will have a separate/standalone API?16:02
*** malini is now known as malini_afk16:02
flaper87kgriffs: mmh, I don't think so.16:06
flaper87mmh, actually, it might make sense16:07
flaper87but I haven't put many thoughts on that16:08
*** malini_afk is now known as malini16:09
acabreraI think amitgandhi has put some thought into notifications.16:10
kgriffsok, for now we just need to decide on directory structure16:10
acabreraI have a suspicion that there will be a separate API, since the purpose of a notification is different than that of a message bus.16:10
kgriffs"queues" is easier to spell16:11
flaper87acabrera: there'll be a separte API16:11
flaper87what I'm not sure of is whether it'll run as a standalone service16:11
flaper87I can see that working either way16:11
flaper87as long as the user doesn't have to deal with that16:11
flaper87I think it's fine16:11
kgriffs "queues" services /v1/queues16:12
kgriffs"notifications" services "/v1/notifications"16:12
kgriffsor something16:12
flaper87kgriffs: yup16:12
kgriffsnotifications will need to be more flexible, optional16:12
*** JRow has joined #openstack-marconi16:13
acabreraI'm happy with the proposed directory structure.16:13
acabrera+1 for the common folder, too.16:13
kgriffsok, so I vote for reorging with (cmd, common, proxy, queues)16:14
acabreraIt seems like that's the kind of utilities we might be able to give back to oslo16:14
acabrera+1 kgriffs16:14
flaper87acabrera: +116:14
* flaper87 votes for that as well16:14
amitgandhii see notifications being a seperate product that uses marconi16:15
amitgandhibut yeh something like v1/notiications16:15
flaper87acabrera: sorry, very slow review. :( I added some comments16:27
acabrerano worries, flaper87! Thanks so much for looking at it. :D16:27
acabreraI'm heading to the comments now.16:28
amitgandhikgriffs: flaper87: any idea what Get Actions is about: https://wiki.openstack.org/wiki/Marconi/specs/api/v1/responsecodes#Get_Actions16:28
kgriffsoh, that's old cruft16:29
amitgandhiok im deleting it16:29
flaper87amitgandhi: what kgriffs said16:34
flaper87:D16:34
flaper87sorry, my brain is working in slow montion16:34
acabreraflaper87: responded to all your review comments. Thanks for doing a thorough run through on that. :)16:35
*** JRow has quit IRC16:35
acabrerabrb office lunch. :)16:46
*** acabrera is now known as acabrera_away16:46
*** acabrera_away is now known as acabrera17:05
acabrerazyuan: unit test patch reviewed. Thanks!17:18
acabreraI'm going to spend the next hour or so putting together/submitting PyCon talk proposals.17:19
acabreraI have three in mind.17:20
kgriffsis Falcon one of them?17:22
kgriffs:D17:22
zyuanLOL17:22
acabreralol17:22
acabreraActually, that was proposal #4. :P17:22
acabreraI'm on the fence about presenting that one, but I might.17:22
zyuanhow many sessions in total17:23
zyuanon the conference?17:23
zyuanyou submit multiple proposals, not all will be presented, i guess17:24
acabreraI intend to give one 30 minute talk. :)17:24
acabreraI'm submitting multiple proposals to increase my chances of being accepted. :D17:24
acabreraMy three target topics are: 1) Migrating to Python 3 - Single Code Base w/ Py2.6 + 2.7 support, 2) Python 3 Pains: Concurrency in Python, 3) [Use Case] Developing a Scalable, Distributed Message Bus w/ Python (Marconi), 4) Falcon: High-performance API Framework17:24
acabreras/three/four17:24
openstackgerritZhihao Yuan proposed a change to stackforge/marconi: WPI: test(proxy): catalog tested  https://review.openstack.org/4636217:25
malinineed to WFH rest of the day..will be back online in ~45 min17:25
acabreraIf I end up getting selected to speak on #4, expect me to bug you a lot, kgriffs. :P17:25
acabreramalini: see you again soon. :)17:25
*** malini is now known as malini_afk17:26
openstackgerritZhihao Yuan proposed a change to stackforge/marconi: test(proxy): partitions  https://review.openstack.org/4636217:27
acabrerazyuan: Your unit tests tend to contain cleverly hidden references to nifty anime. I like it. :D17:29
kgriffsacabrera: I may propose the Falcon one17:29
kgriffslet me consider the idea over the weekend17:30
zyuanChanServ: ;)17:30
zyuanacabrera: ;)17:30
zyuan... i always complete your id with c<Tab>17:31
acabrerahehe17:31
acabreracppcabrera isn't here today. ;P17:31
* acabrera likes cycling through closely related nicknakes17:32
acabreras/nicknanes/nicknames17:32
acabrerakgriffs: PyCon talk proposals are due Monday. :P17:32
zyuanwow17:32
acabreraI've been stewing ideas for about a month now, and I decided to last-minute blitz the proposals. >.>17:32
kgriffsacabrera: gtk, i will email you to let you know by tomorrow re Falcon talk17:33
acabreraawesome, kgriffs17:34
acabreraI'll prepare a falcon proposal, just in case, but I'll only submit my initial three today.17:34
openstackgerritZhihao Yuan proposed a change to stackforge/marconi: test(proxy): partition related endpoints  https://review.openstack.org/4636217:43
acabrerakgriffs: correction - talsk proposals are due Sunday Sept. 15th. Lighting talks/tutorials are also accepted and these are due by October 15th.17:51
*** malini_afk is now known as malini18:17
kgriffsacabrera: proxy?18:29
acabreraready, kgriffs18:29
kgriffsok18:30
kgriffsso, first thing I wanted to clarify was how queues are directed to partitions and webheads within those partitions18:30
acabreraYes, that's a good place to start. I think I overengineered that portion of the proxy.18:31
acabreraSince given a URL to a load balancer as the entry point to a partition, the logic can be handled else where.18:32
acabrera(If a load balancer is even used - leaving options open!)18:32
acabreraI'm thinking of modifying partition handling so that an operator need only specify a single URL rather than a list of URLs, and select() will return the URL to the weight-selected partition.18:33
kgriffslet's try this: http://demo.tutorialzine.com/2012/08/nodejs-drawing-game/18:33
kgriffsoops18:33
kgriffshttp://flockdraw.com/danjvx18:34
acabrerasweet18:34
acabreracool, that's the image that came to my mind after reading your comment, kgriffs.18:37
kgriffsok, so..18:37
acabreraLB level 2 handled at partition level18:37
kgriffsyou can either have basically a VIP per partition18:38
kgriffsor you have to have all the web heads in the catalog18:38
kgriffsseems like we thought about this before and decided to skip the VIP idea - can't remember why18:38
kgriffsanother hop/latency?18:38
acabreraLatency per hop is about ~.7 ms, IIRC.18:39
acabreraIntra-DC18:39
acabrera0.7 ms18:39
acabreraIn any case, that ventures into the realm of things done outside of the proxy.18:40
acabreraPartition deployment is another topic entirely, if we agree that LB level 2 should be handled outside of the proxy.18:40
acabreraThoughts, kgriffs?18:40
kgriffshmmm18:41
kgriffsIf it is just a plain TCP LB it should be almost no overhead other than network latency18:41
acabreraagreed18:42
kgriffsso, cons for proxy going directly to partition nodes?18:42
kgriffs(fwiw 10g links will make latency a non-issue)18:43
acabrerait's over-specified, IMO, and probably hurts the weighting system unless the deployer very carefully sets weights.18:43
kgriffsi mean, you do the following18:43
kgriffsso, weights would only be used when creating a queue (new catalog entry)18:44
kgriffsfor other requests, you just lookup the partition for the queue, pick one of the webheads at random (or round-robin) in that partition, and forward on the request18:45
acabreraAh, I see.18:45
kgriffs#breakout(breakout()) weighting18:46
acabreralol18:46
kgriffs(have a different thought on that to share in a min18:46
kgriffsso, the cons I see to what I just outlined is you are flattening your operations namespace18:46
acabreracreate queue -> assign to partition, forward request -> select node from partition - the conn here is that we enforce a load-balancing strategy at the proxy level.18:47
acabreraI'd love to leave that decision open to the deployer.18:47
kgriffsyou end up having to deal with MIA nodes and stuff in the proxy, updating the catalog when adjusting a given partition's makeup is a pain, etc.18:47
kgriffsoz_akan_: ping18:47
acabrerayup, I hadn't even considered node cleansing18:47
kgriffspersonally, I vote for the VIP tree idea18:48
kgriffsService VIP ——> Partition VIPs18:48
acabreraone partition, one URL - that's my winning idea, which goes in line with VIP / partition18:48
kgriffsthat localizes all the partition ops and we don't have to write half of haproxy in marconi-proxy18:48
acabreras/winning/favored18:48
acabreraIt also simplifies the proxy-storage driver implementation requirements.18:49
acabreraPreviously, I was requiring storage drivers handle LB decisions - baaaad.18:49
kgriffsyep18:49
kgriffsyou said it18:49
acabrerayup. :D18:49
kgriffsso, an operator can deploy basically a single partition with a single VIP (LB)18:50
kgriffsOR,18:50
kgriffsthey can clone a bunch of those (no changes needed)18:50
acabreraor~18:50
kgriffsstick some marconi-proxy boxes in front18:50
acabrerasingle node / partition - wimpy partitions style18:50
kgriffswhich themselves are fronted by a LB VIP (public)18:50
kgriffssound good?18:51
acabrerayup18:51
kgriffsok, now regarding weighting18:51
acabreraone partition, one URL - noted and added to my proxy TODOs.18:51
kgriffsmy question is this:18:52
kgriffsis there a simple way to add a dynamic element to weighting such that hot queues are less likely to end up together on the same partition?18:53
acabrerahmm...18:53
acabrerahow would a hot queue be determined before it is acted upon/created? It seems like some sort of hint would have to be passed at creation time.18:54
acabreraX-Queue-Hotness: Extreme18:54
kgriffsLOL18:54
kgriffshow about some kind of heuristic?18:54
kgriffswhat *do* we know18:55
kgriffs1. how may queues are already assigned to a partition18:55
kgriffs2. How many queues a project has created already18:55
acabreraIf the proxy tracked how many queues were kept at each partition and how many *somethings* were done using that queue, we could approximate a partition's current "queue" load.18:56
acabreraThat could be used as a heuristic together with weight.18:56
kgriffs3. Historical stats of a partition load (maybe based on statsd or something from the partition's LB)18:56
kgriffsacabrera: sounds promising18:56
acabreraI'd like to see that in a later patch, but the base algorithm is very amenable to expansion by heuristic.18:57
kgriffscan you register a blueprint? we can look at it after we get a solid 1.0 proxy done18:57
acabreraThe base idea is that we select from partitions by creating a partition-selection-spectrum. Modifying the length of each partotion's spectrum using a heuristic would be a one line change.18:57
acabrera+1 kgriffs - I'll add that bp now. :)18:57
kgriffsto summarize, we want to factor in a partition's load to the weighting18:57
kgriffsyou would take static weight combined with load factor18:58
kgriffsok, moving on18:58
kgriffsre caching18:58
kgriffsI was thinking to not put that in the storage driver after all18:59
oz_akan_kgriffs: yesp18:59
kgriffsit's actually common pattern in web apps, for example, to just layer cache on top of storage18:59
acabrerare caching: I agree.19:00
kgriffsoz_akan_: do you have any major concerns with having each proxy be fronted with a LB, say haproxy?19:00
kgriffssorry, not proxy19:00
kgriffspartition19:00
kgriffseach partition has a VIP19:00
kgriffsand you would do simple TCP load balancing19:00
acabrerain_cache? | yes = read from cache, no = check primary store, update cache if found19:00
kgriffsok, good on that19:01
kgriffsthe only other thing is when migration lands, don't forget about cache invalidation19:01
kgriffsre what i will be working on, it will be some kind of in-process backend for oslo.cache19:01
kgriffsafter that, redis unless someone beats me to it. :p19:02
acabreradoesn't that already exist with the memory backend for oslo.cache?19:02
kgriffsacabrera: sort of. it doesn't do LRU19:02
acabreraahh, I see.19:03
kgriffsalso, i was toying with the idea of doing a backend for uwsgi's cache19:03
acabreraThat sounds like the kind of policy that would be great to apply to all oslo.cache backends.19:03
kgriffsthat one is interesting because it supports p2p replication19:03
acabreraoptional LRU spec19:03
kgriffsand it's shared between workers19:03
oz_akan_kgriffs: in front of proxies I think we will have cloud load balancers19:03
oz_akan_kgriffs: so do you mean another later between cloud load balancer and proxies?19:04
kgriffsyes19:04
oz_akan_later = layer19:04
kgriffseach partition would be fronted by it's own proxy19:04
kgriffsotherwise we end up writing half of haproxy into marconi-proxy19:04
kgriffsalso means you don't have to update the catalog whenever you add/remove/replace webheads19:04
oz_akan_plan was to have marconi-proxy instances to serve for all partitions19:05
kgriffsone moment19:05
oz_akan_otherwise, we would just put marconi-proxy functionality on marconi app19:06
oz_akan_I think that is why we started to call it proxy19:06
oz_akan_instead of placement service19:06
oz_akan_we may install haproxy on marcon-proxy instances, I am not sure how that would work19:07
kgriffshttps://etherpad.openstack.org/queuing-scratch19:07
acabrera#link https://blueprints.launchpad.net/marconi/+spec/smarter-partition-selection19:08
*** gordonsim has quit IRC19:20
kgriffsso19:24
acabrerawb19:24
kgriffscatalog should be out-of-band19:24
kgriffsmeaning19:24
kgriffsrequest comes in19:24
kgriffsproxy asks catalog driver for list of webheads OR single proxy href (whatever we decide)19:24
kgriffsproxy then forwards request to one of webheads OR the proxy19:25
kgriffs(latter proxy is haproxy on localhost)19:25
kgriffsmake sense?19:25
acabrerayews19:25
acabrera*yes19:25
kgriffsso, the catalog driver doesn't care which webhead gets the request, it is just informational only19:25
oz_akan_I think it gets overly complicated19:26
kgriffsnow, the proxy can just hit webheads round-robin using in-memory counter19:26
oz_akan_each marconi-proxy has haproxy19:26
kgriffsif one fails, it just tries the next19:26
kgriffsthe assumption would be that the operator deploys enough webheads that it is extremely unlikely for all of them to be down at the same time19:27
kgriffsproxy can blacklist the webhead in cache with expiration19:27
kgriffs(TTL for blacklist is configurable)19:27
kgriffsseems like this may not be very hard to implement after all?19:28
kgriffsgiven this means another cache lookup, having in-process, hierarchical would be a real help19:28
kgriffsor just keep in-process cache for blacklist19:28
kgriffswill be small, and won't take long for all workers on all proxies to blacklist a node if it is down hard19:29
kgriffsif it is only down for a second or two, then you don't want everyone blacklisting anyway19:29
* kgriffs is done19:29
* acabrera reads and thinks19:29
oz_akan_I like the idea marconi-proxy sending requests to web servers instead of sending to another proxy19:30
oz_akan_if it has to be another proxy, it could be local haproxy19:30
oz_akan_I don't like another component as it is one more moving part19:30
oz_akan_and doing round-robin and taking dead node out, as kgriffs described, shouldn't be so difficult19:31
acabreraI like the idea on "one partition, one URL" - having marconi-proxy hitting web servers directly w/o additional load balancing could be accomplished right not by regstering a partition containing a single URL per webhead,19:32
acabreraI don't like the idea of doing round-robin in the proxy.19:32
acabrerain the marconi-proxy19:32
acabreraThat forces a load balancing algorithm on future deployers.19:32
oz_akan_lets don't like idea compared to complexity to implement but with the complexity it will bring to the environment19:33
oz_akan_future deployers?19:33
oz_akan_marconi-proxy can support other algorithms later on19:33
acabreraThat will lead to a bloated marconi-proxy over time, while these load balancing algorithms are already a solved problem by other software.19:34
acabreraI'm in favor of marconi-proxy caring about the details of partitions, but not about the various web servers registered under a partition (if more than 1, which I believe is bad idea to have more than one node/contact point per partition).19:35
oz_akan_acabrera: assuming that marconi web servers will be identical, there is tiny chance someone will ever need anything other than round-robin19:37
oz_akan_and if needed, you could just configure one endpoint under marconi-proxy, which will actually be a proxy and then do what ever algorithm is supported19:38
acabreraso the argument to have marconi-proxy implement round robin is that it avoids adding another dependency...19:39
oz_akan_yes, that is my argument19:39
kgriffsguys, consider that you could just have a list of "webheads" with two entries that go to two haproxies or a single entry going to a VIP for the partition19:40
kgriffsthen an operator can do advanced LB stuff19:40
kgriffsotherwise they can just let marconi-proxy do it's simple LB algorithm19:41
kgriffss/it's/its19:41
oz_akan_yes, round robin is just enough, I think for ever.19:42
acabreraalright19:42
acabreraso given this discussion - keep marconi-proxy as-is? (round-robin, lists of nodes per partition)?19:42
oz_akan_acabrera: haven't you already implemented that anywyas?19:42
acabrerayup. :P19:42
oz_akan_so why are we loosing time?19:42
acabreramaking sure it's the right way to go.19:43
oz_akan_makes sense19:43
kgriffsacabrera: not really as-is, i mean the storage driver doesn't do the lb19:45
kgriffsor has that already been moved?19:45
acabreraalright, marconi proxy stays as is. The algorithm for round robin became much easier to use with the mongo driver patch. All a storage driver needs to do is provide a list of [(name, weight), ...] and then pass the chosen partition to round_robin(), which yields a node from that partition after 2 more DB queries...19:45
oz_akan_so even if we had decided to have a proxy, we could just set up current implementation to point to one endpoint which would be a load balancer / proxy19:45
acabrerakgriffs: as-is since the mongo patch19:45
acabreraI simplified it. :)19:45
acabrerastorage doesn't LB.19:46
kgriffswait a sec19:46
oz_akan_storage driver doesn't have to do lb, does it?19:46
kgriffsstorage driver ===> catalog driver19:46
oz_akan_LB is done on transportation level19:46
kgriffs+!19:46
kgriffscatalog driver just returns info19:46
acabrerastorage driver --> (catalogue + partitions)19:46
oz_akan_marcon-proxy will be reading from mongodb. so HA is provided by mogno19:47
kgriffsthat's fine, but it shouldn't have any logic to pick a webhead within the partition19:47
kgriffsi mean, the catalog driver shouldn't19:47
oz_akan_yes19:47
acabreraI need to further move the round-robin index out of storage and place that in-process.19:47
kgriffsI imagine handler like:19:47
kgriffson_get()...19:47
oz_akan_queues will be mapped to partition and partition mapped to web servers19:47
kgriffsdef on_get(…):19:48
kgriffs1. get catalog entry for queue19:49
kgriffs2. pick href from list of webheads in partition record from catalog19:49
kgriffs3. forward request to that href19:49
kgriffs(2) will use the simple round-robin, in-proc blacklisting discussed earllier19:49
acabrerahmm...19:50
oz_akan_1- get partition - web server mapping19:50
kgriffs(1) is the only thing the storage driver thingy is responsible for19:50
kgriffs(2) is a helper function19:50
acabreraI need to make some more updates, since I don't currently support per request LB.19:50
acabreraI need to make it so that queue stores a partition name, rather than an href.19:50
kgriffsyou just have a RoundRobin() object instance for (2)19:51
oz_akan_to keep mapping data small, I think we should map queue to partition id19:51
oz_akan_and parition id to web servers19:51
kgriffsthat is shared by on_* methods19:51
oz_akan_or urls19:51
kgriffsthat instance keeps a blacklist and counter19:51
acabrera+1 oz_akan_ - I was just going to use partition name.19:51
kgriffsrr.choose(list_of_webheads)19:51
kgriffslike random.choice19:51
oz_akan_guy, I will catch acabrera on Monday to go over the plan, I need to finish some stuff now19:51
oz_akan_guy =guya19:52
kgriffskk19:52
oz_akan_guys...19:52
acabrerak19:52
*** vkmc has quit IRC20:01
*** tedross has quit IRC20:19
zyuanacabrera: http://openmp.org/wp/2013/09/clang/20:30
zyuanlooks good http://clang-omp.github.io/20:31
*** amitgandhi has quit IRC20:32
*** amitgandhi has joined #openstack-marconi20:37
acabrerasweet20:45
acabreraIMO, there's no reason AFAICT to continue using gcc, except to compile the linux kernel.20:47
acabreraseems like clang will surpass gcc in every respect before too much longer.20:47
zyuanhahahaha linux kernel20:47
zyuanacabrera: the thing left is transactional memory. gcc's libitm already support haswell's HTM instructions20:58
*** acabrera has quit IRC21:12
*** openstackgerrit has quit IRC21:14
*** flaper87 is now known as flaper87|afk21:27
*** amitgandhi has quit IRC21:29
*** oz_akan_ has quit IRC21:37
*** vkmc has joined #openstack-marconi22:53
*** reed has joined #openstack-marconi23:01
reedi just read that this channel is a source of good laughs :)23:01
reedhttp://www.openstack.org/blog/2013/09/open-mic-spotlight-flavio-percoco/23:02
kgriffshehe23:26
kgriffshang out with us for a week and you'll see what he means. :p23:27
reedwill do :) set the channel to autojoin23:30
*** kgriffs is now known as kgriffs_afk23:40

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!