| *** hazmat has joined #openstack | 00:05 | |
| *** dmd17_ has joined #openstack | 00:06 | |
| *** devcamcar has quit IRC | 00:06 | |
| *** dmd17 has quit IRC | 00:09 | |
| *** dmd17_ is now known as dmd17 | 00:09 | |
| *** devcamcar has joined #openstack | 00:13 | |
| *** adrian_otto has quit IRC | 00:15 | |
| *** adjohn has joined #openstack | 00:16 | |
| *** devcamcar has quit IRC | 00:16 | |
| Glace | hdo you guys support glusterfs? | 00:20 |
|---|---|---|
| alekibango | Glace: try #gluster here on freenode | 00:20 |
| alekibango | i do :D | 00:20 |
| *** hazmat has quit IRC | 00:23 | |
| uvirtbot | New bug: #613264 in nova "api / objectstore should allow specifying ip to bind to" [Undecided,New] https://launchpad.net/bugs/613264 | 00:25 |
| *** hazmat has joined #openstack | 00:31 | |
| *** devcamcar has joined #openstack | 00:32 | |
| *** pcrews has quit IRC | 00:42 | |
| *** zenmatt has quit IRC | 00:44 | |
| *** devcamcar has quit IRC | 00:58 | |
| *** gnrfan has quit IRC | 01:01 | |
| *** jsmith is now known as jsmith-away | 01:02 | |
| *** hazmat has quit IRC | 01:02 | |
| *** kallistec has quit IRC | 01:08 | |
| *** gustavomzw has left #openstack | 01:10 | |
| *** pcrews has joined #openstack | 01:12 | |
| *** maximilien has quit IRC | 01:13 | |
| *** justinsheehy has quit IRC | 01:23 | |
| *** tomo__ has joined #openstack | 01:25 | |
| *** jakedahn has joined #openstack | 01:25 | |
| *** tomo_bot has joined #openstack | 01:28 | |
| *** tomo__ has quit IRC | 01:28 | |
| *** justinsheehy has joined #openstack | 01:29 | |
| *** mrayzenoss has joined #openstack | 01:32 | |
| *** mrayzenoss is now known as mray | 01:32 | |
| *** mray has joined #openstack | 01:32 | |
| *** kallistec has joined #openstack | 01:36 | |
| *** rdw has quit IRC | 01:40 | |
| *** pcrews has quit IRC | 01:40 | |
| *** sirp1 has quit IRC | 01:56 | |
| *** codejunkie has left #openstack | 01:58 | |
| *** RobertLJ has joined #openstack | 02:04 | |
| *** jdmaturen has quit IRC | 02:18 | |
| *** miclorb_ has quit IRC | 02:20 | |
| *** ptremblett has joined #openstack | 02:21 | |
| *** sirp1 has joined #openstack | 02:23 | |
| *** jdmaturen has joined #openstack | 02:43 | |
| *** mray has quit IRC | 02:45 | |
| *** ikk has quit IRC | 02:50 | |
| *** stewart has quit IRC | 02:55 | |
| *** stewart has joined #openstack | 03:07 | |
| *** kashyapc has joined #openstack | 03:12 | |
| *** miclorb_ has joined #openstack | 03:21 | |
| *** RobertLJ has quit IRC | 03:25 | |
| *** silassewell has quit IRC | 03:27 | |
| *** blpiatt has joined #openstack | 03:29 | |
| *** jsgotangco has joined #openstack | 03:47 | |
| *** alii_ has joined #openstack | 03:55 | |
| *** miclorb_ has quit IRC | 03:59 | |
| *** alii_ has joined #openstack | 03:59 | |
| *** alii_ has quit IRC | 04:01 | |
| *** alii_ has joined #openstack | 04:01 | |
| *** jamiew has quit IRC | 04:03 | |
| *** blpiatt has quit IRC | 04:08 | |
| *** dmd17 has quit IRC | 04:12 | |
| *** adjohn has quit IRC | 04:12 | |
| *** adjohn has joined #openstack | 04:12 | |
| *** dmd17 has joined #openstack | 04:12 | |
| *** jakedahn has quit IRC | 04:17 | |
| *** jakedahn has joined #openstack | 04:17 | |
| *** jakedahn has quit IRC | 04:22 | |
| *** jakedahn has joined #openstack | 04:23 | |
| *** tobym has joined #openstack | 04:36 | |
| *** dendro-afk is now known as dendrobates | 04:40 | |
| *** miclorb_ has joined #openstack | 04:43 | |
| *** stewart has quit IRC | 04:48 | |
| *** maximilien has joined #openstack | 04:54 | |
| *** rdw has joined #openstack | 04:59 | |
| *** stewart has joined #openstack | 05:01 | |
| *** maximilien has quit IRC | 05:20 | |
| *** justinsb has quit IRC | 05:35 | |
| *** sophiap has joined #openstack | 05:41 | |
| *** sirp1 has quit IRC | 05:55 | |
| *** allsystemsarego has joined #openstack | 05:58 | |
| *** rajijoom has joined #openstack | 06:05 | |
| *** BartVB has joined #openstack | 06:07 | |
| *** metoikos has joined #openstack | 06:19 | |
| mtaylor | termie: hey - may sound stupid, but mailing list mail from launchpad isn't hitting a spam filter for you do you? | 06:22 |
| *** jdmaturen has quit IRC | 06:25 | |
| *** kallistec has quit IRC | 06:30 | |
| soren | mtaylor: Do you know if anyone is working on packaging Swift for Debian/Ubuntu? Feature freeze is nigh :) | 06:31 |
| mtaylor | soren: well, it's packaged and up in the ppa | 06:33 |
| mtaylor | soren: I don't know if anyone is actively working on uploading it though | 06:33 |
| soren | ok. | 06:34 |
| soren | mtaylor: I'm probably uploading nova today. | 06:34 |
| ttx | soren: \o/ | 06:35 |
| soren | mtaylor: I fixed the problem with Redis in the builds, so we seem to be golden now: https://edge.launchpad.net/~soren/+archive/ppa/+build/1902914 | 06:36 |
| mtaylor | soren: excellent | 06:37 |
| mtaylor | soren: uploading to ubuntu I'm assuming | 06:37 |
| soren | mtaylor: Yup. | 06:37 |
| soren | mtaylor: Debian should be fine, too, really. I think all the dependencies are met there too now. | 06:38 |
| * soren goes to get his daughter ready for day care | 06:42 | |
| *** brd_from_italy has joined #openstack | 06:55 | |
| *** londo__ has joined #openstack | 07:04 | |
| *** londo has quit IRC | 07:04 | |
| *** sophiap has quit IRC | 07:09 | |
| *** jsgotangco has quit IRC | 07:19 | |
| *** calavera has joined #openstack | 07:21 | |
| *** almaisan-away is now known as al-maisan | 07:24 | |
| termie | mtaylor: nope, i've checked a few times and checking right now it still is not, however it seems i've been missing mail from a variety of cute local girls | 07:30 |
| *** rajijoom has quit IRC | 07:58 | |
| *** calavera has quit IRC | 08:00 | |
| *** calavera has joined #openstack | 08:05 | |
| *** miclorb_ has quit IRC | 08:12 | |
| *** rnewson has joined #openstack | 08:16 | |
| *** rnewson has joined #openstack | 08:16 | |
| *** calavera has quit IRC | 08:28 | |
| *** calavera has joined #openstack | 08:31 | |
| *** dmd17 has quit IRC | 08:50 | |
| *** ptremblett has quit IRC | 08:53 | |
| soren | anotherjesse: I'm about to upload Nova to Ubuntu. You're still listed as the maintainer in the package. Do you consider that accurate? Are you intending to be involved in the packaging effort going forward? | 09:08 |
| *** adjohn has quit IRC | 09:12 | |
| *** calavera has quit IRC | 09:14 | |
| alekibango | soren: where is source package? i have debian amd64 -> i will test them for you on debian | 09:16 |
| alekibango | found it... | 09:17 |
| alekibango | btw your glusterfs packages are realy old now | 09:17 |
| alekibango | :D | 09:17 |
| alekibango | soren: this url? http://ppa.launchpad.net/soren/nova/ | 09:18 |
| *** adjohn has joined #openstack | 09:25 | |
| soren | /my/ glusterfs packages? | 09:25 |
| *** veszig has quit IRC | 09:28 | |
| *** jtimberman has quit IRC | 09:28 | |
| *** veszig has joined #openstack | 09:32 | |
| *** jakedahn has quit IRC | 09:33 | |
| *** jakedahn has joined #openstack | 09:34 | |
| soren | ttx: Hm... Maybe you have some good ideas for this: | 09:43 |
| soren | ttx: Nova uses rabbitmq for messaging. | 09:43 |
| soren | ttx: and redis as its datastore | 09:43 |
| soren | ttx: We can have a whole bunch of frontend API servers, but we don't want all of them to have their own rabbitmq and redis instances. | 09:44 |
| soren | ttx: so just making nova-api depend on rabbitmq-server and redis-server won't work. Even doing it as a recommends seems wrong, since really only the first API server needs them. | 09:45 |
| soren | ttx: So I'm thinking of creating a metapackage with some install profiles in it. | 09:46 |
| ttx | hmm, right, looks like you'd need a nova-api-with-extra-stuff | 09:46 |
| soren | ttx: So a meta-package that'll install all of nova-api, redis-server and rabbitmq. | 09:47 |
| soren | ttx: Yeah. Not sure what to call it. | 09:47 |
| ttx | you need *one* server with all the stuff ? or *some* servers with all the stuff ? | 09:47 |
| ttx | "only the first" sounds all spof | 09:48 |
| soren | ttx: Each nova-api server needs to talk to a Redis server and a RabbitMQ-server. | 09:48 |
| ttx | but not necessarily on the same machine ? | 09:48 |
| *** jtimberman has joined #openstack | 09:48 | |
| *** jtimberman has joined #openstack | 09:48 | |
| soren | ttx: There can be any number of rabbitmq servers on the netwrok. | 09:48 |
| soren | ttx: No, not at all. | 09:49 |
| *** miclorb_ has joined #openstack | 09:49 | |
| soren | ttx: None of the components of Nova speak directly to each other. Everything goes through Rabbit. | 09:49 |
| ttx | why not have a nova-redis+rabbit metapackage then, that can be installed (or not) on the same machine as the nova-api ? | 09:49 |
| ttx | soren: you abandoned zeromq completely or are still considering it ? | 09:50 |
| soren | ttx: We haven't really talked about it yet. | 09:50 |
| soren | Much. | 09:50 |
| * ttx is thinking about that install profiles idea... Could be interesting to have some "topology" metapackages that would propose decent defaults for various topologies | 09:52 | |
| * soren wanders over to his whiteboard for a little bit | 09:52 | |
| ttx | though I don't know enough of nova yet to know if it would make any sense :) | 09:52 |
| ttx | maybe documentation is just a better way to do it. "all in one ? just install those:..." | 09:54 |
| ttx | but I'd definitely have some nova-branded metapackages to make logical units of "stuff that should be deployed on a single machine" | 09:55 |
| soren | Package: nova-api-standalone Depends: rabbitmq-server, redis-server, nova-api | 09:57 |
| soren | Maybe. | 09:57 |
| soren | Default config looks on localhost for both rabbit and redis, so that'll "just work2. | 09:57 |
| soren | ". | 09:57 |
| ttx | or just nova-api-extras depend in rabbit and redis, and doc says "single machine ? apt-get install nova-api nova-api-extras" | 09:58 |
| soren | nova-api-extras doesn't sound like a metapackage to my ear. | 09:58 |
| *** ptremblett has joined #openstack | 09:58 | |
| uvirtbot | New bug: #613383 in nova "euca-terminate-instances fails to remove instance" [Undecided,New] https://launchpad.net/bugs/613383 | 10:01 |
| *** ikk has joined #openstack | 10:04 | |
| * soren wonders why branching lp:euca2ools involves downloading >17MB of stuff. | 10:04 | |
| *** jakedahn has quit IRC | 10:06 | |
| soren | Ah, because it used to embed a bunch of deps. *sigh* | 10:21 |
| *** silassewell has joined #openstack | 10:44 | |
| *** Tony[] has joined #openstack | 10:54 | |
| Tony[] | morning, can anyone point me at any architecture diags or howto type docs for openstack so I can start educating myself?? | 10:55 |
| Tony[] | to clarify, I've looked at the sites for each bit, but I'm trying get a good overview of how all the pieces fit together | 11:04 |
| *** justinsheehy has quit IRC | 11:10 | |
| *** Tony[] has quit IRC | 11:11 | |
| *** miclorb_ has quit IRC | 11:17 | |
| *** ctennis has quit IRC | 11:34 | |
| *** rajijoom has joined #openstack | 11:35 | |
| *** Glace has quit IRC | 11:36 | |
| *** justinsheehy has joined #openstack | 11:43 | |
| *** jaypipes has quit IRC | 11:44 | |
| *** Podilarius has left #openstack | 12:02 | |
| *** sophiap has joined #openstack | 12:05 | |
| *** jonesy_ has quit IRC | 12:10 | |
| *** _mattf has quit IRC | 12:10 | |
| *** carolinad28 has joined #openstack | 12:20 | |
| *** maximilien has joined #openstack | 12:22 | |
| *** hazmat has joined #openstack | 12:29 | |
| *** sophiap has quit IRC | 12:30 | |
| *** ctennis has joined #openstack | 12:30 | |
| *** codejunkie has joined #openstack | 12:38 | |
| *** jsmith-away is now known as jsmith | 12:42 | |
| *** aliguori has joined #openstack | 12:43 | |
| *** abecc has joined #openstack | 13:07 | |
| *** Glace has joined #openstack | 13:20 | |
| *** tonywolf has joined #openstack | 13:24 | |
| *** jbd has joined #openstack | 13:26 | |
| *** jsmith is now known as jsmith-away | 13:28 | |
| *** jonesy_ has joined #openstack | 13:34 | |
| *** mtaylor has quit IRC | 13:43 | |
| *** kashyapc has quit IRC | 13:46 | |
| *** cjwilson has quit IRC | 13:49 | |
| *** Podilarius has joined #openstack | 13:50 | |
| *** gustavomzw has joined #openstack | 13:52 | |
| *** Podilarius has left #openstack | 13:55 | |
| *** jaypipes has joined #openstack | 14:05 | |
| *** ptremblett has quit IRC | 14:14 | |
| *** ptremblett has joined #openstack | 14:15 | |
| *** sophiap has joined #openstack | 14:17 | |
| *** pcrews has joined #openstack | 14:24 | |
| *** sirp1 has joined #openstack | 14:25 | |
| *** mrayzenoss has joined #openstack | 14:29 | |
| *** mrayzenoss is now known as mray | 14:29 | |
| *** mray has joined #openstack | 14:29 | |
| *** gundlach has joined #openstack | 14:33 | |
| *** pvo has joined #openstack | 14:38 | |
| *** pvo has joined #openstack | 14:38 | |
| *** ChanServ sets mode: +v pvo | 14:38 | |
| *** pvo has quit IRC | 14:40 | |
| *** pvo has joined #openstack | 14:41 | |
| *** ChanServ sets mode: +v pvo | 14:41 | |
| *** dendrobates is now known as dendro-afk | 14:42 | |
| *** sophiap has quit IRC | 14:44 | |
| *** mray has quit IRC | 14:44 | |
| *** sophiap has joined #openstack | 14:45 | |
| *** sophiap has quit IRC | 14:47 | |
| *** dendro-afk is now known as dendrobates | 14:47 | |
| * hazmat is starting to come around to eventlet over twisted | 14:48 | |
| *** adrian_otto has joined #openstack | 14:50 | |
| *** sophiap has joined #openstack | 14:53 | |
| gundlach | hazmat: why's that? | 14:54 |
| *** jsmith-away is now known as jsmith | 14:55 | |
| *** metoikos has quit IRC | 14:55 | |
| *** tr3buchet has joined #openstack | 14:56 | |
| uvirtbot | New bug: #613479 in nova "Support querying versions in API" [Undecided,New] https://launchpad.net/bugs/613479 | 14:56 |
| *** metoikos has joined #openstack | 14:56 | |
| uvirtbot | New bug: #613481 in nova "Support throwing Faults in API" [Undecided,New] https://launchpad.net/bugs/613481 | 15:01 |
| *** silassewell has quit IRC | 15:02 | |
| *** RobertLJ has joined #openstack | 15:04 | |
| *** matclayton has joined #openstack | 15:05 | |
| *** sophiap has quit IRC | 15:10 | |
| *** cjwilson has joined #openstack | 15:11 | |
| *** blpiatt has joined #openstack | 15:18 | |
| *** codejunkie has quit IRC | 15:21 | |
| *** dendrobates is now known as dendro-afk | 15:22 | |
| hazmat | gundlach, i did a lightning talk on async programming via example w/ twisted and eventlet last night.. it was clear from the discussion that folks found the eventlet approach much more approachable. | 15:30 |
| gundlach | hazmat: cool. had you gone into the talk leaning towards twisted? | 15:31 |
| hazmat | gundlach, i had | 15:31 |
| hazmat | gundlach, actually what convinced me, was a little experiment i just did earlier this morning.. you can actually use pdb w/ greenlet based approaches | 15:32 |
| hazmat | its rather useless with twisted | 15:32 |
| gundlach | that's a good datapoint for the discussion. eday and i think that eventlet is the way to go so that we can use WSGI to make our apps as simple as possible. maybe we're finally coelescing upon a decision :) | 15:32 |
| hazmat | from app debugging perspective that adds huge value. | 15:32 |
| gundlach | wow, pdb with greenlet? cool! | 15:32 |
| gundlach | yeah absolutely. | 15:32 |
| hazmat | it maintains the stack trace within a greenlet | 15:32 |
| gundlach | would you mind (if you haven't already) posting your thoughts to the discussion on the mailing list? if we can settle on an answer it will help us move forward. | 15:33 |
| hazmat | i'll post on the list re the above.. but i have to wait till i'm done with work today.. other things to do. | 15:33 |
| gundlach | ok -- i was about to post anyway, so i'll mention what you said, and you can flesh it out. | 15:34 |
| creiht | Where's mtaylor when you need him :/ | 15:35 |
| creiht | soren: just fyi re: packaging swift, I'm not sure the swift package in the ppa is fully funtionaly yet | 15:35 |
| hazmat | gundlach, sent.. | 15:40 |
| gundlach | hazmat: ty, i was composing mine so i'll adapt appropriately :) | 15:41 |
| *** pstevensza has joined #openstack | 15:45 | |
| *** Phil_FL has quit IRC | 15:46 | |
| *** tobym has quit IRC | 15:46 | |
| gundlach | hazmat: you posted to the "[Nova] Twisted or Eventlet?" thread? | 15:48 |
| hazmat | gundlach, yeah.. | 15:48 |
| hazmat | gundlach, https://lists.launchpad.net/nova/msg00082.html | 15:49 |
| *** dendro-afk is now known as dendrobates | 15:49 | |
| gundlach | hazmat: ty, my mail must be behind | 15:49 |
| hazmat | gundlach, care to comment on gevent vs. greenlet | 15:50 |
| hazmat | er. eventlet | 15:50 |
| *** Podilarius has joined #openstack | 15:50 | |
| creiht | just use eventlet and make your life easy :) | 15:50 |
| gundlach | hazmat: only what i've researched, which is that gevent was made as a fork to eventlet by a guy who wasn't happy with it -- and that the things he complained about are fixed in eventlet. gevent is maintained by one guy, eventlet is a real project. | 15:50 |
| *** pstevensza has quit IRC | 15:51 | |
| creiht | gevent is a little faster since it is much closer to libevent | 15:51 |
| redbo | I don't know if that's true. I could never tell much difference when benchmarking. | 15:51 |
| hazmat | gundlach, i only see one active maintainer on eventlet.. lots of legacy code in eventlet as well, very different stdlib integration techniques, and faster performance in greenlet. | 15:52 |
| hazmat | er.. gevent | 15:52 |
| * creiht is just quoting blogs I read earlier | 15:52 | |
| hazmat | http://nichol.as/benchmark-of-python-web-servers | 15:52 |
| creiht | hazmat: Most of the legacy code is removed from eventlet | 15:52 |
| gundlach | hazmat: i may be mistaken. again, my data is just from reading "gevent vs eventlet" threads on the web. | 15:52 |
| hazmat | http://nichol.as/asynchronous-servers-in-python | 15:53 |
| creiht | there is one maintainer, but several contributors, and he is very easy to work with | 15:53 |
| creiht | the gevent maintainer isn't as easy to work with | 15:53 |
| * redbo waves at rdw | 15:53 | |
| creiht | ^- the main reason he created gevent | 15:53 |
| uvirtbot | creiht: Error: "-" is not a valid command. | 15:53 |
| creiht | not rdw, but that he isn't easy to work with :) | 15:53 |
| * creiht <3 rdw | 15:53 | |
| hazmat | creiht, he seems fine to work w/ on the gevent list | 15:53 |
| hazmat | i guess i'm wondering the differences on a technical basis.. sounds like besides.. its just familiarity w/ the eventlet project. | 15:54 |
| hazmat | on a technical basis things would point towards gevent afaics | 15:56 |
| creiht | he wasn't so easy to work with when things didn't go exactly the way he wanted with eventlet | 15:56 |
| hazmat | creiht, i still some legacy code going through the eventlet source.. from twisted integration to deprecated names support | 15:56 |
| creiht | gevent doesn't work cooperate so well with threads | 15:56 |
| hazmat | creiht, yeah.. the process and thread integration do seem lacking in gevent... i noticed gunicorn basically had to implement its own process pool | 15:57 |
| creiht | eventlet has async support for pgsql and mysql | 15:58 |
| creiht | the twisted stuff was donated by someone, and then never touched again by that person | 15:58 |
| _0x44 | Doesn't eventlet have two "core" maintainers? Donovan and that M(att|ike) guy? | 16:00 |
| creiht | Ryan (rdw) is the maintainer | 16:01 |
| creiht | Donovan was an original maintainer, and he pops in and out | 16:01 |
| _0x44 | I need to be better with names. :S | 16:02 |
| creiht | In fact, Denis is the one who added the twisted support to eventlet :) | 16:04 |
| creiht | https://lists.secondlife.com/pipermail/eventletdev/2008-October/000147.html | 16:04 |
| *** kashyapc has joined #openstack | 16:05 | |
| *** brd_from_italy has quit IRC | 16:07 | |
| *** lbieber has joined #openstack | 16:09 | |
| *** hazmat has quit IRC | 16:10 | |
| *** tobym has joined #openstack | 16:16 | |
| *** hazmat has joined #openstack | 16:18 | |
| *** jdmaturen has joined #openstack | 16:18 | |
| *** tobym has quit IRC | 16:19 | |
| *** rnewson has quit IRC | 16:21 | |
| *** jdmaturen has quit IRC | 16:27 | |
| eday | gundlach: so, we should talk some more about structuring the wsgi layers... I'm thinking we should order the wsgi layers from auth right down to individual request handlers... basically a wsgi tree with selection at each layer. Todd has done some of this in the rackspace/wsgi endpoints already, but we should make each layer use a wsgi interface so we can transparently insert a proxy if needed | 16:29 |
| *** mtaylor has joined #openstack | 16:31 | |
| *** ChanServ sets mode: +v mtaylor | 16:31 | |
| gundlach | eday: ok, if i understand you, you're proposing that we would have a wsgi layer for every / in the URL, basically -- directing traffic to the correct choice of next layer. yes? | 16:32 |
| gundlach | and are you saying that Todd's rsapi branch is using wsgi? i thought it was on twisted? | 16:33 |
| eday | gundlach: no, it's wsgi with simple http server | 16:33 |
| gundlach | eday: ah, gotcha. | 16:33 |
| *** maximilien has quit IRC | 16:34 | |
| creiht | mtaylor: howdy | 16:34 |
| eday | gundlach: well, the wsgi layer may not map directly onto path elements, you might hit two layers for the same URL. most routing between will happen based on URLs though | 16:34 |
| gundlach | re wsgi tree: i'm all for peeling some code off of wsgi layer A to make wsgi layer B, if it makes layer A simpler. i don't quite see yet how peeling once per / would simplify the routing -- one per API version would probably be useful, but i dunno how tall to make the stack | 16:35 |
| creiht | mtaylor: btw.. I don't think the swift package is fully funtional yet | 16:35 |
| mtaylor | creiht: no? well poo | 16:36 |
| gundlach | eday: maybe i'm not on the same page yet -- why do we need a tree instead of a stack of layers? if we choose between APIs as our only "branch" point, and let the API request handler do normal routing a la django routes or whatever, that seems fine to me | 16:36 |
| mtaylor | creiht: letterj was saying he had some bugs to file against the packages compared to what you guys had been cutting internally | 16:37 |
| creiht | yeah | 16:37 |
| creiht | cool... just making sure he had talked to you about it. I will get with him and see if he has filed those bugs yet | 16:38 |
| *** rnewson has joined #openstack | 16:38 | |
| * creiht hopes he didn't scare hazmat away :/ | 16:39 | |
| *** hazmat has quit IRC | 16:40 | |
| eday | gundlach: so, I'm not proposing a layer per /, it's whatever is most efficient and makes sense... probably a combo of hard-coded rules and pattern matching | 16:41 |
| gundlach | eday: ok, sure, i agree with "let's make a layer wherever it makes sense" :) when you said "I'm thinking we should order the wsgi layers from auth right down to individual request handlers" I guess I thought you were suggesting something more specific. | 16:42 |
| gundlach | are you saying we should hash out now exactly what those layers will be, in what order? i don't think i can do that until i'm actually in the code. | 16:42 |
| eday | gundlach: but I think it needs to be more tree than stack because not all states are valid at every level. Take a look at nova/endpoint/rackspace.py it first splits between v1.0 and servers, and then for each, there are differnet handlers | 16:42 |
| eday | gundlach: I don't think we need to hash out the layers, but we need to hash out a framework so we have something sensible to write code in :) | 16:43 |
| gundlach | eday: gotcha. lemme look at rackspace.py again, brb | 16:43 |
| eday | gundlach: I'm looking to see if there is anything existing already, especially one with a proxy option built in | 16:43 |
| *** RobertLJ has quit IRC | 16:46 | |
| gundlach | eday: re "more tree than stack" -- i think i disagree so far. after SSL termination, we could have a layer that picks the right API version and branches at that point to a RS API stack, or to an EC2 stack. The EC2 stack would do EC2 auth as a layer, then maybe more layers, then hit a route-to-handler layer. The RS API stack would have its own auth layer, then maybe more layers, and then a route-to-handler layer. | 16:47 |
| *** mrayzenoss has joined #openstack | 16:47 | |
| *** mrayzenoss is now known as mray | 16:47 | |
| *** mray has joined #openstack | 16:47 | |
| *** rnewson has quit IRC | 16:47 | |
| gundlach | so the only real branching i see is between versions of the API. | 16:47 |
| eday | gundlach: currently, yes, but then in RackspaceCloudServerApi.index it branches on method | 16:48 |
| *** jdmaturen has joined #openstack | 16:49 | |
| eday | gundlach: well, actually before that, it branches on URL based on available method names in the controller classes | 16:49 |
| *** RobertLJ has joined #openstack | 16:49 | |
| *** jdmaturen has quit IRC | 16:49 | |
| *** jdmaturen has joined #openstack | 16:50 | |
| *** rnewson has joined #openstack | 16:53 | |
| *** pvo has quit IRC | 16:55 | |
| *** blpiatt has quit IRC | 17:00 | |
| *** blpiatt has joined #openstack | 17:00 | |
| *** jsmith is now known as jsmith-away | 17:06 | |
| *** mmalgeri1 has joined #openstack | 17:09 | |
| *** maximilien has joined #openstack | 17:11 | |
| *** jaypipes has quit IRC | 17:14 | |
| *** mray has quit IRC | 17:14 | |
| *** Phil_FL has joined #openstack | 17:15 | |
| *** hazmat has joined #openstack | 17:19 | |
| gundlach | a really nice, short wsgi (and webob) tutorial: http://archimedeanco.com/wsgi-tutorial/ | 17:19 |
| gundlach | eday: you're at rev 141 of ~xtoddx/nova/api, right? I'm not sure that i'm seeing what you're seeing, but i'm not familiar enough with bzr to be sure i'm up to date | 17:21 |
| gundlach | RackspaceCloudServerApi.index is line 202, and it just contains "return self.detail(env)" | 17:21 |
| *** matclayton has left #openstack | 17:22 | |
| eday | gundlach: I'm at rev 202 | 17:22 |
| gundlach | ah hah! :) | 17:22 |
| gundlach | michael@conch:~/rackspace/nova_branches/api$ bzr up | 17:23 |
| gundlach | Tree is up to date at revision 141 of branch /home/michael/rackspace/nova_branches/api | 17:23 |
| gundlach | what command do *you* use? | 17:23 |
| gundlach | michael@conch:~/rackspace/nova_branches/api$ bzr info | 17:23 |
| gundlach | Standalone tree (format: 2a) | 17:23 |
| gundlach | Location: | 17:23 |
| gundlach | branch root: . | 17:23 |
| gundlach | Related branches: | 17:23 |
| gundlach | parent branch: bzr+ssh://bazaar.launchpad.net/~xtoddx/nova/api/ | 17:23 |
| eday | bzr pull | 17:23 |
| eday | this is originally branched from lp:nova | 17:24 |
| eday | now at revno 205 (just pulled changes from last night) | 17:24 |
| *** jamiew_ has joined #openstack | 17:25 | |
| *** jamiew_ is now known as jamiew | 17:25 | |
| *** metoikos has quit IRC | 17:25 | |
| * gundlach goes to read Soren's bzr tutorial again because he's lost | 17:27 | |
| *** codejunkie has joined #openstack | 17:28 | |
| eday | gundlach: bzr init-repo nova, cd nova, bzr branch lp:nova, done :) | 17:31 |
| _0x44 | Or git bzr clone lp:nova | 17:31 |
| gundlach | hm, ok i'm catching up with you -- you're on lp:nova, not reading todd's rsapi branch. yes? | 17:31 |
| eday | and then 'bzr pull' just pulls the latest lp:nova down, this keeps your local trunk copy in sync | 17:31 |
| eday | gundlach: nope, not looking at todds, just looking at trunk | 17:32 |
| mtaylor | gundlach: bzr up does something completely different | 17:34 |
| *** pvo has joined #openstack | 17:36 | |
| *** ChanServ sets mode: +v pvo | 17:36 | |
| *** codejunkie has quit IRC | 17:41 | |
| *** mrayzenoss has joined #openstack | 17:44 | |
| *** mrayzenoss is now known as mray | 17:44 | |
| *** mray has joined #openstack | 17:44 | |
| * ianweller finally gets around to the openstack-swift package review for fedora | 17:48 | |
| *** mtaylor has quit IRC | 17:50 | |
| pvo | ianweller: did you ever talk to rackerhacker? | 17:51 |
| ianweller | pvo: about what? i chatted with him a bit on irc | 17:51 |
| rackerhacker | yeah, we met up in #fedora-cloud | 17:51 |
| *** kashyapc has quit IRC | 17:51 | |
| pvo | ianweller: re: fedora packaging | 17:51 |
| ianweller | ok. :) | 17:51 |
| pvo | righton | 17:52 |
| rackerhacker | i think i'm still too green on fedora packaging to assist ianweller | 17:52 |
| *** kashyapc has joined #openstack | 17:52 | |
| ianweller | rackerhacker: nah | 17:52 |
| pvo | rackerhacker: you're more fleshy-pink | 17:52 |
| rackerhacker | orly? | 17:52 |
| ianweller | rackerhacker: you got anything lying around you wanna get in fedora? | 17:52 |
| * rackerhacker looks at his ugly glusterfs spec file on his desktop | 17:52 | |
| ianweller | make it build, post it for review, we'll take a look :) | 17:53 |
| rackerhacker | will do | 17:53 |
| ianweller | rackerhacker: it's not a kernel module, is it? | 17:56 |
| rackerhacker | ianweller: nah, gluster really depends on fuse, and that's about it | 17:57 |
| ianweller | good. :) | 17:57 |
| rackerhacker | if it was a kernel module, i'd surely be in over my head | 17:57 |
| * rackerhacker wonders if pvo likes riding horses | 17:57 | |
| pvo | when I get a good ride, sure. | 17:57 |
| ianweller | yeah kmods are nutso in fedora | 17:58 |
| ianweller | the rule is: no, go away | 17:58 |
| *** wilson has joined #openstack | 18:03 | |
| *** wilson is now known as Guest42248 | 18:03 | |
| Glace | btw.. anyone have worked with zuora before? | 18:04 |
| *** mtaylor has joined #openstack | 18:09 | |
| *** ChanServ sets mode: +v mtaylor | 18:09 | |
| *** silassewell has joined #openstack | 18:12 | |
| *** metoikos has joined #openstack | 18:13 | |
| *** Guest42248 has left #openstack | 18:13 | |
| *** Guest42248 has joined #openstack | 18:14 | |
| *** jsmith-away is now known as jsmith | 18:16 | |
| *** colinnich has joined #openstack | 18:16 | |
| *** maximilien has quit IRC | 18:17 | |
| *** pcrews is now known as pcrews_arroz_con | 18:17 | |
| *** pcrews_arroz_con is now known as pcrews_lunch | 18:17 | |
| *** rdw has quit IRC | 18:17 | |
| *** maximilien has joined #openstack | 18:20 | |
| *** sophiap has joined #openstack | 18:21 | |
| *** blpiatt has quit IRC | 18:22 | |
| *** tonywolf has left #openstack | 18:27 | |
| *** pvo has quit IRC | 18:28 | |
| ianweller | rackerhacker: so when i'm sponsoring packagers, i usually ask them to submit a couple of packages and then do an informal review of another package, just as an FYI | 18:29 |
| rackerhacker | alrighty | 18:29 |
| ianweller | rackerhacker: (just so happens i've got some simple python crap i wrote that i need to get in fedora) :P | 18:30 |
| rackerhacker | i see what you did there | 18:30 |
| ianweller | rackerhacker: here it is HAVE FUN http://pypi.python.org/pypi/simplemediawiki/1.0 | 18:32 |
| * ianweller runs | 18:32 | |
| rackerhacker | haha, i'll have to take a look a bit later - trapped in something at the moment | 18:32 |
| ianweller | <nod> | 18:33 |
| *** ptremblett has quit IRC | 18:44 | |
| *** pvo has joined #openstack | 18:44 | |
| *** pvo has joined #openstack | 18:44 | |
| *** ChanServ sets mode: +v pvo | 18:44 | |
| *** sophiap has quit IRC | 18:54 | |
| *** sophiap has joined #openstack | 19:00 | |
| *** maximilien has quit IRC | 19:02 | |
| *** brd_from_italy has joined #openstack | 19:08 | |
| *** blpiatt has joined #openstack | 19:18 | |
| *** rdw has joined #openstack | 19:19 | |
| soren | For those who missed it: http://twitter.com/ubuntusoren/status/20303973457 | 19:23 |
| creiht | soren: btw, I would argue that twisted can't work with wsgi, not because it is async, but because of how twisted handles async | 19:24 |
| creiht | Eventlet is async, but still plays nicely with wsgi | 19:25 |
| soren | Pardon. Should have said "Twisted has an asynchronous API." | 19:26 |
| creiht | And I would argue with Twisted, you *have* to use a twisted specific implementation of that protocol, you don't see a lot of that with eventlet, because it already works with standard python implementations of those protocols | 19:27 |
| creiht | For example, you can use the standard httplib, urllib, etc from python, where as with twisted you have to use twisted's specific libraries | 19:28 |
| soren | Yes. | 19:28 |
| soren | You're assuming that I want to use standard python of those protocols. | 19:29 |
| soren | Err.. | 19:29 |
| soren | Insert "implementations" somewhere in that sentence. | 19:29 |
| soren | That's part of my argument. | 19:29 |
| creiht | That same argument could be used for twisted :) | 19:29 |
| soren | I don't want to use them. The Twisted ones are usually much better. | 19:29 |
| creiht | assuming I want to use the *one* implementation that is for twisted :) | 19:29 |
| soren | Sure. | 19:30 |
| hazmat | soren, nice re pkgs | 19:31 |
| creiht | So for example, being able to use the wealth of tools/libraries available for wsgi is a big win | 19:31 |
| hazmat | soren, your personal ppa? | 19:31 |
| soren | hazmat: No, Ubuntu proper. | 19:31 |
| hazmat | nice | 19:31 |
| soren | hazmat: It's been in my PPA for weeks :) | 19:31 |
| *** rajijoom has quit IRC | 19:32 | |
| soren | hazmat: Possibly. I've never really used any of them. | 19:32 |
| soren | Err.. | 19:32 |
| soren | hazmat: Sorry, that was for creiht. | 19:32 |
| soren | creiht: ^ | 19:32 |
| hazmat | no worries, i figured it out | 19:32 |
| soren | creiht: ..so I can't really speak for that. | 19:33 |
| creiht | soren: Of course you haven't because you have been bound by the shackles of twisted ;) | 19:33 |
| hazmat | yeah.. its a bit unclear that this stdlib monkeypatching is very robust.. it seems like eventlet itself realizes that and promotes particular ones to selective monkeypatching by inclusion into the green package | 19:33 |
| *** kashyapc has quit IRC | 19:33 | |
| soren | creiht: I've been doing Python (also web stuff) waaaay longer than I've been doing Twisted. | 19:33 |
| hazmat | for the context of nova, there are particular set of libraries of interest.. i don't think we need to generalize to the stdlib at large.. txredis/txamqp/txaws etc. vs. boto/pyredis/pyamqp | 19:34 |
| creiht | wsgi is one of the few really awesome things that has happened to python web development | 19:34 |
| hazmat | creiht, definitely but what's the point of saying python has a thousand wsgi implementations... your only going to use one of them.. | 19:35 |
| creiht | It isn't about the wsgi implementation | 19:35 |
| creiht | wsgi is a spec | 19:35 |
| hazmat | creiht, zope3 for a while used the twisted wsgi implementation as its std wsgi server.. but you can drop in and replace any of them.. i still fail to see the point your trying to make | 19:36 |
| hazmat | er. drop in a replacement ala mod_wsgi or what have you.. | 19:37 |
| hazmat | it effectively did a defer to thread for the wsgi app call from the twisted network code.. for any sync web framework or wsgi app (pretty much all of them), you'd be doing the same thing | 19:38 |
| creiht | hazmat: that is untrue | 19:38 |
| creiht | eventlet doesn't have to defer to threads to work with wsgi | 19:38 |
| creiht | The advantage with wsgi is that it makes available a large pool of useful libraries to choose from | 19:39 |
| hazmat | creiht, neither does twisted.. but if you want to reuse a sync web framework... | 19:39 |
| creiht | hazmat: That is only true if the framework specifically does something blocking | 19:39 |
| hazmat | eventlet monkey patching here is dangerous.. its not going to magically turn a web app into an async app.. that app might be doing some nasty rendering implementation | 19:39 |
| creiht | pylons can run on eventlet | 19:39 |
| creiht | hazmat: that can be true in the twisted world as well | 19:40 |
| hazmat | creiht, except those twisted apps are designed to be async | 19:40 |
| creiht | the monkey patching isn't meant to be magic, it is there as a tool | 19:40 |
| soren | Another things that seems to be ignored in this discussion is that Twisted is a big, mature framework with a lot of cool stuff in it. Eventlet fiddles with some knobs to make the stuff you're used to asynchronous. | 19:40 |
| creiht | hazmat: if you write bad code in a twisted app that blocks it still has the same problem | 19:40 |
| soren | Right, exactly. | 19:40 |
| hazmat | the implicit assumption that eventlet is going to make an existing wsgi app suddenly just work, is a bit of a fallacy without experience | 19:40 |
| soren | Eventlet is more of a tool. it's not a framework. | 19:40 |
| soren | I like frameworks :) | 19:41 |
| hazmat | just work async that is | 19:41 |
| creiht | Eventlet doesn't need to be a framework, because you can leverage any other framework that you want | 19:41 |
| soren | Right. | 19:41 |
| hazmat | its not that the app is bad, but its design/impleentation was sync.. assuming that it can be converted magically to async without audit is dangerous is what i'm trying to say. | 19:41 |
| soren | So the comparison is skewed. | 19:41 |
| soren | of course eventlet is going to have a smaller learning curve, because it's doesn't do much. | 19:42 |
| creiht | hazmat: I'm not trying to say that it is magic | 19:42 |
| *** ptremblett has joined #openstack | 19:42 | |
| soren | when was it ever bad to learn new shit? | 19:42 |
| creiht | All I can say is that it works very well for us in swift, pushing a *ton* of traffic and services | 19:42 |
| *** pcrews_lunch is now known as pcrews | 19:42 | |
| gholt | You guys are going in circles, heh. A won't convince B who can't convince A. Probably should just move on. | 19:42 |
| creiht | And made us be able to develop and debug issues much faster than with twisted | 19:42 |
| hazmat | creiht, someone mentioned swift was originally using twisted.. do you know the reason for the switch to eventlet there? | 19:43 |
| hazmat | i guess that answers that ;-) | 19:43 |
| creiht | :) | 19:43 |
| ianweller | woo, review of fedora's openstack-swift package posted. | 19:43 |
| ianweller | silassewell: ^^ | 19:44 |
| gholt | hazmat: Generically, we didn't need all the stuff twisted provides, and all that stuff kinda got in the way. | 19:44 |
| creiht | Either will work, I'm not trying to say that twisted doesn't | 19:44 |
| creiht | But what I can say from experience is that development with eventlet is much faster, easier to conceptualize, and easier to debug than twisted | 19:44 |
| soren | The fact that you did reworked the architecture and rewrote the whole thing and ended up with a better product doesn't mean you can attribute all of this to some change in technology. Learning from past experiences and starting over often leads to similar results, even if you don't change any of the libraries you used (or even if you switched the other way). | 19:45 |
| *** kashyapc has joined #openstack | 19:45 | |
| gholt | soren: True, which is why I think this line of chatter isn't going anywhere. :) | 19:45 |
| creiht | I'm not trying to say that our issues were due to twisted (they were not), but it made for a system that was harder to debug | 19:45 |
| soren | gholt: I agree. | 19:45 |
| creiht | And I've done twisted development long before I came to work on swift | 19:46 |
| soren | gholt: Let's write some frickin' code and compare it. | 19:46 |
| hazmat | +1 | 19:46 |
| creiht | soren: Indeed | 19:46 |
| gholt | Well, one could write a twisted version of swift and compare. :) | 19:47 |
| creiht | haha | 19:47 |
| eday | soren: It's mostly coming down to wanting to use WSGI at this point for me :) | 19:47 |
| *** codejunkie has joined #openstack | 19:47 | |
| soren | What I'd hate most of all is some sort of decree "now we've decided to use eventlet for these few things, so if you ever want to write anything else for nova, you don't get to use Twisted." Seriously, if you want to write a network server for almost any protocol other than http, you'd be in for a looooong day's work without Twisted. | 19:48 |
| creiht | soren: I would say that is untrue, but we are getting nowhere | 19:49 |
| soren | The python standard library isn't exactly a treasure trove of good server stuff. | 19:49 |
| creiht | But you aren't limited to the standard library | 19:49 |
| gholt | Ah, I hadn't heard that decree. Seems like it should be based on a per merge request basis. | 19:49 |
| soren | or bad server stuff for that matter. | 19:49 |
| _0x44 | I don't know that that's true. Jared Kuolt and _cerberus_ wrote an eventlet BERT RPC server in something like fifteen minutes and 12 of those minutes were making coffee. | 19:49 |
| creiht | hah | 19:50 |
| soren | gholt: No such decree has been made. Yet. | 19:50 |
| eday | soren: I agree, I think each app needs to be evaluated differently. For example, I think the API servers should be WSGI based, which impies Eventlet, and the back-end workers probably just thread based since we're not dealing with high concurrency or web-style apps (just using the AMQP libs). If some component comes up that would be best done in twisted (high concurrency, certain protocol support, ...), I'd see no reason not to choose it | 19:52 |
| gholt | ssh was a lot easier in twisted, as I recall. | 19:53 |
| * gholt is just thinking of examples... | 19:53 | |
| creiht | hehe | 19:53 |
| soren | eday: That would make me much more amenable in this discussion. It's being made out to be the ultimate eventlet vs. twisted standoff. Winner takes all. | 19:53 |
| creiht | I have no doubt that some things are actually better in twisted, but not all things | 19:54 |
| *** niemeyer has joined #openstack | 19:54 | |
| * creiht hides | 19:54 | |
| eday | gholt: yeah, twisted conch is pretty good | 19:54 |
| * soren likes the Manhole thing for debugging. | 19:54 | |
| creiht | soren: Eventlet has that | 19:54 |
| creiht | or something very similar | 19:54 |
| soren | How does Eventlet have that? | 19:54 |
| eday | soren: I guess I've been thinking this was 'twisted vs eventlet for API', it what to switch to from tornado :) | 19:54 |
| creiht | soren: http://eventlet.net/doc/modules/backdoor.html | 19:55 |
| soren | That surprises me. Why is that sort of thing Eventlet's job? | 19:56 |
| *** al-maisan is now known as almaisan-away | 19:56 | |
| * comstud prefers hurricain | 19:56 | |
| redbo | manhole, backdoor... I'm noticing a trend. | 19:57 |
| comstud | hehe | 19:57 |
| creiht | soren: Why would it not? | 19:57 |
| gholt | It can be argued wsgi isn't part of eventlet's core job, etc. It'll grow, maybe not to twisted size, but it'll keep growing a bit here and there. | 19:57 |
| *** gundlach has quit IRC | 19:57 | |
| *** gundlach has joined #openstack | 19:58 | |
| creiht | Several of the original devs came from twisted core contributors | 19:58 |
| eday | soren: what surprises you exactly? | 19:58 |
| soren | creiht: I understand Evenlet's job description to be: Play around with stuff until it acts asynchronously, but people can't tell. | 19:58 |
| creiht | soren: and yes I agree that type of functionality can be very useful | 19:58 |
| creiht | soren: eventlet isn't a play toy | 19:59 |
| soren | creiht: I just don't see how "also, implement some sort of server that lets people log into an interactive python shell to poke around with stuff" falls under that job description. | 19:59 |
| creiht | It is used to backend a lot of Second Life's services at very large scale, and is used in swift at very large scale | 20:00 |
| soren | Sure, no argument there. | 20:00 |
| creiht | soren: It provides tools to create async network servers and clients | 20:00 |
| soren | I've just misunderstood Eventlet's scope, apparantly. | 20:00 |
| soren | Yes. | 20:00 |
| creiht | why would it not want to provide tools to help you debug those? | 20:00 |
| soren | creiht: Tools. | 20:00 |
| soren | creiht: Not the actual servers and clients. | 20:00 |
| soren | creiht: Or so I thought. | 20:00 |
| soren | creiht: Again, apparantly I misunderstood Eventlet's scope. | 20:01 |
| soren | I wonder what else it does that I don't know about.. | 20:01 |
| creiht | soren: http://eventlet.net/doc/ | 20:01 |
| creiht | :) | 20:02 |
| soren | Ok, apparantly nothing. I had just missed the backdoor thing. | 20:02 |
| eday | soren: Thinking only in terms of the API servers, do you think choosing to follow WSGI, and for now using Eventlet would be bad for any reason? This gives use the option to use some other WSGI provider in the future if Eventlet doesn't work out. With twisted.web we're kind of locked in (which may be fine too, but options are always nice) | 20:02 |
| creiht | hah | 20:02 |
| soren | eday: I've never really used WSGI myself. Sure, I've used mod_wsgi to put stuff like moin on the web, but I've never fiddled with middleware and all that jazz, so I have no clue what we'd be missing if we couldn't do WSGI. | 20:04 |
| soren | eday: WSGI is a good idea. I'm not opposed to standards :) | 20:05 |
| eday | soren: ahh, ok. WSGI gives us a good way to layer middleware and request chains with pretty small overhead. it's just passing callables and iteraters for results | 20:05 |
| *** jaypipes has joined #openstack | 20:05 | |
| eday | soren: here are some examples with wsgi middleware: http://wsgi.org/wsgi/Middleware_and_Utilities | 20:06 |
| jaypipes | friggin plane delayed an hour... | 20:07 |
| soren | eday: I'm all for succinctness. If Eventlet+WSGI gives us that for the API server, that's great. Really. My real concern is that going with Eventlet for the API server bars me from using Twisted even if I could do something in 10 lines of Twisted but it would take 500 lines without it. | 20:09 |
| soren | 10 lines of Python is almost invariably going to be easier to maintain the 500 lines of Python. Twisted or not. | 20:10 |
| eday | soren: I apologize if anything I've said implies that, I've mainly been talking about the API servers | 20:10 |
| creiht | I'm failing to see why using eventlet is going to suddenly require you to write 500 lines of code instead of 10 | 20:11 |
| creiht | The entirety of core swift code it like 6k lines. I believe that is quite a statement when you consider what it does | 20:11 |
| eday | creiht: perhaps a protocol driver that exists for twisted but doesn't exist/work well in eventlet | 20:11 |
| eday | creiht: ie, something with sockets in C where eventlet can't monkey patch or something | 20:12 |
| soren | creiht: Anything that Twisted spends thousands of lines of code doing for me isn't going to magically do itself just because I use Eventlet. Also note, I didn't mention eventlet at all :) | 20:13 |
| creiht | And all of this is very hypothetical | 20:13 |
| soren | creiht: It's not. SSH server. There. | 20:13 |
| gholt | Hehe, good lord. I'm tired. | 20:13 |
| eday | soren: having said all that, for things like the compute and volume workers which are simple queue consumers without lots of connections/concurrency I think twisted is a bit overkill. multi-threaded or multi-process would probably be easier than dealing with the async issues Justin is having :) | 20:14 |
| * creiht sighs | 20:14 | |
| creiht | We need an SSH server in compute custom written? | 20:14 |
| soren | No! | 20:14 |
| soren | That's the point. | 20:14 |
| soren | i don't want to write custom shit that someone else has already written. | 20:14 |
| creiht | soren: That isn't what I was asking | 20:15 |
| *** sophiap has quit IRC | 20:15 | |
| soren | No, we probably don't need an SSH server in Nova. Maybe we do, but probably not. | 20:15 |
| creiht | I was asking if we have a need in compute to write a service that speaks ssh? | 20:15 |
| creiht | k | 20:15 |
| soren | I was responding to: | 20:15 |
| soren | 20:11 <+creiht> I'm failing to see why using eventlet is going to suddenly require you to write 500 lines of code instead of 10 | 20:15 |
| soren | Twisted has an SSH server that I can whip up with very few lines of code. Having to do the same without Twisted would be a lot of work. | 20:16 |
| eday | SO, I think we can all agree there are different tools for different jobs. Sometimes there are great tools for both jobs, and it's really a matter of preference. We can treat each case individually as it comes up, generalized statments are always going to lead to disagreements :) | 20:16 |
| soren | I used the SSh server as an example, because someone (you?) mentioned it not too long ago, so it was a common reference point. | 20:16 |
| soren | I can wade through the list of other protocols Twisted has to find other examples, but proving existence really only takes on example. | 20:17 |
| creiht | I just mentioned it is better, I have no doubt | 20:17 |
| soren | Would you be happier if I had said 10 lines vs. 50 lines? | 20:17 |
| creiht | There are pure python implementations for a large number of the same protocols, and those will work with eventlet | 20:18 |
| soren | I'm not saying there isn't. | 20:18 |
| soren | I'm just saying that if something is much, much simpler to express using Twisted, I'd hate to not be able to use it. Examples exist. Same for the other way around. | 20:19 |
| creiht | Want to use redis? Use the python implementation created for it, not another rewrite of it just so it will work with twisted | 20:19 |
| soren | If you can do something in 10 lines without Twisted, but Twisted makes it hella complicated, I prefer the non-Twisted version. | 20:19 |
| creiht | I would argue the opposite, that I can write shorter clearer code with eventlet | 20:19 |
| gholt | creiht: You're physically near me. I wonder if there's some way I can get you to quit perpetuating this conversation? | 20:20 |
| creiht | :) | 20:20 |
| creiht | haha | 20:20 |
| soren | creiht: and I'm agreeing! | 20:20 |
| * creiht digresses | 20:20 | |
| eday | yay | 20:20 |
| soren | creiht: Some things are easier and more straightforward to express /with/ Twisted. Others are easier and more straightforward without Twisted. Sometimes by orders of magnitude. | 20:21 |
| gundlach | soren: i picked about the worst time to stop paying attention to IRC, and just caught up with the conversation :) i think that my email suggesting that we "agree on eventlet" sounded like i was trying to hand down a decision from on high, which i'm not (i'm not on high, for one thing.) | 20:21 |
| gundlach | i agree very much with what you and eday said earlier: that if something is way easier in twisted, and it's painful in eventlet, then let's do it in twisted. | 20:21 |
| eday | gundlach: I think we can say "we've decided on WSGI for the API, and will use EVentlet for now to power the WSGI apps" | 20:22 |
| soren | The discussion just started with mtaylor suggesting we should standardise in OpenStack on one of them. | 20:22 |
| soren | AFAIK, at least. | 20:22 |
| gundlach | soren: ah, i must have missed that. | 20:22 |
| gundlach | eday: i agree with that (though i'd love it if someone tried a twisted, non-wsgi api version to compare it to.) | 20:23 |
| eday | gundlach: for the other workers, that's a different discussion (already happening). public web frontend != queue consumer with fixed number of workers | 20:23 |
| eday | soren: so we can all point and yell at mtaylor? | 20:23 |
| * eday punches mtaylor | 20:23 | |
| gundlach | eday: yep, i like justin's suggestion of simple threads in the backend -- basically, i like whatever is simpliest :) soren, if it turns out that some piece of arch is much easier in twisted, i'm all for that too. | 20:24 |
| eday | gundlach: yeah, or event multi-process like vish and josh were suggesting (due to signals) | 20:24 |
| gundlach | eday: s/event/even/ ? | 20:25 |
| eday | gundlach: something that was missed (and I think due to me starting the original thread with just "Eventlet vs Twisted") is that I was talking about *API* only, ie, what to replace tornado with | 20:25 |
| eday | gundlach: err, yeah, *even* | 20:26 |
| gundlach | got it | 20:27 |
| soren | creiht: Swift clearly demonstrates that Eventlet is solid. I'm not arguing it isn't. I'm also not arguing that you guys are bad Twisted coders because you couldn't do this as elegently using Twisted. :) I'm just saying the Twisted also has merit and in some cases, one approach just happens to be expressable more elegantly using one or the other and in those cases, I'd like the option of choosing the elegant solution. | 20:27 |
| creiht | soren: I think I'm the same boat, just making sure people just don't just automatically dismiss eventlet without taking a look at it | 20:29 |
| jaypipes | eday, gundlach: hehe, event multi-process :) a whole. nuther. level. | 20:29 |
| eday | jaypipes: it actually is event-driven multi-process, queue events from rabbit :) | 20:30 |
| jaypipes | eday: I know :) | 20:30 |
| jaypipes | eday: I was just making a silly joke. | 20:30 |
| eday | jaypipes: not funny. | 20:30 |
| gundlach | jaypipes: i was very glad it was a typo | 20:31 |
| eday | jaypipes: :) so, how was the heat? | 20:31 |
| jaypipes | eday: awful. | 20:31 |
| soren | creiht: My primary argument has been similar: Don't choose eventlet without looking at it. :) This is really not the sort of decision to make based on hearsay. | 20:31 |
| jaypipes | eday: though I hear it was over 100 in Columbus today.. :( | 20:31 |
| creiht | Though I'm making my argument from actually having built large systems in both twisted and eventlet ;) | 20:32 |
| eday | jaypipes: well, you were in florida in august. vacation good otherwise? | 20:32 |
| creiht | soren: Give it a taste, you might find you like it :) | 20:32 |
| jaypipes | eday: ya | 20:32 |
| creiht | The apple is shiney | 20:32 |
| redbo | I just monkey patched eventlet into my silly cloudfiles-sftp server based on paramiko and it worked fine. | 20:32 |
| gholt | redbo: Yeah, when I did that last I discovered paramiko had some nasty bugs. :) But yeah, I'd rather do that. | 20:33 |
| redbo | well, worked fine as best I could test in 2 minutes :) | 20:34 |
| gholt | Then somebody finally filled me in on dsh and I quit worrying about that stuff, heh. | 20:34 |
| creiht | haha | 20:34 |
| gholt | It'd be fun to help the paramiko folks out in fixing those bugs, but I got sidetracked -- actually, I got lazy, I admit it. | 20:35 |
| mtaylor | soren, eday wait! you aren't trying to blame the twisted/eventlet conversation on _me_ ? | 20:40 |
| *** BartVB has quit IRC | 20:40 | |
| soren | mtaylor: I am :) | 20:40 |
| * mtaylor has been purposefully staying way the hell away from that one | 20:40 | |
| * soren attempts to find evidence | 20:40 | |
| mtaylor | promise. wasn't me | 20:40 |
| * mtaylor does not like Twisted - but would _NEVER_ _EVER_ start a thread insinuating that perhaps it not be used, because twisted fans are usually quite vehement in their love of twisted | 20:41 | |
| *** mmalgeri1 has quit IRC | 20:41 | |
| soren | Darn it, apparantly the discussion was sparked IRL. | 20:41 |
| mtaylor | :) | 20:41 |
| eday | soren: I'll back you up, he said it. mtaylor's been a twisted hater since the day I met him | 20:43 |
| mtaylor | eday: yes... but I have not vented any of that in this forum | 20:43 |
| mtaylor | eday: don't make me take a train down to portland and cough on you with my step throat now | 20:44 |
| eday | mtaylor: haha | 20:44 |
| *** justinsheehy has quit IRC | 20:47 | |
| *** ctennis has quit IRC | 20:51 | |
| *** ctennis has joined #openstack | 21:02 | |
| *** ctennis has joined #openstack | 21:02 | |
| *** kashyapc has quit IRC | 21:04 | |
| *** RobertLJ has quit IRC | 21:04 | |
| vish1 | network refactor appears to be complete | 21:09 |
| vish1 | I'm going to try and make some time next week to improve our testing | 21:09 |
| vish1 | that would have been a lot easier if we had a good way of starting up various workers and making sure the api calls worked | 21:09 |
| eday | vish1: cool :) | 21:10 |
| *** mmalgeri1 has joined #openstack | 21:13 | |
| *** sirp1 has quit IRC | 21:18 | |
| *** jdarcy has quit IRC | 21:19 | |
| *** jaypipes has quit IRC | 21:25 | |
| *** sirp1 has joined #openstack | 21:25 | |
| mtaylor | vish1: once jaypipes gets his lazy butt back from vacation, he's going to help me set up an integration testing environment too | 21:26 |
| mtaylor | vish1: potentially not a useful for in tree "python setup.py test" sorts of things - but some "hey, let's spin up 20 servers and hammer the crap out of them" | 21:27 |
| *** jonesy_ has quit IRC | 21:27 | |
| *** justinsheehy has joined #openstack | 21:28 | |
| *** jonesy_ has joined #openstack | 21:30 | |
| *** silassewell has quit IRC | 21:30 | |
| eday | gundlach: you about still? | 21:35 |
| gundlach | yep | 21:35 |
| gundlach | you? | 21:35 |
| gundlach | guess not ;) | 21:35 |
| mtaylor | eday is never actually here. he's a ghost | 21:36 |
| *** rtyler has joined #openstack | 21:36 | |
| rtyler | hey creiht, so what's that you were saying about how awesome Tornado was? | 21:36 |
| * rtyler ducks | 21:36 | |
| eday | hehe | 21:36 |
| eday | gundlach: http://pastebin.com/2wg5LWEs what I'm playing with, for layer management | 21:37 |
| * creiht smacks rtyler | 21:39 | |
| creiht | again | 21:39 |
| creiht | :) | 21:39 |
| eday | gundlach: thinking how fancy we need to make router classes. Should we just do them programatically? Or full on system where you declare routes and make decisions on URL, method, ...? | 21:39 |
| *** blpiatt has quit IRC | 21:40 | |
| *** jonesy_ has quit IRC | 21:40 | |
| gundlach | had to go deal w/ puppy emergencies -- back | 21:41 |
| gundlach | (my dog just learned how to jump onto the dining room table) | 21:41 |
| eday | gundlach: awesome, did you give it a treat!? | 21:42 |
| creiht | eday: How complicated are your paths going to be? (for routing) | 21:43 |
| gundlach | heh, no, but i did run outside and make him go down the stairs (also a new trick) and use the bathroom on command and come back upstairs. hoping that obedience would make him forget about the dining room table. | 21:43 |
| gundlach | it didn't. | 21:43 |
| eday | creiht: not very, so I'm thinking we keep it simple and don't make a big deal of it | 21:43 |
| creiht | eday: then yeah keep it simple (basically what we did for swift) | 21:43 |
| eday | creiht: it's mainly a way to neatly separate different tasks in the requst chain (rate limiting, auth, caching, ...) | 21:43 |
| creiht | I was just going to say if you were going to have complicated routing, then it would be worth looking at some of the available request dispatch libs that are available | 21:44 |
| eday | creiht: mainly want to ability to keep them separate so we could insert HTTP proxy layers easily if needed | 21:45 |
| creiht | yeah | 21:46 |
| *** silassewell has joined #openstack | 21:46 | |
| *** brd_from_italy has quit IRC | 21:46 | |
| *** dendrobates is now known as dendro-afk | 21:47 | |
| gundlach | eday: we could reuse routes (http://routes.groovie.org/) -- behaves like ruby's routes and does this for us already. | 21:47 |
| creiht | gundlach: That's kinda what I'm getting at, but if you have very few paths, and they aren't complicated, then it isn't worth the dependency | 21:48 |
| gundlach | creiht: agreed. though there are about 25 paths in the RS API | 21:48 |
| gundlach | not sure which side of the scales that falls on | 21:49 |
| creiht | then yeah that would probably be worth it | 21:49 |
| creiht | In swift, almost everything revolves around a path that looks more or less like: /account/container/object | 21:49 |
| creiht | so it wasn't worth the extra dependecy for us | 21:49 |
| gundlach | in nova, it's /entity/id/[sometimes subentities] | 21:50 |
| *** tr3buchet has quit IRC | 21:50 | |
| gundlach | yeah, i could see rolling your own when the url pattern is so regular. | 21:50 |
| eday | gundlach: ahh, thats kind of cool. in ubuntu already too, not just pypi :) | 21:51 |
| gundlach | eday: be aware that there exist some frameworks that declaratively stack our WSGI apps for us. i jsut don't know their names offhand :) | 21:51 |
| gundlach | eday: you mean available via apt-get instead of pip install? | 21:51 |
| creiht | yeah routes should already be packaged | 21:51 |
| gundlach | i'm a packaging n00b -- why is apt-get preferable for python packages instead of pip install? | 21:52 |
| gundlach | because we have other dependencies that require apt-get so it's simpler to use one packaging system? | 21:52 |
| creiht | ops people like it a lot more :) | 21:53 |
| gundlach | fair enough | 21:53 |
| *** Phil_FL has quit IRC | 21:53 | |
| * creiht has yet to find an ops person that would not use the os packaging system | 21:53 | |
| creiht | (in production) | 21:53 |
| creiht | not that I blame them either | 21:53 |
| gundlach | eday: django lets you stack your wsgi apps via a config; turbogears 2.0 does the same; basically any wsgi-compliant web framework will give you that. so we may be able to reuse that piece from an existing project rather than trying to manually hook these together (like your code does) or write our own declarative syntax | 21:54 |
| eday | gundlach: automatic updates (apt vs pypi) | 21:55 |
| *** colinnich has quit IRC | 21:55 | |
| eday | gundlach: ahh, ok. hmm. looking at turbogears | 21:55 |
| gundlach | eday: sold | 21:55 |
| *** ptremblett has quit IRC | 21:56 | |
| *** ptremblett has joined #openstack | 21:56 | |
| *** jsmith is now known as jsmith-away | 21:56 | |
| mtaylor | gundlach: oh yeah - as an admin I'm 100% apt-get | 21:56 |
| mtaylor | sorry - realize that's been answered already... but thought I'd toss in anyway :) | 21:57 |
| gundlach | so people still use pip/easy_install/etc just for convenience? or for non-unix systems? | 21:57 |
| gundlach | or maybe to be able to suck modules into a virtualenv for easy distribution to other machines without requiring apt-get to run? | 21:57 |
| eday | gundlach: hmm, so with django (which I've used before) and turbogears (which I;ve not), it you get a lot more than just the WSGI glue... want to make sure we keep things simple | 21:58 |
| eday | gundlach: I use easy_install when it's not in apt still, I just prefer apt | 21:58 |
| gundlach | eday: yeah -- i am not aware if either of those examples actually have cleanly separated their wsgi stacking out, i just know it's such a common thing to have to solve that i hope someone *has*. | 21:58 |
| mtaylor | gundlach: _devs_ like easy_install/pip | 21:58 |
| gundlach | if not, we can end up writing it ourselves. | 21:58 |
| mtaylor | gundlach: devs tend to be very language-centric and just focused on their coding tasks, so language specific things like easy_install or ruby gems mean they're just always in their language all the time | 21:59 |
| gundlach | mtaylor: i admit i (mostly dev, some sysadmin) prefer pip, because i feel like it's doing less magical stuff than apt-get. it fetches the .tar.gz from a website, it puts the files into my site-packages, it's done. | 21:59 |
| mtaylor | but as soon as anything happens outside of the bubble, it all goes to shit | 22:00 |
| gundlach | mtaylor: :) | 22:00 |
| mtaylor | gundlach: heh | 22:00 |
| mtaylor | easy_install's biggest hurdle is dealing with things that depend on c libraries :) | 22:00 |
| mtaylor | properly | 22:00 |
| eday | gundlach: I think the routes + simple WSGI class layers should be simple enough. THe code I pasted is already pretty minimal minues the path selection | 22:00 |
| mtaylor | it can install stuff ... but damn if you don't have the library already installed you're fux0red | 22:01 |
| gundlach | eday: i think your current approach won't allow us to pull a piece into its own app, because you've hardcoded e.g. Auth(Debug(Router())) | 22:01 |
| gundlach | maybe that part you were planning on making more flexible? | 22:01 |
| gundlach | wow, that sounded yiddish | 22:01 |
| mtaylor | nice | 22:01 |
| eday | gundlach: so, instead, you cp the 1 line binary and split: Auth(Proxy()) and the other with just Router() | 22:03 |
| eday | gundlach: that's still simple enough to need a big framework thingy to do | 22:03 |
| gundlach | eday: ok, that will work for .001 release, fair enough. | 22:03 |
| *** niemeyer has quit IRC | 22:04 | |
| eday | gundlach: yeah.. if we start swapping these types of modules around a lot, we can find some other config glue | 22:04 |
| eday | gundlach: but I don't think it will change any of our WSGI callables, just the binaries blueing them together | 22:05 |
| gundlach | eday: absolutely right | 22:05 |
| eday | err, gluing | 22:06 |
| *** rnewson has quit IRC | 22:08 | |
| * eday plays with routes | 22:09 | |
| *** ptremblett has quit IRC | 22:13 | |
| *** ptremblett has joined #openstack | 22:13 | |
| uvirtbot | New bug: #613667 in swift "Bytes In and Bytes Out are incorrectly logged " [Undecided,Confirmed] https://launchpad.net/bugs/613667 | 22:15 |
| *** jonesy_ has joined #openstack | 22:20 | |
| *** dendro-afk is now known as dendrobates | 22:21 | |
| *** allsystemsarego has quit IRC | 22:28 | |
| creiht | gundlach, eday: btw, python paste has stuff for configuring arbitrary wsgi layers | 22:31 |
| gundlach | creiht: thanks. i think i read that in the tutorial i linked to earlier, now that you mention it | 22:31 |
| creiht | more specifically: http://pythonpaste.org/deploy/ | 22:31 |
| creiht | cool | 22:31 |
| gundlach | yep, that was it -- Paste Deploy | 22:32 |
| gundlach | though I was turned off a bit because it required that you wrap your wsgi app in their style of thing in order to make it into a deployable class (or something). aka, you had to play by their rules instead of just specifying the classnames of your wsgi apps. | 22:32 |
| gundlach | But the Paste guys are smart so I imagine there was a reason :) | 22:32 |
| gundlach | http://archimedeanco.com/wsgi-tutorial/ | 22:33 |
| creiht | hah... yeah some people don't like that style, but at the very least it is a good source of ideas | 22:33 |
| creiht | I'm not super fond of it either, but the times I have used it, I opted to do so instead of writing something myself :) | 22:34 |
| eday | gundlach: http://pastebin.org/448065 my simple app converted to use routes (with generic route class) | 22:35 |
| *** adjohn has quit IRC | 22:35 | |
| gundlach | did it feel simpler? | 22:35 |
| *** adjohn has joined #openstack | 22:36 | |
| *** Podilarius has left #openstack | 22:38 | |
| eday | gundlach: routes? yeah, with the base class I made | 22:38 |
| eday | gundlach: I guess we should insert the matched dict into environ as well, since it's easier to access them already parsed | 22:39 |
| gundlach | matched dict? not sure what you're referring to | 22:40 |
| gundlach | Router.map? | 22:40 |
| gundlach | anyway, glad that it worked out, and way to integrate it so quickly :) | 22:42 |
| eday | gundlach: http://pastebin.org/448072 notice file is picked out of matched dict now | 22:43 |
| eday | gundlach: the Router layer injects the map result into environ | 22:43 |
| *** jonesy_ has quit IRC | 22:53 | |
| *** mray has quit IRC | 22:54 | |
| mtaylor | creiht: speaking of paste - I haven't fogotten about you - just been sick :) | 22:55 |
| creiht | mtaylor: no worries | 22:57 |
| *** dmd17 has joined #openstack | 22:57 | |
| *** chilts has joined #openstack | 22:57 | |
| vish1 | sigh, i just realized there is something else that i need to move into network | 22:59 |
| eday | vish1: know if todd is around? Not seen him on IRC today | 23:00 |
| vish1 | eday: not sure, he's in kentucky | 23:00 |
| vish1 | :) | 23:00 |
| eday | vish1: ahh. where you based out of? | 23:01 |
| *** ptremblett has quit IRC | 23:06 | |
| *** ptremblett has joined #openstack | 23:06 | |
| *** chilts has quit IRC | 23:07 | |
| vish1 | eday: silicon valley | 23:07 |
| *** chilts has joined #openstack | 23:09 | |
| *** metoikos has quit IRC | 23:22 | |
| *** rtyler has left #openstack | 23:28 | |
| *** miclorb_ has joined #openstack | 23:28 | |
| *** metoikos has joined #openstack | 23:29 | |
| *** aliguori has quit IRC | 23:30 | |
| *** justinsb has joined #openstack | 23:31 | |
| *** ptremblett has quit IRC | 23:32 | |
| *** ptremblett has joined #openstack | 23:32 | |
| *** lbieber has quit IRC | 23:33 | |
| *** syah has joined #openstack | 23:34 | |
| *** coyotte508 has joined #openstack | 23:35 | |
| *** coyotte508 has left #openstack | 23:35 | |
| *** pcrews has quit IRC | 23:37 | |
| *** pcrews has joined #openstack | 23:43 | |
| *** abecc has quit IRC | 23:44 | |
| *** aliguori has joined #openstack | 23:46 | |
| *** pcrews has quit IRC | 23:56 | |
| *** adrian_otto has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!