| *** vish1-mostly-afk is now known as vish1-not-afk-an | 00:08 | |
| *** vish1-not-afk-an is now known as vish1 | 00:08 | |
| *** gustavomzw has left #openstack | 00:10 | |
| * vish1 needs to find a new nick | 00:12 | |
| *** adjohn has joined #openstack | 00:15 | |
| *** vish1 is now known as vi | 00:23 | |
| *** vi is now known as vishy | 00:24 | |
| *** carolinad28 has quit IRC | 00:30 | |
| *** lbieber has joined #openstack | 00:30 | |
| *** miclorb has quit IRC | 00:34 | |
| *** dendro-afk is now known as dendrobates | 00:38 | |
| *** jdmaturen has quit IRC | 00:39 | |
| eday | ahh, libeatmydata for swift unittests :) | 00:47 |
|---|---|---|
| *** lbieber has quit IRC | 00:52 | |
| *** vishy is now known as vishy-afk | 00:53 | |
| *** littleidea has joined #openstack | 00:58 | |
| *** miclorb_ has joined #openstack | 01:07 | |
| *** dendrobates is now known as dendro-afk | 01:09 | |
| *** mrayzenoss has joined #openstack | 01:21 | |
| *** mrayzenoss is now known as mray | 01:21 | |
| *** mray has joined #openstack | 01:21 | |
| *** mray has quit IRC | 01:27 | |
| creiht | eday: ? | 01:34 |
| *** dendro-afk is now known as dendrobates | 01:36 | |
| eday | creiht: yo | 01:38 |
| creiht | libeatmydata? | 01:38 |
| eday | creiht: oh, yeah. it disables sync() and friends so tests that hit disk run faster. swift unittests on my laptop from 30+s to 5s | 01:38 |
| eday | creiht: https://launchpad.net/libeatmydata | 01:39 |
| creiht | ah | 01:39 |
| creiht | interesting | 01:39 |
| creiht | that's nifty | 01:40 |
| eday | yeah, made a world of difference when running the drizzle test suite | 01:40 |
| eday | creiht: what do you think of webob? does it give you a whole lot? (reading about it now) | 01:47 |
| creiht | It is handy | 01:50 |
| creiht | It is a lot easier than parsing the environ yourself, and if you don't use it (or something like it) you are going to end up implementing something similar anyways :) | 01:50 |
| *** lbieber has joined #openstack | 01:52 | |
| eday | creiht: yeah, it's starting to look pretty useful :) | 01:52 |
| creiht | And it also handles some of the more esoteric things like range requests | 01:53 |
| creiht | and better now that notmyname's patches are in there :) | 01:54 |
| *** dendrobates is now known as dendro-afk | 01:54 | |
| eday | creiht: does it handling streaming on the request and response objects efficiently? (don't need to buffer) | 01:54 |
| creiht | It gives you a file like object so it is up to you | 01:55 |
| creiht | We stream in chunks | 01:55 |
| eday | cool | 01:57 |
| creiht | It works pretty well | 01:57 |
| *** lbieber has quit IRC | 02:05 | |
| *** jdmaturen has joined #openstack | 02:09 | |
| *** gnrfan has quit IRC | 02:14 | |
| *** jamiew has quit IRC | 02:15 | |
| eday | creiht: is there any reason you didn't use more paste components? | 02:15 |
| creiht | 1. We were tyring to keep dependencies down | 02:17 |
| creiht | 2. Most of what we do on that side of things is pretty simple | 02:17 |
| gundlach | eday: i should have been reading this chat -- i've just converted your sample code to webob and made the middlewares dead simple :) | 02:18 |
| eday | gundlach: cool :) | 02:21 |
| *** miclorb_ has quit IRC | 02:24 | |
| *** randybias has quit IRC | 02:26 | |
| *** randybias has joined #openstack | 02:28 | |
| *** miclorb has joined #openstack | 02:32 | |
| *** lbieber has joined #openstack | 02:38 | |
| *** hornbeck has quit IRC | 02:40 | |
| *** rajijoom has joined #openstack | 02:46 | |
| gundlach | eday: http://bazaar.launchpad.net/~gundlach/nova/api/annotate/head:/nova/endpoint/new_wsgi.py | 02:54 |
| gundlach | look in particular at how much simpler Auth() is -- you can add headers in any order, not worrying about wrapping start_response to pull it off | 02:55 |
| *** dendro-afk is now known as dendrobates | 02:57 | |
| *** spy has left #openstack | 03:10 | |
| *** spy has joined #openstack | 03:10 | |
| *** ChanServ sets mode: +v spy | 03:10 | |
| *** pcrews has quit IRC | 03:11 | |
| *** dendrobates is now known as dendro-afk | 03:14 | |
| *** _anm has joined #openstack | 03:19 | |
| _anm | any wiki.openstack.org admins around? | 03:19 |
| spy | _anm: yeah, what's up? | 03:20 |
| _anm | does wiki.openstack.org allow anonymous edits now? has it always? | 03:20 |
| spy | you can log in if you need to make edits, easy to make an account | 03:20 |
| spy | well i guess anon edits are also allowed | 03:21 |
| _anm | justinsb and myself spent 3 days writing the most complete "install from trunk" article to date, and it has gotten pretty trashed by anonymous contributers: | 03:21 |
| _anm | http://wiki.openstack.org/InstallFromSource?action=diff&rev1=14&rev2=15 | 03:21 |
| _anm | not maliciously, but VERY carelessly and now very inacuratly | 03:22 |
| *** dendro-afk is now known as dendrobates | 03:22 | |
| spy | the big dump of yum installs? | 03:22 |
| _anm | yeah | 03:23 |
| _anm | this page would in no way resukt in a runnable nova setup | 03:23 |
| _anm | s/resukt/result/ | 03:23 |
| _anm | just a bit frustrating, I'm just venting | 03:24 |
| spy | yeah, we left it open because we wanted anyone to contribute | 03:24 |
| _anm | since I know how much time i spent trying to get people up and running from source | 03:24 |
| _anm | ok, I'll go try to fix it up | 03:24 |
| spy | yeah, feel free to fix it up | 03:24 |
| spy | maybe copy all of those edits to another page if needed | 03:25 |
| _anm | i dont think a wiki login is too much to ask from a potential contributer | 03:25 |
| *** jbd has quit IRC | 03:25 | |
| spy | yeah, i agree, not hard to sign up either | 03:25 |
| spy | i'll ask dendrobates about it as i know he wanted to keep it very open | 03:25 |
| *** miclorb_ has joined #openstack | 03:26 | |
| _anm | ok | 03:27 |
| notmyname | how possible is it to lock pages down. for example, the approved contributors page also allows anon edits | 03:28 |
| spy | don't know, that would be ideal tho | 03:28 |
| spy | haven't played with moin moin too much | 03:29 |
| * mtaylor would fix - but no havey admin access to wiki | 03:30 | |
| spy | what's your user on the wiki? | 03:30 |
| spy | i'll add you too it too, was just about to add myself | 03:30 |
| _anm | yeah, I 'm not sure either, but I'll bet it's probably simple, most wiki's have a decent admin backend and moin moin is pretty mature and feature rich | 03:31 |
| _anm | spy: I'm NikMartin on the wiki | 03:31 |
| *** kashyapc has joined #openstack | 03:38 | |
| *** gundlach has quit IRC | 03:40 | |
| _anm | ok, i hear branch/merge branch merge all the time. I'm contributing docs to the wiki, not code, so what is my svn up command in bzr? my working copy of trunk is called "trunk" (how original) | 03:46 |
| _anm | i just want to grab trunk every day or two, try to buold and modify the wiki accordingly based on my latest successful attempt | 03:47 |
| _anm | NM, i just found a SWEET bzr for svn users guide | 03:48 |
| *** miclorb_ has quit IRC | 03:53 | |
| *** miclorb has joined #openstack | 04:01 | |
| *** hazmat has quit IRC | 04:11 | |
| mtaylor | _anm: bzr pull... but great! | 04:14 |
| _anm | ok, I did it the old fashioned way: rm -rf nova and fetched clean | 04:15 |
| _anm | :) | 04:15 |
| notmyname | updated the work in progress of the swift stats system https://code.launchpad.net/~notmyname/swift/stats_system/+merge/31924 | 04:17 |
| mtaylor | _anm: heh. | 04:18 |
| _anm | so, I enjoyed Rick on FLOSS this week ;) | 04:19 |
| notmyname | mtaylor: if I set the status for a merge proposal to "work in progress", does it cancel the merge request? | 04:19 |
| _anm | Randal was a a bit underwhelmed when he asked what exactly nova was | 04:20 |
| _anm | and I was like "that's the whole point"\ | 04:20 |
| notmyname | mtaylor: for example, the stats code isn't ready to be merged, but I've created the merge request as a place for people to see the diffs and make comments | 04:20 |
| *** nikmartin has joined #openstack | 04:25 | |
| vishy-afk | you made stats code? | 04:26 |
| vishy-afk | nice | 04:26 |
| *** nikmartin has left #openstack | 04:26 | |
| *** vishy-afk is now known as vishy | 04:26 | |
| *** vishy is now known as beesh | 04:26 | |
| beesh | hehe | 04:27 |
| *** beesh is now known as vishy | 04:27 | |
| *** almaisan-away is now known as al-maisan | 04:33 | |
| notmyname | not sure how useful the swift stats stuff will be for nova, but it some of the processing may be useful to both | 04:33 |
| vishy | notmyname: oh, it is for swift. :( | 04:33 |
| *** stewart has quit IRC | 04:38 | |
| *** rajijoom has quit IRC | 04:42 | |
| *** randybias has quit IRC | 04:49 | |
| *** miclorb has quit IRC | 04:51 | |
| *** lbieber has quit IRC | 04:55 | |
| *** littleidea has quit IRC | 04:56 | |
| *** al-maisan is now known as almaisan-away | 05:04 | |
| *** littleidea has joined #openstack | 05:17 | |
| *** RobertLJ has joined #openstack | 05:18 | |
| _anm | just got an unhandled error: http://pastebin.org/450233 | 05:26 |
| _anm | starting nova-volume | 05:26 |
| *** nettok has joined #openstack | 05:26 | |
| _anm | this is probably based on me previously starting nova-volume using /dev/loop0 vs /dev/loop3 | 05:27 |
| _anm | so it's probably not a 'real' error | 05:28 |
| *** allsystemsarego has joined #openstack | 05:40 | |
| *** jdmaturen has quit IRC | 05:41 | |
| *** _anm has left #openstack | 05:41 | |
| *** stewart has joined #openstack | 05:53 | |
| *** ctennis has quit IRC | 06:00 | |
| *** miclorb has joined #openstack | 06:02 | |
| *** ctennis has joined #openstack | 06:06 | |
| *** RobertLJ has quit IRC | 06:08 | |
| *** sirp1 has quit IRC | 06:09 | |
| *** BartVB has joined #openstack | 06:16 | |
| *** miclorb has quit IRC | 06:20 | |
| *** spackest has joined #openstack | 06:40 | |
| *** miclorb has joined #openstack | 06:49 | |
| *** rajijoom has joined #openstack | 06:57 | |
| *** spackest has quit IRC | 06:59 | |
| *** brd_from_italy has joined #openstack | 07:00 | |
| *** stewart has quit IRC | 07:24 | |
| *** miclorb has quit IRC | 07:32 | |
| *** miclorb_ has joined #openstack | 07:38 | |
| *** rajijoom has quit IRC | 07:48 | |
| *** miclorb_ has quit IRC | 07:53 | |
| *** rnewson has joined #openstack | 07:56 | |
| *** rnewson has joined #openstack | 07:56 | |
| *** miclorb has joined #openstack | 08:01 | |
| *** adjohn has quit IRC | 08:10 | |
| *** miclorb has quit IRC | 08:12 | |
| *** calavera has joined #openstack | 08:18 | |
| *** adjohn has joined #openstack | 08:32 | |
| *** rajijoom has joined #openstack | 08:33 | |
| *** adjohn has quit IRC | 08:36 | |
| *** rnewson has quit IRC | 08:37 | |
| *** rnewson has joined #openstack | 08:38 | |
| *** rnewson has joined #openstack | 08:38 | |
| *** sophiap has quit IRC | 08:39 | |
| *** nettok has left #openstack | 09:03 | |
| *** adjohn has joined #openstack | 10:03 | |
| *** littleidea has quit IRC | 10:10 | |
| *** rajijoom has quit IRC | 10:22 | |
| *** rajijoom has joined #openstack | 10:38 | |
| *** rajijoom has quit IRC | 10:47 | |
| *** Paul_Miller_ has joined #openstack | 10:49 | |
| *** Paul_Miller_ has quit IRC | 10:51 | |
| *** Paul_Miller has joined #openstack | 10:51 | |
| *** Paul_Miller_ has joined #openstack | 10:52 | |
| *** Paul_Miller has left #openstack | 10:53 | |
| *** Paul_Miller_ has quit IRC | 10:53 | |
| *** rajijoom has joined #openstack | 11:03 | |
| *** jonesy_ has quit IRC | 11:08 | |
| *** ctennis has quit IRC | 11:23 | |
| *** ctennis has joined #openstack | 11:35 | |
| *** ctennis has joined #openstack | 11:35 | |
| *** justinsheehy has quit IRC | 11:44 | |
| *** hazmat has joined #openstack | 11:44 | |
| *** justinsheehy has joined #openstack | 11:54 | |
| *** metoikos has quit IRC | 11:59 | |
| *** justinsheehy has quit IRC | 12:04 | |
| *** metoikos has joined #openstack | 12:17 | |
| *** gundlach has joined #openstack | 12:20 | |
| *** metoikos has quit IRC | 12:27 | |
| *** metoikos has joined #openstack | 12:28 | |
| *** justinsheehy has joined #openstack | 12:30 | |
| *** calavera has quit IRC | 12:34 | |
| zul | guys when i try to start nova-volume it compains about gflags.UnrecognizedFlagError: Unknown command line flag 'ca_path | 12:38 |
| *** sophiap has joined #openstack | 12:41 | |
| *** aliguori has joined #openstack | 12:47 | |
| *** jaypipes has quit IRC | 12:56 | |
| gundlach | zul: sorry, i haven't run nova-volume myself yet -- but more people should be here in a couple hours who could probably help :) | 13:14 |
| zul | gundlach: k ill just wait then | 13:14 |
| *** carolinad28 has joined #openstack | 13:15 | |
| *** jaypipes has joined #openstack | 13:18 | |
| *** jonesy_ has joined #openstack | 13:19 | |
| *** carolinad28 has quit IRC | 13:23 | |
| *** sophiap has quit IRC | 13:29 | |
| *** metoikos has quit IRC | 13:30 | |
| *** metoikos has joined #openstack | 13:31 | |
| *** sophiap has joined #openstack | 13:36 | |
| *** Podilarius has joined #openstack | 13:50 | |
| *** pvo has joined #openstack | 13:51 | |
| *** pvo has joined #openstack | 13:51 | |
| *** ChanServ sets mode: +v pvo | 13:51 | |
| jonesy | quick question I've been curious about: does xfs have some killer feature that made it a no-brainer for swift compared to ext3 or reiser or... ? | 13:53 |
| *** sophiap has quit IRC | 13:58 | |
| *** abecc has joined #openstack | 13:59 | |
| gholt | redbo can answer that better as he did most the testing, but xfs ended up faster for use cases. xattr support is the only special fs feature swift needs. | 13:59 |
| jonesy | thanks, gholt | 14:03 |
| *** sophiap has joined #openstack | 14:04 | |
| *** gustavomzw has joined #openstack | 14:16 | |
| *** jdarcy has joined #openstack | 14:19 | |
| *** mrayzenoss has joined #openstack | 14:28 | |
| *** mrayzenoss is now known as mray | 14:28 | |
| *** pcrews has joined #openstack | 14:31 | |
| *** JimCurry has joined #openstack | 14:34 | |
| *** JimCurry has left #openstack | 14:35 | |
| *** sirp1 has joined #openstack | 14:40 | |
| *** bretpiatt has joined #openstack | 14:40 | |
| *** bretpiatt is now known as blpiatt | 14:41 | |
| *** hornbeck has joined #openstack | 14:52 | |
| *** dendrobates is now known as dendro-afk | 14:56 | |
| *** dendro-afk is now known as dendrobates | 14:56 | |
| *** pvo_ has joined #openstack | 15:04 | |
| *** carolinad28 has joined #openstack | 15:08 | |
| comstud | zul- nova-volume only needs a --storage_dev= option | 15:14 |
| zul | oh ok | 15:14 |
| gundlach | zul: wherever you found the docs that claimed you needed the other option, would you edit them (if they're a wiki) or point us to them so someone with the keys can fix it? | 15:15 |
| *** littleidea has joined #openstack | 15:15 | |
| zul | gundlach: sure no problem | 15:16 |
| creiht | jonesy: XFS perfromed the best all around. This was even more evident as the filesystems grew. I think there is still a mode of running Ext4 that redbo wants to experiment with | 15:16 |
| gundlach | zul: ty | 15:16 |
| jonesy | thanks, creiht | 15:17 |
| * jaypipes yawns...more coffee needed.. | 15:17 | |
| creiht | Seems like when you fsync everything, it changes how filesystems perform quite a bit :) | 15:17 |
| jaypipes | creiht: no way! ;) | 15:17 |
| jonesy | haa | 15:17 |
| jonesy | fsync=off --> http://www.flickr.com/photos/gavinmroy/4638958958/ | 15:18 |
| creiht | It also seems like most filesystem benchmarks that you see online start with a clean filesystem and then write just a small amount of data | 15:19 |
| creiht | It is pretty amazing how much performance declines as the filesystem fills up | 15:19 |
| creiht | jonesy: hah! | 15:19 |
| redbo | I'd like to experiment with ext3/4 with journaled data and not fsyncing. That should keep the filesystem consistent if there's a failure, and we'd just rely on having stuff on multiple machines for durability. But I haven't been able to get much lab time lately to test this theory. | 15:20 |
| comstud | zul- take a look at http://wiki.openstack.org/InstallFromSource | 15:21 |
| comstud | you can see the options for the daemons there.. | 15:21 |
| comstud | if they seem different somewhere else, i'd be curious what they say | 15:22 |
| jonesy | redbo: interesting. That seems plausible. | 15:22 |
| jdarcy | Turning fsync off is not unreasonable, but turning barriers off would be. That can result not just in missing data but in a corrupt filesystem. | 15:25 |
| redbo | well, with xfs and no fsync, you can wind up with files that say they're there, but they just contain NULL characters. It isn't corrupt, but it isn't consistent either, and I haven't found any way to detect that short of auditing the whole drive. | 15:27 |
| jdarcy | The worst problem with ext4 IMO is not its normal performance or reliability (assuming proper configuration) but fsck time. | 15:27 |
| jdarcy | Some people near me did a lot of work to improve that, but AFAIK it's still worse than XFS. | 15:27 |
| creiht | jdarcy: yeah that is a good point | 15:27 |
| jdarcy | redbo: Look on the bright side, at least it's NUL and not "whatever data happened to be in those disk blocks from before" | 15:28 |
| jonesy | how much worse does that performance need to be in order to actually decide to use XFS over ext4? | 15:28 |
| creiht | hehe | 15:28 |
| redbo | yeah, but then we think we have users' files, and we don't :) | 15:28 |
| jdarcy | I should be able to dig up some numbers. Just a sec. | 15:28 |
| creiht | jonesy: I think it was more of a fact that in the mode redbo described before, performance didn't trail off like the others did | 15:29 |
| creiht | jdarcy: redbo has a crazy graph, but he kind of needs to be there to interpret it :) | 15:29 |
| jonesy | gotcha. | 15:29 |
| creiht | And it isn't like we aren't happy with how xfs is performing now, but we always like to tinker to try to make things better | 15:30 |
| redbo | yeah, xfs is really nice for the most part. So long as you're not trying to delete anything. | 15:31 |
| jonesy | well, yeah, that's gonna be an issue for me specifically. | 15:32 |
| jdarcy | Huh. According to Ric Wheeler's Red Hat Summit, fsck of a million-file FS was actually slightly faster for ext4 than for XFS - both ~30x faster than ext3. | 15:32 |
| jonesy | I have to figure out how to recover space by insuring that aged-out objects are removed, and there will likely be millions of them to remove at a time. | 15:33 |
| jdarcy | Those would be the specific versions that are going into RHEL6, I believe. YMMV especially on other versions. | 15:33 |
| jonesy | of course, I'm testing swift in a context it wasn't really built for. And it might not work. | 15:34 |
| jonesy | but it's gonna be fun to try. | 15:34 |
| redbo | there are some kernel changes coming through the pipeline to make xfs deletes faster. basically so it can batch metadata changes. | 15:34 |
| *** spackest has joined #openstack | 15:36 | |
| redbo | but yeah, might be worth trying other filesystems if you have a use case significantly different from ours. We don't delete files very often. | 15:36 |
| *** pvo_ has quit IRC | 15:36 | |
| *** pvo_ has joined #openstack | 15:36 | |
| *** adrian_otto has joined #openstack | 15:37 | |
| jdarcy | Ah, here it is. Can anyone else see http://www.redhat.com/f/pdf/summit/degts_1010_rhel_file_systems.pdf or is it just me? | 15:41 |
| creiht | jdarcy: works for me | 15:41 |
| jdarcy | That has some charts (slide 31 on) for fsck and other kinds of performance. | 15:42 |
| jdarcy | Basically ext3 sucks and will always suck, but ext4 is less like ext3 than it used to be. | 15:43 |
| creiht | jdarcy: It would be interesting to see how long a 2T part that is 80% full on ext4 would take to fsck | 15:47 |
| creiht | It just says faster, not how long it takes :) | 15:48 |
| jdarcy | creiht: ISTRC that Ric tried something very like that, and basically got tired of waiting. | 15:48 |
| jdarcy | creiht: He and Eric Sandeen (XFS maintainer just showed up), by purest coincidence. I'll ask them. | 15:48 |
| *** gnrfan has joined #openstack | 15:48 | |
| creiht | hah | 15:49 |
| *** gnrfan has left #openstack | 15:49 | |
| creiht | jdarcy: That presentation has a lot of good information though, thanks! | 15:51 |
| *** rajijoom has quit IRC | 16:03 | |
| jdarcy | Long story short: XFS fscks slower than ext4 primarily due to higher memory footprint, deferred/batch logging should help lots with both create-heavy and delete-heavy workloads. | 16:05 |
| jdarcy | Ric gave up on the XFS fsck test because he ran out of time to create the files, never even got to the fsck. | 16:06 |
| creiht | hah | 16:06 |
| jdarcy | That was, of course, before the logging changes. | 16:06 |
| *** pvo_ has quit IRC | 16:09 | |
| dendrobates | working on the wiki acl's might be up and down for a few minutes | 16:10 |
| *** sirp1 has quit IRC | 16:13 | |
| *** brd_from_italy has quit IRC | 16:14 | |
| eday | gundlach: saw your wsgi work, cool! did you see my example with the Routes wsgi middleware usage? | 16:20 |
| gundlach | eday: not sure -- didn't my wsgi stuff use your FileRouter which uses Routes? or is there some newer code i haven't seen? | 16:21 |
| eday | gundlach: yeah, the one I emailed to Todd.. one sec | 16:21 |
| eday | gundlach: http://pastebin.org/448203 | 16:21 |
| gundlach | eday: any reason you're not doing this work in a branch? | 16:21 |
| eday | gundlach: from routes.middleware import RoutesMiddleware stuff | 16:22 |
| gundlach | oh! | 16:22 |
| eday | gundlach: not really, it just wasn't relevant yet to put in. was hoping to discuss with Todd first too :) | 16:22 |
| gundlach | cerberus points out that Todd's stuff is three days old now -- i'd suggest you not block on him, and you can discuss when he comes back :) | 16:23 |
| eday | gundlach: Yeah, I just didn't want to step on any toes. Plus there were plenty of otehr things to do. Probably fine to hack on it now | 16:24 |
| *** stewart has joined #openstack | 16:28 | |
| *** rokoroko has joined #openstack | 16:31 | |
| *** sirp1 has joined #openstack | 16:31 | |
| *** rokoroko has quit IRC | 16:37 | |
| *** johnpur has joined #openstack | 16:39 | |
| *** ChanServ sets mode: +v johnpur | 16:39 | |
| *** gundlach has quit IRC | 16:40 | |
| *** gundlach has joined #openstack | 16:42 | |
| *** roko has joined #openstack | 16:42 | |
| dendrobates | ok, done with the wiki acl changes. If anything is not working for you please ping me asap. | 16:43 |
| eday | gundlach: are you reworking the rackspace endpoint with new wsgi stuff? if not I was going to start | 16:44 |
| gundlach | eday: go for it -- i'm mostly out today. | 16:44 |
| gundlach | if you haven't noticed, the new_wsgi.py and eventletserver.py code in ~gundlach/nova/api may be useful. | 16:44 |
| eday | gundlach: yeah, was going to start from there | 16:44 |
| gundlach | code away :) | 16:45 |
| *** dendrobates is now known as dendro-afk | 16:48 | |
| *** dendro-afk is now known as dendrobates | 16:48 | |
| blpiatt | Anyone have thoughts on http://www.illumos.org/ for an option as the host machine OS? (It is the new fork of OpenSolaris) | 16:58 |
| *** roko has quit IRC | 16:59 | |
| blpiatt | I like ZFS and the iSCSI stuff is very high quality as well | 16:59 |
| *** rajijoom has joined #openstack | 16:59 | |
| mjmac | blpiatt: i'd want to see what kind of developer commitment there is to the fork before investing much in it. | 16:59 |
| blpiatt | mjmac, agreed, we don't want to add a whole OS to our project scope if nobody else is supporting it | 17:00 |
| * mtaylor agrees ... unless the userspace utilities around that come along _way_ farther than opensolaris was, it'll be a nightmare | 17:00 | |
| * mtaylor says, currently supporting opensolaris in his other project | 17:00 | |
| mtaylor | otoh - if I could spin up illumos guests - it would make my drizzle build farm easier to scale :) | 17:01 |
| *** dendrobates is now known as dendro-afk | 17:01 | |
| *** dendro-afk is now known as dendrobates | 17:01 | |
| mjmac | sadly, i suspect that illumos will be a hobby for a few folks and that's about it. unless someone builds a company around it, but i have a hard time seeing how that would work out well. | 17:02 |
| *** spackest has quit IRC | 17:02 | |
| blpiatt | yeah, feels like it may end up a smaller community like the *bsds | 17:02 |
| dendrobates | has anyone looked closely at sheepdog: http://www.osrg.net/sheepdog/index.html | 17:04 |
| *** jamiew has joined #openstack | 17:05 | |
| *** metoikos has quit IRC | 17:05 | |
| *** gcc has left #openstack | 17:06 | |
| *** gcc has joined #openstack | 17:06 | |
| *** jakedahn has quit IRC | 17:10 | |
| *** blpiatt is now known as blp-away | 17:11 | |
| *** pcrews is now known as pcrews_lunch | 17:15 | |
| *** kashyapc has quit IRC | 17:38 | |
| *** dendrobates is now known as dendro-afk | 17:39 | |
| *** dendro-afk is now known as dendrobates | 17:39 | |
| *** spackest has joined #openstack | 17:49 | |
| *** blp-away is now known as blpiatt | 18:00 | |
| jdarcy | dendrobates: I've looked at it some. Why? | 18:03 |
| *** kashyapc has joined #openstack | 18:03 | |
| *** skeptomai has joined #openstack | 18:08 | |
| blpiatt | jdarcy, looks like sheepdog is being merged in upstream in QEMU | 18:11 |
| *** kashyapc has quit IRC | 18:11 | |
| blpiatt | jdarcy, trying to figure out all of the block storage related use cases it is appropriate for | 18:11 |
| *** kashyapc has joined #openstack | 18:11 | |
| jdarcy | blpiatt: I think the basic idea of doing block storage that way is a good one, but sheepdog's dependency on corosync to do it seems noxious to me. | 18:12 |
| blpiatt | jdarcy, hmm so sheepdog maps blocks to various disks in the cluster using corosync? | 18:16 |
| *** armmer has left #openstack | 18:16 | |
| jdarcy | blpiatt: Not directly, but it uses it for membership/leadership/locking. | 18:19 |
| holoway | jdarcy: what would you rather it used? | 18:19 |
| jdarcy | blpiatt: I'm not sure their way of consistent hashing onto "whoever's there" without maintaining coherent per-VDI maps really handles node addition/deletion that well. | 18:20 |
| holoway | jdarcy: yeah, the question is whether it can do ring rebalancing | 18:21 |
| holoway | jdarcy: riak style, for example - where you set the total ring size to some superset of devices, so you always have space in the ring to rebalance to as new nodes come online | 18:21 |
| holoway | or old ones go out of service | 18:21 |
| jdarcy | blpiatt: I don't think there should be any notion of leadership, locking should be in another layer, and I'd rather see gossip-based membership. | 18:21 |
| jdarcy | Oops, that was to holoway. | 18:22 |
| jdarcy | holoway: Riak style? Why not Swift style? ;) | 18:22 |
| holoway | jdarcy: all love to swift, but Riak is amazing at this | 18:23 |
| holoway | jdarcy: in terms of membership, ring distribution, membership | 18:23 |
| *** lbieber has joined #openstack | 18:23 | |
| jdarcy | holoway: Not going to argue that point today, especially not here. ;) | 18:23 |
| holoway | (ack, I said membership twice!) | 18:23 |
| holoway | also, riak is not swift | 18:23 |
| holoway | totally different critter | 18:24 |
| jdarcy | And Sheepdog isn't either. I think I might put it a hair closer to Swift than to Riak. | 18:24 |
| holoway | but as an EBS style distributed block store, sheepdog does look interesting | 18:24 |
| holoway | yeah, I was talking about the underlying distributed systems algos | 18:24 |
| holoway | (rebalancing a hash ring, for example, on membership addition) | 18:25 |
| jdarcy | I mean, single-level blocks within devices is pretty similar to single-level objects within containers, not so much to arbitrary key/value with links and post-commit hooks. | 18:25 |
| * holoway nods | 18:25 | |
| jdarcy | I think the creation/deletion rate is probably more Swift-like than Riak-like too. | 18:26 |
| jdarcy | That said, when I first heard about sheepdog/vinzvault one of my first thoughts was to implement a QEMU-block interface on top of Voldemort first. ;) | 18:27 |
| jdarcy | Could as easily have been Cassandra, Riak, etc. | 18:27 |
| holoway | jdarcy: it would be interesting to see one built on riak core | 18:27 |
| holoway | jdarcy: since you don't need the key/value, link walking, hooks, etc | 18:28 |
| * jdarcy nods. | 18:28 | |
| holoway | erlang's pretty facile with binary blobs these days | 18:28 |
| *** blpiatt has quit IRC | 18:28 | |
| jdarcy | The tricky part in something like something like sheepdog is finding the server that *really* holds a block, not just the one that you think should based on hash plus current membership. I'd rather delegate that to something else that already does it well. | 18:29 |
| jdarcy | I have a couple of tricks I use for that in CloudFS, but I'm in the middle of (defensive) patent disclosure so I can't say too much more yet. | 18:30 |
| *** rnewson has quit IRC | 18:30 | |
| *** pcrews_lunch is now known as pcrews | 18:34 | |
| *** rnewson has joined #openstack | 18:52 | |
| *** rnewson has joined #openstack | 18:52 | |
| *** Daviey has quit IRC | 18:55 | |
| *** blpiatt has joined #openstack | 18:56 | |
| *** blpiatt has quit IRC | 19:00 | |
| *** binaryWarrior has joined #openstack | 19:00 | |
| *** blpiatt has joined #openstack | 19:00 | |
| *** justinsheehy has quit IRC | 19:02 | |
| *** Daviey has joined #openstack | 19:02 | |
| *** jonesy_ has quit IRC | 19:08 | |
| *** jonesy_ has joined #openstack | 19:11 | |
| *** jaypipes has quit IRC | 19:14 | |
| *** chromakode has quit IRC | 19:14 | |
| *** chromakode has joined #openstack | 19:14 | |
| *** jakedahn has joined #openstack | 19:15 | |
| *** dendrobates is now known as dendro-afk | 19:16 | |
| *** dendro-afk is now known as dendrobates | 19:16 | |
| *** kashyapc has quit IRC | 19:18 | |
| *** rnewson has quit IRC | 19:21 | |
| *** rnewson has joined #openstack | 19:22 | |
| *** justinsheehy has joined #openstack | 19:32 | |
| *** justinsheehy has quit IRC | 19:39 | |
| *** brd_from_italy has joined #openstack | 19:40 | |
| aliguori | are any of the openstack folks going to be at KVM Forum/LinuxCon next week? | 19:49 |
| aliguori | soren, dendrobates, ^ | 19:49 |
| blpiatt | aliguori, I'll be at LinuxCon, arrive Tue afternoon, leaving Thursday afternoon | 19:50 |
| aliguori | blpiatt, is there going to be a talk on OpenStack? | 19:50 |
| blpiatt | Not scheduled, thoughts on a bof on wednesday night? | 19:51 |
| *** blpiatt has quit IRC | 19:53 | |
| zul | im up for that | 19:54 |
| aliguori | yeah, I'd be interested | 19:57 |
| *** jaypipes has joined #openstack | 19:57 | |
| *** chainey has joined #openstack | 19:59 | |
| * mray wishes he were going to LinuxCon now | 20:04 | |
| jdarcy | Maybe I should find a way to sneak into LinuxCon. | 20:05 |
| mtaylor | creiht, letterj: should be merging trunk into the debian packaging branch on each trunk push now | 20:06 |
| *** mray is now known as mrayzenoss | 20:07 | |
| *** mrayzenoss has joined #openstack | 20:07 | |
| *** mrayzenoss is now known as mray | 20:07 | |
| *** codejunkie has left #openstack | 20:08 | |
| *** dendrobates is now known as dendro-afk | 20:16 | |
| *** aliguori has quit IRC | 20:17 | |
| *** dendro-afk is now known as dendrobates | 20:18 | |
| *** jakedahn has quit IRC | 20:21 | |
| *** rnewson has quit IRC | 20:25 | |
| *** aliguori has joined #openstack | 20:32 | |
| *** rajijoom has quit IRC | 20:36 | |
| *** jonesy_ has quit IRC | 20:43 | |
| *** carolinad28 has quit IRC | 20:45 | |
| *** brd_from_italy has quit IRC | 20:46 | |
| *** justinsheehy has joined #openstack | 20:49 | |
| vishy | what did i miss? | 20:54 |
| *** ctennis has quit IRC | 20:54 | |
| vishy | is justsinsb here? | 20:54 |
| *** justinsheehy_ has joined #openstack | 21:00 | |
| *** justinsheehy has quit IRC | 21:02 | |
| *** justinsheehy_ is now known as justinsheehy | 21:02 | |
| *** ctennis has joined #openstack | 21:07 | |
| eday | mtaylor: any reason why https://code.edge.launchpad.net/~vishvananda/nova/network-refactor/+merge/31808 hasn't landed? I marked it approved yesterday morning | 21:09 |
| mtaylor | eday: yes. it has no commit message | 21:10 |
| eday | mtaylor: oh. anything in particular that should be in there? or just "branch approved"? | 21:12 |
| mtaylor | eday: how about a descriptive commit message ? | 21:13 |
| mtaylor | eday: tarmac uses the value of whatever you put there as the commit when it does the merge | 21:14 |
| *** justinsheehy has quit IRC | 21:14 | |
| eday | mtaylor: it should just use the description field :) | 21:15 |
| mtaylor | eday: I've got a todo list item for that | 21:15 |
| mtaylor | eday: although in fairness, in this case, that would result in a ridiculous commit message | 21:15 |
| *** codejunkie has joined #openstack | 21:16 | |
| mtaylor | eday: this is actually a great example of how the merge request description and the commit message may carry different info for different people | 21:16 |
| mtaylor | eday: although I agree that the system should be a bit less... annoying... in the situations where they are the same | 21:16 |
| mtaylor | and I will fix that | 21:16 |
| eday | mtaylor: hehe, just seemed a little redundant, but yeah, I could see it being useful to have something different than the proposal description | 21:18 |
| mtaylor | eday: it's definitely on my todo list | 21:18 |
| eday | mtaylor: perhaps if it were just required during the proposal creation | 21:18 |
| mtaylor | eday: actually - this might get solved just by defaulting to using the description if there isn't a commit message | 21:18 |
| eday | all in all, still very slick :) | 21:18 |
| mtaylor | cause then you can give one piece of data if that's all you've got - but if you're gonna write a novel in the description, you can add a commit message and be set | 21:19 |
| *** justinsheehy has joined #openstack | 21:19 | |
| eday | yeah, that would probably work | 21:19 |
| *** justinsheehy has quit IRC | 21:20 | |
| eday | anyways, thanks! was wondering what was up with it :) | 21:21 |
| mtaylor | np! | 21:21 |
| eday | doh, busted | 21:22 |
| mtaylor | hehe | 21:23 |
| * eday pokes vishy | 21:23 | |
| *** justinsheehy has joined #openstack | 21:24 | |
| *** aliguori has quit IRC | 21:25 | |
| *** spackest has quit IRC | 21:41 | |
| *** BartVB has quit IRC | 21:41 | |
| *** aliguori has joined #openstack | 21:42 | |
| * vishy screams | 21:43 | |
| vishy | eday: don't surprise me like that! | 21:43 |
| vishy | i just noticed a failing test in the branch anyway | 21:44 |
| vishy | due to my awesome sed skills | 21:44 |
| vishy | fixed now | 21:44 |
| *** jdarcy has quit IRC | 21:45 | |
| vishy | has anyone looked at justinsb's threading stuff? | 21:51 |
| vishy | he's doing a lot of neat work and I haven't had time to check it out in detail | 21:51 |
| eday | vishy: not yet, but I'm hoping to later today | 21:51 |
| vishy | i am excited about virtualbox bindings | 21:55 |
| vishy | i just installed a vm so that i can actually test without installing to a remote server | 21:55 |
| *** adrian_otto1 has joined #openstack | 21:58 | |
| *** adrian_otto has quit IRC | 22:02 | |
| *** pcrews has quit IRC | 22:03 | |
| *** binaryWarrior has quit IRC | 22:14 | |
| *** pvo has quit IRC | 22:16 | |
| *** Podilarius has left #openstack | 22:22 | |
| *** mray has quit IRC | 22:25 | |
| *** sirp1 has quit IRC | 22:33 | |
| *** sirp1 has joined #openstack | 22:33 | |
| eday | _cerberus_: I think the first step is to get some better unittests in there around the API so we can be test-driven as we add new things | 22:38 |
| _cerberus_ | eday: Would certainly be a good idea to set that precedent | 22:39 |
| _cerberus_ | eday: though test-driven certainly implies test-first ;-) | 22:39 |
| *** pvo has joined #openstack | 22:45 | |
| *** pvo has joined #openstack | 22:45 | |
| *** ChanServ sets mode: +v pvo | 22:45 | |
| *** that1dude has joined #openstack | 22:45 | |
| *** abecc has quit IRC | 22:48 | |
| *** pvo has quit IRC | 22:56 | |
| *** adrian_otto1 has quit IRC | 22:58 | |
| vishy | +1 unittests | 23:04 |
| vishy | i'm hoping to have some time to work on testing | 23:04 |
| vishy | it is getting unmanagable...I basically have to do a full deploy and do stuff to uncover bugs | 23:05 |
| *** aliguori has quit IRC | 23:07 | |
| *** johnpur has quit IRC | 23:09 | |
| mtaylor | vishy: sorry ... that's been high up on my list of things to get going... | 23:16 |
| mtaylor | vishy: a hudson job you can submit a branch to that will do a fresh deploy and then run stuff | 23:16 |
| *** jakedahn has joined #openstack | 23:22 | |
| *** jakedahn has quit IRC | 23:24 | |
| *** jonesy_ has joined #openstack | 23:26 | |
| *** lbieber has quit IRC | 23:31 | |
| vishy | mtaylor: that would rock | 23:35 |
| mtaylor | vishy: I'd be farther along on that if I hadn't been hobbled by this darned illness | 23:35 |
| * mtaylor curses germs | 23:35 | |
| alekibango | +1 test driven development is good | 23:35 |
| vishy | mtaylor: opscode is doing something like that with chef, only it will support doing it from a freshly booted box | 23:36 |
| *** gustavomzw has quit IRC | 23:36 | |
| vishy | and eventually multiple boxes | 23:36 |
| vishy | which i think would be cool | 23:36 |
| *** allsystemsarego has quit IRC | 23:37 | |
| vishy | my goal is. One click on my mac: download, install, run from a branch with virtualbox bindings | 23:37 |
| vishy | then automatically create 4 ubuntu vms | 23:37 |
| mtaylor | vishy: yeah - I'm actually working on a plugin for hudson that will go sort of hand-in-hand with opscode's work | 23:37 |
| vishy | and deploy nova to them with chef | 23:37 |
| vishy | and run a bunch of tests | 23:38 |
| eday | mtaylor: i was playing with nosetests+nova yesterday, got it down to only two errors :) | 23:38 |
| mtaylor | vishy: which is a) I want to be able to have hudson spin up new boxes as needed for build slaves, b) populate it with what I need via chef and then c) run a bunch of tests | 23:38 |
| mtaylor | eday: excellent! | 23:38 |
| vishy | nova on top of nova on a mac | 23:38 |
| alekibango | vishy: why ubuntu? isnt debian on server much better choice? | 23:38 |
| vishy | whoot! | 23:38 |
| eday | mtaylor: i'd really like to get those converted to work... keep them self contained | 23:38 |
| vishy | alekibango: we are using ubuntu internally, so I want to match our deploy as much as possible | 23:38 |
| mtaylor | eday: yes. | 23:38 |
| alekibango | ic | 23:38 |
| mtaylor | alekibango: ubuntu server is a really really common server platform these days | 23:39 |
| alekibango | vishy: btw great work, have seen your name a lot when i used gource on sources :D | 23:39 |
| eday | vishy: how offended would you and the other nasa guys be with no more branches accepted if pep8/pylint numbers go up? (ie, all new code has to conform) | 23:39 |
| vishy | eday: I think that is fine | 23:39 |
| alekibango | btw debian squeeze is frozen | 23:39 |
| mtaylor | eday: (I'm probably about a week away from being able to enforce that) | 23:40 |
| vishy | eday: we hack in stuff to our own branch as needed | 23:40 |
| vishy | eday: but public repo should be clean as possible imo | 23:40 |
| eday | vishy: awesome | 23:40 |
| vishy | eday: I'll run it by the team, but probably suggest it in mailing list so everyone can have a say | 23:41 |
| eday | I'm going to keep a branch going cleaning up old stuff as I can. great way to dig in and learn more details :) | 23:41 |
| vishy | eday: a few of the pylint complaints are insane imo | 23:41 |
| eday | vishy: yeah.. I don't think many other folks will have a problem | 23:41 |
| mtaylor | ++ ... definitley also need a sane pylintrc :) | 23:41 |
| vishy | like, uses args/kwargs magic? | 23:41 |
| eday | vishy: yeah, we can add exceptions.. I already added a few (like minimum # of methods!?!) | 23:41 |
| vishy | the whole point of python is for args/kwargs magic imo | 23:42 |
| vishy | :) | 23:42 |
| mtaylor | eday: when/if you get one of those in the tree let me know and I can point the hudson pylint job at it | 23:42 |
| eday | mtaylor: already in, and vish approved | 23:43 |
| eday | mtaylor: so, ready to be merged if you want to mark it :) | 23:43 |
| mtaylor | eday: done | 23:45 |
| *** miclorb has joined #openstack | 23:46 | |
| *** sirp1 has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!