Friday, 2010-07-30

sorenOr just one?00:00
*** pcrews has quit IRC00:01
vish1_soren: no idea00:01
vish1_wait though00:01
vish1_another (related?) issue00:01
vish1_ File "/Users/vishvananda/cloudkick/nova/.nova-venv/lib/python2.6/site-packages/boto/utils.py", line 72, in canonical_string00:01
vish1_            provider = boto.provider.get_default()00:01
vish1_        exceptions.AttributeError: 'module' object has no attribute 'provider'00:01
vish1_that is boto actually failing internally?00:02
sorenSemi-related.00:02
sorenYeah.00:02
*** miclorb_ has quit IRC00:02
sorenWe can fix it from nova, though, but yeah, it's boto's own fault.00:02
vish1_so 2.0b1 is a pos00:02
vish1_how? import boto.provider?00:03
sorenShould do the trick.00:03
*** rmillner has quit IRC00:03
*** lbieber has quit IRC00:03
sorenbut it's 2 AM for me. Take whatever I say with a grain of your favourite mineral.00:03
vish1_trying the hack00:04
vish1_ok just importing boto00:05
vish1_instead of boto.utils works00:05
sorenYeah.00:05
sorenThat's why they didn't catch it.00:05
soren"import boto" gets everything imported, probably.00:06
*** miclorb_ has joined #openstack00:11
*** adjohn has joined #openstack00:11
*** iroth has quit IRC00:19
*** gustavomzw has quit IRC00:21
*** gasbakid has quit IRC00:23
*** miclorb_ has quit IRC00:29
vish1_wanted feature:00:34
vish1_flag for hvm vs qemu in libvirt.xml creation00:34
vish1_--libvirt_type00:39
*** _buck has quit IRC00:43
vish1_i went ahead and made it :)00:56
*** miclorb has joined #openstack01:01
*** maximilien has quit IRC01:02
*** jdmaturen has quit IRC01:05
vish1_if there are any people familiar with brew recipes, we could use one for euca2ools01:06
vish1_here are the manual steps01:06
vish1_wget http://open.eucalyptus.com/sites/all/modules/pubdlcnt/pubdlcnt.php?file=http://eucalyptussoftware.com/downloads/releases/euca2ools-1.2.tar.gz&nid=134601:06
vish1_tar zxf  euca2ools-1.2.tar.gz01:06
vish1_cd euca2ools-1.201:06
vish1_sed -i="" "s/-g root/-g wheel/g" Makefile01:06
vish1_sudo make install01:06
*** kallistec has quit IRC01:07
*** tobym has quit IRC01:08
*** vish1_ has quit IRC01:25
*** dendrobates is now known as dendro-afk01:27
*** dendro-afk is now known as dendrobates01:27
*** mrayzenoss has joined #openstack01:28
*** mrayzenoss is now known as mray01:28
*** mray has joined #openstack01:28
*** justinsb_ has quit IRC01:31
*** jdmaturen_ has joined #openstack01:36
*** jdmaturen_ has quit IRC01:37
*** jdmaturen has joined #openstack01:38
*** jdmaturen has quit IRC01:40
*** mray has quit IRC01:41
*** Brainspackle has joined #openstack01:48
*** miclorb has quit IRC01:48
*** iroth has joined #openstack01:48
*** Brainspackle is now known as viewlogic01:49
*** kallistec has joined #openstack01:51
*** lbieber has joined #openstack01:51
*** miclorb_ has joined #openstack01:56
*** silassewell has quit IRC01:59
*** miclorb_ has quit IRC02:01
*** miclorb has joined #openstack02:07
*** littleidea has quit IRC02:09
*** lbieber has quit IRC02:10
*** blpiatt has joined #openstack02:12
*** littleidea has joined #openstack02:13
*** silassewell has joined #openstack02:13
*** littleidea has quit IRC02:16
*** littleidea_ has joined #openstack02:16
*** littleidea_ has quit IRC02:19
*** miclorb has quit IRC02:22
*** miclorb has joined #openstack02:23
*** miclorb has quit IRC02:23
*** mrayzenoss has joined #openstack02:28
*** mrayzenoss is now known as mray02:28
*** lbieber has joined #openstack02:35
*** sophiap has joined #openstack02:39
*** littleidea has joined #openstack02:48
*** littleidea has quit IRC02:54
*** _jcsmith has joined #openstack02:54
_jcsmithanybody know a graphical mac/windows cloud files file transfer/browser app that works with a non-rackspace auth server? the command line stuff works well, just want to do a demo to non techies02:57
_jcsmithfor swift02:57
*** JimCurry has joined #openstack02:59
*** wattersjames_ has joined #openstack03:01
*** littleidea has joined #openstack03:02
*** wattersjames has quit IRC03:03
*** miclorb_ has joined #openstack03:04
creiht_jcsmith: unfortunately none that I am aware of :/03:10
_jcsmithok :)03:11
creihtI'm looking around, but can't seem to find anything03:11
_jcsmithI looked too, if people felt there was a need for a windows one I could write something in C#, but that means I'd have to admit to being a former windows developer ;)03:14
creihtThere is supposedly an iphone app that does, but not sure if that will work for you03:14
creihthaha03:14
creihtIt wouldn't be hard to whip something up with pyQT03:14
creihtor something like that03:14
_jcsmithyeah, I'd prefer the python wrote03:15
_jcsmithI went to the nova meetup in SF today, there was a good showing of people, I look forward to the swift one03:16
creihtswift one?03:16
creihtHadn't heard about it yet03:16
_jcsmithmeetup.com03:16
_jcsmithopenstack group03:16
_jcsmithat nasa ames03:16
creihtahh cool03:17
_jcsmithits at the end of august03:17
creihtwell hopefully they will send out some swift devs :)03:17
_jcsmiththat would be awesome, you deserve it, you've answered a lot of my newbie questions03:18
creihthah03:18
*** tobym has joined #openstack03:19
*** StylusEater has left #openstack03:22
_jcsmithcreiht: are you familiar with python-cloudfiles?03:23
creiht_jcsmith: a little03:23
_jcsmithoff the top of your head do you know how to specify a custom auth url?03:24
cw_jcsmith: fake out DNS ?  or are certs involved?03:24
blpiatt_jcsmith have you checked out the openstack web control panel for interfacing with object storage? you can ping @greenisus about it on twitter or here, or email mike.mayo@rackspace.com03:24
cw_jcsmith: please tell me you're not still working in the city? :-)03:25
_jcsmithcw: no, back in the east bay. I fixed the auth issue03:25
cwoh sweet, let me fire up IM03:25
_jcsmithwant to use something other than 'st' to talk to the cluster03:25
_jcsmithblpiatt: no, but I'll check it out!03:26
creiht_jcsmith: I think you send authurl when creating the connection03:26
cw_jcsmith: ->03:26
creiht_jcsmith: you might also checkout swift/common/client.py for a simpler bear metal api03:27
blpiatt_jcsmith he lives in the bay area too, he was at the meetup tonight so you could have met him today and didn't know it ;)03:27
_jcsmithis client.py the same as 'st'? I got that working03:27
creihtst uses client.py03:27
cwblpiatt: we were in the back left corner03:27
*** jsgotangco has joined #openstack03:28
*** littleidea has quit IRC03:28
blpiattcw, I'm in san antonio so I didn't make it :(  I just talked to him and said "make sure everyone has pizza tonight!"03:28
_jcsmithoh yeah, thanks for the pizza Rackspace :)03:29
jsgotangcomm pizza03:30
creihtblpiatt: Do you know if they are planning on sending out any swift devs to the swift meetup?03:31
blpiattI'm not sure how we're handling getting proper coverage to all of the stuff yet, we need to figure out how to ramp up people all over as meetups are going to be everywhere soon and we won't be able to cover them all03:32
creihtahh03:32
blpiattAwesome that the first was in the bay so the team from Ames could make it up as well as Mike heading over03:32
cwlee was there too03:32
_jcsmithany excuse to go to texas works for me :)03:33
cwget in line03:33
blpiattYeah, Lee lives in Palo Alto as well, we have a few folks in that area too :)03:33
blpiattXen.org is going to have their next summit in TX so come on out for that03:33
*** mray has quit IRC03:33
*** pvo has joined #openstack03:38
*** ChanServ sets mode: +v pvo03:38
blpiattpvo, good evening03:39
pvowhy hello there.03:39
_jcsmithpvo: thanks for hosting us at your digs tonight03:40
pvo_jcsmith: I have digs? I'm in San Antonio...03:40
_jcsmithoops03:40
pvoyou guys were at cloudkick?03:41
_jcsmitha case of mistaken identity03:41
_jcsmithyeah03:41
blpiattI love google translate but it isn't quite perfect yet, checking out Bayoda's website (openstack corporate CLA signed), they have a backup product, translated one portion as "Provides furious fast data loss", somehow that can't really be right :D03:41
pvogood times? lots 'o code?03:41
_jcsmith'o code, heh03:41
blpiattcloudkick = polvi, cloud servers dev lead for rackspace = pvo03:42
_jcsmiththat explains it :)03:42
*** chilts has left #openstack03:44
pvosweet. tests passing today.03:45
blpiattpvo, did you bribe hudson or did the team push some new code?03:45
pvo~ 30 revs since I last checked out.03:47
*** mrayzenoss has joined #openstack03:48
*** mrayzenoss is now known as mray03:48
*** mray has joined #openstack03:48
blpiattawesome, love the pace at which stuff is getting revved03:48
_jcsmithbtw, to answer my own question about python-cloudfiles, this is how you can specify your the url to your own auth server03:49
_jcsmithconn = cloudfiles.get_connection(username='accountXXX:userXXX', api_key='passwdXXX', authurl='http://api.mytestbox.com:11000/v1.0')03:49
creihtyup03:49
*** mray has quit IRC03:49
_jcsmiththanks creiht03:50
creihtand I just looked at the control panel code, and correct me if I'm wrong, but I don't think it has storage support in it yet03:51
blpiattcreiht, hmm that could be true! sounds like it needs help03:52
_jcsmithis the control panel a django app?03:52
creihtworse, it is ruby :)03:53
*** littleidea has joined #openstack03:53
_jcsmithoh man, no comment :)03:53
creihtand a lot of JS03:53
creihtthough to be fair, the ruby is a bit light,03:53
blpiattit is a ruby backend with a cappuccino front end, written by an iphone dev ;)03:53
_jcsmithneat03:53
blpiattit is a prototype, we need a functional control panel, not sure what the best thing to write it in is long term, we'll have to talk more about that at the dev summit in november, maybe hack on this until then03:54
blpiattthe ruby backend is a jsonp proxy so you could connect multiple frontends to it, mike was messing with some ideas when he put it together03:55
jsgotangcoblpiatt:  hey sorry to budge in but you got to talk to yoram earlier yes?03:55
_jcsmithI agree having a control panel is a must to make this as turn key as possible for hosting companies03:55
blpiattjsgotangco, yes I did, great chat03:55
jsgotangcoblpiatt:  great thanks looking forward to collaborating03:57
blpiattwe're really excited to help give a big community a place to collaborate on building the best software together03:57
*** metoikos has quit IRC04:04
*** jernst has joined #openstack04:05
*** jernst has left #openstack04:06
*** miclorb_ has quit IRC04:06
*** JimCurry has left #openstack04:08
*** miclorb_ has joined #openstack04:13
*** cjwilson has joined #openstack04:15
*** silassewell has quit IRC04:16
*** tobym has quit IRC04:19
*** adjohn has quit IRC04:22
*** tobym has joined #openstack04:22
*** miclorb_ has quit IRC04:26
*** miclorb_ has joined #openstack04:26
*** adjohn has joined #openstack04:34
*** jsmith is now known as jsmith-away04:42
*** blpiatt has quit IRC04:45
*** pvo has quit IRC04:55
*** maximilien has joined #openstack05:01
*** alexsolo has joined #openstack05:03
*** miclorb_ has joined #openstack05:09
*** jbarratt has quit IRC05:09
*** alexsolo has quit IRC05:10
*** abecc has quit IRC05:15
*** lbieber has quit IRC05:19
*** maximilien has quit IRC05:21
*** miclorb_ has quit IRC05:22
*** miclorb has joined #openstack05:31
*** Cloudkylemac has joined #openstack05:32
*** Cloudkylemac has left #openstack05:33
*** kylemac has joined #openstack05:34
*** kylemac is now known as topspinkyle05:35
*** topspinkyle has joined #openstack05:35
*** miclorb has quit IRC05:51
*** miclorb_ has joined #openstack05:52
*** wattersjames has joined #openstack05:53
*** miclorb_ has joined #openstack05:53
*** miclorb_ has quit IRC06:25
*** miclorb has joined #openstack06:25
*** _jcsmith has quit IRC06:26
*** rajijoom has joined #openstack06:28
*** wattersjames has quit IRC06:43
*** fortune has joined #openstack06:59
*** joshuamckenty has joined #openstack07:01
*** _buck has joined #openstack07:07
*** calavera has joined #openstack07:19
*** CnC has quit IRC07:48
*** _buck has quit IRC07:59
*** iroth has quit IRC08:18
*** adjohn has left #openstack08:19
*** littleidea has quit IRC08:21
*** allsystemsarego has joined #openstack08:21
termiei think mailing lists are broken for me08:25
termiei am not getting anything form the nova mailing list08:25
*** gasbakid has joined #openstack08:26
termieyet my launchpad ui claims i am08:26
termieand my messages haven't gone through or been given a rejection notice08:28
termieand i thought i was just sitting here waiting for replies08:28
termiewhen nobody has even seen my messages08:28
*** joshuamckenty has quit IRC08:30
termiewhat is the little diamond shaped symbol that is next to project names on launchpad?08:32
termieit says "bazaar branches" when i hover08:33
termiebut that doesn't seem to mean much considering we are on launchpad08:33
sorentermie: Where, exactly?08:44
termieif i go to launchpad.net/~termie08:45
termieunder Most Active In08:45
termiethe three projects i've participated in in launchpad all have this little symbol08:45
sorenRight. Lots of project that don't /live/ on Launchpad, are /registered/ on Launchpad, for instance.08:45
termiethis is OpenStack Compute(nova), gitbzr-ng and Pantheon08:46
sorenOthers just host bugs on Launchpad, but vcs elsewhere, or vice versa.08:46
termieso two of those are all in launchpad08:46
sorenThose symbols denote which parts of the project live on Launchpad.08:46
termieso everything, effectively, wil have the bazaar symbole then?08:47
sorenNo.08:47
soren08:46 < soren> Others just host bugs on Launchpad, but vcs elsewhere, or vice versa.08:47
termieright, i said effectively08:47
termieany project that is being worked on on launchpad08:47
sorenFar, far from it.08:47
sorenLots of project that don't /live/ on Launchpad, are /registered/ on Launchpad.08:47
termieokay fine, plenty of other stuff won't have it08:47
termiethe point is it is very common08:48
termieand it looks very much like the also difficult to read edit symbol08:48
sorenReally. I'm not being pedantic. Thousands of projects are registered on Launchpad without their actually using it.08:48
termieand only provides minimal information once you are already familiar with a project08:48
termiecan we ditch the yellow and just make it a gray symbol that doesn't catch the eye?08:49
termiemy launchpad overview looks like it is calling out for attention from every link08:49
sorenI'm not arguing that the information is superuseful or that it belongs on one's main page.08:49
termiei'm not claiming you are, i think we were off topic08:50
sorenQuite so :)08:50
termiethe point of all this being that it represents something common and has a visual conflict with another symbol08:51
termieperhaps a chrome extension is hte way to go here08:52
termie.sprite { display: none; }08:54
termiesoren: on a different note, any idea why i'm not receiving mail from the mailing list? on my +editemails page it has my email address under 'subscribe with' for both nova and openstack discuss08:59
*** jtdowney has joined #openstack09:00
termiei've tried cycling it09:00
termieso maybe i'll get new messages?09:00
*** adjohn has joined #openstack09:08
*** londo has quit IRC09:08
*** londo has joined #openstack09:11
sorentermie: It landed in the moderation queue for some reason. I'm trying to work out why.09:11
termiesoren: i'm a member of nova-core rather than "nova" perhaps?09:12
termiealthough it says inderirect membership and put the list on my mailing lists section09:12
sorentermie: Yeah, it should be fne.09:12
sorenfine, even.09:12
sorentermie: Could you send something else to the list, please? Then I can approve your real e-mail and use the new one for debugging with the launchpad guys.09:18
*** adjohn_ has joined #openstack09:21
*** adjohn has quit IRC09:21
*** adjohn_ is now known as adjohn09:21
termiesoren: sent09:21
sorenta09:22
sorentermie: Uh... I think maybe someone went ahead and did the moderation thing for me.09:22
sorentermie: And your second one shot straight through. Did you also receive them back?09:24
termiei hven't received anything at all, no :/09:25
termiealthough somebody claims to have gotten my last email09:25
termieor rather, my email about gflags09:25
sorenYup, I got both that one and your test e-mail.09:26
termiewant to reply to the test email so i can see if i get anything?09:26
sorenHan gon.09:27
sorenHang on, even.09:27
termieah man, i was really hoping i was just learning a new cool phrase09:27
*** metoikos has joined #openstack09:28
sorentermie: Is it flowing both ways now?09:36
sorenmtaylor: https://edge.launchpad.net/~soren/+archive/ppa/+build/189330509:37
sorenmtaylor: Apparantly our unit tests require a running redis-server.09:38
*** jsgotangco has quit IRC09:44
termieaye, something i'd love to change but -shrug-09:45
termiesoren: nope, still haven't received any mail09:49
termienor is it in spam or anything like that09:50
termiei do get bug emails09:50
termieand merge emails09:50
termie... also my "Most active in" seems to have disappeared in the last 5 minutes10:00
termiewhich i only mention as possibly useful debug information10:00
*** gasbakid has quit IRC10:05
*** jtdowney has quit IRC10:21
*** gasbakid has joined #openstack10:24
termiesoren: you expect i'll have to wait until u.s. morning to see mailing list email?10:25
*** cjwilson has left #openstack10:31
*** stewart has quit IRC10:33
termiei'll just reply to messages without threading for now10:34
*** stewart has joined #openstack10:34
*** stewart has quit IRC10:41
*** stewart has joined #openstack10:44
*** avsm has joined #openstack10:48
sorentermie: The one guy I know does mailing list stuff on LP (Barry Warzaw, of mailman fame) is on US time, at least.10:53
sorentermie: There might be others, I'm not sure.10:53
sorentermie: IIRC, he's an early riser, though, so it may not be too long.10:54
termieoh man, i used to work on mailman10:55
termiehere's a pretty fancy piece of it that is still there http://bazaar.launchpad.net/~mailman-coders/mailman/2.1/annotate/head:/Mailman/MailList.py#L9810:56
termieat least when i last was using it (which was something like 6 years ago) trying to subclass that class would cause a coredump10:57
termienearly the only time i've ever managed to coredump python10:58
termiei think i managed another time when writing an extension10:58
*** zul has quit IRC10:59
*** jtdowney has joined #openstack11:00
*** Slidey has left #openstack11:02
*** jsmith-away is now known as jsmith11:03
*** miclorb has joined #openstack11:04
termiehttp://term.ie/data/cleanpad_before.png http://term.ie/data/cleanpad_after.png11:07
*** jtdowney has quit IRC11:11
*** miclorb has quit IRC11:15
*** jonesy_ has quit IRC11:16
*** miclorb has joined #openstack11:19
*** miclorb has quit IRC11:24
*** tobym has quit IRC11:26
*** miclorb has joined #openstack11:32
*** miclorb has quit IRC11:43
*** justinsheehy has quit IRC11:45
*** carolinad28 has joined #openstack11:48
*** miclorb has joined #openstack11:50
*** jsmith is now known as jsmith-away12:20
*** miclorb has quit IRC12:32
*** calavera has quit IRC12:32
*** justinsheehy has joined #openstack12:44
*** fortune is now known as communitystring12:47
*** communitystring is now known as CommunityString12:47
*** maximilien has joined #openstack12:51
*** CommunityString has left #openstack12:52
*** joshuamckenty has joined #openstack13:23
*** jonesy_ has joined #openstack13:26
*** jsmith-away is now known as jsmith13:26
*** gundlach has quit IRC13:34
*** abecc has joined #openstack13:37
*** gasbakid has quit IRC13:42
*** pvo has joined #openstack13:43
*** pvo has joined #openstack13:43
*** ChanServ sets mode: +v pvo13:43
*** gasbakid has joined #openstack13:45
*** joshuamckenty has quit IRC13:45
sorenEvery example of eventlet code I've come across either deals with raw sockets or speaks to a server through eventlet.urllib2 or some variation thereof. What if I want to speak another protocol to something? Does it just happen to work because the socket module has been replaced?13:51
creihtsoren: Yes13:51
creihtIn most cases, you can use a python library that implements that protocol13:52
creihtunless it does something out of the ordinary (like uses a c extension to handle socket communication)13:52
soren..or uses some amount of global state or whatnot, I suppose.13:53
creihtI wouldn't think global state would matter13:53
*** Podilarius has joined #openstack13:53
sorencreiht: No?13:53
creihtUnless that global state is used for that one connection I guess13:54
sorencreiht: If its interactions with socket suddenly arent' blocking..13:54
sorenright, exactly.13:54
sorenOk.13:54
creihtsoren: do you have a certain protocol in mind?13:54
creihtOr just wanted to see examples?13:55
sorencreiht: Not really.13:55
soren(To your first question)13:56
sorenWhat about file operations or db access?13:57
creihtfor file operations we do reads/writes in chunks and make sure to let eventlet get control between operations with eventlet.sleep()13:57
creihtthat seems to work pretty well13:58
creihtdb access can be a little more problematic, as db libs usually do weird things13:58
creihtThe easiest thing is to just access them from a thread pool (like you would in twisted with deferr to thread)13:59
creihtThere has been work on both postgres and mysql async support, but I've never messed with either13:59
*** adjohn has quit IRC13:59
sorencreiht: In Twisted, I use adbapi, but that may just be deferToThread behind the scenes. I'm not sure.14:00
creihtsoren: yeah looks like it uses thread14:02
*** jbryce has quit IRC14:03
creihtsoren: eventlet has something similar with db_pool14:03
*** lbieber has joined #openstack14:03
creihtI think that is where the MySQL and pyscopg support comes in, but have never used it14:03
*** carolinad28 has quit IRC14:03
*** jbryce has joined #openstack14:03
*** jsgotangco has joined #openstack14:08
sorenI'll probably miss twisted.web most of all. WSGI feels really amputated next to it.14:08
*** jsmith is now known as jsmith-away14:08
creihtwsgi is quite freeing once you learn it14:09
*** _buck has joined #openstack14:09
creihtBy using wsgi, it opens up tons of other libraries available for use14:09
creihtsoren: which parts of twisted.web do you think you will miss?14:10
*** lbieber has quit IRC14:10
*** binaryWarrior has joined #openstack14:11
sorenResource and the render_* methods.14:11
sorenIt just saves a lot of copy/paste stuff every time I need to do something with http.14:12
sorenSure, it's reasonable easy to write a dispatcher that does mostly the same thing with WSGI, but it's easier to use one that someone else wrote for me :)14:13
creihtsoren: but that is exactly, you can use any of the wsgi dispatchers that are already written14:13
*** rajijoom has quit IRC14:13
creihtYou don't have to write that14:13
creihtand you aren't tied to a specific one14:13
creihtWe didn't use one in swift because we a.) were trying to minimize dependencies and b.) had a very simple request structure anyways14:15
PiotrSikorahmm... is there any way to use swift on FS without xattr right now?14:17
creihtPiotrSikora: no14:17
creihtwhich FS?14:17
PiotrSikoraUFS ;)14:17
PiotrSikoramake it FFS14:18
*** binaryWarrior has quit IRC14:18
creihtsoren: You can even use eventlet with most of the wsgi frameworks available (like pylons, restish,  werkzeug, etc.)14:18
creihtheh14:18
*** binaryWarrior has joined #openstack14:18
PiotrSikoracreiht: are there any plans to? like back-falling to storing meta-data in sqlite?14:19
creihtPiotrSikora: which os?14:19
PiotrSikoraOpenBSD ;)14:19
*** binaryWarrior has quit IRC14:19
*** binaryWarrior has joined #openstack14:20
creihtPiotrSikora: http://en.wikipedia.org/wiki/Extended_file_attributes14:20
creihtsays that ffs supports xattrs14:20
creihtthough it calls them flags14:20
creihtOne of the main reasons that we use xattrs is it makes it fairly easy to make sure object operations are atomic14:22
PiotrSikoracreiht: wikipedia lies14:22
creihtIt's on the internet, it must be true! :)14:22
PiotrSikoraand OpenBSD flags are predefined and used for something else (file immutability, append-only, etc)14:22
creihtBut it becomes more difficult to make things atomic which you are managing a file and some entries in a DB14:23
creihthrm14:23
PiotrSikorawell, swift is "eventaully consistent", so this shouldn't be deal-breaker ;)14:24
creihtIt isn't so much about that14:24
*** mrayzenoss has joined #openstack14:26
*** mrayzenoss is now known as mray14:26
*** mray has joined #openstack14:26
gholtConsistent and atomic are definitely different.14:26
redboI don't know if I'd focus on the atomicity, it's just a good place to put them14:26
creihthah14:26
notmynamewow, that got a response from the swift devs14:27
gholtHeheh14:27
creihtI guess everyone is awake now :)14:27
creihtPiotrSikora: We tried several approaches, and xattrs were the best solution that we found14:28
*** Podilarius has left #openstack14:28
gholtWe did have a sqlite based object server. It was slow, but it had everything in one db, so...14:28
gholtDoes BSD have resource forks? :P14:28
redbowe also tried putting xattrs at the end of the file, but it made the hd seek to the end then back to the front14:28
creihtOf course if anyone comes up with a better way, patches are welcome14:28
PiotrSikoragholt: i know it isn't the same14:28
PiotrSikoragholt: but it seems that when something is EV and resources are versioned, then atomicity shouldn't be required14:29
PiotrSikorabut i might be wrong ;)14:29
redboit's not about atomicity14:29
gholtOh it's not, I was teasing creiht. :)14:29
creihtheh14:29
gholtWe can do all sorts of things to the uploaded data before moving it into place, which is atomic.14:29
creihttrue14:30
redbowe would have had to move the metadata to the beginning of the file, which requires copying the file over again.  If you think not using nginx is inefficient, that's murder.14:30
PiotrSikoraredbo: hehe, i agree14:30
PiotrSikoraand merging meta-data with actual data seems like a no-no to me14:31
PiotrSikorabut i was thinking about making things more portable14:31
redbothe only other option is a second .meta file, which is as bad as having it at the end of the file for performance14:31
PiotrSikora(which means back-falling to less effective way for non-xattr systems)14:32
PiotrSikoraredbo: yes, but its better than not being able to use swift at all14:32
notmynamePiotrSikora: I disagree14:32
redboyeah maybe14:32
*** binaryWarrior has quit IRC14:33
gholtWell, for me at least, I'd prefer not to provide subpar systems. I hope you don't /have to/ use a particular os.14:33
*** Podilarius has joined #openstack14:33
creihtWow... OpenBSD doesn't support Journaling filesystems period14:33
gholtWhat would really rock (probably) is if somebody smart figured out how to do all this to a raw partition and threw out all that file system stuff we don't need. :D14:34
PiotrSikoranotmyname: do you mind elaborating a bit on that? ;)14:34
notmynameya, sorry. got asked a question14:34
redbogholt: quit that14:34
notmynameat some point, portability becomes an excuse to make a system that doesn't work very well. to me, the answer is "don't do that"14:34
notmynamein other words, as gholt said, I think a subpar system is worse than no system at all14:35
redboI'd be fine with abstrcting the xattr stuff so it could be stored in a different file.  It's not *that* much of a performance difference.14:35
redbowell, I wouldn't be fine with doing it myself, but I'd be fine if that existed.14:35
PiotrSikoragholt: actually, i don't... but i prefer using OpenBSD for as my development box :)14:35
PiotrSikoranotmyname: but does it really render it subpar?14:36
PiotrSikorawe are not talking about disabling main features14:36
PiotrSikoraor slashing performance in half14:36
gholtWell, the single db definitely did that. Hehe14:36
notmynamewell, for this particular issue, I don't know. I'd want to see comparison numbers. I mean it more as a general guideline.14:37
creihthah14:37
*** topspinkyle has quit IRC14:37
notmynamegood stuff >> no stuff > bad stuff14:37
gholtBut yeah, metafiles would probably be okay for a hobby system [teasing]14:37
* creiht waits for the person who wants to run swift on Windows14:38
*** sophiap has quit IRC14:38
PiotrSikoranotmyname: good stuff >> so-so stuff > no stuff > bad stuff14:38
PiotrSikora:P14:38
notmynamegood stuff >> no stuff > bad stuff >>> swift on windows14:38
*** jsgotangco has quit IRC14:38
gholtStop it!14:38
gholtOh wait.14:38
redboI don't see why it wouldn't run okay14:38
gholtNo continue, this is fun. hehe14:38
PiotrSikora:)14:39
creihtPiotrSikora: All of that said, if someone were to produce a patch that added something like a .meta for each uploaded file that contained the metadata that a.) didn't hurt performance too much, and b.) didn't add too much complexity to the code, we would consider it14:40
gholtThat's a lot of words for: patches welcome!14:41
redboI don't care so much about performance.  Let people who use the feature worry about that.  But yeah, it'd have to be clean :)14:41
gholt[we use that feature!]14:41
*** tr3buchet has joined #openstack14:42
* gholt thinks it's strange that we're all so froggy on a Friday morning.14:43
creihtheh14:43
cory_lack of sleep14:43
PiotrSikoracreiht: yeah, i'm already looking into it ;)14:43
creihtAnd to be honest, we already have some of that, since we temporarily use .meta files to handle updates to metadata14:44
PiotrSikora:D14:44
redbopermanently temporarily14:44
creihtman... it has been a long time since I have looked at the object server :)14:45
gholtThe trick is, since we did it in the past, making sure only one version of the object meta+data is served at any time. Not a difficult trick, but something to keep in mind.14:45
redbothis is going to be a mess, now that I think about it.  Good luck.14:47
PiotrSikorawell, since files are versioned (and immutable, i assume?) then adding .meta to each version should handle that?14:47
notmynamePiotrSikora: so if I'm hearing you right, you want one big database, served by nginx modules, and doesn't require xattrs? ;-)14:49
creihtoh heh... the .meta file we store for updates is a 0 length file with the metadata in xattrs :)14:49
*** gundlach has joined #openstack14:49
PiotrSikoranotmyname: i never said anything about single database14:49
notmynamesomething about being able to query the system, though, right? ;-)14:50
PiotrSikoranotmyname: i was always talking about sharded RDBMS (drizzle/postgresql)14:50
creihthehe14:51
notmynameah ok. I'm just giving you a hard time14:51
redbocreiht: I was hacking on the object server this morning!  playing with python-sendfile.14:51
creihtoh noes! :)14:51
PiotrSikoranotmyname: which IMHO isn't much different from what you guys are doing... just using sqlite instead, its still sharded data in SQL14:52
PiotrSikoraerm... data in sharded SQL*14:52
notmynames/data/metadata14:52
PiotrSikoraright :)14:52
PiotrSikoracreiht: guess i can't re-use that .meta file then :)14:52
creihtPiotrSikora: not sure14:53
redboit's not central to our design that our databases aren't shared.  It just doesn't add anything, and sqlite is pretty nice that we can just rsync the files around and not worry about configuration or anything.14:53
*** avsm has quit IRC14:54
*** antonym is now known as spy14:54
creihtPiotrSikora: It is more like you would have to work around the .meta file :)14:55
*** spy is now known as Guest5328314:55
redboapparently freebsd supports xattrs14:57
creihtPiotrSikora: And if you got the object server updated, you would also have to make replication work with it14:57
creihtredbo: it does :)14:57
PiotrSikoraredbo: yeah, i got that last time :)14:57
PiotrSikoracreiht: yeah... i know, i know :)14:57
creihtPiotrSikora: The more I look at it, the more messy it seems14:57
redbo"OpenBSD 3.7 supported extended attributes, although support was never built into the default GENERIC kernel."14:57
notmynamewhat about using ZFS on FreeBSD?14:58
termiesoren: i second the "debugging things that make blocking io look like non-blocking io" statement, sends ripples of anticipatory fear through me14:58
redbooh they completely removed extended attributes14:58
PiotrSikoraredbo: yeah, no xattr in there ;)14:58
PiotrSikoranotmyname: ZFS has xattr14:58
redbozfs won't matter if there's no system call to use them :)14:58
*** littleidea has joined #openstack14:58
redbooh, does it have a userland library or something?14:58
PiotrSikoraerm?14:59
termiesoren: but i have to admit after 4 years of twisted i'd be pretty excited to try something else, that's initially one of the reasons we put so much work into tornado before deciding it wasn't worth it14:59
PiotrSikoraredbo: FreeBSD has xattr and zfs in base, OpenBSD doesn't have either14:59
PiotrSikoraneither*14:59
redboah, ok14:59
redboI didn't notice he said zfs on FreeBSD14:59
PiotrSikoraah, ok :)15:00
creihttermie/soren: It isn't as bad as it sounds15:00
jaypipesPiotrSikora: either was correct :)15:00
*** pcrews has joined #openstack15:00
creihtMuch better than trying to debug exceptions in long strings of callbacks :)15:01
jaypipescreiht: ++15:01
creihtI've done quite a bit of twisted development as well, and much prefer eventlet15:02
jaypipescreiht: or if you've ever had the disctinct pleasure of debugging C++ metaprogramming....15:02
creihthah15:02
* creiht doesn't miss C++ in general :)15:02
PiotrSikorajaypipes: thx :P15:03
jaypipesPiotrSikora: :)15:04
PiotrSikorabtw, since you guys mentioned ZFS, it would be probably killer FS for swift, since it has built-in versioning (via snapshoting) and replication (via send/recv)15:04
*** soren has quit IRC15:05
*** soren has joined #openstack15:05
*** ChanServ sets mode: +v soren15:05
creihtThe problem with ZFS is that it is in a very weird state right now, especially with NetAPP being so sue happy about it right now15:05
jaypipescreiht: Python is indeed a joy to code in. most of the time ;)15:05
* creiht may be a little bit tainted from his C++ days of having to use MFC15:06
PiotrSikoracreiht: and who they are going to sue? Oracle? ;)15:06
*** sophiap has joined #openstack15:06
creihtPiotrSikora: They have been suing people distrbuiting NAS systems that use ZFS15:06
creihtAnd it is still uncertain what Oracle is going to do with ZFS15:07
gholtHe really likes ZFS though.15:07
creihthaha15:07
PiotrSikoracreiht: they did? bastards!15:07
PiotrSikoracreiht: well, they'll probably kill it15:07
creihtyeah15:07
PiotrSikoralike they did with rest of their nice stuff15:07
creihtAs much as I like ZFS, I'm just not certain how much of a future it has15:08
PiotrSikorathat they got with Sun15:08
*** mray has quit IRC15:08
notmynameya. opensolaris seems dead too15:08
creihtindeed :/15:08
PiotrSikorayup :(15:08
notmynamemaybe we could switch to btrfs15:08
PiotrSikorajust when I started using it15:08
redbobtrfs deletes stuff so fast compared to xfs I thought it was broken15:09
*** mrayzenoss has joined #openstack15:09
*** mrayzenoss is now known as mray15:09
*** mray has joined #openstack15:09
creihthah15:09
jaypipesPiotrSikora: ya, opensolaris seems to be knocking on death's door...shame.15:09
notmynameI've got a home fileserver running opensolaris/zfs for all backups and media and whatnot. I'm hoping i can keep it running for a while15:09
jaypipesredbo: lol15:09
cory_when does redbofs come out?15:09
jaypipeshehe15:09
PiotrSikoraredbo: lol :D15:09
gholtBarfs, actually.15:09
jaypipes:)15:09
notmynamecory_: we've been asking that on the swift team for a while :-)15:09
creihtAnd then BartOS15:09
PiotrSikorabut doesn't btrfs have the same problems as zfs?15:10
PiotrSikoralegally speaking15:10
PiotrSikoraits also copy-on-write with checksuming on online fs checks15:10
* creiht has no idea, except it has the deep pockets of oracle behind it15:10
*** mray has quit IRC15:10
PiotrSikorathose were the problem that NetAPP have with it, AFAIR15:11
*** mrayzenoss has joined #openstack15:11
*** mrayzenoss is now known as mray15:11
*** mray has joined #openstack15:11
PiotrSikoraare*15:11
*** sophiap has quit IRC15:16
creihtPiotrSikora: replication is going to be interesting, since you have to make sure both the object file and metadata files are synced over, if one doesn't make it for some reason, it would need to be deleted15:18
PiotrSikoracreiht: are you trying to discourage me wrong working on that? :P15:19
PiotrSikoras/wrong/from15:19
creihtPiotrSikora: not at all, just trying to let you know what you are getting into :)15:19
PiotrSikoraduh, my english skills sux at the end of the month :P15:19
creihthah15:19
creihtPiotrSikora: Where are you from, if you don't mind me asking?15:19
PiotrSikoraPoland15:19
* PiotrSikora waits while creiht opens Google Maps15:20
creihthah15:20
cory_haha15:20
* creiht knows where pland is :)15:21
redboWe never forget Poland.15:21
creihtpoland15:21
* creiht actually has some Polish ancestry15:21
notmynameisn't it right between France and Ireland? ;-)15:21
PiotrSikoranotmyname :P15:21
notmyname(I actually do know the geography of Europe pretty well)15:22
*** mattf has quit IRC15:24
PiotrSikoraoh, rsync has --delay-updates, which should take care of "almost-atomic replication"15:31
creihthehe15:31
PiotrSikorait renames all files to final filenames once everything was transferred ok15:32
redboyeah, but we don't want that on all the time15:32
creihtThe only problem with that is that we rsync a partition at a time, which could be a lot of files15:32
termienotmyname: ireland is poland-lite15:32
PiotrSikoratermie: nope, but london is ;)15:33
termienotmyname: although i guess that it is getting less popular as a destination15:33
PiotrSikoracreiht: ugh... partition at a time?15:33
creihtyes15:33
PiotrSikoraguess i'll need to understand internals more before i'll try playing with .meta15:34
creihthehe15:34
pvois todd from anso around?15:34
pvodon't know his nick15:34
creihtpvo: he is usually xtoddx15:34
termiepvo: xtoddx15:34
pvoah, right15:35
*** mattf has joined #openstack15:39
*** lbieber has joined #openstack15:40
*** littleidea has quit IRC15:42
*** carolinad28 has joined #openstack15:43
*** metoikos has quit IRC15:46
*** sophiap has joined #openstack15:46
jaypipesPiotrSikora: don't worry, my Polish skills suck at the end of the month, too.15:47
PiotrSikorajaypipes :)15:48
jaypipestermie: so have all your email woes been fixed?15:48
*** metoikos has joined #openstack15:53
*** Guest53283 is now known as spy15:56
*** asksol has joined #openstack16:02
*** littleidea has joined #openstack16:10
*** holoway has quit IRC16:11
*** holoway has joined #openstack16:13
*** sophiap has quit IRC16:16
mtaylorlbieber: morning!16:21
lbiebermtaylor:  good morning!16:21
*** jdmaturen has joined #openstack16:22
*** jsmith-away is now known as jsmith16:38
*** littleidea has quit IRC16:40
*** blpiatt has joined #openstack16:58
*** jsmith is now known as jsmith-away17:09
*** blpiatt has quit IRC17:10
jaypipesvish1: around? question for you...17:17
*** kallistec has quit IRC17:20
*** Protocol_ has joined #openstack17:21
*** Protocol_ has left #openstack17:21
*** maximilien has quit IRC17:23
*** jc_smith has joined #openstack17:25
*** carolinad28 has quit IRC17:33
*** dendrobates is now known as dendro-afk17:43
*** dendro-afk is now known as dendrobates17:43
*** jdmaturen has quit IRC17:47
*** kallistec has joined #openstack17:50
*** mattf has quit IRC17:50
*** jdmaturen has joined #openstack17:50
*** carolinad28 has joined #openstack17:50
*** doublerr has left #openstack17:54
*** mattf has joined #openstack18:02
edaymtaylor: you drizzle post ended up on planet openstack :)18:02
mtayloreday: I broke tag filtering on my blog - fixing is on my todo list18:04
mtayloreday: I broke _rss feed_ tag filtering on my blog - fixing is on my todo list18:04
edayhehe18:04
mtaylorso sorry for the planet spam everyone :)18:04
mtayloreday: it's ok - it means my openstack posts will wind up on planetmysql too :)18:04
edaymtaylor: unacceptable, I'm removing you.18:05
* mtaylor quickly removes eday's commit access18:05
edaymtaylor: either that or you get 5 lashes18:06
* mtaylor quickly gives eday his commit access back, realizing he doesn't really want to have to directly fix eday's stuff himself18:06
*** maximilien has joined #openstack18:07
*** greenisus has joined #openstack18:09
*** spy is now known as antonym18:09
greenisushey guys, i have someone wanting to contribute a patch on the control panel side, but neither one of us knows how to do that with bzr :(18:10
greenisusanyone know how he would send me the equivalent of a git pull request18:10
*** subirjolly has joined #openstack18:11
creihtgreenisus: http://wiki.openstack.org/LifeWithBzrAndLaunchpad18:11
greenisusduh, i should have RTFM18:12
creiht:)18:12
*** tc__ has joined #openstack18:14
*** tc__ has quit IRC18:18
*** yyyeahTc has joined #openstack18:19
*** allsystemsarego has quit IRC18:19
uvirtbotNew bug: #611799 in openstack-web-control-panel "Incorrect Background Image URL" [Undecided,New] https://launchpad.net/bugs/61179918:21
mtayloreday: oh - I totally forgot I was playing with that vala echo server - are you gonna push your scalestack changes?18:21
sorengreenisus: There are the concept of "bundles", but really people should just push a branch to Launchpad. It's way easier.18:21
sorengreenisus: Where can I see the control panel stuff?18:21
*** tjyang_ has joined #openstack18:22
notmynamemtaylor: on http://wiki.openstack.org/AddingYourBlog there is reference to a heads dir, but the commands seem to mix it with the images dir. what should I be using?18:23
mtaylornotmyname: images18:23
mtaylornotmyname: fixing page18:24
notmynamehmm...ok. I need to rexeamine it then. I'm not seeing my mug show up18:24
greenisussoren: all the projects are here https://launchpad.net/openstack18:24
mtaylornotmyname: thanks!18:24
greenisussoren: OpenStack Web Control Panel, OpenStack iPhone App, OpenStack Android App, OpenStack iPad App18:24
sorengreenisus: Oh, sure, I meant screenshots and stuff.18:24
greenisusoh18:24
sorenSorry, I can see now that I was vague :)18:25
*** jsmith-away is now known as jsmith18:25
greenisussoren: here's one http://c0222312.cdn.cloudfiles.rackspacecloud.com/main-99.png18:25
mtaylorgreenisus: damn man. you're _still_ on my automation todo list too. sorry for the long delay. :)18:25
greenisusit's out of date, but that's the general idea.  it's what i was showing in austin when i was begging everyone for JSONP support18:25
greenisusit's all good mtaylor.  i'm still just catching up with bzr as i go along18:25
sorengreenisus: JSONP, as I understand it, is a JSON variant where you basically avoid having to eval stuff, but can just src it?18:26
greenisusjust so you know, at this point the control panel and mobile apps are speaking rackspace cloud api only18:26
mtaylorgreenisus: best way to do it ... needing to do something, then learning. :)18:26
greenisusmtaylor: that's exactly how i'm learning bzr18:26
greenisusand how i learned Android for that matter18:26
mtaylorthat's how I learn _everything_18:26
greenisussoren: that's right about JSONP.  basically you'd make an API call like GET /whatever?callback=myCallback.  then, the response would be: "myCallback({ the json for whatever you were asking for };"18:27
greenisusthat way, you can make API calls in a web app by inserting <script> tags into the DOM18:28
greenisusso ultimately, if we have JSONP support, we can make a control panel that's entirely html/css/javascript, with no backend at all18:28
creihtsoren: the main reason for it is to get around same-origin security in the browser18:28
greenisuswe could serve the control panel entirely from CDN18:28
greenisuscreiht is right.18:28
greenisusi was really hoping we could get JSONP, so i did this app with a dumb jsonp proxy in ruby that just does the JSONP style and passes it along to the real API18:29
greenisusbut if we do JSONP, the proxy can go away18:29
greenisusam i making sense?18:29
*** littleidea has joined #openstack18:29
edaymtaylor: ahh yeah, I never really got it working properly (vala echo). I'll get the scale stack changes up shortly18:30
mtayloreday: I've got the backlog thing fixed18:30
mtayloreday: I'm asking the vala people about the other...18:30
mtayloreday: I _think_ it may be vala utf8 encoding strings by default18:30
creihtThe only problem with JSONP (at least with current rackspace style auth) is that you would have to have your auth creds in the javascript18:30
mtayloreday: looking for a way to skip that18:30
justinsbAnyone seen this while running the Nova unit tests: ImportError: No module named amqplib.client_0_8.connection18:30
edaymtaylor: ahh18:31
mtayloreday: just a guess though18:32
greenisusyeah that's true.  hopefully we can get around that somehow18:32
*** rdw has quit IRC18:33
justinsb(answering my own question) ImportError is because I had Debian's python-amqplib package installed18:34
*** lbieber has quit IRC18:41
*** gustavomzw has joined #openstack18:43
sorengreenisus: Yes, making perfect sense.19:01
sorengreenisus: I see no reason we couldn't add jsonp support. It sounds like a perfectly reasonable idea.19:01
*** jsmith is now known as jsmith-away19:02
sorencreiht: I don't quite see why the auth stuff doesn't use an actual http cookie.19:02
greenisussoren: thank you :)  i'm really happy about it19:02
creihtsoren: Because that's not how rackspace auth works? :)19:03
sorencreiht: Instead of a piece of data that does the exact same thing. You get it from the server, and the server expect you to pass it with each request.19:03
sorencreiht: Right, exactly. I just don't get why it  doesn't. It seems /really/ weird to me.19:03
gholtCookies are not restful, right?19:03
greenisusyeah i'm assuming a <script> GET would send a cookie, right?  as long as we handled that....19:04
greenisusgholt: JSONP is not restful.  it's a hack to get around browser security policies19:04
gustavomzwirclogs is returning 403 - http://irclogs.openstack.org/logs/2010-07/%23openstack.29.log19:05
gholtAh, I was more trying to answer of why our auth didn't use cookies. But I did jump in at weird point in the conversation.19:05
creihtAlso in the long run, auth is going to be one of those things that is likely to vary a lot from installation to installation19:05
creihtsince most places already have some sort of auth that they will want to use19:06
greenisusright.  auth is one of the weird places for these projects.  i'm assuming we're going to have some sort of reference auth implementation though19:08
greenisusso that's what i'll go with for these apps19:08
greenisusat this point i'm assuming rackspace style, but i'd really like for us to do better19:08
greenisusi'd especially like for us to support some sort of human password auth19:08
greenisus(like for rackspace, letting someone auth to the api with their manage.rackspacecloud.com password)19:09
greenisusthe reason for that is that typing a huge API key on a cell phone is a miserable experience19:09
sorenOpenID + OAuth are high on my list of wishes for a authentication.19:09
creihtYeah, not sure what is going to happen on the auth side of things19:09
greenisusyeah i'm down with that19:09
creihtOAuth++19:09
notmynamegreenisus: swift has a reference implementation that is compatible with RAX auth19:09
sorenThat and HOTP/TOTP19:09
greenisusnotmyname: okay good.  i need to dig into that19:09
greenisushave to admit i've been very lightweight on looking at swift and nova19:10
mtayloreday: ooh. I think I may have gotten it working19:10
notmynamegreenisus: on a related note, that's one reason I use Cloud Mobile on the iPhone rather than your app19:10
greenisusnotmyname: exactly!  but i wasn't comfortable screen scraping for an API key19:11
notmynamenot sure that would be necessary. typing an API key is painful. i use 1password and copy/paste it19:11
notmynameCloud Mobile has 3 entry fields: username, api key, and auth server19:12
greenisusnotmyname: yeah, i copy/paste it too, but not everyone realizes they can do that.  so in the App Store i end up with all these bad reviews because no one wants to type the key19:12
greenisuswait, Cloud Mobile prompts for the API key as well19:12
notmynameya19:12
mtayloreday: pull from lp:~mordred/+junk/vala-echo-test and try again19:12
notmynamegreenisus: sorry, I was told I wasn't being clear. One reason I like Cloud Mobile better is because I can set the auth server19:15
edaymtaylor: it's saving the buffer, so no19:16
edaymtaylor: nc to it and type a few lines, you'll see what I mean19:16
mtayloreday: well piss19:16
mtayloreday: oh - you mean I need to make it flush the buffer before I read more into it don't you?19:17
edayoh, you're peeking, not reading19:17
edaymust be a simple way to just read instead of peek19:18
mtayloreday: potentially - lemme go look at api again19:18
sorenmtaylor: Did you have any ideas for the failing unit test on the buildd's?19:18
mtaylorsoren: the failedtobuild I saw looked like some weirdness with doc/build not being there ... is there something else you're referencing?19:21
sorenmtaylor: Look again19:22
sorenmtaylor: redis.exceptions.ConnectionError: Error 111 connecting 127.0.0.1:6379. Connection refused.19:23
sorenmtaylor: It wants a running redis-server.19:23
sorenMaybe this is less of a problem than I think it is.19:23
mtaylorsoren: ah yes! well, we should put in a start-redis command into debian/rules19:24
sorenDo buildd's have a policy-rc.d that stops them from running daemons?19:24
sorenmtaylor: Yeah, that will fix it on the buildd's, but probably make stuff weird on local dev boxes.19:24
mtaylorI'm not sure- in libmemcached and drizzle I have the test suite start daemons one a random open port and then use that port in the test suite19:24
*** stewart_ has joined #openstack19:25
mtaylorsoren: we should probably do something similar, because having make test connect to a system instance of a service is a good way to accidentally hose data when someone is doing a rollout and building from source19:25
*** stewart has quit IRC19:25
*** stewart_ is now known as stewart19:26
sorenmtaylor: That's totally what we should do.19:26
stewartmtaylor, i was thinking we should launch a private storage thing for drizzle tests of the cloud engine.19:26
sorenmtaylor: Sorry, I'm not thinking entirely clearly. It's been a long week.19:26
mtaylorsoren: yes. busy around here and stuff19:27
mtaylorstewart: we have a cloud engine?19:27
stewartmtaylor, a SoC project19:27
stewartmtaylor, also for pbms19:27
mtaylorstewart: but yeah - I'm working on hudson plugins to spin up cloud resources19:27
stewartmtaylor, if we can do it as part of the test run... it means people can actually (easily) run "make test" and test everything too19:28
stewart(assuming that installing a single machine small cloud storage thing isn't batshit insanely hard to do)19:28
greenisusnotmyname: oh i see.  i'll be adding that to the openstack app.  i never did for rackspace because it doesn't change (as far as i know)19:28
mtaylorstewart: well... right- we'll get that sorted - but I also want to test it in hudson against a real cloud thing19:28
*** rdw has joined #openstack19:29
greenisusnotmyname: i generally try to have as few settings as possible, because they come at a cost19:29
stewartmtaylor, yeah19:29
notmynamegreenisus: the only use case I know of was when were were testing swift internally and I wanted to point it at our staging env :-)19:29
notmynamebut that has an audience of about 419:29
notmynameit makes a lot of sense on the openstack app, though19:29
*** allsystemsarego has joined #openstack19:30
greenisusnotmyname:  gotcha.  you guys should have reached out to me :)  i would have sent you a configurable ad hoc build19:30
greenisusnotmyname: you also could have gotten the source at github if you have a mac19:30
notmynamegreenisus: while we're talking about features....  the other think I like about cloud mobile is the additional metadata that is shown about containers and objects19:31
*** jsmith-away is now known as jsmith19:31
greenisusnotmyname: yeah, i always meant to add that and never got around to it19:31
greenisusnotmyname: i want it on servers too, but they're not editable so i didn't do it19:32
notmynamegreenisus: will the openstack app allow for multiple accounts?19:34
greenisusnotmyname: yes.  that's currently in progress, and the ipad app already does19:34
greenisusnotmyname: eventually, i'll merge the iphone/ipad apps into a single universal app19:34
notmynamevery nice. sounds like I'm going to like it19:34
greenisusyeah i have a lot of big ideas for it.  right now i'm working on hooking it up to chef19:35
greenisusprovision nodes into a role so it can bootstrap itself19:35
greenisusreally hoping others get involved in the mobile apps so we can do more than just the stuff i come up with19:38
*** jakedahn has joined #openstack19:38
* notmyname needs to learn ObjC first19:38
greenisusnotmyname: learning objective-c and cocoa is definitely the biggest hurdle :)  i find most iphone devs would rather go off on their own trying to strike it rich on the app store anyway19:39
*** n0cturne has joined #openstack19:43
*** jsmith is now known as jsmith-away19:57
*** n0cturne has quit IRC19:58
*** n0cturne has joined #openstack19:58
*** Podilarius has quit IRC20:01
*** Podilarius has joined #openstack20:07
*** carolinad28 has quit IRC20:08
creihtmtaylor: how do I fix the conflic in https://code.launchpad.net/~cthier/swift/admin_guide/+merge/3141920:10
creiht?20:10
creihtconflice in .bzrignore20:11
uvirtbotNew bug: #611839 in swift "Add stats report documentation to admin guide" [Medium,Triaged] https://launchpad.net/bugs/61183920:11
creihtman I can't type today... must be friday20:12
*** justinsheehy has quit IRC20:12
mtaylorcreiht: you need to re-merge your branch with trunk, fix, and push again20:12
mtaylorcreiht: so when you merge with trunk, you should see the conflict in that file - fix it, run "bzr resolve .bzrignore" and then commit20:13
creihtk20:13
creihtso I need to branch from trunk, merge my branch into that new branch and then push that?20:13
creihtmtaylor: ok I think I got it20:19
*** antonym has quit IRC20:23
*** antonym has joined #openstack20:23
*** asimov.freenode.net sets mode: +v antonym20:23
*** snowboarder04 has quit IRC20:25
*** thinkscientist has joined #openstack20:25
*** pvo has quit IRC20:25
*** pvo has joined #openstack20:25
*** asimov.freenode.net sets mode: +v pvo20:25
*** lbieber has joined #openstack20:25
*** jonesy_ has quit IRC20:45
*** aliguori has quit IRC20:56
*** justinsheehy has joined #openstack20:56
gustavomzwfollowing http://wiki.openstack.org/InstallationNova20100729 - install_venv breaks installing python-gflags20:59
*** justinsheehy_ has joined #openstack21:01
*** justinsheehy has quit IRC21:02
*** justinsheehy_ is now known as justinsheehy21:02
*** allsystemsarego has quit IRC21:02
*** jakedahn has quit IRC21:03
*** jakedahn has joined #openstack21:03
*** jakedahn has joined #openstack21:03
*** n0cturne has quit IRC21:05
*** 64MAAAAEF has quit IRC21:06
*** aliguori has joined #openstack21:11
gustavomzwwhen I run pip install using with_venv.sh it installs21:11
jc_smithI had to do easy_install http://python-gflags.googlecode.com/files/python_gflags-1.3-py2.5.egg21:12
*** justinsheehy has quit IRC21:15
vish1anyone here?21:16
vish1I have a dependency question21:16
vish1should we put python-ldap in the debian dependencies?21:16
vish1my installs are choking on not having it21:18
vish1in general the question is, if we have optional plugins, do we install the deps for them?21:19
jc_smithdid you run /nova/auth/slap.sh21:20
gustavomzwinstall_venv using pip -E breaks, running pip install commands using with_venv.sh works21:22
*** sirp1 has joined #openstack21:23
*** gnrfan has joined #openstack21:23
creihtis there a reason to use gflags rather than the built in OptParse?21:26
*** justinsheehy has joined #openstack21:26
*** mtaylor has quit IRC21:28
jc_smithcreiht: it was part of the deps for nova, but I found this on the site with regard to OptParse: the ability to define flags in the source file in which they're used21:30
*** gnrfan has left #openstack21:30
creihtk.. just curious21:30
jc_smithcreiht: in swift, what is being stored in memcache via the proxy and via the auth server? looks like the proxy is caching response codes?21:32
jc_smithresult codes, rather21:32
jc_smithis this for like negative caching for people stating files that don't exist?21:33
creihtIn the proxy we store auth tokens, and account/container existance21:33
creihtI think we also store error counts, so that we can error limit nodes if it is erroring out too much21:33
* creiht goes to look21:34
*** jk0 has quit IRC21:34
*** jk0 has joined #openstack21:34
*** asimov.freenode.net sets mode: +v jk021:34
jc_smithoh, I can go dig, no worries21:34
creihtAnd we also use it for rate limiting21:34
jc_smithyou just seem to know everything about swift ;)21:34
*** justinsb_ has joined #openstack21:34
creihthah21:34
creihtI know little bits, and think I remember other bits21:35
*** jc_smith has quit IRC21:35
creiht:)21:35
creihtfor account/container existence we cache the result code21:35
*** zenmatt has joined #openstack21:35
creiht200, it is there, 404 it isn't :)21:36
*** jc_smith has joined #openstack21:37
creihtjc_smith: for account/container existence we cache the result code21:37
creihtthat way the check for existence is the same weather we got the result from a webservice call or from memcache21:38
jc_smithah, ok thats what I was looking at21:38
*** mattf has quit IRC21:38
creihtand I guess the error_limiting is in process21:38
creihtnot in memcache21:38
notmynameif my error limiting stuff is still in it, rate limiting uses memcache to store the current rate counts for accounts, etc21:39
creihtnotmyname: correct21:39
creihtjc_smith: and I like refreshing my memory, because you aren't the only one that comes in with questions :)21:39
jc_smithif I had two proxies, would there be two separate memcaches independant of one another?21:40
creihtNo the memcaches are shared accross the proxies21:40
creihtI actually have docs awaiting merge that explain some of this :)21:41
creihtThe memcached servers are in the proxy-server.conf21:41
notmynameyou should get someone to review it :-)21:41
* creiht smacks notmyname 21:41
creiht:)21:41
creihtjc_smith: https://code.launchpad.net/~cthier/swift/admin_guide/+merge/3141921:41
creihtif you want a preview :)21:42
jc_smithhmm, my proxy-server.conf doesn't have a memcache specified21:42
*** _buck has quit IRC21:42
jc_smithI assume that it uses the local one then21:42
creihtIt defaults to localhost if not specified21:42
creihtotherwise it is a comma separated list of ip:port21:43
notmynameI see a typo in your docs! I'll have to deny your merge request :-)21:43
creihthah21:44
notmynameline 279 of the diff21:44
creihthehe21:44
jk0:D21:45
gustavomzwInstallationNova20100729 is the cleanest guide21:45
*** tjyang has quit IRC21:45
jc_smithcreiht: great doc, cw an I were just talking this morning about what to do about disk failures and stuff, this will help a lot21:45
creihtcool21:46
creihtjc_smith: let me know if there are other questions21:46
jc_smiththanks21:46
creihtso we can get them in21:46
creihtThough, I may take a break from docs next week :)21:46
redbowe were gonna automate it, but then we were like pandemicsyn is on call all weekend anyway21:46
creihthaha21:47
creihtnotmyname: fix pushed... should be updated shortly21:47
* creiht waits for lp's eventual consistency to kick in21:48
*** justinsb_ has quit IRC21:48
notmynameyay21:48
*** jhalogen has quit IRC21:49
creihtand there it is21:53
redbohow do I merge trunk into my branch?21:56
creihthrm... mtaylor is gone already21:57
_0x44redbo: bzr merge lp:foo/trunk?21:57
notmynamecreiht: reviewed and approved21:57
creihtWith the way we do merge proposals, I wonder if it is alright to merge directly from trunk to your branch21:58
creihtnotmyname: awesome thanks21:58
redboshould be fine21:58
creihtredbo: if you have trunk locally you can try: bzr merge ../trunk21:59
creihtfrom your branch21:59
redbo_0x44: thanks, bzr merge lp:swift seemed to work21:59
*** Phil_FL has quit IRC21:59
_0x44redbo: No problem, please don't tell the ozone guys I helped ;)21:59
creihthah21:59
* jk0 saw it21:59
* jk0 is going to tell21:59
*** selenamarie has quit IRC22:00
*** selenamarie has joined #openstack22:00
* _0x44 is never going to hear the end of that. "You helped with bzr? Who are you?"22:01
creihthaha22:01
pvo_0x44: impostor!22:01
jc_smithcreiht: for memcache on the proxies, would it be best to put the ip of both proxies? so they are pointing to each other?22:02
creihtcorrect22:02
creihtSo in our case, we run memcache on all of our proxies, and each proxy-server.conf has the same line, listing the ip:port of all the memcache servers22:03
redboyou know, we probably don't cache anything in swift long enough for that memcached consistent hashing to have any point.22:03
_0x44pvo: I would have gotten away with it too, if it hadn't been for you meddling kids and your talking Wisconsiner too.22:03
creihtAs a side note, if this is on a public network, you might want to run memcache on a private network, if available22:04
jc_smithcreight: thanks22:04
creihtredbo: I think it helps for highly concurrent users, but in general you are probably right22:04
redbowell, modular hashing would distribute fine and take less work22:05
notmynameredbo is normally probably right22:05
creihtredbo: would it still work to handle failure cases sanely?22:05
redboyou think it handles failure cases sanely?22:06
creihtwell somewhat sanely22:06
*** littleidea has quit IRC22:06
creihtDoes it still handoff to another memcache server, if one of them goes down?22:06
redbothe only thing consistent hashing gives you is if you add or remove a server, less stuff moves around22:06
*** antonym is now known as spy22:06
creihttrue22:06
* creiht wonders why antonym likes to spy so much22:07
_0x44creiht: It's because he's required to maintain a 100' distance from most of us.22:07
creihtheh22:07
redboWe never add or remove memcached servers anyway.22:08
creihttrue22:08
spycreiht: finally reclaimed my nick22:08
creihtspy: ahh cool22:08
spyone of the freenode staff hadn't used it in 5 years22:08
spygave it to me when i asked for it heh22:09
redboand the once every bazillion years we do add or remove memcached nodes, we'd lose like... our auth token caches.22:10
creihthehe22:10
creihtMaybe at the time we were thinking that we would use memcache more?22:11
redboyeah22:11
redboeh, no compelling reason to remove it now though.22:13
creihtThere are some new docs up: http://swift.openstack.org/22:16
creihtNotably a start to an Administrator's Guide, and added some to the Deployment guide22:17
*** Podilarius has left #openstack22:19
*** gundlach has quit IRC22:22
*** joearnold has joined #openstack22:25
*** pvo has quit IRC22:26
creihtohhh... how do I get openstack in my nick?22:26
*** mray has quit IRC22:29
*** gustavomzw has quit IRC22:29
*** jdmaturen has quit IRC22:33
*** strtok has joined #openstack22:35
*** subirjolly has quit IRC22:36
rbergeronis the list of planet pages in the sidebar on planet.openstack.org pulled from planetplanet.org? or manually selected?22:38
jk0rbergeron: under the Subscriptions?22:48
rbergeronjk0: no, planetarium.22:48
rbergeronsorry, should have scrolled up to see the distinction. ;)22:49
rbergeronbut yes, the planetarium - the fedora link (sorry to toot the horn here) is like... woefully such a 2007 link, but I didn't want to bother anyone if it's a feed from elsewhere.22:49
jk0rbergeron: I think those are just sitting in the template, but I could be wrong22:56
*** lbieber has quit IRC22:58
*** msparks has joined #openstack23:01
msparkscontributing.23:01
*** anm_ has left #openstack23:04
*** thinkscientist has quit IRC23:17
*** pvo has joined #openstack23:18
*** pvo has joined #openstack23:18
*** ChanServ sets mode: +v pvo23:18
*** wattersjames has joined #openstack23:19
*** thinkscientist has joined #openstack23:22
*** littleidea has joined #openstack23:22
*** wattersjames_ has joined #openstack23:26
*** jakedahn has quit IRC23:27
*** wattersjames_ has quit IRC23:28
*** jonesy_ has joined #openstack23:28
*** wattersjames_ has joined #openstack23:29
*** jakedahn has joined #openstack23:29
*** wattersjames has quit IRC23:30
*** wattersjames_ is now known as wattersjames23:30
*** kevnfx has joined #openstack23:32
*** aliguori has quit IRC23:32
*** wattersjames has joined #openstack23:38
*** ghchinoy has left #openstack23:49

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!