Monday, 2010-10-25

gholtCool! And thanks! I don't know if someone already mentioned, but if you use the debian packages you'll get init scripts from there. Unfortunately, I don't think they're 100% atm; so that'll have to wait until they are.
uvirtbotNew bug: #666083 in nova "remove tornado.escape" [Undecided,New]
*** Cybodog has quit IRC00:13
*** masumotok has quit IRC00:22
mtaylorgholt: oh, are our init scripts not great?00:32
mtaylorgholt: actually, I've been meaning to make upstart scripts for ubuntu... (never enough time in the day)00:32
*** mtaylor has quit IRC00:32
*** sophiap has joined #openstack00:33
*** aliguori has quit IRC00:37
*** miclorb has quit IRC00:45
*** codejunkie has joined #openstack00:50
*** mfoemmel has quit IRC01:03
*** ar1 has joined #openstack01:03
*** zul_ has joined #openstack01:10
gholtmtaylor: Ah no. Reading back my line before I realize I forgot a key word or three. I meant the packages weren't 100% up to date. I don't see the latest version there yet, assuming we're all 1.1.0 official and such.01:23
*** mfoemmel has joined #openstack01:34
*** chmouel has quit IRC01:34
*** sandywalsh has quit IRC01:39
*** mtaylor has joined #openstack01:51
*** ChanServ sets mode: +v mtaylor01:51
*** codejunkie has quit IRC01:56
*** mtaylor has quit IRC01:59
*** miclorb has joined #openstack02:14
*** iammartian has quit IRC02:30
*** romain_ has joined #openstack02:39
*** romain_ has left #openstack02:39
*** ppetraki has joined #openstack03:09
*** miclorb has quit IRC03:16
*** ppetraki has quit IRC03:43
*** scarabx has joined #openstack03:44
*** joeallens has joined #openstack03:47
joeallenscan two swift proxies adress the same storage?03:48
gholtYes, in fact that's usually the case. As many proxy servers as needed behind a load balancer.03:51
*** scarabx has quit IRC03:53
joeallensthanks. how do they not step on each other? don't you need a kind of dlm to avoid that?03:56
*** ArdRigh has joined #openstack03:56
*** hazmat has quit IRC03:56
gholtdlm? distributed lock manager?03:56
gholtI'm not sure which part your thinking about, but no dlm is needed.03:58
gholtFor instance, if two concurrent PUTs for the same object occur, the last one wins once it completes.03:59
joeallensok, I was just thinking about two write accesses to the same file at the same time.03:59
joeallensok I see.03:59
gholtYes, there's a requirement that the proxy servers have very close times. The start time of the request indicates who wins.04:00
*** miclorb has joined #openstack04:00
joeallenshow do the proxies know about each other? and do they all manage the same set of rings?04:00
gholtThe rings are managed externally for the whole of the cluster. Once new rings are created, updated, they are pushed/pulled out to all the servers. The proxies don't actually need to know about one another, with the exception that usually they each also run memcached and share all of those.04:02
joeallensah, ok, so they find out about the avaialable targets when everyone else does...04:03
gholtYes, the way the ring works no two partitions are ever moved at the same time for a given object. With 3 replicas, even if a proxy hasn't received the latest ring yet, it'll still have the proper address to 2 of the 3 replicas.04:05
joeallensok i see. do you know why raid is discouraged? is it just not needed given the redundancy model or is there some specific penalty involved?04:06
gholtPerformance was dramatically poorer with raid in testing.04:07
*** kashyapc has joined #openstack04:08
joeallensah ok, does that mean software raid or an external array (something like fiber channel)04:09
gholtWith raid, if a drive fails you just replace it and the raid rebuilds. Which seems like a big win. But it took forever to rebuild (many hours). Without it, the drive unmount is detected and Swift starts replicating a new third copy of the affected objects, usually completing within an hour (depending on cluster size of course).04:10
gholtWell, I don't know the specifics on the hardware used (sorry). I do know it was a hardware raid controller, not software. And that a couple folks have played around with it with different hardware.04:11
joeallensI see, so you avoid rebuilds altogether and fail the drive immediately to invoke new copies. that makes sense. thank you for answering.04:12
gholtNo problem! I happen to have the computer nearby while watching some tv. :)04:12
*** joeallens has quit IRC04:13
*** dendro-afk is now known as dendrobates04:15
*** Grizzletooth has quit IRC04:43
*** dendrobates is now known as dendro-afk05:18
*** omidhdl has joined #openstack05:51
*** mfoemmel has quit IRC05:57
*** mfoemmel has joined #openstack06:06
*** DubLo7 has quit IRC06:08
*** BK_man has joined #openstack06:46
*** stewart has quit IRC06:55
*** guigui1 has joined #openstack07:01
*** Cybodog has joined #openstack07:01
*** miclorb has quit IRC07:06
*** stewart has joined #openstack07:12
*** ibarrera has joined #openstack07:13
*** befreax has joined #openstack07:13
*** calavera has joined #openstack07:13
*** omidhdl has quit IRC07:22
*** omidhdl1 has joined #openstack07:23
*** befreax has quit IRC07:28
*** stewart has joined #openstack07:29
*** omidhdl has joined #openstack07:33
*** omidhdl1 has quit IRC07:34
*** allsystemsarego has joined #openstack07:37
*** taihen has quit IRC07:50
*** mfoemmel has quit IRC07:59
*** krish has joined #openstack08:02
*** Ryan_Lane has joined #openstack08:14
*** Ryan_Lane has quit IRC08:17
*** miclorb_ has joined #openstack08:25
*** mfoemmel has joined #openstack08:28
*** ArdRigh has quit IRC08:29
*** miclorb_ has quit IRC08:35
*** ramkrsna has joined #openstack08:36
*** almaisan-away is now known as al-maisan08:37
*** sophiap has quit IRC08:43
*** taihen has joined #openstack08:46
*** irahgel has quit IRC09:03
*** irahgel has joined #openstack09:04
*** ramkrsna has quit IRC09:05
*** ramkrsna has joined #openstack09:19
*** miclorb has joined #openstack09:20
*** irahgel has quit IRC09:20
*** befreax has joined #openstack09:22
*** irahgel has joined #openstack09:30
*** irahgel has left #openstack09:41
vishysoren: are you around?09:42
vishysoren: I just discovered that if you do a virsh detach-disk on a logical volume, it leaves an open file descriptor09:43
*** irahgel has joined #openstack09:44
*** scaraffe has joined #openstack09:48
vishyreally really annoying09:55
*** chmouel has joined #openstack09:58
*** mfoemmel has quit IRC10:05
*** mfoemmel has joined #openstack10:17
*** befreax has quit IRC10:19
*** mfoemmel has quit IRC10:26
*** ambo has left #openstack10:26
*** veiko has joined #openstack10:28
*** scaraffe has left #openstack10:35
*** gustavomzw has joined #openstack10:38
*** kashyapc has quit IRC10:46
*** dendro-afk is now known as dendrobates10:49
*** ramkrsna has quit IRC10:50
*** miclorb has quit IRC11:03
*** itguru has joined #openstack11:14
*** mfoemmel has joined #openstack11:33
*** dendrobates is now known as dendro-afk11:34
*** ctennis has quit IRC11:36
*** dendro-afk is now known as dendrobates11:44
*** ramkrsna has joined #openstack11:46
*** westmaas has joined #openstack11:46
*** irahgel has left #openstack11:55
*** aliguori has joined #openstack11:59
*** sophiap has joined #openstack12:01
*** ctennis has joined #openstack12:12
*** ctennis has joined #openstack12:12
*** sophiap has quit IRC12:19
*** dendrobates is now known as dendro-afk12:22
*** hazmat has joined #openstack12:34
*** mischer has joined #openstack12:40
*** masumotok has joined #openstack12:42
*** BK_man has quit IRC12:42
*** itguru has quit IRC12:44
*** kashyapc has joined #openstack12:46
*** perestrelka has quit IRC12:47
masumotokHi there12:47
*** perestrelka has joined #openstack12:51
*** jdarcy has joined #openstack12:57
*** hazmat has quit IRC13:00
*** omidhdl has left #openstack13:02
*** hazmat has joined #openstack13:03
masumotokHas anyone encountered "ebiptables" problem?13:05
*** littleidea has joined #openstack13:06
masumotoklibvirtError: internal error Could not get access to ACL tech driver 'ebiptables'13:06
masumotokI installed ebtables iptables gawk13:06
sorenmasumotok: Which distro?13:07
masumotokUbuntu Lucid 10.0413:07
masumotokAustin release for nova13:07
masumotokI checked libvirtd source code, but "grep -r ebiptables libvirtd-0.7.5" does not show anything.13:08
*** hazmat has quit IRC13:09
*** zul_ has joined #openstack13:10
*** zul_ is now known as zul13:11
*** zul has joined #openstack13:11
*** zul has joined #openstack13:18
*** krish has quit IRC13:19
*** iammartian has joined #openstack13:24
*** mtaylor has joined #openstack13:29
*** ChanServ sets mode: +v mtaylor13:29
*** dendro-afk is now known as dendrobates13:29
*** krish has joined #openstack13:29
*** zul has quit IRC13:30
*** AimanA has joined #openstack13:32
*** hazmat has joined #openstack13:34
*** ramkrsna has quit IRC13:38
*** mischer has quit IRC13:44
*** zul has joined #openstack13:44
*** piken has joined #openstack13:44
annegentleGrizzletooth: thanks a bunch for the MultiServerSwift install info13:51
*** schisamo has joined #openstack13:51
*** hazmat has quit IRC13:56
*** irahgel has joined #openstack13:57
*** Podilarius has joined #openstack13:57
*** krish has quit IRC13:57
*** electroniceagle has joined #openstack13:59
*** pvo has joined #openstack14:00
*** ChanServ sets mode: +v pvo14:00
*** ppetraki has joined #openstack14:04
*** mordred_ has joined #openstack14:05
*** ChanServ sets mode: +v mordred_14:05
*** mtaylor has quit IRC14:06
*** Grizzletooth has joined #openstack14:10
*** al-maisan has quit IRC14:11
*** Grizzletooth has quit IRC14:11
*** gondoi has joined #openstack14:11
*** grizzletooth has joined #openstack14:11
*** mordred_ has quit IRC14:19
*** krish has joined #openstack14:20
*** zul has quit IRC14:25
*** zul has joined #openstack14:27
grizzletoothannegentle: glad to be of service - FYI I signed the CDA this weekend, and the project is certainly welcome to use that howto in distro or wiki or whatever.14:31
annegentleGrizzletooth: awesome. I'm pondering an RST to Wiki to RST process.14:32
annegentleGrizzletooth: I'll edit in Etherpad and hopefully get the RST into the code base today14:33
*** dendrobates is now known as dendro-afk14:34
*** zul has quit IRC14:38
*** rnirmal has joined #openstack14:38
*** spectorclan has joined #openstack14:42
grizzletoothannegentle: I noticed it is a little complicated trying to format back and forth between the Etherpad, RST, and the Wiki.  All different markup languages.  Looks like a would be useful.  As well as an etherpad->RST exporter.14:42
*** gundlach has joined #openstack14:42
*** pharkmillups has joined #openstack14:48
*** sirp has quit IRC14:48
*** sirp has joined #openstack14:49
annegentleGrizzletooth: yup, agreed.14:49
annegentleGrizzletooth: it's interesting, the Swift team uses Etherpad to collaborative author RST, then brings it into code. So we're doing their method basically.14:49
creihtRST is basically a text file14:50
creihtetherpad doesn't have any RST helpers, but the markup gets pretty easy the more you use it14:51
*** ttx has joined #openstack14:52
*** ttx has joined #openstack14:52
*** schisamo has quit IRC14:53
*** schisamo has joined #openstack14:53
*** silassewell has joined #openstack14:54
*** littleidea has quit IRC14:54
*** schisamo_ has joined #openstack14:54
*** hazmat has joined #openstack14:57
*** jarrod_ has quit IRC14:57
*** jarrod has joined #openstack14:57
*** mtaylor has joined #openstack14:58
*** ChanServ sets mode: +v mtaylor14:58
*** Daviey has quit IRC14:59
grizzletoothmy first draft was directly in Etherpad's built-in.  I exported to text and re-built the markup using RST, and pastebin'd that into the channel.  Then creiht re-loaded that into a named etherpad to do more editing :)15:00
*** schisamo has quit IRC15:00
grizzletoothon another topic...  does anyone have any quick hints on setting up the stats process on multi-server?  The config file seems generally assume it will be run on a server with access to the proxy logs, but it also wants access to an account-server config.  In my multiserver implementation those are on separate servers.15:00
*** kashyapc has quit IRC15:00
*** schisamo_ is now known as schisamo15:00
creihtnotmyname: -^ :)15:00
*** 52AACIHR4 has joined #openstack15:01
*** mtaylor has quit IRC15:01
*** ptremblett has joined #openstack15:01
*** nijaba has joined #openstack15:02
*** mischer has joined #openstack15:03
*** Daviey has joined #openstack15:05
notmynamegrizzletooth: you ready? I could talk for a long time about the stats stuff. I lived and breathed it for about 2 or 3 months15:06
creihtnotmyname: get a channel15:06
notmynamegrizzletooth: there are 3 parts to the stats system: 1) generate logs 2) upload logs to swift 3) process logs15:07
*** jkakar has joined #openstack15:07
notmynamepart 1 is done across the entire cluster15:07
notmynamepart 2 is done similarly to part 1, just as a second step (allows for more code reuse)15:08
notmynamepart 3 is done on a box with internal access to the cluster, but on one box (with lots of RAM and CPU)15:08
*** zul has joined #openstack15:08
notmynamegrizzletooth: which part are you curious about?15:09
grizzletoothnotmyname: I was trying to adapt the methods in to my multiserver howto15:09
grizzletoothnotmyname: the portion of the config that didn't make sense was the [log-processor-stats] section15:11
notmynameit's simple to me (because I wrote it), so please help me clarify where I am being obtuse. the instructions are the same, but where they are run changes in a multi-server config15:11
grizzletoothsince it assumes local access to the account-server configs15:11
notmynamethat is a stats system plugin15:11
creihtgrizzletooth: don't feel bad, he tells us the same thing when we ask questions about it :)15:11
notmynamegrizzletooth: not really15:11
grizzletoothI was assuming that I would run the log processor on the proxy server since there it had access to the proxy logs.  but sounds like I need to be running some version of it on all of the servers?15:12
vishymasumotok: maybe you already figured this out, but you need a much newer libvirt, try gettting the package from the nova-core ppa15:12
notmynameI mean, yes it does have references to "local" things. but that is because it's written to have one config file for all of the parts. just in some stages not all of the config options are used15:12
grizzletoothwhich in the case of the SAIO is the case15:13
*** pvo_ has joined #openstack15:13
notmynameyes, but it's the case for non-SAIO too15:13
notmynamelet me back up15:14
grizzletoothgak - I have to run out and work with battery and ups techs.   will have to learn more on this later15:15
notmynameour ops guys have one config file for the stats system that they put in svn, sync to all the machines, and use as an argument in all the scipts15:15
notmynamethe config file is a union of all the options for each of the 3 step15:15
*** allsystemsarego has quit IRC15:16
notmynamebut there is overlap, so 3 separate config files would repeat a bunch of stuff15:16
notmynamegrizzletooth: let me know when you have time15:16
*** gondoi_ has joined #openstack15:20
*** kashyapc has joined #openstack15:21
grizzletoothnotmyname:  okay, I think I get that part.  for the three phases you mentioned above, what named process gets run on each different type of server?15:24
*** gondoi has quit IRC15:24
*** littleidea has joined #openstack15:24
*** silassewell has quit IRC15:25
notmynamepart 1) the proxy servers generate logs (via syslong-ng to split them hourly). also the swift-account-stats-logger generates the account stats log files15:25
*** gondoi_ has quit IRC15:25
notmynamepart 2) swift-log-uploader15:25
notmynamepart 3) swift-log-stats-collector15:25
grizzletoothso all 1) and 2) run on the proxy server, and then 3) runs on a dedicated stats processor?15:27
notmynamefor the access logs, yes15:28
notmyname1 and 2 run on every account server box (every box storing account server data) for the stats logs15:29
notmynameand it written so that, if you want to, you can add your own plugin. we do this internally for CDN logs15:30
grizzletoothokay, so referring to the steps in your SAIO-stats howto:  1-7 on every proxy and account server.  8-9 on all account servers.  10 on all proxy servers.  11 on the stats processor box.  is that right?15:31
*** ttx has quit IRC15:32
grizzletootherr 7 only on proxy servers, not account servers.15:32
*** gondoi has joined #openstack15:32
notmynameerr..let me pull up that doc again :-)15:32
*** gondoi has quit IRC15:32
*** gondoi has joined #openstack15:33
*** dendro-afk is now known as dendrobates15:33
notmynamestep 1 is done once15:33
notmynamestep 2 on the proxy servers (at a minimum15:33
*** gondoi has quit IRC15:34
notmynamesteps 3 and 4 are for the proxy servers15:34
grizzletoothok, so the syslog breakouts don't apply to the account-servers.  got it15:34
*** adrian_otto has joined #openstack15:35
notmynameso step 5 is split15:35
notmynameanything local under [log-processor-stats] should be local to the account server boxes15:36
grizzletoothswift-account-stats-logger generates the log files that get put in /var/log/swift/stats?15:36
*** hazmat has quit IRC15:36
notmynamethey get put in log_dir (from the config)15:36
notmynameand for the cron jobs, access stuff is for the proxy servers and stats logs is for the account servers15:37
*** scaraffe has joined #openstack15:37
*** jkakar has quit IRC15:37
grizzletoothok, I think I have that.  what uses /etc/swift/stats.conf?15:38
notmynameand now you get to the dark side of this whole mess :-)15:39
notmynamestats is an overloaded term in the codebase now15:39
notmynamestats.conf actually has nothing to do with the stats system15:39
*** adrian_otto1 has joined #openstack15:40
grizzletooththat is for statistics about the overall operation of swift?15:40
grizzletoothas opposed to user accounting15:40
*** adrian_otto2 has joined #openstack15:41
notmynamegholt can explain15:41
edayjaypipes: you have access to the openstack hudson machine?15:41
grizzletoothokay, I will get to that later then.  plenty to work on now to get the stats collectors working on my setup :)15:41
*** adrian_otto has quit IRC15:41
grizzletoothok, will look at that gholt15:41
gholtThat may be the explanation you're looking for for swift-stats-report15:41
jaypipeseday: no.15:42
edayjaypipes: ahh, bummer. nice pug pic btw :)15:43
grizzletooththanks notmyname and gholt - will study and test some more15:44
*** adrian_otto1 has quit IRC15:44
jaypipeseday: :) thx15:44
*** kevnfx has joined #openstack15:45
*** 52AACIHR4 has quit IRC15:45
*** zul has quit IRC15:52
*** dendrobates is now known as dendro-afk15:53
*** electroniceagle has quit IRC15:56
*** adrian_otto2 has quit IRC15:57
*** dendro-afk is now known as dendrobates15:57
*** adrian_otto has joined #openstack15:58
*** metcalfc has joined #openstack15:58
*** hazmat has joined #openstack15:59
jaypipestermie: ++ :) /me likey.15:59
*** hazmat has quit IRC16:00
*** hazmat has joined #openstack16:00
*** ttx has joined #openstack16:04
*** calavera has quit IRC16:04
creihtjaypipes, termie: fwiw, a lot of that type of stuff is implemented in swift if you are interested in looking at it16:05
jaypipescreiht: a lot of it is implemented in Nova, too ;)16:05
jaypipescreiht: it's just that things aren't consistent...16:05
creihtIt is consistent within swift :)16:06
creihtjust noting in case there is anything useful16:06
grizzletoothnotmyname: should I run swift-account-stats-logger on all of the account-servers?  or just one?16:07
notmynameit only generates account stats logs for the machine it is running on (it walks the filesystem and finds the account dbs)16:07
gundlachjaypipes: in cleanup-unittests, your plan is to have nova/tests/ have a setUp() module level function that installs test fixtures and fake flags, and an equivalent tearDown(), right?16:08
grizzletoothah, ok, that makes sense.  forgot that not all accounts end up on each account server16:08
jaypipesgundlach: yep.16:09
jaypipescreiht: yep, noted :)16:09
*** littleidea has quit IRC16:09
gundlachjaypipes: if i say "cd nova; nosetests tests/api/" will the tests/__init__ setup() and teardown() still be run?16:09
notmynamegrizzletooth: another "Feature" is that you can not run step 3 and write your own parser. by that point all the logs are in swift, so it's a matter of downloading and tracking them16:09
jdarcyStatements like "patterns more consistently meet the expectations of developers for simple tasks" could spark an interesting conversation at summit.16:10
notmynamegrizzletooth: there is one clarification I need to make16:12
*** mordred_ has joined #openstack16:13
*** ChanServ sets mode: +v mordred_16:13
jaypipesgundlach: it should, yes.16:13
edaymordred_: there you are :)16:13
*** mordred_ is now known as mtaylor16:13
notmynameI may have spoken prematurely16:14
gundlachjaypipes: great, thanks.16:14
mtayloreday: morning16:14
edaymtaylor: what version of pep8 is on the hudson machine? it seems to be reporting issues, although 0.5.0 and 0.6.1 (ubuntu/pypi versions) don't16:14
edaymtaylor: morning, hows uds :)16:14
grizzletoothnotmyname: about the clarification, or the need to make it ;)16:15
notmynamethe need to make the clarification16:15
notmynamelooks like there is a config section in the code that is no longer used, but still there16:15
*** zul has joined #openstack16:16
notmynameI thought I still might be using it16:16
notmyname[account-stats] section in account-server.conf16:16
notmynamebut, it isn't referenced in the code anywhere16:16
mtayloreday: uds is great - I'll look at pep8 version in a sec16:16
* notmyname lunch16:16
grizzletoothnot referenced in my implementation16:17
edaymtaylor: cool, thanks. I also think we're ready for the merge check too, assuming it's just a version issue there16:17
*** Ryan_Lane has joined #openstack16:18
grizzletoothso it was successfully elided from the SAIO howto16:18
edaymtaylor: one of these days I should check out a UDS, you always speak so highly of them :)16:18
jaypipesmtaylor: heads up...
jaypipesmtaylor: I'll be proposing it at the summit...16:24
*** dendrobates is now known as dendro-afk16:26
mtaylorjaypipes: cool. I'll look at it in a sec - my laptop is running the project with my browser in another desktop16:28
jaypipesmtaylor: no worries :)16:29
*** pvo_ has quit IRC16:29
sorenvishy: libvirt does?16:30
vishysoren: yeah16:30
vishysoren: testing using python instead of virsh16:30
vishysoren: atm16:30
jaypipesgundlach: got any code to merge into cleanup-unittests?16:31
vishyafk for a few bbs16:31
*** silassewell has joined #openstack16:34
*** Ryan_Lane has quit IRC16:36
*** hazmat has quit IRC16:38
*** Cybodog has quit IRC16:42
sorenvishy: That won't change anything.16:43
*** ttx has quit IRC16:45
gundlachjaypipes: nothing worth mentioning.  I spent most of Thursday reading Twisted documentation trying to wrap my head around it enough to know how to clean up the Twisted tests... and failed :)16:45
gundlachI'm going through the low hanging fruit now.16:46
gundlachand then probably going to go on a docstring campaign so I can understand what's actually going on in the code.16:46
*** electroniceagle has joined #openstack16:46
jaypipesgundlach: :) you may want to check out termie's work here then:
sorengundlach: I'd be happy to help with Twisted, if needed.16:50
gundlachjaypipes: hey thanks :)16:50
gundlachsoren: cool, thanks.  well, until i started reading UnifiedServiceArchitecture I assumed we'd need to be converting Twisted unittests to run without the help of Trial; but perhaps the right thing to do is to leave Trial in place until everything's off of Twisted?16:50
sorenI don't see a good reason not to use trial.16:51
gundlachsoren: i think some of the inputs were that being able to use nosetests is a good thing; being able to run tests in just one part of the code without requiring hand-hacking of is nice; and not having another dependency is always nice.16:53
gundlachsoren: but as i understand it, Trial is mostly for Twisted testing, right?  And I'm assuming based on UnifiedServiceArchitecture that we're moving away from Twisted?16:54
*** guigui1 has quit IRC16:54
sorengundlach: Anyone is free to submit a blueprint. Just because someone did, doesn't mean it represents a decision of any kind.16:56
gundlachsoren: ah, i didn't realize that was a blueprint.16:56
sorenAlso, trial does (afaik) most of the interesting stuff nosetests also does.16:56
sorengundlach: Well, s/blueprint/wiki page/ on my statement, and it'll still be true.16:57
jaypipessoren, gundlach: it is a blueprint with a spec:
gundlachsoren: ok, well, i'm definitely for reducing the number of architectures we run upon, so i don't know what process it requires to come to a formal decision, but i'm happy to throw my effort into Getting It Done.16:58
jaypipessoren: I don't see a good reason *not* to just use Eventlet, to be honest.16:58
sorenjaypipes: I see a few, but that's really besides the point.16:58
sorenI'm just saying that just because there's a wiki page that lays out a plan for something, doesn't mean that we've (as a group) decided that it's a good plan or something we should pursue.16:59
jaypipessoren: agreed.16:59
gundlachsoren: agreed.  when jaypipes pointed me to that page, i assumed i had missed some meeting, and that we had decided to move in that direction.16:59
mtaylorjaypipes: yes. I agree17:00
mtaylorjaypipes: we need to figure out how to call hudson jobs from tarmac... which is on my todo list17:00
jaypipesmtaylor: cool17:00
* soren heads lunchwards17:01
* mtaylor also lunch17:01
*** mtaylor has quit IRC17:01
*** zul has quit IRC17:04
*** DubLo7 has joined #openstack17:05
*** masudo has joined #openstack17:07
*** gundlach has quit IRC17:13
spectorclanFun break, search for your location and add it to the OpenStack Community Page at
*** abecc has joined #openstack17:17
*** rlucio has joined #openstack17:17
*** abecc has joined #openstack17:17
*** ambo has joined #openstack17:20
*** ambo has left #openstack17:20
*** Ryan_Lane has joined #openstack17:22
*** jc_smith has joined #openstack17:30
*** littleidea has joined #openstack17:32
*** littleidea has quit IRC17:36
*** littleidea has joined #openstack17:43
*** pharkmillups has quit IRC17:48
* creiht thinks you guys should just use eventlet, but yeah that's really besides the point ;)17:51
*** adrian_otto1 has joined #openstack17:53
*** adrian_otto has quit IRC17:53
*** kevnfx has quit IRC17:55
*** sophiap has joined #openstack17:56
*** gundlach has joined #openstack17:56
*** adrian_otto has joined #openstack17:56
*** spectorclan has quit IRC17:57
*** sirp has quit IRC17:58
*** adrian_otto1 has quit IRC18:00
*** electroniceagle has quit IRC18:00
*** adrian_otto has quit IRC18:01
*** joearnold has joined #openstack18:01
vishysoren: I think i found a workaround for the attach issue18:02
*** mdomsch_ has joined #openstack18:03
*** scaraffe has quit IRC18:03
*** adrian_otto has joined #openstack18:03
edaycreiht: we are where it matters (API server). THe workers would probably be fine with simple threads/processes even (just a couple things reading the queue)18:03
* creiht is just poking at soren :)18:04
*** anotherjesse_ has joined #openstack18:05
*** anotherjesse_ has quit IRC18:07
*** electroniceagle has joined #openstack18:07
*** sirp has joined #openstack18:08
*** adrian_otto1 has joined #openstack18:10
*** anotherjesse_ has joined #openstack18:11
*** sirp has quit IRC18:12
*** adrian_otto has quit IRC18:12
*** sirp1 has joined #openstack18:12
*** spectorclan has joined #openstack18:14
vishywow, I don't know why that works18:16
vishyreally odd18:16
*** krish has quit IRC18:16
*** anotherjesse_ has joined #openstack18:21
*** zul has joined #openstack18:23
*** neogenix has joined #openstack18:23
*** neogenix has quit IRC18:24
*** neogenix has joined #openstack18:25
*** mtaylor has joined #openstack18:26
*** mtaylor has joined #openstack18:26
*** ChanServ sets mode: +v mtaylor18:26
*** masudo has left #openstack18:27
*** kw1 has joined #openstack18:30
*** mischer has quit IRC18:30
*** zul has quit IRC18:32
*** kakoni has joined #openstack18:32
kakonicongrats on the release!18:32
kakonicreiht: around?18:32
creihtkakoni: howdy18:33
*** londo has quit IRC18:34
kakonihowdy. question. I guess in the current swift design all the traffic goes thru the proxy server(s). Has there been some thought at anypoint to do this differently (lets say you would initiate the connection with proxy and the redirect to storage server...)?18:34
creihtThat question seems to come up often :)18:34
*** zul has joined #openstack18:35
*** ptremblett has quit IRC18:35
creihtDuring the initial development, we talked about several different designs which also included something like what you are asking about18:35
*** zaitcev has joined #openstack18:36
_0x44jaypipes: Did you have any comments about that blueprint?18:36
creihtThe current design was simpler, and also gave us a lot of flexibility and control18:36
creihtand it worked well enough that we haven't considered trying anything else18:37
creihtthere are a lot of smarts in the proxy that become more difficult if you hand the connection off to the backend servers18:37
creihtit is also easier to handle failure scenarios in a reasonable way18:38
creihtall that said, if someone comes up with a better idea/implementation we are all ears :)18:38
kakonivalid points. thanks for the detailed answer.18:39
*** pharkmillups has joined #openstack18:40
jaypipes_0x44: not yet, sorry :)18:42
_0x44jaypipes: No worries, I was just curious :)18:43
pikenIs there a way to have a vm mount a mount from the node it is on?18:46
*** zul has quit IRC18:46
*** niks has joined #openstack18:48
niksHi All,18:49
niksI have downloaded nova and trying to install it on RHEL VM18:49
niksis there a doc which i can use to proceed with install?18:49
niksi see current rhel instructions are more for swift..anything for nova @ documented yet?18:50
*** mischer has joined #openstack18:51
kakoniniks: btw, what rhel version do you have?18:56
annegentleniks: I haven't seen anything about nova with rhel yet on the wiki or mailing list in the last month or os18:57
annegentleer, so18:57
pikenShould be easy as long as you install all the pre-req's18:58
pikenredis, rabbit, and all python libs needed.18:58
pikenI had the most issue with gflags that was needed for redis on RHEL5.518:58
*** ttx has joined #openstack18:58
niksi am using centos 5.4 (not RHEL)18:59
pikenit is binary compatible18:59
jaypipes_0x44: ok, added some comments to that blueprint18:59
pikenniks: make sure you use epel yum repo as it will help with the prereq's for python19:00
_0x44jaypipes: Thanks!19:00
jaypipespiken: I don't think gflags is a requirement for Redis... but it IS a requirement for Nova.19:01
jaypipes_0x44: lemme know if the comments make sense :)19:01
nikspiken: ok..where do i get other prerequs redis and rabbit mq19:01
pikenjaypipes: To build redis I need gflags c lib installed.19:01
jaypipespiken: hmm, I see...19:01
pikenniks: redis you have to build from source. rabbit is in epel19:02
jaypipespiken: so not the python-gflags19:02
pikenjaypipes: That was solved for nova when I built gflags for redis as it includes the python and ruby bindings when you do the install.19:02
nikspiken: will do..thanks19:02
pikenniks: np19:03
annegentleniks: scroll to the bottom of for Centos instructions (with wget for python-gflags)19:03
jaypipesgundlach: FYI, I pushed a few more revisions with more unit tests of that new test utils module to cleanup-unittests... so bzr merge em in when you want (shouldn't overlap anything you are doing)19:03
jaypipespiken: gotcha :)19:03
gundlachjaypipes: thx.  my work's in ~gundlach/nova/cleanup-unittests fyi.19:03
jaypipesgundlach: sweet.  can I merge it?19:03
*** AimanA has quit IRC19:03
gundlachjaypipes: should be able to; be aware that cloud_unittest breaks until the module-level fixtures are in place to wipe the database between tests.19:04
jaypipesgundlach: gotcha.19:04
_0x44jaypipes: Good call on the testing problem, neither sirp or I had thought of that.19:04
pikenpretty soon libcloud will support openstack directly as I am building the driver set as an extenstion to the rackspace as euca has one extending ec2.19:04
pikenJust a heads up for all19:04
niksannegentle: looks good to get me let me know if there is something that you know is out there...will let you guys know how it goes :)19:04
jaypipesgundlach: merged and pushed to my lp branch with commit comment "Merge Gundy."19:05
vishysoren: so the fd issue goes away if i use the python libvirt conn to attach and detach19:05
jaypipes_0x44: yeah, otherwise in order to test, we'll basically have to install every library...which is kinda annoying ;)19:05
*** jkakar has joined #openstack19:05
vishyperhaps simple_execute isn't closing the session to virsh properly so it leaves the file open?19:05
gundlachjaypipes: great!19:05
*** rlucio has quit IRC19:06
*** westmaas has quit IRC19:07
*** westmaas has joined #openstack19:08
*** dendro-afk is now known as dendrobates19:08
*** westmaas has quit IRC19:08
*** zul has joined #openstack19:09
*** westmaas has joined #openstack19:09
*** mischer has quit IRC19:10
*** ppetraki has quit IRC19:10
*** AimanA has joined #openstack19:12
*** rlucio has joined #openstack19:12
*** londo has joined #openstack19:13
*** AimanA has quit IRC19:14
*** AimanA has joined #openstack19:15
*** miclorb has joined #openstack19:16
*** anotherjesse_ has quit IRC19:16
*** CodeBlock[s] has joined #openstack19:19
*** kakoni has quit IRC19:21
sorenvishy: Perhaps. I seriusly doubt just using the python bindings would make a difference. The python bindings are a very thin wrapper around the C lib.19:22
*** miclorb has quit IRC19:23
*** Ryan_Lane is now known as Ryan_Lane|food19:24
gundlachjaypipes: hmm, looks like nose module-level setup is run once, before all tests are run in the module.  your goal was for reset_test_environment() to be called before every test, i believe?19:27
jaypipesgundlach: yeah, was just going to put that into a base test case class for use when the test data environment needed to be wiped between each test case execution.19:30
gundlachjaypipes: fair enough.19:30
jaypipesgundlach: as well as use nose's @with_setup decorator for more custom stuff.19:30
*** electroniceagle has quit IRC19:32
*** electroniceagle has joined #openstack19:33
edaysoren: you have access to the hudson machine don't you? can you see what version of pep8 is on there?19:36
*** littleidea has quit IRC19:38
gundlachs/get off of Twisted/get off of Trial/19:40
*** spectorclan has quit IRC19:48
*** ctennis has quit IRC19:50
*** dendrobates is now known as dendro-afk19:50
*** neogenix has quit IRC19:52
*** westmaas has quit IRC19:52
*** hornbeck has joined #openstack19:55
vishysoren: I blame threading, but it could be something in the way that virsh attach-disk is implemented, I notice that the xml shows driver name='qemu' instead of name='phy'19:57
*** pvo_ has joined #openstack19:58
*** pvo_ has joined #openstack19:58
*** ChanServ sets mode: +v pvo_19:58
*** ttx has quit IRC19:59
*** pvo_ has quit IRC19:59
*** pvo_ has joined #openstack20:00
*** ChanServ sets mode: +v pvo_20:00
*** pvo has quit IRC20:00
*** pvo_ is now known as pvo20:00
*** anotherjesse_ has joined #openstack20:01
*** pvo has quit IRC20:01
*** pvo has joined #openstack20:02
*** ChanServ sets mode: +v pvo20:02
*** ptremblett has joined #openstack20:02
*** zul has quit IRC20:03
*** ctennis has joined #openstack20:03
*** ctennis has joined #openstack20:03
pikenIs there a way to have a vm mount a directory from a mount on the node?20:08
*** jdmaturen has joined #openstack20:08
pikenI want to take advantage of the infiniband for lustre even though kvm doesn't support infiniband20:08
*** electroniceagle has left #openstack20:13
*** pvo has quit IRC20:14
*** vnaren has joined #openstack20:15
*** joearnold has quit IRC20:21
*** silassewell has quit IRC20:23
*** Ryan_Lane|food is now known as Ryan_Lane20:25
*** pvo has joined #openstack20:26
*** pvo has joined #openstack20:26
*** ChanServ sets mode: +v pvo20:26
*** pvo has quit IRC20:26
*** pvo has joined #openstack20:27
*** pvo has joined #openstack20:27
*** ChanServ sets mode: +v pvo20:27
jaypipeseday: on a bug-status-changing rampage today, eh!? :)20:28
vishypiken: you can attach block devices to vms, i don't know about straight dirs20:29
edayjaypipes: trying to clean up blueprints/bugs for summit :)20:31
jaypipeseday: nice job :)20:32
edayjaypipes: I figure this is a good janitor week.. pep8/pylint/lp cleanup :)20:32
jaypipeseday: yup! badly needed...20:33
jdarcyThat was for piken - paravirtualized FS for qemu-based virt environments.20:34
pikenjdarcy: ty very much. That should do what we want. :)20:34
*** Ryan_Lane has quit IRC20:36
*** dendro-afk is now known as dendrobates20:36
pikennow to find out a way to get that option added to the vm creation line for openstack along with custom params based on the user creating the vm20:36
*** hazmat has joined #openstack20:37
*** joearnold has joined #openstack20:37
*** zul has joined #openstack20:38
*** miclorb has joined #openstack20:40
*** Ryan_Lane has joined #openstack20:40
grizzletoothgak - cyberduck's problem reading objects from swift with slashes in their name is annoying20:41
*** sirp1 has quit IRC20:42
vishypiken: it would be great to have a volume driver that can use it20:42
*** kevnfx has joined #openstack20:44
*** joearnold has quit IRC20:44
pikenWell, I will most likely have to write it and submit it as it will be needed for my project since kvm doesn't support virtual infiniband devices20:44
*** ttx has joined #openstack20:44
*** silassewell has joined #openstack20:47
*** joearnold has joined #openstack20:48
*** ppetraki has joined #openstack20:49
*** sirp1 has joined #openstack20:50
*** arthurc has quit IRC20:53
*** gundlach has quit IRC20:55
*** gaveen has joined #openstack20:58
*** gaveen has joined #openstack20:58
vishyiscsi branch proposed:21:03
sorenvishy: *awesome*21:03
vishysoren: i noticed in the xen libvirt xml that it attaches disks to sda instead of vda?  Xen doesn't support virtio?21:04
sorenvishy: I'm.... not sure.21:05
vishysoren: also, i'd really like to get rid of this:              <!--   <model type='virtio'/>  CANT RUN virtio network right now -->21:06
vishysoren: but only the maverick kernel actually works with virtio network turned on21:07
*** abecc has quit IRC21:07
*** jdarcy has quit IRC21:08
kevnfxHas there ever been any discussion in the team wrt using/integrating Openvswitch/OpenFlow with Nova?21:09
vishysoren: I'm doing this ugly hack at the moment in deploy:
vishykevnfx: it has been mentioned as a nice to have a few times, but i don't think there is an official blue print21:10
Ryan_LaneI have a few patches to submit. how do I go about that for this project? :)21:11
Ryan_LaneI was told I need to submit something?21:11
vishyRyan_Lane: have you signed the cla?21:11
Ryan_Lanehow do I go about that21:11
kevnfxvishy: cool ... perhaps something I'm interested in looking into ....21:12
kevnfxjust wanted to make sure it hadn't already been done or dismissed ...21:15
pvokevnfx: yea, we've been talking about it.21:16
pvokevnfx: but haven't moved to start working on it yet.21:16
kevnfxpvo: was this a private discussion, or transcripts I can find in the IRC history?21:17
sorenvishy: Yeah, I wish there was a cleaner way.21:18
sorenvishy: Ideally, there would be a clever way for the guest to switch the disk from IDE to virtio on the fly.21:18
vishysoren: we can stick a flag in kernel i suppose, but that fails as soon as we have whole disk images.21:19
vishysoren: is that possible?21:19
sorenvishy: Not at the moment, no.21:19
zulvishy: pv xen does support xvd21:19
sorenvishy: I've wanted to do it for years, but never had the time.21:19
*** rlucio_ has joined #openstack21:20
*** DubLo7 has quit IRC21:21
*** ttx has quit IRC21:22
pvokevnfx: this was "we" being rackspace.21:22
*** sanjib has joined #openstack21:22
pvokevnfx: but if you want to get a leap on us, feel free. : )21:23
kevnfxpvo: understood ... just wanted to try and leverage any previous discussion or requirements banter ...21:24
*** rlucio has quit IRC21:25
*** rlucio_ is now known as rlucio21:25
*** pothos has joined #openstack21:28
*** joearnold has quit IRC21:28
*** gundlach has joined #openstack21:29
zulvishy: the xen pv-ops kernel uses the xvda1 devices etc21:29
*** pothos_ has quit IRC21:30
vishyzul: interesting21:30
*** ttx has joined #openstack21:33
*** ttx has joined #openstack21:33
*** kevnfx has quit IRC21:35
*** joearnold has joined #openstack21:36
*** zul has quit IRC21:38
* eday pokes mtaylor :)21:39
mtaylorhey eday21:41
mtayloreday: was I supposed to be looking at something for you?21:41
mtayloreday: oh! pep821:41
mtayloreday:   Installed: 0.4.2-321:41
edaymtaylor: ahh, would it be a pain to upgrade?21:42
mtaylorit depends21:42
mtayloreasiest way would be to upgrade the hudson box to maverick21:42
edaypypi would be better, as it's 0.6.1 (maverick is only 0.5.0)21:43
edayso probably easier too21:43
niksdo i need tornado for nova install?21:44
*** dysinger has joined #openstack21:45
edayniks: not as of today, the last dependency was just removed21:46
edayniks: but the release still had a dep on it with the object server21:46
nikseday: I am installing austin release..did that had the dependency on it?21:46
edayniks: yes21:47
niksoh! ok..well trying to install nova on centos 5.4 which comes with python 2.4 but looks like tornado needs python 2.5+21:48
nikslooks like i would have to upgrade python to get it working :(21:48
vishybtw, I'd like to state on record that i find instance-342423534524 highly annoying21:49
edayvishy: make a blueprint, I think we should discuss canonical/secondary ID management :)21:49
vishyeday: ok.  I think we need to be able to specify which id to use for the 'name' value, since that shows up in libvirt/filesystem etc.21:50
vishyI'm stuck with my standard command line ui which uses ec2_id for everything21:51
*** sophiap has quit IRC21:54
*** ttx has quit IRC21:55
*** dendrobates is now known as dendro-afk21:55
kw1niks, EPEL has pre-built python2.6 distros for CentOS 521:57
kw1that run alongside the system python21:57
kw1that work well for tornado21:57
nikskw1: did you mean
kw1nah, EPEL21:58
niksgot it21:58
kw1install their RPM to get the distro, then 'yum search python26'21:58
nikskw1: :)21:59
*** Grizzletooth_ has joined #openstack22:01
*** rnirmal has quit IRC22:02
*** jkakar has quit IRC22:05
vishykw1: have you managed to get nova running on centos?22:06
kw1no, haven't tried; I'm new22:07
*** metcalfc has quit IRC22:10
*** hazmat has quit IRC22:11
*** stewart has quit IRC22:13
edaysoren, dendro-afk: could one of you mark as implemented? was the ORM branch22:19
niksi have installed all dependencies mentioned in bottom part of idea what next ?22:20
*** rlucio has quit IRC22:21
edaysoren, dendro-afk:   this one too please :)22:21
*** metcalfc has joined #openstack22:23
*** jc_smith has quit IRC22:24
*** metcalfc has joined #openstack22:24
nikshi..i get following error while trying to build austin release of nova22:28
niks  File "", line 37     with open("ChangeLog", "w") as changelog_file:SyntaxError: invalid syntax22:28
*** silassewell has quit IRC22:29
niksany idea what could be wrong??22:32
*** vnaren has quit IRC22:34
vishyniks: is unneccessary22:34
vishyniks: if you're just trying to run it22:34
*** pvo has quit IRC22:35
niksvishy: All dependencies seems to be what? python install also seems to throw lot of Syntax errors22:36
niksam i doing something wrong? is it expected to work against Python 2.4 or do i need other version of python?22:37
*** gundlach has quit IRC22:38
*** niks is now known as niks222:39
*** niks2 is now known as niks22:39
vishythat is odd22:40
*** dysinger has quit IRC22:40
*** mtaylor has quit IRC22:40
vishyniks: no 2.622:40
vishyit could probably be modified to work against 2.5 with some changes22:40
vishy2.4 will be a big pain though22:40
niksvishy u mean it needs 2.6 or it doesnot need 2.622:41
vishyniks: it needs 2.622:41
*** hazmat has joined #openstack22:42
*** jc_smith has joined #openstack22:42
*** bwalk7125 has joined #openstack22:43
*** zuhaib has joined #openstack22:43
*** lukey has joined #openstack22:46
*** lukey has left #openstack22:47
*** hazmat has quit IRC22:49
*** joearnold has quit IRC22:52
*** jc_smith_ has joined #openstack22:54
Ryan_LaneI've added a branch for ldap support improvements:
*** joearnold has joined #openstack22:56
Ryan_LaneI moved the openldap schema out of, and added the sun (opends/opendj/sun directory server/fedora ds) schemas22:56
*** jc_smith has quit IRC22:57
*** jc_smith_ is now known as jc_smith22:57
*** DubLo7 has joined #openstack23:01
*** adrian_otto1 has quit IRC23:02
*** metoikos has joined #openstack23:04
uvirtbotNew bug: #666554 in nova "_inject_net_info_fs in compute/ fails if /etc/network doesn't exist" [Undecided,New]
*** stewart has joined #openstack23:19
edaywhew, I think that's the last of the bug/blueprint cleaning23:23
*** mtaylor has joined #openstack23:23
*** mtaylor has joined #openstack23:23
*** ChanServ sets mode: +v mtaylor23:23
*** littleidea has joined #openstack23:24
*** DubLo7 has quit IRC23:24
*** mtaylor has quit IRC23:27
*** Ryan_Lane has quit IRC23:29
*** Ryan_Lane has joined #openstack23:30
anotherjesse_eday: wow, thank god23:30
*** pvo has joined #openstack23:31
*** ChanServ sets mode: +v pvo23:31
*** Podilarius has left #openstack23:36
*** hornbeck has quit IRC23:41
edayvishy: you would make a good salesman :)  (nova-debug prop)23:42
vishyeday: hehe, i was getting a little punchy :)23:43
*** sirp1 has quit IRC23:51
nikshmm..had to install all modules again in 2.623:56
niksnova-manage user now throwing errors23:56
niksAttributeError: 'module' object has no attribute 'BrokerConnection'23:56
niksany guesses guys ? :)23:57
*** arcane has quit IRC23:58
*** arcane has joined #openstack23:59
*** RamD has joined #openstack23:59

Generated by 2.14.0 by Marius Gedminas - find it at!