Wednesday, 2010-10-27

*** rds__ has joined #openstack00:00
*** Ryan_Lane has quit IRC00:03
*** rds__ has quit IRC00:09
*** Cybodog has quit IRC00:09
*** rds__ has joined #openstack00:10
*** hogstrom_ has joined #openstack00:13
*** tedd has quit IRC00:14
grizzletoothdo any swift servers, other than proxy-server, use memcache?00:14
*** hogstrom has quit IRC00:18
*** hogstrom_ has quit IRC00:18
xtoddxgirzzletooth: check what service definitions include the memcache middleware, i don't know off-hand00:23
*** rds__ has quit IRC00:24
*** rds__ has joined #openstack00:25
*** joearnold has quit IRC00:26
grizzletoothxtoddx: I grepped through the source tree, and saw references to proxy/server and a bunch of common/middleware/* stuff00:26
*** kw1 has left #openstack00:26
*** bwalk7125 has quit IRC00:27
*** adjohn has joined #openstack00:27
*** kevnfx has quit IRC00:28
*** blakeyeager has quit IRC00:29
xtoddxgrizzletooth: grepping for memcache in the etc/ directory, it looks like proxy is the only one that loads it00:30
*** schisamo has quit IRC00:32
grizzletoothxtoddx:  the part that started me looking was this phrase in the docs:  "Several of the Services rely on Memcached for caching certain types of lookups..."00:37
grizzletoothxtoddx: I knew that proxy used it, but wondered what others would make "several"00:37
grizzletooth  "Memcached Considerations"00:38
xtoddxgrizzletooth: hrm, i think it is used for auth caching, and that is always done at the proxy level before the request gets forwarded.  maybe creiht can chime in, though.00:38
*** hazmat has quit IRC00:39
*** jc_smith has quit IRC00:41
*** kevnfx has joined #openstack00:45
*** Talvine has quit IRC00:49
creihtdwight_: I'm going to update the doc to do some better logging00:53
creihtgrizzletooth: I'm pretty sure only the proxy uses the memcache00:53
dwight_creiht: cool.00:54
creihtI was hoping to do that today, but got side tracked working on some other stuff00:55
creihtThere are a couple of other things that I would like to update the doc with as well00:56
creihtrate limiting uses memcache, but that is on the proxy00:57
creihtand some of the stats stuff will use memcache when uploading the logs to swift00:57
creihtactually scratch that, the stats stuff caches locally, so it doesn't use memcache00:58
*** ar1 has joined #openstack01:10
*** jdmaturen has quit IRC01:14
*** miclorb has quit IRC01:17
*** Dweezahr has quit IRC01:26
*** Dweezahr has joined #openstack01:27
*** ambo has quit IRC01:34
*** ambo has joined #openstack01:35
*** silassewell has joined #openstack01:36
*** rds__ has quit IRC01:47
*** neogenix has joined #openstack01:52
*** gaveen has joined #openstack01:55
*** miclorb has joined #openstack01:55
*** kw1 has joined #openstack01:56
*** miclorb has quit IRC01:56
*** miclorb_ has joined #openstack01:57
*** kevnfx has quit IRC02:00
*** kw1 has quit IRC02:01
*** miclorb__ has joined #openstack02:09
*** miclorb_ has quit IRC02:12
*** metcalfc has quit IRC02:13
*** Talvine has joined #openstack02:19
*** Talvine1 has joined #openstack02:22
*** Talvine has quit IRC02:23
*** Talvine1 has quit IRC02:24
*** kashyapc has quit IRC02:30
*** miclorb__ has quit IRC02:42
*** hogstrom has joined #openstack02:46
*** Cybodog has joined #openstack02:51
*** littleidea has quit IRC02:56
*** kevnfx has joined #openstack03:00
*** joearnold has joined #openstack03:02
*** blakeyeager has joined #openstack03:02
*** miclorb_ has joined #openstack03:02
*** perestrelka has joined #openstack03:20
*** kashyapc has joined #openstack03:36
*** BK_man has joined #openstack03:41
*** kashyapc has quit IRC03:45
*** gaveen has quit IRC03:56
*** gaveen has joined #openstack03:57
*** mtaylor has joined #openstack04:00
*** ChanServ sets mode: +v mtaylor04:00
*** BK_man has quit IRC04:01
*** sophiap has quit IRC04:06
*** gaveen has quit IRC04:09
*** metcalfc has joined #openstack04:12
*** coredump|br has quit IRC04:15
*** kashyapc has joined #openstack04:26
*** adjohn has quit IRC04:30
*** BK_man has joined #openstack04:31
*** gaveen has joined #openstack04:33
*** sirp1 has quit IRC04:33
*** coredump|br has joined #openstack04:46
*** toddmorey has quit IRC04:51
*** blakeyeager has quit IRC04:52
*** miclorb_ has quit IRC04:56
*** miclorb has joined #openstack04:56
*** metcalfc has quit IRC04:57
*** littleidea has joined #openstack05:01
*** Orman has joined #openstack05:06
OrmanHey guys05:06
Ormanannegentle: How are you man?05:07
*** Orman_ has joined #openstack05:10
Orman_I've been doing some documentation writing on the security side of Nova.05:11
*** Orman has quit IRC05:12
*** Ryan_Lane has joined #openstack05:14
Orman_Hey guys been working the security documentation for Nova. ;D05:17
*** dendro-afk is now known as dendrobates05:19
*** Orman_ has quit IRC05:25
*** kevnfx has quit IRC05:33
*** joearnold has quit IRC05:37
*** omidhdl has joined #openstack05:41
*** joearnold has joined #openstack05:43
*** miclorb has quit IRC05:45
*** kainam has joined #openstack05:50
*** arcane has quit IRC05:50
*** Podilarius has quit IRC05:59
*** adjohn has joined #openstack06:03
*** EdwinGrubbs has quit IRC06:06
*** arthurc has joined #openstack06:09
*** EdwinGrubbs has joined #openstack06:10
*** EdwinGrubbs has joined #openstack06:10
*** guigui1 has joined #openstack06:10
*** omidhdl has left #openstack06:15
*** joearnold has quit IRC06:30
*** guigui1 has quit IRC06:50
*** guigui has joined #openstack06:51
*** calavera has joined #openstack07:08
*** miclorb_ has joined #openstack07:12
*** ibarrera has joined #openstack07:29
*** coolhandluke has joined #openstack07:32
edaymtaylor: thanks for fixing pep8. ready to have it check for 0 now on merge if we can add that before test suite runs or something07:36
*** allsystemsarego has joined #openstack07:37
*** Ryan_Lane has quit IRC07:37
mtayloreday: done07:42
mtayloreday: now you just need to clean pylint :)07:44
vishythat one will take longer :)07:44
mtaylorwell - you guys will now block on pep8 violations07:45
*** arcane has joined #openstack07:47
*** kainam has quit IRC07:48
*** guigui has quit IRC07:53
*** matiu has quit IRC07:53
*** matiu has joined #openstack07:54
*** matiu has joined #openstack07:54
*** guigui1 has joined #openstack08:03
*** BK_man has quit IRC08:19
*** BK_man has joined #openstack08:22
*** ramkrsna has joined #openstack08:26
*** ptremblett has quit IRC08:53
*** miclorb_ has quit IRC08:54
*** dendrobates is now known as dendro-afk09:02
*** miclorb has joined #openstack09:11
*** rbergeron has quit IRC09:40
*** rbergeron has joined #openstack09:47
*** littleidea has quit IRC09:50
*** littleidea has joined #openstack09:54
*** ptremblett has joined #openstack09:58
termiegundlach: There shouldn't be anything that needs to be BaseTestCase, I think it only still existed for tornado stuff, of which there are now none10:09
*** miclorb has quit IRC10:10
*** BK_man has joined #openstack10:21
*** silassewell has quit IRC10:24
*** matiu has joined #openstack10:42
*** gustavomzw has joined #openstack10:48
*** rds__ has joined #openstack10:58
*** adjohn has quit IRC11:02
*** kashyapc has quit IRC11:03
*** fitzdsl has joined #openstack11:14
*** coolhandluke has quit IRC11:20
*** coolhandluke has joined #openstack11:24
*** schisamo has joined #openstack11:31
*** sirp1 has joined #openstack11:31
*** sophiap has joined #openstack11:33
*** schisamo has quit IRC11:40
*** sophiap has quit IRC11:42
*** hazmat has joined #openstack12:02
*** mdomsch has joined #openstack12:05
*** hazmat has quit IRC12:16
*** hogstrom has quit IRC12:19
*** befreax has joined #openstack12:22
*** befreax has quit IRC12:24
*** ctennis has quit IRC12:44
*** aliguori has joined #openstack12:45
*** matiu has quit IRC12:48
*** alekibango has quit IRC12:50
*** toddmorey has joined #openstack12:53
*** Podilarius has joined #openstack12:56
*** guigui4 has joined #openstack12:56
*** JTol has joined #openstack13:00
*** guigui1 has quit IRC13:00
*** ttx has joined #openstack13:01
*** ttx has joined #openstack13:01
*** dendro-afk is now known as dendrobates13:03
*** kashyapc has joined #openstack13:05
*** ramkrsna has quit IRC13:08
*** westmaas has joined #openstack13:09
*** ppetraki has joined #openstack13:09
*** coolhandluke has quit IRC13:10
*** jkakar has joined #openstack13:12
*** zul has joined #openstack13:14
*** mdomsch has quit IRC13:15
*** mtaylor has quit IRC13:16
uvirtbotNew bug: #667278 in nova "Instance monitor uses hardcoded (wrong) crendentials" [Undecided,New]
*** dionjp has joined #openstack13:36
*** westmaas1 has joined #openstack13:39
*** westmaas has quit IRC13:40
*** ctennis has joined #openstack13:40
*** ctennis has joined #openstack13:40
*** pvo has joined #openstack13:41
*** ChanServ sets mode: +v pvo13:41
dionjpHi, installing nova I got the error "ImportError: no module named twisted.internet.threads". Tried to reinstall python-twisted, browsed the web, solution. Any idea ? Thanks. jean-pierre13:42
*** kevnfx has joined #openstack13:43
*** reldan has joined #openstack13:44
*** masumotok has joined #openstack13:47
*** rbergeron has quit IRC13:47
*** rbergeron has joined #openstack13:47
*** rds__ has quit IRC13:50
*** rds__ has joined #openstack13:50
*** dionjp has quit IRC13:52
*** eldarnugaev has joined #openstack13:52
*** blakeyeager has joined #openstack13:53
*** reldan has quit IRC13:53
*** eldarnugaev has quit IRC13:54
*** mtaylor has joined #openstack13:55
*** mtaylor has joined #openstack13:55
*** ChanServ sets mode: +v mtaylor13:55
*** ttx has quit IRC13:55
*** dysinger has joined #openstack13:55
*** kevnfx has quit IRC13:58
annegentlediojp: I got that error when first installing, I think I installed python-twisted and fixed it though. Let me look through my notes.13:59
*** zul has quit IRC14:00
*** gaveen has quit IRC14:01
gundlachtermie: yep, thanks -- ~gundlach/nova/cleanup-unittests has excised it.  i'm helping jaypipes get rid of next.14:01
*** eldarnugaev has joined #openstack14:02
*** ttx has joined #openstack14:02
*** ttx has joined #openstack14:02
*** cole has joined #openstack14:03
*** zul has joined #openstack14:05
*** dendrobates is now known as dendro-afk14:10
*** mtaylor has quit IRC14:11
annegentlediojp: yep, I was on Ubuntu 10.04 and sudo apt-get install python-twisted did it for me. Then I ran into "No module named gflags" because it's not packaged on 10.04 (but it is on 10.10) so I used the OpenStack PPA.14:13
*** hazmat has joined #openstack14:14
*** abecc has joined #openstack14:14
*** cole has quit IRC14:15
*** abecc has joined #openstack14:16
*** mtaylor has joined #openstack14:20
*** ChanServ sets mode: +v mtaylor14:20
*** jdarcy has joined #openstack14:21
*** neogenix has quit IRC14:23
BK_manI have a problem with objectstore in nova: openssl rsautl -decrypt -inkey /root/nova/nova/../CA/private/cakey.pem -> Error: Failed to decrypt private key: RSA operation error -> 2845:error:04065084:rsa routines:RSA_EAY_PRIVATE_DECRYPT:data too large for modulus:rsa_eay.c:515:14:25
BK_manany help?14:25
BK_manit comes from aws_register_image14:26
*** guigui4 has quit IRC14:28
*** metcalfc has joined #openstack14:32
*** jtimberman has quit IRC14:34
jaypipescreiht, gholt, redbo: any help for BK_man ? :)14:34
*** stuart_ has joined #openstack14:35
*** alekibango has joined #openstack14:37
*** jtimberman has joined #openstack14:37
creihtjaypipes: Isn't that in the nova built in object store?14:41
BK_mancreiht: yep14:46
jaypipescreiht: doh, sorry :(14:46
jaypipesBK_man: how large is image?14:46
BK_manjaypipes: standard test image14:47
BK_manjaypipes: ami-tiny14:47
*** zul has quit IRC14:47
jaypipesBK_man: hmm, weird...14:47
BK_manjaypipes: this is yesterday's snapshot of nova14:48
jaypipesBK_man: I would ask you to report a bug and make sure you list all your platforms specs, ok?14:48
BK_manjaypipes: I'm new to nova. I will try to do my best14:49
jaypipesBK_man: thanks!14:49
jaypipessoren: any ideas on BK_man's issue above?14:49
BK_manjaypipes: do you have a good getting started guide for newbies?14:49
*** schisamo has joined #openstack14:50
BK_manjaypipes: I'm not sure that I did everything right14:50
jaypipesBK_man: unfortunately, docs are pretty light right now for usage. :(  Did you install using vishy's nova install script?14:50
*** ttx has quit IRC14:50
BK_manjaypipes: correct, I used a script. I tried to install Austin code but failed.14:51
*** westmaas1 is now known as westmaas14:51
jaypipesBK_man: by "failed", did you get an error?14:52
BK_manI'm on Maverick (10.10)14:52
BK_manjaypipes: yep. I'm really newbie and feel bad - copying & pasting commands like monkey and don't understand what it do14:53
jaypipesBK_man: no worries. :)14:53
BK_manjaypipes: I'm using instructions from this page:
jaypipesBK_man: and what was the error you got when installing?14:54
pikenNova uses libvirt to spawn the VM's on nodes correct?14:54
BK_manjaypipes: I can't complete nova-manage project zipfile on Austin release14:54
jaypipespiken: it can, yes.14:54
jaypipespiken: can also use XenAPI14:55
jaypipespiken: and hyper-v support is almost done..14:55
jaypipesBK_man: ok, lemme look into that...14:55
BK_manjaypipes: it could be due missed deps - I hope that vishy's script installed all required packages after that14:55
jaypipesBK_man: that's the command you get the RSA error on, right?14:55
annegentleBK_man: I'm glad you're walking through those instructions, I want to update them as needed. I tested on 10.04 but not sure if the install is complete yet.14:56
BK_manjaypipes: back to RSA error - i think that was immediately after euca-upload-bundle14:56
pikenjaypipes: Is there anyway with libvirt to setup a node to use virtfs the way you would with qemu? I have a need to use virtfs to allow the vm's to mount a local directory on the node.14:56
BK_manannegentle: thanks :)14:56
jaypipesanneg: I didn't realize you were an IRT fan.  I'm partial to Swamp Loggers, myself :)14:57
annegentleBK_man: on Ubuntu 10.10, you might be better off installing from the package, see "Single Machine Installation" on
jaypipespiken: I'm not sure.  vishy and soren would be better help than me on that :(14:57
annegentleBK_man: then again, you're far enough along that you may have stumbled on bugs :)14:57
BK_manannegentle: ok. I will try it in few mins - just need another cup of tea :)14:57
pikenvishy: Soren: either of you around at the moment?14:58
annegentlejaypipes: LOL my hubby controls the remote most evenings so it's History and Discovery channel for us14:58
BK_manit's easy to erase all and begin from scratch on that early stage :)14:58
jaypipesBK_man: did the nova-manage create project command succeed successfully?14:58
pikenannegentle: and what is wrong with a little learning ;)14:58
annegentlepiken: you betcha :)14:58
BK_manjaypipes: I don't get any errors14:58
jaypipesannegentle: oh, pls no Sarah Palin :P14:58
*** eldarnugaev has quit IRC14:59
jaypipesBK_man: ok.  but you *do* get an error while executing nova-manage project zipfile?14:59
annegentlejaypipes: thanks I just threw up in my mouth a little14:59
jaypipesannegentle: :)14:59
annegentlejaypipes: :)14:59
BK_manjaypipes: that was on Austin and not on trunk. Will try now15:00
jaypipesBK_man: k.15:00
pikenI am wondering if I can setup libvirt.qemu.xml.template to setup a virtfs partition and also pass to it an id associated with the user spawning the vm's....15:01
*** rnirmal has joined #openstack15:02
*** dionjp has joined #openstack15:07
* jaypipes notes that his wife is already nagging him to find his passport for a trip to Greece...that we're going on in May next year. A bit excessive he feels...15:08
*** ttx has joined #openstack15:08
*** HouseAway is now known as AimanA15:10
BK_manOh no: Connection refused15:11
jaypipesBK_man: hmmm....that may be the source of the problem :)15:12
jaypipeseday: ok, I fixed up things from your review (thx!)
dionjpannegentle: thank you. I reinstalled python-twisted but that is still nok. I keep trying...15:15
*** schisamo_ has joined #openstack15:19
BK_manlibvirtError: internal error process exited while connecting to monitor: chardev: opening backend "file" failed15:20
BK_manthis is in compute window15:20
*** schisamo has quit IRC15:22
*** schisamo_ is now known as schisamo15:23
*** JTol has quit IRC15:23
*** zul has joined #openstack15:23
*** zul__ has joined #openstack15:24
*** pharkmillups has joined #openstack15:26
*** calavera has quit IRC15:28
*** schisamo has quit IRC15:28
*** dizz has joined #openstack15:31
*** schisamo has joined #openstack15:33
sirp1jaypipes: i just roughed out a first cut of the Teller caching blueprint
jaypipessirp1: sweetness.15:40
creihtgrizzletooth: I'm hacking on your multi-server install doc if you want to follow along :)15:40
*** jk0 has left #openstack15:41
*** jk0 has joined #openstack15:41
*** ChanServ sets mode: +v jk015:41
*** littleidea has quit IRC15:43
*** zul__ has quit IRC15:48
*** kevnfx has joined #openstack15:49
jarrodno docs on multi-server compute setup for nova?15:53
*** ttx has quit IRC15:56
alekibangojarrod:  try those, and put your thoughts in, please...
jarrodoh i didnt see that15:57
jarrodi will, i have a rack of servers i want to deploy15:57
alekibangojarrod: yes i see. please write your thoughts/results/methods in the document15:57
alekibangoit will become wiki sooner or later15:57
grizzletoothcreiht: in what forum?  etherpad?15:58
grizzletoothcreiht: cool, I found an issue I wanted to correct as well :)15:59
alekibangojarrod: how many servers you have?15:59
alekibangoplease document your way of installing on your system in teh etherpad15:59
alekibangojarrod: will you use some configuration management system?16:00
alekibangoi would thing for 10 servers its becoming needed16:00
jarrodwithout question16:00
alekibangowhich one? which os?16:00
jarrodim using ubuntu16:01
alekibangohow will you install servers?16:01
alekibangousing FAI?  foreman?16:01
jarrodthe ubuntu?16:01
alekibangohow you will partition drives? :)16:01
jarrodthe main storage will all be on EMC16:02
jarrodfor server images etc16:02
grizzletoothjarrod: at a minimum lookup dsh - group oriented ssh batching16:02
*** schisamo has quit IRC16:02
alekibangogrizzletooth:  i would say m-collective :)16:02
alekibangobut that is not solvign the partition/install part16:02
*** sirp1 has quit IRC16:03
*** eldarnugaev has joined #openstack16:04
alekibangojarrod: which EMC? can you tell more?16:04
alekibangoplease use the etherpad. dont be scared16:04
alekibangotry even to document architecture16:05
alekibangowe will together edit it, so we  all will benefit16:06
*** Ryan_Lane has joined #openstack16:06
alekibangojarrod: so, or ? or something similar?16:08
jarrodsorry, in and out, fai is what i was going with16:10
jarrodis there something better?16:10
alekibangojarrod: i am not sure... i am right now trying to decide which one to take16:10
alekibangotheforeman looks to be somewhat friendly with puppet16:11
alekibangoand might be nice to use16:11
jarrodyea, i just picked fai16:11
alekibangoor fai16:11
jarrodwill figure it out along the way16:11
alekibangojarrod: so i will take fai too and we can start preparing fai config for nova ok?16:11
*** hazmat has quit IRC16:12
jarrodmost excellent16:12
alekibangojarrod: do you have fai working?16:12
jarrodno i still have to unbox all of these servers16:12
jarrodwe just ordered them just for this16:12
alekibangoheh, i have 4 servers ready :)16:12
alekibangobut i am on debian :)16:12
jarroddebian is so behind on software16:13
alekibangonot really16:13
*** gaveen has joined #openstack16:13
*** gaveen has joined #openstack16:13
alekibangojarrod: debian stable might be, but its stable :)16:14
alekibangoi use sid on desktop16:14
jarrodyea i would only go stable16:14
jarrodthats what i use for my voice network16:14
alekibangosqueeze will be stable soon - and has newer fai which is able to install ubuntu too (if i am not mistaken)16:14
alekibangojarrod: anyway, fai will be very similar for debian/ubuntu16:15
*** pharkmillups has quit IRC16:17
jarrodwhat servers are you using16:17
alekibangojarrod: i have 4 nice xeon 8core servers, each having (only) 2 1TB drives, (only) 16 GB RAM, and 2 1GB network cards16:17
alekibangonever seen them :)16:17
alekibangothey are 200km from me16:18
jarrodi have AMD16:18
*** JimCurry has joined #openstack16:18
jarrod16gig, 2 1gb, and same TB drives16:18
alekibangoyes 2 or 3 netwrok cards?16:18
alekibangoi think we will need later 316:18
alekibango1 for public ips16:18
alekibango2nd for sheepdog storage16:19
alekibango3rd for admin access -- including web consoles and stuff16:19
*** zul__ has joined #openstack16:19
jarrodcould always vlan sheepdog/admin16:19
alekibangoyes, i am having similar hardware i would say16:20
alekibangosheepding is not yet integrated, but i would love to use it16:20
alekibangoor similar storage solution16:20
alekibangojarrod: where are u? (which timezone?)16:21
jarrodCST houston, tx16:21
*** zul__ has quit IRC16:21
*** zul__ has joined #openstack16:21
alekibangojarrod: i am in eu, but i dont sleep often16:22
*** eldarnugaev has quit IRC16:22
alekibangojarrod: when you will ahve servers ready ?16:23
jarrodnext week16:23
alekibangoah, was thinking about tonight, when i finish my internet crawler engine :)16:23
jarrodha i have 4 projects rolling16:24
alekibangojarrod: dont be afraid to pvtmsg me, i will answer16:24
alekibango(when i will be afk)16:24
*** hazmat has joined #openstack16:32
*** Orman has joined #openstack16:40
OrmanHey guys16:40
annegentlehey Orman16:41
*** ibarrera has quit IRC16:42
OrmanI wish I could be there at the design summit.16:42
*** JimCurry has left #openstack16:42
OrmanAnnegentle: How is it going?16:44
annegentleOrman: getting hungry, it's nearly lunchtime :)16:44
*** jkakar has quit IRC16:45
*** hazmat has quit IRC16:46
*** zul__ has quit IRC16:54
*** pharkmillups has joined #openstack16:57
*** sebastianstadil has joined #openstack17:01
*** Orman has quit IRC17:05
mjmachmm.  anyone running swift on centos5 using python26 from EPEL?17:05
*** ppetraki has quit IRC17:07
pikenmjmac: yes, I am17:12
pikenHardest part to get setup and working correctly will be carrot17:12
pikeneverything else is easy17:12
*** ppetraki has joined #openstack17:13
mjmacperhaps...  i can't even seem to run swift-init auth-server start.  well, i can, but nothing happens.  when i use trace i get a bizarre error that 'sys' is undefined.  i can clearly see that it's been imported in, though.  this is all from packages (except swift), so i just wanted to make sure i wasn't hitting some known problem with the EPEL stuff.  i'll keep poking.17:15
dwight_mjmac: are you following the SAIO?17:19
mjmacnope, just trying to install using
*** jdmaturen has joined #openstack17:21
dwight_I haven't tried that. I did the SAIO with Fedora 13.17:21
dwight_The Fedora 13 changes are on a message I posted on the mailing list.17:21
*** kw1 has joined #openstack17:21
*** kw1 has left #openstack17:22
dwight_There is also a pointer there to RH 6 IIRC changes, but those might be out of date.17:22
BK_manKhm. Interesting. I have that bug!
uvirtbotLaunchpad bug 655217 in nova "euca-run-instances fails to run VM" [High,Fix released]17:22
dwight_Wish they were in the wiki, but that doesn't seem to be in the cards.17:23
*** dwight_ is now known as dwight17:24
*** jcannava has joined #openstack17:25
*** dizz has quit IRC17:25
annegentledwight: feel free to put 'em in the wiki17:26
dwightThanks. That's the only feedback I've gotten. Can I get a link to them on the SAIO page?17:27
annegentledwight: I've got that in a merge request, but unfortunately I also bundled that merge request in with the swift multi install :)17:28
*** johnbergoon has joined #openstack17:29
dwightannegentle: :) Ok. Will do. I'll try to do it today or tomorrow, but I just got in two new serious storage boxes that need to be brought up and benchmarked.17:29
annegentledwight: excellent, thanks a bunch!17:30
*** schisamo has joined #openstack17:30
dwightannegentle: Hey, my pleasure. It's always great to help with Open Source.17:30
*** kevnfx has quit IRC17:31
annegentledwight: I did minor changes to the wiki page (see but you're welcome to keep editing, enhancing, etc.17:31
jaypipeseday: ok, making those're right. :)17:31
jaypipeseday: sorry, wasn't quite thinking clearly about your suggestions earlier.17:31
dwightannegentle: That's seriously cool. Thanks.17:32
annegentledwight: will follow up on the mailing list too so others are aware17:33
edayjaypipes: sorry for being a nag :)17:34
dwightannegentle: As I mentioned on the mailling list, I think a separate page is needed for Fedora. The Fedora instructions are a lot simpler.17:34
jaypipeseday: hey, I appreciate it :)17:34
*** BK_man has quit IRC17:35
annegentledwight: okay, just to make sure I understand, we need a page with Fedora/RHEL/Centos SAIO instructions?17:37
annegentleheh he must be off unboxing. Sent to the mailing list instead. :)17:40
*** rds__ has quit IRC17:51
mtaylorjaypipes: have you looked at zookeeper at all?17:52
dwightannegentle: Sorry, was AFK. I was thinking on the SAIO page, a link to a general page which points to the final page for a distribution. Does that make sense?17:56
jaypipesmtaylor: yes, very briefly.17:56
dwightannegentle: Sort of like Wikipedia's disambiguation page.17:56
mtaylorjaypipes: k. guy here at UDS was just talking to me about it as a centralized/distributed conf-management thing. thought I'd toss if your way in case it was useful for anything17:56
*** burris has quit IRC17:56
annegentledwight: very helpful, thanks. Yes, the disambiguation list makes sense!17:56
edaymtaylor, jaypipes: It's in Java and doesn't have Python bindings yet AFAIK, so not too interesting yet :/17:57
jaypipesmtaylor: when I looked at it, I saw the same as eday.  The code has some good parts, but it doesn't look particularly friendly to install or use.18:00
mtaylorjaypipes: lovely18:00
mtaylork. well - I have now done my due-diligence :)18:00
edaymtaylor: was this for the context of nova, or just in general?18:01
mtayloreday: nova/openstack/i don'18:01
mtayloreday: nova/openstack/i don't know18:02
gustavomzweday: python bindings since 3.2 -
*** eldarnugaev has joined #openstack18:04
*** neogenix has joined #openstack18:07
*** Orman has joined #openstack18:07
edaygustavomzw: ahh, good to know :)18:07
gustavomzweday: Twisted library for Apache Zookeeper -
*** jcannava has quit IRC18:12
*** dendro-afk is now known as dendrobates18:12
edaygustavomzw: their wiki:  states there are only Java/C interfaces... probably should be updated :)18:14
*** hazmat has joined #openstack18:14
gustavomzweday: agreed :)18:17
*** pvo_ has joined #openstack18:19
OrmanHey guys!18:21
alekibangohi orman, nice to see you again..18:21
OrmanThanks man you too.18:21
OrmanBeen busy.18:22
alekibangoOrman: so, what do you think we need to protect? :)18:22
alekibangoOrman: integrity!18:22
OrmanYes oh yes!18:23
alekibangothats very valuable, especially when you think about silent disk corruption problem18:23
OrmanYeah and plus more storage security.18:23
OrmanAuthentication is always a plus to keep updated as well.18:25
jaypipeseday: ping18:26
jaypipeseday: so, I tried the exact code you wrote in the code review, and I get various errors.  Did you try that code on your local box?18:27
edayjaypipes: nope :)18:28
edayjaypipes: can you paste the errors?18:28
jaypipeseday: sure, one sec18:29
jaypipeseday: this is what shows up in nosetests:
OrmanWell I am going to head out.18:31
OrmanI'll be on back on later some time.18:32
OrmanGod Bless guys18:32
jaypipeseday: forget it.  I'm an idiot.18:32
jaypipeseday: I missed a comma...18:32
edayjaypipes: :)18:33
*** Orman has quit IRC18:33
*** ptremblett has quit IRC18:34
dwightjaypipes: don't feel bad. It was a missing semicolon that led to the creation of the EFF. :)18:34
jaypipesdwight: hehe :)18:35
*** jc_smith has joined #openstack18:39
sorenpiken: I am now. Is it still relevant?18:45
* soren scrolls back18:45
pikenLet me know if you need to explain in further detail what I am trying to do18:45
vishyBK_man: don't know if you're still here, but the libvirt console error is usually do to having a parent directory for instances that is only readable by root. For example, trying to run in /root.  You can usually fix it with chmod 755 /root (which is super secure btw) :p18:46
sorenpiken: Please do. I'm trying to pay attention to too many things at once, so a brief summary would be helpful.18:49
pikenI am trying to obtain full infiniband performance for my lustre san. To do that, I cannot mount them in the vm as virtio-net will not emulate a 40GBps interface.18:50
pikenTo get around it I am on a few qemu images started by hand using virtfs to mount a local directory on the kvm node in the vm.18:50
pikenI am trying to figure out how I can do the same for nodes launched using qemu-kvm from libvirt.18:51
pikenthat url shows how I am using virtfs to mount the local directory as a disc in the vm18:52
*** westmaas has quit IRC18:53
*** rlucio has joined #openstack18:55
*** dysinger has quit IRC18:57
gundlachjaypipes: does reset_test_environment take 15 seconds for you?  it does for me, most of it spent in sqlite3.Connection.commit().18:58
*** burris has joined #openstack19:00
*** rnirmal has quit IRC19:00
*** rnirmal_ has joined #openstack19:00
*** pvo_ has quit IRC19:00
*** ttx has joined #openstack19:11
*** ttx has joined #openstack19:11
sorenpiken: Well, you are free to use a different template for libvirt.19:11
sorenpiken: I honestly don't remember if libvirt exposes that functionality.19:13
*** zul__ has joined #openstack19:14
*** zul__ has quit IRC19:18
*** zul__ has joined #openstack19:18
*** arthurc has quit IRC19:19
*** jkakar has joined #openstack19:21
*** zul__ is now known as zul19:25
*** zul has quit IRC19:25
*** zul has joined #openstack19:25
*** rnirmal_ has quit IRC19:25
*** rnirmal has joined #openstack19:25
*** stewart has quit IRC19:28
*** kevnfx has joined #openstack19:29
*** dendrobates is now known as dendro-afk19:33
*** dendro-afk is now known as dendrobates19:35
*** jcannava has joined #openstack19:38
*** ttx has quit IRC19:41
*** schisamo has quit IRC19:43
*** ptremblett has joined #openstack19:43
*** zul has quit IRC19:45
jaypipesgundlach: not 15 seconds, but about 4, yes.  It's because it completely removes the models and reconstructs them... but I have a plan to improve that :)19:45
vishyguys: we (anso) are doing heavy editing of the nova docs.  We are starting with a basic skeleton and filling in as we go.  Feel free to follow the progress at lp:~anso/nova/trunkdoc19:46
gundlachjaypipes: i must have a really slow hard drive or cpu :)  i was thinking we could just replace the nova.sqlite file with a pre-built one, unless the user has custom fixtures.19:46
jaypipesgundlach: I tried that...wasn't working :(19:47
gundlachjaypipes: hmm, maybe you have to disconnect and reconnect sqlalchemy from it?19:47
*** rlucio has quit IRC19:48
*** gaveen has quit IRC19:48
vishygundlach, jaypipes: does it reset between each test?19:48
*** pvo has quit IRC19:49
*** bwalk7125 has joined #openstack19:50
*** jcannava has quit IRC19:59
*** kevnfx has quit IRC20:01
*** gaveen has joined #openstack20:02
*** Cybodog has quit IRC20:02
*** Cybodog has joined #openstack20:03
edaygundlach: try libeatmydata, it makes everything faster :)
*** mdomsch has joined #openstack20:03
soreneday: Is that was is backing s4?20:04
edaygundlach: I ld preload them for my terminal that runs tests, makes a big difference20:04
soreneday: <--- S420:04
soreneday: Oh, not at all.20:05
soreneday: Just realised what libeatmydata is. :)20:05
alekibangosoren: heh. and your data are ok? :)20:06
*** ctennis has quit IRC20:06
edaysoren: haha :)20:06
edaysoren: I wonder if folks have actually sent them money20:07
soreneday: I would.20:08
sorenIf I wanted to get rid of some data, it seems like a good deal to me. What am I missing?20:08
sorenI tried their example SOAP call. It failed. I smell a scam.20:09
*** pvo has joined #openstack20:12
*** ChanServ sets mode: +v pvo20:12
*** dendrobates is now known as dendro-afk20:12
alekibangohaha, s420:12
alekibangogood one20:12
alekibangoreminds me how nsa operates :)20:12
alekibango  -- sell your soul or get online quotation, measuring value of your soul is priceless20:15
gundlachvishy: yes, we're trying to reset test fixtures between every test so that each test is independent20:15
gundlacheday: thanks, hadn't heard of that -- i bet it'll help :)20:15
*** dysinger has joined #openstack20:18
*** ttx has joined #openstack20:20
*** zul has joined #openstack20:22
pikensoren: ty for looking. I might try a custom qemu template for it. Now to find a way to pass a userid to it in order to make sure it creates the mount for the right directory if it works.20:26
*** schisamo has joined #openstack20:27
*** pvo_ has joined #openstack20:28
*** pvo_ has joined #openstack20:28
*** ChanServ sets mode: +v pvo_20:28
jaypipesvishy: gundlach and I are talking about some new code that resets the test environment fixtures...still a work in progress :)20:30
vishyjaypipes: yeah i figured20:30
bwalk7125does openstack have something similar to Amazon's metadata service to get console output for instances?20:31
*** pvo has quit IRC20:31
*** pvo_ is now known as pvo20:31
jaypipesvishy: but, for the record, doing destroy_all() and then create_all() on SQLAlchemy's Metadata object is slowwww.. :)20:31
bwalk7125or where is there log info for instances?20:31
*** dendro-afk is now known as dendrobates20:35
*** AimanA has quit IRC20:36
pikenanyone able to give me the link to the nova and swift mailing list subscriptions. I can't seem to find it anymore. lol20:40
alekibangopiken: see second link in nova/swift    on page
*** rlucio has joined #openstack20:47
*** johnbergoon has quit IRC20:51
*** zul has quit IRC20:53
*** ttx has quit IRC20:53
*** dendrobates is now known as dendro-afk20:55
*** tr3buchet has joined #openstack20:56
*** tr3buchet has joined #openstack20:57
xtoddxbwalk7125: yea, there is metadata20:58
*** allsystemsarego has quit IRC21:00
*** jdarcy has quit IRC21:02
bwalk7125xtoddx: is it just in the console.log files?21:05
xtoddxbwalk7125: look at get_metadata in nova/api/ec2/cloud.py21:07
*** ctennis has joined #openstack21:08
*** ctennis has joined #openstack21:08
bwalk7125xtoddx: thanks, I see it now, I was trying to find where this was kept b/c after bundling some instances and then launching them, they would shut down immediately21:10
bwalk7125this should help though21:10
*** ttx has joined #openstack21:12
*** dendro-afk is now known as dendrobates21:25
*** zul__ has joined #openstack21:25
*** rnirmal has quit IRC21:26
*** zul__ has quit IRC21:38
*** pvo has quit IRC21:41
*** pvo has joined #openstack21:41
*** pvo has joined #openstack21:41
*** ChanServ sets mode: +v pvo21:41
*** alekibango has quit IRC21:52
*** AimanA has joined #openstack21:54
*** AimanA has joined #openstack21:54
bwalk7125has anyone else had a problem with bundled instances going immediately into the shutdown state after launch?  I'm on Ubuntu 10.04, and I've tried the ubuntu-lucid.img from canoncial and using qemu-create to an image file.  After uploading them into the bucket and registering them, they always go from lauching->shutdown.  These are m1.small images21:56
bwalk7125I tried m1.large and it remains perpetually in a launching state21:56
bwalk7125and the console.log files are empty21:57
*** ttx has quit IRC21:58
*** Cybodog has joined #openstack21:59
*** Ryan_Lane has quit IRC22:05
*** Podilarius has left #openstack22:11
*** pvo_ has joined #openstack22:13
*** ChanServ sets mode: +v pvo_22:13
*** dendrobates is now known as dendro-afk22:13
mtaylorsoren: you guys doing anyting interesting tonight?22:14
*** jkakar has quit IRC22:14
dwightbwalk7125: Are you running all 64-bit?22:15
*** alekibango has joined #openstack22:15
*** neogenix has quit IRC22:16
bwalk7125dwight: yes22:17
*** pvo has quit IRC22:18
*** pvo_ is now known as pvo22:18
*** ppetraki has quit IRC22:18
vishybwalk7125: i saw that happen to someone else, but haven't been able to see what might be causing it22:18
dwightbwalk7125: I ran into that a while back. I don't have a solution, as I gave up on nova at that point.22:18
vishydoes /var/log/libvirt/qemu/<instance-xxxxxxx>.log show anything?22:19
vishyand is there an error message in the nova-compute log?22:19
bwalk7125vishy: nothing unusual in the nova-compute log22:20
vishyit says booted as normal?22:20
bwalk7125let me check the other files real quick22:20
bwalk7125actually, I must be looking at the wrong nova-compute file, as the dates are all wrong, nothing about booting images at all22:21
bwalk7125in /bin/nova-compute.log22:21
bwalk7125sorry, /nova/bin/nova-compute.log22:21
*** mtaylor has quit IRC22:23
bwalk7125vishy: in the /var/log/libvirt/qemu/instance logs, I could find: "failed to find romfile pxe-rt18139.bin" in some of the files22:24
bwalk7125the others were empty22:24
vishythat isn't an issue22:24
vishydo some instances start and others don't?22:24
bwalk7125the standard ami-tiny instances start22:24
vishyoh rly22:24
*** miclorb has joined #openstack22:24
vishythen i would suspect you are bundling the image incorrectly22:25
vishythe image needs to be a single partition22:25
vishyand needs a separate kernel and ramdisk22:26
bwalk7125the single parition might be the issue22:26
bwalk7125I downloaded the image file as listed in the 'Install from source' instructions on the wiki22:26
bwalk7125wget -q -O - | tar xSv22:26
vishythere is a tool nova-import-canonical-imagestore22:27
vishyto grab and register a lucid image22:27
vishyfrom uec22:27
bwalk7125ok, I'll try it with that then22:28
vishyi know that ubuntu-lucid image worked at one point, though, so it is odd that it isn't22:28
vishyif you do a describe instances does the image state show as available22:28
vishy(for the new one you registered)22:29
bwalk7125hmm, for the image, it shows available22:29
bwalk7125but for the ramdisk and kernel, it says decryting22:30
bwalk7125actually, for that ubuntu-lucid.img it does say decrypting as well22:31
bwalk7125sorry I misread that22:31
*** pvo has quit IRC22:31
*** krish has joined #openstack22:34
vishythere is your problem22:35
vishythe system is failing to decrypt the image properly22:35
vishyso you could track that issue down or just cheat22:35
vishyand copy the images into the image store directly22:36
vishyyou just need a folder called ami-whatever22:36
vishya file called image22:36
bwalk7125vishy: ok22:36
vishyand an info.json (just copy the one from ami-tiny)22:36
vishyand edit the image name in the file22:36
bwalk7125ok, I'll give that a try then22:37
vishyto be the same as the folder name22:37
*** krish has quit IRC22:37
*** pharkmillups has quit IRC22:43
*** schisamo has quit IRC22:53
*** piken has quit IRC22:58
*** gundlach has quit IRC23:00
rluciorlucio@box3:/var/log/nova$ ls | wc -l23:01
rluciomaybe a bug? :)23:02
dwightcreiht: What's the easiest way to debug the proxy server? It's hanging when I do the multi-server approach, when running the swift-auth-add-user.23:03
creihtyou can check /var/log/syslog (or messages on some OSes) to see if the proxy is logging any messages23:05
dwightthanks. I tried that, and there weren't any, alas.23:06
*** stewart has joined #openstack23:07
creihtI would also check that you can reach the auth server you are sending, and the proxy (the host that is set as default_cluster_url)23:07
dwightok. will do. thanks.23:07
creihtI had that happen once when the default_cluster_url was a URL to a public interface, but I was on an internal machine that couldn't access the external network23:08
creihtdwight: First thing I would try is to ensure that you can 1. auth from that server using curl, and then 2. do a head on your account using curl, as described at the end of the saio doc23:09
creihtthat should show you pretty quickly if there are any connectivity issues23:09
*** Cybodog has quit IRC23:09
dwighthmmm. that's a good idea. Re: the default_cluster_url, that looks fine. Even netstat is showing that the connection is established, but it's like the socket connection isn't fully going through.23:10
*** jc_smith has quit IRC23:10
creihtdwight: also make sure you don't have any funky iptables rules23:10
dwightyeah. I always shut off iptables completely when dealing with something new. Then bring them back up later when things are working.23:11
dwightthe curl is hanging too. But when I remove the certs from the proxy config file, the socket completes, and the proxy complains.23:13
creihtare you making the request with http or https?23:13
dwightWhatever swift-auth-add-user uses. For curl, it's http.23:14
dwightAhh. https is better.23:14
creihtwell it is based on the url you send it :)23:14
dwightthere's no url in the swift-auth-add-user command.23:15
creihtI think it looks in your auth-server.conf and gets the url from default_cluster_url23:16
*** schisamo has joined #openstack23:17
vishyjaypipes: are you here?23:18
*** dysinger has quit IRC23:18
*** pharkmillups has joined #openstack23:20
dwightcreiht: Yep. That's the problem. Now I'm getting a service unavailable 503 error.23:20
creihtok, then you should be able to see in the logs more info as to why it returned a 50323:21
dwightStill, the proxy shouldn't be hanging, it should abort immediately, I would expect.23:21
creihtdwight: the proxy wasn't hanging, it was the call from the command23:21
creihtthe 503 there is most likely due to the having issues talking to the account servers23:22
dwightWell, a telnet was also hanging. Which isn't particularly helpful in trying to debug. I would expect an error or timeout.23:23
dwightOk, I'll look at the account servers. Thanks.23:23
creihtI'll have to look into the http/https issue, but for all normal requests, it would timeout quickly in case of errors23:24
creihtand also if you were making the http request to the default port, then it was freezing trying to hit some other service running on port 80, not the proxy server23:25
dwightThis was to port 8080. I have nothing running on port 80. No connection would be established.23:27
dwightGood point though.23:27
creihtthen it is some issue with the https handling code23:27
*** jc_smith has joined #openstack23:27
creihtwhich we don't use in production23:27
creihtwe terminate ssl at the load-balancer23:27
dwightAha. Cheater. :)23:28
creihtall other types of errors should timeout quickly23:29
dwightYeah. That's what it looked like to me. Which is why I was wondering if there was an easy way to debug it.23:29
creihtyou can run any of the services in the terminal manually like:23:30
creihtswift-proxy-server /etc/swift/proxy-server.conf23:30
*** jc_smith has quit IRC23:31
creihtThough not sure that would help in the problem you saw23:31
*** jc_smith has joined #openstack23:31
dwightthanks. It would be nice to have configurable debugging output messages.23:31
*** eldarnugaev has joined #openstack23:32
dwightjust a wishlist item. no biggie.23:32
creihtyou can set log_level = DEBUG23:32
*** jc_smith_ has joined #openstack23:32
creihtand feel free to add debug log messages anywhere you like :)23:32
creihtthere is some debugging stuff in there, but not a whole lot23:32
dwightSo I noticed. :)23:32
creihtpatches welcome :)23:34
dwightheh. now you're asking me to do some work. :P23:35
*** matiu has joined #openstack23:35
dwightbe careful of what ye wish for.23:35
creihtdoesn't mean we have to accept the patch :)23:35
*** jc_smith has quit IRC23:36
*** jc_smith_ is now known as jc_smith23:36
creihtdinner time23:36
dwightenjoy. and thanks again.23:37
*** blakeyeager has quit IRC23:37
*** rlucio_ has joined #openstack23:42
*** rlucio has quit IRC23:46
*** rlucio_ is now known as rlucio23:46
*** rlucio has quit IRC23:53
*** hornbeck has quit IRC23:59

Generated by 2.14.0 by Marius Gedminas - find it at!