Wednesday, 2011-02-16

*** joearnold has quit IRC00:03
openflymatches to the ip in nova.conf in hosts00:04
*** joearnold has joined #openstack00:05
*** siebrand has left #openstack00:07
kpeppleopenfly: is your machine connected to the internet ?00:08
*** j05h has quit IRC00:09
kpeppleopenfly: for some reason, it's querying networks for something similar to "select * from networks WHERE deleted = False AND host = barad-dur" but your networks table entry has a NULL for the hosts ...00:10
openflyit's on the internet00:11
openflynatted though00:11
openflynot using FQDN00:12
openflymaybe that's the issue.00:12
kpeppleopenfly: hold on, checking code on the nova-manage network create ...00:13
*** kashyapc_ has quit IRC00:14
*** jbaker_ has quit IRC00:14
*** kashyapc_ has joined #openstack00:14
*** MarkAtwood has joined #openstack00:15
*** ccustine has quit IRC00:16
kpeppleopenfly: what command did you use to start nova-network ? what did the logfile include above what you posted -- there is usually a dump of flags when it first starts. we are looking for the sql_connection flag to make sure it is using mysql and not sqlite00:17
openflyin nova-network log?00:17
openflyor nova-compute / api00:17
*** desai has joined #openstack00:17
kpeppleopenfly: yes, in nova-network log ... it looks like the nova-compute and nova-api ...00:19
openflygonna have to wait00:19
openflyi've accidently knocked myself out of that box for the next couple hours00:19
*** aimon has joined #openstack00:20
kpeppleokay ... you should be starting nova-network with the "--flagfile=/etc/nova/nova.conf" option to make sure it looks for your flag file00:20
kpeppleopenfly: if you don't, it will default to sqlite300:20
openflykpepple where is that set?00:21
*** jbaker_ has joined #openstack00:21
kpeppleopenfly: in /etc/nova/nova.conf, it's your --sql_connection flag -- which is set in the nova.conf file that you posted. however, you have to make sure and point each of the nova-* workers at the same nova.conf file to make sure they are all querying the same database00:22
openflyis there an example somewhere?00:23
*** pcoca has quit IRC00:23
openflylike just put it in /etc/nova/nova-network.conf ?00:23
openflyand i'm good?00:24
kpeppleopenfly: no, you should have one nova.conf file and point all your nova-* workers (nova-compute, nova-network, nova-scheduler, etc.) and that same file by adding "--flagfile=/etc/nova/nova.conf" to each of their startup commands. you can check and make sure that they are using the right conf file by looking in their log files as they dump their flags when they startup.00:25
*** msassak has quit IRC00:28
*** kashyapc_ has quit IRC00:34
*** aimon_ has joined #openstack00:40
*** clauden_ has quit IRC00:41
*** jbaker_ has quit IRC00:42
*** aimon has quit IRC00:42
*** zul has quit IRC00:46
*** evtoews has joined #openstack00:46
*** diegoparrilla has quit IRC00:47
desaiare availability zones used for anything at this point?00:58
sirp_justinsb: i must have missed the merge-prop related to - i'm a tad worried by it since deep within eventlet it clears exceptions by calling sys.clear_exc; this  means that a 'raise' with no args can fail00:58
*** eikke has quit IRC00:59
sirp_justinsb: i've run into this before, so i just wanted to raise that issue; may not necessarily be a problem this time around01:00
*** kashyapc has joined #openstack01:03
*** aimon has joined #openstack01:05
*** jaypipes has quit IRC01:06
*** desai has quit IRC01:09
*** desai has joined #openstack01:09
*** jaypipes has joined #openstack01:10
*** dirakx has joined #openstack01:10
*** littleidea has joined #openstack01:17
*** aimon_ has joined #openstack01:20
*** aimon has quit IRC01:22
*** aimon_ is now known as aimon01:22
*** RJD22 is now known as RJD22|away01:29
*** clauden_ has joined #openstack01:38
justinsbsirp_: termie raised that bug.  He seemed to think it was OK.  What's the right thing to do here so we don't lose the stack trace?  The 3-arg "raise" form?01:44
*** pvo has joined #openstack01:45
sirp_justinsb: i had solved it with just 'raise e' in my case; but i think you're right, 3-arg raise might be the way to go01:45
*** dendro-afk is now known as dendrobates01:47
justinsbsirp_: The problem with raise e is that it swallows the inner exception; I don't really know python that well so didn't really grok what the 3 args should be though :-)01:49
sirp_justinsb: yeah, that hadn't occurred to me until now; yeah reading seems to indicate we should use 3-arg, and use a tb object01:52
*** jheiss has quit IRC01:53
*** RJD22|away is now known as RJD2201:54
*** pvo has quit IRC01:54
*** zul has joined #openstack01:55
justinsbsirp_: I'm about to log off for the night, but if you tell me what the syntax should be I can revise the patch.  Is an exception a 'traceback object'?  Do you know of any examples of how to do this stuff?01:56
sirp_justinsb: not sure offhand, but ill look into it tonight, we can catch up on this tomorrow morn :)01:57
*** joearnold has quit IRC02:00
*** jheiss has joined #openstack02:01
*** joearnold has joined #openstack02:06
*** ryker has quit IRC02:06
*** dendrobates is now known as dendro-afk02:11
*** vbn has joined #openstack02:13
*** vbn has quit IRC02:16
*** dendro-afk is now known as dendrobates02:26
*** dragondm has quit IRC02:27
*** pvo has joined #openstack02:32
*** clauden_ has quit IRC02:33
*** Xenith_ is now known as Xenith02:36
*** mray has joined #openstack02:37
*** pvo has quit IRC02:45
*** fcarsten has quit IRC02:54
*** nati has joined #openstack02:55
*** aimon has quit IRC03:04
*** aimon has joined #openstack03:05
*** nati has quit IRC03:11
*** MarkAtwood has quit IRC03:14
*** jheiss has quit IRC03:26
*** freeflying has quit IRC03:27
*** jheiss has joined #openstack03:28
*** freeflying has joined #openstack03:28
*** jheiss_ has joined #openstack03:31
*** jheiss has quit IRC03:32
*** jheiss_ is now known as jheiss03:32
*** jheiss has joined #openstack03:33
*** hadrian has quit IRC03:36
*** anticw has quit IRC03:38
*** anticw has joined #openstack03:38
uvirtbotNew bug: #719765 in nova "Network Worker crashes on start with Eventlet Error" [Undecided,New]
*** aimon has quit IRC03:53
*** jk0 has joined #openstack03:53
*** jk0 has joined #openstack03:53
*** ChanServ sets mode: +v jk003:53
*** aimon has joined #openstack03:53
*** desai has quit IRC03:54
*** miclorb_ has quit IRC03:54
*** aimon has quit IRC03:57
*** aimon has joined #openstack03:57
*** aimon_ has joined #openstack03:58
*** aimon has quit IRC04:01
*** aimon_ is now known as aimon04:01
*** jheiss has quit IRC04:03
*** vvuksan has quit IRC04:07
*** maplebed has quit IRC04:13
*** jheiss has joined #openstack04:15
*** jheiss has quit IRC04:16
*** jheiss has joined #openstack04:17
*** miclorb has joined #openstack04:19
*** pvo has joined #openstack04:24
*** Gh0sty has quit IRC04:26
*** jheiss has quit IRC04:26
*** jheiss has joined #openstack04:27
*** miclorb has quit IRC04:27
*** Gh0sty has joined #openstack04:27
*** jbaker_ has joined #openstack04:37
*** jbaker__ has joined #openstack04:38
*** jbaker_ has quit IRC04:38
*** jheiss_ has joined #openstack04:41
*** jheiss has quit IRC04:41
*** jheiss_ is now known as jheiss04:41
*** omidhdl has joined #openstack04:47
*** miclorb_ has joined #openstack04:48
dsockwelldoes there exist a puppet recipe for installing openstack on debian squeeze?04:57
*** kashyapc has quit IRC05:00
*** MarkAtwood has joined #openstack05:09
*** jtran has quit IRC05:13
*** ichristo has joined #openstack05:18
*** kashyapc has joined #openstack05:20
*** sebastianstadil has joined #openstack05:20
uvirtbotNew bug: #719798 in nova "Instance fails to access metadata server" [Undecided,New]
*** jheiss has quit IRC05:21
*** jheiss has joined #openstack05:25
*** jheiss has quit IRC05:27
*** aimon has quit IRC05:28
*** jheiss has joined #openstack05:33
*** miclorb_ has quit IRC05:37
*** miclorb_ has joined #openstack05:39
*** littleidea has quit IRC05:40
*** f4m8_ is now known as f4m805:42
*** miclorb_ has quit IRC05:45
*** dirakx has quit IRC05:45
*** nati has joined #openstack05:47
*** dirakx has joined #openstack05:52
*** pvo has quit IRC05:54
*** jheiss has quit IRC06:00
*** mark is now known as mark|coffee06:01
natiThe bexer version with mysql is working? I faild to launch instance06:01
natiI got (nova.root): TRACE: InstanceNotFound: Instance 2 not found06:02
natiThis is same for trunk.06:02
*** jheiss has joined #openstack06:07
kpepplenati: are you having problems with nova-networking ?06:26
natikpeople: no nova-compute fails06:27
kpepplenati: does the instance stick in the "networking" state ? that is what i am seeing06:28
natikpeople: It looks no database entry created, before "networking" state.06:28
natiI checked instances table, but nothing in there.06:29
kpepplenati: hmmm ... how does nova-compute fail in the logs ?06:30
natikpeople: instance is "scheduling" state06:30
natiI could find no other error logs. X:06:30
kpepplenati: when i have "scheduling" state problems, most of the time it has something to do with KVM/Xen not working06:31
natikpeople: KVM with SQLLite works well.06:32
*** kashyapc has quit IRC06:32
natikpeople: Thank you for your help!  I'm looking forward to debug. :)06:34
kpepplenati: sorry i couldn't be of more help :)06:34
*** DigitalFlux has joined #openstack06:37
*** mark|coffee is now known as mark06:37
*** kashyapc has joined #openstack06:44
uvirtbotNew bug: #719825 in glance "POST /images should return 201 on success" [Low,Confirmed]
*** littleidea has joined #openstack06:56
*** westmaas_away has quit IRC07:13
*** ramkrsna has joined #openstack07:20
*** ramkrsna has joined #openstack07:20
*** guigui has joined #openstack07:20
*** Cybo has joined #openstack07:25
*** littleidea has quit IRC07:26
*** rchavik has joined #openstack07:29
*** thimble has joined #openstack07:29
*** thimble has left #openstack07:29
uvirtbotNew bug: #719834 in glance "sqlalchemy-migrate ImportError" [High,In progress]
*** aimon has joined #openstack07:42
*** thimble has joined #openstack07:43
*** berendt has joined #openstack07:46
alekibangonati: please look into logs, there must be more info somewhere... some errors07:52
natialekibango: Yes Thank you!07:53
*** gaveen has joined #openstack08:06
*** gaveen has joined #openstack08:06
*** MarkAtwood has quit IRC08:11
*** winston-d has quit IRC08:17
openstackhudsonProject nova build #519: SUCCESS in 1 min 31 sec:
openstackhudsonTarmac: This branch should resolve nova bug #718675 (
uvirtbotLaunchpad bug 718675 in nova "Can't create db with PostgreSQL backend" [Medium,Fix committed]
openstackhudsonAlso added dropping of tables in case of failures so `nova-manage sync db` will can restart next time.08:22
uvirtbotLaunchpad bug 718675 in nova "Can't create db with PostgreSQL backend" [Medium,Fix committed]08:22
*** CloudChris has joined #openstack08:26
sorenjaypipes: Oh, you're still here?08:29
sorenjaypipes: I addressed your comment on if you care to take a peek.08:29
jaypipessoren: ya08:34
jaypipessoren: can't sleep...08:34
sorenjaypipes: :(08:35
*** Nacx has joined #openstack08:36
jaypipessoren: approved. :)08:37
* ttx wonders how many times we'll fix one by breaking the other :
*** calavera has joined #openstack08:44
*** ramkrsna has quit IRC08:44
*** allsystemsarego has joined #openstack08:54
*** allsystemsarego has joined #openstack08:54
*** drico has joined #openstack08:54
*** Cybo has quit IRC08:56
jaypipessoren: unittest2 issue?
sorenjaypipes: Weird.08:57
sorenjaypipes: Yeah, maybe.08:57
*** aimon has quit IRC08:59
sorenjaypipes: Ah, assertIsNone is new in Python 2.7.09:00
*** ramkrsna has joined #openstack09:00
*** ramkrsna has joined #openstack09:00
* soren fixes09:00
*** omidhdl has left #openstack09:07
*** aimon has joined #openstack09:08
*** aimon_ has joined #openstack09:11
ttxjaypipes: still around ?09:13
*** aimon has quit IRC09:14
*** aimon_ is now known as aimon09:14
ttxok, good :)09:14
*** sateesh has joined #openstack09:18
*** sateesh is now known as bird09:18
jaypipesttx: yup09:18
DigitalFluxOh, the cloudservers command has been renamed to novatools :-)09:20
DigitalFluxI hope i can find more new stuff/fixes with this push :)09:21
DigitalFluxOK first change, the generated novarc file in Bexar doesn't work09:22
DigitalFluxthe env vars names have also been changed09:22
sorenjaypipes: Oh, in that case, could you re-approve ? I fixed the 2.7-ism.09:24
ttxjaypipes: a number of glance specs are "pending approval" with me set as approver. Do you want me to look at them and approve their design ?09:25
jaypipesttx: yes, please :)09:27
ttxok :)09:27
*** irahgel has joined #openstack09:28
jaypipessoren: done09:30
sorenWe need more core-devs in this timezone.09:30
sorenOr more insomniacs in "the other" timezone.09:31
*** gaveen has quit IRC09:31
*** eikke has joined #openstack09:32
sorenzul: What do we need python-paramiko for?09:33
ttxsoren: so that you can review it, I proposed for merging the adminclient split on the ubuntu packaging09:34
sorenttx: Just now?09:36
sorenttx: Link to the bmp?09:36
DigitalFluxWhat's up with this error -> "novatools.exceptions.OpenStackException: list index out of range (HTTP 500)" ?09:37
DigitalFluxI can almost do everything with novatools09:37
DigitalFluxBut not launch an instance !09:37
ttxsoren: yesterday09:37
jaypipessoren: hehe, ++ on insomniacs, eh? ;)09:38
DigitalFluxon a side note, good to know that ttx is working on the adminclient ;)09:38
ttxDigitalFlux: you should try to corner sandywalsh about that error, he is da novatools master09:39
sorenttx: Ta.09:40
DigitalFluxttx: Great, will do, Thanks09:40
DigitalFluxIRC needs a "timezone" field :/09:42
*** MarcMorata has joined #openstack09:42
*** gaveen has joined #openstack09:43
jaypipessoren: bitten by pep8.09:44
sorenThat makes no sense.09:44
sorenI copied the style from somewhere else.09:44
sorenAnd I thought would run pep8? I fixed that a while ago.09:45
sorenOh. Not if you're passing extra args to nose...  which I always do.09:46
soren(I always pass -x)09:47
sorenttx: adminclient stuff looks great. Merged and pushed. Thanks!09:56
ttxok, cool09:56
ttxI haven't created a dependency for nova on nova.adminclient09:56
ttxI don't think it needs it09:56
openstackhudsonProject nova build #520: SUCCESS in 1 min 35 sec:
openstackhudsonTarmac: Add back --logdir=DIR option. If set, a logfile named after the binary (e.g. nova-api.log) will be kept in DIR.09:57
*** miclorb_ has joined #openstack09:58
*** guigui has quit IRC09:59
*** nati has quit IRC10:03
sorenttx: I thought about that too and decided against it.10:04
jaypipesalright, g'night guys... gonna try and get a few hours of sleep. night. anyone feel like reviewing any/all of the little branches in Glance here would be most appreciated:
* soren takes a look10:08
*** kashyapc has quit IRC10:08
*** kashyapc has joined #openstack10:08
*** ramkrsna__ has joined #openstack10:08
*** aimon has quit IRC10:09
*** ramkrsna has quit IRC10:11
*** miclorb_ has quit IRC10:15
*** miclorb_ has joined #openstack10:23
*** bird has quit IRC10:27
*** sateesh has joined #openstack10:27
*** sateesh is now known as bird10:28
*** herki has quit IRC10:29
*** aimon has joined #openstack10:35
*** miclor___ has joined #openstack10:59
*** aimon has quit IRC11:00
*** miclorb_ has quit IRC11:01
*** aimon has joined #openstack11:17
*** miclor___ has quit IRC11:31
uvirtbotNew bug: #719967 in nova "UnicodeDecodeError on project description" [Undecided,New]
*** h0cin has joined #openstack11:38
*** thimble has quit IRC11:46
*** berendt has quit IRC11:47
*** CloudChris has quit IRC11:52
*** CloudChris has joined #openstack11:52
*** drico has quit IRC11:54
*** ctennis has quit IRC11:56
*** DigitalFlux has left #openstack11:58
*** DigitalFlux has joined #openstack11:58
*** DigitalFlux has left #openstack11:58
*** DigitalFlux has joined #openstack11:58
*** jheiss has quit IRC12:00
*** j05h has joined #openstack12:00
*** jheiss has joined #openstack12:07
*** fabiand_ has joined #openstack12:15
sorenxtoddx: I'd appreciate it if you could approve since you had reservations about it.12:25
*** guigui has joined #openstack12:37
*** eikke has quit IRC12:42
*** CloudChris has quit IRC12:42
*** CloudChris has joined #openstack12:43
*** bird has quit IRC12:43
*** bird has joined #openstack12:43
ttxwhile you're at it, please also have a look into xtoddx: since you wrote the original code12:44
ttxxtoddx: while you're at it, please also have a look into since you wrote the original code12:44
*** eikke has joined #openstack12:49
sorenjaypipes: Ok, so I want to make Glance support EC2 style image uploads.12:57
sorenjaypipes: I'm hoping you can guide me a bit.12:57
sorenjaypipes: EC2 style image uploads work as follows:12:57
sorenjaypipes: 1. You have an image locally. You use a special tool to turn that image into a bunch of compressed, signed (maybe encrypted, I don't remember), parts plus a manifest.12:58
*** eikke has quit IRC12:59
sorenjaypipes: 2. You use an upload tool to upload all these things to S3. The tool just reads the manifest and uploads everything, it doesn't process the contents or anything cool like that.12:59
zulsoren: its needed because of nova/volume/san.py13:00
sorenzul: Ah, so it is.13:00
sorenzul: Cool, thanks.13:00
*** berendt has joined #openstack13:01
sorenjaypipes: 3. You call EC2's RegisterImage method with an S3 URL pointing to the manifest. This makes EC2 grab the manifest and the image parts, checks the signature, (decrypts the parts?), and stitches the parts together, forming a complete image.13:02
sorenjaypipes: ...and in response, you get back an ID for the image.13:02
sorenjaypipes: Would I just let the register part contain the S3 URL, and then do the download when asked to run the image for the first time?13:03
*** desai has joined #openstack13:03
sorenjaypipes: Is there no validation of the given URL during the registration phase?13:03
*** rossij has quit IRC13:04
*** thimble has joined #openstack13:05
*** eikke has joined #openstack13:11
*** rchavik has quit IRC13:17
*** hggdh_ has joined #openstack13:23
*** hggdh has quit IRC13:23
*** kashyapc has quit IRC13:25
*** DanEllis has joined #openstack13:38
*** jbaker__ has quit IRC13:42
*** nelson has quit IRC13:52
*** nelson has joined #openstack13:52
*** dprince has joined #openstack13:54
*** aimon has quit IRC13:58
*** hadrian has joined #openstack14:00
*** fabiand_ has quit IRC14:06
*** allsystemsarego has quit IRC14:13
*** westmaas has joined #openstack14:15
*** hggdh_ is now known as hggdh14:16
*** fabiand_ has joined #openstack14:17
*** dprince has quit IRC14:17
*** dprince has joined #openstack14:19
*** desai has quit IRC14:20
*** desai has joined #openstack14:21
*** desai has quit IRC14:23
*** Ryan_Lane has joined #openstack14:28
*** thimble has quit IRC14:30
DigitalFluxSo i just fired an instance with novatools14:33
*** ppetraki has joined #openstack14:33
DigitalFluxhowever looks like curl can't get the image from the nova node14:33
*** thimble has joined #openstack14:35
*** aryko has joined #openstack14:41
*** gaveen has quit IRC14:42
*** Paul_Miller has joined #openstack14:42
*** f4m8 is now known as f4m8_14:49
*** Ryan_Lane is now known as Ryan_Lane|dc14:49
*** ramkrsna__ has quit IRC14:52
*** dirakx has quit IRC14:53
*** kashyapc has joined #openstack14:56
*** skiold has joined #openstack15:03
*** pvo has joined #openstack15:03
*** m_3 has quit IRC15:07
*** Paul_Miller has quit IRC15:08
ttxDigitalFlux: what was the problem with that 500 error you had ?15:10
ttxDigitalFlux: what do you use for an image service ?15:13
*** m_3 has joined #openstack15:13
ttxDigitalFlux: If you use Glance you should run with --image_service=nova.image.glance.GlanceImageService15:14
*** hub_cap has joined #openstack15:16
DigitalFluxttx: yes it's Glance15:17
DigitalFluxwith the flag you mentioned already in nova.conf15:17
jarrodcan libvirt/kvm make volume snapshots yet?15:17
*** gondoi has joined #openstack15:17
DigitalFluxjarrod: I truly need that too15:18
DigitalFluxttx: so what do you think ?15:18
jarrodhmm.. i can look into contributing15:19
ttxDigitalFlux: what version are you running ?15:20
DigitalFluxttx: Glance ?15:20
*** tmarble_ is now known as tmarble15:20
DigitalFluxttx: The trunk, installed a couple of hours ago15:20
ttxDigitalFlux: I hit that bug, but fixed it in
uvirtbotLaunchpad bug 708673 in nova "libvirt/hyperv do not support Glance" [Wishlist,Fix released]15:20
DigitalFluxttx: I see, I am already using Bexar15:21
*** aryko has quit IRC15:22
ttxDigitalFlux: if you look at the patch if image_service=nova.image.glance.GlanceImageService it should not fetch a :3333 url15:22
*** westmaas has quit IRC15:23
ttxDigitalFlux: the fix is in Bexar release so if you run that it should work...15:24
*** westmaas has joined #openstack15:24
DigitalFluxttx: it should NOT fetch a :3333 URL ?15:25
DigitalFluxit's fetching a :3333 URL currently15:25
ttxDigitalFlux: right, that's where I don't understand. With the config you claim to have it should not happen.15:26
DigitalFluxttx: i see15:26
DigitalFluxwell, i have --image_service=nova.image.glance.GlanceImageService on nova.conf of the main node and also the compute node15:26
DigitalFluxthat's weird ..15:26
*** msassak has joined #openstack15:27
DigitalFluxI was this close to launching an instance via the novatools :(15:28
DigitalFluxBut i hit this image issue ..15:28
berendtat the moment only on instance per compute node is launched at one point. how is it possible to launch more instances at the same time on one node?15:29
*** westmaas has quit IRC15:29
*** joearnold has joined #openstack15:32
*** joearnold has quit IRC15:41
*** vvuksan has joined #openstack15:41
jaypipessoren: I'm up now, after a terrible mornings sleep. I'll respond to your EC2/Glance stuff after I grab some coffee and breakfast.15:42
ttxsoren / nova-core: you can probably switch to approved now that fix-unittest is in15:42
*** dendrobates is now known as dendro-afk15:42
*** westmaas has joined #openstack15:43
jaypipesDigitalFlux, jarrod: Can I ask a favour? Could you start a conversation on the mailing list about what you were discussing above re: Glance, libvirt/kvm snapshots?  I'd like to get more opinion on that. IRC is great, but mailing list can generate some excellent threads as well...cheers.15:44
DigitalFluxjaypipes: OK, will do ;)15:47
uvirtbotNew bug: #720133 in nova "Nova returns non-compliant tag for DescribeKeypairs" [Undecided,New]
*** DigitalFlux has quit IRC15:47
*** fabiand_ has quit IRC15:54
*** dragondm has joined #openstack15:58
*** troytoman-away is now known as troytoman16:03
Kegarohello.  Where does swift write it's error logs?16:03
KegaroI see this in an strace:   "write(2, "Error: [swift-hash]: swift_hash_"..., 78) = 78"    but I can't figure out where it is writing this file to read the rest of the error.16:04
vish1soren: I think we need to separate out crypto to make it actually work in glance.  Glance needs a way to retrieve the projects key to decrypt.16:05
uvirtbotNew bug: #720157 in nova "Nova returns HTTP 400 for SignatureVersion=1 requests" [Undecided,New]
*** dirakx has joined #openstack16:12
sorenvish1: That was the plan, yeah.16:13
sorenttx: Marked as approved. Thanks for the reminder.16:15
*** vish1 is now known as vishy16:16
vishysoren: I was considering making it a separate worker16:16
vishysoren nova-crypto or something16:16
vishysoren: was that your plan?16:16
vishysoren: the code for unpacking the image should actually go in to ec2_api imo16:17
sorenvishy: It was one of the options.16:17
sorenAnyways, /me goes to do dinner stuff16:17
vishyi think rather than making glance support an s3 front end16:17
*** berendt has quit IRC16:19
ttx2011.1.1 nova/bexar branch merge proposals up at
*** thimble has quit IRC16:19
ttxnote that I'd prefer if the translationbs branch landed in cactus first16:20
ttxso give some love please :)16:20
creihtKegaro: by default the logs will go to /var/log/syslog or messages16:21
*** CloudChris has quit IRC16:22
*** CloudChris has joined #openstack16:22
openstackhudsonProject nova build #521: SUCCESS in 1 min 34 sec:
openstackhudsonTarmac: Fix for bug #71684716:22
uvirtbotLaunchpad bug 716847 in nova "euca-delete-volume cannot delete volumes in error state" [Medium,In progress]
openstackhudsonIf a volume has not yet been assigned to a host, then have "nova-manage volume delete" delete from database and skip the rpc.16:22
*** desai has joined #openstack16:23
ttxsoren: i suspect that tarmac won't pick up approved BMPs on lp:nova/bexar, and we'll have to land them manually ?16:23
*** sebastianstadil has quit IRC16:24
*** bird has quit IRC16:25
*** guigui has quit IRC16:26
*** skiold has quit IRC16:31
*** kashyapc has quit IRC16:33
*** sebastianstadil has joined #openstack16:34
ttxjaypipes: could you look into sirp's last comment on ? I'd hate it if we were breaking one while fixing the other16:34
*** rds__ has joined #openstack16:35
zulsoren: what do you think of shipping things like virt/libvirt virt/xenapi in their own seperate packages16:35
*** skiold has joined #openstack16:36
*** skiold has quit IRC16:42
mtaylorsoren: ping16:43
*** reldan has joined #openstack16:44
mtaylorttx: we can set up a tarmac job that watches lp:nova/bexar16:44
*** sebastianstadil_ has joined #openstack16:45
*** sebastianstadil has quit IRC16:47
*** sebastianstadil_ is now known as sebastianstadil16:47
*** kashyapc has joined #openstack16:50
*** CloudChris has quit IRC16:51
*** rds__ has quit IRC16:54
*** calavera has quit IRC16:55
*** ccustine has joined #openstack16:56
jaypipesttx: we aren't. dprince's fix is correct. I'll note on the merge prop.16:58
dprincettx, jaypipes: Sorry. I just saw that comment. I'll update the merge proposal.17:00
*** littleidea has joined #openstack17:00
jaypipesdprince: no, please don't. your fix is correct.17:00
dprincejaypipes: Sorry. I just meant I'd add a comment that it works fine.17:01
jaypipesdprince: ah, sorry, misunderstood :)17:01
dprincejaypipes: Hey. While I've got your attention I'm looking at now.17:01
uvirtbotLaunchpad bug 709355 in nova ""kernel_id property not found" Openstack API w/ S3ImageService " [Medium,In progress]17:01
jaypipesdprince: looking..17:02
dprincejaypipes: I've pretty much got it however are you cool with relaxing that check on kernel_id and ramdisk so that it just passes.17:02
dprincejaypipes: the _get_kernel_ramdisk_from_image function.17:03
jaypipesdprince: lemme look. gimme a few ;)17:04
dprincejaypipes: Okay. One more thing. Glance uses image['properties']['kernel_id'] where S3 just uses image['kernel_id'].17:06
dprincejaypipes: should we standardize these further?17:06
jaypipesdprince: unfortunately, your bugs about the s3/glance/nova/objectstore inconsistencies hit while I was down in San Antonio, and that whole week was a bit of a mess :)17:07
dprincejaypipes: or at least put the check in the compute/api code?17:07
dprincejaypipes: Sure. Understood. :) I was waiting on devcamcar's patch anyway.17:07
jaypipesdprince: as for "standardizing", we just need to make sure the API represented by the BaseImageService explains what is expected to be returned. Right now, only "top-level" attributes like type, is_public, etc, are explained in the API.  What do you think? Should be standardize on kernel_id and ramdisk_id in properties? Or should they be "top-level" attributes?17:08
dprincejaypipes: I can go either way. A slight preference for them being top level within the nova code.17:10
jaypipesdprince: k. devcamcar, any opinion?17:10
doude_Hi all, the bug makes Glance unable to fetch images with a Swift backend (in the bexar releases) ?17:11
uvirtbotLaunchpad bug 717431 in glance "Swift backend not taking into account new Swift account:user:key in URI" [High,Confirmed]17:11
jaypipesdprince: even though kernel_id and ramdisk_id can very likely be None?17:11
jaypipesdprince: I don't mind, either way, to be honest :)17:11
*** rcc has quit IRC17:11
jaypipesdprince: let's just go ahead and make them "top-level attributes" as far as Nova is concerned, then.17:11
dprincejaypipes: gotcha. So another question since I just saw this ticket. Should I do #717431 at the same time?17:12
dprincejaypipes: These are related issues here.17:12
dprincejaypipes: I might as well get them all at once...17:12
jaypipesdprince: so, the "fix" is 1) modify BaseImageService to be 100% clear about what attributes are expected to be returned from each implementing image service 2) Make sure the various ImageService subclasses are translating any keys (or lack of keys) coming from the host image service (glance, local or nova-objectstore) into the keys that are required by the documented API in BaseImageService.17:13
jaypipesdprince: that sound like a plan to you? ^^17:13
dprincejaypipes: gotcha17:13
jaypipesdprince: yes, I would do both in a single branch, since they're interrelated.17:13
jaypipesdprince: you might even mark 717431 a dup?17:14
jaypipesdprince: and finally, 3) tests, tests, tests ;)17:14
dprincejaypipes: Okay. I'll try and get this stuff worked out today then. Thanks.17:14
*** sebastianstadil has quit IRC17:14
dprincejaypipes: That brings up another question... Glance. So the cactus branch now uses a config file again?17:15
*** littleidea has quit IRC17:16
jaypipesdprince: config file is for paste.deploy. And yes, check the docs here for info on starting/stopping/restarting Glance servers in Cactus:
dprincejaypipes: Thank you sir.17:17
dprinceback in a bit...17:17
*** dprince has quit IRC17:17
doude_jaypipes:Hi, the bug that you opened, makes Glance unable to fetch images with a Swift backend (in the bexar releases) ?17:20
uvirtbotLaunchpad bug 717431 in glance "Swift backend not taking into account new Swift account:user:key in URI" [High,Confirmed]17:20
*** jbaker_ has joined #openstack17:21
doude_jaypipes: no workaround ?17:21
*** maplebed has joined #openstack17:22
*** vvuksan has quit IRC17:23
mtaylorcreiht: hey. ok - let's chat about packages real quick17:25
mtaylorcreiht: I'm cutting 1.2 packages (sorry for the delay) ... but I'm a little confused about versioning. do we want them in the 1.2 ppa? or in the swift-core ppa?17:26
*** maplebed has quit IRC17:26
mtaylorcreiht: it feels to me like the want to be in the normal ppa - but I suppose if you want people to still be able to install 1.1 for some reason?17:27
*** maplebed has joined #openstack17:27
jaypipesdoude_: looking...17:28
*** blueadept has joined #openstack17:28
jaypipesdoude_: I didn't think there was a workaround, no. Did you know of one?17:28
jaypipesdoude_: or were you making a subtle suggestion that I should increase the priority of the bug? ;)17:30
*** reldan has quit IRC17:30
doude_jaypipes: no I don't find one and I don't want you increase the priority. But perhaps, it will be nice to update the release note of Bexar release17:34
jaypipesdoude_: much agreed. :)17:36
doude_jaypipes: the release note says "The S3 and Swift backends do not currently support the POST /images/ API command in the Glance API. These backends only currently support fetching disk images via GET calls. Support for storing disk images in S3 and Swift directly through the Glance API is planned for the Cactus release"17:36
doude_jaypipes: Ok, bye17:37
jaypipesdoude_: I'll update the release notes accordingly. cheers!17:38
*** joearnold has joined #openstack17:41
*** Nacx has quit IRC17:42
jaypipessandywalsh_: what are you using as a mail client to respond to dabo?  MS Word? ;P17:43
sandywalsh_bad enough ... outlook web (the rs standard)17:43
*** Ryan_Lane|dc has quit IRC17:43
jaypipessandywalsh_: or possibly Rackspace's wonderful Outlook Web Access!? :)17:43
jaypipessandywalsh_: yup, that's what I figured. god I hate MS Outlook Web Access...17:43
mtaylorrs standard is dumb in this case17:43
* jaypipes runs from johnpur17:43
*** Ryan_Lane|dc has joined #openstack17:44
jaypipesthe preferred mail client for monkeys.17:44
* mtaylor stands his ground ... anything that thinks that a) executing code found in emails and b) sending emails in ms word format are good ideas is obviously flawed17:44
jaypipessandywalsh_: you in SA or something? not at home in Nova Scotia?17:45
sandywalsh_jaypipes, I'm home17:46
jaypipessandywalsh_: hmm, I won't ask then :)17:46
sandywalsh_jaypipes, heh. I use gmail for my normal email. Don't want to mix business and pleasure :)17:46
jk0kpepple: how close are you to proposing that branch to trunk?17:47
jk0kpepple: I only have a few more changes -- do you think it's possible to have it proposed by tomorrow sometime?17:47
jk0kpepple: if it's good to go now, I can always propose my changes later too17:48
kpepplejk0: it's feature functional and passing all new tests ... i only have coding style issues (mainly that i'd like to mox instance_types instead of leaving the constant in tests)17:48
jk0all I'm working on is adding create/destroy to the OS API17:48
kpepplejk0: why don't you finish up tonight and we'll push for code review tomorrow pm ?17:49
jk0other than that, it's done17:49
jk0that sounds great17:49
kpepplejk0: i'm traveling today, so can you hold nova-core from merging the config file patch that was proposed ?17:50
jk0which one was that?17:50
kpepplejk0: and
uvirtbotLaunchpad bug 681411 in nova "Instance types should not be hard coded" [Wishlist,In progress]17:52
jk0yeah, that wont go in. we talked about it in email this morning17:52
kpepplejk0: okay. cool.17:52
*** sebastianstadil has joined #openstack17:54
tr3buchetopenstack process question: suppose i have a cactus approved blueprint and I'd like to split it into multiple blueprints.. since we're after the blueprint freeze, what can I do?18:05
*** hub_cap_ has joined #openstack18:08
*** vvuksan has joined #openstack18:08
*** hub_cap has quit IRC18:09
*** hub_cap_ is now known as hub_cap18:09
vishyRyan_Lane: are you here?18:11
uvirtbotNew bug: #720252 in nova "adminclient create_user fails with --auth_driver=nova.auth.ldapdriver.LdapDriver" [Undecided,New]
sirp_dprince: whenever you're available, have a question re: imageservice change18:23
*** RobertLaptop has left #openstack18:24
*** drico has joined #openstack18:24
*** evtoews has quit IRC18:27
*** allsystemsarego has joined #openstack18:28
openflyhow do I diagnose a problem with the meta-data server operating on 8773 ?18:33
openflygetting 500 internal errors.18:33
jaypipestr3buchet: just split it.18:35
vvuksanopenfly: watch nova-api debug log18:35
*** littleidea has joined #openstack18:51
creihtmtaylor: hrm18:51
creihtmtaylor: can't you still have multiple versions available in the PPA?18:51
mtaylorno. not in a single ppa18:51
creihtand I'm not sure where the other PPA sections came from18:51
openflyvvuksan thanks18:52
openflywill do18:52
mtaylora single ppa is like a miniature debian archive, so it only keeps the most recent version18:52
openfly011-02-16 10:53:55,324 CRITICAL nova.root [-] [Errno 2] No such file or directory: '/usr/lib/pymodules/python2.6/nova/..//CA'18:54
openflythat's the symlink to /var/lib/nova/CA right?18:54
vvuksanopenfly: read Find section that talks about copying to /var/lib/nova/CA18:55
creihtmtaylor: let me do a little research18:55
* jaypipes taking a short break before tackling the live-migrations review for the first time....deep breath...18:58
openflyi fixed the CA issue19:01
openflythis i can't seem to figure out though19:01
*** Pentheus has joined #openstack19:01
*** jheiss has quit IRC19:02
openflynova-api  14130     nova    5u  IPv4 336485      0t0  TCP *:8773 (LISTEN)19:03
jaypipesdevcamcar: what the heck was that latest DEFINE_integer thing on soren's merge?19:03
openflywell that's just plain wierd19:03
henrichrubinanyone know why this nova-networks fails from the latest trunk during launching an instance?
henrichrubinare the instances supposed to be removed from the instances db after a euca-terminate-instance?19:04
openflyhenrichrubin beyond me.19:08
vvuksanopenfly: there is an existing nova-api running19:09
vvuksanopenfly: kill it19:09
*** dprince has joined #openstack19:09
dprincesirp_: hi. What is your question?19:11
henrichrubinopenfly:  thanks.  anyone else have a clue on this error w/ the latest trunk?
vvuksanhenrichrubin: doesn't look like instances table is cleaned up19:11
vvuksanhenrichrubin: I assume it's needed for accounting purposes19:12
henrichrubinvvuksan:  if a euca-run-instances stays in the networking state b/c nova-network fails, then i terminate the instance, it is still in the instances table in the 'shutdown' state.  would this cause this problem?  if so, then this is a bug.19:13
*** clauden has joined #openstack19:13
vvuksanhenrichrubin: i suppose it's possible. You are using a /28 block so you may be exhausting the 14 ips19:14
*** MarcMorata has quit IRC19:14
*** bcwaldon has quit IRC19:17
*** sebastianstadil has quit IRC19:19
*** fabiand_ has joined #openstack19:21
*** hub_cap has quit IRC19:21
henrichrubinvvuksan:  i deleted all the instances from the table and have the same problem19:22
*** hub_cap has joined #openstack19:22
sorenjaypipes: I think it's a mutiny by my brain. It's telling me to stop proposing branches at such an ungodly hour.19:23
*** imsplitbit has joined #openstack19:23
openflyokay so vvuksan i shut down the nova-api19:24
openflythen i ran it through watch19:24
openflyand got no output at all19:24
openflystill getting 500 internal errors on the meta-data service19:24
vvuksanI run nova-api in a screen session with nova-api --nodaemon --flagfile /etc/nova/nova.conf19:25
vvuksanhenrichrubin: hmmm19:25
openflyokay did that vvuksan i see it starting up... binding to a port19:26
vvuksanhenrichrubin: type ps auxww | grep dnsmasq on the nova-network node.19:26
asksolsandywalsh_: what are the quirky errors you refer to here? ?19:26
openflybut when i do a
openflyi get an internal 500 error19:26
openflyand nothing in the api logs19:26
*** elastic_dog has quit IRC19:27
vvuksanthat's odd19:27
sandywalsh_asksol, ok, it was the error passing, which you fixed in that patch. I should update that page19:28
asksolsandywalsh_: ok :)19:28
openflymaybe i should try bouncing the api server through strace...19:28
*** elastic_dog has joined #openstack19:28
asksolthat looks really interesting, btw19:28
sandywalsh_interesting is a good word for it :)19:28
henrichrubinvvuksan: .  this is the line that causes the error "network_ref = db.fixed_ip_get_network(context, address)"19:29
henrichrubincould it be b/c the network table id doesn't start at 1?19:30
vvuksanhenrichrubin: i know but I don't quite get that piece of code :-)19:30
vvuksanhenrichrubin: btw has this ever worked for you ?19:31
vvuksanhenrichrubin: the nova network that is ?19:31
henrichrubinvvuksan:  i think all it does is look in the network table and returns a context based upon the address it returns from the DHCP command19:31
vvuksanhenrichrubin: what it does it looks for the bridge name then returns the proper network settings19:31
vvuksanDHCP settings19:32
henrichrubinthis trunk has never worked for me.  i had an earlier branch working on single node, but am trying on multi-node now and i can't get past this.19:32
vvuksani have had trouble with it since this dhcp-range= has to be in the range of the br100 interface19:32
vvuksanie. i was using addresses19:32
asksolsandywalsh_: do you know what the problems with AMQP the newly proposed QueueService is supposed to solve?19:32
vvuksanand since dhcp-range said dnsmasq would refuse to hand out addresses19:33
*** jc_smith has joined #openstack19:33
henrichrubinvvuksan:  so what address did you use for your nova-network?19:33
vvuksanwhat I ended up doing19:33
sandywalsh_asksol, I haven't read the entire thread on the ML yet, perhaps eday has a blue print he can point you to?19:33
vvuksanhenrichrubin: add following to /etc/nova/nova.conf --flat_network_dhcp_start=
devcamcarjaypipes: what DEFINE_integer thing on which soren's merge? :)19:34
henrichrubini am using the same address for "nova-network create" as "fixed_range" in nova.conf.19:34
*** sebastianstadil has joined #openstack19:34
vvuksanhenrichrubin: assuming that br100 is e.g.
* soren hugs devcamcar 19:35
vvuksanhenrichrubin: it has to be in the range19:35
henrichrubinok, my br100 is different than, so that may be the problem?19:35
devcamcarsoren: <319:35
* soren is rather embarrassed about it, really.19:35
vvuksanhenrichrubin: yes it has to be in the range. However that shouldn't break nova-network19:35
*** colinnich_ has quit IRC19:36
vvuksanhenrichrubin: also these are helpful --dhcpbridge=/usr/bin/nova-dhcpbridge19:36
sorenI seem to have this dreadful habit of doing all the hard bits (admittedly, in this case, the hard bits weren't really that hard), running the unit tests, going "cool, it works, I'll just make this last change, so the patch looks prettier." and then apparantly forget to rerun the test suite.19:36
* soren has an idea for a bzr plugin19:37
*** colinnich has joined #openstack19:37
*** troytoman has quit IRC19:38
*** troytoman has joined #openstack19:38
ttxmtaylor: might be the easiest solution, even if it's a one-off, we can disable it afterwards19:39
*** drico has quit IRC19:40
sorenmtaylor: wazzup?19:41
devcamcarsoren: i had a moment of concern the other day when i submitted a patch that had nothing to do with sec groups, and then all the sec group tests broke.  it was a real head scratcher, but turned out something else broke the sec group tests.  i'm still confused about how that happened actually19:41
sorendevcamcar: I can explain that.19:41
sorendevcamcar: It took almost a day to work out.19:42
mtaylorsoren: I was going to ask you questions about what you'd done to swift packaging - but I got it worked out19:42
sorendevcamcar: It turned out to be because we now order security groups in DescribeSecurityGroups by name or something.19:42
sorendevcamcar: ...and the test case generates a random name.19:42
sorendevcamcar: If the random name sorted earlier than "default", it failed. If not, it succeeded.19:43
devcamcarah hah!19:43
sorendevcamcar: Before, we'd always get them in the order in which they were found in the DB.19:43
sorendevcamcar: It had *all* the character traits of a race condition, so I was looking in all the wrong places.19:43
henrichrubinvvuksan:  i tried the dhcp_start in nova.conf but it gives an error.  seems like before dnsmasq starts....19:44
Ryan_Lane|dcvishy: yes/no. I'm building out a new datacenter, so I'm here, but will be not terribly responsive19:44
vvuksanhenrichrubin: kill dnsmasq before you rerun nova-network19:45
henrichrubinvvuksan:  it is already dead.  and it doesn't start when restarting nova-network19:46
mtaylorttx: ok.
mtaylorttx, soren: ^^^ I set it up to do tarmac and docs publishing. I did not set up pep8 or pylint or tarball publishing19:48
ttxmtaylor/soren: I guess I culd use tarball publishing.19:48
annegentlemtaylor: just Sphinx dox publishing or also the new DocBook publishing (ant and maven)?19:49
ttxthat would prevent me from doing it manually, which is errorprone19:49
creihtmtaylor: for now lets just do the release in the current release ppa19:49
sorenttx, mtaylor: Yeah, tarballs should be safe.19:50
mtaylorannegentle: there is new docbook publishing?19:50
creihtI don't think there is a reason for keeping around the 1.1 ppa19:50
mtaylorcreiht: ok. cool.19:50
creihtIf there becaomes one at some point, then we can facilitate that19:51
annegentlemtaylor: yeah I sent an email with a description and request... with details. I can meet with you to go through it if needed. Maybe you filter out anne@openstack :)19:51
ttxmtaylor/soren: once we have tarballs publishing set up, we can start approving the bmps on lp:nova/bexar19:51
creihtBut I don't want to add extra burden unless really needed19:51
mtaylorannegentle: I filter _many_ things :)19:51
annegentlemtaylor: ayup! email is for email! Or something like that. :)19:51
annegentlemtaylor: completely see my request as further down on the todo list though19:52
mtaylorannegentle: oh hey look... there's the email19:52
openstackhudsonProject nova-bexar build #1: FAILURE in 1 min 32 sec:
mtaylorannegentle: should I ask why we want two doc systems now? or just be snarky about it next time I see you in person? ;)19:53
ttxmtaylor: failure ? ^19:53
mtaylorrsync: mkstemp "/srv/docs/nova/bexar/.nova.concepts.html.Q2KkGp" failed: Permission denied (13)19:53
annegentlemtaylor: snark is welcome :) RST didn't meet all the requirements - I hemmed and hawwed (just ask jbryce) and went with XML19:53
*** rnirmal has joined #openstack19:54
mtaylorwell, as long as it caused you and continues to cause you pain, I guess I'm ok with it. :)19:54
devcamcarsoren: yea, that's good times!19:55
*** olivier_ has quit IRC19:55
mtaylorsoren: do you have root on
mtaylorsoren: and if so, can you please add me to the sudoers file?19:56
mtaylorsoren: or, just do chown hudson:hudson /srv/docs/nova/bexar/19:57
mtaylorantonym: ^^^19:57
sorenmtaylor: I am, and I can.19:59
*** fabiand_ has quit IRC19:59
antonymsoren: you have it?19:59
sorenmtaylor: You don20:00
sorent even have a user on that box, do you?20:00
sorenOh, mordred.20:00
soreni see it.20:00
sorenmtaylor: Done.20:00
sorenmtaylor: Enjoy.20:00
openstackhudsonProject swift build #198: SUCCESS in 29 sec:
openstackhudsonTarmac: creating a Ring will ensure a valid HASH_PATH_SUFFIX20:02
openstackhudsonProject nova build #522: SUCCESS in 1 min 38 sec:
openstackhudsonTarmac: Use a threadpool for handling requests coming in through RPC.20:02
mtaylorsoren: you rock20:07
*** dendro-afk is now known as dendrobates20:08
mtaylorsoren: sorry to bug you - but I don't actually seem to have a working password on that machine :)20:08
asksolsandywalsh_: i only saw the wiki page, and it had very little rationale for such a big move.  but is the plan to replace rabbitmq in Nova with this?20:08
sandywalsh_asksol, I don't think so initially, but if it evolves to a point where it could ...20:09
sandywalsh_asksol, I think they have some internal plans for it out of the gate20:09
*** littleidea has quit IRC20:10
sorenmtaylor: on it20:10
asksolsandywalsh_: ok, thank you20:11
*** h0cin has quit IRC20:11
desaidoes anyone have any pointers on tuning kvm?20:11
*** joearnold has quit IRC20:13
*** olivier_ has joined #openstack20:14
openstackhudsonYippie, build fixed!20:15
openstackhudsonProject nova-bexar build #2: FIXED in 1 min 36 sec:
openstackhudsonProject nova build #523: SUCCESS in 1 min 33 sec:
openstackhudsonTarmac: Use RotatingFileHandler instead of FileHandler.20:17
* soren adjusts packaging20:18
*** eikke has quit IRC20:22
*** joearnold has joined #openstack20:23
*** troytoman is now known as troytoman-away20:23
*** lvaughn_ has quit IRC20:26
*** lvaughn has joined #openstack20:27
*** irahgel has left #openstack20:33
*** sebastianstadil has quit IRC20:34
desaiso it looks like nova-compute starts up kvm with a restricted set of cpu features passed through to instances; is there a way to tune that? i'm not seeing anything in the code20:35
openflywell i got nothing,. my metadata server is still spitting out internal error 500 and i still have no logs anywhere telling me why that could be20:36
*** drico has joined #openstack20:37
desaiopenfly: whenever i have metadata server problems, i get errors in the nova-api.log20:38
desaisometimes it looks like log rotation gets off kilter tho; you might check nova-api.log.*20:39
*** lvaughn_ has joined #openstack20:39
*** lvaughn has quit IRC20:42
vvuksandesai: like what options ?20:42
openflydesai there is nothing in that log20:43
openflyit all looks like it's operating correctly.20:43
openflyjust the running instances keep erroring out trying to contact the meta data controller...20:43
openflythe nat / iptables rules are there20:43
openflyand locally i can't hit the metadata url either without that 500 internal error20:44
desaivvuksan: i'm trying to get sse3 passed through to the vm, otherwise our apps take a performance hit20:44
desaiopenfly: any metadata url or a particular one?20:44
desaiwe had a problem with the availability-zone one20:44
desaibut not with the others20:44
*** joearnold has quit IRC20:44
openflyi can hit the list of dates20:44
vvuksandesai: if you have an idea how to pass it in in the libvirt.xml you can change the template20:44
openflybut anything inside the dates 50020:44
tr3buchetcan't get nova network to start:
tr3buchetany ideas?20:45
tr3buchethappens in trunk and in the past few reviews20:45
desaivvuksan: no love with that yet; it looks like you can tell qemu/kvm directly with a command line flag, but i can't figure out how to get that through libvirt20:45
vvuksandesai: what's the flag?20:45
desai-cpu on the command line20:46
vvuksandesai: perhaps this
vvuksanwhat is the invocation for -cpu ie. complete args ?20:49
*** sebastianstadil has joined #openstack20:49
*** nelson has quit IRC20:50
*** nelson has joined #openstack20:50
desai-cpu host, will pass through the host system capabilities20:51
openflyman installing openstack is way too hard.20:51
*** sebastianstadil has quit IRC20:51
*** decramy has quit IRC20:51
tr3buchetso is dating XD20:52
tr3buchetopenstack has better docs though20:52
openflyI dunno20:52
openflyI am pretty sure openstack docs are comparable to docs on dating20:52
openflya million methods that all don't work.20:52
vvuksanopenfly: yes it has been a challenge for me as well20:52
tr3bucheti have more trouble keeping it working20:53
jaypipesopenfly: Nova contestant #3, what is your preferred hypervisor? ooooh, XenServer? Good answer, contestant #3.20:54
vvuksanopenfly: actually you know what :-)20:54
*** Ryan_Lane|dc is now known as Ryan_Lane20:54
vvuksanopenfly: I get 500 doing 2009-04-04 as well20:54
vvuksanthat may be quite normal20:54
tr3buchetjaypipes: nice :D20:55
openflyvvuksan is the correct next argument instanceid?20:55
vvuksanlet me look20:55
openfly2011-02-16 20:54:46,086 -[WARNING]: waiting for metadata service at http:\/\/\/2009-04-04\/meta-data\/instance-id20:55
openflythat's what i am seeing in my instances20:55
vvuksanthat could be it20:56
*** sebastianstadil has joined #openstack20:56
openflyno joy on that20:57
openflystill 50020:57
vvuksanare you executing that from the instance itself ?20:59
*** Ryan_Lane is now known as Ryan_Lane|dc20:59
openflyi can't get into the instances20:59
openflythey are hanging trying21:00
* jaypipes realized that his reference to the 70s game show "The Dating Game" may have been lost on some poor souls in the IRC channel... (
vvuksanon the network node21:00
openfly2011-02-16 21:00:11,455 -[WARNING]:   21:00:11 [47\/100]: url error [timed out]21:00
vvuksaniptables -t nat -L21:00
vvuksandoes 169.254 show up ?21:00
openflyDNAT       udp  --  anywhere             barad-dur           udp dpt:1000 to:
openflyDNAT       tcp  --  anywhere        tcp dpt:www to:
vvuksanthat looks good21:01
vvuksantry this21:01
vvuksanifconfig eth0:100 inet
vvuksanyou can ping the IP now ?21:02
openflythe instances are still timing out21:03
vvuksantry restarting instances21:03
jarrodis there anyway to use VlanManager, but not use NAT and allow the instances to possess the public IP?21:03
openflysame error21:05
openflyi can't ping the instance from the api system21:06
openflyi added a default allow in the availability zone21:06
openflyand i see it in the iptables21:06
openflymaybe there is a forwarding issue between bridges21:06
openflymight have found it21:07
vvuksanopenfly: r u using DHCP ?21:07
vvuksanturn of stp21:07
*** ironcamel2 has joined #openstack21:07
openflydidn't see sysctl net.ipv4.ip_forward21:08
openflyin the sysctl.conf21:08
openflybut it looks right in /proc21:08
openflyso guess it's not that21:08
vvuksansysctl -a | grep forwa21:08
vvuksanyes brctl stp br100 off21:08
openflynet.ipv4.conf.all.forwarding = 121:09
openflynet.ipv4.conf.all.mc_forwarding = 021:09
openflynet.ipv4.conf.default.forwarding = 121:09
openflynet.ipv4.conf.default.mc_forwarding = 021:09
vvuksannet.ipv4.ip_forward = 121:09
vvuksan ?21:09
henrichrubintr3buchet:  did you get that error fixed yet?   if not try apt-get update; apt-get install python-eventlet21:09
vvuksani would have assumed libvirt would have turned on ip forward21:09
openflyroot@barad-dur:~# sysctl net.ipv4.ip_forward21:10
openflynet.ipv4.ip_forward = 121:10
henrichrubinanyone else have any idea on this error?  i think vvuksan ran out of ideas.21:10
openflyyeah i was just confirming21:10
openflystill no joy21:10
openflystp didn't do it.21:10
*** dendrobates is now known as dendro-afk21:11
tr3buchethenrichrubin: did that, need to add the ppa also21:13
henrichrubintr3buchet:  so did you get paste that error?  i had the same problem, but now i am stuck on the error here:
*** brd_from_italy has joined #openstack21:14
vvuksanopenfly: what instances are you using ?21:16
vvuksanwhat images that is ?21:16
openstackhudsonProject swift build #199: SUCCESS in 31 sec:
openstackhudsonTarmac: I rewrote swift-init cause I wanted it to do somethings it didn't, and I21:16
openstackhudsontried to make some of the error conditions a little better too.  It has21:16
openstackhudsonsome new features, but overall it's behavior is mostly compatible with21:16
openstackhudsonthe old swift-init (input, return codes) - but if someone had some weird21:16
openstackhudsongreps on the output they're hozed.21:16
openstackhudsonswift-init --help is a good place to start21:16
openstackhudsonsome highlights might be:21:16
openstackhudson?$swift-init start main --wait  # wait for all "main" services to start,21:16
openstackhudsonprinting any errors to console21:16
openstackhudson?$swift-init rest wait  # same as above but for replicators, updaters & auditors21:16
openstackhudson?$swift-init stop object -c1  # kill the first object-server "node"21:16
openstackhudson?$swift-init object-server status  # make sure it's dead - yay status!21:16
openstackhudson?$swift-init proxy auth reload  # devauth?  srsly?21:16
openstackhudson?$swift-init *-replicator once -n  # run the object, container,21:16
openstackhudsonand account replicators in "once mode" and watch them log to console21:16
openstackhudsonuntil they're finished21:16
openstackhudson... probably some other stuff too.  Looking forward to feedback...21:16
openflyvvuksan i've tried it with 10.10 and 10.04 ubuntu cloud images21:17
*** dirakx has quit IRC21:17
tr3buchethenrichrubin: i'll let you know when i get there21:18
vvuksanopenfly: are you using dhcp or flat ?21:18
openflyttylinux-uec-amd64-12.1_2.6.35-22_1.tar.gz  <--- also tried that21:18
openflyno success21:18
openflywhatever is default21:19
openflyi think dhcp21:19
openflyi'm not setting the flag21:19
vvuksanon the network node21:19
vvuksanverify dnsmasq is running ?21:19
openflynetwork node is compute / api node21:19
vvuksansame here21:20
vvuksani have one that does all jobs21:20
openflyI see it21:20
vvuksanand one that has compute and network21:20
openflydnsmasq is running21:20
vvuksanpaste the ps output21:20
openflythat looks off21:21
openflythe instance is saying it's
openflybut dnsmasq is set for 100.3?21:21
vvuksanignore the 100.3 part21:21
vvuksanthis is the key file21:22
vvuksani would however21:22
vvuksankill the dnsmasq that was invoked by libvirt21:22
vvuksanone dishing 122 addresses21:22
vvuksanit likely doesn't matter21:22
vvuksanbut who knows21:22
vvuksancan you ping ?21:23
openflyno idea why21:25
vvuksancheck syslog21:25
vvuksanfor messages from dnsmasq21:25
vvuksansee if there is anything suspicious21:25
openflyi see some app armor issues on libvirt i think21:27
openflynot sure how relevant they are21:27
openflyand some vmnet issues21:27
tr3buchethenrichrubin: not hitting your error, but instead compute manager is getting stuck at "plugging VBD..."21:27
openflydid i mention i hate app armor?21:28
vvuksanopenfly: turn it off :-)21:28
openflyapp armor, network manager, and upstart21:28
openflyhate em all21:28
henrichrubintr3buchet:  send me the trace from nova-compute.log21:29
vvuksanopenfly: how about this then21:29
vvuksankill dnsmasq21:29
vvuksanthen run it with exact same options21:29
vvuksanbut append -d for debug21:30
vvuksanuse virsh to destroy the instance21:30
vvuksanthen start it up with virsh create21:30
*** berendt has joined #openstack21:31
openflythat's with app armor unloaded21:31
openflygive me a sec for the other stuff21:31
openstackhudsonProject nova build #524: SUCCESS in 1 min 33 sec:
openstackhudsonTarmac: -from migrate.versioning import exceptions as versioning_exceptions21:32
openstackhudson+    from migrate.versioning import exceptions as versioning_exceptions21:32
openstackhudson+except ImportError:21:32
openstackhudson+    try:21:32
openstackhudson+        # python-migration changed location of exceptions after 1.6.321:32
openstackhudson+        # See LP Bug #71746721:32
openstackhudson+        from migrate import exceptions as versioning_exceptions21:32
openstackhudson+    except ImportError:21:32
uvirtbotLaunchpad bug 717467 in nova "from migrate.versioning import exceptions fails with recent venv" [Medium,In progress]
openstackhudson+        sys.exit(_("python-migrate is not installed. Exiting."))21:32
berendti'm creating a new network with "nova-manage network create 1 255" but I only got 128 entries in the fixed_ips tables. is that normal? I thought I should got 255 entries in fixed_ips.. (
openflyis vnet0 supposed to get an address?21:33
berendtand the new network in networks is "" with netmask ""21:34
vishytry 25621:34
vishynot 25521:34
vishynetwork_size is should be a power of 221:35
*** joearnold has joined #openstack21:35
jaypipesblamar: still an issue with the Authors file?21:35
berendtvishy: ok i'll try it.. is it useful to add a check there?21:35
openflyyeah dnsmasq won't even load21:36
vishyyes we need lots of checks21:36
jaypipesblamar: I think you need to change this: 8+<> <>21:36
vishyin nova-manage21:36
jaypipesblamar: reverse that, so your gmail address maps to your rackspace address, which is what is in AUTHORS.21:36
berendtvishy: it's working fine with 256.. it's so simple.. :)21:36
berendtvishy: thanks21:36
*** DanEllis has quit IRC21:36
*** miclorb has joined #openstack21:36
vishyopenfly: looks like you are missing a flag21:37
vishyare you running from packages21:37
vishy--dhcpbridge_flagfile=/path/to/nova.conf (usually /etc/nova/nova.conf)21:37
openflyvishy yes21:38
tr3buchetnot a trace..  that's just where it hangs21:38
henrichrubinvishy:  any idea on
blamarjaypipes, arg, mea culpa, fixed and pushed21:40
nelsonAre there python bindings for swift other than python-cloudfiles?
jaypipesblamar: :) cool, I'll re-approve21:41
vishyhenrichrubin: weird it looks like db.fixed_ip_get_network is failing21:42
notmynamenelson: any *-cloudfiles repo at should work with swift21:42
henrichrubintr3buchet:  i don't see an error.   what does euca-run-instances say21:42
vishydo you have some old fixed ips in your db that are part of a network that has been deleted?21:42
henrichrubinvishy:  i know.  for some reason it returns a null db reference21:43
notmynamenelson: ah. but those are the only python bindings21:43
nelsonnotmyname: right; that's all I need.21:43
tr3buchethenrichrubin: i'm using the cloudservers api and it returns as usual, a list says status is in "build"21:43
henrichrubinvishy:  my network table is shown on the paste21:43
*** nelson has quit IRC21:43
vishyhenrichrubin: check your fixed_ips table21:44
*** nelson has joined #openstack21:44
tr3buchethenrichrubin: i'm going to upgrade to xenserver 5.6 and see if that helps, may have some random dirtiness hidden away somewhere21:44
openflyvishy no21:44
openflyeverything is brand new21:44
vishyhenrichrubin: I'm guessing there are some in there that have network_id=1 or network_id=221:45
henrichrubinvishy:  ok, there are some references to some old networks which i deleted.  i'll delete those.21:45
vishyopenfly: so do you have a flagfile?21:45
notmynamenelson: you could use the swift.common.client bindings distributed with swift itself. actually, they will be better than using python-cloudfiles21:46
nelsonfrig; those bindings aren't going to work for my use case either. Oh well, there's always s = socket.socket()21:46
henrichrubinvishy:  i deleted them and relauched an instance.    so when you remove a network from the networks table, you also have to remove it from the fixed_ips table?21:46
openflyvishy when i execute from nova-network service it pops up fine21:48
openflyi can't manually kick off dnsmasq to get debug output for whatever reason21:48
openfly2011-02-16 21:42:45,918 -[WARNING]:   21:42:45 [75\/100]: url error [timed out]21:48
openflythat is my primary issue at the moment21:49
openflyand i can't figure out what the cause is.21:49
creihtnelson: what use case?21:49
jaypipesblamar: gah, again Hudson isn't happy with the Authors file. :(21:51
blamarjaypipes, just looking at that, any details on what it doesn't like?21:51
jaypipesblamar: :( I don't know.21:54
*** dendro-afk is now known as dendrobates21:55
vishyhenrichrubin: correct21:55
*** berendt has quit IRC21:55
vishyopenfly: it is nova-dhcpbridge failing21:56
vishyopenfly: hence the mention of needing a flagfile21:57
henrichrubinvishy:  if you try a bunch of instances that never get to the running state, then terminate the instance, they are never cleaned from the fixed_ips table.  so you would run out of ips eventually, even though you have no instances running.  shouldn't there be a mechanism to clean this table up if the instance isn't running or is terminated?21:57
*** vvuksan has quit IRC21:57
nelsoncreiht: reading from a URL, writing to the storage AND returning a copy to the swift client.21:57
creihtthat is a pretty specific use case :)21:58
vishyhenrichrubin: they are cleaned up after 10 minutes or so21:58
vishyby default21:58
jarrodis there anyway to use VlanManager, but not use NAT and allow the instances to possess the public IP?21:58
henrichrubinor send a warning to the user that they need to clean up21:58
creihtnelson: but I highly recommend using swift/common/ for the writing to the storage part21:58
vishyhenrichrubin: at least they are supposed to be21:58
nelsonreading it now.21:58
henrichrubinvishy: i am not sure about the default time.  the old fixed_ips were still there from yesterday.21:59
creihtnelson: it is a pretty direct mapping to the various swift functions21:59
vishyhenrichrubin: the ones that no longer had a network?21:59
creihtand we use it for all of our internal tools21:59
henrichrubinvishy:  yes.  plus there were ones that did have a network that i killed this morning.22:00
openflyvishy i have one22:00
openflyit's part of the default package22:00
vishyhenrichrubin: hmm let me check something, there may be a bug with flatdhcp22:00
nelsoncreiht: yeah, but it expects a function with a 'read' so it can consume the entire stream. doesn't work for me, since the *other* data sink also wants an iterator. :)22:01
vishyhenrichrubin: you are correct, the disassociate only happens in vlan manage22:01
*** hub_cap has quit IRC22:01
jaypipeseday: cheers, those were the details I was looking for.  sorry for being a pest about details on that thread. I know you feel you've repeated yourself, oh, about a million times, on the distributed scheduler blueprint. :)22:01
creihtnelson: ahh... so you want it to stream through the whole path22:02
vishyhenrichrubin: the periodic_tasks code needs to be copied over, and made to work without a host, if you report it as a bug i will make a fix.22:02
henrichrubinvishy:  ok, i'll file a bug report.  also, why wouldn't my dnsmasq be starting now after i cleaned up the fixed_ips table.22:02
vishyhenrichrubin: not sure, are you getting an error?22:02
henrichrubinvishy:  the error was here:  but now i see no error.22:03
vishyyou might try deleting the networks/br100.conf and pid and restarting nova-network22:03
nelsoncreiht: I need put_object to be put_object_start, which returns an opaque object with two methods: write() and close().22:03
henrichrubinvishy:  wrong pastie.22:03
henrichrubinno more error.  just doesn't start22:04
vishyhenrichrubin: it won't start it unless there is an instance running22:04
nelsoncreiht: maybe I'll cut-n-paste out of and structure it that way. Maybe other people will find that useful and you can pull it into swift?22:04
*** alekibango has quit IRC22:04
*** grapex has joined #openstack22:04
vishyhenrichrubin: otherwise it starts when you run an instance22:04
henrichrubinvishy:  ok, then i get no error from nova-network and it is in the "networking" state22:04
creihtnelson: sure22:05
vishypossible the message isn't getting to rabbit22:05
creihtnelson: as long as it is pep8 compliant of course ;)22:05
vishydo your fixed_ips now all have the proper network_id?22:05
nelsonwouldn't DREAM of not.22:06
henrichrubinvishy:  i got to run, but here is the bug report...
uvirtbotLaunchpad bug 720393 in nova "fixed_ips table is not cleaned up by default" [Undecided,New]22:07
edayjaypipes: no worries, happy to share. without the details it would just be hand-wavey, which is not helpful :)22:08
* nelson goes for a walk.22:08
openflystill nothing.22:08
openflyfor whatever reason instances just cannot hit the meta-data server22:09
openflyand i have no clue why22:09
*** MarkAtwood has joined #openstack22:12
*** dendrobates is now known as dendro-afk22:13
uvirtbotNew bug: #720393 in nova "fixed_ips table is not cleaned up by default using FlatDHCPManager" [Undecided,New]
openflyOkay does someone... anyone at all have functional actually tested and working instructions for installing bexar on a stand alone ubuntu 10.10 system?22:18
*** blueadept has quit IRC22:22
*** CloudChris has joined #openstack22:25
vishyopenfly: which instructions are you following?22:25
openflyvishy i've tried everything I could find22:26
openflywiki and website.22:26
vishyopenfly: I'm working on bexar++ cuz there were some bugs22:26
*** littleidea has joined #openstack22:26
vishyopenfly: but I think bexar packages work just fine22:26
openflynot working for me22:26
openflyi can get instances to kick off22:27
openflybut they get stuck trying to hit the metadata server22:27
openflyand then... nothing22:27
openflyno errors22:27
openflyjust... nothing22:27
*** dprince has quit IRC22:27
vishyopenfly: do you have two interfaces on the machine?22:27
openflyphysical anyways22:27
*** glenc has quit IRC22:28
vishyopenfly: can you pastie me the output of iptables -t nat -L -n -v22:28
*** glenc has joined #openstack22:28
vishyopenfly: can you ping the instances?22:28
*** dprince has joined #openstack22:29
vishyopenfly: do you get any messages in /etc/nova/nova-network.log about ips getting leased?22:29
openflyi think i might have it22:30
vishyyou may have an extra copy of dnsmasq running as well22:30
openflydoes the packaged version of bexar assume you use ?22:30
vishythat is a common issue22:30
vishyopenfly: yes22:30
openflybecause that appears to be the case from the iptables.22:30
vishyopenfly: do you have a conflict?22:30
openflyi don't use at all in my config22:31
vishyopenfly: looks like your host machine is using 10.100.7022:31
vishyyeah that will cause issues22:31
openflyhow do i change nova?22:32
*** m_3 has quit IRC22:33
*** allsystemsarego has quit IRC22:35
vishyopenfly: --fixed_range= in /etc/nova/nova.conf22:35
vishyor something like that22:35
vishyyou'll probably need to flush iptables22:35
vishyand restart nova-network22:35
vishyyou should also kill your running instances first as well22:35
openflywill that be fine if I use 192.168.100 address space?22:35
vishysure you can use that if you want22:35
*** benlake_ has joined #openstack22:38
openflythat chagned stuff22:40
openflybut not 100%22:41
openflystill can't ping still seeing natting rules for
*** benlake has quit IRC22:41
*** hazmat has quit IRC22:41
*** hazmat has joined #openstack22:41
jaypipessoren: if you have a moment to re-check this one, would be appreciaed:
*** m_3 has joined #openstack22:41
*** miclorb has quit IRC22:42
vishydid you flush iptables first22:42
vishyiptables -f22:42
vishyi have to go, sorry, you should still see natting rules showing the ip of your host machine22:43
jaypipesdprince: know if Brian Waldon hangs out on IRC? Wanted to see what his plans were on
sorenjaypipes: Which editor do you use?22:46
jaypipessoren: vim. why?22:46
sorenjaypipes: There's a lovely pyflakes plugin for it that highlights unused variables and a bunch of other things.22:47
sorenjaypipes: That's how I spotted those things in my review.22:47
jaypipessoren: ah, cool. link?22:47
sorenjaypipes: It is *awesome*.22:49
* soren heads bedwards22:49
jaypipessoren: cheers mate.22:49
sorenjaypipes: Let me know if you can't get it working. I remember it took a bit of tweaking back when I set it up.22:49
sorenThat might be fixed now. It was about a year ago.22:49
* soren wanders off22:49
*** imsplitbit has quit IRC22:50
openstackhudsonProject nova build #525: SUCCESS in 1 min 33 sec:
openstackhudsonTarmac: Update to our HACKING doc to add examples of our docstring style.22:57
*** grapex has left #openstack23:01
jaypipessandywalsh_ or dabo: one more set of eyes on tr3buchet's patch would be appreciated:
*** rnirmal has quit IRC23:05
*** gondoi has quit IRC23:07
*** pvo has quit IRC23:07
*** adiantum has joined #openstack23:11
*** brd_from_italy has quit IRC23:12
tr3buchetnow that's GM Goodwrench Service PLUS23:17
tr3buchetthink i can get cerberus to take a look at it to23:18
*** desai has quit IRC23:20
dsockwellthe wiki mentions installing openstack on centos 6; is there documentation for that case?23:20
*** adiantum has quit IRC23:20
dsockwellerm -- centos 6 being unreleased i'm going to guess 'maybe not'23:22
*** adiantum has joined #openstack23:24
*** GasbaKid has joined #openstack23:30
*** dprince has quit IRC23:31
*** eikke has joined #openstack23:32
nelsoncreiht: split up put_object into a class with three methods: open, write, and close23:37
nelsoncreiht: seems like the obvious set of names.23:37
kpeppledsockwell: RHEL 6 packages are available so you should be ready when CentOS 6 hits23:37
*** adiantum has quit IRC23:38
*** adiantum has joined #openstack23:40
*** msassak has quit IRC23:52

Generated by 2.14.0 by Marius Gedminas - find it at!