Tuesday, 2011-01-04

nelson__Hrm. I have setuptools installed. When I look at trunk/README, it says that I can build the documentation using "python setup.py build_sphinx"00:01
nelson__does that work for anybody else?00:01
*** MikeZ1 has quit IRC00:04
creihtnelson__: install python sphinx00:07
nelson__I'll bet that will fix it. Sounds like something to add to my documentation fixes. :)00:08
nelson__  File "/home/nelson/openstack/trunk/doc/source/./_theme/layout.html", line 2, in top-level template code00:09
nelson__    {% set css_files = css_files + ['_static/tweaks.css'] %}00:09
nelson__jinja2.exceptions.UndefinedError: 'css_files' is undefined00:09
nelson__which sounds like it expects that css_files was defined by sphinx_doc/layout.html00:11
creihtnelson__: is this for swift, or nova?00:12
nelson__swift00:12
creihthrm00:13
creihtnelson__: which version of sphinx?00:14
creihtnelson__: current trunk builds for me00:15
*** jimbaker has quit IRC00:17
*** MikeZ has joined #openstack00:18
*** Cybo has quit IRC00:30
*** ccustine has quit IRC00:32
nelson__python-sphinx (0.6.4-1)00:32
*** reldan has quit IRC00:33
nelson__On Ubuntu 10.04.100:33
*** reldan has joined #openstack00:35
nelson__argh. And auth-server is silently crashing.00:40
Ryan_Lanenelson__: you running trunk, or a stable release?00:41
nelson__trunk00:41
* Ryan_Lane nods00:41
*** dendrobates is now known as dendro-afk00:41
nelson__Doh! The config file is b0rked. swift-auth-server is printing an appropriate error to fd1, but swift-init isn't printing that to stderr00:43
*** winston-d has joined #openstack00:44
*** Cybo has joined #openstack00:45
nelson__fixing the config file solved the problem and swift-auth-server is now running, but ... but ... swift-init ought to print an error message,00:45
nelson__particularly if the user install instructions are going to tell people to run "swift-init auth start"00:45
nelson__and there's nothing in any /var/log/ file mentioning swift.00:47
*** dirakx has joined #openstack00:48
nelson__and without meaning to whine *too* loudly, but there's no man page for /usr/bin/swift* in the 'swift' package.00:49
*** Cybo has quit IRC00:49
*** gaveen has quit IRC00:51
nelson__haha, oops, the first thing swift-init does is close fd1. :)00:51
* nelson__ rewrites swift-init so that it relies on Dan Bernstein's daemontools.00:52
*** jdurgin has quit IRC00:52
nelson__sudo apt-get install daemontools daemontools-run00:56
*** Jordandev has joined #openstack00:56
*** reldan has quit IRC00:56
alekibangowhy not runit?00:56
termieupstart00:57
nelson__because daemontools handles logging of program output.00:58
nelson__it also lets you send signals to running programs,00:58
*** dirakx has quit IRC00:59
nelson__without disgusting hackery like /var/run/*.pid00:59
termienelson__: making things rely on a specific implementation is generally a bad thing00:59
termiethere are quite a lot of people who vehemently hate daemontools00:59
nelson__daemontools runs everywhere00:59
termiethat isn't much of an argument01:00
nelson__I'm sure they have bad reasons for hating daemontools ... starting with the personality of the author.01:00
termiethere are plenty of things to dislike about it01:00
nelson__daemontools running everywhere is a GREAT argument against "making things rely on a specific implementation is generally a bad thing"01:00
nelson__termie: like what?01:01
nelson__daemontools solves a lot of stupid little problems all at once.01:01
termienelson__: it has a very separate environment from the rest of the system and does not integrate with standard setups nor is supported by the os writers01:01
alekibangonelson__: and introduces some of its own :)01:02
nelson__so put it in /etc/service like the debian/ubuntu packages does.  Next?01:02
alekibangonelson__: did you read license?01:02
nelson__public domain. next?01:02
*** anotherjesse has joined #openstack01:02
alekibangoreally?,  i need to check that  :)01:02
termienelson__: well first off, talk about the decision on the list01:03
termiealekibango: yeah it is public domain01:03
nelson__termie: that's a reason not to change anything, not a reason not to use daemontools. Next?01:03
nelson__So far, #define plenty 201:03
*** jc_smith has quit IRC01:03
XenithI think the question is if nelson__ is making daemontools required or not.01:04
XenithNothing wrong with making it supported but optional01:04
termienelson__: daemontools is a perfectly reasonable way to run the code, it is not a reasonable dependency01:04
*** Cybo has joined #openstack01:04
nelson__and neither of those reasons were true so far.01:04
nelson__termie: I'm looking for rationality here, not hatred of djb.01:05
nelson__(and I will cheerfully acknowledge that he has been an ass in the past)01:05
termienelson__: i don't know djb, or care about him, i have used daemontools before and I personally don't like it and I am not alone in that01:05
termienelson__: i offered three reasons that you skipped over01:05
nelson__why don't you like it?01:05
termienelson__: scroll back01:05
nelson__I've answered all three of your reasons.01:05
Ryan_Lanenelson__: we are unlikely to use daemontools01:06
termienelson__: putting it in /etc/service is not a solution to that problem01:06
Ryan_Lanewe are going to use init.d, and start-stop-daemon01:06
nelson__Ryan_Lane: I'm sorry to hear that. Do you have a rational reason for objecting to a superior solution?01:06
nelson__Or again, are you a djb-hater?01:06
nelson__(which is fine ... it's just irrational)01:06
Ryan_Lanebecause we do things the ubuntu way01:07
Ryan_Laneit's more consistent if we do so01:07
alekibangoi cant say i am hater of djb, but his licensing was crazy last time i checked01:07
nelson__it sucks consistently.01:07
Ryan_Lanealekibango: most of his stuff was released public domain recently01:07
Xenithalekibango: A while back he just threw everything of his into public domain01:07
nelson__no, seriously, init.d is more than twenty years old, and it started sucking when it was first created.01:07
Ryan_Lanelike in the last few years01:07
alekibangothats great news, really01:07
Ryan_Lanenelson__: meh. works for me01:08
nelson__sucks for me.01:08
alekibangoi must say i liked few features of daemontools01:08
nelson__Look, I'm not suggesting this for no reason.01:08
nelson__I'm suggesting it because swift-init is broken.01:08
Ryan_LaneI like the init system of solaris and os x personally01:08
nelson__1) it has no documentation.01:08
nelson__2) it discards error messages.01:08
nelson__3) it has no internal help.01:08
nelson__4) it doesn't report when a program it started exits.01:09
nelson__there's a GAZILLION reasons to prefer daemontools to swift-init in particular, and init.d in general.01:09
Ryan_Laneall of those should simply be fixed.01:09
Ryan_Laneno distro uses daemontools by default01:09
nelson__init.d can't be fixed. It's broken by design.01:09
nelson__well that's FAIL, then, isn't it?01:10
Ryan_Laneit would be silly to add that requirement by default01:10
nelson__it's silly to FAIL when success is within your grasp.01:10
Ryan_Lanenelson__: that's something to take up with fedora, ubuntu, debian, rhel, suse, and all other major distros :)01:10
nelson__"hey, let's do this good thing."  "No, we prefer to fail."01:10
nelson__Ryan_Lane: the reason they don't use it by default is because of conservative sysadmins who want everything done the same way it was done twenty years ago.01:11
Ryan_LaneI'd love a system like launchd in linux01:11
nelson__yeah, I hate it when ifconfig eth0:0 goes away, but y'know "ip address" makes a lot more sense.01:12
Ryan_Laneeither way, making things work properly in init.d is likely best01:12
nelson__except that init.d doesn't actually solve the problem of starting long-running problems.01:12
nelson__Why do you insist on using something that doesn't actually work?01:13
nelson__... just because everybody else is using it?01:13
Ryan_Lanenelson__: they do work for me though01:13
termienelson__: we don't insist on using anythign at this point, and i'm perfectly willing to accept that swift-init is doing somethign wrong01:13
termienelson__: but the solution is to make our code compatible with whichever methods people choose to run it with01:13
Ryan_Lanetermie: +101:14
nelson__Ryan_Lane: okay .... where does the stdout of, say, rsyncd go?01:14
termienelson__: within the realm of reason (i'd say supporting daemontools, init.d and upstart is a good start)01:14
Ryan_Lanenelson__: if the daemon is written properly, whereever that daemon says it goes. hopefully to the system logger01:14
nelson__or, more to the point, swift-auth-server ?01:14
termienelson__: but none of those are exclusive or really very complex to support01:14
termiei'd love to stop the argument over which system is better because it is largely a waste of our time01:15
nelson__Ryan_Lane: when you use daemontools, the output ALWAYS goes into the .../log/main directory.01:15
*** MikeZ1 has joined #openstack01:15
Ryan_Lanenelson__: I don't believe that is correct though01:15
nelson__termie: because ... you refuse to accept that something else is better.01:15
Ryan_Lanetermie: agreed01:15
termieas simply making our services act normally will automatically support them all01:15
Ryan_Laneexactly01:15
termienelson__: no, because i am not going to force somebody else to agree with me01:15
alekibangosupport all methods -- thats good for me01:15
alekibangoprovide patches01:15
nelson__termie: and since you're agreed that you want to use FAIL, then we'll continue to have to deal with FAIL.01:15
termienelson__: at this point you are just trolling01:16
nelson__sigh.01:16
nelson__nahhhh, I'm frustrated that you're not being rational.01:16
nelson__"We're using init.d because everybody else is using it."01:16
Ryan_Lanenelson__: meh. if they write logs to the system logger, or add an option for where logs go, daemontools is unneccessary for that01:16
nelson__That's monkey-talk.01:16
nelson__more monkey-talk.01:16
termieRyan_Lane: if you could stop talking to him it would make my /ignore feel a lot more successful ;)01:16
nelson__I have work to do.01:16
alekibangoeasy ppl...01:16
Ryan_Lanetermie: :)01:17
*** MikeZ has quit IRC01:17
Jordandevfast question: any plan for Bexar to have a Dashboard? (like nebula dashboard?) or another GUI?01:17
Ryan_LaneJordandev: there are a few being worked on right now01:18
anotherjesseJordandev: we're just finishing open sourcing the nebula django dashboard as django apps01:18
Ryan_LaneJordandev: there's one that was supposed to be released by the end of last year (did that happen?) written in django01:18
alekibangothere is no need to be harsh on others for having relations with some technology or the other... those are only tools -- some work for some people, some work for another...01:18
Jordandevcoool01:18
alekibangoRyan_Lane: it didnt happen yet01:18
Ryan_Lanealekibango: k01:19
Jordandevand when it will?01:19
alekibangoreally soon now (tm)01:19
Ryan_LaneJordandev: I'm also writing one, but it isn't likely to be as useful as the django for non wikimedia foundation folks at first01:19
anotherjesseJordandev / alekibango the guy leading it is back from vacation01:19
*** rlucio has quit IRC01:19
alekibangoanotherjesse: ping me if you will want some testing of it please01:19
alekibangoi am back from vacation too, pretty ruined my body there, need some rest...01:20
alekibangoheh01:20
Jordandevstill in vacation but ;)01:20
anotherjessealekibango: we still need to re-work the auth for it01:20
nelson__alekibango: I'm just frustrated that I'm being asked to provide a quality solution, and my technical choices are vetoed left and right.01:20
anotherjessebut then again we need to move it to the openstack api :)01:20
alekibangodarn... some version to play with would be sweet...   iam really curious01:21
alekibangonelson__:   i know what you mean01:21
JordandevRyan_Lane: we can test it and contribute01:21
Ryan_LaneI'd love that01:22
Jordandevany public link?01:22
Ryan_Laneyep01:22
Ryan_LaneIt's for a specific project right now01:22
Ryan_Lanehttp://ryandlane.com/blog/2011/01/02/building-a-test-and-development-infrastructure-using-openstack/01:22
alekibangonelson__: i often talk to people about things they cant accept... or imagine...   and its tough job01:22
Ryan_Lanewhich is why it won't be as useful for non-wmf people, as it does some pretty specific things01:23
Ryan_Lanelike add puppet configuration and dns information when instances are created01:23
alekibangonelson__: provide patches for daemontools support  -- they might get included and i might use them on my servers01:23
Ryan_Laneand creates the users with posix information01:23
alekibangoyou know, i was thinking about doing this for runit01:24
Ryan_LaneJordandev: test interface is here: http://nova-controller.tesla.usability.wikimedia.org/sandbox.1/Main_Page01:24
alekibango(which is similar to daemontools in a way)01:24
Ryan_Lanethat's where I'm working on features right now anyway01:24
Ryan_LaneI guess I should add information about how to actually use it :D01:25
JordandevRyan_Lane: still an empty page!01:25
alekibangoRyan_Lane: i see empty page too01:25
nelson__alekibango: I'm not generally seeking to improve swift. I'm seeking to improve it for Wikimedia use. But if Ryan_Lane says that he's not going to use it, there's no point in writing it.01:26
alekibangoah, ic01:26
Ryan_Lanetry now01:27
Ryan_Lanenelson__: using daemontools for this when we are using init.d for everything else adds complexity, which makes it harder for us to manage things with a small number of admins01:28
alekibangoRyan_Lane: shellaccountnamehelp ?01:28
Ryan_Laneyeah01:28
Ryan_LaneI haven't started doing localization yet01:28
alekibangothats username?01:28
nelson__except that daemontools works better.01:28
Ryan_Laneyeah.01:28
alekibangonelson__: i loven initng01:29
alekibangoloved*01:29
Ryan_Lanewiki user name and shell account name are separate. per request by a ton of people :)01:29
alekibangonever liked upstart01:29
Ryan_LaneI haven't started doing localization because we have an army of volunteers, and until I finalize the messages, I'm wasting their time01:29
anotherjesseRyan_Lane: any other articles about mediawiki + openstack?01:30
Ryan_Laneanotherjesse: just that one right now01:30
*** johnpur has quit IRC01:30
Ryan_LaneI do have a spec sheet here: http://www.mediawiki.org/wiki/Extension:OpenStackManager01:30
Ryan_Laneplus documentation that needs to be updated01:31
alekibangokey is what type of key? ssh public key?01:32
*** reldan has joined #openstack01:32
Ryan_Laneyep01:32
Ryan_Lanethis is something I need to fix01:33
Ryan_LaneI'm inserting keys directly into ldap right now01:33
Ryan_Lanenova should be doing it instead01:33
Ryan_LaneI need to add that part of the api, and also figure out a way to reliably import the keys01:34
Ryan_Laneand handle key names01:34
Ryan_Lanewithout adding schema :)01:34
Ryan_Lanealekibango: added you to a project so that you can check out instance creation/modifcation01:35
alekibangothanks :) i added my own project atm01:35
Ryan_Laneami-gr0xqtsz is a bad image01:35
Ryan_Laneheh01:35
Ryan_LaneI haven't limited access to project creation yet :)01:35
Ryan_Laneand though you can create instances, you can't ssh to them, as they are in a backend-network01:36
Ryan_Lanefor security purposes01:36
alekibangoyou are using nagios?01:36
alekibangotry zabbix :)01:36
nelson__alekibango: swift-auth-server (at least) runs just fine under daemontools.01:36
Ryan_Lanealekibango: I01:37
Ryan_Laneerr01:37
Ryan_Lanealekibango: I'll take a look at it01:37
alekibangodo,  its cool01:37
alekibangozabbix.com01:37
Ryan_Lanewe have very few people, and have nagios integrated with puppet, so it's unlikely until someone gets some free time :)01:38
alekibangoit takes at least a day to make it right if its complex, but it will be worth it, believe me01:38
Ryan_Lanehave I mentioned we like volunteers? :)01:38
nelson__alekibango: and unlike swift-init, the error message is right there in the daemontool log file.01:38
alekibangonelson__: i just love the simplicity01:38
*** anotherjesse has quit IRC01:38
nelson__alekibango: I just love that it works.01:39
alekibangoyes, and its simple01:39
alekibangosimple => no magic (=no problems)01:39
nelson__alekibango: I know where to find the log output; I know how to start and stop processes; I know how to send signals to the process.; I know how to tell the process to go down and stay down.01:39
alekibangoand especially i love that it restarts itself when it fails01:40
alekibangounlike some init scripts01:40
alekibango(or should i say most)01:40
nelson__alekibango: init scripts aren't running and can't see that a process has exited.01:41
alekibangoyes i know01:41
alekibangosome of them run some watching daemon which watches the actual service01:41
alekibangowhich is crazy :)01:41
nelson__but other people haven't felt that pain, and don't appreciate self-restarting services.01:41
alekibangobut the only way to make it work, i hear you nelson01:42
alekibangofor openstack cluster, it might make sense to use daemontools - and even depend on them01:43
alekibangoas the Nova or SWIFT  will be the only service the machine will serve01:43
alekibango(mostly)01:43
Ryan_Laneif services are dying, it for a reason01:43
Ryan_Laneerr it's01:43
alekibangoRyan_Lane: yes, but with initscript, you sometimes cant find out why it dies01:44
alekibangoor why it is not running01:44
Ryan_Lanehow's daemontools help with that?01:44
alekibangoa lot :)01:44
alekibangothats where the simplicity comes in01:44
Ryan_Lanedon't get me wrong. init.d sucks.01:45
alekibangoit captures output of the process into file  ->  its not lost somewhere in system logs (guess which one)  or in buggy init script01:45
Ryan_Laneforcing people to use something non-standard for their distro just isn't a good idea01:45
alekibango(guess where)01:45
alekibangoand why01:46
Ryan_Laneheh01:46
Ryan_Lanebut...01:46
alekibangoam not for forcing01:46
Ryan_Laneif you are writing to the system logger, the admin can choose where to write that file01:46
alekibangobut having this possibility might make sense01:46
alekibangoRyan_Lane: yes, but mostly they do not01:46
alekibangoand i even lost my nova logs few times01:46
alekibangothanks to some bugs01:46
alekibangothey always bite01:47
Ryan_Lanethat's a problem fixed by documentation01:47
alekibangoRyan_Lane: not really01:47
Ryan_Laneah. you mean bugs causing the logs to go to the wrong spot01:48
alekibangoright!01:48
alekibangothats the simplicity i am talking about01:48
alekibangocomplex ==> will be broken01:48
alekibangoand init scripts are complex :)01:49
Ryan_Laneinconsistency with a distro is also complexity though ;)01:49
alekibangoon my debian, i need to restart nova after first boot01:49
alekibangothanks to not using daemontools :)01:49
Ryan_Lanewhy?01:49
Ryan_Lanesounds like a bug to fix01:49
alekibangoall nova services are trying to create tables  at the same moment -- in the empty database01:50
alekibangosome of them fail01:50
alekibangoand abort01:50
alekibangoand whatever01:50
Ryan_Lanethat's not a problem in a production system though01:51
alekibangoit is01:51
alekibangoi need it to be deployed automatically01:51
Ryan_Lanewhy would the database be empty?01:51
alekibangowithout glitch01:51
alekibangoi install nova in 3 minutes on 3 servers01:51
* Ryan_Lane nods01:51
alekibangoincluding partitioning and raid and stuff01:51
alekibangoand i want it to RUN sucessfully immediatelly01:52
alekibangoseconds are important for me01:52
alekibangoyou know, for fast deployment, automated functional testing, etc etc01:52
Ryan_Laneyep01:52
alekibangoand after first start, few services just fail somehow :)01:53
alekibangousing daemontools with restart might help there01:53
Ryan_Lanefixing the bug would be better :)01:53
alekibangowell, might be, but then, another 5 bugs come in01:53
alekibango:)01:54
Ryan_Laneheh01:54
alekibangoRyan_Lane: its not a bug, its feature... maybe01:54
alekibangoits feature of SQLAlchemy01:54
termiealekibango: certainly would, as would other approaches, the bug should also be fixed to handle the error case in popular setups as well, as well as any bugs related to it running under less popular but still supported setups01:54
Ryan_Lanelol01:54
alekibangoits creating tables and stuff -which is missing in db01:54
alekibangowrong part is, they all are trying to do it at the same moment01:55
alekibango:)01:55
alekibangoall those nova services01:55
alekibangomaybe we should prepare the db as part of   postinstall01:55
alekibangothat would make it work better01:55
Ryan_Lanethat assumes the database is configured at install time01:56
alekibango... even installation manual imho mentions restarts01:56
alekibangomy database is configured01:56
alekibangobut empty01:56
termiealekibango: a tools/prep_db.py would certainly allow such a thing to be written01:56
XenithI've switched to icinga, wasn't too impressed with zabbix when my old company used it.01:56
* Ryan_Lane doesn't like when services auto-start themselves01:56
Ryan_Laneon install, that is01:56
alekibangoXenith: why?01:57
Xenithalekibango: Personally, I didn't work with it, but my coworkers who were trying to configure it were not happy at all. Always complaining :)01:57
alekibangoheh, as i said, its not very easy  when you are not doing large deployment01:58
*** joearnold has quit IRC01:58
XenithAlso, it wouldn't shut up. I got tons of SMS messages from it, but that was probably due to the previously mentioned coworkers not being able to configuring it properly :)01:58
alekibangoi agree its not easy to configure :)    but still -- just look on screenshots :)01:58
XenithWell, the new icinga-web stuff looks pretty snazzy as well :)01:59
Ryan_Laneyeah, i'm looking at the demo for that now01:59
Ryan_Laneit's quite pretty02:00
*** mdomsch has joined #openstack02:00
alekibangoit certainly is... i remember using nagios some 15 years ago and it was crazy02:00
Ryan_Lanenagios has a terrible UI02:00
alekibangothansk, never heard of icinga, will check it later02:01
XenithThat's because the Nagios UI is(was) hard-coded in the C/CGI stuff.02:01
alekibangoyes, iremember that02:01
*** littleidea_ has quit IRC02:01
Xenithalekibango: icinga is a fork of nagios, so it can use all the nagios plugins and stuff02:01
alekibangomy hairs are standing when i remember looking into source02:01
alekibangosomething like PHP, but in C (ouch)02:02
alekibangoor perl?02:02
Ryan_Laneheh. one of our devs did a security audit of nagios02:03
Ryan_Lanehe was less than impressed02:03
alekibangothats the spirit, yes02:03
zaitcevDid he find a useable exploit though?02:03
XenithThat's why you don't run nagios over public internet.02:04
Ryan_Lanehe sent a list of bugs to them02:04
XenithBackplane or VPNs :)02:04
Ryan_LaneXenith: wuss :) http://nagios.wikimedia.org/02:04
*** reldan has quit IRC02:04
XenithWell, I have the web front end available on a public IP, of course. :)02:05
Ryan_Laneah. right. the daemon itself isn't publicly accessible02:05
alekibangotomorrow in news: wikimedia hacked by unanonymouse02:05
Ryan_Lane:D02:05
*** littleidea has joined #openstack02:06
alekibangooh that nagios is really crazy ui :)02:10
Ryan_LaneI pretty much only use Services: unhandled and Hosts: unhandled02:12
*** daleolds has quit IRC02:23
*** schisamo has quit IRC02:28
*** jdurgin has joined #openstack02:30
*** jimbaker has joined #openstack02:33
*** anotherjesse has joined #openstack02:40
Ryan_Lanewhen upgrading nova, is it automatically supposed to upgrade the database schema too?02:51
Ryan_Laneor is there some other way to go about that?02:51
anotherjesseRyan_Lane: the idea is that once we migrate the schema we should move from the auto-generate database to a migration based02:52
Ryan_Laneso if all of my nova services are throwing schema errors right now, how do I fix it? :)02:53
anotherjesseyikes - hmm, perhaps then something got merged that changed schema :(02:54
anotherjessedid you just upgrade?02:54
Ryan_Lane"Unknown column 'instances.availability_zone' in 'field list'"02:54
Ryan_Laneyeah. using trunk debs02:54
Ryan_Lanetrunk ppa that is02:54
Ryan_Laneif I need to wipe out the database I won't be amazingly upset, but I'd like to know this won't happen when I upgrade in production. heh02:55
Ryan_Lanethough I won't be using trunk ppa in production02:55
anotherjesseya - let me do the same thing to my test cluster - haven't upgraded it in a couple days02:56
*** GeorgeSebastian has joined #openstack02:56
Ryan_LaneI wanted raw image support ^_^02:56
anotherjesseme to02:57
XenithI'm looking forward to that.02:58
XenithHaven't had a chance to play with it yet.02:58
*** rchavik has joined #openstack02:59
creihtnelson__: the error stuff on swift-init is a known bug03:06
creihtnelson__: re: daemontools, we leave those decisions up to the ops guys, and they don't like it.  That said, I'm not against having a swift-init, or like script that cooperates better with daemontools.  It just isn't what we use right now03:07
creihtnelson__: and for future debugging, you can see the output by running the auth server manually like:03:14
creihtswift-auth-server /etc/swift/auth.conf03:14
creihtwill display the console error03:14
creihtI know it is a pain, but it is on our list to fix03:14
creihtswift-init can definately be made better, and of course patches are welcome :)03:15
creihtmy personal experience with daemontools in the past is that it was a pain to setup/configure, but once you got it set up it worked reasonably well03:16
creihtI think most people dislike it, just because it is different03:16
*** blpiatt has joined #openstack03:17
creihtAll of that said, ops people are all going to want to control their services differently, so I'm much more inclined to keep it flexible, rather than say this is how you have to do it (eg. daemontools)03:17
alekibango+103:17
creihtswift-init is just there as a piece of legacy code since it was what grew somewhat organically to suite our needs03:18
creihtand the requests of our ops guys03:18
creihtnelson__, Ryan_Lane: Please let me know if you guys run into any issues with swift03:20
anotherjessealekibango: my system was up-to-date with that change already :(03:20
*** dendro-afk is now known as dendrobates03:20
Ryan_Lanecreiht: I'm not too involved with swift as of right now. nelson__ and a couple other people are more involved03:21
Ryan_LaneI'll let you know though, as I'm sure I'll be knee deep in it at some point in time :D03:21
creihthehe03:22
Ryan_Lanethe plus side of having few people? you get to work with a lot of stuff? the negative side? you have to work with a lot of stuff03:22
creihtWe know that there are still probably some rough areas, and areas where the docs need improving03:22
Ryan_Laneminus a question mark somewhere in there.03:23
creihtRyan_Lane: indeed03:23
*** littleidea has quit IRC03:27
*** hadrian has quit IRC03:31
*** hadrian has joined #openstack03:32
*** littleidea has joined #openstack03:32
anotherjessecreiht: was their any progress on swift-lite for nova?03:37
creihtanotherjesse: I think gholt played with the idea for a bit, but then it got pushed to the side as more pressing stuff came around03:38
*** Ryan_Lane is now known as Ryan_Lane|away03:38
*** guynaor has joined #openstack03:39
anotherjessecreiht: termie spent some time moving all the components of nova to eventlet except the naive nova-objectstore03:39
*** guynaor has left #openstack03:39
anotherjessecreiht: it would be nice to be able to kill nova-objectstore and have a swift-lite03:39
*** gondoi_ has joined #openstack03:39
creihtanotherjesse: k, I'll bring it up again with gholt, but I know that we are all going to be quite busy for a bit03:39
anotherjessecreiht: nah, it's a new year, there is time ;)03:40
creihtlol03:40
anotherjesseswift-lite would only be for dev mode btw03:42
creihtanotherjesse: of course :)03:43
*** gondoi_ has quit IRC03:44
creihtWe've also talked how it would be good for anyone who is developing client side apps for swift to test against03:44
anotherjessehmm, but won't that cost rax money ...03:45
anotherjessethey could make $0.43 in the testing process03:45
creihtanotherjesse: the approach we talked about was to have a package that could be installed that sets everything up for a 1-replica "cluster" on one machine03:46
creihtreplication, and aux services wouldn't have to run03:46
creihtIn theory it is mostly packaging work03:46
aimonhi all03:50
aimoncreiht: A KickStart iso for that would be pretty great03:51
aimonwe are doing similar for our product .. it allows non-HA one resource node configuration03:51
aimongives people a chance to test it out with out ivolving a bunch of support from us03:52
aimon.. I couldnt help but to jump in.. This thread was exaclty like one we just had internally03:52
creihtaimon: there has actually been talk of such a thing.  A live ISO for nova already exists03:53
creihtNot sure if there has been any work on a swift one03:53
aimonwould be a great accompnyment..03:55
*** jimbaker has quit IRC03:55
aimona few iso installs and you are up and runnng03:55
aimonreduces time to implementation for testing considerably03:55
aimonof courser as you said.. it is mostly a packaging job03:55
aimonwe are doing same atm.. TRPMing everything in our case03:56
aimon*RPMing03:56
*** trin_cz has quit IRC03:56
aimonthen can easily uise KS file for centos to instal and configure03:56
aimon.. well we have a bit of puppet action also.. but our situation is more complex03:57
aimonwe have puplic puppet server for KS boot.. it configures itself03:57
aimoncreiht: Who are you btw? Were you at the design summit in Texas?03:58
aimonperhaps we met03:58
creiht<- Chuck from the swift team03:58
creihtand yes I have been to all the design summits03:59
anotherjessecreiht yells at other rackers ...03:59
*** hadrian has quit IRC03:59
creihtlol03:59
aimonhehe03:59
aimonI m with Morph Labs03:59
creihtI wouldn't exactly call it yelling :)03:59
anotherjesseyou yell because you care04:00
aimonI am still amazed by that summit :)04:00
gholtBadgering?04:00
aimonstrongly engouraging  <-- ftw04:00
aimon*encouraging04:00
notmyname"creiht is a very aggressive person. He/She attacked others 9 times." from http://12.am/openstack/04:00
gholthttp://www.badgerbadgerbadger.com/04:00
aimonomg lol04:01
creihthahah04:01
aimonbtw does B. Piatt stalk this forum?04:01
aimon*channel04:01
gholtProbably....04:02
creihtaimon: periodically04:02
gholtblpiatt: ^^04:02
aimoncool04:02
aimonLOL love the Amanita Muscaria (Fly Agaric) reference in the bdger dance04:02
*** jdurgin has quit IRC04:09
*** dendrobates is now known as dendro-afk04:16
*** MikeZ1 has quit IRC04:23
blpiattaimon, I'm around sometimes :)04:30
blpiattcreiht, I'm working on a testbed for openstack so we can have an environment to test API clients, UIs, and run full end to end system tests with performance metrics04:33
blpiattwe'll get that setup rather than a Rackspace Cloud testbed as the product offerings won't be against today's trunk and we'll want to be testing against those when doing new development04:34
creihtblpiatt: cool04:35
creihtI have hardware in our lab now to do a reference swift setup04:35
creihtjust don't have time now to mess with it04:35
alekibangoblpiatt, creiht:  i have fai-project.org  installing openstack system  automatically if you are interested...    i would love to  get automated functional testing of openstack...04:35
creihtalekibango: I think jay pipes is taking the lead on getting the automated functional testing going04:36
alekibangoi will talk to him then :)04:36
blpiattalekibango, looks like good stuff, FAI is in the same bare metal provisioning space as cobbler and barracus?04:36
alekibangosimilar, but for debian, ubuntu04:36
alekibango(and also can work wth other distros if you provide  some images)04:37
alekibangodebian and ubuntu works very well, using packages04:37
blpiattcreiht, alekibango: yeah, jay is going to lead the functional testing, emellor wants to help, so does jordanrinke04:37
alekibangoso i do04:37
blpiattok, so more like barracus than cobbler04:38
alekibangoi dont know barracus :)04:38
blpiatthttp://baracus-project.org/Site/Baracus.html04:38
blpiattNovell just recently open sourced it04:38
alekibangointeresting..04:40
alekibangoblpiatt: fai is not using images with debian/ubuntu04:41
alekibangoit installs packages04:41
alekibangoits 'the debian way'04:41
alekibango(ubuntu also works well now)04:41
alekibangoimages = pain04:43
anotherjessebaracus = perl?04:43
anotherjessehttp://gitorious.org/baracus/baracus/blobs/master/root/usr/sbin/bapower04:43
aimonhey blpiatt!04:43
alekibangofai = shell (or whatever)04:44
blpiattanotherjesse, yeah barracus = perl (mostly)04:44
blpiattalekibango, with barracus you'd use a base image then preseed to install the packages04:45
alekibangosimilar for fai + nondeb distro...04:46
blpiattyeah, sounds like it, I haven't looked at FAI before, I'll do so tomorrow04:47
alekibangoits not very easy, but after some fight, it works well for you04:47
alekibangoand its highly configurable04:47
alekibangoi think we should build and TEST automatically   functionality  of all versions of nova trunk  and branches with merge requests...04:50
blpiattagreed, I want to come up with a NovaMIPS and SwiftMIPS so we can run a set of performance tests as well to see if our changes are improving speed or not on each merge to trunk04:51
alekibangogood idea04:51
blpiattthat way if we have performance tuning to do in the future we can go see the impact of each change made in the past to know where to start looking04:51
* alekibango needs to sleep now, see you tomorow...04:52
winston-dblpiatt: what will NovaMIPS & SwiftMIPS be like?04:53
creihtMIPS?04:53
xtoddxhardware architecture?04:54
blpiattwinston-d, creiht: I use MIPS in reference to BogoMIPS, it'll be some metric we define, it isn't "millions of instructions per second"04:54
creihtahh04:54
winston-dok04:55
creihthehe04:55
creihtwell for swift, we already have swift-bench as a start04:55
blpiattcreiht, nice, I think we have a lot of the pieces we need, it'll be good to pull it all together and automate it04:56
winston-dcreiht: is swift-bench in swift branch? can I use it?04:56
creihtwinston-d: yup04:56
winston-dwhat does swift-bench actually bench?04:56
creihtswift04:56
creiht:)04:56
winston-dI mean what metrics does it report04:57
creihtreminds me that I need to write some docs for it04:57
winston-dcreiht: Write Docs for Swift-Bench, Please04:57
creihtIt will do a number of PUTS GETS and DELETES of objects to a swift cluster04:57
creihtdoh04:58
*** kashyapc has joined #openstack04:58
creihtlooks like some import changed and swift-bench is broken :/04:58
winston-dI'm looking at swift revno 148 source.  where is swift-bench?04:59
creihtwinston-d: in bin05:00
winston-dOK. find it.  thanks05:01
creihtit is configurable for number of objects and concurrency05:01
winston-dHmm, nice.  I'd like to have a simple test in my swift setup05:02
*** MikeZ has joined #openstack05:02
creihtwinston-d: at your rev, it is possible it may still be working05:03
winston-dI've a swift testbed with one proxy, 6 account nodes.  any test configuration recommendation for this setup?05:05
creihtstart with running swift-bench from one machine that has good connectivity to the proxy05:06
winston-dOK05:06
*** jimbaker has joined #openstack05:08
*** Cybo has quit IRC05:14
winston-dcreiht: to use swift-bench, do I need to create user account?05:15
creihtyeah it is probably a good idea to create a user account05:17
creihtfor benchmarking05:17
winston-d2011-01-04 13:46:45,695 INFO swift-bench 5 PUTS [0 failures], 2.3/s05:28
winston-d2011-01-04 13:47:00,988 INFO swift-bench 48 PUTS [0 failures], 2.7/s05:28
winston-d2011-01-04 13:47:16,282 INFO swift-bench 97 PUTS [0 failures], 3.0/s05:28
winston-d2011-01-04 13:47:17,389 INFO swift-bench 100 PUTS **FINAL** [0 failures], 2.9/s05:28
winston-d2011-01-04 13:47:19,831 INFO swift-bench 30 GETS [0 failures], 12.3/s05:28
winston-d2011-01-04 13:47:35,318 INFO swift-bench 319 GETS [0 failures], 17.8/s05:28
winston-d2011-01-04 13:47:50,346 INFO swift-bench 652 GETS [0 failures], 19.8/s05:28
winston-d2011-01-04 13:48:05,835 INFO swift-bench 937 GETS [0 failures], 19.3/s05:28
winston-d2011-01-04 13:48:08,591 INFO swift-bench 1000 GETS **FINAL** [0 failures], 19.5/s05:28
winston-d2011-01-04 13:48:11,011 INFO swift-bench 10 DEL [0 failures], 4.1/s05:28
winston-d2011-01-04 13:48:26,242 INFO swift-bench 59 DEL [0 failures], 3.3/s05:28
winston-d2011-01-04 13:48:41,182 INFO swift-bench 100 DEL **FINAL** [0 failures], 3.1/s05:28
winston-dcreiht: how does this result look like?05:29
creihtincredibly slow05:29
winston-dor, how should i interpret it?05:29
winston-d:)05:30
creihtwinston-d: and next time I would recommend pasting to paste.openstack.org05:30
winston-di see, thanks.05:30
creihtwinston-d: Have you tuned any of the config settings? (like workers)05:31
winston-dno, everything is default05:31
creihtok... default is only one worker05:31
winston-doh, I just check 2 workers for accoutn/container/object05:32
winston-dhow to tune proper number of workers? some ratio to # account/container/object nodes and replicates?05:34
winston-dmaybe also the device per nodes?05:35
creihtwinston-d: Would you mind if we worked through this tomorrow?05:36
creiht(It is time for me to get some sleep)05:36
winston-dof course.  I know it is late for you.05:36
creiht:)05:36
winston-dthank you for your help, creiht. have a good nite05:37
creihta decent starting point for good hardware is:05:37
*** mdomsch has quit IRC05:37
creihtproxy workers: 2x number of cores05:37
creihtacct/cont/obj workers: start with 1x number of cores05:37
winston-dgood. let me do that. I'll re-do the test and discuss w/ u tomorrow. :)05:38
anticwcreiht: what about HT 'cores' ?05:41
anticwcreiht: wrt to obj workers ... why base that on cores?05:41
anticwit's not cpu bound, more IO bound05:41
anticwso something like n x spindles sure makes more sense?05:41
creihtanticw: yeah the obj workers are more complicated, but 1x core is just something to get him going :)05:42
winston-d:)05:42
*** reldan has joined #openstack05:46
winston-danticw: i assume the configuration for rsyncd should be altered too?05:48
anticwno05:48
anticwi think the connections aren't capped05:49
anticwso inbound you'll get a lot but not insane numbers05:49
winston-dalrite05:49
*** GeorgeSebastian has quit IRC05:50
*** sassyn has joined #openstack06:17
*** jeevan_ullas has quit IRC06:28
*** jeevan_ullas has joined #openstack06:28
*** sassyn has quit IRC06:29
*** reldan has quit IRC06:32
*** jbaker has joined #openstack06:36
*** jimbaker has quit IRC06:36
*** jbaker is now known as jimbaker06:36
*** ramkrsna has joined #openstack06:37
*** ramkrsna has joined #openstack06:37
*** sandywalsh has quit IRC06:43
*** sandywalsh has joined #openstack06:56
*** guigui1 has joined #openstack07:00
*** anotherjesse has quit IRC07:12
*** MikeZ1 has joined #openstack07:26
*** MikeZ has quit IRC07:28
*** rcc has joined #openstack07:38
*** rcc has joined #openstack07:39
*** ArdRigh has joined #openstack07:42
*** jimbaker has quit IRC07:50
*** allsystemsarego has joined #openstack07:54
*** allsystemsarego has joined #openstack07:54
*** brd_from_italy has joined #openstack07:58
*** MikeZ has joined #openstack08:01
*** gasbakid has joined #openstack08:02
*** MikeZ1 has quit IRC08:03
*** blpiatt has quit IRC08:09
*** fabiand_ has joined #openstack08:17
*** trin_cz has joined #openstack08:20
*** befreax has joined #openstack08:23
*** reldan has joined #openstack08:26
*** littleidea has quit IRC08:29
*** fabiand_ has quit IRC08:33
*** calavera has joined #openstack08:35
sorenvishy: My concern isn't just about euca2ools. It's about everything that consumes the EC2 API.08:40
sorenvishy: I'm confident that there are many tools out there that expect instance ID's to not be 39 characters long.08:41
*** calavera has quit IRC08:44
*** allsystemsarego has quit IRC08:44
*** kashyapc has quit IRC08:44
*** jtimberman has quit IRC08:44
*** perestrelka has quit IRC08:44
*** lool has quit IRC08:44
*** JordanRinke has quit IRC08:44
*** brainproxy has quit IRC08:44
*** _cerberus_ has quit IRC08:44
*** klumpie has quit IRC08:44
*** cory__ has quit IRC08:44
*** fitzdsl has quit IRC08:44
*** smoser` has quit IRC08:44
*** Dweezahr has quit IRC08:44
*** letterj_ has quit IRC08:44
*** devcamcar has quit IRC08:44
*** sunech has quit IRC08:44
*** deshantm has quit IRC08:44
*** Kdecherf has quit IRC08:44
*** tmarble has quit IRC08:44
*** deshantm_cosi has quit IRC08:44
*** annegentle has quit IRC08:44
*** cloud0_ has quit IRC08:44
*** ioso has quit IRC08:44
*** brd_from_italy has quit IRC08:44
*** gondoi has quit IRC08:44
*** karmabot has quit IRC08:44
*** _0x44 has quit IRC08:44
*** filler has quit IRC08:44
*** sophiap has quit IRC08:44
*** ksteward has quit IRC08:44
*** jeremyb has quit IRC08:44
*** MikeZ has quit IRC08:44
*** jbryce has quit IRC08:44
*** arcane has quit IRC08:44
*** cw has quit IRC08:44
*** clayg has quit IRC08:44
*** yosh has quit IRC08:44
*** reldan has quit IRC08:44
*** gholt has quit IRC08:44
*** Daviey has quit IRC08:44
*** zul has quit IRC08:44
*** dabo has quit IRC08:44
*** soosfarm_ has quit IRC08:44
*** mtaylor has quit IRC08:44
*** herki_ has quit IRC08:44
*** olivier_ has quit IRC08:44
*** styro has quit IRC08:44
*** mjmac has quit IRC08:44
*** termie has quit IRC08:44
*** michaeldreamhost has quit IRC08:44
*** zykes- has quit IRC08:44
*** creiht has quit IRC08:44
*** pandemicsyn has quit IRC08:44
*** Adri2000 has quit IRC08:44
*** patri0t has quit IRC08:44
*** xtoddx has quit IRC08:44
*** ChrisAM has quit IRC08:44
*** morfeas has quit IRC08:44
sorenvishy: So it's also not just about usability.08:44
*** brd_from_italy has joined #openstack08:47
*** filler has joined #openstack08:47
*** sophiap has joined #openstack08:47
*** gondoi has joined #openstack08:47
*** karmabot has joined #openstack08:47
*** _0x44 has joined #openstack08:47
*** ioso has joined #openstack08:47
*** cloud0_ has joined #openstack08:47
*** annegentle has joined #openstack08:47
*** deshantm_cosi has joined #openstack08:47
*** tmarble has joined #openstack08:47
*** Kdecherf has joined #openstack08:47
*** deshantm has joined #openstack08:47
*** ksteward has joined #openstack08:47
*** jeremyb has joined #openstack08:47
*** zelazny.freenode.net sets mode: +v _0x4408:47
*** MikeZ has joined #openstack08:47
*** jbryce has joined #openstack08:47
*** arcane has joined #openstack08:47
*** cw has joined #openstack08:47
*** clayg has joined #openstack08:47
*** yosh has joined #openstack08:47
*** reldan has joined #openstack08:48
*** gholt has joined #openstack08:48
*** Daviey has joined #openstack08:48
*** zul has joined #openstack08:48
*** dabo has joined #openstack08:48
*** soosfarm_ has joined #openstack08:48
*** mtaylor has joined #openstack08:48
*** herki_ has joined #openstack08:48
*** olivier_ has joined #openstack08:48
*** styro has joined #openstack08:48
*** mjmac has joined #openstack08:48
*** termie has joined #openstack08:48
*** michaeldreamhost has joined #openstack08:48
*** Adri2000 has joined #openstack08:48
*** zykes- has joined #openstack08:48
*** creiht has joined #openstack08:48
*** pandemicsyn has joined #openstack08:48
*** patri0t has joined #openstack08:48
*** xtoddx has joined #openstack08:48
*** ChrisAM has joined #openstack08:48
*** morfeas has joined #openstack08:48
*** zelazny.freenode.net sets mode: +vvv mtaylor creiht pandemicsyn08:48
*** localhost has quit IRC08:48
*** mtaylor has quit IRC08:48
*** mtaylor has joined #openstack08:48
*** kashyapc has joined #openstack08:48
*** calavera has joined #openstack08:48
*** allsystemsarego has joined #openstack08:48
*** jtimberman has joined #openstack08:48
*** perestrelka has joined #openstack08:48
*** lool has joined #openstack08:48
*** JordanRinke has joined #openstack08:48
*** Dweezahr has joined #openstack08:48
*** brainproxy has joined #openstack08:48
*** _cerberus_ has joined #openstack08:48
*** klumpie has joined #openstack08:48
*** cory__ has joined #openstack08:48
*** fitzdsl has joined #openstack08:48
*** smoser` has joined #openstack08:48
*** letterj_ has joined #openstack08:48
*** devcamcar has joined #openstack08:48
*** sunech has joined #openstack08:48
*** zelazny.freenode.net sets mode: +v _cerberus_08:48
*** mtaylor has quit IRC08:49
*** mtaylor has joined #openstack08:49
*** localhost has joined #openstack08:50
*** ibarrera has joined #openstack08:53
*** mtaylor has quit IRC08:58
*** mtaylor has joined #openstack08:58
*** ChanServ sets mode: +v mtaylor08:58
*** fabiand_ has joined #openstack09:01
*** trin_cz has quit IRC09:01
*** rchavik has quit IRC09:07
*** reldan has quit IRC09:16
*** Jordandev has quit IRC09:20
*** rchavik has joined #openstack09:26
*** littleidea has joined #openstack09:29
*** sassyn has joined #openstack09:33
*** DigitalFlux has joined #openstack09:49
*** colinnich has joined #openstack09:55
*** aku1 has joined #openstack10:01
*** aku1 has quit IRC10:06
*** fabiand_ has left #openstack10:06
*** sassyn has quit IRC10:12
*** sassyn has joined #openstack10:14
*** MarkAtwood has quit IRC10:16
*** littleidea has quit IRC10:17
*** MarkAtwood has joined #openstack10:18
*** littleidea has joined #openstack10:18
*** gasbakid has quit IRC10:23
*** sassyn has quit IRC10:23
*** gasbakid has joined #openstack10:24
*** gasbakid has quit IRC10:27
*** gasbakid has joined #openstack10:29
*** gasbakid has quit IRC10:36
ttxcreiht: ok :)10:43
*** littleidea has quit IRC10:49
*** _skrusty has quit IRC11:04
ttxso 2011 so far seems to be the year of huge scrollbacks and hot technical discussions (UUIDs vs. readability, daemontools vs. what the world actually uses)11:09
*** _skrusty has joined #openstack11:18
* soren missed the daemontools one11:28
*** ArdRigh has quit IRC11:29
* soren lunches11:30
*** rchavik has quit IRC11:32
*** gasbakid has joined #openstack11:36
*** schisamo has joined #openstack12:14
*** sandywalsh has quit IRC12:24
*** lool has quit IRC12:24
*** lool has joined #openstack12:25
*** MarkAtwood has quit IRC12:33
*** arthurc has joined #openstack12:35
*** sandywalsh has joined #openstack12:38
*** lool has quit IRC12:39
*** jbryce has quit IRC12:40
*** jbryce has joined #openstack12:41
*** arthurc has quit IRC12:42
*** arthurc has joined #openstack12:43
*** MarkAtwood has joined #openstack12:45
*** lool has joined #openstack12:49
*** arthurc has quit IRC12:50
*** arthurc has joined #openstack12:55
*** kashyapc has quit IRC13:01
*** b_erb has joined #openstack13:03
b_erbhi13:05
b_erbi followed the SAIO instructions and additionally edited the default_cluster_url entry in auth-server.conf into a local network address (instead of 127.0.0.1).13:05
b_erbbut after auth'ing, i still get back 127.0.0.1 URIs as X-Storage-Url13:05
b_erbi'm using 1.1.013:05
b_erbany idea how to get back the right url?13:06
*** nelson__ has quit IRC13:10
*** nelson__ has joined #openstack13:11
nelson__ttx: there's a reason why daemontools exists: because what the world actually uses actually sucks.13:14
sorennelson__: I just don't really see daemontools solving any problems that I have.13:15
nelson__and when people defend the suck by saying "but that's what everybody uses" I kinda am unimpressed by that argument, because suck is still suck even if everybody uses it.13:15
ttxnelson__: I don't defend the suck. But distributions have already chosen how they want to resolve the suck (upstart, for example). Consistency has value.13:16
ttx(personally, I think this needs to be solved at distro packaging level, not inside our core code)13:16
nelson__soren: I understand that. Oftentimes, people don't realize that they have a problem until they see the solution. That's the role of the entrepreneur. But ahhh, I'll just put my lampshade on to continue talking about economics. :)13:17
nelson__ttx: the problem is that people demand init.d, and the distros listen to them.  But then to say that we should use init.d because the distros use them is circular reasoning.13:18
ttxnelson__: in case you have missed the memo, distros are moving away from init.d. And not towards daemontools.13:19
nelson__ttx: and I agree with you that consistency has value, but not sucking also has value.13:19
nelson__by distros you mean Ubuntu.13:19
ttxand Fedora.13:19
nelson__what's Fedora using?13:20
ttxmoving towards systemd, last time I looked.13:20
alekibangoi would say lets support both... its possible, not much work, its nice to most ppl13:20
nelson__Oh, I see, so now we can throw out consistency as a reason not to use daemontools.13:20
ttxDebian also moving away, though they still hesitate. daemontools is not part of their options.13:21
ttxnelson__: no, I throw out consistency as a reason not to make any choice.13:21
ttx(from the upstream project perspective)13:22
ttxinit scripts are distribution integration. Let the distributions handle it (for consistency with the rest of their stuff) in packaging.13:22
nelson__urgh, I'm not following you13:22
nelson__okay, I give. Clearly I'm not going to change the world on this one.13:23
ttxnelson__: I see no reason for an upstream project to provide init scripts. Or daemontools scripts. Or upstart scripts.13:23
nelson__I much prefer daemontools to syslog /var/log init.d /var/run and that farrago.13:23
nelson__ttx: you should go read Dan Bernstein's reasoning.13:24
nelson__Unix fragmentation is, in his mind, a very bad thing, because it reproduces the FAIL that caused Unix to suffer twenty years ago.13:24
alekibangonelson__: he is right that is packaging issue... but we have at least one  packager right here :)13:25
nelson__But I also realize that a lot of people pushing the suck weren't alive then.13:25
b_erbwhat do i have to configure to get non-"127.0.0.1" storage URLs? already changed default_cluster_url entry in auth-server.conf13:25
ttxI'm not sure I want to go deeper into djb's mind :)13:25
*** kashyapc has joined #openstack13:25
nelson__ttx: yeah, see, now that's where you're going irrational on me.13:25
nelson__djb has a very bad reputation in the Unix community ... but that's because he was very pushy. But he was pushy because he was right.13:26
nelson__http://lesswrong.com/lw/372/defecting_by_accident_a_flaw_common_to_analytical/13:26
ttxnelson__: being right is often not enough.13:26
nelson__yes, you have to be powerful.13:27
ttxbeing accepted by others is more important.13:27
nelson__and by being offensive and radical, he gave up power.13:27
nelson__ja.13:27
nelson__and sometimes the best solution is too much of a change for people to accept.13:27
*** fraggeln has joined #openstack13:29
*** aliguori has joined #openstack13:32
*** jfluhmann has quit IRC13:35
*** jfluhmann_ has joined #openstack13:35
*** jbryce has quit IRC13:35
*** localhost has quit IRC13:41
*** jbryce has joined #openstack13:50
*** hadrian has joined #openstack13:51
*** Podilarius has joined #openstack13:51
*** roamin9_ has joined #openstack13:54
*** Ryan_Lane|away is now known as Ryan_Lane13:54
*** hggdh has quit IRC13:58
*** hadrian has quit IRC14:00
*** hggdh has joined #openstack14:01
*** stewart has quit IRC14:04
*** roamin9_ has quit IRC14:05
*** stewart has joined #openstack14:07
*** ccustine has joined #openstack14:08
daboHow do I update my database to match a new model.py that adds columns? Is there a programmatic way, or do I do it manually?14:09
sandywalshI think sqlalchemy does it, but migrations are being done in nova currently (afaik)14:13
sandywalsh*aren't14:13
*** Lcfseth has joined #openstack14:13
daboso I should just use ALTER TABLE?14:15
sandywalshif it's not faster to just drop the db14:16
*** b_erb has quit IRC14:16
*** mark has joined #openstack14:17
*** ppetraki has joined #openstack14:17
dabonah, alter table is quicker14:20
*** kashyapc has quit IRC14:21
*** hadrian has joined #openstack14:27
sorenAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARGH!14:29
Ryan_Lanesoren: one of those days?14:30
sorenI've spent absolutely ridiculous amounts of time trying to get the SOAP API to work. Turns out what I've been doing all along has been great, except my xmlns property on my FooResponse wasn't what the client expected.14:31
soren...so it just segfaulted.14:31
Ryan_Lane:D14:31
*** b_erb has joined #openstack14:31
sorenNow to unwind all my horrid debugging hacks.14:31
ttxsoren: SOAP is bad for you.14:32
*** jdarcy has joined #openstack14:32
sorenttx: This is where Rick would usually drop a comment about Europeans and body odour.14:32
ttxsoren: too bad he can't talk.14:33
sorenYeah. Too bad.14:37
*** littleidea has joined #openstack14:43
*** f4m8 is now known as f4m8_14:47
*** reldan has joined #openstack14:48
*** sophiap has quit IRC14:48
*** sophiap has joined #openstack14:57
*** ramkrsna has quit IRC14:59
ttxRyan_Lane: you should add your blog to http://planet.openstack.org/15:02
creihtb_erb: The storage url is stored in the users record in the auth.db, so if you change it in the config, any users you created previously will still have the old url15:02
creihtyou need to either edit the db, and update the rows, or just delete and recreate your users15:03
creihtnelson__: Nothing prevents you from using daemontools.  If there are things we can do to make it easier for you to do so, please let us know15:04
creihtnelson__: And if our init script is the worst thing that someone complains about for swift, then I am a happy developer :)15:05
nelson__creiht: I did a trial run of it last night and swift-auth-server seems to be exiting from time to time. Not sure why.15:05
*** nijaba has quit IRC15:05
creihtnelson__: did you try running it manually?15:06
Ryan_Lanettx: ah. didn't realize I could. will do15:06
creihtand also just to make sure we are clear, the auth server is just made for dev purposes15:06
*** smoser` is now known as smoser15:06
annegentleRyan_Lane: yep, instructions for adding your blog to the planet: http://wiki.openstack.org/AddingYourBlog15:07
nelson__am doing now.  right; I know that we'll need our own auth ... but of course almost nothing we do is private.15:07
creihtand by exiting, do you mean at start, or at some point after running?15:07
*** nijaba has joined #openstack15:07
nelson__after about twenty seconds.  but now, from the command line, it hasn't existed.15:08
nelson__I'll look into it some more.15:08
creihtok15:08
creihtwhen it was exiting, were you testing at any load?15:08
nelson__no15:08
creihthrm15:09
creihtnelson__: let me know if you find anything else15:13
colinnichcreiht: while you are on the subject, do you know if SWAuth is ready for production?15:13
nelson__creiht: will do.15:14
creihtcolinnich: depends on how you define production :)15:14
colinnichcreiht: does it work? :-)15:14
gholtIt's not even merged to trunk yet. Heh.15:15
creihtit works for as much as we have tested it so far15:15
creihtoh heh15:15
creihtdidn't realize it wasn't merged yet :)15:15
nelson__Oh, creiht, in #7 of http://swift.openstack.org/howto_installmultinode.html#configure-the-proxy-node why do the second two names end in _1 ?15:15
nelson__I'm guessing that's a typo.15:15
colinnichgholt: what's the best way to get it?15:15
creihtnelson__: yes, copy paste error15:16
*** dendro-afk is now known as dendrobates15:17
gholtcolinnich: This is the latest: https://code.launchpad.net/~gholt/swift/swauth2/+merge/43312 I've tested the heck out of it on my own, but that's never enough. And it hasn't been used in a large prod environ.15:17
creihtcolinnich: At some point the plan is to replace the dev_auth with it15:17
colinnichwould I take the latest swift and merge in that branch?15:18
gholtYou can just check that branch out directly. bzr branch lp:~gholt/swift/swauth215:19
*** jdarcy has quit IRC15:19
*** WonTu has joined #openstack15:19
*** WonTu has left #openstack15:19
gholtI do my best to keep it up to date with changes from trunk; it currently is up to date.15:19
colinnichgholt: that was my next question :-)15:20
colinnichgholt: thanks15:20
Ryan_Lanein nova, am I able to have the private IP network on one vlan, and the public floating IPs on another?15:20
colinnichbetter get my skates on though, as I see there is some competition over this side of the pond now :-)15:21
creihtlol15:21
* creiht guesses that colinnich is referring to our release of cloud in the UK? :)15:22
colinnichyup :-)15:22
creihtcompetition makes everything better right? :)15:24
colinnichcan't complain since you are supplying the software......15:24
creihthehe15:24
*** blpiatt has joined #openstack15:26
*** westmaas has joined #openstack15:26
*** reldan has quit IRC15:26
*** reldan has joined #openstack15:27
*** jimbaker has joined #openstack15:28
*** jimbaker has joined #openstack15:28
*** _skrusty is now known as sdkrusty15:41
*** sdkrusty is now known as skrusty15:41
*** reldan has quit IRC15:43
*** rnirmal has joined #openstack15:43
* soren breaks for dinner15:46
czajkowskisoren: no break for you! slides all done for your talk at FOSDEM :)15:47
ttxcan you do mines as well ?15:49
czajkowskittx: looking forward to going ?15:51
ttxczajkowski: I'm going. And presenting.15:51
ttxczajkowski: "Why Linux distributions hate Java", in the Java devroom.15:51
czajkowskittx: excellent, are you giving a lightning talk or presentation?15:51
czajkowskiahhhh nice15:51
ttxshould be lots of fun, if I survive it.15:54
czajkowskiwell if you surrive the Friday night you'll be fine :)16:00
vishysoren: the change I'm talking about is internal so it will work with all tools16:01
*** johnpur has joined #openstack16:03
*** ChanServ sets mode: +v johnpur16:03
*** guigui1 has quit IRC16:05
*** littleidea has quit IRC16:08
*** mdomsch has joined #openstack16:08
*** opengeard has quit IRC16:12
*** crago has joined #openstack16:15
nelson__creiht: remember that sphinx build problem I had yesterday? It seems to be a sphinx version thing: my version of sphinx doesn't define css_files, so when _theme/layout.html attempts to append something to it, it whines because it doesn't exist.16:18
nelson__The fix is to simply set, rather than to append to css_files. But the next question is: why am I getting this error under Ubuntu 10.04.1 ?16:18
nelson__especially since it's a supported platform.16:19
*** calavera has quit IRC16:19
*** dirakx has joined #openstack16:19
creihtnelson__: Most of us have sphinx v1.016:20
ttxvishy: +116:21
creihtlooks like lucid's default packaging is 0.616:21
*** reldan has joined #openstack16:22
creihtthe docs use to build with the older version16:22
creihtI'll add that as a bug so that we can fix it16:22
ttxvishy: who at Anso works on https://blueprints.launchpad.net/nova/+spec/snapshot-instance , if anyone ?16:22
creihtnelson__: do you still have a copy of the traceback?16:22
vishyttx: no one has it, but I think it should be a relatively easy addition to the cow code I'm working on16:23
vishyin fact I'm not really sure how to do it without cow16:23
nelson__creiht: yes, but the problem isn't in the Python code.16:23
nelson__it's in the mini-language that jinja2 implements.16:23
ttxvishy: you think that cow code you're working on may make it in time for the Bexar feature merge window ?16:24
nelson__actually, I shouldn't say that. perhaps it has the concept of "add this to the end of that even if that doesn't exist yet"16:24
creihthehe16:24
creihtnelson__: well whatever error message that was there :)16:24
nelson__do you want me to paste it here?16:25
ttxvishy: is that https://blueprints.launchpad.net/nova/+spec/cow-instances ?16:25
creihtpaste.openstack.org16:25
nelson__(hehe)16:25
creihtplease16:25
nelson__http://paste.openstack.org/show/376/16:26
nelson___theme/layout.html really does have a line adding tweaks to css_files, and I looked in the files that it includes, and nobody sets css_files in the first place.16:27
creihtMy guess is that in 1.0 it is predefined16:27
nelson__obviously, if it's working for you ... yeah.16:27
vishyttx: yes, almost done with that one, trying to figure out key injection using cow16:32
*** burris has quit IRC16:33
ttxvishy: ok. For reference will assign snapshot-instance to you and consider it "Not started". Feel free to reassign if needed16:33
*** MarkAtwood has quit IRC16:35
*** MarkAtwood has joined #openstack16:35
uvirtbotNew bug: #697288 in swift "Building docs with sphinx <1.0 results in an error" [Low,Triaged] https://launchpad.net/bugs/69728816:37
*** burris has joined #openstack16:38
*** DigitalFlux has quit IRC16:38
*** glen_ has joined #openstack16:41
*** Edwin__ is now known as EdwinGrubbs16:41
*** glen_ is now known as glenc16:41
nelson__hrm. As far as I can tell, _static/tweaks.css isn't being used by anything.16:42
*** dirakx has quit IRC16:42
nelson__and neither is css_files.16:42
creihthrm16:42
nelson__and it's not referred-to by ANYTHING in my /usr/lib/pymodules16:43
nelson__sounds like dead code to me.16:43
creihtThe theme was changed a bit ago, so it might just be from the previous theme16:43
creihtnelson__: thanks16:43
nelson__I'll boldly remove it, and if it breaks something that somebody cares about, they can revert.16:44
*** befreax has quit IRC16:46
*** joearnold has joined #openstack16:46
*** trin_cz has joined #openstack16:47
*** evilrob has left #openstack16:47
*** ctennis has quit IRC16:48
*** kdsingh has joined #openstack16:48
*** joearnold has quit IRC16:51
*** eday has joined #openstack16:56
*** ChanServ sets mode: +v eday16:56
*** ctennis has joined #openstack17:00
*** daleolds has joined #openstack17:00
*** singh_ has joined #openstack17:02
*** jsgotangco has joined #openstack17:04
ttxjbryce: do you have the next POC meeting scheduled ?17:05
*** ibarrera has quit IRC17:08
*** jsgotangco has quit IRC17:08
*** reldan has left #openstack17:10
Ryan_Laneis it possible to have nova create and connect multiple NICs connected to multiple VLANs?17:11
Ryan_Laneor multiple bridges?17:12
*** dirakx has joined #openstack17:12
*** brd_from_italy has quit IRC17:13
*** dubsquared has joined #openstack17:16
*** cynb has joined #openstack17:16
*** rcc has quit IRC17:17
*** jdurgin has joined #openstack17:18
sandywalshRyan_Lane, currently nova instances can only have 1 NIC17:21
Ryan_Lane:(17:22
Ryan_Lanethat's what I was assuming17:22
sandywalshthere is a bp for multi-nic that I was working on, but it was pushed to cactus17:22
Ryan_Laneso that means all IPs must be on a public IP range17:22
sandywalshcurrently, yes17:22
* Ryan_Lane gets ready for some dirty hacks17:22
sandywalsh'friad so17:23
Ryan_Lanewhen's ipv6 support slated?17:23
sandywalshhmm, good question. I'd have to search the bp's17:23
Ryan_Lane(we are running out of public IPs, and can't get any more :( )17:23
* creiht is stashing IPs under his matress for the IPocalypse17:24
dendrobatesRyan_Lane: it is planned for bexar17:25
Ryan_Laneipv6 support? great17:25
dendrobatesNTT is working on it and should be submitting a merge request soon17:26
sandywalshRyan_Lane, https://blueprints.launchpad.net/nova/+spec/ipv6-support17:26
dendrobateshttp://wiki.openstack.org/BexarIpv6supportReadme17:26
*** rlucio has joined #openstack17:29
*** singh_ has left #openstack17:30
*** schisamo has quit IRC17:34
*** hggdh has quit IRC17:35
*** maplebed has joined #openstack17:35
*** hggdh has joined #openstack17:39
*** schisamo has joined #openstack17:39
zaitcevIt was quite educational in the NTT IPv6 preso that we need a replacement for NAT due to cloud, no matter if global addresses are used (incl. IPV4 global addresses).17:49
*** joearnold has joined #openstack17:51
termievishy: are you at anso?17:56
termievishy: i'm about to get a sandwich at jump start and i would like to head over17:56
termievishy: also i need to get a key17:56
*** Mr_T has joined #openstack17:58
*** Mr_T has left #openstack17:58
*** colinnich has quit IRC17:58
*** Ryan_Lane is now known as Ryan_Lane|food17:58
b_erbcreiht: zhx18:09
b_erbcreiht: thx18:09
xtoddxcan I bum some reviews from nova-core?  https://code.launchpad.net/~anso/nova/newlog2/+merge/4510018:15
xtoddxLogging is ready for merge, the rebranch was to fix the deleting and re-adding of the cloudpipe directory18:16
*** rlucio has quit IRC18:18
*** jamfl__ has joined #openstack18:45
*** glenc has quit IRC18:55
*** glenc has joined #openstack18:55
*** glenc has quit IRC18:58
*** dendrobates is now known as dendro-afk18:58
*** dendro-afk is now known as dendrobates18:58
*** jamfl__ has quit IRC19:02
*** glenc has joined #openstack19:03
*** glenc has left #openstack19:04
*** glenc has joined #openstack19:04
*** glenc has joined #openstack19:05
*** glenc has quit IRC19:05
*** Ryan_Lane|food is now known as Ryan_Lane19:06
*** MikeZ has quit IRC19:14
sorenvishy: Er... Whcih change is this?19:17
vishyshorter ec2 references19:17
sorenczajkowski: Slides? Whuh?19:17
vishysoren: so the idea is we can have full uuids but allow users to refer to them using only enough to make them unique19:18
sorenvishy: Once again, your ideas are fascinating to me, and I still wish to subscribe to your newsletter.19:18
vishy:)19:18
edayvishy: how can we guarantee this? we still have the same problem as if we have a smaller, secondary key19:18
soreneday: There's a problem with the ipv6 key thing, by the way.19:19
edaysoren: I hadn't thought much about it, was an idea from pvo :)19:19
soreneday: While it indeed is 128 bit, we don't control the whole thing. Maybe we just have a /64 and then we're down to a 64 bit key space.19:19
vishysoren, eday: say your uuid is 234af34fdc2343bd34342 for example19:19
*** glenc has joined #openstack19:20
vishyyou can do: euca-describe-instances ami-23419:20
sorenOh.19:20
sorenThat doesn't really help.19:20
vishythe ec2_api can do a fuzzy search19:20
sorenWell, a bit.19:20
edayvishy: what if another instance is created with that prefix before the next req?19:20
sorenIt doesn't solve the problem of clients suddenly receiving ID's massively longer than they expect.19:21
vishyit just gives an error, duplicate19:21
vishysoren: true, but it helps a lot with usability19:21
sorenvishy: True.19:21
vishyalternatively, I really like the idea of nicknaming making display_name unique per project19:22
vishyand also using it for reverse dns19:22
vishyunfortunately it would require some kind of availability zone hack to allow for it on run19:22
vishyeuca-run-instances -z east1:blue19:22
sorenI still believe 128 bit keys are overkill.19:22
vishyeuca-describe-instances ami-blue19:23
vishyer19:23
vishyeuca-describe-instances i-blue19:23
glencd19:23
vishydon't know why I was using ami above, which is the prefix for images not instances19:23
* nelson__ has his own class C.19:23
glencexcuse me19:23
*** Zlasher has joined #openstack19:24
sorenvishy: 40 bit keys still lets us have a million physical hosts with a million VM's on each.19:25
sorenThat's about as far as my ambitions go.19:25
sorenFor this lifetime.19:26
vishysoren: just looking at your email, compatibility concerns are a bit annoying19:26
sorenThey are.19:26
sorenMost things we're working on are :)19:26
ZlasherIs't possible to run game servers like Minecraft, Battlefield and such in a cloud enviorment?19:27
vishysoren: can we use a uuid and just use a portion of it for the ec2api?19:27
vishysoren: so internally openstack can support a much larger space?19:27
edayvishy: we could, but it depends on the UUID algorithm we use19:27
sorenvishy: There'll be a miniscule chance of collisions.19:27
edayvishy: some bits are more static than others for different algos19:28
sorenOne thing we haven't really anaylised though:19:28
eday(ie, mac address of generating host)19:28
sorenWhat will happen if there's a collision?19:28
edaysoren: thats what I asked above, vish said respond with an error19:28
sorenOh, er... no.19:28
sorenThat's not what I meant.19:28
edayand then you'd need to relist your instances, and get a different ID set19:28
edaywhich could be a problem19:28
edayif you store your instance list and it changes underneath you (although it's the same instances with longer IDs to make them unique)19:29
sorenI mean.. Not a collision after we truncate the full uuid to the ec2 id.19:29
sorenBut an actual collision.19:29
sorenIf two hosts generated the same uuid, independently.19:29
edaysoren: we can guarantee we don't with UUIDs, some implementations give you that19:29
edayunless you use the same mac address for two hosts, in which case you should be shot19:29
sorenYes, yes, but.19:29
sorenThis is not the point.19:30
*** troytoman has joined #openstack19:30
sorenI'm trying to determine, in the theoretical case where you accidentally assign the same uuid to two instances, what happens?19:30
sorenWhat's the effect?19:30
sorenLet's forget uuid's for a second.19:30
sorenIf we had a 4 bit keyspace.19:31
sorenYes, four.19:31
sorenCollisions are guaranteed. :)19:31
sorenWhat happens when you get a collision?19:31
edaywell, you could detect and log it for admin resolution, but, if you use a proper UUID lib where this never happens, it's a non-issue :)19:31
sorenNo, no.19:31
sorenI'm..19:31
sorenErm..19:31
sorenOk, let's try it like this:19:32
edayI mean, why consider cases that we can guarantee don't happen?19:32
sorenI just think it's interesting to understand exactly what we're trying to avoid.19:32
edayif we did19:33
sorenIf the only effect is that one instance every 28 years across the entire infrastructure fails to start...19:33
edayany request to the system would return an internall error19:33
sorenI don't care much.19:33
edayand an admin notification is sent (email, log, ...) to resolve it19:33
edaywould be the easy way19:33
sorenIf it means that every once in a while, people can steal each other's data, I care.19:33
edayright now the DB looks have first() appended to the sqlalchemy queries, so you would just get the first one according to insert order19:34
edaywhich would be very bad19:34
sorenWell..19:34
edaywe would want to make sure only one was returned19:34
edayfor a given key19:34
sorenRight.19:34
edayand if > 1, report errors19:34
edaybut that, to me, is an unacceptable case for a distributed system :)19:35
sorenLet's start over..19:35
sorenAre we in agreement that 128 bit keys brings along some problems?19:35
edayIf we care about ec2 api, it may have some problems, yes :)19:35
sorenOk.19:36
sorenThere are a number of ways to attack this:19:36
soren1) Ignore it.19:36
soren2) Keep the 128 internally (and in the OS api) and find a reaonsably good hashing algorithm to turn 128 bit keys into 40 bit ones for EC2.19:37
soren3) Use smaller keys.19:37
vishyhttp://www.jackofallclouds.com/2009/09/anatomy-of-an-amazon-ec2-resource-id/19:37
soren4) Change the EC2 api in curious ways (that people also don't expect).19:37
edayI see 2 and 3 as the same really19:38
edaywhat matters is how unique the smallest key is19:39
edayand if you can't guarantee it, what you do to resolve19:39
ttxYou could make 1-3 configurable19:39
edayso hashing a larger key to make a smaller one is really just using a smaller one19:39
*** glenc has left #openstack19:40
termiei am late to the conversation but what examples do we have of people not being able to fit our keys?19:40
ttxso that people can tune their EC2-lookalikeness level19:40
sorenscalr.19:40
sorenFor one.19:40
sorenIt was the first I looked at.19:40
termiesoren: varchar(20) you said?19:40
edayttx: I'm afraid options for something so fundamental will lead to extra complexity and bugs :/19:40
sorentermie: Yeah.19:40
termiesoren: 128 bits fit in that as base-36 don't they?19:40
sorentermie: Nope.19:40
sorenln(2^128)/ln(36) ~24.7585796630106615597319:41
soren25 chars.19:41
sorenplus "i-"19:41
soren27.19:41
*** Lcfseth has quit IRC19:41
*** MikeZ has joined #openstack19:41
*** MikeZ1 has joined #openstack19:41
sorenI still rather like the idea of handing out ID's in a great big bulks.19:42
sorenOnce a new server is started that needs to assign ID's for something, it requests a great big pile of them.19:43
vishyyou and your natural logs19:43
sorenThey're handed out from some sort of central authority, ensureing uniqueness.19:43
termiehmm, amazon claims that their implementation details for their ids are subject to change19:43
vishyor 30 for "snap-"19:43
sorentermie: They certainly do.19:43
sorentermie: ...but they haven't, so clients don't expect it.19:44
termiei don't know whether 1 example is enough19:44
termiedoes euca care?19:44
sorenThey generate ID's of similar size to EC2.19:44
termiedo their tools care19:44
termiedoes boto19:44
sorenNo.19:44
sorenPython rarely cares about such things.19:45
termiemy gut instinct would say it probably isn't actually a very widespread problem19:45
sorenC and Java do.19:45
sorenAnything that stores anything in a database does.19:45
*** MikeZ has quit IRC19:46
sorenThe EC2 API isn't an open standard. Their API reference can say whatever it pleases. If the actual service behaves differently, that's what we need to do, too.19:46
sorenWe're not trying to be better at implementing their API according to their spec than they are.19:46
vishyso do we just keep a separate ec2 id?19:46
termienot claiming we are, just trying to get some examples of tools that care19:47
vishyperhaps even in the compatibility layer?19:47
termiefor example, they claim things should just take strings19:47
termieso base-64 is fine19:47
termieas an encoding format, however plenty of things probably aren't escaping their output properly19:48
vishybase 64 only saves a couple chars19:48
termiebecause they don't expect fancy chars19:48
sorenRight.19:49
termiewe are using base-36 right now from what i gather, which appears already different from ec219:49
*** b_erb has left #openstack19:49
sorenHow so?19:49
termiethey are using hex :/19:50
sorenI thought that's what they used.19:50
vishythey use base-16 i think19:50
sorenOh. Wow.19:50
sorenbc19:50
sorenwhoops19:50
termiethey don't claim to use hex, but it is what they are doing19:50
sorenOk, so they're actually using 32 bit keys. Wow.19:51
vishyagreed19:51
termiesoren: based on that link though their keys are also rather silo'd19:51
sorentermie: Yup.19:51
sorenI think that's less of an issue than the length, though.19:51
*** glenc has joined #openstack19:51
termiewhich means, amongst other things, that they are probably using batches assigned by different master counters19:52
sorenI doubt very many consumers of the api are taking their time to strip off the "i-" prefix and converting to an int and converting back when they need it.19:52
sorenThe length of the thing is much more likely to be a problem.19:52
termiebut, again, a problem for who19:53
termiedo we expect a website that is built against amazon to be able to switch to openstack seamlessly?19:53
sorenThe fact that there are seams isn't must of a justification to add more.19:53
sorenI'd hope they could do it seamlessly.19:54
termieno, but it is important to decide where our goals are19:54
*** joearnold has quit IRC19:54
termiewe could possibly go through a lot of effort to make a system that is that compatible without actually needing to19:54
sorenMine is quite short: I want to let people currently using EC2 to be able to point their stuff at nova and not be surprised.19:54
pvoI don't think our goals are to provide seamless compatability19:55
sorenWho are "we"?19:55
termieif all of our users are built for openstack and can leverage most existing api libraries i think we are pretty much there19:55
sorenIf we don't try to make it seamless, why do we try at all?19:55
termiesoren: us? myself, you, vishy...19:55
pvoi interpreted we as "openstack community"19:55
sorenWell, I just said it was one of my goals. pvo said it wasn't one of "ours". I wonder who "we" are.19:56
vishysoren: there isn't really a good solution to this19:56
vishyI don't want to adopt amazons keysize particularly19:56
sorenvishy: That's really not been established.19:56
sorenvishy: Hang on, let me run some numbers real quick.19:56
vishysoren: so it seems like a compatibility layer is the "best" option19:56
sorenHow about this:19:58
sorenThe EC2 API grows a new flag:19:58
ttx--use_short_ec2like_ids_that_increase_collisions ?19:59
sorenec2_id_key_bits19:59
sorenPretty much, yes.19:59
sorenIf someone actually has a collision, I'll buy them a beer.19:59
sorenSeriously, if they run a million instances, there's a 1% chance they'll have had 1 (one) collision.19:59
sorenEver.20:00
sorenWith 40 bit keys, that is.20:00
* ttx makes mental note. I generated a collision on a eucalyptus setup, but that might just be their rng20:00
sandywalshhow about our access keys with project tacked on the end?20:00
sorenttx: Maybe they copied the RNG from openssl.20:00
* soren runs20:00
* soren needs to tend to other business for a while20:01
sandywalsh(speaking of not exactly compatible)20:01
vishyI'm happy with the flag, does it just fill the rest of the internal_id with zeros?20:02
sleepson-soren - isn't there a  50% chance of collision at 1M?20:04
*** anotherjesse has joined #openstack20:05
*** trin_cz has quit IRC20:05
*** glenc_ has joined #openstack20:20
*** glenc has left #openstack20:20
vishyis anyone else getting TypeError: argument must be 9-item sequence, not datetime.datetime in trunk?20:25
vishyah nm, my bad20:26
ttxTeam meeting in 35 minutes in #openstack-meeting !20:27
*** joearnold has joined #openstack20:27
*** glenc_ has quit IRC20:27
*** troytoman has left #openstack20:27
*** troytoman has joined #openstack20:27
*** glenc has joined #openstack20:28
*** glenc_rackspace has joined #openstack20:28
*** glenc_rackspace has quit IRC20:28
*** pothos_ has joined #openstack20:29
*** pothos has quit IRC20:30
*** pothos_ is now known as pothos20:30
Ryan_Laneshould uec images work now that raw image support is in trunk?20:32
dubsquared+120:33
*** trin_cz has joined #openstack20:36
ttxRyan_Lane: they did last time I checked20:38
Ryan_Lanehmm20:38
Ryan_LaneI must have some other problem20:38
ttxRyan_Lane: did you use uec-publish-tarball to register them ?20:39
Ryan_Laneyep20:39
ttxRyan_Lane: what are the symptoms ?20:39
Ryan_Lanesec20:40
sorensleepson-: How do you figure that?20:40
Ryan_Lanettx: http://pastebin.com/NQCvqtKn20:40
Ryan_Lanein nova-compute log20:40
sorensleepson-: Darn it, I closed the terminal where I had run my numbers.20:42
ttxdoesn't ring a bell. Do the ttylinux images work any better ?20:42
ttxRyan_Lane: ^20:42
Ryan_LaneI'm using a flatnetwork20:43
Ryan_Lanettylinux images don't work20:43
sorenRyan_Lane: Look in /var/log/libvirt/qemu/instance-1315349274.log20:43
Ryan_Laneunless that's been changed recently20:43
zuldevcamcar: ping where is the web ui stuff, i thought it was proposed already?20:43
Ryan_Lanesoren: not much there: http://pastebin.com/bsnK69Ng20:44
edaysoren: birthday paradox20:44
Ryan_LaneI wonder if apparmor is causing problems20:44
sorenRyan_Lane: Plenty of space left?20:44
Ryan_Laneah. lemme check that20:44
Ryan_Lanebingo20:45
Ryan_Lanethat may be it20:45
Ryan_Lanewell. maybe, maybe not. 3GB left20:45
Ryan_Lanebut to be sure, clearing that out and trying again20:45
soreneday: Sure, I understand the maths. :)20:45
soreneday: I just suspect there's a bracket that got misplaced, because now that I try again, I get ~50%.20:46
*** ctennis has quit IRC20:46
edaysoren: ahh, ok20:48
ttxTeam meeting in 10 minutes in #openstack-meeting !20:49
soreneday: Oh, heh.20:51
soreneday: Nope, I just missed a 0 in my million :)20:51
sorenFor 100,000 instances, it comes out to about 1%.20:51
*** arthurc has quit IRC20:53
sorensleepson-: For 1,000,000 it's probably around 50%, yeah. My pessimistic estimate says 59%, so 50% is probably more accurate.20:53
*** arthurc has joined #openstack20:54
Ryan_Lane:( it isn't a space issue20:54
sorenRyan_Lane: Is it reproducable?20:54
Ryan_Laneit happens every time i try to run an instance :)20:55
Ryan_Laneusing version 2011.1~bzr515-0ubuntu0ppa1~lucid120:55
*** colinnich has joined #openstack20:56
sorenRyan_Lane: Ok, pastebin the output of "file /var/lib/nova/instances/*/*"20:56
Ryan_Lanesoren: http://pastebin.com/L3WjhC5E20:57
*** arthurc has quit IRC20:59
sorenRyan_Lane: No ramdisk?20:59
sorenRyan_Lane: is that intentional?20:59
Ryan_Lanesoren: that's how it comes from uec20:59
*** enigma has joined #openstack20:59
ttxTeam meeting NOW in #openstack-meeting.21:00
*** DigitalFlux has joined #openstack21:07
*** littleidea has joined #openstack21:12
sorensleepson-: Ok, 36%. That's my final offer :)21:16
*** blpiatt has quit IRC21:20
sleepson-soren - heh :) I usually just ballpark bday problems with sqrt(n) for 50% collision probability, but it is closer to 1.18*sqrt(n), so 36% sounds right21:23
*** EdwinGrubbs has quit IRC21:25
sorensleepson-: Oh. I did 100*(1-((2^40)!/((2^40)^1000000*((2^40)-1000000)!)).21:25
sorensleepson-: ...with Stirling approximation for the large factorials.21:26
sorenWhoops, missed a bracket at the end.21:27
*** rlucio has joined #openstack21:28
*** dfg_ has joined #openstack21:29
*** kdsingh has quit IRC21:30
*** MarkAtwood has quit IRC21:34
anotherjesselittleidea / DigitalFlux  - sorry - had a call - want to continue talk about puppet stuff here?21:35
*** EdwinGrubbs has joined #openstack21:36
*** EdwinGrubbs has joined #openstack21:36
*** DigitalFlux has quit IRC21:37
*** bfschott has joined #openstack21:38
*** colinnich has quit IRC21:39
sorensleepson-: http://paste.openstack.org/show/382/ if you care.21:39
*** spectorclan has joined #openstack21:42
* ttx calls it a day21:42
littleideaanotherjesse: let's take to email, I have to do some stuff for a meeting in 20 minutes.21:42
anotherjesselittleidea: sounds good21:43
*** bfschott has left #openstack21:49
*** ArdRigh has joined #openstack21:51
*** masumotok_ has joined #openstack21:51
*** DigitalFlux has joined #openstack21:58
*** sophiap_ has joined #openstack22:01
*** sophiap has quit IRC22:01
*** sophiap_ is now known as sophiap22:01
spectorclanSorry I missed meeting today, had another meeting. If there are any issues for me - feel free to let me know22:03
*** miclorb has joined #openstack22:03
*** MarkAtwood has joined #openstack22:06
*** blpiatt has joined #openstack22:06
*** MarkAtwood has quit IRC22:07
anotherjesseDigitalFlux: I can talk about the existing puppet stuff ...22:10
sleepson-soren - cool - need to brush up on my math a bit though :)22:10
*** DigitalFlux has quit IRC22:11
*** crago has left #openstack22:15
sorensleepson-: :)22:17
* soren heads bedwards22:17
dubsquaredRyan_Lane:  Did you ever find out the issue with this error —> libvir: QEMU error : Domain not found: no domain with matching name 'instance-407726969'22:18
Ryan_Lanedubsquared: nope22:18
dubsquaredI just ran into it...22:18
dubsquared:|22:18
dubsquared10.04 i get that error…10.10…i do not22:18
dubsquared10.10 i am getting this in the nova-compute.log —> NotFound: No instance for id 322:19
dubsquaredbut…it still boots22:19
*** westmaas has quit IRC22:20
rlucioyou should turn on verbose/debug messaging22:21
rluciothe domain errors are just the surface error message22:21
rluciobasically saying that a query to libvirt for a particular VM instance id failed22:22
dubsquaredoh wow…i didn't know that isn't default22:22
dubsquaredlemme do...thx..22:22
rlucionp22:23
rlucioif things were working before, it could be something silly like the partitioning failed because your drive is out of space ro something22:24
rlucioits happened to me a few times :)22:24
dubsquaredthis is my first run with images w/o a ramdisk22:24
dubsquaredthe UEC images...22:24
rlucioah22:24
*** MikeZ1 has quit IRC22:25
rlucioregardless though, it should show some sort of relevant trace in the compute log22:26
*** dendrobates is now known as dendro-afk22:27
dubsquaredand you just use the public key to login i presume?22:28
rluciohuh?  you mean to log into the vm?22:29
edayvishy: https://code.launchpad.net/~eday/nova/internal-api-cleanup/+merge/45152  could use some lovin' when you have a moment :)22:29
*** opengeard has joined #openstack22:29
*** johnpur has quit IRC22:31
*** MikeZ has joined #openstack22:31
dubsquaredYeah22:34
*** Podilarius has quit IRC22:37
*** MarkAtwood has joined #openstack22:41
*** Jordandev has joined #openstack22:44
rluciodubsquared: i honestly dont know how to login to new uec vms, i'd have to look it up.  im sure there are docs on it though on ubuntu's site22:46
*** dendro-afk is now known as dendrobates22:46
dubsquaredrlucio:  what?  documentation?  shenanigans!22:46
rluciohttps://help.ubuntu.com/community/UEC22:48
rluciolooks like there is some web url you are supposed to use to create an account22:48
rlucio /shrug22:49
*** Zlasher has quit IRC22:51
*** kevnfx has joined #openstack22:53
dubsquaredinteresting…22:53
dubsquaredand not in a good way22:53
rluciowell, if you can connect to the vm via serial console while it boots apparently it will print the password somewhere along the line22:55
rlucioand once you set up the vm the way you like it, you can always bundle it up as a new images so you have a built-in acct22:55
*** MikeZ1 has joined #openstack23:03
*** nelson__ has quit IRC23:04
dubsquaredYeah, I was  just reading about that23:04
dubsquaredgreat info..23:04
*** nelson__ has joined #openstack23:04
*** mdomsch has quit IRC23:05
*** anotherjesse has quit IRC23:06
*** MikeZ has quit IRC23:06
*** enigma has left #openstack23:07
*** rnirmal has quit IRC23:12
*** ppetraki has quit IRC23:12
annegentlejust fyi to all - for all the nova docs, I've changed Copyright 2010 to 2010-2011. I'll do the same for swift next. I haven't touched .py files though.23:15
annegentlethis is the branch where the 2010-2011 statements are: https://code.launchpad.net/~annegentle/nova/fixnewscript/+merge/4508523:15
*** gasbakid has quit IRC23:25
*** ctennis has joined #openstack23:26
*** ctennis has joined #openstack23:26
*** daleolds has quit IRC23:32
*** blpiatt has quit IRC23:35
*** dendrobates is now known as dendro-afk23:39
*** spectorclan has quit IRC23:41
*** dendro-afk is now known as dendrobates23:46
*** daleolds has joined #openstack23:47
*** troytoman has left #openstack23:49
*** gondoi has quit IRC23:54
nelson__http://twitter.com/#!/search/Swift%20%26%20Jake%20Gyllenhaal23:57
nelson__apparently a trending topic on twitter. I didn't know Jake was a geek.23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!