Wednesday, 2015-04-08

*** thumpba_ has quit IRC00:04
*** wojdev has joined #openstack-chef00:25
*** tmichael has quit IRC00:33
*** wojdev has quit IRC00:37
*** tmichael has joined #openstack-chef00:46
*** emagana has quit IRC00:46
*** tmichael has quit IRC00:49
*** stevemar has joined #openstack-chef00:57
*** tmichael has joined #openstack-chef01:17
*** tmichael has quit IRC01:19
*** malfaitc has joined #openstack-chef01:42
*** otter768 has joined #openstack-chef01:54
*** otter768 has quit IRC01:59
*** cheeseplus has quit IRC02:02
*** mgagne has quit IRC02:02
*** cheeseplus has joined #openstack-chef02:04
*** mgagne has joined #openstack-chef02:08
*** mgagne is now known as Guest3992602:08
*** jmickle has quit IRC02:09
openstackgerritNeo proposed stackforge/cookbook-openstack-network: Add rabbitmq max_retries and retry_interval in neutron.conf.erb  https://review.openstack.org/17108702:16
*** harlowja is now known as harlowja_away02:22
openstackgerritNeo proposed stackforge/cookbook-openstack-common: Add rabbitmq max_retries and retry_interval attr  https://review.openstack.org/17107402:36
*** tmichael has joined #openstack-chef02:46
*** otter768 has joined #openstack-chef02:48
*** thumpba has joined #openstack-chef02:52
sc`hm. aio_nova fails on creating the admin tenant02:59
j^2Yep that's what we were working on at Chefconf03:08
j^2It looks like a upstream pkg issue at the moment03:09
*** tmichael has joined #openstack-chef03:10
sc`i'm seeing keystone-paste.ini still getting populated with xml03:16
sc`and my keystone is barfing on xml03:16
sc`cookbooks from... about an hour ago03:16
sc`cloned master and i don't see the template having xml03:17
sc`that's... a little weird if i do say so myself03:20
*** malfaitc has left #openstack-chef03:21
sc`yeah... the cookbook is adding xml stuff into keystone-paste.ini. obviously, we don't want that03:25
j^2Yep03:25
sc`rhetorical question, but... how?03:26
sc`whatever berks is getting seems to be what gets applied03:33
sc`and whatever berks is getting is stale03:34
sc`even though it's master?03:34
sc`¿que?03:34
sc`j^2: it's maddening to look for too long03:35
sc`does 'berks vendor' dump stuff into ~/.berkshelf/cookbooks by chance?03:38
*** otter768 has quit IRC03:39
j^2nope03:43
j^2it's in <your_testing_repo>/cookbooks03:44
j^2yeah; sorry i didnt have irccloud on this i wasnt watching my phone. now i am...but yeah even if the berks gets vendored the pkgs arent pulled down till the converge happens03:45
j^2i think03:45
openstackgerritNeo proposed stackforge/cookbook-openstack-common: Add rabbitmq max_retries and retry_interval attr  https://review.openstack.org/17107403:45
sc`hm. i see stuff in my ~/.berkshelf/cookbooks from the last time i played with aio_nova03:45
sc`perhaps it still copies them there temporarily?03:46
j^2yeah but not the pkgs?03:46
j^2wait they do?03:46
j^2i thought only the cookbooks get dl'd03:46
sc`just the cookbooks03:46
j^2yeah03:47
sc`but i don't have the latest master of openstack-identity, thus my keystone-paste.ini gets splatted out with xml... even though i ran from a fresh clone03:47
j^2that what's i thought, the challange is we believe that the openstack-identity pkgs from upstream have been updated causing the....errors because originallly they didnt care but now they do?03:48
j^2something to that effect03:48
sc`i think they finally deprecated xml03:48
j^2yah i don't think keystone cared that xml was in there till the updates in the pkgs03:48
j^2exactly03:48
j^2and now our shit is broken03:48
sc`yeah03:48
j^2so we need to fix it03:48
sc`but this has been fixed in the cookbook master03:48
j^2not sure how though honestly03:48
sc`keystone-paste.ini is all jsonified03:49
sc`but berkshelf won't pull the latest master03:49
sc`at least i think that's what's happening03:49
sc`because my openstack-identity has a template with xml in it, which differs if i clone openstack-identity master from github03:50
sc`can i consider my environment hosed? i've already toasted a couple of macbooks in this endeavor, may as well add a third *cough*03:51
sc`ok... this is messed up03:54
sc`i nuked ~/.berkshelf/cookbooks and openstack-chef-repo/cookbooks03:54
sc`ran chef exec berks_vendor and out popped a keystone-paste.ini with xml03:54
*** pradipta has joined #openstack-chef03:55
sc`that part is very obviously fixed: https://github.com/stackforge/cookbook-openstack-identity/blob/master/templates/default/keystone-paste.ini.erb03:55
sc`so why is berkshelf getting the wrong data?03:55
*** thumpba has quit IRC03:56
sc`maybe it needs a version bump for berks to be happy?03:57
sc`i dunno. "master" for openstack-identity isn't giving me the proper keystone-paste.ini.erb03:57
*** tmichael has quit IRC03:58
sc`yeah... this is the commit berks vendor is getting: https://github.com/stackforge/cookbook-openstack-identity/commit/2f54d6f77c9b1f4ec998d141a10ae13d28dbbd6b04:01
sc`that's kinda old04:01
*** tmichael has joined #openstack-chef04:01
sc`that still doesn't explain anything though04:02
sc`actually, maybe it does04:04
sc`i dunno. that's the first commit that introduced 11.0.004:05
sc`i'm wondering if bumping to 11.0.1 would make berks happy04:05
sc`but how to prove my theory other than just pushing a commit to metadata?04:06
sc`the only reason i'm focusing on the tool is because it's doing Unexpected Things04:11
sc`openstack-compute is from 5 days ago04:14
sc`even though changes have been merged since04:14
sc`i'm retracting my Unexpected Things and replacing it with Really Weird Shit04:14
sc`i think that's enough poking at things. it's delving into sheer madness04:16
sc`would it be safe to say that anyone using the github tag in their berksfile the way openstack-chef-repo is, is silently getting the same behavior?04:18
sc`i think the end solution is just to bump the version for identity and see if that fixes it *shrug*04:20
sc`i'd say a version bump all around for good measure04:21
*** seizadi has joined #openstack-chef04:22
sc`i guess that's a bug in berkshelf?04:28
j^2sorry, got called into something, let me read the backlog04:30
j^2oh dang04:30
j^2berks is pulling old code?04:31
j^2so we might have an inconsistant dependancy tree?04:31
j^2we know we have trouble with our dependancies being out of sync04:32
sc`yep yep04:33
sc`it looks pretty inconsistent to me04:33
j^2there is no good way to keep it all in line04:33
sc`well, berks needs to pull master04:33
sc`not "master"04:33
sc`but master04:33
sc`how/why it's getting such old commit ids, i don't know04:34
j^2yeah that's a problem04:34
j^2can you figure out the verison of that old commit that it is pulling?04:34
sc`i found the one for keystone04:34
j^2in the metadata.rb04:34
sc`the keystone commit is when 11.0.0 was introduced04:34
j^2lovely04:35
j^2that's the push for kilo iirc04:35
sc`but it's not consistent, because another one for compute is more recent04:35
j^2hmm04:36
j^2can you put this in a lanchpad bug report so we can try to track it there? it's way past my bedtime and i think i need to sleep on this04:37
sc`yeah, no worries04:37
sc`it's getting early here too04:37
j^2just through the bug number in here when you got as much info as you can there so we can revisit it tomorrow04:37
sc`i'll put it in lp04:37
j^2:D04:37
j^2i'm also trying to learn libvirt and qemu and integrate it with MAAS as i talk to you plus drinking a good beer, my brain is all over the place04:38
sc`it's all good. i needed to mostly stream of consciousness while i 'unwound' anyway04:39
sc`but the outcome is a bug report, which is a Good Thing04:40
j^2yep04:40
j^2hey man i'm one of the best rubber ducks you'll ever find04:40
j^2you can OH that if you want ;)04:41
sc`i'm gonna gather my thoughts in lp so we can revisit fresh04:43
j^2:)04:43
j^2sounds good04:43
*** thumpba has joined #openstack-chef04:57
*** thumpba has quit IRC05:02
*** thumpba has joined #openstack-chef05:39
*** otter768 has joined #openstack-chef05:40
*** thumpba has quit IRC05:41
*** otter768 has quit IRC05:44
*** wojdev has joined #openstack-chef05:47
*** wojdev has quit IRC05:48
*** emagana has joined #openstack-chef05:50
*** stevemar has quit IRC05:54
*** seizadi has quit IRC05:55
sc`@core https://bugs.launchpad.net/openstack-chef/+bug/1441442 for berkshelf inconsistency (see scrollback)06:07
openstackLaunchpad bug 1441442 in OpenStack + Chef "[all] Berkshelf pulls inconsistent cookbook sets" [Undecided,New]06:07
os-chef-bot@j^2 @markvan @mattray @wenchma @jklare @cmluciano https://bugs.launchpad.net/openstack-chef/+bug/1441442 for berkshelf inconsistency (see scrollback)06:07
*** emagana has quit IRC06:42
*** emagana has joined #openstack-chef06:42
*** welldannit has quit IRC06:47
*** emagana has quit IRC06:47
*** opscode-logger has quit IRC07:00
*** opscode-logger has joined #openstack-chef07:00
*** otter768 has joined #openstack-chef07:41
*** otter768 has quit IRC07:45
*** cheeseplus has quit IRC08:15
*** cheeseplus has joined #openstack-chef08:16
*** pradipta has quit IRC09:30
*** otter768 has joined #openstack-chef09:41
*** otter768 has quit IRC09:47
*** j^2 has quit IRC10:04
*** j^2 has joined #openstack-chef10:06
*** ChanServ sets mode: +o j^210:06
*** rtheis has joined #openstack-chef11:01
*** otter768 has joined #openstack-chef11:42
*** otter768 has quit IRC11:47
*** smccully has joined #openstack-chef12:20
*** neelashah has joined #openstack-chef12:57
sc`i hope my bug makes sense. that was all mostly one eye open :)13:18
*** stevemar has joined #openstack-chef13:19
*** stevemar has quit IRC13:27
*** stevemar has joined #openstack-chef13:28
sc`perhaps it has something to do with berkshelf's cache/scratch directory business it does13:42
sc`like, somehow master is getting cached somewhere?13:42
sc`i dunno, that doesn't sound like it makes sense13:43
*** otter768 has joined #openstack-chef13:43
*** otter768 has quit IRC13:48
*** mattray has joined #openstack-chef13:55
*** ChanServ sets mode: +o mattray13:55
*** chlong has quit IRC13:58
sc`i'm going to go ahead and file a confusing issue against berkshelf since this seems like a not-necessarily-us problem14:00
sc`master should be a snapshot from "right now" instead of last week or a month ago14:00
sc`no matter how the versions are handled in the cookbooks14:01
*** ncerny has joined #openstack-chef14:26
*** stevemar has quit IRC14:34
*** pradipta has joined #openstack-chef14:34
j^2Yep14:43
sc`i dug around in the source code for berks, and it all seems to be shellout calls to the git binary, which adds to the confusion14:45
sc`i suspect it's something with berks' caching14:45
*** emagana has joined #openstack-chef14:46
sc`this is slipping ever further into madness :)14:47
*** emagana_ has joined #openstack-chef14:49
*** emagana has quit IRC14:50
sc`Seth Vargo suggested it's a .lock issue, but i saw this without a preexisting .lock from a fresh clone of chef-repo14:53
sc`which just adds to the madness14:54
sc`messed up. yet another fresh clone and berks is all "haha, made you look. here's the right code"14:56
cmlucianoso it works with a fresh clone?14:57
sc`as of about 3 minutes ago, for now, yes14:57
sc`last night, i was getting commit IDs spanning 1-3 weeks14:57
sc`the tooling is making me out like i'm crying wolf :/14:58
cmlucianothese things happen14:59
j^2sc`: wait aio_nova is working now?15:01
j^2Last attempt with a fresh clone didn't work either15:01
sc`no, it's still broken bitching about saml2 this time15:01
sc`still some xml errors in keystone.log though15:01
j^2Lovely15:01
sc`but /etc/keystone has no references to xml15:02
j^2Hmmm15:02
j^2🚗15:02
sc`a couple of deprecated messages, assertion errors about greenthreads, and saml2 missing is what my keystone.log has, which is most likely the current goings on15:03
*** emagana_ has quit IRC15:05
*** emagana has joined #openstack-chef15:06
sc`apt-get install python-pysaml2 gets me past the admin tenant creation15:09
*** stannie has joined #openstack-chef15:09
sc`glance fails with a 40315:10
sc`but keystone seems to be happy15:10
markvansc`: As I noted above, that saml2 issue is fixed in the ubuntu repo trunk.    I just posted a patch for the glance 403 see https://review.openstack.org/#/c/171330/15:11
sc`ah, see how much i pay attention15:11
sc`i'll pull that patch for glance in15:11
*** stevemar has joined #openstack-chef15:13
*** wojdev has joined #openstack-chef15:15
*** wojdev has quit IRC15:15
markvansc`: saw your bug on Berks....yeah, I'm always fighting the cache it has in ~/.berkshelf/cookbooks     I think one of the issues your seeing is that all our metadata is pointing to => 11.0.0, so for master, almost any commit level resolves that dependency.   And I think that is the right way to handle master.   Else we would be updating the versions for EVERY patch.   I just blast that cache, I wish there was a way to limit the caching15:17
markvanof our cookbooks or the whole cache.15:17
sc`markvan: the weird thing about what i saw last night is that blowing away ~/.berkshelf/cookbooks made no difference15:18
sc`and this morning it spontaneously does the right thing and gives me latest and greatest from github15:18
markvanhumm, when I see that, there's the 2nd cache to worry about, it usually in the current dir somewhere15:19
sc`maybe it's worthwhile to cut versions a little more often and ship to supermarket?15:20
sc`just spitballing15:21
sc`that may not even make sense to do given the velocity of the cookbooks15:21
markvanyeah, it's an interesting topic.  The base openstack tries to have RC milestones/releases, so we could consider something like that, but to make that work, would need to be able to "lock" the repo versions we run against as well15:26
markvancmluciano: have you tried using the trusty-proposed  (truck) for ubuntu aio_nova?  I was going to give that a spin next.15:43
*** neelashah has quit IRC15:43
cmlucianoHave not15:43
openstackgerritMark Vanderwiel proposed stackforge/cookbook-openstack-network: dnsmasq attributes defined in wrong order  https://review.openstack.org/17134215:43
sc`policyfile perhaps? dunno if that would translate to github well15:43
*** otter768 has joined #openstack-chef15:44
*** ozialien has joined #openstack-chef15:44
*** ozialien has quit IRC15:47
*** otter768 has quit IRC15:48
*** neelashah has joined #openstack-chef15:57
*** neelashah1 has joined #openstack-chef15:57
*** ozialien has joined #openstack-chef15:59
*** ozialien has quit IRC16:01
*** neelashah has quit IRC16:01
*** ozialien has joined #openstack-chef16:05
*** ozialien has quit IRC16:06
*** ozialien has joined #openstack-chef16:19
*** welldannit has joined #openstack-chef16:21
*** ozialien has quit IRC16:23
*** jmickle has joined #openstack-chef16:24
markvanok, trying a spin with ubuntu trunk from here: https://launchpad.net/~openstack-ubuntu-testing/+archive/ubuntu/kilo  I think that's the right ppa.16:25
*** jmickle has quit IRC16:31
*** jmickle has joined #openstack-chef16:31
*** harlowja_away is now known as harlowja17:21
*** ozialien has joined #openstack-chef17:22
*** Guest39926 is now known as mgagne17:29
*** mgagne has joined #openstack-chef17:29
*** otter768 has joined #openstack-chef17:45
*** otter768 has quit IRC17:49
*** seizadi has joined #openstack-chef18:06
*** seizadi1 has joined #openstack-chef18:06
*** seizadi has quit IRC18:06
*** seizadi1 has quit IRC18:07
*** seizadi has joined #openstack-chef18:08
*** stannie has quit IRC18:19
*** mattray has quit IRC18:33
openstackgerritMark Vanderwiel proposed stackforge/cookbook-openstack-compute: osapi_compute_link_prefix should not be set by default  https://review.openstack.org/17176818:33
*** stevemar has quit IRC18:48
*** petertr7 has joined #openstack-chef19:12
markvanI tried spin with ubuntu trunk, but too many package issues right now.   Going back to ubuntu kilo and maybe doc the couple of hacks needed to bring up aio_nova, so we can move a bit forward.19:30
cmluciano:P19:31
cmlucianooh truck19:31
markvancmluciano: I have also gotten very close with aio_neutron with our internal repo...it takes a few changes to set up.19:42
*** otter768 has joined #openstack-chef19:46
*** otter768 has quit IRC19:51
cmlucianomarkvan: lol define very close19:52
*** thumpba has joined #openstack-chef19:52
markvanable to do a sucessful converge, but neutron not starting properly, still tweaking attrs for it19:52
cmlucianoyep that sounds like the rabbit hole19:52
markvanyeah, but at least it good to see a converge go thru again...19:53
*** ozialien has quit IRC19:59
*** stevemar has joined #openstack-chef20:08
sc`update on rdo: packages are still broken for c7. they seem to be working on the delorean repo, but haven't made it out of delorean yet20:23
sc`they had their irc meeting this morning and i've been digging through the post-meeting scrollback20:23
*** smccully has quit IRC20:26
*** _joel has joined #openstack-chef20:28
_joelim working to get the stackforge swift install fully working for juno.  it seems that the disk/mount part is not working as expected.  anyone experiencing similar issues?20:31
*** stevemar has quit IRC20:34
*** stevemar has joined #openstack-chef20:34
*** stevemar has quit IRC20:42
*** stevemar has joined #openstack-chef20:42
*** ncerny has quit IRC20:45
*** jmickle has quit IRC20:51
markvan_joel: I did a bunch of cleanup work in master for swift cookbook, maybe do some compares with that.20:52
*** rtheis_ has joined #openstack-chef20:54
*** rtheis has quit IRC20:56
_joelmarkvan: i'll assume you are kramvan1.  ive been going through your edits and trying to apply your changes cd4233a.  were you able to verify that the object nodes are able to mount disks and then publish them to chef?21:01
markvanyup, that part is working with latest cookbook21:02
_joelk.  thats good to know.  i see chef formats the disks but then thats about it.  any suggestions on what to look at next?21:03
*** pradipta has quit IRC21:12
*** ncerny has joined #openstack-chef21:25
*** rtheis_ has quit IRC21:26
*** otter768 has joined #openstack-chef21:47
*** otter768 has quit IRC21:52
markvan_joel:  humm, there was a bug in how the device info was being passed around thru the node attributes.  I had to use a slightly different key to make it unique and findable, so the mount would kick in and then the ring would pick it up.   I that is in the master branch.21:57
markvanj^2: cmluciano sc`  I'm trying hard to get the aio_nova working again.  Here's where I at right now with hacks/workarounds.   It's very close.  http://paste.openstack.org/show/200736/    If you could confirm my findings, then we can push to get that nova bug fixed.21:58
_joel@markvan:  i switched my test swift node from a virtual to bare metal and it picked up the disks just fine.  i was using instances built on top of openstack and attaching volumes and apparently it had some problems.  now my next problem is that it cant find the zone for when generating rings.  it doesnt have the fqdn it looks like.21:59
_joelalso, thanks for the help and for the work on the cookbook cleanup21:59
markvanA couple patches a really needed to get in right now, the one mentioned in the paste, https://review.openstack.org/#/c/171330/  and https://review.openstack.org/#/c/167321/  As we use lvm in our default set, so this missing package is a blocker right now.22:00
*** chlong has joined #openstack-chef22:01
markvan_joel: I added both zone and region to the attr now: https://github.com/stackforge/cookbook-openstack-object-storage/blob/master/attributes/default.rb#L182  not sure if that's what your looking for...22:02
*** seizadi has quit IRC22:05
* markvan heading out...will check back in later tonight22:05
_joel@markvan:  this is an excerpt of the output i'm seeing - maybe this helps clarify:  http://pastebin.com/niyZyMWs22:08
*** ncerny has quit IRC22:11
_joel@markvan: i think i see my problem.  ill have to switch up to your latest edits. thanks again22:13
*** mancdaz has quit IRC22:21
*** mancdaz has joined #openstack-chef22:22
*** seizadi has joined #openstack-chef22:25
*** seizadi has quit IRC22:29
*** stevemar has quit IRC22:30
*** neelashah1 has quit IRC22:38
*** andymccr has quit IRC22:56
*** mancdaz has quit IRC22:57
*** chlong has quit IRC23:05
*** mancdaz has joined #openstack-chef23:14
*** andymccr has joined #openstack-chef23:14
*** emagana has quit IRC23:25
*** kei_yama has joined #openstack-chef23:32
*** seizadi has joined #openstack-chef23:36
*** seizadi has quit IRC23:37
*** seizadi1 has joined #openstack-chef23:37
*** seizadi1 has quit IRC23:41
*** os-chef-bot has quit IRC23:47
*** os-chef-bot has joined #openstack-chef23:47
*** otter768 has joined #openstack-chef23:48
*** otter768 has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!