Wednesday, 2014-03-12

derekhMerge it now before the jenkins gods change their minds https://review.openstack.org/#/c/79631/00:01
openstackgerrittom-howley proposed a change to openstack/tripleo-incubator: Wait for baremetal nodes before creating undercloud stack.  https://review.openstack.org/7863600:02
StevenKderekh: Needs 79750 +A'd first00:02
openstackgerritA change was merged to openstack/tripleo-image-elements: Use upstream/package keystone-paste.ini.  https://review.openstack.org/7975000:03
derekhStevenK: done, was just typing a message on it00:03
derekh1h 55m 23s ran like a hot snot00:03
dividehexanyone know where I can find info on redirecting metadata requests (aka 169.254.169.254) on a baremetal instance to the nova-api-metadata service on the compute host?00:04
rpodolyakadividehex: correct routes to 169.254.169.254 are provided via dhcp00:05
rpodolyakadividehex: what's the problem you have?00:06
dividehexrpodolyaka: i'm not using FlatDHCP only Flat for the network manager00:07
lifelesswow the zbook chews through battery00:07
dividehexI have a baremetal instance that doesn't respond to metadata http requests00:07
openstackgerritA change was merged to openstack/tripleo-image-elements: List filesystem and swift as known glance stores  https://review.openstack.org/7963100:08
rpodolyakadividehex: in tripleo we use neutron for providing pxe configs for baremetal instances00:08
dividehexrpodolyaka: ahh that is good to know.  I'm using nova-network at the moment00:09
lifelessrechecked https://review.openstack.org/#/c/79220/00:09
rpodolyakadividehex: not sure if it's possible to provide host routes for baremetal instances if you don't use neutron00:10
lifelessdividehex: the instance doesn't? whats running on the instance00:10
rbradybnemec: that change must have gotten lost.  patch to fix here: https://review.openstack.org/#/c/79815/00:10
openstackgerritRyan Brady proposed a change to openstack/tripleo-heat-templates: Remove Duplicate Rabbit Parameters  https://review.openstack.org/7981500:10
lifelessrpodolyaka: nova baremetal doesn't work with nova-network unless you use file injection00:10
dividehexrpodolyaka: i suspect that is the case.00:10
lifelessbah00:11
lifelessdividehex: ^00:11
dividehexlifeless: i am using file injection for the network template00:11
dividehexlifeless: I was follow this guide00:11
dividehexhttps://wiki.openstack.org/wiki/Baremetal00:11
dividehexdoing a baremetal proof of concept at work00:12
StevenKlifeless: I've been trying to work how to get a testenv locally to test ensure-test-env. Is elements/tripleo-cd/bin/deploy-testenv the script you're talking about, since that seems hardcoded against the current cloud00:12
lifelessdividehex: you'd be much better off with neutron - you get working ec2 metadata, simpler network setup, and its a configuration other people use ;)00:14
lifelessStevenK: it is the script (and see the readme there for notes on building an image)00:14
lifelessStevenK: it takes parameters00:14
StevenKlifeless: elements/tripleo-cd/README.md has no notes about building an image00:15
dividehexlifeless: thanks.  I'll defnitely go down that road then.  I'm new to openstack and the networking has beeen the most brain warping part of it ;)00:15
*** bauzas has quit IRC00:19
mordredtchaypo: I try to be useful00:24
derekhStevenK: ./diskimage-builder/bin/disk-image-create -a amd64 -o testenv-worker fedora testenv-worker dhcp-all-interfaces hosts modprobe-blacklist selinux-permissive baremetal stackuser00:25
lifelessin smartbuy on the zbook 15 page - lol = 'Please be aware that this model PC has a higher weight compared to all the other models'00:25
derekhStevenK: or ubuntu if your into that kind of thing00:25
*** matsuhashi has joined #tripleo00:26
StevenKderekh: Heh. I've never actually used Fedora00:27
StevenKI switched to Debian before Fedora existed.00:27
derekhStevenK: give it a whirl you might like it, or for ubuntu switch fedora and selinux-permissive for ubuntu00:28
lifelessStevenK: you can boot the image you build as a VM guest to evaluate whether it worked or not00:31
StevenKWhich means I should use the vm element, rather than baremetal?00:32
lifelessyes00:33
* StevenK has ten minutes to wait for the ubuntu tarball00:34
lifelessStevenK: where are you building? [what cloud are you going to test deploy it on ?]00:35
StevenKlifeless: Locally00:35
derekhlifeless: StevenK I think I've been using it successfully on the devtest undercloud as is , you'll want to change the heat template to *-per-env to 1 so it actually creates a test env00:36
lifelessderekh: StevenK isn't a cd-admin yet00:37
derekhlifeless: I meant a local virt unsercloud00:37
derekh*undercloud00:37
lifelessderekh: oh right00:37
lifeless(heh)00:38
StevenKWhich is what I understood given 'devtest undercloud'00:38
lifelesscody-somerville: so you've seen a 512GB SSD on smartbuy? I cannae find it00:38
derekhStevenK: you'll also need to add some extra bm nodes to the undercloud, and change some network/user params so it works against the overcloud00:40
lifelessderekh: why more nodes?00:40
lifelessderekh: oh, I see00:40
derekhlifeless: the overcloud will have used the others, yup00:41
derekhStevenK: the diff I pasted on your review seems to work, but if you have an update tomorrow I'll try it out00:42
StevenKderekh: I already had a change locally, I've adapted parts of your diff as a good idea00:42
derekhlifeless: once we get this nailed down I think I can switch on the new cloud as a new provider in nodepool00:42
StevenKderekh: I can paste a diff if you wish00:42
derekhStevenK: ok, go for it00:43
*** matty_dubs|gone has quit IRC00:43
lifelessderekh: \o/00:43
StevenKderekh: http://paste.openstack.org/show/7320400:44
*** petertoft has joined #tripleo00:46
StevenKdib-run-parts Wed Mar 12 00:48:12 UTC 2014 Running /tmp/in_target.d/install.d/80-modprobe-blacklist00:50
StevenK/tmp/in_target.d/install.d/80-modprobe-blacklist: line 4: DIB_MODPROBE_BLACKLIST: Please set DIB_MODPROBE_BLACKLIST.00:50
StevenKHandy.00:50
*** matty_dubs|gone has joined #tripleo00:50
*** derekh has quit IRC00:50
*** petertoft has quit IRC00:50
*** CaptTofu has joined #tripleo00:51
SpamapSlifeless: did you have any thoughts on the best way to get logs out of the box while we're "Waiting for the overcloud stack to be ready" ?00:52
SpamapSlifeless: I was toying with the idea of starting a thing that would just try and try and try to SSH in and tail logs..00:52
SpamapSlifeless: but then I started to wonder if we shouldn't try to push the user visible logs feature into Heat.00:53
SpamapS\o/00:56
SpamapS2014-03-12 00:00:43.931 | Finished: SUCCESS00:57
SpamapSstupid jenkins page didn't refresh00:57
SpamapSwas like "whoa wait it has been waiting an hour"00:57
*** kiall has quit IRC00:57
*** StevenK has quit IRC00:57
*** sdake has quit IRC00:57
*** bogdando has quit IRC00:57
*** tzumainn has quit IRC00:57
*** dividehex has quit IRC00:57
*** nijaba_ has quit IRC00:57
*** rollerj has quit IRC00:58
*** jrist has quit IRC00:58
*** newell has quit IRC00:58
*** fungi has quit IRC00:58
lifelessSpamapS: logstash00:58
lifelessSpamapS: but more specifically, whats the issue you're looking at - deploys? CI?00:58
*** rollerj_ has joined #tripleo00:58
*** CaptTofu has quit IRC00:58
*** GheRivero has quit IRC00:58
*** ProfFalken has quit IRC00:58
*** nijaba has joined #tripleo00:58
*** tzumainn_ has joined #tripleo00:58
*** newell_ has joined #tripleo00:58
*** dividebin has joined #tripleo00:58
*** jrist has joined #tripleo00:58
*** GheRivero has joined #tripleo00:58
*** openstack has joined #tripleo01:05
SpamapSis there a way to cancel a jenkins run and make it start over from scratch?01:06
SpamapSI was hoping recheck no bug would do that.01:06
StevenKSpamapS: Abandon and restore?01:07
StevenKOr push a new patchset01:07
*** killer_prince has joined #tripleo01:07
lifelessnew changset01:08
SpamapSwell WIP->Ready does not do it01:14
SpamapSbah01:14
* SpamapS hits EOD01:14
lifelessSpamapS: so re waiting01:15
lifelessSpamapS: I think heat event-list is probably 90% of the answer01:15
SpamapSlifeless: agreed and was thinking the same thing01:15
SpamapSlifeless: we can even be smart about it and only show events we haven't shown yet fairly easily01:15
SpamapSbut anyway.. EOD01:16
lifelessSpamapS: meep, I hate this new tz separation thing :)01:18
*** ohadlevy has joined #tripleo01:19
*** mkerrin has joined #tripleo01:19
*** noslzzp has joined #tripleo01:19
*** tomh_ has joined #tripleo01:19
*** tomh_ has quit IRC01:19
*** tomh_ has joined #tripleo01:19
*** tteggel has quit IRC01:22
*** tteggel has joined #tripleo01:23
*** nosnos has joined #tripleo01:39
StevenKlifeless: Would you mind chiming on https://review.openstack.org/#/c/79051/1 about John-Paul's comment?01:50
lifelessStevenK: I disagree with him and have said so now.01:54
lifelessok, lets see if I need more reviewers for https://review.openstack.org/#/c/79220/01:55
lifelessnup01:55
StevenKlifeless: Given our discussions on Friday morning at the sprint, I thought you would.01:56
lifelessStevenK: I mean, I've answered in the review for you01:56
openstackgerritA change was merged to openstack-infra/tripleo-ci: Run overcloud deploy tests against the seed.  https://review.openstack.org/7922001:58
StevenKlifeless: Yes, I saw.01:58
*** matsuhashi has quit IRC01:58
clarkbpeople in here run saucy and maybe use hp virtualrooms. what i386 libs do I need to isntall to make it work?01:59
clarkbwith precise I could easily install ia32-libs and that worked, but they killed it in saucy01:59
lifelessclarkb: FIIK I haven't managed on the zbook14 yet02:00
clarkbldd is no help as it shows it happily linked to a bunch of amd64 stuff02:01
StevenKclarkb: I've not gotten it working either02:02
greghaynesclarkb: for me it 'just worked'02:04
clarkbTo run HPVR on 64 bit versions of Linux some additional 32 bit libraries may be required to be installed. For example: the 32 bit Xlib Libraries. <- so useless02:04
StevenKYeah02:04
*** vkozhukalov has joined #tripleo02:07
clarkbthey do have telephone support I should give them a call >_>02:07
StevenKclarkb: GLWT? :-P02:07
cody-somervilleclarkb: Do you get any debug output or error?02:09
clarkbcody-somerville: nope02:09
clarkbapparently if you run firefox out of a shell you might though02:09
clarkbI should give that a try02:09
*** matsuhashi has joined #tripleo02:17
clarkblifeless: StevenK I got it working installing the deps for precise's ia32-libs-multiarch in saucy (using equivalents where necessary)02:26
clarkblifeless: StevenK I can paste that list somewhere if you are interested02:26
clarkbnot very pretty but it works02:26
lifelessclarkb: put it on hpedia ?02:29
*** killer_prince is now known as lazy_prince02:29
clarkblifeless: StevenK mordred http://paste.openstack.org/show/73210/ good idea will throw it on the wiki when I vpn in tomorrow02:29
lifelessclarkb: https://hpedia.hp.com/wiki/Virtual_rooms (has a similar list already, I'm sure updating it is a good idea)02:30
clarkb~.02:31
clarkbgah02:31
lifelesshaha02:31
*** rcarrillocruz has joined #tripleo02:32
lifelesscody-somerville: that 256GB SSD for my productivity laptop hasn't arrived here or in the CHC office AFAICT; have you had confirmation of it shipping etc?02:33
lifelesscody-somerville: if it *hasn't* shipped, I'd like to cancel it (since I can focus on the zbook for now, and look at a 512GB drive when they become available)02:34
*** vkozhukalov has quit IRC02:36
lifelesscody-somerville: but I can't seem to find it in smartbuy02:36
*** rcarrillocruz has quit IRC02:37
*** rpodolyaka has quit IRC02:38
StevenKHm, I've been waiting >1 month for the RAM and slice battery, awesome.02:45
StevenKlifeless: Is your toci_devtest.sh change picked up by the test environment immeadiately?02:47
openstackgerritA change was merged to openstack/os-cloud-config: Ignore H302 as other TripleO projects  https://review.openstack.org/7814702:51
*** CaptTofu has joined #tripleo02:52
*** rsacharya has joined #tripleo02:53
lifelessStevenK: should be02:56
*** CaptTofu has quit IRC02:57
cody-somervillelifeless: It's currently scheduled to be delivered April 28th.02:58
lifelesscody-somerville: wow02:58
lifelesscody-somerville: thanks for figuring that out!02:59
StevenKcody-somerville: I have an order number for you, too03:01
StevenKcody-somerville: Do you need the PR number from SmartBuy, or the order id?03:01
cody-somervilleStevenK: Order ID please03:02
lifelesscody-somerville: is it worth trying to cancel that SSD?03:02
StevenKcody-somerville: SBB242385003:02
lifelesscody-somerville: april 28th is so far away I can't imagine that its left the factory03:02
StevenKcody-somerville: Was ordered on the 5th of Feb03:02
cody-somervillelifeless: It hasn't. I... don't think it's even been built yet.03:03
cody-somervilleStevenK: Delivered03:03
cody-somervilleToday in fact03:03
StevenKBlink03:03
StevenKGuess I might be driving to Rhodes, then03:03
lifelesscody-somerville: ok lets cancel it then, no point when I have a 256GB in this laptop. (But, would like to figure out how to get a 512GB one when they re available)03:04
lifelessStevenK: check outside your door03:04
cody-somervilleIt was delivered to Rhodes facility.03:04
StevenKlifeless: My address is set to the Rhodes office, and I can't change it03:04
lifelessStevenK: oh, in smartbuy you can change that03:04
cody-somervilleStevenK: You just need to create a new address in the system.03:04
lifelessStevenK: go to smartbuy, preferences, your profile03:04
StevenKI can't see the option to create a new address03:07
StevenKThe drop down in Ship To has HP and Search for more... which is less than useful03:08
lifelessStevenK: search for more03:08
lifelessStevenK: then add new03:08
lifelessStevenK: (its even in the help bar at the top of the screen :))03:08
cody-somervillelifeless: Ok. I filed a ticket to have the SSD order cancelled.03:14
lifelesscody-somerville: thanks03:15
lifelesscody-somerville: I guess we check every month or so for 512GB SSD :)03:16
lifelesscody-somerville: (Specifically for when you can see them, since APJ smartbuy seems crippled)03:16
cody-somervilleI can order you whatever you want. But as you've seen, it's question of when you might get it ;)03:16
lifelesscody-somerville: I ordered a slice battery today, at least, I hope its a slice battry for the zbook.03:17
StevenKHeh03:17
lifelesscody-somerville: can you see 512GB SSD's in smartbuy atm ?03:17
* cody-somerville takes a looksy.03:18
*** rwsu has quit IRC03:19
lifelesshmm03:21
lifeless2014-03-12 03:16:53.775 3296 TRACE nova.compute.manager [instance: d5cfebb9-9c88-4ce4-8b1f-a705842923c8] InstanceDeployFailure: PXE deploy failed for instance d5cfebb9-9c88-4ce4-8b1f-a705842923c803:21
cody-somervillelifeless: Not in the catalogue. I can make a non-catalogue purchase for you. How important is this for you? If you say very important, I can make sure this happens.03:21
lifelesscody-somerville: I'm happy to wait for it to be a catalogue item. Its an optimisation for travel, vs a must-have.03:22
lifelesscody-somerville: I have a 1TB spinning rust external drive I carry my mirrors around on for now.03:22
*** CaptTofu has joined #tripleo03:23
* StevenK disappears to Rhodes03:23
*** matsuhashi has quit IRC03:23
*** petertoft has joined #tripleo03:48
*** rwsu has joined #tripleo03:50
*** petertoft has quit IRC03:53
openstackgerritA change was merged to openstack/tripleo-incubator: Fix baremetal flavor image size  https://review.openstack.org/7934404:04
*** killer_prince has joined #tripleo04:10
lifelessSpamapS: welcome back04:20
StevenKHm, I think this box only contains RAM. Which displeases me.04:21
lifelessyay systemd. 'This makes /etc/fstab and the root= kernel command line option unnecessary'04:21
lifelessby yay, I mean WTF04:21
clarkbhuh04:22
SpamapSlifeless: you didn't hear?04:22
SpamapSlifeless: systemd thinks for you04:22
SpamapSno thinking04:22
SpamapSbad user04:22
* StevenK sobs at Lennart, and systemd.04:22
lifeless'To make this reality, in the 211 release systemd-nspawn also gained support for dissecting disk images and mounting according to this specification. With the new "-i" switch you can now point it to a disk image file '04:22
SpamapSlifeless: and o/ .. just trying to keep the tubes full of changes ;)04:22
StevenKHaha. The box contains two other boxes, each of which contains a 8GiB SODIMM04:22
lifelessso, all the effort that went into making Nova use guestfish etc04:23
StevenKcody-somerville: I've changed my address -- when the second part ships, the slice battery, that will use the new address?04:24
*** CaptTofu has quit IRC04:24
*** matsuhashi has joined #tripleo04:25
*** slagle has quit IRC04:29
*** slagle has joined #tripleo04:30
* StevenK finally manages to deconstruct his laptop enough to get to the SODIMM slots04:30
clarkbStevenK: you have to remove all of those bottom plates iirc04:31
clarkbbecausethey overlap04:31
StevenKclarkb: Er, yes? Since I just said that I've gotten to the SODIMM slots? :_)04:31
lifelessStevenK: no, each order has address encoded in it04:38
lifelessStevenK: I think clarkb was commiserating :)04:38
StevenKAh04:39
* StevenK sighs. Didn't check all the clips on the service cover before I put the hard disk cover on, and only noticed when I went to turn the laptop over04:39
StevenKsteven@vanquished:~% free -m | grep '^Mem'04:43
StevenKMem:         15913       1223      14689          0         42        72904:43
StevenKWoot04:43
lifelesscool04:45
lifelessnow go forth and devtest04:45
StevenKNo, now it's back to working out to plug this image I have into my devtest on my local machine04:47
lifelessStevenK: run cleanup-env04:49
lifelessStevenK: then walk through devtest.sh by hand, but for devtest_testenv build another node over the default count04:49
*** petertoft has joined #tripleo04:50
lifelessStevenK: then call seed, but you can skip undercloud - just register the rest of the nodes04:51
lifelesse.g. by setup-baremetal $TE_DATAFILE --service-host seed --not-first04:51
lifelessfinally devtest_overcloud.sh04:51
lifelessand after that you can use the seed heat to deploy the testenv04:52
lifelessyou'll need to make a network in the overcloud neutron for the testenv to create ports on04:52
lifelessthe default-net can be used for that actually04:52
*** petertoft has quit IRC04:54
StevenKRight, testenv done04:55
*** akuznetsov has joined #tripleo04:56
*** ramishra has joined #tripleo05:09
rbradyif anyone has a chance, I'm looking for reviews of https://review.openstack.org/#/c/74563/05:15
*** mandeep has joined #tripleo05:16
*** mandeep has left #tripleo05:16
*** rbrady is now known as rbrady-afk05:21
*** akuznetsov has quit IRC05:23
*** akuznetsov has joined #tripleo05:27
*** rpodolyaka has joined #tripleo05:31
*** rsacharya has quit IRC05:34
lifelessStevenK: what would be most cool05:38
lifelessStevenK: would be a test job we could run via check experimental or some such that would deploy a testenv itself05:39
lifelessStevenK: so we could CI check this stuff :)05:39
StevenKAgreed05:39
StevenKlifeless: Seed just about up05:40
StevenKlifeless: If the new node is in the JSON, that's bad, we don't want it registered by setup-baremetal ?05:41
lifelessStevenK: you want it registered05:44
lifelessStevenK: since you'll be deploying the testenv as a parallel instance to the overcloud05:44
StevenKBlah, where is the PATH stuff done05:46
*** mandeep has joined #tripleo05:50
*** petertoft has joined #tripleo05:51
lifelessStevenK: _variables05:51
*** vkozhukalov has joined #tripleo05:53
*** petertoft has quit IRC05:55
*** akuznetsov has quit IRC05:56
StevenKTiming out after 600 seconds:05:58
StevenKCOMMAND=nova baremetal-node-create devtest_canary 1 1 1 11:22:33:44:55:6605:58
StevenK:-(05:58
*** akuznetsov has joined #tripleo05:59
*** CaptTofu has joined #tripleo06:00
*** ramishra has quit IRC06:05
lifelessStevenK: hmm, epic06:05
*** rdopieralski has joined #tripleo06:17
*** ramishra has joined #tripleo06:35
*** tzumainn_ is now known as tzumainn06:36
*** ramishra_ has joined #tripleo06:37
*** ramishra has quit IRC06:37
*** akuznets_ has joined #tripleo06:40
*** akuznetsov has quit IRC06:40
*** ramishra_ has quit IRC06:45
*** saju_m has joined #tripleo06:52
*** mandeep has quit IRC06:54
openstackgerritSteve Kowalik proposed a change to openstack-infra/tripleo-ci: Switch to nodes list in the JSON  https://review.openstack.org/7774106:58
*** rpodolyaka has quit IRC07:00
*** pblaho has joined #tripleo07:08
*** mrunge has joined #tripleo07:19
*** AaronGreen has quit IRC07:21
*** AaronGr has joined #tripleo07:23
*** killer_p- has joined #tripleo07:23
*** lazy_prince has quit IRC07:24
*** Qiming has joined #tripleo07:25
*** Qiming has quit IRC07:33
*** CaptTofu has quit IRC07:35
*** gcha has joined #tripleo07:37
*** vkozhukalov has quit IRC07:38
*** rsacharya has joined #tripleo07:44
GheRiveromorning all07:48
*** rcarrillocruz has joined #tripleo07:50
*** rlandy has joined #tripleo07:50
*** petertoft has joined #tripleo07:53
*** jprovazn has joined #tripleo07:54
*** petertoft has quit IRC07:57
*** jcoufal has joined #tripleo07:59
*** bauzas has joined #tripleo08:02
* SpamapS waits patiently as firefox loses its mind08:02
*** ifarkas has joined #tripleo08:05
openstackgerritA change was merged to openstack/tripleo-heat-templates: Swift storage heat templates  https://review.openstack.org/7844608:05
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-heat-templates: Add heat.stack_domain_admin_password  https://review.openstack.org/7801708:11
*** jistr has joined #tripleo08:13
openstackgerritOm Kumar proposed a change to openstack/diskimage-builder: Refactor code to select boot kernel  https://review.openstack.org/7987308:13
SpamapShm interesting failure..08:16
SpamapS2014-03-12 04:01:48.075 | Calling <function virsh_destroy at 0x7f6479a0fa28> with: ['destroy', 'seed_2']08:16
SpamapS2014-03-12 05:52:45.080 | Build timed out (after 125 minutes). Marking the build as failed.08:16
SpamapSwhile testing seed in https://review.openstack.org/#/c/79654/08:17
*** morganfainberg is now known as morganfainberg_Z08:17
*** d0ugal_ has joined #tripleo08:17
SpamapSI wonder if we somehow dead-locked libvirt or something08:17
*** jistr has quit IRC08:20
*** matsuhashi has quit IRC08:21
*** jistr has joined #tripleo08:22
openstackgerritJan Provaznik proposed a change to openstack/diskimage-builder: Adds libmariadb-dev package mapping  https://review.openstack.org/7987408:26
jprovazngreghaynes: hi08:26
*** d0ugal_ is now known as d0ugakl08:36
*** d0ugakl is now known as d0ugal08:37
*** xuhaiwei has joined #tripleo08:37
*** e0ne has joined #tripleo08:40
*** saju_m has quit IRC08:41
*** hashar has joined #tripleo08:41
*** giulivo has joined #tripleo08:41
openstackgerritOm Kumar proposed a change to openstack/diskimage-builder: Refactor code to select boot kernel  https://review.openstack.org/7987308:45
*** ekarlso has quit IRC08:47
*** ekarlso has joined #tripleo08:47
*** e0ne has quit IRC08:50
*** e0ne has joined #tripleo08:56
*** e0ne_ has joined #tripleo08:57
*** athomas has joined #tripleo08:58
*** e0ne has quit IRC09:01
*** petertoft has joined #tripleo09:03
*** hewbrocc` is now known as hewbrocca09:07
*** saju_m has joined #tripleo09:09
openstackgerritA change was merged to openstack/tuskar-ui: Create node profile details page  https://review.openstack.org/7720809:10
*** derekh has joined #tripleo09:12
*** al has quit IRC09:14
*** al has joined #tripleo09:15
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mariadb element  https://review.openstack.org/7988409:16
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-common element  https://review.openstack.org/7988509:16
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Install mariadb devel package if mariadb is used  https://review.openstack.org/7988609:16
*** jcoufal has quit IRC09:20
derekhStevenK: I noticed in the eavesdrop irc logs that, it didn't log my last messages to you, so just incase you missed them09:21
derekhMar 12 00:44:10 <StevenK>derekh: http://paste.openstack.org/show/7320409:21
derekhMar 12 00:48:53 <derekh>StevenK: seems to make sense, the jq vodoo goes over my head09:21
derekhMar 12 00:50:40 <derekh>StevenK: the one thing I did notice today, I had trouble running ci jobs until I bumped the node disk size upto 31G, but I'm not 100% sure if its required once https://review.openstack.org/#/c/79344/ merges09:21
derekhalthough its probably not helpfull anymore09:21
openstackgerritJiri Stransky proposed a change to openstack/tuskar: Using flavors from Roles  https://review.openstack.org/7682609:22
openstackgerritJiri Stransky proposed a change to openstack/tuskar: Fix of Overcloud Update  https://review.openstack.org/7685009:22
SpamapSderekh: careful making flavor changes .. it confuses nodepool09:22
*** jcoufal has joined #tripleo09:22
derekhSpamapS: on the ci-overcloud ? noted09:23
derekhSpamapS: ^^ wasn't a flavor change, it was a test env change09:23
SpamapSderekh: we changed RAM from 8G to 16G and nodepool went "say whua?!" and stopped spinning up nodes until they poked it with a reconfig stick.09:23
SpamapSderekh: OH that size09:24
derekhSpamapS: cool, I wont be doing that so09:24
SpamapSreminds me that we need to fix the dib heuristics so we can keep trying to build the raw image in tmpfs09:24
SpamapSbut it's late09:24
SpamapSand I should try that sleep thing again09:25
derekhSpamapS: as a easy alternative we could remove the git repo's and make the image smaller09:25
derekhSpamapS: night09:26
SpamapSderekh: Yeah, I had a change to make that somewhat intuitive .. got bogged down in "need to switch to yaml"09:26
SpamapSderekh: Note that I've been poking things through CI as I find them.. there are a bunch of things in the queue now... we are at least getting through the backlog now. :)09:27
derekhSpamapS: Cool, will keep and eye on them through the day09:27
*** jrist has quit IRC09:27
* SpamapS out09:28
*** ifarkas has quit IRC09:28
*** akrivoka has joined #tripleo09:29
*** ifarkas has joined #tripleo09:31
*** CaptTofu has joined #tripleo09:32
*** xuhaiwei has quit IRC09:33
*** rsacharya has quit IRC09:34
*** killer_prince has quit IRC09:34
*** killer_p- is now known as killer_prince09:34
*** CaptTofu has quit IRC09:37
*** jrist has joined #tripleo09:40
*** athomas has quit IRC09:43
*** lucasagomes has joined #tripleo09:47
*** lucasagomes has joined #tripleo09:48
*** killer_prince2 has joined #tripleo09:50
*** athomas has joined #tripleo09:51
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Get logs of all instance on seed  https://review.openstack.org/7977109:53
*** hashar has quit IRC09:58
*** pbelanyi has joined #tripleo10:01
*** lucasagomes has quit IRC10:10
*** lucasagomes has joined #tripleo10:10
*** hashar has joined #tripleo10:10
*** jp_at_hp has joined #tripleo10:13
*** xuhaiwei has joined #tripleo10:16
*** martyntaylor has joined #tripleo10:18
*** rlandy has quit IRC10:18
*** andreaf has joined #tripleo10:26
*** rlandy has joined #tripleo10:31
*** rsacharya has joined #tripleo10:44
*** killer_prince2 has quit IRC10:45
openstackgerritA change was merged to openstack/diskimage-builder: Fixup all occurrences of REPONAME for replacing '-'  https://review.openstack.org/7960210:45
*** saju_m has quit IRC10:49
*** killer_prince2 has joined #tripleo10:53
*** xuhaiwei has left #tripleo10:56
openstackgerritA change was merged to openstack/diskimage-builder: Add libaio1 to libaio pkg map for fedora  https://review.openstack.org/7728310:59
openstackgerritA change was merged to openstack/diskimage-builder: Fix unbound variable in debian element  https://review.openstack.org/7581410:59
openstackgerritA change was merged to openstack/tripleo-incubator: Removing unused options from getopts call in boot-seed-vm  https://review.openstack.org/7860011:00
*** spm has joined #tripleo11:01
*** rsacharya has quit IRC11:01
*** AaronGr has quit IRC11:02
*** AaronGr has joined #tripleo11:06
*** jang1 has joined #tripleo11:10
*** morazi has quit IRC11:11
*** dtantsur has quit IRC11:19
*** hashar has quit IRC11:20
openstackgerritOm Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails.  https://review.openstack.org/7928911:21
*** e0ne_ has quit IRC11:22
*** e0ne has joined #tripleo11:23
*** CaptTofu has joined #tripleo11:24
*** e0ne_ has joined #tripleo11:26
*** ChanServ changes topic to "test/gate jobs are queuing now in preparation for gerrit maintenance at 12:00 utc (eta to resume is 12:30 utc)"11:28
*** CaptTofu has quit IRC11:28
*** e0ne_ has quit IRC11:29
*** e0ne has quit IRC11:29
*** e0ne_ has joined #tripleo11:29
*** e0ne_ has quit IRC11:29
*** CaptTofu has joined #tripleo11:38
derekhci seems a lot more stable now, I've seen intermittent errors contacting the testenv (looks like we have 2 intermittent bugs)11:41
derekhhttps://bugs.launchpad.net/tripleo/+bug/129130611:41
derekhhttps://bugs.launchpad.net/tripleo/+bug/129132311:41
derekhwill look into them later11:41
derekhrbrady-afk: ^^ these are the 2 problems you hit with your last 2 runs11:42
*** mkerrin has quit IRC11:53
*** morazi has joined #tripleo11:56
*** rbrady-afk is now known as rbrady11:56
rbradythanks derekh11:56
*** spm has left #tripleo12:00
*** mkerrin has joined #tripleo12:01
rbradyderekh: if you have any review cycles today, I'm looking for reviews of https://review.openstack.org/#/c/74563/.  It's already passed CI...12:01
*** rlandy_ has joined #tripleo12:02
derekhrbrady: will do12:02
*** rlandy has quit IRC12:04
*** ChanServ changes topic to "gerrit on review.openstack.org is down for maintenance (eta to resume is 12:30 utc)"12:07
*** e0ne has joined #tripleo12:13
*** derekh changes topic to " gerrit on review.openstack.org is down for maintenance (eta to resume is 12:30 utc) | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"12:14
*** julim has joined #tripleo12:17
*** dprince has joined #tripleo12:18
*** dtantsur has joined #tripleo12:19
*** ChanServ changes topic to "gerrit on review.openstack.org is down for maintenance (revised eta to resume is 13:00 utc)"12:24
rbradygerrit is down12:25
*** derekh changes topic to " gerrit on review.openstack.org is down for maintenance (revised eta to resume is 13:00 utc) | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"12:25
*** bogdando has quit IRC12:33
*** lucasagomes is now known as lucas-hungry12:36
*** saju_m has joined #tripleo12:37
mariosgerrit down?12:38
boris-42marios savanna -> sahara renaming as I know12:39
mariosboris-42: thx12:39
*** hashar has joined #tripleo12:46
*** CaptTofu has quit IRC12:47
*** CaptTofu has joined #tripleo12:47
*** rlandy__ has joined #tripleo12:50
*** CaptTofu has quit IRC12:52
*** rlandy_ has quit IRC12:53
*** rlandy has joined #tripleo12:53
*** openstackgerrit has quit IRC12:54
*** openstackgerrit has joined #tripleo12:54
*** rlandy__ has quit IRC12:54
*** bogdando has joined #tripleo12:55
*** weshay has quit IRC12:55
*** morazi has quit IRC12:59
*** ramishra has joined #tripleo13:01
*** mrunge has quit IRC13:02
*** rlandy_ has joined #tripleo13:03
*** rlandy has quit IRC13:05
*** akuznets_ has quit IRC13:06
*** lblanchard has joined #tripleo13:06
*** openstackgerrit has quit IRC13:08
*** openstackgerrit has joined #tripleo13:08
*** rcarrillocruz1 has joined #tripleo13:12
*** morazi has joined #tripleo13:12
*** rcarrillocruz has quit IRC13:12
*** rlandy__ has joined #tripleo13:15
*** akuznetsov has joined #tripleo13:15
*** rlandy_ has quit IRC13:18
*** rlandy_ has joined #tripleo13:25
*** jdob has joined #tripleo13:26
*** rlandy__ has quit IRC13:28
killer_prince2seems like gerrit is up again..13:30
openstackgerrittom-howley proposed a change to openstack/tripleo-incubator: Add standalone script to build all images.  https://review.openstack.org/7746213:30
rbradymarios: ping13:30
*** CaptTofu has joined #tripleo13:34
openstackgerritOm Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails.  https://review.openstack.org/7928913:34
Ngmorning13:34
*** rlandy__ has joined #tripleo13:34
*** Qiming has joined #tripleo13:36
*** rlandy_ has quit IRC13:37
*** lucas-hungry is now known as lucasagomes13:37
mariosrbrady: pong13:38
rbradymarios: you have a moment to help me out with tuskar usage?13:39
*** mrunge has joined #tripleo13:39
mariosrbrady: can sure try13:41
*** rlandy__ is now known as rlandy13:42
*** nosnos has quit IRC13:43
rbradymarios: https://github.com/agroup/instack-undercloud/blob/master/scripts/deploy-overcloud-tuskarcli13:44
rbradymarios: ^^ that's the script13:44
rbradymarios: here's overcloud and role list: http://paste.openstack.org/show/73244/13:45
mariosrbrady: ok whats the issue13:46
rbradymarios: heat stack-create fails (notcompute wait condition).  Still looking into this, but wanted to confirm the usage looked right13:47
*** mrunge has quit IRC13:48
*** weshay has joined #tripleo13:48
mariosrbrady: tbh, i use the ui (i.e. setup https://github.com/openstack/tuskar-ui/blob/master/docs/install.rst and use horizon) - so not that experienced with the cli. gimme sec to scan again13:49
hewbroccapblaho: ^^^13:50
hewbroccare cli13:50
mariosalso jistr or lsmola ^^^13:50
*** matsuhashi has joined #tripleo13:50
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Neutron: don't hard code the venv dir  https://review.openstack.org/7662613:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-daemon -i.  https://review.openstack.org/7696313:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-install -i.  https://review.openstack.org/7696413:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Wire in OPENSTACK_VENV_DIR default if set  https://review.openstack.org/7696513:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS...  https://review.openstack.org/7696613:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element  https://review.openstack.org/7696713:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Glance/Cinder/Keystone/Swift: don't hard code venv  https://review.openstack.org/7666613:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Stop using the os-svc-install -n -c options.  https://review.openstack.org/7660413:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Drop the os-svc-install -n -c options.  https://review.openstack.org/7660513:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Link db sync utilities to /usr/local/bin...  https://review.openstack.org/7657913:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Heat/Horizon/Tempest: don't hard code venv dir  https://review.openstack.org/7688813:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Ceilo/Ironic/Tuskar: don't hard code venv dir  https://review.openstack.org/7688713:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Nova: don't hard code the venv dir  https://review.openstack.org/7661913:51
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add -i options to os-svc-install/os-svc-daemon...  https://review.openstack.org/7695213:51
*** geerdest has joined #tripleo13:52
mariosrbrady: this looks awesome man13:52
*** matty_dubs|gone is now known as matty_dubs13:52
mariosrbrady: so once heat says create-complete you go ahead with the config too?13:52
mariosrbrady: where you at summit last week, i don't know if the latest 'overcloud config' work robert was going to work on was discussed at all?13:53
mariosrbrady: do you have a trace of the heat fail?13:53
rbradymarios: I as not there.13:54
* jistr looks at the CLI issue13:56
*** matsuhashi has quit IRC13:56
rbradymarios: looking at the logs now13:56
mariosjistr: no issue, just does https://github.com/openstack/tuskar-ui/blob/master/docs/install.rst look sane enough for trying to bring up overcloud13:56
mariosjistr: e.g. can you see any obvious issues. rbrady is hitting a heat issue13:57
jistrmarios: you meant this link? https://github.com/agroup/instack-undercloud/blob/master/scripts/deploy-overcloud-tuskarcli13:57
mariosjistr: hehe yeah13:58
rbradymarios: we had just before this deployed using heat + tripleo-heat-templates successfully13:58
mariosrbrady: yeah, that's as far as i got tuskar with the ui. but i was running that on top of slagles undercloud-live images (plus the overcloud-cinder-volume)13:58
marioshttps://gist.github.com/marios/cdc82a55ac80e0fac03613:59
mariosrbrady: everything was running but tuskar didn't do the config (your script does it outside of tuskar, e.g. direct call to keystone13:59
jistrrbrady: you'll need double quotes around these, not single quotes https://github.com/agroup/instack-undercloud/blob/master/scripts/deploy-overcloud-tuskarcli#L60-L7014:00
*** matsuhashi has joined #tripleo14:00
jistrrbrady: here is visible that the values didn't get substituted http://paste.openstack.org/show/73244/14:00
rbradyjistr: thanks14:01
jistrnp :)14:01
rbradymarios: here was the stack trace.  going to fix the script and try again.  http://paste.openstack.org/show/73249/14:01
*** e0ne_ has joined #tripleo14:03
*** ftcjeff has joined #tripleo14:03
*** openstack has joined #tripleo14:04
*** e0ne has quit IRC14:05
openstackgerritTrent Geerdes proposed a change to openstack/tripleo-image-elements: Adding more .deb dependencies for pypi-mirror on 13.10  https://review.openstack.org/7995114:07
*** Qiming has quit IRC14:13
derekhany cd-admin's looking for something todo we seem to have lost one of the test env's https://bugs.launchpad.net/tripleo/+bug/1291323 , I killed the testenv-workers so it shouldn't be picking up ci jobs any more, the others can pick up the slack14:14
*** ramishra has quit IRC14:16
*** akuznetsov has joined #tripleo14:21
*** matsuhashi has quit IRC14:23
openstackgerritStuart McLaren proposed a change to openstack/tripleo-image-elements: Glance: Change seed json to use config hierarchy  https://review.openstack.org/7827114:28
openstackgerritStuart McLaren proposed a change to openstack/tripleo-image-elements: Glance: Add configuration hierarchy  https://review.openstack.org/7822614:28
*** e0ne_ has quit IRC14:30
*** matsuhashi has joined #tripleo14:30
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Stop using all ports option w/ neutron-ovs-cleanup  https://review.openstack.org/6993714:32
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Refactor ensure-bridge for persistance  https://review.openstack.org/6991814:32
openstackgerritJay Dobies proposed a change to openstack/python-tuskarclient: Removed unreachable code  https://review.openstack.org/7995514:35
*** e0ne has joined #tripleo14:35
*** ChanServ changes topic to "gerrit on review.openstack.org is down for maintenance (revised eta to resume is 13:00 utc) | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"14:43
*** ChanServ changes topic to "TripleO | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"14:44
openstackgerritA change was merged to openstack/tuskar: Do not use __builtin__ in python3  https://review.openstack.org/7926914:46
openstackgerritStuart McLaren proposed a change to openstack/tripleo-image-elements: Glance: Make image_cache_dir configurable  https://review.openstack.org/7794514:51
*** killer_prince2 is now known as lazy_prince14:52
*** andreaf has quit IRC14:52
openstackgerritDougal Matthews proposed a change to openstack/python-tuskarclient: Remove multivalue attributes and roles from CLI  https://review.openstack.org/7755614:55
*** saju_m has quit IRC15:03
*** bogdando has quit IRC15:03
*** bogdando has joined #tripleo15:03
*** lazy_prince2 has joined #tripleo15:04
openstackgerrittom-howley proposed a change to openstack/tripleo-incubator: Add standalone script to build all images.  https://review.openstack.org/7746215:05
*** lazy_prince has quit IRC15:07
rbradymarios, jistr: ping15:15
jistrrbrady: pong15:15
openstackgerritOm Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails.  https://review.openstack.org/7928915:15
mariosrbrady: pong15:16
rbradymarios, jistr: we changed the quoting, checked perms, ran again.  it's not creating any events related to block storage15:17
rbradyin heat15:17
jistrrbrady: does "heat resource-list overcloud" show a block storage machine among other resources?15:18
*** matsuhashi has quit IRC15:18
rbradyjistr: nope15:18
*** rcarrillocruz has joined #tripleo15:18
*** lazy_prince2 is now known as lazy_prince15:19
rbradymarios, jistr: what needs to be done other than create a role and point it at the right image and baremetal flavor?15:19
mariosrbrady: sec15:19
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Get logs of all instance on seed  https://review.openstack.org/7977115:19
*** rcarrillocruz1 has quit IRC15:20
rbradymarios, jistr: maybe this has something to do with it: https://github.com/openstack/tuskar/blob/master/tuskar/heat/template_tools.py#L2715:21
mariosrbrady: what does virsh list say and what about nova list?15:21
mariosrbrady: are you only getting 1 compute and 1 controller?15:22
jistrrbrady, marios: hmm i'm thinking what you described should be sufficient... Would it be possible that you run a version of tuskar or tripleo-heat-templates that doesn't support block storage yet?15:22
rbradymarios: 1 compute, 1 controller15:22
*** sseago has quit IRC15:22
rbradyjistr: the link I pasted looks like the name has to be "overcloud-cinder-volume"15:23
mariosrbrady: this is what i'm chasing - might be the issue with naming of the overcloud cinder role15:23
rbradymarios: no virsh, this is all baremetal15:23
mariosrbrady: oh nice15:23
*** sseago has joined #tripleo15:24
mariosrbrady: fyi, there is a script that should be run to create the overcloud roles... sec15:25
*** rpodolyaka has joined #tripleo15:25
jistrrbrady: hmm yeah and that's what you have in the script, so that shouldn't be a problem right? https://github.com/agroup/instack-undercloud/blob/master/scripts/deploy-overcloud-tuskarcli#L4915:26
mariosrbrady: in tuskar/tools/initial_data.py15:27
*** martyntaylor has left #tripleo15:27
mariosrbrady: jistr: oh right http://paste.openstack.org/show/73244/ seems that they are created ok15:27
*** ramishra has joined #tripleo15:28
rbradymarios, jistr: thanks.  the commands in the script we're being run manually and the role name was different15:28
mariosjistr: not sure if something changed in UI but i can't change the number of compute vs block-storage... lemme pull from master again15:28
rbradymarios: yeah...the name in that paste for the role is "blockstorage"15:28
mariosjistr: oh. horizon changed a lot in a few days15:30
*** ramishra has quit IRC15:32
jistrrbrady: i'm not sure that the name of the role matters that much (i can't recall matching against role name anywhere, i only recall matching against the image name) but i suppose it's worth a try to sync values with the script that marios mentioned15:33
mariosjistr: lsmola: http://i.imgur.com/EoGyfZ0.png this is what UI looks like for me after git pull on master15:34
mariosjistr: lsmola : i can't change the numbers of nodes for a role15:34
jistrwow15:34
mariosis it just me?15:34
mariosjistr: see it all moved to the left15:34
mariosjistr: but i think ui is broken somewhere15:35
*** giulivo has quit IRC15:35
*** akuznetsov has quit IRC15:35
jistrmarios: i don't have a running/up-to-date UI presently but if UI devs don't have it either i can fire it up and verify. But i suppose you're right that it got broken.15:37
*** jprovazn has quit IRC15:37
*** giulivo has joined #tripleo15:38
*** jprovazn has joined #tripleo15:38
lsmolamarios: yeah15:38
lsmolamarios: you need to first associate the node profiles15:39
lsmolamarios: got me confused too :-D15:39
*** jprovazn has quit IRC15:39
lsmolamarios: and yeah, the layout has been changed15:39
lsmolamarios: and there are few UI bugs :-D15:40
rbradymarios, jistr: mystery solved...we have an older version of tuskar installed15:40
lsmolarbrady: ach :-)15:40
*** rpodolyaka has quit IRC15:40
jistryeah i thought it's going to be old tuskar or old t-h-t :)15:41
mariosrbrady: great15:46
rbradymarios, jistr: here's the source of the packages http://repos.fedorapeople.org/repos/openstack-m/openstack-m/fedora-20/noarch/.  Do you know who built these?15:46
rlandymarios: jistr: the package installed was built on 02/28 ... the change we need went in on 02/2015:51
openstackgerritDougal Matthews proposed a change to openstack/python-tuskarclient: Remove multivalue attributes and roles from CLI  https://review.openstack.org/7755615:51
openstackgerritJon-Paul Sullivan proposed a change to openstack/tripleo-incubator: Add source-repository element var setting script  https://review.openstack.org/7439315:52
jistrrbrady: not sure, but athomas asked me and pblaho some questions about packaging tuskarclient some time ago, so he might know15:54
*** lazy_prince has quit IRC15:55
jistrrlandy: i guess the package uses Tuskar from pypi https://pypi.python.org/pypi/tuskar15:55
*** matty_dubs is now known as matty_dubs|lunch15:56
jistrrlandy: if i'm correct about that, then it's more important in what version was the change released rather than when it went to master, and also how that compares with what version is in the package15:57
openstackgerritA change was merged to openstack/tripleo-image-elements: Stop using the os-svc-install -n -c options.  https://review.openstack.org/7660415:57
*** rpodolyaka has joined #tripleo15:58
rlandyjistr: correct - all I meant to point out is that it was the latest package available here: http://repos.fedorapeople.org/repos/openstack-m/openstack-m/fedora-20/noarch/15:58
openstackgerritA change was merged to openstack/tripleo-image-elements: Drop the os-svc-install -n -c options.  https://review.openstack.org/7660515:59
*** cwolferh has joined #tripleo15:59
*** rpodolyaka has quit IRC16:00
jistryeah the package is named openstack-tuskar-0.0.8 while in pypi there's 0.2.1, so that doesn't sound up to date :-/16:00
*** rpodolyaka has joined #tripleo16:00
*** CaptTofu has quit IRC16:00
*** spzala has quit IRC16:00
*** pblaho has quit IRC16:00
*** CaptTofu has joined #tripleo16:01
*** akuznetsov has joined #tripleo16:03
*** CaptTofu has quit IRC16:05
*** rbrady is now known as rbrady-afk16:13
*** jistr has quit IRC16:14
*** vkozhukalov has joined #tripleo16:18
*** lazy_prince has joined #tripleo16:18
lazy_princepRiya2.rahul116:18
*** lazy_prince has quit IRC16:19
*** pbelanyi has quit IRC16:22
*** akrivoka has quit IRC16:23
openstackgerritA change was merged to openstack/tripleo-heat-templates: Remove Duplicate Rabbit Parameters  https://review.openstack.org/7981516:24
*** rpodolyaka has quit IRC16:27
*** rpodolyaka has joined #tripleo16:29
*** CaptTofu has joined #tripleo16:30
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Openstack-clients: don't hard code venv  https://review.openstack.org/7998816:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv  https://review.openstack.org/7998916:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-daemon -i.  https://review.openstack.org/7696316:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-install -i.  https://review.openstack.org/7696416:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Wire in OPENSTACK_VENV_DIR default if set  https://review.openstack.org/7696516:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS...  https://review.openstack.org/7696616:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Heat/Horizon/Tempest: don't hard code venv dir  https://review.openstack.org/7688816:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element  https://review.openstack.org/7696716:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Ceilo/Ironic/Tuskar: don't hard code venv dir  https://review.openstack.org/7688716:33
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add -i options to os-svc-install/os-svc-daemon...  https://review.openstack.org/7695216:33
*** jang1 has quit IRC16:40
*** matty_dubs|lunch is now known as matty_dubs16:42
*** rdopieralski has quit IRC16:42
*** rbrady-afk is now known as rbrady16:48
* derekh is hosting a race with two PXE servers, the one I don't want keeps winning16:48
*** sseago has quit IRC16:49
*** rcarrillocruz1 has joined #tripleo16:51
*** rcarrillocruz has quit IRC16:53
*** e0ne has quit IRC16:54
mordredderekh: fun race16:54
*** bauzas has quit IRC16:58
*** cody-somerville has quit IRC16:58
*** cody-somerville has joined #tripleo16:59
*** cody-somerville has quit IRC16:59
*** cody-somerville has joined #tripleo16:59
*** sseago has joined #tripleo17:02
*** howleyt has joined #tripleo17:02
* SpamapS awakens17:03
*** cody-somerville has quit IRC17:03
*** cody-somerville_ has joined #tripleo17:03
*** cody-somerville_ has joined #tripleo17:03
*** sballe has joined #tripleo17:03
openstackgerritA change was merged to openstack/tripleo-incubator: add check to create ~/.ssh/ to install-dependencies  https://review.openstack.org/7744517:05
*** rsacharya has joined #tripleo17:07
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Adds NFS backend support for cinder-volume  https://review.openstack.org/7456317:08
*** sballe has quit IRC17:10
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Adds NFS backend support for cinder-volume  https://review.openstack.org/7456317:11
*** andreaf has joined #tripleo17:11
*** rollerj_ has quit IRC17:11
*** rollerj_ has joined #tripleo17:11
*** morganfainberg_Z is now known as morganfainberg17:13
lifelesso/17:13
lifelessderekh: why sucsuch a race?17:13
*** AaronGr has left #tripleo17:14
derekhlifeless: problems with one of the testenv hosts https://bugs.launchpad.net/tripleo/+bug/129132317:15
derekhlifeless: so I eventually bounced it, now its getting dhcp from the overcloud17:16
*** saurabhs has joined #tripleo17:16
derekhlifeless: I'm now trying to figure out if I can disable dhcp on the port on the overcloud but don't look like its possible with neutron client17:17
lifelessderekh: so this is the problem: https://bugs.launchpad.net/tripleo/+bug/127280317:17
derekhlifeless: yup, trying to figure out a way around it17:17
lifelessderekh: you need an internal port with unique mac to be the mac for one of the dhcp servers, and to dhcp that port too17:18
lifelessI may not have explained that well enough in the bug17:18
lifelessderekh: if you're fixing in situ, kill dhcp, use ip addr to set both adresses :)17:19
*** bauzas has joined #tripleo17:19
derekhlifeless: yup, I get all that, my problem at the moment is I can't boot17:19
derekhlifeless: because the overcloud dhcp keeps winning the race, I could just delete the port on the overcloud but I was trying to avoid that17:20
derekhlifeless: also we need to rebuild our test envs soon with a new geard, or restart the test-env workers lsof | wc -l is returning 792912 on testenv417:26
SpamapSDOHHHH17:26
lifelessderekh: oh right, so probably delete the port for it in the overcloud17:26
lifelessdis the right thing to do17:26
lifelessderekh: is the right think g to do17:27
lifelessmmm 630MBps sustained IO in my server. meep :)17:27
derekhlifeless: ya, I really wanted to figure this out without doing that but its probably sunk enough time now17:27
SpamapS2014-03-12 11:27:30.661 | Calling <function virsh_start at 0x7fa931bdbc80> with: ['start', 'seed_4']17:28
SpamapS2014-03-12 11:27:30.698 | error: failed to connect to the hypervisor17:29
derekhSpamapS: https://bugs.launchpad.net/tripleo/+bug/129132317:29
SpamapSderekh: I wonder if that is related to geard problems?17:30
SpamapShmm..17:31
SpamapSgoogling suggests mutex problems17:31
SpamapSwhich I suspect is a libvirt issue17:31
*** jcoufal has quit IRC17:31
*** rsacharya has quit IRC17:31
derekhSpamapS: I suspected ports and pipes being left open by gear at first but now now so sure, killing the python processes (which freed them up) didn't help17:33
*** dprince has quit IRC17:34
derekhSpamapS: if it is a problem triggered by gear then expect another one to go soon, they would die in the same order in which they registered with gear17:35
lifelessderekh: well/win 6117:36
derekhlifeless: ??17:37
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296917:38
*** openstackstatus has quit IRC17:38
*** openstackstatus has joined #tripleo17:39
lifelessbah17:40
lifelessderekh: nvm :)17:40
derekhStevenK: I never got a chance to try out your patch properly today, First time I tried it I hit a problem with absense of jq, https://review.openstack.org/#/c/7966317:43
*** geerdest has quit IRC17:43
derekhStevenK: then I got distracted with testenv problems, it looks ok, just want to bring up the testenvs on new cloud using it and verify everything befor I add +217:44
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Add RHEL 7 image element  https://review.openstack.org/7414617:45
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Add redhat-common element  https://review.openstack.org/7486617:45
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Use redhat-common in fedora element  https://review.openstack.org/7486717:45
*** geerdest has joined #tripleo17:46
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296917:51
*** rpodolyaka has quit IRC17:52
*** hashar has quit IRC17:54
*** rpodolyaka has joined #tripleo17:56
*** rpodolyaka has quit IRC17:57
*** athomas has quit IRC17:57
*** rlandy_ has joined #tripleo17:57
lifelessderekh: have you poked at the testenv vm writeback mode yet ?17:58
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Use os-svc-restart in boot-stack reset-db  https://review.openstack.org/7803117:58
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Add mysql-mariadb-compat element  https://review.openstack.org/7803417:58
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Add cinder-lio element  https://review.openstack.org/7846317:58
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Don't install percona-toolkit if distro is RHEL7  https://review.openstack.org/7803217:58
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Factor out tgt-specific parts of cinder element  https://review.openstack.org/7846217:58
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Allow os-svc-daemon to specify dir name with -d option  https://review.openstack.org/7803317:58
derekhlifeless: testenv8 is back up, I'll resetup the test envs later and register the workers so its doing something17:58
lifelessderekh: just a crazy note for you17:58
*** vkozhukalov has quit IRC17:58
lifelessderekh: if you delete the ports for it on overcloud neutron then issue a nove rebuild on the undercloud, it should just DTRT17:59
derekhlifeless: yes, I have, using the unsafe mode on ci overcloud testenvs (with images cached), it runs about 5 minutes faster17:59
derekhlifeless: cool, I'll try that, thanks18:00
lifelessderekh: is there a patchset up? I'm considering poking my local setup is all ....18:00
derekhlifeless: actually I'll leave it as is for a bit incase we need to get debug info off it18:00
SpamapSunsafe mode?18:00
* SpamapS want18:00
SpamapSsafety is for wussies18:00
*** rlandy has quit IRC18:00
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv  https://review.openstack.org/7998918:00
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element  https://review.openstack.org/7696718:00
*** rpodolyaka has joined #tripleo18:01
openstackgerritA change was merged to openstack/tripleo-heat-templates: Merge Block Storage into Overcloud  https://review.openstack.org/7965418:01
derekhlifeless: SpamapS https://review.openstack.org/#/c/79663 it worked faster on a ci test env setup no doubt about that several runs compared, (about 5 minutes on overcloud job, every minute counts), I wanted to test on local devtest before pushing it up.18:03
derekhlifeless: SpamapS  wronge link http://paste.openstack.org/show/73277/18:04
derekhok, gotta run18:05
lifelessderekh: up it comes....18:05
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Switch to unsafe cache mode for test VMs.  https://review.openstack.org/8003718:06
derekhlifeless: :-), as usual we'll have to have to rebuild test envs for it to take effect :-(18:07
*** derekh has quit IRC18:07
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Delete obsolete template.  https://review.openstack.org/8003918:09
rbradyderekh, jan: https://review.openstack.org/#/c/74563/18:16
openstackgerritBen Nemec proposed a change to openstack/os-cloud-config: Switch from oslo.sphinx to oslosphinx  https://review.openstack.org/8004218:17
*** lucasagomes is now known as lucas-afk18:18
*** rpodolyaka has quit IRC18:20
*** jistr has joined #tripleo18:22
*** rpodolyaka has joined #tripleo18:23
*** sandeepr has joined #tripleo18:26
*** killer_prince is now known as lazy_prince18:26
*** sandeepr has left #tripleo18:27
openstackgerritA change was merged to openstack/python-tuskarclient: Remove multivalue attributes and roles from CLI  https://review.openstack.org/7755618:27
*** cwolferh has quit IRC18:27
*** cwolferh has joined #tripleo18:29
*** e0ne has joined #tripleo18:29
*** cody-somerville_ is now known as cody-somerville18:39
*** rlandy__ has joined #tripleo18:39
*** sdake has quit IRC18:41
*** petertoft has quit IRC18:41
*** sdake_ has quit IRC18:42
*** rlandy_ has quit IRC18:43
*** sdake_ has joined #tripleo18:44
*** jistr has quit IRC18:44
*** sdake has joined #tripleo18:45
*** andreaf has quit IRC18:45
*** weshay has quit IRC18:45
*** rpodolyaka has quit IRC18:47
*** e0ne has quit IRC18:49
*** morazi is now known as morazi-mtg18:49
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296918:53
openstackgerritJay Dobies proposed a change to openstack/python-tuskarclient: Changed role count assignment to use role name  https://review.openstack.org/8005218:58
*** jb11211 has joined #tripleo18:58
*** e0ne has joined #tripleo19:00
*** weshay has joined #tripleo19:01
*** rpodolyaka has joined #tripleo19:05
openstackgerritA change was merged to openstack/tuskar: Updates CONTRIBUTING docs sort bug in fedora 20  https://review.openstack.org/7960919:06
adam_ggreghaynes, around?19:08
*** Lotus907efi has joined #tripleo19:14
*** Lotus907efi has quit IRC19:14
*** jprovazn has joined #tripleo19:17
openstackgerritBen Nemec proposed a change to openstack/tuskar: Remove tools/flakes.py  https://review.openstack.org/7978319:25
greghaynesadam_g: lunching, will be in about 1019:27
*** rpodolyaka has quit IRC19:31
*** rpodolyaka has joined #tripleo19:31
greghaynesadam_g: here now, whats up19:33
adam_ggreghaynes, i think im good now actually. was hitting something that sounded similar to what you hit last week: https://bugs.launchpad.net/diskimage-builder/+bug/129153319:35
adam_gjust needed to resolve differences in how devstack and tripleO sets up its tftp server19:36
greghaynesah19:36
*** geerdest has quit IRC19:42
jprovazngreghaynes, hi, did you have any luck yesterday with reproducing the memory issue with xtradb?19:42
greghaynesjprovazn: Hey. Yes I was able to reproduce. Did some looking around for fixes but nothing stuck out obviously - just email threads from people complaining about galera using a lot of mem19:43
greghaynesWas going to try and use some of our local mysql knowledge for possible fixes19:44
jprovazngreghaynes, ah, thanks, will look at it now too19:45
greghaynesok19:45
rbradyjprovazn: https://review.openstack.org/#/c/74563/ (same as when you approved earlier, now with more Readme.md)19:45
jprovaznrbrady, looking19:47
rbradyjprovazn: thanks19:48
*** akuznetsov has quit IRC19:48
openstackgerritA change was merged to openstack/tuskar: Create stack in right order  https://review.openstack.org/7626719:49
*** saurabhs has quit IRC19:52
*** Lotus907efi has joined #tripleo19:52
*** Lotus907efi has quit IRC19:53
*** Lotus907efi has joined #tripleo19:53
*** jp_at_hp has quit IRC19:54
*** jang1 has joined #tripleo19:59
*** weshay has quit IRC20:12
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Adds NFS backend support for cinder-volume  https://review.openstack.org/7456320:16
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296920:18
*** saurabhs has joined #tripleo20:20
*** dprince has joined #tripleo20:20
*** rpodolyaka has quit IRC20:20
*** rpodolyaka has joined #tripleo20:21
*** rpodolyaka has quit IRC20:21
*** rpodolyaka1 has joined #tripleo20:21
bnemechttp://logs.openstack.org/66/74866/8/check-tripleo/check-tripleo-overcloud-precise/aa490f1/console.html#_2014-03-12_20_01_46_50920:22
bnemecIs ^ any kind of known issue or should I open a bug?20:22
*** Slower_ is now known as Slower20:22
*** geerdest has joined #tripleo20:23
greghaynesso pidgin is definitely they way to go for Lync... that web interface and firefox on linux seem to have a hard time getting along20:24
greghaynesway too much time spent learning that lesson20:25
*** morazi-mtg is now known as morazi20:29
openstackgerritA change was merged to openstack/tripleo-image-elements: Use os-svc-restart in boot-stack reset-db  https://review.openstack.org/7803120:33
*** e0ne has quit IRC20:36
lifelessNg: SpamapS: slagle: https://review.openstack.org/#/c/80037/ \o/ passed CI20:36
*** e0ne has joined #tripleo20:36
lifelessgreghaynes: does pidgin need a VPN connection? and does it do voice for Lync?20:37
slaglenice, now maybe virtio is next :)20:37
SpamapSlifeless: boom goes the dynamite20:37
greghayneslifeless: I think it needs vpn, no idea about voice20:37
Ngdoh, too late :)20:37
lifelessgotta get in early on the approve train20:38
dprincelifeless: how much does 80037 speed us up?20:38
Ngnothing wrong with a +6 ;)20:38
Ngdprince: commit message says 5m20:38
lifelessdprince: 5m per run20:38
lifelessNg: if you're looking for a thing... https://review.openstack.org/#/c/80039/20:38
dprincelifeless: with the common venv thing I've seen as much as 9-10 minutes shaved off.20:39
dprincewe should do them both IMO20:39
*** e0ne has quit IRC20:40
lifelessdprince: sure; I was going to ask if you'd started on the nova feature to control vm cache mode via Nova20:40
*** funzo has quit IRC20:40
*** morazi has quit IRC20:41
*** funzo has joined #tripleo20:41
dprincelifeless: I have not. Although I could probably look into it if we want it20:41
*** morazi has joined #tripleo20:41
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296920:42
*** e0ne has joined #tripleo20:42
lifelessdprince: we don't have stats yet, but I am concerned about sync contention on the nova compute hosts20:44
openstackgerritA change was merged to openstack/tripleo-incubator: Switch to unsafe cache mode for test VMs.  https://review.openstack.org/8003720:44
dprincelifeless: I think this is just one of those things you just have to try... and see what happens20:45
SpamapSlike spaghetti.. just have to throw it at the cabinet door and see if it sticks20:48
lifelessSpamapS: speaking of spaghetti hows graceful heat ?20:49
greghaynesspaghetti merge, one of the lesser known git merge strategies20:49
SpamapSlifeless: -> #openstack-meeting20:50
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: baremetal element: Enable stable interface names  https://review.openstack.org/7200320:51
*** morazi has quit IRC20:56
*** morazi has joined #tripleo20:58
*** saurabhs has quit IRC20:59
SpamapSlifeless: so to answer your question directly: Now that dust has settled and softwareconfig and deployers are landed in Heat, I can try adding a graceful rebuilding server subclass that leverages that code.21:00
lifelessSpamapS: Do It Do It Do it Do It Do It21:01
*** vkozhukalov has joined #tripleo21:02
*** saurabhs has joined #tripleo21:02
SpamapSlifeless: still terrified of UPDATE_FAILED .. so I want to push my spike on that just a little further.21:04
*** rlandy_ has joined #tripleo21:04
lifelessSpamapS: I'm going to beg.21:04
lifelessSpamapS: without graceful we cannot complete the HA work.21:04
SpamapSlifeless: here's my point.. tripleo-cd would be a great place to prove this graceful stuff works. _IF_ we didn't have to delete the stack every 8 hours.21:04
*** rpodolyaka1 has quit IRC21:05
lifelessSpamapS: update failed doesn't block other work.. graceful does.21:05
*** matty_dubs is now known as matty_dubs|gone21:05
lifelessSpamapS: doing update failed first blocks the rest of the team21:05
lifelessSpamapS: and that said, its up to you :)21:05
SpamapSlifeless: well.. it makes availability worse for compute.. right? You could just deal with nodes being gone for a while.21:05
SpamapSHm actually Galera would crap out21:06
SpamapSok n/m I'm just whining21:06
SpamapSfine..21:06
* SpamapS shifts gears from "RETRY" to "GRACEFUL"21:06
SpamapS:-/21:06
*** rlandy__ has quit IRC21:07
*** rpodolyaka has joined #tripleo21:09
greghaynesSpamapS: You see our issue with galera taking *all* the rams on the seed box? Wondering if you might have pointers on where we might be able to fix21:09
*** geerdest has quit IRC21:09
*** weshay has joined #tripleo21:09
SpamapSgreghaynes: no, pointer?21:09
greghayneswell, my google search was extremely unhelpful... I can dig deeper just wondering if you knew of anything offhand21:10
*** dprince has quit IRC21:11
lifelessSpamapS: VMs get interrupted if we don't migrate them first, is the issue.21:12
*** rlandy__ has joined #tripleo21:13
*** jdob has quit IRC21:13
*** rlandy_ has quit IRC21:15
lifeless      [===================>.]  reshape = 99.8% (2924957696/2929556480) finish=1.7min speed=42526K/sec21:16
*** e0ne has quit IRC21:16
lifelessnearly done21:16
*** e0ne has joined #tripleo21:17
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Move install type enablement out of source-repositories  https://review.openstack.org/7879421:22
*** rlandy__ is now known as rlandy21:24
*** cody-somerville- has joined #tripleo21:26
*** rcarrillocruz has joined #tripleo21:28
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: os-*-config package install support  https://review.openstack.org/7881621:28
*** rcarrillocruz1 has quit IRC21:29
lifelesslol21:31
lifelessresize2fs: New size too large to be expressed in 32 bits21:31
lifelessaka 'stb'21:32
lifelesserm21:32
lifeless'stab'21:32
* lifeless settles for 15T21:34
*** rbrady is now known as rbrady-afk21:35
greghayneswah21:36
greghaynesThats a lot of bytes21:36
*** saurabhs has quit IRC21:37
SpamapSlifeless: time for XFS :)21:37
lifelessold_desc_blocks = 524, new_desc_blocks = 96021:38
lifelessThe filesystem on /dev/lifelesswks-vg/lifelesswks-lvs is now 4026531840 blocks long.21:38
SpamapSyou just need a second 15TB array to use as an intermediary ;)21:38
lifelessSpamapS: I might play with it with the the spare 3TB at the end21:38
lifelessSpamapS: nah, I have /dev/mapper/lifelesswks--vg-lifelesswks--lvs   15T  7.1T  7.2T  50% /21:38
SpamapSlifeless: big drawback in your case is that XFS cann't be shrunk. :-/21:38
lifelessSpamapS: ted t'so gets sad when folk shrunk ext4 ;)21:39
clarkblifeless: what make 7TB of data?21:39
clarkbs/make/made/21:39
clarkbI mean that is pretty impressive21:39
clarkbyou could host almost 6 months of our build logs in the remaining 50% >_>21:40
lifelessclarkb: its miy digital history since circa 199321:40
clarkbnice21:40
lifelessclarkb: photos, films, code, mail, everything21:41
lifelessclarkb: plus Lynnes21:41
*** d0ugal has quit IRC21:41
lifelessclarkb: plus backups of our DVD collection, etc.21:41
lifelessclarkb: WD RED drives are awesome.21:41
*** saurabhs has joined #tripleo21:41
greghaynesWhat do you house them in?21:41
tchaypomorning21:42
tchaypo+21:44
tchaypo[2~21:44
tchaypo++++++++++++++++++++21:44
tchaypo~21:44
tchaypo[6~[6~21:44
*** CaptTofu has quit IRC21:44
lifeless  VG             #PV #LV #SN Attr   VSize  VFree21:45
lifeless  lifelesswks-vg   2   1   0 wz--n- 18.19t 3.19t21:45
lifelessgreghaynes: http://www.nanoxia-world.com/product/1/55/Deep+Silence+6+21:45
lifelessgreghaynes: amazingly quiet case21:46
greghaynesooo. About to say - I have a box with a bunch of HD's but the noise makes me not want to leave it on21:46
lifelessnuts, I've got a zombie stack that I can't delete :(21:47
lifelessnova vms are gone21:47
lifelessbut something else is quirky21:47
greghaynesYep, thats what I ran into yesterday21:48
greghayneshttps://bugs.launchpad.net/tripleo/+bug/129106021:48
lifelessgreghaynes: I've 8 drives, and a good overpowered quiet PSU, + a i7 K in there with an MAXIMUS VI EXTREME M/B and 16GB of ram21:48
*** jprovazn has quit IRC21:48
lifelessoh, and a 2nd hand mellanox IB card :)21:49
greghaynesooo21:49
lifelessactually, I lie, 32GB of memory21:50
greghayneshah, necessary for the file server21:50
greghaynesOh, how do people hipchat - is there some special magic of how we should sign up?21:51
lifelessgreghaynes: ping collette if you don't have a hipchat account21:51
*** rlandy has quit IRC21:51
lifelessgreghaynes: so when we have Ironic support21:51
*** rpodolyaka has quit IRC21:52
lifelessgreghaynes: I plan to distribute my test VMs across my home workstation (i7, 16GB, 4 disks in RAID10, 2x work laptops and the beast server)21:53
lifelessgreghaynes: I figure that that should give fairly good performance21:53
greghaynesaha21:53
greghaynesI had two smallish 1u's with 32gb ram and raid0 ssd's.. im hoping to play with some baremetal deploying on them in the very near future21:54
lifelesshad ?21:54
greghaynesalso a silly high clockspeed quad core xeon. Used to be build boxen21:55
greghayneswell, have. Need to install one back into my little mini rack21:56
*** rpodolyaka has joined #tripleo21:56
*** rpodolyaka has quit IRC22:03
lifelessSpamapS: did you eyeball https://bugs.launchpad.net/tripleo/+bug/1291060 ?22:03
lifelessgreghaynes: why didn't you triage  ^ ?22:03
greghaynesWas hoping for a confirmation. *triages now*22:04
greghaynescritical, correct?22:06
lifelessI rather think so22:08
lifelesssince deleting your DB to delet a stack is... bad22:08
*** lblanchard has quit IRC22:09
SpamapSlooks like a heat bug22:09
SpamapSdelete should basically never fail22:10
lifelessSpamapS: I have added heat to the bug22:10
lifelessSpamapS: on that bassis22:10
SpamapSlifeless: yeah, I"m looking at the trace now to see why that 404 isn't handled as "yay it is already gone"22:10
*** rpodolyaka has joined #tripleo22:10
SpamapSlifeless: might become moot for us, as softwareconfig will alleviate us of having to create users and keys and policies22:10
lifelessSpamapS: I'm going to suggest greghaynes dive deep into it22:11
SpamapSand thats what I'm playing with right now.. :)22:11
lifelessSpamapS: since he has it happening and more folk with heat codebase familiarity++22:11
greghaynesSounds good22:11
SpamapSgreghaynes: see my suggestion above.. look at how the user resource handles the same type of error.22:11
* greghaynes hopes his email quiets down a bit for this22:12
* SpamapS back to building a seed to poke graceful things into22:12
SpamapShmm22:12
SpamapSthat sounded ... inappropriate22:12
greghaynesindeed22:12
lifelessSpamapS: 'graceful' things. hah.22:14
SpamapSgreghaynes: I usually quiet my email program down by exitting it :)22:16
greghaynesI like this idea!22:17
lifelessWAT22:19
*** e0ne has quit IRC22:25
*** e0ne has joined #tripleo22:25
*** e0ne has quit IRC22:25
*** rpodolyaka has quit IRC22:28
Lotus907efihello all. I am trying to follow the pypi mirror setup directions in pypi/README.md22:29
jang1go on.22:30
*** rpodolyaka has joined #tripleo22:30
Lotus907efibut when I try running run-mirror command I keep getting an error about "No local packages or download links found for pbr"22:30
Lotus907efiso I looked around on google to see what I could find22:30
Lotus907efiI see an openstack build test bot was getting this same issue over and over in May of last year22:31
Lotus907efibut I have not found any particular solution22:31
Lotus907efiis this error normal or to be expected?22:32
lifelessLotus907efi: not expected22:33
lifelessmordred: ^ thoughts?22:33
Lotus907efiah, any ideas what the cause might be?22:34
jang1can you pip install pbr manually?22:34
Lotus907efiI am inside the HP corp network but I have env variables set for http_proxy and https_proxy22:34
Lotus907efilet me try that22:35
Lotus907efiyup, I just did a "pip install pbr" and it said it completed with success22:36
Lotus907efiSuccessfully installed pbr22:36
Lotus907efiCleaning up...22:36
Lotus907efibut the run-mirror command still fails22:37
Lotus907efirun-mirror -b remotes/origin/master --verbose -c /home/ops/mirror.yaml22:37
Lotus907efiis what I am running22:37
jang1where does your mirror.yaml file point? (IIRC it has an upstream source declared in it, let me just have a quick look)22:37
jang1tell a li,e, that's bandersnatch22:38
Lotus907efimirrors:22:38
Lotus907efi - name: openstack22:38
Lotus907efi   projects:22:38
Lotus907efi     - https://git.openstack.org/openstack/requirements22:38
Lotus907efi   output: /home/opsauto/.cache/image-create/pypi/mirror22:38
tchaypoLotus907efi: I'm wondering if maybe you haven't exported the variable and the script isn't seeing it22:39
Lotus907efihmmm22:39
tchaypoyou could check with "export | grep proxy"22:39
mordredlifeless: aroo?22:39
Lotus907efiI did not see anything in pypi/README.md about exporting any variables22:39
jang1just https?_proxy22:39
lifelessmordred: pypi-mirror failing in the HP corp network22:39
mordredyes. it will22:40
lifelessmordred: workarounds ?22:40
*** slagle has quit IRC22:40
mordredbecause easy_install does not obey proxy settings22:40
Lotus907efithe other pypi modules seem to be successfully mirrored before it gets to pbr22:40
Lotus907efihmmm22:40
mordredand setup_requires does easy_install22:40
mordredSO22:40
mordredthe answer22:40
Lotus907efihmmm22:40
tchaypoLotus907efi: or, it could be what mordred said :(22:40
jang1easy_install has a bunch of problems. like, not grokking wheels.22:41
mordredis a small patch to pypi-mirror22:41
mordredeasy_install is the worst thing ever22:41
Lotus907efihehehehe22:41
mordredlet me dig up the patch22:41
lifelessan alternative would be to run bandersnatch22:41
Lotus907efi'k22:41
jang1python package "management" has gone through a number of generations, most of which have been terrible.22:41
mordredalso, we're working on having an inside-the-firewall pypi mirror22:41
lifelessand then run pypi-mirror against the resulting full pypi mirror22:41
lifelessmordred: hah! exactly. :)22:41
Lotus907efiha!22:41
Lotus907efithat would be cool22:41
Lotus907efiand quite helpful22:42
mordredLotus907efi: gimme just a bit - let me elevate it in personal priority22:42
Lotus907efiok22:42
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296922:42
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Put overcloud state into testenv.json.  https://review.openstack.org/8010422:43
jang1Lotus907efi: you _can_ install something like .. hrm, what is it? tsocks, or any number of other transparent proxies, and wrap up pypi-mirror to lean on those - but all of those are painful.22:43
jang1Or, spin up pypi-mirror in the public cloud and pull the result down inside the hp net. Unlike bandersnatch, pypi-mirror's output is of a reasonable size.22:44
mordredjang1: do we have any tsocks docs anywhere?22:44
jang1I don't think so - it's a bit of a black art. My desktop version doesn't even come with man pages.22:44
lifelessjang1: heh, 70G isn't unreasonable for a whole office etc :)22:45
jang170G isn't unreasonable for an office, but the time taken to grab the initial mirror is a bit of a pain ("come back tomorrow")22:45
jang1you can tell that the python universe sprang up largely outside corporate firewalls.22:46
lifelessjang1: agreed :). The similar size for Ubuntu mirror has the same issue.22:46
clarkbmordred: jang1 it doesn't really need documentation22:46
clarkbmordred: you tsocks thing22:46
lifelessjang1: well, python universe != setuptools, which is widely regarded as the most evil tool ever.22:46
*** Daviey has quit IRC22:46
Lotus907efiI am going to echo a proposal to Ulf that a pypi inside corp mirror is really needed22:47
clarkball it does is set LD_PRELOAD22:47
lifelessclarkb: what variable do you set to make it work?22:47
mordredLotus907efi: I'm working on one22:47
clarkblifeless: nothing you just tsocks command22:47
jang1I can't remember if tsocks will follow fork/exec. If it does (and thinking about it, it must) then you just need a tsocks.conf22:47
lifelessclarkb: how does it discover the proxy ?22:48
clarkbI think it uses 1080 by default22:48
jang1from http_proxy or socks_proxy22:48
clarkbwhatever the semi official port is22:48
lifelessclarkb: hostname! :)22:48
clarkbon localhost22:48
lifelessclarkb: which is entirely not what the HP corp net needs :)22:48
clarkbssh -D is all anyone ever needs :)22:48
lifelessclarkb: so I think my point is made, docs needed.22:48
greghaynesYes, I can confirm that ssh -D was the path to success during tripleo sprint at hp22:49
clarkbyou can use it to do rootless android tethers if you have adb going22:49
*** morazi has quit IRC22:49
clarkb>_>22:49
jang1hrm. Lotus907efi, unlike my desktop, this laptop gets man pages when tsocks gets installed.22:50
clarkband fwiw google is a great man page searcher22:51
Lotus907efitrue22:52
Lotus907efimordred: is the pypi mirror you are working on located somewhere permanent / reliable like a system in a data center?22:54
mordredyes22:55
mordredLotus907efi: it will be part of the gozer standup22:56
Lotus907efiI think I would like additional mirrors inside the two major hpcloud data centers if that is not where your mirror would already be located22:56
Lotus907efiah, ok22:56
mordredthe data centers shoudln't e behinda  blocking firewall22:56
Lotus907efiwhere is gozer being stood up?22:56
mordredLotus907efi: Fort Collins22:56
Lotus907efiah22:56
Lotus907efiare you in FtC ?22:57
mordredalthough we're also working on putting a mirror in galway22:57
mordredLotus907efi: I'm not - I'm in Sunnyvale22:57
Lotus907efioh, I am in FtC22:57
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Stop editing config.json.  https://review.openstack.org/8011022:57
mordredLotus907efi: well, then gozer is going to be great for you!22:57
Lotus907efijust so I don't have to fix it when it breaks  :D22:58
*** Daviey has joined #tripleo22:58
Lotus907efii.e. no 3AM phone calls22:58
lifelessnah. we schedule those for 4AM22:58
Lotus907efiah, that is OK then22:58
mordredLotus907efi: nah. we're getting a follow the sun team in place :)22:58
*** lblanchard has joined #tripleo22:58
ccrouch1roll up roll up reviewers :-)22:59
ccrouch1we have passing CI https://review.openstack.org/#/c/72003/22:59
mordredLotus907efi: any chance you know the socks proxy info?22:59
mordredccrouch1: w00t!22:59
tchaypomordred: we are?22:59
Lotus907efifor FtC ?22:59
mordredtchaypo: yes22:59
mordredLotus907efi: for anywhere22:59
lifelessdoes one typically run XFS on LVM, or XFS directly on component devices?22:59
tchaypomordred: I look forward to hearing more about this :)22:59
Lotus907efisocks info I am using is socks-server.hp.com port 108023:00
Lotus907efithat is what I had to use to get on freenode from my desktop system here23:01
Lotus907efibut the https_proxy I am using for our dev boxes in USeast is different23:01
Lotus907efilifeless: I have done both23:02
* Lotus907efi likes XFS filesystems23:02
mordredLotus907efi: cool. let me try that one real quick23:02
*** ftcjeff has quit IRC23:02
lifelessLotus907efi: I'm just noodling - I have a 18T RAID array here and found ext4 won't resize to use it all23:03
Lotus907efiah, yes23:03
Lotus907efiuse XFS23:03
lifelessLotus907efi: so I'm speculating on experimenting with XFS on the upper 3T23:03
Lotus907efiyeah, that would be interesting23:03
lifelessLotus907efi: but should I shrink the md device (raid6) down by 3TB, and run XFS on the raw devices (does it have a RAID mode?) or stay on top of the mdadm device...23:03
Lotus907efiI would run on top of the mdadm device23:04
Lotus907efino need to do the shrinking23:04
openstackgerritA change was merged to openstack-infra/tripleo-ci: Get logs of all instance on seed  https://review.openstack.org/7977123:07
*** jang1 has quit IRC23:09
openstackgerritA change was merged to openstack/diskimage-builder: baremetal element: Enable stable interface names  https://review.openstack.org/7200323:09
Lotus907efilifeless: one time when I was working at the Intel HPC benchmarking lab I formated a 3PB device with XFS and then when I did a "df -h" it reported the units in "P"23:11
Lotus907efithat was the first time I got to see a df output with "P" in the units23:12
mordredLotus907efi: I keep forgetting, I worked on this some already ...23:17
SpamapShmm.. I think we might be able to improve parallelism in our deploys if we create the ports for our servers before the servers...23:17
SpamapSBecause what we want is the address for the port.. if it isn't actually "ACTIVE" yet, that is not really important.. we can still configure nova-compute to try to talk to rabbitmq.23:18
SpamapSWhich should result in less churn at the moment that notCompute0 goes "ACTIVE"23:18
SpamapSnot sure that would relieve us from any waiting time tho23:19
* SpamapS goes back to playing with software deployers23:19
lifelessSpamapS: we need that for the VIP for APIs23:19
SpamapSlifeless: yeah that makes sense23:20
SpamapSjust sitting here watching NovaCompute#Config's waiting for notCompute0 to deploy23:21
SpamapSwhich btw, takes a long time now.. I think we need to attack image size23:21
SpamapS| notCompute0                     | 1b33eaca-df90-4ae7-9c59-551a869804bb | state changed          | CREATE_COMPLETE    | 2014-03-12T23:11:18Z |23:21
SpamapS| notCompute0                     | 88ee51e7-d5c1-421e-931b-522a2249c810 | state changed          | CREATE_IN_PROGRESS | 2014-03-12T22:44:35Z |23:21
SpamapShm that includes the dd time for NovaCompute0 and 123:22
SpamapS| NovaCompute1                    | bcf69e44-34ca-4dd0-852d-c62ae0810241 | state changed          | CREATE_COMPLETE    | 2014-03-12T22:56:29Z |23:22
SpamapS| NovaCompute0                    | f72d80de-6193-452d-977d-0858fe39bd8a | state changed          | CREATE_COMPLETE    | 2014-03-12T23:02:33Z |23:22
SpamapSsince we only dd one thing at a time23:22
SpamapSso NovaCompute1 went first, and took 20 minutes (presumably including image conversion to raw in there)23:23
SpamapSnot 20 minutes, 1223:23
SpamapSNovaCompute0 took 6 minutes ..23:23
SpamapShm and then notCompute0 9 minutes23:24
SpamapSso n/m me23:24
lifelessSpamapS: on VMs ?23:24
SpamapSlifeless: yeah, vms on my laptop23:24
SpamapSlifeless: doing --offline and devtest.sh -c ...23:24
*** saurabhs has quit IRC23:24
SpamapS133.37user 34.98system 1:01:34elapsed 4%CPU (0avgtext+0avgdata 32924maxresident)k23:25
SpamapS14338552inputs+6298768outputs (399major+7185863minor)pagefaults 0swaps23:25
SpamapSall images were already built23:25
SpamapSso --offline is inconsequential.. but with -c .. still 1 hour.23:25
SpamapS27 of which was deploying the overcloud in baremetal23:26
*** saurabhs has joined #tripleo23:27
* SpamapS squelches distractions and returns to dissecting heat stuff23:27
devanandaSpamapS: create ports before nodes -- non-starter with Ironic.23:28
devanandaSpamapS: without substantial changes, anyway23:29
* devananda wonders if that distraction was squelched23:29
lifelessdevananda: port-update :>23:30
lifelessdevananda: nova boot --nic accepts port ids23:30
devanandalifeless: sure. but port-create requires a node23:30
devanandaooh you mean neutron ports :)23:30
lifelessdevananda: I did say that calling Ironic's things ports was a bad idea.23:30
devanandahah23:30
lifelessdevananda: now, happy to say, I get to say 'told you so'23:30
lifelessTold you so!23:31
devananda:-D23:31
devanandalifeless: clearly we need more overloaded terms23:31
lifelessdevananda: clearly...23:31
devanandaquick! let's invent another thing called node!23:32
*** ramishra has joined #tripleo23:32
lifelessSpamapS: I just race using -c :)23:33
*** lsmola has quit IRC23:36
*** lsmola has joined #tripleo23:36
*** bauzas has quit IRC23:38
greghaynesSpamapS: https://review.openstack.org/#/c/80116/23:44
*** CaptTofu has joined #tripleo23:46
*** lucas-afk has quit IRC23:46
SpamapSgreghaynes: very nice.. -1'd ;)23:47
greghayneshaha, ty23:47
*** saurabhs has quit IRC23:47
greghaynesoh yay, only my commit msg was -1'd!23:48
SpamapSan excellent achievement23:48
*** saurabhs has joined #tripleo23:48
lifelessgreghaynes: also missing a test case.23:49
SpamapSand you forgot to hunt for snipes23:50
lifelessthough it's an interesting question *why* the user is missing23:50
*** CaptTofu has quit IRC23:50
lifelessmore like a rat23:50
greghayneswhat? snipes? where?23:50
SpamapSlifeless: the user is there. The access key is not... I think.23:50
openstackgerrittom-howley proposed a change to openstack/tripleo-incubator: Wait for baremetal nodes before creating undercloud stack.  https://review.openstack.org/7863623:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!