Wednesday, 2014-03-26

tchaypoII take it back, there is a bug00:01
tchaypohttps://bugs.launchpad.net/openstack-ci/+bug/127038200:01
uvirtbotLaunchpad bug 1270382 in openstack-ci "Name resolution failure on gate job" [Medium,Confirmed]00:01
StevenKBleh, looking at the wrong console log00:03
StevenK2014-03-25 23:42:46.216 | + '[' overcloud = overcloud ']'00:04
StevenK2014-03-25 23:42:46.216 | + setup-baremetal --service-host seed --nodes /dev/fd/6300:04
StevenK2014-03-25 23:42:46.252 | ++ jq '.nodes - [.nodes[0]]' /tmp/tmp5JOyMb00:04
StevenKlifeless: ^00:04
StevenK2014-03-25 23:42:46.252 | new00:04
tchaypouvirtbot: oh, you're playing nice today, well done!00:04
uvirtbottchaypo: Error: "oh," is not a valid command.00:04
StevenKHaha00:04
StevenKuvirtbot: botsnack00:04
uvirtbotStevenK: Error: "botsnack" is not a valid command.00:04
StevenKHeh00:04
lifelessStevenK: so its good ?00:22
*** mestery has quit IRC00:23
*** newell has quit IRC00:26
*** matsuhashi has joined #tripleo00:27
*** morganfainberg is now known as morganfainberg_Z00:33
*** yamahata has quit IRC00:34
StevenKlifeless: Hm, not sure why undercloud-ironic broke00:36
* tchaypo heads into the office for vaccination00:47
*** d0ugal has quit IRC00:53
*** d0ugal has joined #tripleo01:07
*** akrivoka has quit IRC01:09
*** matsuhashi has quit IRC01:12
*** matsuhashi has joined #tripleo01:14
*** sdake_ has quit IRC01:15
*** sdake has quit IRC01:16
*** eguz has quit IRC01:18
*** weshay has quit IRC01:25
*** nosnos has joined #tripleo01:27
*** sseago has joined #tripleo01:28
*** ramishra has joined #tripleo01:30
*** sdake has joined #tripleo01:35
*** sdake_ has joined #tripleo01:35
lifelessSpamapS: slagle: https://review.openstack.org/#/c/82066/01:45
lifelessStevenK: the deploy failed01:46
lifelessStevenK: pull the log tarball01:46
lifeless2014-03-26 00:07:49.063 4480 ERROR nova.compute.manager [req-5c546f0f-c3ab-4531-a963-746c51791f99 d6ceae50d0d74b1abc080f1bf99e7761 7094b5e353a3441fabe3242425ffcadf] [instance: 3630003f-52dd-45e1-a004-0815618b83da] Instance failed to spawn01:48
lifeless2014-03-26 00:07:49.063 4480 TRACE nova.compute.manager [instance: 3630003f-52dd-45e1-a004-0815618b83da] Traceback (most recent call last):01:48
lifeless2014-03-26 00:07:49.063 4480 TRACE nova.compute.manager [instance: 3630003f-52dd-45e1-a004-0815618b83da]   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 1718, in _spawn01:48
lifeless2014-03-26 00:07:49.063 4480 TRACE nova.compute.manager [instance: 3630003f-52dd-45e1-a004-0815618b83da]     block_device_info)01:48
lifelessthen check nova-baremetal-deploy-helper01:48
lifeless2014-03-26 00:07:39.615 4429 ERROR nova.virt.baremetal.deploy_helper [req-cf9f97eb-583b-48f9-b2eb-1d8771942699 None None] Cmd     : sudo nova-rootwrap /etc/nova/rootwrap.conf dd if=/mnt/state/var/lib/nova/instances/instance-00000001/disk of=/dev/disk/by-path/ip-192.0.2.3:3260-iscsi-iqn-3630003f-52dd-45e1-a004-0815618b83da-lun-1-part1 bs=1M oflag=direct01:49
lifeless2014-03-26 00:07:39.622 4429 ERROR nova.virt.baremetal.deploy_helper [req-cf9f97eb-583b-48f9-b2eb-1d8771942699 None None] StdOut  : ''01:49
lifeless2014-03-26 00:07:39.623 4429 ERROR nova.virt.baremetal.deploy_helper [req-cf9f97eb-583b-48f9-b2eb-1d8771942699 None None] StdErr  : '/bin/dd: writing \xe2\x80\x98/dev/disk/by-path/ip-192.0.2.3:3260-iscsi-iqn-3630003f-52dd-45e1-a004-0815618b83da-lun-1-part1\xe2\x80\x99: Input/output error\n1+0 records in\n0+0 records out\n0 bytes (0 B) copied, 120.693 s, 0.0 kB/s\n'01:49
*** yamahata has joined #tripleo01:58
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Remove backwards compatability in setup-baremetal  https://review.openstack.org/7846702:00
*** matsuhashi has quit IRC02:00
*** matsuhashi has joined #tripleo02:08
lifelessgreghaynes: so how goes HA? With Ironic sorted (pending reviews I have more bw for HA and perf)02:09
StevenKBlah, that remove backwards compat branch will need a recheck no bug when the toci change lands02:10
tchaypohttps://bugs.launchpad.net/tripleo/+bug/1297324 is doing my head in02:14
uvirtbotLaunchpad bug 1297324 in tripleo "iptables element scripts don't work on Ubuntu" [Undecided,New]02:14
tchaypocan't figure out if there's actually a problem, or if the requestor is just seeing rules and assuming they need to be applied02:14
tchaypoit looks like the add-rules script very deliberately targets only the redhat builds, which suggests this is deliberate... but why would redhat need iptables rules and debian-based distros not require them? but in any case if the lack of firewall rules was a problem I'd expect to see CI fail02:15
clarkbtchaypo: perhaps because rhel images enforce rules by default whereas debian/ubuntu allow all traffic to flow?02:18
tchaypoah. https://review.openstack.org/#/c/68075/ has a comment saying that as of F20 iptables doesn't run on fedora anyway so these probably aren't even needed there any more02:18
tchaypoclarkb: this is the impression I'm getting.02:19
tchaypoI'm going to mark the bug as incomplete with a suggestion that unless there are actual problems observed with the traffic flow this should probably be closed02:19
lifelessso we may want to firewall things up02:20
lifelessbut yes the rules exist to unbreak fedora02:21
tchaypowe may want to, yes; but it seems that everything we have so far has been designed to unbreak fedora02:25
tchaypoit seems to me that switching to "here's a set of iptables rules that will work generically" is a bigger goal02:25
tchaypoalso, it bugs me that launchpad lets me triage/update priority/comment on a bug and still doesn't think I want to get emails about changes. I wish I could find a setting that lets me get auto-subscribed to everything I touch02:25
*** noslzzp has quit IRC02:26
*** vkozhukalov has joined #tripleo02:31
*** saurabhs1 has quit IRC02:32
openstackgerritA change was merged to openstack-infra/tripleo-ci: Kill commands that outlive test env  https://review.openstack.org/7540202:33
openstackgerritA change was merged to openstack-infra/tripleo-ci: Set the worker timeout  https://review.openstack.org/7617202:33
lifelesstchaypo: you can subscribe to the whole feed02:34
*** fandi has quit IRC02:34
lifelesstchaypo: or per bug basis, I don't *think* there is 'if I commented', but I may be wrong - try putting a custom feed subscription in place02:35
StevenKThere's also a setting for e-mailing you about things you changed02:36
lifelessStevenK: https://review.openstack.org/#/c/79051 needs tweaks02:36
StevenKYeah, let me fix that up.02:38
*** BadCub01 has quit IRC02:39
*** lazy_prince has quit IRC02:39
*** killer_prince has joined #tripleo02:39
StevenKlifeless: I was also trying to work out how to store the route-dev in the object02:39
StevenKSince we currently use a specified value in toci, and the fallback for devtest.02:40
*** nati_ueno has quit IRC02:42
lifelessStevenK: so that patch isn't about toci using testenv; as such you could just document it.02:43
*** ramishra has quit IRC02:45
*** ramishra has joined #tripleo02:45
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Update documentation.  https://review.openstack.org/8274102:49
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Default to 30G of disk per bm VM.  https://review.openstack.org/8275302:49
*** ramishra has quit IRC02:50
*** cwolferh has quit IRC02:54
*** ramishra has joined #tripleo02:54
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Update documentation.  https://review.openstack.org/8274102:57
*** ramishra has quit IRC03:02
*** ramishra has joined #tripleo03:02
*** cwolferh has joined #tripleo03:06
*** ramishra has quit IRC03:07
openstackgerritA change was merged to openstack/tripleo-heat-templates: Adding image update policy to undercloud template  https://review.openstack.org/8206603:11
openstackgerritA change was merged to openstack/tripleo-image-elements: Create pip manifest files of installed packages  https://review.openstack.org/8064403:12
*** matsuhashi has quit IRC03:15
greghayneslifeless: not a lot of progress, got distracted by trying to actually learn how heat works. Got the percona config to be templated out by a hosts list from heat this afternoon so hoping to work on the bootstrapping logic this eve/tomorrow morning.03:17
lifelessawesome03:17
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Update documentation.  https://review.openstack.org/8274103:17
lifeless[2014-03-26 16:17:15] Total runtime: 1396 s03:18
lifeless[2014-03-26 16:17:15]   ramdisk    : 0 s03:18
lifeless[2014-03-26 16:17:15]   seed       : 213 s03:18
lifeless[2014-03-26 16:17:15]   undercloud : 353 s03:18
lifeless[2014-03-26 16:17:15]   overcloud  : 820 s03:18
lifelessreal    23m16.304s03:18
lifelessuser    0m57.494s03:18
lifelesssys     0m16.543s03:18
lifeless time with -c03:18
greghayneswat03:18
tchaypolifeless: ^^^ seemed easier than just telling you about the nit03:18
lifelesstchaypo: thanks :)03:18
tchaypogreghaynes: oh, there you are. i was just wondering where your in-progress change is because i want to start learning from it03:18
SpamapSlifeless: hey, so heat-admin is no longer necessary...03:18
SpamapSlifeless: if we stop setting it in heat.conf, we'll get whatever the default from the image is.03:19
lifelessSpamapS: youknowwhattodo03:19
greghaynestchaypo: not sure what change you are referring to?03:19
lifelessSpamapS: where do we set it? Also how will we know what user to ssh in to setup keystone with ?03:19
tchaypogreghaynes: the work you're doing on percona. I was hoping there was a change03:20
lifelessbbiab, C time03:20
* tchaypo should have looked at trello03:20
greghaynesah. The switch to xtradb landed, just getting started on actual clustering. Ill try to push a WIP soon and hl you03:21
SpamapSlifeless: 1: heat.conf, admin_user=heat-admin. 2: I know.. trade one problem for another one. ;)03:21
greghaynesoh, trello. Forgot about that :)03:21
*** Matt1 has quit IRC03:21
SpamapSlifeless: I was thinking we could bake a locked down user into the images for it.03:22
tchaypothanks :)03:23
SpamapSotherwise we have to map distro -> cloud-init default.03:23
openstackgerritA change was merged to openstack/tripleo-incubator: Default to 30G of disk per bm VM.  https://review.openstack.org/8275303:25
* SpamapS releases os-collect-config 0.1.13 with cfn deployments support03:27
*** nosnos has quit IRC03:28
StevenKHeh, I guess you can't create RHEL images without a RHN account03:29
SpamapSAFAIK, yes03:32
SpamapSIt's one of the reasons I've never logged into a RHEL server. :)03:33
mordredSpamapS: those are a thing? I thought they were only a myth03:33
StevenKHeh03:33
SpamapSmordred: A story told to the stock holders to help them sleep at night? ;)03:34
mordredSpamapS: :)03:34
StevenKI'm trying to work out if we can use 'ip route replace'03:34
StevenKFedora 20 is fine, at least03:35
tchaypo"control plane node" == "notCompute", right?03:36
tchaypols03:36
greghaynes. ..03:38
* tchaypo cuddles tmux and its quirky some-times keystroke eating03:38
* tchaypo is sitting in an office right near a myITpc03:39
* tchaypo learns from the advertising on the screen that he will soon have 2 whole Gb of storage space in outlook03:39
StevenKtchaypo: From like Aug 201303:39
tchaypoStevenK: tmux, or noComputer?03:40
tchaypos/noComputer/notCompute/03:40
StevenKtchaypo: The 2 whole Gb of storage in Outlook03:40
tchaypooh right. according to the screen it's "COMING SOON"03:40
* StevenK considers connecting to the VPN to find docs to back up his comment.03:41
tchaypooh, i believe you03:41
StevenKI do wonder just how many machines we have dedicated to Exchange03:44
tchaypoone.03:44
tchaypoit might be spread across multiple chassis but effectively it's one computer *nods*03:44
StevenKtchaypo: Exchange supports clustering natively03:44
greghaynesIm sorry you know that03:45
StevenKgreghaynes: I discussed it vaguely with a friend of mine03:45
clarkbStevenK: you need a lot03:45
StevenKHe knows Exchange, the poor bastard, not me.03:45
clarkbwe had a ton at Intel and only a quarter the employees03:46
StevenKclarkb: Yeah, my rough calculations were on the order of like 4-6 racks per region03:46
*** ramishra has joined #tripleo03:46
StevenK(With 50 machines per rack)03:46
*** blamar has quit IRC03:46
* tchaypo gives dirty looks to th eman with the Escorted Visitor badge and the macbook air03:49
StevenKHaha03:49
clarkbtchaypo: huh everyone in the seattle office has a macbook03:49
StevenKtchaypo: That was you in Moffet? :-P03:50
clarkbeveryone but the tiny corner I sit in but slowly we are losing out03:50
StevenKclarkb: http://www.youtube.com/watch?v=Jln3mi0vfJU03:50
*** mestery has joined #tripleo03:51
*** killer_prince is now known as lazy_prince03:53
tchaypoStevenK: i didn't have an "escorted visitor" badge03:55
tchaypoclarkb: I've got an 8570w.I need to glare at people with macbook airs to take my mind of the awful crushing weight I carry with me.03:55
*** blamar has joined #tripleo03:56
StevenKHaha03:56
clarkbtchaypo: we must resist03:56
clarkbthough the bricks are pretty terrible03:56
tchaypothe office muzak is curerntly "i wanna dance with somebody"03:56
StevenKI'm happy enough with the 9470m03:57
StevenKI would like to know where my slice battery is, though03:57
clarkbStevenK: its a decent machine. The touchpad and display could be better though03:57
clarkbtchaypo: you have office music!?03:57
StevenKclarkb: He's in the cafeteria03:57
tchaypoclarkb: we do in the cafeteria03:57
clarkbI tried using the couches in the office today but they didn't agree with my back03:57
*** blamar has quit IRC03:57
StevenKtchaypo: Oh, did you have to get another prox card stapled to the one you got in Moffet?03:58
*** blamar has joined #tripleo03:58
tchaypono, the one i got in moffet works dandy03:59
tchaypogreghaynes: if i'm reading things correctly, you're working on getting percona working on 1 node first, and you're planning to attack the building of multiple nodes later03:59
tchaypogreghaynes: is that right?03:59
openstackgerritA change was merged to openstack/diskimage-builder: Only create a tmpfs big enough for DIB_MIN_TMPFS  https://review.openstack.org/7907404:00
greghaynestchaypo: works on one node right now, moving on to the >1 node setup now04:01
tchaypoah, hence why you spent the day learning about heat04:02
greghaynestchaypo: for that its basically template out configs so the nodes know about each other, and correctly bootstrap a first node04:02
*** nati_ueno has joined #tripleo04:02
greghaynesyep, and also I kinda have been wanting to know it :)04:02
greghaynestherse probably more stuff for handling updates, but not there yet04:03
greghayneser, theres04:03
lifelessStevenK: the zbook14 slice arrived this morning04:03
tchaypoI was planning to attack the problem from the other direction04:03
greghaynesoh?04:04
tchaypoie, get multiple nodes running, then figure out how to get one of the services (rabbitmq seems like a good target) to know about the other nodes04:04
greghaynesI think rabbitmq should be "working"04:04
greghaynesI havent actually tested it, but in theory its all there04:04
tchaypoyeah, that's why i think it's a good target04:04
lifelesstchaypo: ++04:05
greghaynesI think jprovazn posted a heat template that might be a good place to poke at04:05
StevenKlifeless: Nice. It's been 7 weeks since I ordered it.04:05
greghaynes-> bbiab04:05
tchaypogreghaynes: it sounds like i'm going to be duplicating some of your effort - but i think i need to learn about heat too, so i think i'm going to go ahead anyway and assume I'll be throwing away my work04:06
tchaypoI got my slice about 3 days after ordering it04:06
StevenK*sadface*04:06
* StevenK goes to prepare a portion of dinner04:06
tchaypoIt'd be more excited if my RAM had turned up04:07
tchaypolifeless: as an aside - I was thinking that at some point when I'm looking for a pre-coffee task I'd like to rewrite README.md as README.rst and get it linked into the rest of the docs04:08
tchaypoit seems like it makes a more better landing page than the current index.html or devtest.html04:08
tchaypoespecially with the stuff you added today.04:09
tchaypoAlso I think I'd like to collect the notes on setting up squid, pypi-mirror etc into one place04:09
*** blamar has quit IRC04:10
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update selinux policies for nova's use of /mnt/state  https://review.openstack.org/8297604:10
lifelesstchaypo: +1, I believe setup.cfg can cause doc/source/index.rst to be the pypi registered doc04:12
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-heat-templates: WIP: Switch overcloud to software-config  https://review.openstack.org/8166604:12
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update selinux policies for glance's use of /mnt/state  https://review.openstack.org/8297804:12
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update selinux policies for neutron's use of /mnt/state and /var/run  https://review.openstack.org/8298004:13
StevenKlifeless, SpamapS: Do you two want to push https://review.openstack.org/#/c/82941/ along?04:14
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update selinux policies to allow openssl to read keystone ssl files  https://review.openstack.org/8298104:15
lifelessits 9pm for SpamapS probably still doing familty time04:15
SpamapSlifeless: I did family time sprinkled throughout the day actually04:15
SpamapSso I've got an evening pass04:15
* tchaypo suddenly understands what "merge.py" does04:16
* StevenK fights with jq04:17
lifelessSpamapS: .oO04:17
lifelessSpamapS: so land 82941 PLOX04:17
openstackgerritA change was merged to openstack-infra/tripleo-ci: Use --nodes for setup-baremetal in toci_devtest  https://review.openstack.org/8294104:18
SpamapSlifeless: as you wish04:18
StevenKSpamapS: Heh, re: your comment on 78467, it requires ^04:19
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Scrub ironic config in the seed config.  https://review.openstack.org/8298304:23
lifelessSpamapS: \o/04:23
StevenKBlah04:23
lifeless82983 is needed to land the tie patch for ironic without breaking stuff04:23
StevenKThat one liner is horrible04:24
* tchaypo puts on One Direction in order to drown out the dreadful office muzak04:24
*** matsuhashi has joined #tripleo04:25
*** mestery has quit IRC04:25
StevenKtchaypo: So you're replacing dreadful music with other dreadful music?04:25
StevenKSeems kinda redudant.04:25
tchaypospeaking of which - when is eurovision?04:26
tchaypooh pooples. the final is the 10th may, which means it's going to air in Aus while I'm in the air from Aus04:26
SpamapSStevenK: and a rebase.. :-P04:27
*** killer_prince has joined #tripleo04:27
*** eghobo has joined #tripleo04:27
StevenKtchaypo: Are you going to raise a session at the summit that it must not conflict with eurovision?04:27
SpamapSok.. brief switch back to family time and then back to the review treadmill04:27
*** vkozhukalov has quit IRC04:28
StevenKSpamapS: Bah :-)04:28
tchaypoNo. What I am going to raise is a suggestion that the next mid-cycle should be ~august 4-8 and should be located in Brisbane04:30
tchaypowhich will be great, because everyone can turn up a few days early and attend pycon-au04:30
clarkbthat conflicts with all the summer plans :P04:31
clarkbwe only get that one week of summer here04:31
StevenKHaha04:31
tchaypohah04:31
tchaypoI'm sure brisbane mid-winter will be nicer than your summer anyway04:32
tchaypowherever it i s that you are (seattle, right?)04:32
StevenKmid-lack-of-summer04:32
StevenKEr04:32
clarkbya seattle04:32
StevenKWe don't have winter in Australia, we have lack-of-summer04:32
StevenKAnd the US doesn't have summer, it has lack-of-winter04:33
tchaypopfft. sydney has winter from late feb through to early december.04:33
clarkbno most of the US has proper summer04:33
clarkbhot humid days for months in massive chunks of the country04:34
StevenKclarkb: It isn't a proper summer if it doesn't hit 10404:34
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Remove backwards compatability in setup-baremetal  https://review.openstack.org/7846704:35
tchayposo the good news is, all the eurovision songs are up on the website already, so i just need to scrape them and i can take them onto the plane with me04:35
StevenK... yay ?04:36
tchaypooh goodie. A song about baking a cake with lyrics including "tap tap tap to-coo-coo"04:36
*** nosnos has joined #tripleo04:36
*** blamar has joined #tripleo04:46
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Store seed details in the JSON  https://review.openstack.org/7905104:58
*** ramishra_ has joined #tripleo05:04
*** ramishra has quit IRC05:07
*** cwolferh has quit IRC05:24
*** rpodolyaka1 has joined #tripleo05:26
*** cwolferh has joined #tripleo05:27
lifelessSpamapS: https://review.openstack.org/#/c/82741/ should be ok now05:28
*** lazy_prince has quit IRC05:28
*** cwolferh has quit IRC05:32
openstackgerritA change was merged to openstack/diskimage-builder: serial-console: Use udev rules to startup getty  https://review.openstack.org/8161605:33
*** fandi has joined #tripleo05:44
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Teach testenv about numbers and bridges  https://review.openstack.org/8087405:45
*** e0ne has joined #tripleo05:55
*** nati_ueno has quit IRC05:57
StevenK2014-03-26 05:29:04.980 |   File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 13, in <module>05:59
StevenK2014-03-26 05:29:04.981 |     from netaddr.strategy import ipv4 as _ipv4, ipv6 as _ipv605:59
StevenK2014-03-26 05:29:04.981 | ValueError: bad marshal data (unknown type code)05:59
StevenKBlink05:59
*** lazy_prince has joined #tripleo06:00
killer_princelifeless: regarding your comment about "Refactor code to select boot kernel (https://review.openstack.org/#/c/79873)"06:10
*** sdake has quit IRC06:12
killer_princelifeless: first comment about syslinux, i looked at the code and i see that it is guarded by os only for kernel selection and not for syslinux configuration..06:12
*** sdake_ has quit IRC06:12
killer_princelifeless: this is the reason, i put a check for syslinux directory and then create config file..06:13
lifelesskiller_prince: hmmm, will need to think about that06:14
killer_princelifeless: okay, for the 2nd comment, i have put the common code in a function as its getting reused in baremetal element and vm element..06:14
lifelesskiller_prince: but it seems like there would be no refactoring to do if the deprecated CLI was just removed06:14
killer_princelifeless: and i intend to euse the same code later in one of the localboot element for tripleo which is pending review (https://review.openstack.org/#/c/79289/) Adds local storage Boot support in case PXE boot fails.06:16
killer_princes/euse/reuse06:16
lifelessok, I'll re-review tomorrow in that light06:16
killer_princethanks..06:16
*** rpodolyaka1 has quit IRC06:17
killer_princei am fine with deleting the cli but we still have some common code shared between vm and baremetal element which needs to be consolidated (as per community feedback)...06:18
killer_princeI will wait for your 2nd review.. :)06:19
*** sdake has joined #tripleo06:24
*** sdake has quit IRC06:24
*** sdake has joined #tripleo06:24
*** sdake_ has joined #tripleo06:27
*** matsuhashi has quit IRC06:33
*** matsuhashi has joined #tripleo06:35
*** eghobo has quit IRC06:44
*** e0ne has quit IRC06:47
*** e0ne has joined #tripleo06:48
*** e0ne has quit IRC06:51
*** rdopieralski has joined #tripleo06:56
StevenK2014-03-26 06:22:04.504 | Waiting for keystone to initialise...07:01
StevenK2014-03-26 06:22:05.966 | An unexpected error prevented the server from fulfilling your request. (HTTP 500)07:01
StevenKGar07:01
*** vkozhukalov has joined #tripleo07:02
*** matsuhas_ has joined #tripleo07:03
*** matsuhas_ has quit IRC07:04
*** jcoufal has joined #tripleo07:04
*** matsuhas_ has joined #tripleo07:06
*** matsuhashi has quit IRC07:07
*** mrunge has joined #tripleo07:32
*** ramishra_ has quit IRC07:34
*** jprovazn has joined #tripleo07:39
*** thouveng has joined #tripleo07:49
*** matsuhas_ has quit IRC07:59
*** matsuhashi has joined #tripleo08:04
GheRiveromorning all08:12
lsmola_SpamapS: hello, are you around?08:17
*** pblaho has joined #tripleo08:18
*** panda has joined #tripleo08:18
lifelessGheRivero: morning!08:19
GheRiverolifeless: morning08:20
lifelessGheRivero: Ng: https://review.openstack.org/#/c/82683/ passed now08:21
GheRiverocool08:23
*** hashar has joined #tripleo08:23
*** Matt2 has joined #tripleo08:31
rpodolyakamorning all08:34
killer_princelifeless: i am trying to fix bug id 1231946 (Fedora images built on a UEFI host are configured to boot UEFI). but as i understand, uefi needs GPT formatted disk. so is it okay to assume that if user is creating a uefi compatible disk image using vm element, vm element should create a GPT fromatted disk and configure UEFI boot partition, and in all other case, use MBR formatted disk and install MBR compatible boot loaders..08:35
lifelesskiller_prince: anything deployed via nova-bm will be partitioned MBR today.08:35
lifelesskiller_prince: we haven't added GPT support.08:36
*** jistr has joined #tripleo08:37
killer_princelifeless: is there any work going on for GPT support in nova-bm/ironic.. i just wanted to get involved08:37
lifelesskiller_prince: I think so, don't know the status08:37
lifelesskiller_prince: it would be ironic08:37
lifelesskiller_prince: so, vm images will always be MBR for now, because booting UEFI is a pita with kvm (doable, but sheese)08:38
lifelesskiller_prince: and non-vm will always be MBR partitions because thats what nova-bm creates.08:38
openstackgerritA change was merged to openstack/diskimage-builder: Remove an excess cp of disk images.  https://review.openstack.org/8268308:38
lifelessGheRivero: Ng: can I get two hell yeahs for https://review.openstack.org/#/c/82983/ ?08:39
lifelessStevenK: https://review.openstack.org/#/c/78467/08:39
killer_princelifeless: k.. but i see some code related to efi in vm element.. is it going to be removed soon or should vm element always setup a mbr compatible boot loader..08:39
lifelesskiller_prince: you do ?08:40
lifelesskiller_prince: if its from a1beca7b44a788dc627a210550ab2d96714f391c it seems to be disabling efi boot targets08:41
lifelesskiller_prince: under some circumstances08:42
*** ifarkas has joined #tripleo08:42
killer_princeokay..08:44
killer_princeso then how to go about bug id 1231946.. Fedora images built on a UEFI host are configured to boot UEFI08:44
*** rpodolyaka1 has joined #tripleo08:45
*** rlandy has joined #tripleo08:48
*** rpodolyaka1 has quit IRC08:48
*** gcha has joined #tripleo08:48
rpodolyakaa quick question on 'untriaged bugs bot'. Should we list 'Incomplete' bugs as untriaged?08:49
openstackgerritA change was merged to openstack/tripleo-incubator: Scrub ironic config in the seed config.  https://review.openstack.org/8298308:51
openstackgerritAlexis Lee proposed a change to openstack/tripleo-incubator: Fall back to $0 if BASH_SOURCE is unset  https://review.openstack.org/8153908:53
lifelessrpodolyaka: incomplete_with_response yes08:56
lifelessrpodolyaka: incomplete_without_response, no08:57
*** jcoufal has quit IRC08:59
*** jcoufal has joined #tripleo09:00
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Add Ironic support to incubator.  https://review.openstack.org/8195909:02
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Remove backwards compatability in setup-baremetal  https://review.openstack.org/7846709:02
openstackgerritMatthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install the check_mk Agent  https://review.openstack.org/8148509:04
*** athomas has joined #tripleo09:10
*** martyntaylor has joined #tripleo09:10
*** derekh has joined #tripleo09:11
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: debian: install cloud-init on Wheezy  https://review.openstack.org/8232509:21
rpodolyakalifeless: ack, thanks!09:22
*** andreaf has joined #tripleo09:22
gonery lifeless thanks for the review09:23
*** gonery is now known as Goneri09:23
*** hashar has quit IRC09:28
openstackgerritA change was merged to openstack/diskimage-builder: Adds "element-provides" support to element dependencies  https://review.openstack.org/7942109:30
lifelessStevenK: ^^^ was a rebase only09:30
*** lucasagomes has joined #tripleo09:31
openstackgerritGonéri Le Bouder proposed a change to openstack/tripleo-incubator: doc: ReST syntax changes to improve readability  https://review.openstack.org/8231509:33
*** howleyt_ has joined #tripleo09:34
*** hashar has joined #tripleo09:35
*** Rakesh5 has joined #tripleo09:36
openstackgerritA change was merged to openstack/tripleo-heat-templates: Remove unnecessary cinder metadata  https://review.openstack.org/8060009:38
*** giulivo has joined #tripleo09:39
*** e0ne has joined #tripleo09:39
andrearosaquestion about tripleo-geat-templates: can anyone tell me where the notcompute.yaml file is used? It's not used in the makefile and I can't see other places where it's used09:39
andrearosas/tripleo-geat-templates/tripleo-heat-templates09:39
*** derekh has quit IRC09:42
*** derekh has joined #tripleo09:42
*** pbelanyi has joined #tripleo09:44
*** rpodolyaka1 has joined #tripleo09:46
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-incubator: WIP - configure a node ephemeral disk  https://review.openstack.org/8277609:46
openstackgerritA change was merged to openstack/tripleo-image-elements: Add fedora-rdo-icehouse element  https://review.openstack.org/8065809:47
openstackgerritA change was merged to openstack/diskimage-builder: Add console kernel parameters to extlinux configuration  https://review.openstack.org/8282209:47
*** bauzas has joined #tripleo09:51
openstackgerrittom-howley proposed a change to openstack/tripleo-image-elements: Add keepalive option to rabbitmq config  https://review.openstack.org/8192009:51
*** rpodolyaka1 has quit IRC09:51
lifelessGheRivero: hey - in https://bugs.launchpad.net/tripleo/+bug/129573209:54
uvirtbotLaunchpad bug 1295732 in tripleo "tripleo devtest overcloud startup takes too long " [High,Triaged]09:54
lifelessGheRivero: virtio makes it fly for me :)09:54
openstackgerritA change was merged to openstack/tripleo-incubator: Fall back to $0 if BASH_SOURCE is unset  https://review.openstack.org/8153909:56
openstackgerritA change was merged to openstack/tripleo-image-elements: qpidd user should own sasldb file  https://review.openstack.org/8190809:57
GheRiverolifeless: will post data with virtio and no virtio to the bug later this morning just to compare09:57
openstackgerritA change was merged to openstack-infra/tripleo-ci: Add hostname of Test env to json  https://review.openstack.org/8060309:57
GheRiveroapparently my builds where dying because the user image wasn't built with apt-sources and failed to fetch the mirrors09:59
GheRiverobut it's solved now09:59
derekhlifeless: did you mean to approve https://review.openstack.org/#/c/79074/ , I didn't pass CI and I believe will cause us failures in all other CI runs09:59
derekh*It didn't10:00
lifelessGheRivero: cool10:00
howleyt_GheRivero, lifeless: I believe that smulcahy tried with virtio and saw no noticeable difference10:00
lifelesshowleyt_: I see 23m drop to 16m10:01
lifelesserm 16m50s, so 17m10:01
GheRiverobut i don't like the solution. Passing elements via NODE_DIST will make to add those elements also to the build_ramdisk image10:01
*** jp_at_hp has joined #tripleo10:01
smulcahyah, someone called?10:02
smulcahy:)10:02
smulcahyI need a communications channel multiplexer or something10:02
smulcahyor more eyes10:02
lifelessGheRivero: sure, I see you're working on a nicer thing10:02
GheRiverolifeless: https://review.openstack.org/#/c/82739/210:02
lifelessGheRivero: but - the ramdisk image needs the same apt-sources etc too :)10:02
openstackgerritA change was merged to openstack/tripleo-image-elements: Inject admin username and tenant name in config  https://review.openstack.org/8181310:03
lifelessderekh: Apparently I was blind.10:03
lifelessderekh: merged at 5pm. Now 11pm. Have we had passes since then ?10:03
smulcahyI need to get my workstation back on the network and run a bunch more speed tests with a few configs, to make sure I'm seeing consistent numbers10:03
derekhlifeless: don't think so http://goodsquishy.com/downloads/tripleo-jobs.html10:04
smulcahybut I need to wait for IT to fix my network first so I can't say when that will happen10:04
lifelessderekh: nuts; can has revert?10:04
derekhlifeless: we can revert, move image outside of tmpfs or increase DIB_MIN_TMPFS10:04
lifelessderekh: revert is fast10:05
derekhlifeless: ok, will do10:05
lifelessderekh: the rest we have to iterate on; so lets revert, and we can reinstate with the other stuff at the same time10:05
lifelessderekh: btw - moving image outside tmpfs, did you see someone suggesting using a new tmpfs? I like that idea10:05
derekhlifeless: Ya, I saw that, works for me10:06
howleyt_lifeless: maybe I'm misrepresenting smulcahy and it was just that he still found it slow with virtio enabled.10:06
openstackgerritDerek Higgins proposed a change to openstack/diskimage-builder: Revert "Only create a tmpfs big enough for DIB_MIN_TMPFS"  https://review.openstack.org/8302110:07
*** matsuhashi has quit IRC10:09
howleyt_On a different note, in Mustache, can I differentiate empty vs. false values?10:09
derekhlifeless Ng ; revert to fix CI, should we wait on CI to prove that this was the problem?10:10
lifelessderekh: the failure is the one in the failed run on what I wrongly landed. No.10:10
lifelesshowleyt_: empty is falsey. So no.10:11
lifelessat least, I don't know how to ;)10:11
derekhlifeless: yup it is10:11
lifelessderekh: I mean - no need to wait10:12
howleyt_lifeless: ya, that's what I thought.10:12
derekhlifeless: ok, agreed, lets land it10:12
*** yamahata has quit IRC10:15
howleyt_lifeless: any chance you could take a quick look at https://review.openstack.org/#/c/81920/ so that I know have the right idea on use of symlinks, etc.10:21
openstackgerritA change was merged to openstack/tripleo-incubator: Teach testenv about numbers and bridges  https://review.openstack.org/8087410:21
dtantsurslagle, ping re instack-undercloud10:30
dtantsurslagle, I'm interesting now in running instack-undercloud with Ironic. Am I right assuming by default it's going to use nova baremetal (judging by answers file)10:31
*** akrivoka has joined #tripleo10:33
*** matsuhashi has joined #tripleo10:39
dtantsur... but the first instack question is what are VIRTUAL_POWER_HOST? "instack" vm IP? My laptop IP?10:40
dtantsurifarkas, maybe you know ^^^10:40
lifelesshowleyt_: broadly yes; have reviewed the install.d file only, and given some detailed feedback10:42
lifelessdtantsur: what are you trying to do ?10:43
dtantsurlifeless, I'm trying to use instack for installing undercloud on VMs to develop Ironic10:43
dtantsurhowever, it's the first time I actually see instack, thus my questions :)10:44
lifelessah10:44
lifelessI've never used instack either10:44
lifelesshowever trunk tripleo doesn't support ironic yet10:44
lifelessso it may be premature to use instack10:44
lifelesshave you tried basic devtest ?10:45
*** hashar has quit IRC10:45
dtantsurlifeless, actually not10:46
dtantsurit seemed to me instack could be easier (which I'm no longer sure is true)10:46
howleyt_lifeless: thank you kindly.10:47
*** rpodolyaka1 has joined #tripleo10:47
*** hashar has joined #tripleo10:48
dtantsurlifeless, instack-undercloud is based on tripleo mostly, right? So for me it does not matter whether I use devtest or instack, I still have to setup Ironic manually?10:50
lifelessdtantsur: not at all10:50
*** howleyt_ has quit IRC10:50
lifelessdtantsur: see the instructions at the top of https://bugs.launchpad.net/ironic/+bug/129706310:50
uvirtbotLaunchpad bug 1297063 in ironic "failed to deploy overcloud node" [Undecided,New]10:50
lifelessthat will deploy a seed vm with ironic in it, use it to deploy an undercloud with ironic in it, use that to deploy an overcloud etc10:51
dtantsurlifeless, cool, thanks10:51
dtantsurobject has no attribute '__name__'  <-- the same with devstack, oh10:52
*** rpodolyaka1 has quit IRC10:52
lifelessdtantsur: iz bug :)10:53
*** hashar has quit IRC10:54
*** hashar has joined #tripleo10:54
openstackgerritA change was merged to openstack/diskimage-builder: Revert "Only create a tmpfs big enough for DIB_MIN_TMPFS"  https://review.openstack.org/8302111:05
openstackgerritStuart McLaren proposed a change to openstack/tripleo-image-elements: Glance: Allow configuring default_log_levels  https://review.openstack.org/7655711:07
*** noslzzp has joined #tripleo11:13
*** fandi has quit IRC11:21
openstackgerritCian O'Driscoll proposed a change to openstack/tripleo-heat-templates: Update templates to default to preserve ephemeral  https://review.openstack.org/8303711:22
*** e0ne has quit IRC11:23
*** e0ne has joined #tripleo11:23
*** e0ne has quit IRC11:29
*** yamahata has joined #tripleo11:40
*** hashar has quit IRC11:43
*** rpodolyaka1 has joined #tripleo11:47
*** jcoufal has quit IRC11:52
*** rpodolyaka1 has quit IRC11:52
*** IvanBerezovskiy has joined #tripleo11:54
*** jcoufal has joined #tripleo11:55
*** bogdando has quit IRC11:58
*** athomas has quit IRC11:59
openstackgerritLadislav Smola proposed a change to openstack/tuskar: Adding swift role back  https://review.openstack.org/8249911:59
*** e0ne has joined #tripleo12:00
*** lblanchard has joined #tripleo12:00
*** e0ne has quit IRC12:04
*** e0ne has joined #tripleo12:05
*** Slower has quit IRC12:05
*** athomas has joined #tripleo12:05
derekhjogo: I'm trying to plot the run times for successful tripleo runs, took a look at graphite and the graph you linked on monday, the timers don't seem to be returning values I'm expecting12:05
derekhjogo: any pointers ? http://graphite.openstack.org/render/?width=586&height=308&from=-36hours&target=movingAverage(stats.timers.zuul.pipeline.check-tripleo.resident_time.mean,%20%2712hours%27)12:05
*** yamahata has quit IRC12:06
*** fandi has joined #tripleo12:09
*** e0ne has quit IRC12:10
*** matsuhashi has quit IRC12:10
*** bogdando has joined #tripleo12:10
*** matsuhashi has joined #tripleo12:11
*** matsuhas_ has joined #tripleo12:13
*** matsuhashi has quit IRC12:13
*** e0ne has joined #tripleo12:14
*** hewbrocca has joined #tripleo12:14
chmoueli am still pretty confuse how the os-*-config tools works with heat, is there any  image somewhere  that describes how that stuff works12:18
chmoueli understand from the description of each projects in that they are applying configuration from a json/yaml file12:18
chmouelbut where is that configuration come from?12:18
hewbroccachmouel: configuration comes from cloud-init12:21
chmouelhewbrocca: or cfn right?12:21
hewbroccaWhen Heat launches each instance it passes config to nova12:21
hewbroccawhich makes it available to the instance12:21
hewbroccaos-* tools can then retrieve it and apply it12:21
chmouelok but that's only on provisioning, is it on stack-update as well?12:21
hewbroccayes12:22
hewbroccaos-refresh-config polls, IIUC12:22
hewbroccawhen it finds new config it pulls it down and apply it12:22
chmouelcool thanks, so what kind of config this is supposed to do?12:23
*** e0ne_ has joined #tripleo12:23
chmouelit can be anything or just launch other configuration management12:23
chmoueli.e: puppet/ansible etc..12:23
*** e0ne has quit IRC12:23
hewbroccafor tripleo specifically, the os-* tools run scripts from each element12:24
hewbroccabut12:24
hewbroccaif you're not using tripleo, you can do whatever you want12:24
hewbroccathere's more new config stuff in Heat now12:25
hewbroccacloud config12:25
*** e0ne has joined #tripleo12:25
chmouelyeah saw the software-config thing, and that's where my question were coming from, if that's tighten to the os-*-config tools12:26
hewbroccait's not12:26
hewbroccathe os-* tools are tripleo specific12:26
chmouel(i indeed meant OS::Config::CloudConfig)12:27
chmouelok12:27
hewbroccaso, yeah, definitely worth checking with Heat guys for the latest12:28
chmouelwould you be able to use CloudConfig directly without the os-*-config?12:28
hewbroccaI *think* so12:28
hewbroccabut talk to the heat guys12:28
*** e0ne_ has quit IRC12:28
*** jcoufal has quit IRC12:30
chmouelcool, okay thanks a lot, i guess i have to test it to see how stuff fits together in real12:30
*** jcoufal has joined #tripleo12:30
hewbroccano problem12:31
hewbroccaSteve Baker or Steve Hardy will be delighted to guide you through12:31
hewbroccathey're excited to get folks using this stuff I think12:31
*** geerdest has joined #tripleo12:36
*** geerdest has joined #tripleo12:36
*** dprince has joined #tripleo12:41
*** lucasagomes is now known as lucas-hungry12:44
openstackgerritTrent Geerdes proposed a change to openstack/diskimage-builder: Adding pypi-mirror dependencies for redhat  https://review.openstack.org/8251312:44
*** matty_dubs|gone is now known as matty_dubs12:46
*** rpodolyaka1 has joined #tripleo12:48
*** hashar has joined #tripleo12:49
*** rpodolyaka1 has quit IRC12:52
*** weshay has joined #tripleo13:05
*** e0ne_ has joined #tripleo13:07
*** matsuhas_ has quit IRC13:07
*** CaptTofu has quit IRC13:07
*** mrunge has quit IRC13:08
*** CaptTofu has joined #tripleo13:08
*** CaptTofu has quit IRC13:09
*** CaptTofu has joined #tripleo13:09
*** e0ne has quit IRC13:10
*** CaptTofu has quit IRC13:14
*** CaptTofu_ has joined #tripleo13:14
*** julim has joined #tripleo13:16
*** yamahata has joined #tripleo13:18
SpamapShewbrocca: you got a few details wrong in relaying to chmouel how os-*-config works13:27
SpamapSchmouel: ^^13:27
hewbroccaSpamapS: LOL I'm confident that is the case13:28
SpamapSos-collect-config gathers information from multiple sources13:28
hewbroccaI hope I didn't completely butcher them...13:28
SpamapSec2 metadata, /var/lib/heat-cfntools/cfn-init-data (which is written at the request of Heat by cloud-init), and the CFN API of Heat.13:28
SpamapSI think its time I make a picture actually :)13:29
chmouelSpamapS: heh thanks, yeah that would be very nice for newcommer as this is the confusing part of tripleO :)13:29
SpamapShewbrocca: os-refresh-config is really really dumb, and just runs like run-parts through 4 dirs, /opt/stack/os-config-refresh/{pre-configure.d,configure.d,post-configure.d,migration.d}13:29
chmouelSpamapS: is the multiple sources very important in the context of tripleO ?13:29
hewbroccaSpamapS: picture worth a thousand words, yes...13:30
SpamapSchmouel: yes, we use them all13:30
SpamapSwhenever any of them change (as determined by comparing to the previously known version), the command is run with an environment variable pointing to the new metadata.13:30
SpamapSthe command == os-refresh-config in our scenario13:31
chmouelSpamapS:   ah ok i see, that's because there is no a unifomize way at the moment to communicate with instance among openstack service ?13:31
SpamapSbut could very well be puppet, chef, or "your magical configuration machine of choice"13:31
SpamapSchmouel: there is, but it is through ec2 metadata which has.. some issues. ;)13:32
chmouelhehe yeah i mean openstack services in an native openstack way ;)13:32
*** yassine_ has joined #tripleo13:32
*** noslzzp has quit IRC13:34
*** nosnos has quit IRC13:35
hewbroccaSpamapS: OK, so I had the basic idea right, I just messed up the delivery mechanism13:37
hewbroccaI didn't realize there were 4 of them...13:37
* hewbrocca boggles13:37
*** Rakesh5 has quit IRC13:39
Ngmorning13:44
*** killer_prince has quit IRC13:46
*** lazy_prince is now known as killer_prince13:46
*** rpodolyaka1 has joined #tripleo13:49
*** lucas-hungry is now known as lucasagomes13:50
openstackgerritA change was merged to openstack/tripleo-image-elements: Install python-keystoneclient in the swiftclient venv  https://review.openstack.org/7593213:52
*** rpodolyaka1 has quit IRC13:53
*** noslzzp has joined #tripleo13:56
openstackgerritThom Leggett proposed a change to openstack/diskimage-builder: Uses policy-rc.d to prevent dpkg starting daemons  https://review.openstack.org/8207413:56
Gonerichmouel, hewbrocca, SpamapS: Do I get the idea right? http://people.debian.org/~goneri/stuff/os-refresh-config-flow.pdf14:02
ifarkasslagle, ping14:04
lsmola_SpamapS: hello14:07
*** rdopieralski has quit IRC14:07
lsmola_SpamapS: so seems to me like there is not an easy way around https://bugs.launchpad.net/tripleo/+bug/129190514:07
uvirtbotLaunchpad bug 1291905 in tripleo "new resources are not available for signalling during an update -> heat stack update timeout" [High,In progress]14:07
lsmola_SpamapS: as the os-collect-config relies also on non-existent resource14:08
lsmola_SpamapS: i could hack it around by explicitly sending the signal defined in the template, like shardy advised me http://paste.fedoraproject.org/88017/60519139/14:10
lsmola_SpamapS: but then the completion condition would say: os-apply-config was finished14:10
lsmola_SpamapS: which is probably what we want14:10
lsmola_SpamapS: what do you htink?14:10
*** lblanchard has quit IRC14:12
*** lblanchard has joined #tripleo14:13
*** CaptTofu_ has quit IRC14:15
openstackgerritLadislav Smola proposed a change to openstack/tuskar: Adding swift role back  https://review.openstack.org/8249914:15
*** CaptTofu has joined #tripleo14:16
openstackgerritA change was merged to openstack/diskimage-builder: Adding pypi-mirror dependencies for redhat  https://review.openstack.org/8251314:17
*** CaptTofu has quit IRC14:20
lsmola_ifarkas: jistr guys can you look at https://review.openstack.org/#/c/81200/14:33
jistrlsmola_: does it pick nodes with less RAM but only those with *at least as much RAM as the flavor specifies*?14:35
lsmola_jistr: yes14:36
lsmola_jistr: the filter is unchanged14:36
*** e0ne has joined #tripleo14:36
lsmola_jistr: this just sets Weight, so basically what to pick first14:36
jistrright. cool, thx.14:37
lsmola_jistr: but only from the set of already filleted machines :-)14:37
ifarkaslsmola_, do you have a bug against nova for this^?14:37
lsmola_ifarkas: yes, the one it partially fix14:38
lsmola_ifarkas: the wanted solution is to create exact_match filter14:38
lsmola_ifarkas: and put that to nova.conf14:38
ifarkaslsmola_, cool, thx14:39
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-dev and mariadb-dev elements  https://review.openstack.org/7988614:39
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mariadb element  https://review.openstack.org/7988414:39
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-common element  https://review.openstack.org/7988514:39
*** e0ne_ has quit IRC14:39
openstackgerritLadislav Smola proposed a change to openstack/tuskar: Adding swift role back  https://review.openstack.org/8249914:43
lsmola_ifarkas: jistr and please also this, tuskar is currently broken https://review.openstack.org/#/c/82499/14:44
derekhIs it just me or is devtest fedora now completely broken ? on the undercloud I got eth0 with e1000 and ens4 with virtio , non of these if em1 https://github.com/openstack/tripleo-incubator/blob/master/scripts/devtest_undercloud.sh#L3914:45
jistrlsmola_: i'll try at some point, but currently i'm being pulled in different hi-prio directions :(14:45
lsmola_derekh: it was broken yesterday AFAIK, akrivoka tried it14:45
lsmola_derekh: I will try today14:45
openstackgerritA change was merged to openstack/tripleo-image-elements: Setting negative RamWeigher  https://review.openstack.org/8120014:45
*** mestery has joined #tripleo14:46
akrivokayes, I tried multiple times, each time it timed out while waiting for the undercloud heat stack14:47
*** mclaren has joined #tripleo14:48
lsmola_akrivoka: any log errors?14:48
*** rpodolyaka1 has joined #tripleo14:49
derekhdprince: was https://review.openstack.org/#/c/72003/4 intended to set net devices to em* ?14:49
*** CaptTofu has joined #tripleo14:50
derekhlsmola_: thanks14:51
dprincederekh: yes. But there was a fix to that because I messed up.14:51
dprincederekh: https://review.openstack.org/#/c/82093/14:52
akrivokalsmola_: unfortunately I wiped my env, but I will try again today14:52
*** hashar has quit IRC14:52
dprincederekh: both of those should be landed now though14:52
derekhdprince: I seem to have that patch aswell14:52
lsmola_akrivoka: ok, I will try14:53
dprincederekh: this too? https://review.openstack.org/#/c/82173/14:53
*** rpodolyaka1 has quit IRC14:53
derekhMar 26 12:53:32 localhost systemd-udevd[219]: renamed network interface eth0 to ens414:53
lsmola_dprince: cool14:53
derekhdprince: no I don't have that last one, will try it now14:54
dprincederekh: there is a race without that14:54
derekhlooks like I was betting on the slow horse14:54
dprincederekh: now you've got me thinking I forgot something else? Man, we need Fedora in the CI...14:56
derekhdprince: agreed we do, the commits to add it are all up there :-(14:56
dprincederekh: That udev rules rename was a bad link. I thought it was something else. You won't hit any issues with that.14:58
dprincederekh: So disregard my link to 8217314:58
derekhdprince: okie dokie14:58
dprincederekh: what issues are you seeing?14:58
dprincederekh: presumably you just aren't seeing em114:59
derekhdprince: my undercloud came up with ens4 instead of em114:59
dprincederekh: oh...14:59
dprincederekh: so biosdevname is working then.14:59
dprincederekh: man, I hate VMs for baremetal testing15:00
*** julim_ has joined #tripleo15:00
dprincederekh: this is why the Fedora cloud image uses eth015:00
*** julim has quit IRC15:01
derekhdprince: I also tried with virtio and the device was eth0 (still not em1)15:01
*** fungi has joined #tripleo15:03
*** yamahata has quit IRC15:03
dprincederekh: well that might be something different15:03
*** fungi has left #tripleo15:03
dprincederekh: so you've got this https://review.openstack.org/#/c/81596/15:03
dprincederekh: basically, just wanting to know if everything is up to date15:03
dprincederekh: there is nothing unlanded that you should need (that I know of)15:04
dprincederekh: The ens4 interface name means it is working. And switching the virtio type would have been my suggestion.15:05
*** mestery has quit IRC15:05
*** weshay has quit IRC15:05
*** yamahata has joined #tripleo15:05
*** hashar has joined #tripleo15:06
*** gcha has quit IRC15:07
dprincederekh: The complication here is we now require the 'baremetal' element in devtest for other reasons (extracting kernel and ramdisk). So I can't just remove it... but I could split out the stable interfaces stuff into another element... which is actually what I initially did but during review it was decided living under the 'baremetal' element is best.15:08
dprincederekh: Just sayin, if we need to step back from this for VM testing we can.15:08
*** yamahata has quit IRC15:09
*** yamahata has joined #tripleo15:10
*** UtahDave has joined #tripleo15:13
derekhdprince: sorry was on phone, ya I'm uptodate I have a diskimage-builder from this morning, so if we split it out were would need a different element set for real baremetal and virt?15:15
SpamapSare vms unhappy w/ biosdevname changes?15:15
dprincederekh: yes, and is why I sort of disliked adding this to baremetal... cause it is used for other required things that aren't really baremetally anyway15:16
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Alphabetize all element-deps  https://review.openstack.org/8163515:17
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Add dib-lint script  https://review.openstack.org/8163415:17
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: WIP: Make sure all scripts are set -e  https://review.openstack.org/8163715:17
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Enforce alphabetical ordering of element-deps  https://review.openstack.org/8163615:17
derekhSpamapS: devtest fedora net device name is now eth0 (e1000) or ens4 (virtio), devtest is looking for em115:17
dprincederekh: Like I said, with virtio the ens4 means it is working. We could just default devtest to use that.15:18
bnemecGrr, pushed the wrong branch.15:18
derekhSpamapS: while your there you may want to resubmit the tmpfs change it was merged this morning and then reverted (as it brought us back to a 4G tmpfs and CI started failing)15:18
*** lsmola_ has quit IRC15:19
*** markmc has joined #tripleo15:19
derekhdprince: but virtio isn't the default in devtest we need the default to work, lifeless doesn't want to make that switch until we have tested are full matrix of support distros with it15:20
derekhdprince: which I could do15:20
derekhgiven time15:20
*** UtahDave has left #tripleo15:20
*** weshay has joined #tripleo15:21
derekhso I'll need to check virtio works with F20, ubuntu precise and raring on both F20 and Ubuntu Hosts, 6 runs I think15:23
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Alphabetize all element-deps  https://review.openstack.org/8163515:23
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Add dib-lint script  https://review.openstack.org/8163415:23
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: WIP: Make sure all scripts are set -e  https://review.openstack.org/8163715:23
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Enforce alphabetical ordering of element-deps  https://review.openstack.org/8163615:23
bnemecI really hosed up that series of commits. :-/15:24
bnemecShould be fixed now though.15:24
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Add clarification on required permissions  https://review.openstack.org/8310215:24
dprincederekh: I think the quickest thing to unbreak it but keep the feature is to split this out into a new element.15:25
dprincederekh: I say lets go with that which should be much less disruptive to everyone I think. And then we can revert my change to devtest that landed yesterday too15:26
derekhdprince: ok, in the meantime I'm gonna test virtio and see what happens15:26
dprincederekh: cool. Sorry for the disruptions man15:26
derekhdprince: no prob, needed a poke to finally get the virt io work done anyways15:27
slagledprince: if you're going to be poking around in there, i'm seeing that all the dhcp-interface@.services for the generated interfaces are coming up as failed15:27
dprinceslagle: oh?15:28
slagledprince: the ExecStartPre=/usr/local/sbin/dhcp-all-interfaces.sh %I, does not exit non-zero15:28
slaglefor the generated interfaces15:28
slagleso then the ifup command still runs, and fails15:29
slagledoesn't seem to cause any issue, fwiw15:30
slaglebut does not look very nice in "systemctl --failed"15:30
dprinceslagle: which node?15:30
slagledprince: any of them15:31
slaglei see it on control, compute, blockstorage, and swiftstorage15:31
* dprince wants to log off IRC now and go back to bed15:32
*** lsmola_ has joined #tripleo15:34
*** vkozhukalov has quit IRC15:34
dprinceslagle: so systemd is very aggressive about starting things as soon as it sees an interface.15:35
dprinceslagle: udev rule fires off, tags teh systemd device, which causes systemd to startup that interface (or try to)15:35
dprinceslagle: I wonder if dhcp-all-interfaces.sh is timing out for beeing too slow15:36
*** ifarkas has quit IRC15:36
*** IvanBerezovskiy has left #tripleo15:37
dprinceslagle: I'll mess w/ it as best I can. The new way of doing things (using a udev rule to instantiate the systemd service) is much cleaner IMO. But we may need to tweak the service or dhcp-all-interfaces.sh script a bit...15:37
*** petertoft has joined #tripleo15:39
dprincederekh: have you tried getting a substring w/ the jq version in Fedora?15:39
* dprince needs to slice and dice the json15:40
*** ifarkas has joined #tripleo15:40
derekhdprince: nope, I haven't tried much jq vodoo much at all15:40
dprincederekh: who is our resident expert then?15:40
derekhdprince: lifeless and StevenK  seem to have read the manual15:41
dprinceStevenK: around?15:41
derekhdprince: probably not, I think its night in his TZ15:41
dprincederekh: okay, well I read the man page on Fedora and we don't have a couple functions that upstream jq has... makes me sad but I'll figure something out15:45
derekh:-(15:45
*** spzala has joined #tripleo15:46
SpamapSdprince: I've done a bit of jq15:46
SpamapSdprince: whats up?15:46
dprinceSpamapS: http://paste.openstack.org/show/74366/15:48
*** ifarkas has quit IRC15:48
dprinceSpamapS: So I reformatted the jq in devtest_seed.sh so I can inject the baremetal-network JSON into it (for example if I don't want to use 192.0.2.0/24)15:49
dprinceSpamapS: All is well really, except on line 1415:49
dprinceSpamapS: in Fedora we have JQ 1.3 and that last [-3:3] is meant to slice off the cidr netmask15:50
*** rpodolyaka1 has joined #tripleo15:50
dprinceSpamapS: What I get is jq: error: Cannot index string with object15:50
dprinceSpamapS: I've tinkered around a bit, it looks like upstream has a toarray function and has also added some more capabilities to slice/dice maybe. We just don't have those functions yet.15:51
dprinceSpamapS: So the question is am I missing something? Or... should I just bang in an individual cidr/netmask parameter and be done w/ it.15:52
*** rpodolyaka1 has quit IRC15:52
SpamapSdprince: hm15:53
SpamapSdprince: It is a bit lazy to just slice it with [-2:3]15:53
SpamapSsince /8 will not work15:53
SpamapSerr [-3:3]15:53
dprinceSpamapS: good point, gotta start somewhere though15:54
SpamapSso I think adding it as a field is a good way to go unless there is a split function where we can split on /15:54
dprinceSpamapS: heh. And I thought the lazy approach was to add the extra field.15:54
dprinceSpamapS: FWIW the split function is what I really wanted (everything after the /)15:55
dprinceSpamapS: but thanks for the confidence man15:56
SpamapSdprince: well you could also defer to shell for that15:56
SpamapSjq is just for parsing the json IMO15:56
SpamapSdoing more than that with it is fun, but kind of evil15:56
SpamapS"do one thing"15:56
dprinceSpamapS: right, would significantly dirty the jq statement though as we'd need to switch from '' to "" and I'd have to escape like mad15:57
tteggelci poorly? lots of hudson.remoting errors.15:57
dprinceSpamapS: Well, it appears jq already crossed that bridge...15:57
dprinceSpamapS: we just don't have it yet15:57
SpamapSI don't think it is a good idea to lean on it for that.15:57
SpamapSI get _why_ it has it15:58
*** matty_dubs is now known as matty_dubs|lunch15:58
SpamapSbut to me if jq expressions get more complicated than pulling out keys.. they're reaching too far.15:58
*** lazy_prince has joined #tripleo15:58
dprinceSpamapS: fair enough, I've gotten quite a bit of pushback in TripleO when adding new params though... so go figure :)15:58
*** killer_prince has quit IRC15:59
*** lazy_prince is now known as killer_prince15:59
dprinceSpamapS: funny how we switch sides on things sometimes :)15:59
dprinceSpamapS: lifeless wasn't a big fan of splitting the network cidr off FWIW16:00
dprinceSpamapS: Already tried it in the ensure-bridge thing... even though all the distro scripts use it sepearately16:00
*** mestery has joined #tripleo16:01
*** eghobo has joined #tripleo16:04
*** eghobo has quit IRC16:04
*** pblaho has quit IRC16:04
*** eghobo has joined #tripleo16:05
*** eghobo has quit IRC16:05
*** eghobo has joined #tripleo16:06
*** lsmola_ has quit IRC16:06
SpamapSdprince: we should probably stop smoking crack if we want consistency.. but.. whats the fun in that?16:06
SpamapShewbrocca: https://wiki.openstack.org/wiki/OsCollectConfig16:07
SpamapSchmouel: https://wiki.openstack.org/wiki/OsCollectConfig16:07
SpamapSpictures!16:07
chmouelSpamapS:  woo woo awesome!16:07
chmouelGoneri: ^16:07
Gonerioutch :)16:07
GoneriSpamapS: thanks16:08
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/os-collect-config: Add diagram to os-collect-config README.rst  https://review.openstack.org/8311516:13
SpamapSanybody know a quick way to render rst -> HTML the same way the markdown command does?16:14
slaglei wish i did16:14
slaglei admit to using the github live edit as a "scratch space"16:14
SpamapSslagle: hah nice :)16:14
*** weshay has quit IRC16:14
SpamapSmaybe I should just fork os-collect-config on github and push my changes into that16:15
*** weshay has joined #tripleo16:15
slaglewell, it's got a Preview button16:15
slagleso you can edit, preview, w/o committing16:15
*** killer_prince has quit IRC16:17
SpamapShm, github doesn't seem to like the .. image:: syntax16:18
*** killer_prince has joined #tripleo16:18
clarkbSpamapS: rst2html?16:20
*** lsmola_ has joined #tripleo16:20
dprinceSpamapS: a simple jq --arg did the trick (/me has his variable and avoids adding a new parameter)16:20
*** e0ne_ has joined #tripleo16:21
SpamapSdprince: not sure I understand but I look forward to your patch.. ?16:22
dprinceSpamapS: cool, I munged a variable in bash, and then can pass it into jq w/ --arg16:23
*** e0ne has quit IRC16:24
*** boris-42 has quit IRC16:24
SpamapShrm, looks like github won't do any fancy tricks to let you use a non-absolute link to their raw image URL in the README16:25
SpamapSclarkb: thanks.. so simple ;)16:26
openstackgerritRyan Moore proposed a change to openstack/tripleo-image-elements: Allow setting snapshots image format and directory  https://review.openstack.org/8312216:27
*** bauzas has quit IRC16:29
*** epim has joined #tripleo16:34
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Write network configuration into seeds config.json  https://review.openstack.org/8312516:35
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Don't hard code the baremetal seed IP in seedrc  https://review.openstack.org/8312616:35
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Make the baremetal-network configurable  https://review.openstack.org/8232716:35
*** rlandy has quit IRC16:35
*** boris-42 has joined #tripleo16:36
openstackgerritPeter Belanyi proposed a change to openstack/tuskar-ui: Fixed a few typos in runtest.sh  https://review.openstack.org/8312716:36
jogoderekh: I had trouble using graphite for that as well16:37
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Drop dnsmasq_range from the seeds config.json.  https://review.openstack.org/8312816:38
jogoderekh: I just looked at the jenkinses https://jenkins02.openstack.org/job/check-tripleo-undercloud-precise/buildTimeTrend16:38
derekhjogo: ya, I'm trying to get that consolidates all the jenkins servers together, guess I'll have to dig a bit more16:40
derekhjogo: do you know if there is any precedence for ci jobs sending their own statsd stats?16:40
jogoderekh: ask in #infra16:40
derekhwill do, thanks16:41
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Drop dnsmasq_range from the seeds config.json.  https://review.openstack.org/8312816:41
bnemecSpamapS: http://www.tele3.cz/jbar/rest/rest.html ?16:41
openstackgerritDan Prince proposed a change to openstack/tripleo-heat-templates: Drop dnsmasq_range from the undercloud source.  https://review.openstack.org/8313016:42
jogoderekh: I was never sure how to get a good time measurement on these things myself16:43
derekhjogo: ok16:45
*** pbelanyi has quit IRC16:47
*** hashar has quit IRC16:48
*** mestery has quit IRC16:49
smulcahyI just logged https://bugs.launchpad.net/tripleo/+bug/1297948 - happy to discuss here, I think its a critical16:49
uvirtbotLaunchpad bug 1297948 in tripleo "tripleo devtest fails sometimes when running /tmp/in_target.d/install.d/01-install-pip" [High,Triaged]16:49
smulcahyit makes using pypi mirrors and proxy for tripleo devtest totally unreliable16:49
*** rpodolyaka1 has joined #tripleo16:50
*** vkozhukalov has joined #tripleo16:51
GheRiveroI think i fix it by export PIP_INDEX_URL=$PYPI_MIRROR_URL16:52
GheRiverobut have to test in the server, once the current tripleo run finish16:52
*** matty_dubs|lunch is now known as matty_dubs16:53
*** saurabhs has joined #tripleo16:54
SpamapSbnemec: thanks, but I needed something that I can put in the same web root16:54
GheRiverosmulcahy: ok. just test it ad that fix it.16:54
*** rpodolyaka1 has quit IRC16:55
bnemecSpamapS: Ah, gotcha.  I had just used that a couple of times to verify doc output on changes.16:56
smulcahygherhivero: I can give that a try16:58
smulcahyI'd like someone to explain what that fixes and why its neccesary though16:59
*** nati_ueno has joined #tripleo17:00
*** rollerj__ has quit IRC17:02
*** mestery has joined #tripleo17:09
*** jistr has quit IRC17:10
*** e0ne_ has quit IRC17:15
smulcahyI suspect the bug I just raised https://bugs.launchpad.net/tripleo/+bug/1297970 may be related to the use of pypi mirrors and/or wheels and maybe related to https://bugs.launchpad.net/tripleo/+bug/1297948 also17:23
uvirtbotLaunchpad bug 1297970 in tripleo ""No local packages or download links found for six>=1.4.1" error when using a pypi mirror" [Undecided,New]17:23
*** cwolferh has joined #tripleo17:24
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Remove extra TMP variable from devtest_seed.sh  https://review.openstack.org/8313917:41
GheRiverosmulcahy: I just remember the cause of your first problem17:42
GheRiverothe images are built with the pypi element, but somehow, it cannot find the env variable to the mirror, so use the local cache (~/.cache/image-create/pypi) for it17:43
GheRiveroin my case was a misspelled PYPI_MIRROR instead of PYPI_MIRROR_URL17:44
openstackgerritA change was merged to openstack/diskimage-builder: Add dib-lint script  https://review.openstack.org/8163417:44
openstackgerritA change was merged to openstack/diskimage-builder: Alphabetize all element-deps  https://review.openstack.org/8163517:45
openstackgerritA change was merged to openstack/diskimage-builder: Enforce alphabetical ordering of element-deps  https://review.openstack.org/8163617:45
*** petertoft has quit IRC17:46
bnemec\o/17:47
*** saurabhs has quit IRC17:50
*** morganfainberg_Z is now known as morganfainberg17:50
*** rpodolyaka1 has joined #tripleo17:51
*** epim has quit IRC17:54
*** rpodolyaka1 has quit IRC17:56
slaglebnemec: i had my +A finger going there for a moment...17:58
bnemecslagle: Indeed, thanks :-)17:58
*** epim has joined #tripleo18:00
*** saurabhs has joined #tripleo18:00
greghayneslifeless: in your comment on https://bugs.launchpad.net/tripleo/+bug/1295732 you have PYPI_MIRROR_URL as the non-wheel mirror and _1 as the wheel mirror, that seems backwards to me?18:01
uvirtbotLaunchpad bug 1295732 in tripleo "tripleo devtest overcloud startup takes too long " [High,Triaged]18:01
*** athomas has quit IRC18:02
*** lucasagomes is now known as lucas-afk18:02
*** derekh has quit IRC18:04
*** jcoufal has quit IRC18:07
* SpamapS lunches18:10
SpamapSbtw for those that missed it.. I made a pretty: https://wiki.openstack.org/wiki/OsCollectConfig18:11
greghaynesoo such pretty18:12
*** martyntaylor has left #tripleo18:15
lifelessgreghaynes: and yet, thats what I have18:19
lifelessgreghaynes: I reported what I'm using, not what I should be using :018:19
greghayneshah18:19
greghaynesI mentioned that I was surprised you have numbers to back up your settings18:19
lifelessSpamapS: so the reason I wanted the cidr is that its much harder to get skew in the config18:20
lifelessSpamapS: and we foot gunned ourselvs repeatedly without it18:20
lifelessdprince: ^18:21
dprincelifeless: no foot gunning here. I'm just questioning a config that might should get split up because most of the places that uses it need individual peicies (not the whole)18:22
dprincelifeless: in any case I've got my jq issue sorted so we can skip it for now18:23
dprincelifeless: it was the ensure-bridge change the I really wanted to change this for anyways. But that still hasn't landed yet18:23
lifelessack18:29
lifelessdprince: btw passing in a new network ++18:29
*** vkozhukalov has quit IRC18:29
dprincelifeless: :) Hasn't made it guantlet yet but it worked for me locally... (perhaps I messed it up before submitting it)18:30
*** e0ne has joined #tripleo18:30
*** spzala has quit IRC18:41
openstackgerritJon-Paul Sullivan proposed a change to openstack/diskimage-builder: Write a dpkg manifest to list installed packages  https://review.openstack.org/8317118:46
hewbroccaBTW -- marios_ made a couple of tripleo-in-action videos yesterday18:49
hewbroccahttps://www.youtube.com/watch?v=Ranfkx34dhg shows using the UI setting up the overcloud18:49
lifelessdprince: I want to land https://review.openstack.org/#/c/78467/ today - its blocking for Ironic18:50
hewbroccaand https://www.youtube.com/watch?v=s2DAngZ8__E18:50
lifelessdprince: what tests do you need done?18:50
hewbroccashows using the the UI to launch it18:50
dprincelifeless: I just haven't switched over to actually use 'nodes' yet.18:51
dprincelifeless: should be easy to do, just wanted to kick the tires on it first18:51
lifelessdprince: can you do that today?18:51
*** rpodolyaka1 has joined #tripleo18:52
lifelessdprince: my goal is to get Ironic useful check results from toci asap so they stop breaking :)18:52
dprincelifeless: I'll try18:52
lifelessdprince: thanks18:53
lifelessslagle: SpamapS: https://review.openstack.org/#/c/82741/ has all the requested changes and passed CI18:54
*** rpodolyaka1 has quit IRC18:56
*** rpodolyaka1 has joined #tripleo18:57
lifelessGheRivero: ^18:59
*** epim has quit IRC19:01
greghaynesI love when I leave a long review comment only to see that it was addressed in the next file in the review19:03
lifeless:)19:03
greghaynesI left a nice one about "why dont we make a userguide?" in the readme19:03
lifelessgreghaynes: thats a great idea19:03
*** spzala has joined #tripleo19:08
*** epim has joined #tripleo19:10
*** jp_at_hp has quit IRC19:20
*** andreaf has quit IRC19:34
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Make it possible to pass real hardware in.  https://review.openstack.org/8318819:34
lifelessdprince: ^ may be useful to you19:34
dprincelifeless: cool, I may come in handy sometime, I typically don't call the top level devtest.sh however19:36
*** spzala has quit IRC19:38
*** rdopieralski has joined #tripleo19:39
*** rdopieralski has quit IRC19:43
lifelessdprince: it also lets you pass --nodes to devtest_testenv.sh too19:44
SpamapSlifeless: /win 819:50
jprovazngreghaynes, Hi, are you around?19:50
SpamapSdoh19:50
SpamapSlifeless: ACK19:50
greghaynesjprovazn: Hey19:50
openstackgerritColeman Corrigan proposed a change to openstack/tripleo-incubator: Add set-pip-vars script to simplify pip manifests  https://review.openstack.org/8319419:51
jprovazngreghaynes, https://review.openstack.org/#/c/79884/4/elements/mariadb/install.d/10-mariadb - you mean add rhel-based check because of percona-toolkit package is being installed or something else?19:51
lifelessGheRivero: Ng: hey so hp build perf19:52
greghaynesjprovazn: because of the mariadb-galera19:52
lifelessGheRivero: Ng: I note 600 seconds to build a ramdisk; thats 10x slower than for me: Can we fix that?19:52
*** hashar has joined #tripleo19:52
greghaynesjprovazn: I assume theres really no future in that line succeeding in !redhat packages, right?19:53
Nglifeless: orly, where are you seeing 600s?19:53
jprovazngreghaynes, the mariadb-galera pkgs are available once mariadb.org repo is added - this is done in mariadb-dev pre-install script19:54
greghaynesah, I was looking for something like that but forgot about the patch dependency19:55
* greghaynes rechecks19:55
jprovazngreghaynes, no worries - you remind me I should check package names for ubuntu19:56
greghaynesYep, I was just doing that :)19:56
jprovazn:) great19:56
greghaynesjprovazn: looks like they are 'mariadb-galera-server galera' on ubuntu19:58
jprovaznyea19:58
jprovaznwill add mapping19:59
jprovaznactually, IIRC yum should be case insensitive19:59
greghayneshaha20:00
jprovaznand it's not20:01
lifelessNg: https://bugs.launchpad.net/tripleo/+bug/129573220:01
uvirtbotLaunchpad bug 1295732 in tripleo "tripleo devtest overcloud startup takes too long " [High,Triaged]20:01
lifelessNg: comment 720:02
*** jang1 has joined #tripleo20:03
*** julim_ has quit IRC20:03
*** jang1 has quit IRC20:04
*** jang1 has joined #tripleo20:04
Nglifeless: attempting to reproduce20:05
NgI wonder if this is the recent trend of overusing things like NODE_DIST, causing weird elements to leak into the ramdisk build, but I have no evidence of anything yet :)20:06
*** e0ne has quit IRC20:07
*** bauzas has joined #tripleo20:10
lifelessNg: indeed thats possible20:10
lifelessNg: OTOH 10 minutes sounds like either downloading a new base ubuntu image or some such cruft to me20:11
Ngpossibly20:11
Nga run on the lab hardware that has minimal elements ("ubuntu deploy") and no http proxy, but that didn't need to fetch a new base image, just finished after 4m17s20:12
lifelessGheRivero: when you say -u in NODE_DIST doesn't work... what happens?20:12
lifelessNg: so thats 4 times slower still20:13
openstackgerritJan Provaznik proposed a change to openstack/diskimage-builder: Adds package mapping for mariadb packages  https://review.openstack.org/8319920:13
*** epim has quit IRC20:13
lifelessNg: is it using a local mirror ?20:13
Nglifeless: no, I wanted to start from the very basics and add things one at a time to see where it gets better or worse20:14
lifeless++20:14
Nglifeless: even going through the http proxy, fetching the cloud image is never getting even dangerously close to 500KB/s20:15
NgI'd guess the average is about 46520:15
lifelessNg: well you know whats serving that :P20:16
lifelessNg: (so you're in a better place than I to comment on whether thats expected)20:16
Nglifeless: that is not the ubuntu.com servers :)20:17
NgI'm in the same city as them and I can max out my line20:17
Ngaverage of 11.6MB20:17
Ng18 seconds to grab the cloud image :)20:17
*** yassine_ has quit IRC20:18
lifelessNg: random thought, mayeb the office egress to internet is via NY or something20:19
lifelessNg: anyhow, why is it downloading the base image again? versimilitude?20:19
Nglifeless: it wasn't doing so before, but you made the good point that it might be that, so I forced the issue :)20:21
Ngbut the cloud image was updated about 7 hours ago, going by the mtime20:21
*** rpodolyaka1 has quit IRC20:21
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-dev and mariadb-dev elements  https://review.openstack.org/7988620:22
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mariadb element  https://review.openstack.org/7988420:22
SpamapSFor me, cloud-images.ubuntu.com is always a _lot_ slower than archive.ubuntu.com20:22
SpamapSPerhaps we should mirror it into hp cloud's swift20:22
Nglifeless: anyway, if this is a consistent internet speed that I'm seeing, we can rule this out, because the total time for downloading a cloud image to these lab machines appears to be 16 minutes20:22
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-dev and mariadb-dev elements  https://review.openstack.org/7988620:22
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mariadb element  https://review.openstack.org/7988420:22
openstackgerritJan Provaznik proposed a change to openstack/tripleo-image-elements: Adds mysql-common element  https://review.openstack.org/7988520:22
lifelessdprince: so https://review.openstack.org/#/c/78467/ has now passed full CI; is there any chance you can migrate to --nodes now ? I hate to pressure you, but getting Ironic good testing - well, its been a long road.20:23
SpamapSNg: just for giggles, try ssh -D to a box you know is fast, and then bounce through it using tsocks20:23
lifelessNg: right, so 10m is < 16m.20:23
Nglifeless: I'm not going to totally rule it out, because maybe the lab internet is faster at other times, but right now it's not an obvious contender20:24
lifelessNg: shockingly scientific thinking there20:24
dprincelifeless: go for it, if it breaks me I'll fix it20:25
lifelessdprince: gentleman and a scholar@20:25
lifelessbah, s/@/!/20:25
*** jprovazn is now known as jprovazn_afk20:25
Nglifeless: we shall be victorious through the judicious application of science :)20:25
openstackgerritA change was merged to openstack/tripleo-incubator: Remove backwards compatability in setup-baremetal  https://review.openstack.org/7846720:27
*** mrunge has joined #tripleo20:27
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Update documentation.  https://review.openstack.org/8274120:30
lifelessEWAT20:31
lifeless2014-03-26 05:29:04.980 |   File "/opt/stack/venvs/keystone/local/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 13, in <module>20:31
lifeless2014-03-26 05:29:04.981 |     from netaddr.strategy import ipv4 as _ipv4, ipv6 as _ipv620:31
lifeless2014-03-26 05:29:04.981 | ValueError: bad marshal data (unknown type code)20:31
lifelesshttp://logs.openstack.org/51/79051/7/check-tripleo/check-tripleo-overcloud-precise/49d7695/console.html20:31
lifelesseven weirder20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper   File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/cmd/baremetal_deploy_helper.py", line 167, in notify20:33
*** testing__ has joined #tripleo20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper     s.connect((address, port))20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 194, in connect20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper     socket_checkerr(fd)20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper   File "/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 46, in socket_checkerr20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper     raise socket.error(err, errno.errorcode[err])20:33
lifeless2014-03-26 15:27:25.292 4470 TRACE nova.virt.baremetal.deploy_helper error: [Errno 111] ECONNREFUSED20:34
lifelesswe're starting to see lovely corner cases here :)20:34
*** rpodolyaka1 has joined #tripleo20:38
*** julim has joined #tripleo20:45
Nginteresting, curl was predicting badly, the actual download time was 12 minutes, which is by far the worst case I've been able to induce so far. Interestingly, the best case is 1m21s20:46
Ngthat's with local ubuntu and pypi mirrors20:46
lifelessNg: 1m21 to build it?20:46
Nglocal to the lab network, I mean20:46
lifelessso 80s vs 60, thats tolerable20:47
Nglifeless: just for the raw diskimage-builder command to build a deploy ramdisk20:47
lifelessNg: could you use 'time testenv_ramdisk.sh' ?20:47
lifelessNg: that should get the same thing the stuff in the bug measures20:47
*** testing__ has quit IRC20:47
*** cwolferh has quit IRC20:48
*** cwolferh has joined #tripleo20:48
Nglifeless: I will do so, but there are still things like NODE_DIST/DIB_COMMON_ELEMENTS that could be affecting it20:48
*** rpodolyaka1 has quit IRC20:50
lifelessNg: indeed20:51
lifelessq20:51
lifelesswoo I68d46c4da8715df03c3a88393b55665dc57045a3 has landed20:51
lifelessnow the tie patch and incubator and we have ironic ... wheeeee20:51
Nglifeless: 1m41s20:52
SpamapSsweeeeeeeeeeet20:52
lifelessSpamapS: assuming it actualy, yknow, works20:53
SpamapSI ALWAYS assume that20:53
SpamapSAmerican Optimism™20:53
openstackgerritJonathan Brownell proposed a change to openstack/tripleo-image-elements: Update the "arch" value in config.json to match $ARCH  https://review.openstack.org/8321120:55
*** weshay has quit IRC20:56
Nglifeless: I have posted my numbers and a request for some settings info, to the bug20:56
greghaynessomehow I made a heat template that heat stack-create accepted, but causes the cfn api to return http://paste.ubuntu.com/7159007/ when called from os-collect-config20:56
greghaynesas a side note, only way I found that error message is via tcpdump which seemed less than ideal20:56
*** e0ne has joined #tripleo20:57
greghaynesI assume that means somewhere in heat it barfed on that resource?20:58
*** boris-42 has quit IRC20:59
openstackgerritThom Leggett proposed a change to openstack/diskimage-builder: Uses policy-rc.d to prevent dpkg starting daemons  https://review.openstack.org/8207420:59
GheRiveroand back21:04
GheRiverolifeless: the problem is that NODE_DIST is used to call boot-seed-vm21:05
GheRiveroboot-seed-vm -a $NODE_ARCH $NODE_DIST21:05
*** epim has joined #tripleo21:05
GheRiverobut boot-seed-vm doesn't recognize the -u param21:05
*** akrivoka has quit IRC21:06
greghaynesah, found the actual error... bad type to a fn::join. This does seem like a pretty crappy way to fail for that though21:06
*** e0ne has quit IRC21:07
GheRiverobut it we move the user image to use cirros, there is no need for using NODE_DIST and the -u can be moved to DIB_COMMON_ELEMENTS21:08
*** jp_at_hp has joined #tripleo21:10
*** boris-42 has joined #tripleo21:15
*** jp_at_hp has quit IRC21:16
*** jprovazn_afk has quit IRC21:16
*** marun has left #tripleo21:17
*** lifeless has quit IRC21:19
*** dprince has quit IRC21:20
*** e0ne has joined #tripleo21:22
*** CaptTofu has quit IRC21:24
*** CaptTofu has joined #tripleo21:25
*** CaptTofu has quit IRC21:26
*** CaptTofu_ has joined #tripleo21:26
openstackgerritJonathan Brownell proposed a change to openstack/diskimage-builder: Fix "(None)" seed hostname on Debian  https://review.openstack.org/7730121:27
geerdestnoticing nova console-log not working.  is there any open bug for that? not seeing one.21:31
*** markmc has quit IRC21:32
SpamapSgeerdest: It works on our CI overcloud.21:35
SpamapSgeerdest: you getting blank consoles?21:35
*** markmc has joined #tripleo21:35
geerdestjust built all of this today21:35
geerdestERROR (HTTPNotImplemented): Unable to get console log, functionality not implemented (HTTP 501) (Request-ID: req-5ff06cb5-8a55-46d8-99a6-137c30381651)21:35
geerdestthats on undercloud though21:38
geerdestlooks like my latest build still doesn’t have a working overcloud… so much for today21:41
SpamapSoh21:42
SpamapSundercloud would not work no21:42
SpamapSundercloud is baremetal21:42
SpamapSor ironic21:42
SpamapSand thus needs a console driver for that21:42
*** stevehuang has joined #tripleo21:43
geerdestSpamapS: ah ok.. found an issue and yes.. it does work on my overcloud.  thanks21:43
SpamapSThere is some intention to have console-log work on baremetal servers.. it's just.. complicated :)21:44
*** hashar has quit IRC21:45
*** matty_dubs is now known as matty_dubs|gone21:46
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Add clarification on required permissions  https://review.openstack.org/8310221:46
*** rpodolyaka1 has joined #tripleo21:50
*** rpodolyaka1 has quit IRC21:55
*** untriaged-bot has joined #tripleo22:00
untriaged-botUntriaged bugs so far:22:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129684722:00
uvirtbotLaunchpad bug 1296847 in tripleo "Patch needed to build undercloud on baremetal" [Undecided,New]22:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129048822:00
uvirtbotLaunchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete]22:00
*** untriaged-bot has quit IRC22:00
bnemec^Nice!22:00
Ngcool22:03
*** blamar has quit IRC22:03
*** lynxman has quit IRC22:04
greghaynes\O/22:04
*** lynxman has joined #tripleo22:04
tchaypomorning *grunts*22:07
*** jang1 has quit IRC22:08
*** lynxman has quit IRC22:11
*** lynxman has joined #tripleo22:12
*** robertc has joined #tripleo22:13
robertcmy home net is offline for a few hours while we get the house spider-cleaned22:13
*** robertc is now known as lifeless22:13
greghaynesMeanwhile, in NZ...22:14
devanandalifeless: so https://bugs.launchpad.net/ironic/+bug/1295503 landed22:16
uvirtbotLaunchpad bug 1295503 in ironic "nova-compute errors on startup when ironic user isn't registered in keystone" [High,Fix committed]22:16
devananda*the fix for22:16
lifelessdevananda: yup22:16
lifelessjust waiting on https://review.openstack.org/81627 to complete a check run22:17
* devananda goes t ofind the other bug22:17
devanandaah, thx22:17
* Ng shivers. I do not even why we inhabit any part of the world that needs "spider-cleaning" to be a thing ;o22:17
bnemecWrong country, but I can't help thinking of http://www.sluniverse.com/php/vb/attachments/zomgwtfbbqgtfololcats/11529d1272801642-why-australia-sucks-health_bar_spider.jpg22:17
lifelessis the last remaining patch https://jenkins07.openstack.org/job/check-tripleo-overcloud-precise/298/22:17
lifelesss/patch/job/22:17
lifelesswhich it should, its running the qemu vm up atm22:18
devanandaawesome22:18
lifelessslagle: SpamapS: for great irony: https://review.openstack.org/#/c/81627/22:19
lifelessdevananda: then we can recheck the incubator patch that enables ironic22:20
lifelessslagle: SpamapS: thats passed CI and is one of only two patches needed to get Ironic support for TripleO (and tripleo CI for Ironic)22:23
*** lblanchard has quit IRC22:27
*** bauzas has quit IRC22:27
*** blamar has joined #tripleo22:42
*** cwolferh has quit IRC22:44
*** cwolferh has joined #tripleo22:49
*** e0ne has quit IRC22:49
*** rpodolyaka1 has joined #tripleo22:51
*** blamar has quit IRC22:53
*** lifeless has quit IRC22:54
*** rpodolyaka1 has quit IRC22:55
Lotus907efiif I want to build overcloud images with precise do I need a pypi mirror built on a precise system?22:57
Lotus907efii.e. if I have a pypi mirror that I build usinf run-mirror on a saucy system will that not work for a precise system due to wheel files?22:58
Lotus907efior will the image get built using the tar.gz files to build stuff for the new precise image?22:58
SpamapSLotus907efi: building _with_ precise, no. Building overcloud images that _are_ precise.. well.. IMO, don't do that. ;)22:58
Lotus907efiok, but why?22:58
SpamapSLotus907efi: libvirt.. ovs.. kvm... kernel...22:59
SpamapSthe list goes on and on and on22:59
Lotus907efihmm, I used vmbuilder earlier today and built a qcow2 image from a precise apt repo22:59
SpamapSLotus907efi: it is entirely possible, but a bit futile given that we're designing a system that _embraces_ rapid constant change.. rather than "ship it and forget it" :)22:59
Lotus907efiyeah, but all of our current infrastructure is built around precise23:00
StevenKtchaypo: That's what namecheap.com is for.23:00
Lotus907efiand I have been "directed" to see if it will work23:00
SpamapSIt will work.23:01
SpamapSJust..23:01
SpamapSYou'll spend quite a large percentage of your time backporting things.23:01
SpamapSThe cloud archive should help.23:01
StevenKExcept for libvirt, which infra keep tripping over?23:01
SpamapSBut at that point, you're 99% already at saucy. :-P23:01
SpamapSso like.. why bother? :-P23:01
Lotus907efiso I deleted my current overcloud, exported DIB_RELEASE=precise and pointed DIB_APT_SOURCES at a sources.list file that points at our internal precise apt repo23:02
Lotus907efiand then ran devtest_overcloud.sh23:02
Lotus907efijust to see what would happen23:02
*** mrunge has quit IRC23:02
SpamapSLotus907efi: did you feed that cloud after midnight.. because... didn't we tell you not to do that? ;)23:03
Lotus907efiwhat/where is cloud archive ?23:03
Lotus907efihehehehe23:03
Lotus907efivery funny, I like that23:03
SpamapSLotus907efi: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CGAQFjAB&url=http%3A%2F%2Fwww.ubuntu.com%2Fdownload%2Fcloud%2Fcloud-archive-instructions&ei=4VwzU6KNIJH9oATVhoGQDQ&usg=AFQjCNHHU1SzdtLcSXQDLExe2KmcT6JsYQ&sig2=etsNwAlk6VBe0n-tJpgZtw&bvm=bv.63808443,d.cGU23:04
SpamapSgah23:04
SpamapSdamn you google23:04
SpamapShttp://www.ubuntu.com/download/cloud/cloud-archive-instructions23:04
Lotus907efitinyurl is your friend23:04
SpamapSLotus907efi: that has all the stuff backported for havana on precise.23:04
*** geerdest has quit IRC23:04
Lotus907efiah, ok23:04
SpamapSLotus907efi: that wasn't a tinyurl problem, that was google all up in our bizness problem..23:04
Lotus907efiyeah, google rules all, nine rings and all the rest23:05
*** julim has quit IRC23:05
*** markmc has quit IRC23:06
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Add instructions for setting up sudo  https://review.openstack.org/8323823:06
*** lifeless has joined #tripleo23:06
SpamapSLotus907efi: so what happened?23:07
*** noslzzp has quit IRC23:09
*** juice has quit IRC23:09
*** hewbrocca has quit IRC23:10
*** lifeless has quit IRC23:12
*** lifeless has joined #tripleo23:12
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-heat-templates: WIP: Switch overcloud to software-config  https://review.openstack.org/8166623:15
Lotus907efiwell it quit after pulling quite a few packages23:18
lifelessdammit need one more core23:19
Lotus907efiand it quit after spitting out an error about a couple of packages could not be authenticated23:19
lifelessNg: don't support you are still around?23:19
lifelesssuppose23:19
Lotus907efiis Ng your covert core supplier?  :)23:20
lifelessng / slagle / SpamapS / GheRivero atm yeah23:20
Lotus907efiPsst! hey kid. wanna buy a core?23:20
SpamapSreal cheap23:22
SpamapShigh quality23:22
SpamapSpriced to MOVE23:22
*** nati_ueno has quit IRC23:23
*** nati_ueno has joined #tripleo23:23
Lotus907efithis core was only driven on sundays by a little old lady from Pasadena23:24
*** jtomasek has quit IRC23:25
*** juice has joined #tripleo23:26
openstackgerritA change was merged to openstack/diskimage-builder: debian: install cloud-init on Wheezy  https://review.openstack.org/8232523:28
tchayponames23:34
tchaypogah23:34
tchaypoi hate today.23:34
lifelessbecause its Monday?23:34
tchaypono, thursday has always been the day i hate23:35
tchaypoanything that goes wrong on monday is caused by thursday making initial plans; tuesday is a small-scale test run for thursday; wednesday and friday get affected by ambient leakage from thursday, and the weekend is when thursday just breaks things for fun23:36
tchaypoin other news23:37
*** yamahata has quit IRC23:37
tchaypohttps://review.openstack.org/#/c/82807/23:37
lifelesstchaypo: there speaks a sysadmin23:38
tchaypojps seems to be assuming the devtest_* scripts will be run from $TRIPLEO_ROOT/tripleo-incubator/scripts - that's where his script looks for tripleo-overcloud-passwords23:38
tchaypoJB has his own script which assumes they'll be in $TRIPLEO_ROOT/23:39
tchaypoin my case they're in ~/src/tripleo-incubator23:39
lifelesstchaypo: what do you think of my suggestion?23:39
*** saurabhs has quit IRC23:39
tchaypois there any standard location where we suggest people run the scripts from?23:39
lifelessthat this is a nascent 'manage my cloud credentials' thing23:39
lifelesstchaypo: $TRIPLEO_ROOT for incubator stuff23:39
lifelessother things we assume path / installed23:39
lifelessexcept where we don't23:39
lifelessbut we should23:40
tchaypotaking a look at http://docs.openstack.org/developer/tripleo-incubator/devtest.html for instance, it just says to do a git clone, then cd down to tripleo-incubator and run things from there23:40
lifelessyes23:41
lifelessthe incubator isn't installable23:41
tchaypowhich is why my stuff has ended up in ~/src/tripleo-incubator rather than $TRIPLEO_ROOT23:41
lifelessdeliberately - thats a forcing function to divide 'experiment' and 'thing we know we want'23:41
tchaypobut I notice that all the environment rc files do get written to $TRIPLEO_ROOT/tripleo-incubator/seedrc etc - maybe we should be dumping the password files there as well?23:42
lifelesswe should be putting the passwords into a heat json environment file which heat can use directly and we can query via jq23:44
lifelessprobably in $TRIPLEO_ROOT23:44
lifelessIMO23:44
greghaynesand the private ssh key23:45
greghaynesfor nova bm23:45
greghaynesI love when that thing scrolls by on me screen passed as a shell arg23:46
tchaypoI love it when I paste some logs to someone and it includes a private ssh key and i have a little panic23:48
lifelessgreghaynes: the private key is already in JSON :)23:48
lifelessat least with Ironic (when someone gets time to change the patches) we can post that directly to Ironic and Heat won't know it at all23:49
tchaypoI love it when I ask my registrar why their UI tells me that I must urgently renew my domain, but that I'm not allowed to renew my domain, and their only response is to ask me if I've got access to the email address I just responded to their last query from23:49
lifeless'yes I do'23:49
StevenK"Nope, I'm an autoresponder."23:49
*** epim has quit IRC23:51
openstackgerritSteve Baker proposed a change to openstack/os-collect-config: When extracting deployments, only delete config  https://review.openstack.org/8324823:52
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Use ip route replace in devtest_seed  https://review.openstack.org/8324923:54
*** epim has joined #tripleo23:55
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Standardise location of environment password files.  https://review.openstack.org/8325023:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!