Thursday, 2014-02-06

*** akrivoka has quit IRC00:04
*** mestery has joined #tripleo00:07
*** mestery has quit IRC00:08
*** matsuhashi has joined #tripleo00:20
*** CaptTofu has quit IRC00:34
*** CaptTofu has joined #tripleo00:34
*** jcoufal has joined #tripleo00:39
*** cwolferh has joined #tripleo00:52
*** TravT has quit IRC00:56
*** nosnos has joined #tripleo00:56
*** markmcclain has joined #tripleo01:00
*** cd-undercloud has joined #tripleo01:01
cd-undercloud************** overcloud complete status=1 ************01:01
*** cd-undercloud has quit IRC01:01
openstackgerritA change was merged to openstack/tripleo-image-elements: Account for amd64 architecture in snmpd element  https://review.openstack.org/7017101:01
openstackgerritRichard Su proposed a change to openstack/tripleo-heat-templates: Add username parameter for rabbit  https://review.openstack.org/7143001:11
*** jdurgin has quit IRC01:15
*** markmcclain has quit IRC01:21
*** esmute has quit IRC01:22
*** markmcclain has joined #tripleo01:23
*** cody-somerville has quit IRC01:25
*** markmcclain has left #tripleo01:36
*** taps has quit IRC01:38
*** jcoufal has quit IRC01:39
*** jcoufal has joined #tripleo01:40
*** TravT has joined #tripleo01:43
*** weshay has joined #tripleo01:54
*** mestery has joined #tripleo01:57
*** peoplemerge has quit IRC02:07
*** epim has quit IRC02:09
*** cd-undercloud has joined #tripleo02:13
cd-undercloud************** overcloud complete status=1 ************02:13
*** cd-undercloud has quit IRC02:13
*** mestery has quit IRC02:15
*** jcoufal has quit IRC02:18
*** newell has quit IRC02:35
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Support multiple message brokers with boot-stack  https://review.openstack.org/7144402:50
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Add username to rabbit in local heat metadata  https://review.openstack.org/7144602:53
*** spzala has quit IRC02:58
*** vkozhukalov has joined #tripleo02:59
*** peoplemerge has joined #tripleo03:04
*** briancline has quit IRC03:10
*** coolsvap_away has quit IRC03:13
*** weshay has quit IRC03:29
*** cd-undercloud has joined #tripleo03:29
cd-undercloud************** overcloud complete status=1 ************03:29
*** cd-undercloud has quit IRC03:29
*** cwolferh has quit IRC03:57
*** cwolferh has joined #tripleo03:57
*** cd-undercloud has joined #tripleo04:37
cd-undercloud************** overcloud complete status=1 ************04:37
*** cd-undercloud has quit IRC04:37
*** TravT has quit IRC04:37
SpamapSrabbitmq 31323  5.2  0.9 3530324 963516 ?      Sl    2013 3521:38 /usr/lib/erlang/erts-5.9.1/bin/beam.smp -W w -K true -A30 -P 1048504:38
SpamapSerlang.. hrm04:38
SpamapSlifeless: 2014-02-06 04:38:16,004.004 4774 TRACE nova.api.metadata.handler ConnectionFailed: Connection to neutron failed: Maximum attempts reached04:39
SpamapSlifeless: is that what you've been seeing too?04:39
SpamapSoslo.config.cfg.ConfigFileParseError: Failed to parse /etc/neutron/dnsmasq-neutron.conf: at /etc/neutron/dnsmasq-neutron.conf:1, Section must be started before assignment: None04:39
StevenKSpamapS: If /etc/neutron/dnsmasq-neutron.conf is an empty file, I will laugh04:40
SpamapSno04:41
SpamapSit just had no [DEFAULT] header04:41
lifelessSpamapS: yes04:41
SpamapSit looks like something custom04:41
lifelesswe have a dnsmasq-neutron.conf file now? mtu setting probably...04:42
*** CaptTofu has quit IRC04:42
*** jcooley_ has quit IRC04:43
*** coolsvap has joined #tripleo04:44
lifelessSpamapS: is that heat bug under address now?04:44
lifelessok so 1/3 instances booted successfully04:44
SpamapSlifeless: dnsmasq-neutron.conf is setting ntp04:44
SpamapSit is not in os-config-applier so no idea where it came from.04:45
lifelessSpamapS: oh, is this on the undercloud ?04:45
SpamapSaye04:45
lifelessI mean, the prod undercloud04:45
SpamapSlifeless: right04:45
lifelesscd-undercloud04:45
lifelesssomeone probably just put it there :P04:45
SpamapS04:45
*** akuznetsov has joined #tripleo04:45
lifelessSpamapS: was it breaking neutron on cd-undercloud ?04:46
lifelessSpamapS: I have neutron failing here in devtest04:46
lifelesshmm /var/log/upstart/nova-compute.log:2014-02-05 21:55:25.132 4378 TRACE nova.compute.manager [instance: 9384e3ed-9e24-4bce-9f82-2747102eb171] InstancePowerOnFailure: Baremetal power manager failed to start node for instance '9384e3ed-9e24-4bce-9f82-2747102eb171'04:47
SpamapSlifeless: Yes it was breaking neutron .. neutron wasn't able to restart04:47
SpamapSlifeless: ssh fail?04:48
lifelessSpamapS: 1 of the 3 machines deployed OK04:48
lifelessSpamapS: so not a systemic fault04:48
SpamapSfull again04:49
SpamapS<sigh>04:50
SpamapS-bash: cannot create temp file for here-document: No space left on device04:50
lifelessSpamapS: hey so - I think we need graceful coordination with node before we need fancy rolling, given a simple 2 control 2 overcloud scenario04:50
SpamapSlifeless: yes if we are just going to auto-generate, we can just auto-generate fancy coordination bits04:51
lifelessSpamapS: we can make one control depend on the other, same with the overcloud, and so we'll get nodowntime with graceful coordination even without nice rolling logic.04:51
lifelessSpamapS: so I'm going to suggest we do the graceful bit first, what do you think ?04:51
SpamapSlifeless: Yeah I think we can also fit that into icehouse more than what we want for rolling updates04:52
lifelessSpamapS: I don't know if you saw my strawman syntax for it; how crackful am I?04:53
SpamapSlifeless: still catching up on the email from the last 6 hours..04:53
lifeless:)04:53
SpamapSlifeless: spent a couple hours trying to help peoplemerge get a set of 4 boxes under his desk booting TripleO04:53
lifelesscool04:54
SpamapSlifeless: I was totally lost at how to bridge the seed VM onto a real network04:54
SpamapSlifeless: that's our prescribed method of datacenter assimilation.. but I couldn't find a simple way to do it.04:54
lifelessSpamapS: ovs-vsctl add-port brbm eth2 ?04:54
SpamapSsimple in your head04:54
SpamapSmystical voodoo in mine04:55
SpamapSlifeless: so we basically did that actually04:55
SpamapSlifeless: it just took me 45 minutes of futzing04:55
SpamapSlifeless: and after that, the seed VM did not have 192.0.2.104:55
SpamapSbr-ctlplane was not brought up04:55
lifelessthe alternative is dhcp relay agents04:55
SpamapSand I couldn't remember or find the script which does that04:56
SpamapSSuspecting that I have to mess with config.json in boot-stack04:56
lifelessuhm, the add-port is on the seed host, so shouldn't have any impact on the seed :)04:57
SpamapSlifeless: anyway, we decided we'd figure out what we were missing, and throw together a 1 page "so you want to put TripleO on real servers" document.. _OR_ find the existing one and make sure it works.04:57
lifelessbut, yes, good idea04:57
lifelessyou can look at the seed in the DC to see how I kludged it for us04:58
lifelessbut that was complicated by network port fail if you recall04:58
SpamapSlifeless: right, I don't think that the add-port had an effect.. I don't know why the seed did not get 192.0.2.1 assigned to br-ctlplane.04:58
lifelessso I suspect it's also not shortest-path-to-sunshine04:58
lifelessSpamapS: probably you rebooted it and we haven't got that working properly yet04:58
lifelessor something04:59
SpamapSoh we did reboot04:59
SpamapSBut for seed that _should_ sort of work. ;)04:59
SpamapSsince it is all in heat_local04:59
lifelesshmm, virsh power manager is putting a vm into state 'paused'05:00
lifelesswtf05:00
SpamapSweird05:00
*** nosnos_ has joined #tripleo05:02
*** vkozhukalov has quit IRC05:04
*** jrist has quit IRC05:04
*** nosnos has quit IRC05:06
*** vkozhukalov has joined #tripleo05:07
*** blamar has quit IRC05:21
*** rsacharya has joined #tripleo05:22
*** NearlyFunctional has quit IRC05:27
*** edmund1 has quit IRC05:29
*** akuznetsov has quit IRC05:34
*** akuznetsov has joined #tripleo05:36
*** jrist has joined #tripleo05:39
*** e0ne has joined #tripleo05:40
*** jrist has quit IRC05:43
*** jrist has joined #tripleo05:43
*** cd-undercloud has joined #tripleo05:44
cd-undercloud************** overcloud complete status=1 ************05:44
*** cd-undercloud has quit IRC05:44
*** NearlyFunctional has joined #tripleo05:45
*** jcooley_ has joined #tripleo05:49
*** jrist has quit IRC05:52
lifelessugh05:53
lifelessvirsh start baremetal_405:53
lifelesserror: Failed to start domain baremetal_405:53
lifelesserror: Unable to read from monitor: Connection reset by peer05:53
*** rsacharya has quit IRC06:00
*** julim has joined #tripleo06:00
*** tzumainn has quit IRC06:01
*** matsuhashi has quit IRC06:07
*** matsuhashi has joined #tripleo06:07
*** matsuhas_ has joined #tripleo06:09
*** matsuhashi has quit IRC06:09
*** jcooley_ has quit IRC06:11
*** vkozhukalov has quit IRC06:12
SpamapSlifeless: hey where is your strawman syntax btw?06:21
*** jcooley_ has joined #tripleo06:24
*** e0ne has quit IRC06:27
lifelessSpamapS: in the thread06:31
lifelessSpamapS: about your blueprint06:31
*** cwolferh has quit IRC06:33
*** nosnos has joined #tripleo06:40
*** nosnos_ has quit IRC06:42
*** julim has quit IRC06:42
*** CaptTofu has joined #tripleo06:42
SpamapSlifeless: mmk will re-read06:49
*** cd-undercloud has joined #tripleo06:50
cd-undercloud************** overcloud complete status=1 ************06:50
*** cd-undercloud has quit IRC06:50
*** coolsvap_away has joined #tripleo06:51
*** coolsvap has quit IRC06:52
*** yamahata__ has quit IRC06:57
*** yamahata__ has joined #tripleo06:57
tchaypoquit06:58
tchaypo*sigh*06:58
*** tchaypo has quit IRC06:58
*** coolsvap_away has quit IRC07:02
*** funzo has quit IRC07:05
*** akuznetsov has quit IRC07:06
*** tchaypo has joined #tripleo07:06
*** coolsvap has joined #tripleo07:07
*** CaptTofu has quit IRC07:08
*** akuznetsov has joined #tripleo07:09
lifeless Running /opt/stack/os-config-refresh/post-configure.d/74-cinder-volume07:10
lifelesstgtd: unrecognized service07:10
lifeless[2014-02-06 07:09:35,082] (os-refresh-config) [ERROR] during post-configure phase. [Command '['dib-run-parts', '/opt/stack/os-config-refresh/post-configure.d']' returned non-zero exit status 1]07:10
*** coolsvap_away has joined #tripleo07:11
*** coolsvap has quit IRC07:12
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Fix cinder-volume: service is called tgt.  https://review.openstack.org/7146607:12
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Fix os-svc-restart to exit correctly without -n.  https://review.openstack.org/7115607:12
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Add a script to deploy the ci-overcloud.  https://review.openstack.org/7115407:12
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: os-svc-enable should error when no svcname given.  https://review.openstack.org/7115507:12
*** funzo has joined #tripleo07:13
*** jprovazn has joined #tripleo07:20
lifelessSpamapS: ^ critical there07:23
*** rlandy has joined #tripleo07:27
GheRiveromorning TripleO07:30
*** coolsvap_away is now known as coolsvap07:37
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Adding nova-baremetal, heat and nova API  https://review.openstack.org/6897307:42
*** e0ne has joined #tripleo07:47
*** vkozhukalov has joined #tripleo07:48
*** julim has joined #tripleo07:54
*** cd-undercloud has joined #tripleo07:57
cd-undercloud************** overcloud complete status=1 ************07:57
*** cd-undercloud has quit IRC07:57
*** kevinbenton has joined #tripleo08:01
greghaynesjprovazn: Hey, sorry I keep pestering your rabbit reviews - trying to get up to speed on the state of HA control nodes :)08:03
*** killer_prince has quit IRC08:03
jprovazngreghaynes: Hi, sure, np :)08:04
*** pblaho has joined #tripleo08:07
jprovazngreghaynes: would this README help? https://review.openstack.org/#/c/68392/4/elements/rabbitmq-server/README.md08:07
greghaynesYep, I somehow overlooked that review but makes a lot more sense now08:08
openstackgerritGhe Rivero proposed a change to openstack/diskimage-builder: Add tgt service mapping for Fedora  https://review.openstack.org/7147008:10
*** jistr has joined #tripleo08:14
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Adding nova-baremetal, heat and nova API  https://review.openstack.org/6897308:16
*** jcoufal has joined #tripleo08:17
greghaynesjprovazn: Have you been provisioning multiple control nodes with heat yet?08:17
jprovazngreghaynes: yes08:17
jprovazngreghaynes: though my testing template is very ugly08:18
greghayneshrm, not using merge.py and computescale?08:18
greghaynesAlso, mind sharing your template?08:19
jprovazngreghaynes: no merge.py and computescale (which is quite fresh stuff) - sure, mmt08:19
lifelessman, now I have a VM kernel oopsing on boot. wtf08:25
greghaynesbtrfs woes?08:25
greghaynesWhen you run btrfs, we can just attribute everything to that :p08:26
jprovazngreghaynes: https://gist.github.com/jprovaznik/8840234 - it's just the old overcloud.yaml with notCompute node duplicated to notComputeSlave008:27
greghaynesjprovazn: awesome, thanks08:27
jprovazngreghaynes: this is not worth to send as WIP patch until it's properly extracted into separate scalable config file08:28
*** bogdando has joined #tripleo08:30
lifelessgreghaynes: the vm is ext4 per normal08:32
*** athomas has joined #tripleo08:33
lifelessI think I've found a race condition in the nova bm driver, the state-observing code interacting with the deploy code08:33
*** athomas has quit IRC08:34
*** athomas has joined #tripleo08:35
*** ifarkas has joined #tripleo08:45
*** killer_prince has joined #tripleo08:47
*** killer_prince is now known as lazy_prince08:48
lifelesslol ERROR: HTTPConnectionPool(host='192.0.2.3', port=5000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 113] No route to host)08:49
*** jcoufal has quit IRC08:50
openstackgerritA change was merged to openstack/tuskar: Renamed resource category to overcloud role  https://review.openstack.org/7133608:51
lifelessoh man, and another leak in virt-manager08:52
lifeless1.1G resident08:52
*** hashar has joined #tripleo08:53
*** jtomasek has joined #tripleo08:55
openstackgerritA change was merged to openstack/tripleo-image-elements: Fix cinder-volume: service is called tgt.  https://review.openstack.org/7146609:01
*** akrivoka has joined #tripleo09:02
*** cd-undercloud has joined #tripleo09:04
cd-undercloud************** overcloud complete status=1 ************09:04
*** cd-undercloud has quit IRC09:04
*** lazy_prince is now known as killer_prince09:05
*** jcoufal has joined #tripleo09:06
*** rsacharya has joined #tripleo09:09
*** sheep_ has joined #tripleo09:09
*** sheep_ is now known as rdopieralski09:09
*** CaptTofu has joined #tripleo09:09
killer_princeHi lifeless09:10
killer_princei have submitted a blueprint on build-windows-image: https://blueprints.launchpad.net/diskimage-builder/+spec/build-windows-images09:10
killer_princecan you please review it once you get time..09:10
*** e0ne_ has joined #tripleo09:12
*** CaptTofu has quit IRC09:14
*** e0ne has quit IRC09:14
SpamapSkiller_prince: do you see such a thing sharing any code with the existing diskimage-builder ?09:20
SpamapSkiller_prince: it seems like a _completely_ different concept.09:20
SpamapSanyway.. sleep beckons09:21
*** gcha has joined #tripleo09:24
openstackgerritDougal Matthews proposed a change to openstack/python-tuskarclient: Add bindings for Overcloud Role API entrypoints  https://review.openstack.org/6931309:26
*** cody-somerville has joined #tripleo09:28
*** cody-somerville has joined #tripleo09:28
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Adding nova-baremetal, heat and nova API  https://review.openstack.org/6897309:31
*** jcooley_ has quit IRC09:32
killer_princeSpamapS: no.. its not sharing any code from  existing diskimage-builder. however, the tool will be similar to it but only that it will be for Windows and work on Windows os.09:32
killer_princeSpamapS: so personally me, i feel to keep them together so that same project can be extended to for windows image too..09:33
*** markmc has joined #tripleo09:34
SpamapSkiller_prince: I suppose they could share the same element format09:34
SpamapSkiller_prince: feels a bit like a work-alike instead of same-project09:35
SpamapSkiller_prince: unless of course we can use cygwin ... ;)09:35
killer_princeSpamapS: yes.. but windows elements have to written in a different way: may be using windows batch or powershell..09:35
SpamapSkiller_prince: Well I'm glad you think it would be useful. :)09:37
SpamapSkiller_prince: I think that is one for the mailing list.. I suspect there will be various opinions.09:37
SpamapSok really.. bed09:38
*** derekh has joined #tripleo09:38
killer_princeSpamapS: we cannot reuse existing linux elements as they will not work for Windows in its current format. also, there are some linux tools which do have not been ported for windows.09:38
*** lucasagomes has joined #tripleo09:42
*** hashar_ has joined #tripleo09:44
*** hashar has quit IRC09:48
*** hashar has joined #tripleo09:51
*** hashar_ has quit IRC09:55
*** hashar_ has joined #tripleo09:57
openstackgerritA change was merged to openstack/tripleo-image-elements: Package install support for python-ironicclient  https://review.openstack.org/7112710:00
*** hashar has quit IRC10:01
derekhlifeless: should we try and get the tripleo jobs removed from the zuul queue until we got a overcloud running again ?10:07
*** jprovazn has quit IRC10:10
*** cd-undercloud has joined #tripleo10:11
cd-undercloud************** overcloud complete status=1 ************10:11
*** cd-undercloud has quit IRC10:11
lifelessderekh: we should be able to deploy one now I think, with all the cinder bugs fixed10:11
derekhlifeless: ok, cool10:11
lifelessderekh: though my machine finally had oom killer kill dbus and it went badly from there10:12
lifelessso I still don't know for a fact that it all works ;P10:12
derekhworth a try at least10:12
lifelessderekh: I have a partial script written to automate deploys of it (in my review stack)10:14
lifelessderekh: I was planning on 10 hypervisors10:15
lifelessderekh: btw you said you had errors from my patches - what error, where?10:15
derekhlifeless: looking at the patch at the moment,10:17
derekhlifeless: the error was on both nova compute nodes on the overcloud10:17
derekhthe nova-compute service died10:17
derekhbut besides the error I pasted I didn't get any traceback10:18
lifelessderekh: my nova patches?10:18
lifelessderekh: which ones - I have N outstanding ATM :)10:18
derekhlifeless: ahh it was you tripleo patches with master nova10:18
derekhlifeless: so it may not be related to you patches at all10:19
lifelessderekh: note that you need 072e4ad20a6a7f4fff3b59eb7ef3f6fef9aa19d1 to avoid a deadlock running without file injection10:19
lifelessderekh: my bm patches won't affect overcloud at all10:19
derekhlifeless: I am being up a new setup at the moment hopefully I can get more info if it happens again10:19
lifelesskk10:19
lifelessgnight! If you get a ci-overcloud up - cool, if not thats fine, leave me details somewhere about what you tried and what happened.10:20
derekhlifeless: ok, have you kicked off a script ? or will I give it a whirl?10:22
*** martyntaylor has joined #tripleo10:25
*** martyntaylor1 has joined #tripleo10:25
*** max_lobur_afk is now known as max_lobur10:26
*** e0ne_ has quit IRC10:26
*** martyntaylor has quit IRC10:29
*** hashar_ is now known as hashar10:32
*** jistr has quit IRC10:44
*** rsacharya has quit IRC10:59
*** jistr has joined #tripleo11:10
*** CaptTofu has joined #tripleo11:10
*** jprovazn has joined #tripleo11:11
*** CaptTofu has quit IRC11:15
*** cd-undercloud has joined #tripleo11:17
cd-undercloud************** overcloud complete status=1 ************11:17
*** cd-undercloud has quit IRC11:17
*** boris-42_ has quit IRC11:18
openstackgerritA change was merged to openstack/tuskar: Update the HTTP 500 JSON structure to match oslo  https://review.openstack.org/7128811:27
*** coolsvap_away has joined #tripleo11:29
*** coolsvap has quit IRC11:30
jprovaznderekh, ping11:32
*** martyntaylor1 has quit IRC11:33
*** martyntaylor has joined #tripleo11:44
*** CaptTofu has joined #tripleo11:46
*** coolsvap_away is now known as coolsvap11:46
*** athomas has quit IRC11:50
*** athomas has joined #tripleo11:51
lsmolalifeless: heya, any chance you are around?11:56
*** CaptTofu has quit IRC11:57
*** CaptTofu has joined #tripleo11:58
derekhjpeeler: ack12:02
derekhjpeeler: sorry, jprovazn ack12:02
jprovaznderekh, Hi, you were doing a fresh devtest setup yesterday, right? How far did you get?12:04
derekhjprovazn: if failed, nova-compute on the overcloud compute nodes crashed12:05
derekhjprovazn: http://paste.openstack.org/show/62780/12:05
jprovaznderekh, ha, so you managed to create undercloud?12:06
derekhjprovazn: yup12:06
jprovaznI'm trying to find out why my undercloud doesn't work and I'm hitting this bug when kernel boots: http://www.spinics.net/linux/lists/linux-ext4/msg41852.html12:07
derekhjprovazn: turns out it was a bug in oslo.messaging , that had been fixed but needed to be released, markmc did a new oslo.messaging release12:07
derekhjprovazn: which fixed the problem I was having so I got "devtest.sh completed."12:07
lsmoladerekh: when was the release?12:07
jprovaznderekh, nice12:08
derekhlsmola: maybe about 2 hours ago or so12:08
lsmoladerekh: ok12:08
lsmoladerekh: and have both of your compute nodes crashed?12:08
lsmoladerekh: i had one of the mactive12:08
jprovaznderekh, thanks for the info - it seems that the issue I'm hittint is specific to my setup then I guess12:08
lsmolajprovazn: seems like second run of devtest with -c will timeout on waiting for undercloud to be ready, for me12:10
jprovaznderekh, though one more question - could you please check on your undercloud if you have any "kernel BUG" in logs?12:10
jprovazn(in dmesg)12:10
jprovaznlsmola, can you open VM console in virt-manager and check if you see a kernel trace/error there?12:11
lsmolajprovazn: on it12:11
derekhjprovazn: I havn't seen a resize error like that in a few months, last time I saw it this seemed to work https://review.openstack.org/#/c/53818/ but maybe the issue your seeing is unreated since that patch went in ages ago12:11
*** hashar has quit IRC12:12
derekhjprovazn: checking12:12
derekhjprovazn: no "kernel BUG" in logs12:12
jprovaznderekh, thanks12:13
derekhlsmola: both were active but the nova-compute service had crashed on them both12:13
lsmoladerekh: ah, ok12:14
derekhlsmola: if running devtest -c try adding this http://paste.openstack.org/show/62790/  I think we need a delay  (which is normally provided by the time it takes to build an image)12:15
lsmolajprovazn: ok, weird I have four nodes running12:15
lsmolajprovazn: and seems like there are some errors on undercloud, but then there is a login prompt12:16
lsmolajprovazn: will send you image12:16
jprovaznlsmola, I have a login prompt too12:16
jprovaznlsmola, but if I try log in, it hangs12:17
lsmolajprovazn: https://drive.google.com/file/d/0Bw7VaRThy4PLcmNIRDJuNVA4bVU/edit?usp=sharing12:19
jprovaznlsmola, can you log in?12:20
lsmolajprovazn: where I can find the credentials?12:20
jprovaznlsmola, both heat-admin and stack should work (heat-admin with ssh key only)12:21
lsmolajprovazn: and where is the password for stack?12:21
jprovaznlsmola, pwd is same as login12:22
*** gcha has quit IRC12:22
lsmolajprovazn: yeah I am in12:22
lsmolajprovazn: also ssh heat-admin@192.0.2.3 works12:22
lsmolajprovazn: but I got the timeout12:22
*** hewbrocca has quit IRC12:23
jprovaznlsmola, ok - anyway you are lucky to hit different error than me12:23
lsmolajprovazn: yaaay12:23
lsmolajprovazn: hm12:23
lsmolajprovazn: seed says that it didn't found a valid host for udercloud12:24
derekhlsmola: if running devtest -c try adding this http://paste.openstack.org/show/62790/  I think we need a delay  (which is normally provided by the time it takes to build an image)12:24
lsmolajprovazn: I suspect it didn't deleted my machines from first run12:24
derekhlsmola: if things wen't delete may try adding > export TRIPLEO_CLEANUP=112:25
lsmoladerekh: should devtest.sh -c clear the machiens from previous run?12:25
lsmoladerekh: cause I still have 4 running machines, but from what seed nova says, it did't find a host for undercloud12:25
*** jcooley_ has joined #tripleo12:26
lsmoladerekh: or is it enough to run cleanup script?12:26
lsmoladerekh: ok, so I suppose this is running the heat update. but somehow it ignores the existing stuff?12:26
lsmoladerekh: should update work?12:26
*** cd-undercloud has joined #tripleo12:27
cd-undercloud************** overcloud complete status=1 ************12:27
*** cd-undercloud has quit IRC12:27
*** CaptTofu has quit IRC12:28
derekhlsmola: hmm, maybe the problem that in devtest.sh we have  TRIPLEO_CLEANUP=112:28
derekhlsmola: but it isn't exported12:29
derekhlsmola: so sevtest_setup doesn't call cleanup-env12:29
*** e0ne has joined #tripleo12:29
derekhbecause in there it defaults to 0, hmm let me submit something12:29
lsmoladerekh: ok12:29
lsmoladerekh: but it makes sense when we are calling heat stack-update12:30
*** boris-42_ has joined #tripleo12:30
lsmoladerekh: but update is called only when the previous stack operation succeeded?12:30
*** jcooley_ has quit IRC12:31
derekhlsmola: ahh so your running devtest_overcloud on a already setup env ?12:32
lsmoladerekh: well i am running devtest.sh -c12:32
lsmoladerekh: but I got 4 nodes out of 58 in previous run12:33
lsmoladerekh: 512:33
lsmoladerekh: so i am running it again12:33
lsmoladerekh: but this time it failed on no valid host found for undercloud12:33
lsmoladerekh: and seems like the machines from previous run are still there12:33
derekhlsmola: ok so I think there is 2 issues,12:34
*** coolsvap has quit IRC12:34
*** coolsvap has joined #tripleo12:34
derekhlsmola: 1. cleanup isn't happening correctly  so you see lots of nodes from previous runs, setting export TRIPLEO_CLEANUP=1 should workaround that12:34
derekhlsmola: 2 when using devtest -c the heat stack update happens too fast after the baremetal node register, and you end up getting a "no valid  host found" error12:35
derekhlsmola: add the sleep statments I pasted above works around that12:36
lsmoladerekh: ok, adding sleep a cleanup12:36
lsmoladerekh: and running again12:36
lsmoladerekh: thank you12:36
derekhlsmola: no prob, if all that works you may hit the same error I had this morning , with nova-compute crashing on the compute node, thats a matter or building a new overcloud image with the new oslo, as a workaround you can just restart nova-compute on overcloud12:37
lsmoladerekh: ok12:38
lsmoladerekh: will check the nova services then12:38
lsmoladerekh: and your heat-stack failed in this case?12:38
derekhlsmola: no heat stack was ok, the faulure was at this line12:40
*** gcha has joined #tripleo12:40
derekhwait_for 30 10 nova service-list --binary nova-compute 2\>/dev/null \| grep 'enabled.*\ up\ '12:40
lsmoladerekh: right, ok then12:42
openstackgerritA change was merged to openstack/diskimage-builder: Add tgt service mapping for Fedora  https://review.openstack.org/7147012:42
openstackgerritA change was merged to openstack/diskimage-builder: Update openSUSE package mappings for OpenStack  https://review.openstack.org/7017412:45
*** weshay has joined #tripleo12:53
*** coolsvap has quit IRC12:55
*** CaptTofu has joined #tripleo12:57
*** CaptTofu has quit IRC13:01
*** Shrews has joined #tripleo13:06
openstackgerritA change was merged to openstack/tripleo-incubator: Add -m memory option to setup-seed-vm  https://review.openstack.org/7137013:09
openstackgerritA change was merged to openstack/tripleo-incubator: Add -c cpus option to setup-seed-vm.  https://review.openstack.org/7137113:09
*** gcha has quit IRC13:12
openstackgerritDerek Higgins proposed a change to openstack/tripleo-incubator: Export TRIPLEO_CLEANUP  https://review.openstack.org/7152213:16
lsmoladerekh: ok devtest completed with 1 compute node13:17
*** max_lobur is now known as max_lobur_afk13:17
lsmoladerekh: I had to just restart the nova-compute.service as you said13:17
lsmolajprovazn: ^13:19
derekhlsmola: cool, if you build a new overcloud-compute image that problem should be gone, that patch I submitted should solve the cleanup problem, which leave the race condition, I don't know that the right thing to do about that is13:20
lsmoladerekh: cool13:20
*** julim has quit IRC13:23
*** max_lobur_afk is now known as max_lobur13:29
*** hashar has joined #tripleo13:29
*** cd-undercloud has joined #tripleo13:33
cd-undercloud************** overcloud complete status=1 ************13:33
*** cd-undercloud has quit IRC13:33
Ngmorning13:34
*** gcha has joined #tripleo13:35
*** rbrady-afk is now known as rbrady13:36
*** rbrady has left #tripleo13:37
*** rbrady has joined #tripleo13:38
*** hashar has quit IRC13:40
openstackgerritMarios Andreou proposed a change to openstack/tuskar: WIP: Using merge.py from tuskar to generate overcloud.yaml  https://review.openstack.org/7152713:41
mariosdamnit - changed changeid13:41
mariosignore ^^13:41
openstackgerritMarios Andreou proposed a change to openstack/tuskar: WIP: Using merge.py from tuskar to generate overcloud.yaml  https://review.openstack.org/5204513:43
*** rbrady1 has joined #tripleo13:46
*** rbrady has quit IRC13:47
*** rbrady1 has quit IRC13:47
openstackgerritDougal Matthews proposed a change to openstack/python-tuskarclient: Add bindings for Overcloud Role API entrypoints  https://review.openstack.org/6931313:47
*** jdob has joined #tripleo13:49
*** hashar has joined #tripleo13:49
*** rbrady has joined #tripleo13:50
*** e0ne_ has joined #tripleo13:51
*** e0ne has quit IRC13:54
derekhgonna kill the cd-overcloud job so I can try a ci-overcloud13:55
*** dprince has joined #tripleo13:56
derekhSpamapS: ^its in a UPDATE_FAILED state anyways so I guess doing nothing productive13:57
*** jcooley_ has joined #tripleo13:57
*** ci-overcloud2 has joined #tripleo13:59
ci-overcloud2************** ci-overcloud2 complete status=32 ************13:59
*** ci-overcloud2 has quit IRC13:59
*** tzumainn has joined #tripleo14:01
*** jcooley_ has quit IRC14:03
*** hewbrocca has joined #tripleo14:03
*** rbrady is now known as rbrady-brb14:07
*** ftcjeff has joined #tripleo14:07
*** lblanchard has joined #tripleo14:07
openstackgerritDirk Mueller proposed a change to openstack/tripleo-image-elements: openSUSE package install support for neutron  https://review.openstack.org/7072114:08
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove the ignore F841 (unused variables)  https://review.openstack.org/7153014:10
openstackgerritDerek Higgins proposed a change to openstack/diskimage-builder: Make the MIRROR_TARGET directory if it didn't exist  https://review.openstack.org/7153114:11
*** edmund has joined #tripleo14:12
*** rbrady-brb is now known as rbrady14:17
openstackgerritDirk Mueller proposed a change to openstack/diskimage-builder: Add mapping for openvswitch daemon (openSUSE)  https://review.openstack.org/7153914:29
*** bauzas has joined #tripleo14:30
openstackgerritDirk Mueller proposed a change to openstack/tripleo-image-elements: Nova package install support (openSUSE)  https://review.openstack.org/7113914:31
*** edmund has quit IRC14:32
*** jrist has joined #tripleo14:33
*** edmund has joined #tripleo14:35
*** matty_dubs|gone is now known as matty_dubs14:36
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Fix violations and enable gating on F841  https://review.openstack.org/7153014:38
*** julim has joined #tripleo14:41
*** julim has quit IRC14:42
*** julim has joined #tripleo14:43
*** jayg|g0n3 is now known as jayg14:54
*** blamar has joined #tripleo14:55
*** cody-somerville has quit IRC14:57
*** boris-42_ has quit IRC15:02
*** mestery has joined #tripleo15:09
*** matsuhas_ has quit IRC15:17
*** mestery has quit IRC15:20
*** markmc has quit IRC15:20
*** nosnos has quit IRC15:21
*** mestery has joined #tripleo15:21
openstackgerritA change was merged to openstack/tuskar: Fix violations and enable gating on F841  https://review.openstack.org/7153015:25
*** spzala has joined #tripleo15:33
openstackgerritJay Dobies proposed a change to openstack/tuskar: Added overcloud reload after create  https://review.openstack.org/7156915:39
*** ifarkas_ has joined #tripleo15:40
*** ifarkas__ has joined #tripleo15:41
*** rdopieralski has quit IRC15:42
*** ifarkas has quit IRC15:43
*** ifarkas has joined #tripleo15:43
*** jcooley_ has joined #tripleo15:46
openstackgerritJay Dobies proposed a change to openstack/tuskar: Added overcloud reload after create  https://review.openstack.org/7156915:46
*** mestery has quit IRC15:47
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Added missing licence and enabled gating on H102  https://review.openstack.org/7157315:47
*** jistr has quit IRC15:48
*** ifarkas has quit IRC15:49
*** ifarkas__ has quit IRC15:49
*** jcooley_ has quit IRC15:53
*** jistr has joined #tripleo15:54
*** shakayumi has joined #tripleo15:54
*** akrivoka has quit IRC15:55
*** akrivoka has joined #tripleo15:55
*** bauzas has quit IRC15:58
*** markmc has joined #tripleo16:02
*** max_lobur is now known as max_lobur_afk16:02
*** ci-overcloud2 has joined #tripleo16:05
ci-overcloud2************** ci-overcloud2 complete status=0 ************16:05
*** ci-overcloud2 has quit IRC16:05
openstackgerritA change was merged to openstack/python-tuskarclient: Add bindings for Overcloud Role API entrypoints  https://review.openstack.org/6931316:13
*** shakayumi has quit IRC16:14
*** boris-42_ has joined #tripleo16:15
*** akuznetsov has quit IRC16:16
*** mestery has joined #tripleo16:18
SpamapSwoooot16:19
SpamapSstatus=0!16:19
SpamapSoh16:19
SpamapSci-overcloud.. :(16:19
*** vkozhukalov has quit IRC16:22
*** jprovazn is now known as jprovazn_afk16:23
jristha16:23
derekhSpamapS: so I temporarily stopped the cd-overcloud job to try and get a ci one up and running16:26
derekhSpamapS: cd-overcloud is in the update failed state16:27
derekhSpamapS: want me to restart the job /16:27
derekh?16:27
SpamapSderekh: does tripleo-cd interfere with ci-overcloud ?16:27
SpamapSIf so.. I'm thinking.. -> bugs16:27
SpamapS;)16:27
derekhSpamapS: I don't thing both can run in parrallel if thats what you mean ? as it is I manually changed TRIPLEO_ROOT for ci-overcloud so it shouldn't interfere with cd-overcloud16:28
*** akuznetsov has joined #tripleo16:29
SpamapSderekh: hm. We use pip-cache... I odn't know why because we have pypi .. so we should probably drop pip-cache and then they will be concurrency safe16:30
*** jcooley_ has joined #tripleo16:30
derekhSpamapS: lifeless had added OVERCLOUD_DIB_EXTRA_ARGS="pypi" into deploy-ci-overcloud , but It ran with an error installing pbr, not sure why yet, I just commented it out16:32
*** bauzas has joined #tripleo16:33
derekhlifeless: when your back, for reference here is what I changed in your branches http://paste.openstack.org/show/62828/16:34
derekh I guess I next need to add a image to glance16:35
*** killer_prince has quit IRC16:38
openstackgerritA change was merged to openstack/tuskar-ui: Adding nova-baremetal, heat and nova API  https://review.openstack.org/6897316:38
*** max_lobur_afk is now known as max_lobur16:38
*** pblaho has quit IRC16:39
*** sdake has quit IRC16:41
*** hashar has quit IRC16:42
*** mestery has quit IRC16:44
*** athomas has quit IRC16:48
*** athomas has joined #tripleo16:48
*** jistr has quit IRC16:49
*** athomas has quit IRC16:49
*** akrivoka has quit IRC16:50
*** jcoufal has quit IRC16:52
*** lsmola has quit IRC16:53
*** jtomasek has quit IRC16:54
*** ifarkas_ has quit IRC16:55
*** mestery has joined #tripleo16:56
*** jcooley_ has quit IRC16:56
*** jcooley_ has joined #tripleo16:57
*** matty_dubs is now known as matty_dubs|lunch16:59
*** markmc has quit IRC17:00
*** peoplemerge has quit IRC17:01
*** akuznetsov has quit IRC17:02
*** dtantsur has quit IRC17:03
*** TravT has joined #tripleo17:04
*** e0ne_ has quit IRC17:05
*** e0ne has joined #tripleo17:05
*** e0ne has quit IRC17:08
*** kiall has quit IRC17:08
*** GheRivero has quit IRC17:09
*** sdake has joined #tripleo17:09
*** sdake has joined #tripleo17:09
*** Shrews has quit IRC17:10
*** taps has joined #tripleo17:13
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Fix neutron package install  https://review.openstack.org/7160617:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Correct owner for glance api log file  https://review.openstack.org/7160717:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Fix glance package install config  https://review.openstack.org/7160817:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Install openstack-swift-object  https://review.openstack.org/7033517:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Ability to add *-create-dir service separately  https://review.openstack.org/7039917:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add needed swift storage dirs for packaged install  https://review.openstack.org/7038117:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add create-dir service for neutron  https://review.openstack.org/7079317:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add create-dir service for nova  https://review.openstack.org/7079417:32
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add package install support to cinder  https://review.openstack.org/7027117:32
*** taps has quit IRC17:34
*** taps has joined #tripleo17:34
*** martyntaylor has quit IRC17:35
*** gcha has quit IRC17:37
*** julim has quit IRC17:38
*** bcrochet has quit IRC17:38
*** bcrochet has joined #tripleo17:40
*** jcooley_ has quit IRC17:41
*** jcooley_ has joined #tripleo17:42
*** bcrochet has quit IRC17:42
*** akuznetsov has joined #tripleo17:42
*** bcrochet has joined #tripleo17:43
*** rlandy is now known as rlandy|bbl17:46
*** peoplemerge has joined #tripleo17:51
*** cadenzajon has joined #tripleo17:51
*** john-n-seattle has quit IRC17:55
*** john-n-seattle has joined #tripleo17:55
*** derekh has quit IRC18:02
*** lucasagomes has quit IRC18:04
*** bauzas has quit IRC18:07
*** matty_dubs|lunch is now known as matty_dubs18:10
*** mestery has quit IRC18:20
*** jcooley_ has quit IRC18:21
*** GheRivero has joined #tripleo18:23
*** CaptTofu has joined #tripleo18:24
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Fix glance package install config  https://review.openstack.org/7160818:27
*** newell has joined #tripleo18:34
*** akuznetsov has quit IRC18:37
*** mestery has joined #tripleo18:41
*** e0ne has joined #tripleo18:43
*** rdopieralski has joined #tripleo18:43
*** rdopieralski has quit IRC18:44
*** rdopieralski has joined #tripleo18:44
*** spzala has quit IRC18:44
*** rdopieralski has quit IRC18:45
*** rdopieralski has joined #tripleo18:45
*** e0ne has quit IRC18:45
*** jprovazn_afk has quit IRC18:50
*** jcooley_ has joined #tripleo18:58
*** epim has joined #tripleo19:08
ccrouchlifeless:  cody-somerville was looking at groups rates for *everyone* right?19:12
ccrouchre: sunnyvale19:12
ccrouchor did I misinterpret that, and its a HP only thing?19:12
*** spzala has joined #tripleo19:14
*** e0ne has joined #tripleo19:14
lifelessccrouch: yes19:16
*** mestery has quit IRC19:19
*** ftcjeff has quit IRC19:19
*** ftcjeff has joined #tripleo19:19
*** jwalls has joined #tripleo19:20
*** ftcjeff has quit IRC19:20
*** kiall has joined #tripleo19:24
ccrouchlifeless: awesome thanks19:26
openstackgerritJay Dobies proposed a change to openstack/tuskar: Fix tuskar docs building  https://review.openstack.org/7163319:28
*** spzala has quit IRC19:28
*** spzala has joined #tripleo19:29
*** max_lobur is now known as max_lobur_afk19:30
devanandalifeless: when you have a minute, could you take a look at the whiteboard on https://blueprints.launchpad.net/ironic/+spec/ironic-ilo-virtualmedia-driver and confirm that is inline with your expectations for diskimage-builder19:30
devanandalifeless: and le tme know if you feel that needs a separate artifact in tripleo to track it19:31
Ngooh nice19:33
Ngis that more for early stage hardware provisioning? (like firmware upgrades and whatnot) or are we targetting the vmedia deploy as an alternative to the ramdisk deploy?19:34
*** rlandy|bbl is now known as rlandy19:35
devanandaNg: alternative to PXE-booting the ramdisk initially19:35
devanandaNg: but it will be leveraged for other actions as well19:35
Ngfunky19:35
*** jcooley_ has quit IRC19:35
*** jcooley_ has joined #tripleo19:36
devanandaNg: afaiu, this is more secure as all the pre-boot env can be transferred OOB19:36
devanandaNg: by passing all the credentials and commands to mount virtual media over the BMC, instead of as kernel params over PXE19:37
*** wusWWW has joined #tripleo19:37
Ngdevananda: so would we generate a small filesystem image per deploy and attach that as well as the ISO?19:38
devanandaNg: for?19:38
Ngdevananda: per-machine stuff19:39
devanandaNg: cloud-init over https?19:39
*** jcooley_ has quit IRC19:40
*** jcooley_ has joined #tripleo19:41
lifelessthat is how one might support config-drive19:41
lifelessif one wanted to19:41
devanandaNg: security doesnt' seem to be the initial aim of this proposal -- I think there are internal customer demands to not use tftp/pxe (policy, shmolicy)19:41
devanandabut yes, one could also do config-drive over the virtual media channel, if the file were converted to ISO format19:42
lifelessdevananda: I think its small enough that I don't care about a blueprint; its certainly not prioritised work upstream19:42
*** jcooley_ has quit IRC19:45
*** taps has quit IRC19:48
devanandalifeless: ack19:48
*** taps has joined #tripleo19:48
*** d0ugal has quit IRC19:49
*** jcooley_ has joined #tripleo19:50
*** lblanchard has quit IRC19:52
*** jcooley_ has quit IRC19:56
*** jwalls has quit IRC19:57
*** ftcjeff has joined #tripleo19:57
*** jcooley_ has joined #tripleo19:57
*** bauzas has joined #tripleo20:10
*** jcooley_ has quit IRC20:12
*** jcooley_ has joined #tripleo20:12
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Update power_state method to return Ironic states  https://review.openstack.org/7163920:13
*** TravT has quit IRC20:17
* bnemec was looking at old docs20:22
bnemecNo wonder my images weren't working20:22
lifelessStevenK: whats the part # for your SSD (and how big was it again ?)20:26
*** jcoufal has joined #tripleo20:28
*** cadenzajon has quit IRC20:40
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Renamed resource category to overcloud role  https://review.openstack.org/7164620:40
hewbroccatzumainn: \o/20:41
tzumainnhewbrocca, yay for consistency?20:42
hewbroccayes20:44
hewbroccaand for having a name20:44
*** morazi has joined #tripleo20:51
*** julim has joined #tripleo20:52
*** julim has quit IRC20:53
dprincelifeless: is there any reason we would ever use ensure-bridge without the physical NIC? I can't think of any... and we don't seem to currently.20:54
lifelessdprince: not offhand, though brbm is kindof like that20:55
*** jcoufal has quit IRC20:56
dprincelifeless: cool. we don't use ensure-bridge for that so I'm proposing we nuke it as an optional argument (that complicates my script considerably)20:56
lifelessdprince: sure20:56
*** julim has joined #tripleo20:58
*** TravT has joined #tripleo20:59
SpamapSdprince: Hey while you're here. Have you ever run through devtest with physical machines?21:01
*** gcha has joined #tripleo21:01
SpamapSdprince: or just toci?21:01
dprinceSpamapS: That is what I always do man!21:01
dprinceSpamapS: I've been rebasing my patch for months now... :(21:02
dprinceSpamapS: https://review.openstack.org/#/c/67926/21:02
dprinceSpamapS: would love it if that landed so I could spend less time rebasing and more time just having it work.21:02
dprinceSpamapS: Oh. Back when TOCI was the cool cat... I had a note in there about changing your brbm MTU for it to work. I have to do that... otherwise the iscsi connection fails from the seed to the undercloud.21:04
dprinceSpamapS: I've been meaning to re-add that to devtest because without it my guess is anyone would be hosed...21:04
dprinceSpamapS: I set my MTU to 1504 on brbm and the seeds vnet1 and it works fine.21:05
*** lblanchard has joined #tripleo21:06
SpamapSdprince: peoplemerge and I worked on getting 4 random boxes under his desk going with devtest yesterday and we kind of stalled on the networking21:09
*** athomas has joined #tripleo21:09
dprinceSpamapS: oh, what specifically? are you doing something complicated?21:10
SpamapSdprince: no, just want to treat eth1 on the seed VM's host like brbm21:10
SpamapSdprince: I may have overthunk it...21:10
SpamapSdprince: lifeless seemed to think the only thing I needed to do was 'ovs-vsctl add-port brbm eth1'21:11
dprinceSpamapS: that is all I do actually21:12
lifelessSpamapS: may have :P21:12
SpamapSOk yeah I think I overthought it21:12
SpamapSpeoplemerge: I wonder if you can start over.. :)21:13
dprinceSpamapS: I'd be curious to see if it works for you without settig your MTU higher21:13
SpamapSThe other thing we had to try and work around was that we didn't have real power management21:14
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Renamed resource category to overcloud role  https://review.openstack.org/7164621:14
dprinceSpamapS/lifeless: also, would love to have the powermanagement stuff in there too21:14
SpamapSSo we just used the fake driver21:14
SpamapSbut that made it fairly difficult to know which box to turn on/off ;)21:14
lifelessdprince: would love to have it too :P21:14
dprincelifeless: well, I'm blocked on a either a reply from you (guiding me more specifically how to refactor it) or.... you can just land my existing branch which is like totally fine IMO21:15
* dprince checks to make sure it is still up to date21:15
lifelessdprince: sorry, didn't realise you needed a reply21:15
lifelessdprince: basically change node_macs from a list of macs to a list of machine descriptions21:15
lifelessdprince: with power driver, address/user/creds, and a list of macs for the node.21:16
dprincelifeless: my take is that *may* be a good refactoring suggestion but I'd rather not spend the time on it right now when I'm just doing things the same way we already do them.21:16
lifelessdprince: use oac or jq to pull out the tings you need21:16
lifelessdprince: thing is, the way we do them is glitchy and horrible21:16
dprincelifeless: so when I thought it out it seemed to me the existing format (while clunky for json) fits the bash world nicely where we don't have complex data structures21:17
SpamapSlifeless: orthogonal21:17
SpamapSI don't think dprince's patch makes it more glitchy or horrible.21:17
SpamapSbut it would be fantastic to ALSO make it less glitchy and less horrible.21:18
lifelessso I'm expecting in the next couple of weeks we really need the better structure21:18
SpamapSjq needs a "bash associative array" output21:18
lifelessand I think it would be easier to do the change to the better structure without the other stuff21:18
lifelessrefactor-then-improve not improve-then-refactor21:19
SpamapSlifeless: btw I'm poking at your straw-man .. might work as-is21:19
SpamapSlifeless: (regarding heat)21:19
lifelessSpamapS: :)21:19
dprincelifeless: I'm up for helping to raise that bar... its just on a daily basis I've been rebasing this for a while now (noting this is my second implementation of it)21:19
dprincesemi-daily basis we'll say21:19
* dprince overstates it21:19
*** rlandy has quit IRC21:20
openstackgerritTzu-Mainn Chen proposed a change to openstack/tuskar-ui: Update power_state method to return Ironic states  https://review.openstack.org/7163921:21
*** vkozhukalov has joined #tripleo21:21
lifelessdevananda: btw would like your thoughts on https://review.openstack.org/#/c/71219/ and its dependents w.r.t. does ironic need this or does it have it?21:25
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Stop using all ports option w/ neutron-ovs-cleanup  https://review.openstack.org/6993721:27
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Refactor ensure-bridge for persistance  https://review.openstack.org/6991821:27
*** jcooley_ has quit IRC21:28
*** jcooley_ has joined #tripleo21:29
*** cadenzajon has joined #tripleo21:29
*** vkozhukalov has quit IRC21:31
*** jcoufal-mobile has joined #tripleo21:31
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Stop using all ports option w/ neutron-ovs-cleanup  https://review.openstack.org/6993721:32
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Refactor ensure-bridge for persistance  https://review.openstack.org/6991821:32
dprinceSpamapS/lifeless: I've testing ^^ well on Fedora. It works great. The /etc/network/interfaces impl needs some real testing too.21:33
*** d0ugal has joined #tripleo21:34
*** d0ugal has quit IRC21:34
*** d0ugal has joined #tripleo21:34
dprinceSpamapS/lifeless: which I will happily leave to you guys :). But I did some dev script testing and my sed work should be good I think.21:34
* dprince signs off21:35
*** dprince has quit IRC21:35
*** d0ugal has quit IRC21:35
*** e0ne has quit IRC21:36
openstackgerritJames Slagle proposed a change to openstack/tripleo-incubator: Reintegrate devtest actions with docs.  https://review.openstack.org/7086021:39
peoplemergeSpamaps: absolutely!21:39
lifelessslagle: thanks!21:41
*** d0ugal has joined #tripleo21:41
*** d0ugal has joined #tripleo21:41
peoplemergedprince: will try again, the "easy" way21:43
slaglenow if we can just awaken the spirit of jenkins, some of this might start to merge21:45
SpamapSlifeless: looks like this is holding things up btw https://review.openstack.org/#/c/69797/621:46
SpamapSpeoplemerge: I think you'll still have to step through the scripts and stop between seed and undercloud, and add that bridge port. Maybe we should make that an option actually.21:46
SpamapSpeoplemerge: actually there's probably more to it... hrm21:47
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Fix chattr +C of VM images.  https://review.openstack.org/7107521:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Support flat-networks for the overcloud.  https://review.openstack.org/7088021:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Don't source devtest_end.sh.  https://review.openstack.org/7086121:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Reintegrate devtest actions with docs.  https://review.openstack.org/7086021:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Long overdue change to use multiple -P's with heat  https://review.openstack.org/7087921:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Minor syntax cleanup in devtest.sh.  https://review.openstack.org/7085721:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: When debugging setup-endpoints set -x as well.  https://review.openstack.org/7084621:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Attempt to make devtest_testenv docs work better.  https://review.openstack.org/7087321:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Allow more time for overcloud bring-up.  https://review.openstack.org/7117021:52
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Stop sourcing devtest_undercloud.  https://review.openstack.org/6979721:52
*** UtahDave has joined #tripleo21:52
lifelessanyone know if we can run a 2.5" sata + msata at the same time in the 9470ms ?21:53
peoplemergeSpamapS: what are your thoughts?  Try again with manual setup of the bridge port?21:59
openstackgerritBen Nemec proposed a change to openstack/tripleo-incubator: Separate image load and id assignment  https://review.openstack.org/7166222:00
*** epim has quit IRC22:01
*** epim has joined #tripleo22:03
*** rdopieralski has quit IRC22:03
*** shakayumi has joined #tripleo22:05
*** jcoufal-mobile has quit IRC22:06
StevenKlifeless: Its 256G22:06
StevenKlifeless: The part number is harder22:06
lifelessStevenK: lshw ?22:06
openstackgerritDirk Mueller proposed a change to openstack/tripleo-image-elements: openSUSE package install support for neutron  https://review.openstack.org/7072122:08
StevenKlifeless: Mind you, it's a SATA disk, since I pulled the bottom off. product: MTFDDAK256MAM-1K22:09
*** shakayumi has quit IRC22:11
openstackgerritDirk Mueller proposed a change to openstack/diskimage-builder: Adjust neutron package install for Fedora  https://review.openstack.org/7166722:12
lifelessSpamapS: fixed derekhs feedack on the review you linked ...22:14
lifelessStevenK: thanks22:14
*** A-G has joined #tripleo22:15
openstackgerritA change was merged to openstack/tripleo-incubator: Stop sourcing devtest_undercloud.  https://review.openstack.org/6979722:17
openstackgerritA change was merged to openstack/tripleo-incubator: When debugging setup-endpoints set -x as well.  https://review.openstack.org/7084622:17
openstackgerritA change was merged to openstack/tripleo-incubator: Minor syntax cleanup in devtest.sh.  https://review.openstack.org/7085722:17
openstackgerritDirk Mueller proposed a change to openstack/tripleo-image-elements: Package install support for os-config-applier  https://review.openstack.org/7166822:20
*** jdob has quit IRC22:23
*** epim has quit IRC22:26
*** epim has joined #tripleo22:27
openstackgerritA change was merged to openstack/tripleo-incubator: Reintegrate devtest actions with docs.  https://review.openstack.org/7086022:28
*** morazi has quit IRC22:29
lifelessclarkb: btw we don't necessarily review all of a long chain22:29
lifelessclarkb: the intent is that each change is reviewed in isolation for good/bad.22:30
clarkblifeless: does that mean you do review each change?22:30
lifelessyes22:30
lifelessand land separately22:30
clarkbright, and I think a lot of those can be squashed together. It might make reviewing an individual change in isolation slightly easier as a long chain but is harder to review in aggregate22:31
clarkb* can be squashed without impacting review ability or by breaking the one logical change per commit rule22:32
openstackgerritA change was merged to openstack/tripleo-incubator: Don't source devtest_end.sh.  https://review.openstack.org/7086122:34
openstackgerritA change was merged to openstack/tripleo-incubator: Attempt to make devtest_testenv docs work better.  https://review.openstack.org/7087322:34
openstackgerritA change was merged to openstack/tripleo-incubator: Long overdue change to use multiple -P's with heat  https://review.openstack.org/7087922:34
openstackgerritA change was merged to openstack/tripleo-incubator: Support flat-networks for the overcloud.  https://review.openstack.org/7088022:34
openstackgerritA change was merged to openstack/tripleo-incubator: Fix chattr +C of VM images.  https://review.openstack.org/7107522:34
openstackgerritA change was merged to openstack/tripleo-incubator: Allow more time for overcloud bring-up.  https://review.openstack.org/7117022:34
openstackgerritA change was merged to openstack-infra/tripleo-ci: Document building a testenv image.  https://review.openstack.org/7064322:39
openstackgerritA change was merged to openstack-infra/tripleo-ci: local-config isn't needed for te-workers.  https://review.openstack.org/7064422:39
*** morazi has joined #tripleo22:43
lifelessSpamapS: funny thing22:44
lifelessSpamapS: nova bm is booting the two hypervisors first, every time22:44
openstackgerritDirk Mueller proposed a change to openstack/diskimage-builder: Fix kernel extraction on openSUSE  https://review.openstack.org/7167522:46
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Install openstack-swift-object  https://review.openstack.org/7033522:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Fix neutron package install  https://review.openstack.org/7160622:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Correct owner for glance api log file  https://review.openstack.org/7160722:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Ability to add *-create-dir service separately  https://review.openstack.org/7039922:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add needed swift storage dirs for packaged install  https://review.openstack.org/7038122:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add create-dir service for neutron  https://review.openstack.org/7079322:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add create-dir service for nova  https://review.openstack.org/7079422:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Fix glance package install config  https://review.openstack.org/7160822:50
openstackgerritJames Slagle proposed a change to openstack/tripleo-image-elements: Add package install support to cinder  https://review.openstack.org/7027122:50
openstackgerritDirk Mueller proposed a change to openstack/tripleo-image-elements: Nova package install support (openSUSE)  https://review.openstack.org/7113922:51
*** morazi has quit IRC22:53
devanandaGheRivero: https://review.openstack.org/#/c/71219/2 -- something we probably want to port to ironic PXE driver22:58
devanandalifeless: no, we don't have that yet22:59
devanandaGheRivero: also would be good to implement 397     #TODO(ghe):parallized downloads22:59
lifelessdevananda: GheRivero: downloads from where?22:59
devanandalifeless: from glance23:00
lifelessdevananda: yet thats exactly what my patch disables23:00
devanandalifeless: reading the code now, it looks like a previous statement i made was wrong -- it's still a TODO, not implemented23:01
lifelessdevananda: color me confused23:01
devanandalifeless: right, not parallel dl of different images23:01
devanandabut of the same image23:01
lifelessdevananda: eh, now I'm really confused23:01
devanandamake clients after the first request wait for the first request to finish23:01
devanandaif N clients request same image, make the pxe driver smart enough to recognize this23:02
lifelessoh23:02
lifelessso thats super simple, no ?23:02
lifelessyou cache it by image uuid, yes ?23:02
devanandaright. that's done23:02
devanandai think23:02
devanandabut the smarts around making others wait loks like a $TODO23:02
lifelessjust put a semaphore across all downloads23:02
lifelesslike mine23:02
devanandaright23:03
lifelessand check the cache for the image as the first thing after you get the semaphore23:03
devanandaya23:04
lifelessk :) sounds like tell you to suck eggs :P23:04
lifelesshow does one enable Ironic for devtest btw?23:04
*** epim has quit IRC23:04
devanandaNobodyCam: is your wiki current? ^23:05
SpamapSlifeless: heat's just a slave to dict.iteritems() I'm sure.23:07
SpamapSlifeless: so it is likely nova booting in "dict order"23:07
SpamapSwhich is usually stable in a small case like this23:08
*** matty_dubs is now known as matty_dubs|gone23:08
NobodyCamdevananda: not with the new nova patches23:09
NobodyCam:(23:09
*** jayg is now known as jayg|g0n323:09
*** epim has joined #tripleo23:09
lifelessomg23:13
lifelessI got a two-hypervisor boot. yay23:13
lifeless\o/ \o/ \o/ time to do ci-overcloud again23:16
SpamapSlifeless: what's blocking tripleo-cd ATM?23:17
lifelessSpamapS: well, overcloud being broken was kindof a big thing23:20
lifelessSpamapS: those cinder patches, if you recall...23:20
SpamapSright23:20
SpamapSlast I checked we were still getting 500's from nova-api23:20
SpamapScleared up?23:20
*** kevinbenton has left #tripleo23:20
lifelessSpamapS: which nova-api ?23:21
SpamapSlifeless: cd-undercloud23:21
lifelessSpamapS: can you look at that?23:21
SpamapSlifeless: I did, it was Neutron being dead23:22
lifelessSpamapS: did you fix it?23:22
SpamapSbut then there was concern over ci-overcloud building concurrently with tripleo-cd23:22
SpamapSbecause they both use pip-cache23:22
SpamapSwhich.. IMO.. they shouldn't ;)23:22
lifelesshuh23:22
SpamapSlifeless: yeah neutron is running again. :)23:22
lifelessok23:22
SpamapSlifeless: derekh said he stoppedoh23:23
SpamapSlifeless: derek said he stopped tripleo-cd to build ci-overcloud23:23
SpamapSbut now23:23
SpamapSagain23:23
lifelessI don't understand all of derekhs patch23:23
SpamapS/dev/sda1        19G   19G     0 100% /23:23
lifelessugh23:23
SpamapSI'm moving /opt/stack23:23
lifelesshttp://paste.openstack.org/show/62828/23:23
lifelessok23:23
SpamapSI don't think anything is actively depending on it working. Verifying that now.23:23
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Fix os-svc-restart to exit correctly without -n.  https://review.openstack.org/7115623:24
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Add a script to deploy the ci-overcloud.  https://review.openstack.org/7115423:24
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: os-svc-enable should error when no svcname given.  https://review.openstack.org/7115523:24
SpamapSI think /var/log actually23:24
SpamapSis more important, I should say23:24
SpamapSbut really.. both23:25
*** julim has quit IRC23:29
*** lblanchard has quit IRC23:29
*** A-G has quit IRC23:37
*** A-G has joined #tripleo23:37
*** A-G has quit IRC23:38
*** ftcjeff has quit IRC23:48
*** bauzas has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!