Friday, 2016-05-13

*** fpan has joined #tripleo00:04
*** dmacpher has joined #tripleo00:13
*** ccamacho has quit IRC00:15
*** ayoung has joined #tripleo00:20
*** rcernin has quit IRC00:24
*** saneax is now known as saneax_AFK00:28
*** dmacpher has quit IRC00:32
*** nijaba has quit IRC00:34
*** nijaba has joined #tripleo00:35
openstackgerritMerged openstack/diskimage-builder: Add some output to dib-lint  https://review.openstack.org/30922000:38
openstackgerritOpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/31372100:39
openstackgerritMerged openstack/python-tripleoclient: Use a released version of os-cloud-config  https://review.openstack.org/31407700:40
openstackgerritMerged openstack/diskimage-builder: Split YAML & JSON parsing  https://review.openstack.org/30923400:42
*** dmacpher has joined #tripleo00:44
*** mbound has quit IRC00:44
openstackgerritOpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements  https://review.openstack.org/26852800:45
*** olap has quit IRC00:58
*** openstackgerrit has quit IRC01:02
*** openstackgerrit has joined #tripleo01:03
openstackgerritMerged openstack/diskimage-builder: Fix up EPEL element  https://review.openstack.org/30621301:04
openstackgerritMerged openstack/diskimage-builder: yum-minimal: strip locale archive  https://review.openstack.org/30439901:04
openstackgerritMerged openstack/diskimage-builder: yum-minimal : better cleanup of initial yum failure  https://review.openstack.org/30807701:04
*** chlong has joined #tripleo01:13
openstackgerritIan Wienand proposed openstack/diskimage-builder: tests/elements/fake-os: add '/tmp' as top level dir  https://review.openstack.org/31366001:22
*** chlong has quit IRC01:31
*** fpan has quit IRC01:39
*** saneax_AFK is now known as saneax01:43
*** shivrao has quit IRC01:43
*** mbound has joined #tripleo01:45
*** mbound has quit IRC01:49
*** ccamacho has joined #tripleo01:51
openstackgerritIan Wienand proposed openstack/diskimage-builder: Add install-bin element  https://review.openstack.org/31164601:53
openstackgerritMerged openstack/diskimage-builder: Install proliantutils in IPA's virtualenv  https://review.openstack.org/29911601:54
ianw./safe-devstack-vm-gate-wrap.sh: line 606: /tmp/ansible/bin/ansible: No such file or directory01:56
ianwhttp://logs.openstack.org/91/311591/1/check-tripleo/gate-tripleo-ci-f22-nonha/20de6c6/console.html.gz01:56
ianwwhat's up with that ? ...01:56
ianwoh, hang on, i'm getting confused, that's an old run01:57
*** ccamacho has quit IRC01:58
*** ccamacho has joined #tripleo02:00
*** yamahata has quit IRC02:08
*** rhallisey has quit IRC02:09
openstackgerritMerged openstack/diskimage-builder: Updated from global requirements  https://review.openstack.org/31372102:11
*** rbrady has quit IRC02:11
openstackgerritMerged openstack/diskimage-builder: tests/elements/fake-os: add '/tmp' as top level dir  https://review.openstack.org/31366002:17
*** coolsvap has joined #tripleo02:19
*** cwolferh has quit IRC02:21
*** ccamacho has quit IRC02:30
*** ccamacho has joined #tripleo02:31
*** mbound has joined #tripleo02:46
*** rbrady has joined #tripleo02:49
*** mbound has quit IRC02:50
*** thrash is now known as thrash|g0ne02:53
*** absubram has joined #tripleo02:57
*** absubram_ has joined #tripleo02:59
openstackgerritMerged openstack/diskimage-builder: Add install-bin element  https://review.openstack.org/31164603:00
*** absubram has quit IRC03:02
*** absubram_ is now known as absubram03:02
*** chlong has joined #tripleo03:08
*** ebalduf has joined #tripleo03:12
openstackgerritMerged openstack/diskimage-builder: Add test dependency installation on Gentoo  https://review.openstack.org/31159103:18
openstackgerritMerged openstack/diskimage-builder: Add EPEL as requirement of centos-minimal  https://review.openstack.org/30622003:18
*** saneax is now known as saneax_AFK03:22
*** rlandy has quit IRC03:25
*** tzumainn has quit IRC03:28
*** absubram has quit IRC03:39
*** panda has quit IRC03:42
*** panda has joined #tripleo03:42
openstackgerritayoung proposed openstack/tripleo-quickstart: WIP Allow for multiple undercloud nodes  https://review.openstack.org/31574903:51
*** shivrao has joined #tripleo03:53
*** shivrao has quit IRC03:55
*** ayoung has quit IRC04:07
*** ccamacho has quit IRC04:20
*** links has joined #tripleo04:23
*** shivrao has joined #tripleo04:27
*** masco has joined #tripleo04:40
*** ccamacho has joined #tripleo04:46
*** shivrao_ has joined #tripleo04:47
*** shivrao has quit IRC04:51
*** shivrao_ is now known as shivrao04:51
*** ccamacho has quit IRC04:53
*** rbrady has quit IRC04:54
*** jprovazn has joined #tripleo05:08
*** fragatina has quit IRC05:10
*** ccamacho has joined #tripleo05:18
*** jaosorior has joined #tripleo05:21
*** chlong has quit IRC05:29
*** shivrao has quit IRC05:30
*** ccamacho has quit IRC05:39
*** shivrao has joined #tripleo05:44
*** saneax_AFK is now known as saneax05:45
*** ccamacho has joined #tripleo05:47
*** rcernin has joined #tripleo05:50
*** aufi has joined #tripleo05:53
*** dbecker has joined #tripleo05:53
*** fragatina has joined #tripleo05:54
*** bvandenh has joined #tripleo05:55
*** yamahata has joined #tripleo05:56
*** fragatina has quit IRC05:58
*** veteran has joined #tripleo06:00
*** veteran has quit IRC06:00
*** ccamacho has quit IRC06:08
*** fragatin_ has joined #tripleo06:11
*** fragatin_ has quit IRC06:11
*** ccamacho has joined #tripleo06:12
*** fragatin_ has joined #tripleo06:12
*** dmacpher has quit IRC06:14
*** leanderthal|afk has quit IRC06:14
*** ccamacho has quit IRC06:16
*** fragatin_ has quit IRC06:16
*** ccamacho has joined #tripleo06:24
*** hjensas has quit IRC06:31
*** apetrich has joined #tripleo06:31
*** apetrich has quit IRC06:32
*** fragatina has joined #tripleo06:32
*** apetrich has joined #tripleo06:32
*** fragatina has quit IRC06:33
*** fragatina has joined #tripleo06:33
*** veteran has joined #tripleo06:42
*** veteran has quit IRC06:42
*** florianf is now known as florianf-afk06:49
*** florianf-afk is now known as florianf06:50
*** tesseract has joined #tripleo06:56
EmilienMmichchap_: hey, on french timezone for 1 week :)06:58
EmilienMmaybe we can work a bit more together during this time06:58
michchap_EmilienM: ah, nice!06:58
EmilienMyeah06:58
marioso/ EmilienM *06:59
michchap_EmilienM: do you know the answer to my question about pcmk resources?06:59
EmilienMhey marios06:59
EmilienMmichchap_: not yet, I just read it06:59
michchap_EmilienM: if they can ensure all services are up, we can create an anchor.06:59
EmilienMmichchap_: the thing is, the anchor is for one node07:00
EmilienMit's not shared across nodes07:00
michchap_EmilienM: yep, but if pacemaker is ensuring all services are up on all nodes, we can use it.07:00
michchap_EmilienM: only for keystone though, not other services.07:00
EmilienMI need to check what does step 4 and 5 again07:01
EmilienMright! that's why we might move things to step407:01
EmilienMI need to read THT again07:01
EmilienMmichchap_: I've looked at THT, we need to try moving pacemaker bits from step5 to 407:04
EmilienMmichchap_: I think it's doable07:05
EmilienMif we add require correctly07:07
michchap_EmilienM: I'm too inexperienced with pcmk to really know if it's possible. The config is done in step 4, if the pcmk master tries to bring the service up in that step and one of the nodes has not finished doing config, are we going to get errors?07:07
EmilienMmaybe I could try with glance first07:07
*** oshvartz has joined #tripleo07:07
EmilienMright07:07
EmilienMmaybe at step 3 then07:08
EmilienMreally we just want config files in place07:08
*** shivrao has quit IRC07:08
EmilienMI think it's doable to do config at step 3, start everything at step 4 and manage resources like endpoints at step 507:08
EmilienMmaybe I'm wrong but it's worth to try07:08
michchap_No I think you're right about that, and step 3 is mostly vacant iirc07:09
michchap_in fact you could set config in step 107:09
EmilienMyeah07:09
EmilienMthe thing to take care is the dbsync thing07:09
EmilienMI'm going to work on a PoC this week end07:09
michchap_unless I'm missing something all the data is already present at that time07:09
EmilienMit's a bit hard to work on that because we're moving stuffs07:09
EmilienMI would rather wait until we moved everything07:10
michchap_db sync happens in step 307:10
EmilienMand then, try to solve that issue07:10
michchap_but since it's only going to happen on one node, we can use dependencies to order that correctly07:10
michchap_but yeah, I think config can move to step 1 or step 2 unless there is some piece of data produced by those steps that must be consumed and fed back into hiera.07:11
EmilienMsince glance & keystone are already in puppet-tripleo, maybe I can prepare a PoC with them07:11
michchap_I think that's a good idea07:11
EmilienMbut don't trust me now, I didn't sleep last night, was in a flight from canada to france07:12
michchap_oh. Jetlagged?07:12
EmilienMwell, actually fine. I drank 5 coffees07:12
michchap_so you're buzzed :)07:12
EmilienMahah07:13
*** sshnaidm has quit IRC07:16
*** ebalduf has quit IRC07:17
*** paramite has joined #tripleo07:20
*** athomas has joined #tripleo07:23
*** sshnaidm has joined #tripleo07:24
EmilienMmichchap_: I prepared a PoC but i realized we also need to move openstack-core-then-httpd-constraint and galera-then-openstack-core-constraint resources in step4, but they are not in puppet-tripleo yet07:27
michchap_EmilienM: make another patch and depends on is the only option I guess07:28
EmilienMyeah07:28
michchap_annoying though07:28
EmilienMI'll wait for tripleo CI to be back though07:28
EmilienMpabelanger is working on it07:28
*** mikelk has joined #tripleo07:28
openstackgerritAndreas Florath proposed openstack/diskimage-builder: Fix apt-sources configuration for debian-minimal  https://review.openstack.org/31526507:28
pabelangerEmilienM: centos-7 slave is online, so that is good07:29
pabelangerstill waiting for new fedora-22 slave to finish building07:29
EmilienMcool07:29
pabelangerEmilienM: another reason to land https://review.openstack.org/#/c/312725/ and switch to centos-7 :) We'd be running jobs already if we did07:30
*** mcornea has joined #tripleo07:34
*** hjensas has joined #tripleo07:34
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: WIP: move pacemaker resource to step 4  https://review.openstack.org/31593807:36
*** shardy has joined #tripleo07:36
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: deploy keystone at step 3 and 4  https://review.openstack.org/31594307:37
EmilienMmichchap_: ok this is really PoC, I haven't tested it07:37
EmilienMbut feel free to review07:37
*** mdnadeem has joined #tripleo07:40
*** panda has quit IRC07:42
*** anshul has joined #tripleo07:42
*** anshul is now known as Guest5672607:42
*** panda has joined #tripleo07:42
michchap_EmilienM: is this going to be a problem: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_controller_pacemaker.pp#L161107:42
michchap_EmilienM: because horizon has to come up at the same time.07:43
*** mdnadeem_ has quit IRC07:43
EmilienMoh right07:43
*** jcoufal has joined #tripleo07:44
EmilienMdamn07:44
EmilienMmichchap_: should I move it to step 3 also?07:44
EmilienMerr, 407:44
EmilienMlet's try07:44
michchap_EmilienM: I guess.07:45
EmilienMok we should be good in next PS07:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: WIP: move pacemaker resource to step 4  https://review.openstack.org/31593807:46
pabelangerokay, experimental-tripleo is working (as expect).07:48
pabelangerstill waiting on fedora-22 node to finish building07:48
pabelangerlooks like a possible network issue07:48
EmilienMok so check-tripleo is still in progress07:48
shardymorning all07:50
shardythanks to everyone involved with bringing CI back up07:50
EmilienMwaves on pabelanger for staying so late07:50
EmilienMand waves on coffee07:51
*** dbecker has quit IRC07:51
*** zoli_gone-proxy is now known as zoliXXL07:53
*** tdasilva has quit IRC07:54
pabelangerfedora-22 snapshot finished, waiting on hardware to launch the nodes08:04
pabelangerlikely hammering the image service08:05
*** ohamada has joined #tripleo08:06
openstackgerritMartin AndrĂ© proposed openstack/tripleo-puppet-elements: Fix puppet modules in mixed src/pkg environments  https://review.openstack.org/28586008:07
*** ohamada has quit IRC08:08
*** ohamada has joined #tripleo08:09
*** tdasilva has joined #tripleo08:09
zoliXXLgood morning08:14
*** ebarrera has joined #tripleo08:21
*** gfidente has joined #tripleo08:22
*** leanderthal|afk has joined #tripleo08:24
*** oshvartz has quit IRC08:26
*** devvesa has joined #tripleo08:26
*** devvesa has quit IRC08:26
*** devvesa has joined #tripleo08:26
pabelangerokay, zuul still trying to launch tripleo-f22 nodes. Somebody with SSH access with need to look into the cloud debug logs to see what is going on08:29
pabelangerso far we've only launched 1 node, and it failed because of DNS issues08:30
pabelangerhttp://logs.openstack.org/64/310564/4/check-tripleo/gate-tripleo-ci-f22-ha/2055488/console.html#_2016-05-13_08_21_03_24608:30
*** dbecker has joined #tripleo08:30
*** lucas-dinner is now known as lucasagomes08:31
shardypabelanger: we'll have to wait for derekh I think, he's the only one with SSH access who's not in the US08:32
*** ramishra has quit IRC08:34
pabelangershardy: okay, there isn't much more I or openstack-infra can do at this point in time.  Everything has been restored to pre outage configuration08:34
shardypabelanger: ack - looks like DNS issues I guess, thanks for your help, hopefully derekh will show up soon08:35
pabelangerwill see what happens in another 15mins, nodepool will delete the current tripleo-f22 server because of the 1hr timeout08:39
*** jtomasek_ has joined #tripleo08:46
*** jistr has joined #tripleo08:46
*** ccamacho has quit IRC08:47
*** zoliXXL is now known as zoli|afk08:48
*** leanderthal|afk is now known as leanderthal08:48
*** jtomasek_ has quit IRC08:51
*** jpena|off is now known as jpena08:52
*** ccamacho has joined #tripleo08:53
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: composable neutron l3 service  https://review.openstack.org/30356208:56
mariosshardy: you may want to revote there ^^^ you were +2 i just removed the tag thx08:57
shardymarios: will do, thanks08:57
jaosorioranybody getting these errors when installing the undercloud? Error: Duplicate declaration: Ironic_config[DEFAULT/my_ip] is already declared in file /etc/puppet/modules/ironic/manifests/init.pp:324; cannot redeclare at /etc/puppet/manifests/puppet-stack-config.pp:412 on node undercloud08:59
jaosoriorError: Duplicate declaration: Ironic_config[DEFAULT/my_ip] is already declared in file /etc/puppet/modules/ironic/manifests/init.pp:324; cannot redeclare at /etc/puppet/manifests/puppet-stack-config.pp:412 on node undercloud08:59
pabelangerokay, looks like we are launching nodes now09:00
pabelangerafter the 1 hour time out, I am seeing tripleo-f22 coming online09:00
pabelangeronly 5 nodes ATM09:01
pabelangerstill having issues launching more09:01
pabelangernow 609:02
pabelangerYa, something will have to check the debug logs of the cloud and let me know what is the issue09:02
pabelangerseems they are eventually launching after some timeout period09:02
jaosoriorlarsks: Hey dude, did you guys come up with something regarding the Ironic_config issue?09:04
shardyjaosorior: are you installing it with tripleo.sh --undercloud?09:07
*** electrofelix has joined #tripleo09:07
gfidentemarios, see my comment there :(09:07
jaosoriorshardy: yes09:07
shardyhttps://github.com/openstack/puppet-ironic/commit/50fed86586a8519d99bcf8769303423809e1245e09:08
shardylooks like puppet-ironic broke us while CI was down09:08
shardyjaosorior: we'll have to adjust the puppet-stack-config.pp to account for that change I think09:09
*** dmk0202 has joined #tripleo09:09
shardywhenever you see an error from a puppet module, first thing to check is the recent commits to that module ;)09:09
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Ensure manage_service and enabled are false for Pacemaker  https://review.openstack.org/31365009:10
jaosoriorshardy: where was the puppet-stack-config.pp?09:10
jaosoriorgonna change that inline so before proposing a fix09:10
mariosgfidente: ah damnit i missed you are removing them from neutron base there already :/09:10
mariosgfidente: right09:10
gfidenteit's not bad09:10
gfidentejust need to keep track of things09:10
shardyhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp09:10
pabelangerokay, I am logging for a bit. Things are limping along now09:11
jaosorioraaah that09:11
shardyjaosorior: /usr/share/instack-undercloud on the undercloud09:11
gfidenteI've updated the tht change to do this for dhcp agent09:11
gfidentemarios, yeah because the puppet manifests landed all in a single change09:11
mariosgfidente: yeah cos on the tht side you just added them to a couple09:12
gfidentemarios, because those are the only tht roles which landed09:12
mariosgfidente: so i was confused but on the puppet side you removed all of them09:12
gfidenteyes because puppet side landed all together09:12
gfidenteso I can't do it differently09:12
gfidentethe tht roles instead could me squashed09:13
gfidente*be09:13
mariosgfidente: so we can add it, and make it depend on that puppet-tripleo?09:13
gfidentemarios, ack, that's cool09:13
mariosgfidente: i mean add it to the pacemaker manifest tht side... k doing now hope dprince doesn't object :)09:13
shardypabelanger: thanks for all the help!09:13
gfidentemarios, wait actually it's the other way around09:15
gfidentethe puppet change won't pass unless we add those in the templates09:15
mariosgfidente: yeah it can't land until we are passing it from here09:15
gfidenteso if you could just add the two lines in the role09:15
mariosgfidente: so updating anyway won't add a depends on09:16
mariosgfidente: thanks09:16
gfidenteyeah and both should have green ci09:16
gfidentety :)09:16
rcerningfidente, is the triple-heat-templates good to discuss here?09:17
gfidentercernin, sure! :)09:17
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: composable neutron l3 service  https://review.openstack.org/30356209:18
rcerninI would like to help on tripleo-heat-templates do we have use cases for the network configuration, I remember dsneddon wanted to do something but haven't had much time, I thought because it is quite important as some customer are still fighting, whether it would make sense to prepare something.09:19
jaosoriorshardy: EmilienM had actually a fix for that uploaded already https://review.openstack.org/#/c/315238/ got a +2? :D09:20
rcerningfidente, maybe could be part of openstack-manuals in networking-guide.09:20
mariosgfidente: thanks gfidente also the leaeding :: i think09:21
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: composable neutron l3 service  https://review.openstack.org/30356209:22
gfidentercernin, you mean a config guide?09:22
rcerningfidente, yes like config-heat-templates in the networking-guide09:24
rcerninAnd put there some of the general use cases.09:24
gfidenteyeah that would be great09:26
gfidenteand yes I think dsneddon was drafting something but I am afraid I think we need to wait for him to see where this document is09:27
ccamachorcerning, hey! I'm putting some info related to composable THT in https://review.openstack.org/#/c/311512/, folks from Madrid (field) also wanted to have more info09:27
ccamacho* rcernin09:27
gfidenteso there are two sides of it, the nic templates and the actual template parameters09:27
gfidentethe nic templates can be customized to use whatever feature os-net-config provides09:28
gfidentenesting the resources09:28
gfidenteI'd start documenting the sample scenarios we have09:28
gfidenteand the parameters09:28
gfidentebecause the os-net-config features probably belong more to the os-net-config project rather than tht09:29
gfidenteshardy, ^^ wdyt?09:29
gfidentefor os-net-config there are same sample use cases in the README https://github.com/openstack/os-net-config/blob/master/README.rst09:30
rcerningfidente, agree on os-net-config, but they share same openstack-manuals right?09:30
gfidenteI suppose we could provide in openstack-manuals some guidance on how we 'generate' a valid os-net-config yaml with heat09:32
gfidentebecause that is actually what we do09:32
gfidenteuse heat as a sort of template generator09:32
gfidentewhere the addresses/subnets or vlans dumped in the o-n-c template are customized based on user input09:33
shardygfidente: The problem is most of the examples in t-h-t are coupled to the parameters of the heat template09:34
shardyso I don't think we can move all the examples to os-net-config09:34
shardyit would be good to reduce the duplication in t-h-t though09:34
shardyI agree that docs, better validation and wizard style tooling would be good09:35
gfidentercernin, shardy so I'd approach it this way; first let's make sue in os-net-config README we have 'basic' and static nic config examples which match our use case in tht09:36
gfidentethen in openstack-manuals we explain how we generate a valid o-n-c template like the one in the README from the heat parameters09:36
shardygfidente: sure - before we expend a lot of effort documenting the t-h-t stuff I'd like to revive and land https://review.openstack.org/#/c/271450/09:37
shardyas it will change things slightly09:37
gfidenteyeah we should land that09:38
shardyI need to rebase it, will try to do it later today09:38
openstackgerritMerged openstack/instack-undercloud: Load Mistral 'workbooks' (not 'workflows').  https://review.openstack.org/31504109:38
*** jcoufal has quit IRC09:39
*** zoli|afk is now known as zoli09:41
openstackgerritBabu Shanmugam proposed openstack/puppet-tripleo: Add OVN support for neutron profile  https://review.openstack.org/31487509:43
*** chem has joined #tripleo09:45
openstackgerritBabu Shanmugam proposed openstack/puppet-tripleo: Add OVN support for neutron profile  https://review.openstack.org/31487509:45
*** tosky has joined #tripleo09:55
*** zoli is now known as zoli|lunch09:59
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071310:08
*** jprovazn has quit IRC10:08
*** yamahata has quit IRC10:13
*** leanderthal has quit IRC10:19
*** d0ugal has quit IRC10:20
*** d0ugal has joined #tripleo10:21
jaosoriorHas anybody seen this errors in nova compute? http://paste.openstack.org/show/497034/ Getting a bunch of "No valid host was found" and what I see are those errors in the log10:27
jaosoriorthe ironic logs seem pretty normal though10:28
jaosoriorAlso this error in neutron seems pretty fishy http://paste.openstack.org/show/497035/10:29
jaosoriorAll of this is when deploying the overcloud10:30
*** jtomasek_ has joined #tripleo10:33
jaosorioranybody?10:35
*** jtomasek_ has quit IRC10:42
*** apetrich has quit IRC10:46
*** pkovar has joined #tripleo10:47
*** mkovacik__ has joined #tripleo11:02
*** mkovacik__ is now known as _milan_11:03
*** leanderthal has joined #tripleo11:03
*** jprovazn has joined #tripleo11:13
*** mgould has joined #tripleo11:14
slagleEmilienM: i guess we'll need to merge this asap, https://review.openstack.org/#/c/315238/ since puppet-ironic has gone ahead and broke us11:19
jaosoriorslagle: You can +A that :D11:19
*** thrash|g0ne is now known as thrash11:20
*** zoli|lunch is now known as zoli11:20
slaglei'm aware11:20
slaglemight be nice to test that ironic actually still works11:22
slaglesince we aren't getting far enough in ci to verify that11:22
jaosoriorI didn't apply the whole patch, but I am seeing some funky things in nova. Not sure if it's ironic related as the logs there seem pretty normal11:23
jaosoriorI asked for help above but nobody replied11:24
*** dprince has quit IRC11:29
*** jaosorior has quit IRC11:30
*** jaosorior has joined #tripleo11:30
openstackgerritvinay kumar muddu proposed openstack/diskimage-builder: Included "sudo" for devuser element  https://review.openstack.org/30874711:31
slaglejaosorior: we can see how far tripleo-ci gets on https://review.openstack.org/#/c/316039/11:31
*** morazi has joined #tripleo11:32
*** lblanchard has joined #tripleo11:34
shardyjaosorior: FYI https://bugs.launchpad.net/tripleo/+bug/158147211:35
openstackLaunchpad bug 1581472 in tripleo "CloudDomain doesn't correctly set hostname" [High,Triaged] - Assigned to Steven Hardy (shardy)11:35
*** masco has quit IRC11:38
*** panda has quit IRC11:42
jaosoriorshardy: Ever seen the errors I posted above?11:42
openstackgerritvinay kumar muddu proposed openstack/diskimage-builder: Included "sudo" for devuser element  https://review.openstack.org/30874711:42
jaosoriorstill don't have a working dev environment x_x11:42
*** derekh has joined #tripleo11:42
*** panda has joined #tripleo11:43
*** leanderthal has quit IRC11:45
EmilienMslagle: damn11:45
*** lucasagomes is now known as lucas-hungry11:45
EmilienMslagle: I'm going to revert it11:45
EmilienMand sorry for that11:45
slagleEmilienM: https://review.openstack.org/#/c/316039/11:45
EmilienMslagle: +2 +A11:45
slagleEmilienM: thanks, i'm wondering if we will need to revert https://review.openstack.org/#/c/315233/ as well?11:49
slaglei guess we can wait and see what happens in tripleo-ci11:49
*** weshay has joined #tripleo11:50
*** MaxPC has joined #tripleo11:51
EmilienMI don't think so11:51
EmilienMIt's a new class11:51
slagleok11:51
EmilienMIt won't break11:51
EmilienMI'm afk for lunch a bit, ttyl11:51
slaglebon appetit11:52
*** rodrigods has quit IRC11:52
*** rodrigods has joined #tripleo11:52
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB  https://review.openstack.org/31505112:01
ansiwenI want to install my new 64GB machine, and naivly would do that with a CentOS USB installer... but maybe that is already a chance to play around with other technologies?12:03
ansiwenhow does ironic work there, could I install the hardware with heat/ironic from my laptop?12:04
openstackgerritJiri Stransky proposed openstack/puppet-tripleo: Add a service manifest  https://review.openstack.org/31354312:05
*** apetrich has joined #tripleo12:06
*** fultonj has joined #tripleo12:08
openstackgerritJiri Stransky proposed openstack/puppet-tripleo: Add a service manifest  https://review.openstack.org/31354312:10
openstackgerritBrad P. Crochet proposed openstack/puppet-tripleo: Add Heat profiles  https://review.openstack.org/31301912:12
shardyansiwen: FWIW I'd just install it with centos7 via USB, then use http://paste.fedoraproject.org/365874/27735146/ or tripleo-quickstart to get a dev env running12:12
shardyansiwen: you *could* install an undercloud on a laptop and deploy to the hardware using the fake_pxe ironic driver12:13
ansiwenshardy: ok, I thought there is some fancy PXE stuff I could do without USB drive :-)12:13
shardybut there's not a lot of point unless you have at least two nodes12:13
*** links has quit IRC12:14
shardyansiwen: well I maintain some kickstart files and a local pxe server so I can easily pxe boot nodes12:14
openstackgerritBrad P. Crochet proposed openstack/tripleo-heat-templates: composable heat services  https://review.openstack.org/31357712:14
shardyyou don't need ironic to do that12:14
*** MaxPC has quit IRC12:14
ansiwenwell, the advantage for me would be, that I don't have to take care where that USB drive is, when I want to reinstall the machine later12:15
ansiwenshardy: is there a simple-to-setup pxe server?12:15
shardyansiwen: even with just apache you can create a kickstart and put in on a webserver $anywhere, then use ks=$url to do a hands-off centos7 install12:15
shardyansiwen: there's pretty good stuff in the centos docs about it12:15
ansiwenok, I'll check that out12:16
*** leanderthal has joined #tripleo12:16
ansiwenshardy: this: https://www.centos.org/docs/5/html/Installation_Guide-en-US/ch-pxe.html12:16
ansiwen?12:16
*** mbound has joined #tripleo12:16
ansiwenit's centos 5 I think12:17
*** jroll has quit IRC12:17
*** jayg|g0n3 is now known as jayg12:19
*** chem has quit IRC12:19
*** jroll has joined #tripleo12:22
*** jroll has quit IRC12:22
*** jroll has joined #tripleo12:22
*** chem has joined #tripleo12:23
*** leanderthal has quit IRC12:25
*** shardy has quit IRC12:28
*** gfidente has quit IRC12:30
*** dprince has joined #tripleo12:34
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: gate injection: update only gated packages  https://review.openstack.org/31606312:38
*** jpena is now known as jpena|lunch12:39
*** Goneri has joined #tripleo12:41
*** rcernin_ has joined #tripleo12:42
*** rcernin has quit IRC12:42
EmilienMfyi about https://review.openstack.org/#/c/316039/ - it's not merged yet because our gate was also broken, we just fixed it with https://review.openstack.org/#/c/315770/12:48
EmilienMso once https://review.openstack.org/#/c/315770/ is merged, the revert will land12:48
EmilienMand all should be green again12:48
derekhdprince: can you redo the ssl cert update, I didn't spend time on it last night when I redeployed the overcloud12:48
*** rlandy has joined #tripleo12:49
*** rbrady has joined #tripleo12:49
openstackgerritPradeep Kilambi proposed openstack/python-tripleoclient: Aodh integration to tripleoclient  https://review.openstack.org/31478412:50
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071312:51
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB  https://review.openstack.org/31505112:51
*** lucas-hungry is now known as lucasagomes12:53
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Deploy Aodh services, replacing Ceilometer Alarm  https://review.openstack.org/31579612:53
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Write service manifest on the controllers  https://review.openstack.org/31354412:56
dprincederekh: yes12:57
dprincederekh: I will re-do the ssl update12:57
derekhdprince: thanks12:57
*** tzumainn has joined #tripleo12:58
*** ayoung has joined #tripleo13:00
slaglefyi, hopefully once https://review.openstack.org/#/c/316039/ merges we will get some passing obs13:03
slaglejobs13:03
*** MaxPC has joined #tripleo13:03
adarazslarsks: hi, can I bug you with this little change for the gating rpm injection? https://review.openstack.org/316063 -- I realized doing a full update of the image gives me more headache at that step than I need. :)13:05
larsksadarazs: sure, soon as i finish a con call.13:05
adarazsokay13:05
*** pkovar has quit IRC13:06
*** dprince has quit IRC13:11
*** pkovar has joined #tripleo13:11
*** dprince has joined #tripleo13:11
*** rcernin_ has quit IRC13:16
*** egafford has joined #tripleo13:20
dprincedprince: new ssl certs deployed valid through Monday, June 12, 2017 at 7:59:59 PM13:20
dprincederekh: ^^^13:21
dprincederekh: they are also backed up to /root/new_certs on the controller13:21
derekhdprince: cool, thanks13:21
openstackgerritMartin AndrĂ© proposed openstack/tripleo-common: Validations actions and workbook  https://review.openstack.org/31363213:24
*** rcernin has joined #tripleo13:27
*** Erming_ has quit IRC13:28
*** absubram has joined #tripleo13:28
*** absubram_ has joined #tripleo13:32
*** jcoufal has joined #tripleo13:32
*** absubram has quit IRC13:32
*** absubram_ is now known as absubram13:32
*** athomas has quit IRC13:41
openstackgerritPradeep Kilambi proposed openstack/tripleo-puppet-elements: Add Aodh component  https://review.openstack.org/28811913:42
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo: loadbalancer: add Aodh API support  https://review.openstack.org/28812213:43
*** openstack has joined #tripleo13:45
*** jcoufal has quit IRC13:47
*** jpena|lunch is now known as jpena13:47
*** athomas has joined #tripleo13:48
*** ebalduf has joined #tripleo13:52
*** openstack has joined #tripleo13:53
openstackgerritSanjay Upadhyay proposed openstack/tripleo-specs: new spec: tripleo-sriov  https://review.openstack.org/31387213:58
*** derekh has quit IRC13:59
rbradydprince: the mistral api bug I mentioned to you at summit has become a blocker14:00
*** dtrainor has quit IRC14:01
*** dtrainor has joined #tripleo14:01
*** bvandenh has quit IRC14:02
*** absubram has quit IRC14:06
*** absubram has joined #tripleo14:06
dprincerbrady: link?14:07
rbradydprince: http://paste.openstack.org/show/497080/14:08
rbradydprince: it doesn't just happen on the cli, but all calls to api (js and curl too)14:08
EmilienM@all tripleo CI should be green now, the puppet-ironic revert has been merged14:09
EmilienMslagle: ^14:09
rbradydprince: also http://pastebin.test.redhat.com/37406414:09
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Cleanup Puppet code that deploys Ironic  https://review.openstack.org/31523814:12
*** jobewan has joined #tripleo14:13
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Cleanup Puppet code that deploys Ironic  https://review.openstack.org/31523814:13
*** absubram has quit IRC14:14
*** jcoufal has quit IRC14:16
dprincerbrady: how old is your undercloud?14:16
rbradydprince: I'm seeing this issue on underclouds as new as 8 days old.  Mine is from 4/28/201614:19
*** jaosorior has quit IRC14:19
rbradydprince: I have an older undercloud from january still running without the issue14:19
*** links has joined #tripleo14:19
dprincerbrady: mine is 15 days old and those seem to work14:24
dprincerbrady: the zaqar one did anyway14:24
dprincerbrady: yeah, both work for me. So maybe it is a recent regression ?14:25
rbradydprince: it's at least prior to summit.14:27
*** Erming_ has joined #tripleo14:28
*** d0ugal has quit IRC14:29
*** links has quit IRC14:29
jtomasekrbrady, dprince: the response of the request which creates the execution timeouts14:29
jtomasekrbrady, dprince: the execution gets created just fine, but the response for the request which creates it never arrives (error 500)14:29
dprincecould be an oslo.messaging bug too14:30
dprincehttp://git.openstack.org/cgit/openstack/oslo.messaging/commit/?id=776871f1a12690d80abb78007318330630ef0cdf14:30
*** d0ugal has joined #tripleo14:30
*** krickwix has joined #tripleo14:30
*** Erming_ has quit IRC14:32
*** jcoufal has joined #tripleo14:37
*** d0ugal has quit IRC14:38
*** shardy has joined #tripleo14:39
openstackgerritSanjay Upadhyay proposed openstack/tripleo-specs: New Spec: tripleo-ovs-dpdk  https://review.openstack.org/31387114:39
openstackgerritImre Farkas proposed openstack/tripleo-heat-templates: [WIP] Add Ironic to overcloud  https://review.openstack.org/31612814:40
openstackgerritSanjay Upadhyay proposed openstack/tripleo-specs: New Spec: tripleo-ovs-dpdk  https://review.openstack.org/31387114:42
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071314:42
openstackgerritImre Farkas proposed openstack/tripleo-puppet-elements: Add Ironic packages to overcloud controller  https://review.openstack.org/31613214:43
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Composable manila tripleo-heat-templates side  https://review.openstack.org/31565814:44
*** mbound has quit IRC14:48
*** gfidente has joined #tripleo14:48
slagledprince: should i try force-deleting the nodepool instances that are stuck in deleting in our overcloud?14:51
dprinceslagle: I think that would be fine, try it14:52
*** zoli is now known as zoli|gone14:53
*** rbrady has quit IRC14:54
*** zoli|gone is now known as zoliXXL14:55
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071314:55
zoliXXLhave a good weekend14:55
*** zoliXXL is now known as zoli|gone14:55
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB  https://review.openstack.org/31505114:55
*** zoli|gone is now known as zoli_gone-proxy14:56
openstackgerritMerged openstack/tripleo-quickstart: gate injection: update only gated packages  https://review.openstack.org/31606314:56
slagledprince: k, all cleaned up14:57
openstackgerritImre Farkas proposed openstack/python-tripleoclient: Add Ironic password generation during deployment  https://review.openstack.org/31614614:59
dprinceslagle: thanks14:59
slaglewe're up to the 54 jenkins instances now15:00
slagle*slaves15:00
*** dbecker has quit IRC15:00
openstackgerritImre Farkas proposed openstack/tripleo-heat-templates: [WIP] Add Ironic to overcloud  https://review.openstack.org/31612815:01
openstackgerritMarios Andreou proposed openstack/puppet-tripleo: Adds the puppet-tripleo manifests for manila  https://review.openstack.org/31352715:01
*** julim has quit IRC15:05
*** d0ugal has joined #tripleo15:06
openstackgerritBrad P. Crochet proposed openstack/puppet-tripleo: Add Heat profiles  https://review.openstack.org/31301915:07
*** julim has joined #tripleo15:08
*** rbrady has joined #tripleo15:13
*** dmk0202 has quit IRC15:13
*** saneax is now known as saneax_AFK15:13
slagleSlower: jpeeler : do you think we still need the containers job in tripleo-ci?15:17
slaglei wasn't sure if anyone was still actively working on getting it to pass again15:17
*** openstackgerrit has quit IRC15:17
*** openstackgerrit has joined #tripleo15:18
shardyslagle: rhallisey was working on it at one point, but last I heard he'd stopped pending the composable services w/containers work15:18
jistrccamacho: can you explain a bit more about the thing you mentioned on scrum -- did i get it right you were looking for some tips/tricks how to make it easier to test changes?15:18
slagleshardy: ok, i'm mostly wondering if it's worth the cost of keeping testenv's tied up for 15-20 minutes at a time, for every patchset15:19
shardyslagle: it's already wired to fail-fast, so we could just remove it until such time as we've got something to test15:19
jpeeleryeah i'm sure ryan would prefer to have it temporarily disabled. the intention is definitely to make it work15:20
ccamachojistr, hey!  yeahp something like ::tripleo::services::noop to be called from step_config that gets all parameters and record them in /var/log/something.. In order to be able to separate THT from puppet-tripleo logic15:22
shardyOk sounds like we need a clearer confirmation if it's going to start working in the next days/weeks15:22
slaglejpeeler: i'm not sure there's a way to completely disable it w/o removing it, but even if we end up romving it, it'd be very easy to add back15:22
ccamachonot sure if already exists something similar15:22
* slagle adds to meeting agenda for next week15:23
jistrccamacho: so something like "unit testing" for the interface between t-h-t and puppet?15:23
jpeelerslagle: hrm, well unfortunately ryan is out today getting widsom teeth removed. if it needs to be removed due to resources, i think that's ok for now.15:23
ccamachojistr, indeed15:23
*** apetrich has quit IRC15:24
jistrccamacho: i don't think anything like that exists yet. Personally i like testing the things in parallel actually, so that i can verify that both my t-h-t and puppet changes work together correct and do the thing i intended to do.15:24
jistrccamacho: are you using the deployment artifacts upload via swift, btw? That makes testing changes to puppet modules much easier than e.g. amending the overcloud-full image.15:25
*** honza has joined #tripleo15:25
ccamachoyour right, but even is good, rebuild the overcloud images takes more or less time.. and for simple tests might be helpful15:26
ccamachojistr, any docs about it (deployment artifacts upload via swift) ?15:26
*** ebarrera has quit IRC15:27
jistrccamacho: here's a spec but it doesn't describe the actual commands much https://github.com/openstack/tripleo-specs/blob/master/specs/mitaka/puppet-modules-deployment-via-swift.rst15:28
ccamachojistr thanks! that will work15:29
jistrccamacho: i have two related scripts in the repo i linked on scrum, it's not public b/c it contains some internal-only scriptlets too, i'll repaste here15:29
jistrccamacho: so first on undercloud as stack user i clone all the openstack puppet modules from master branches using this script https://paste.fedoraproject.org/366057/53381146/raw/15:30
pabelangerjust checking: http://tripleo.org/cistatus.html looks like only 1 success job since bringing tripleo-ci back online. Looks like an issue with puppet-ironic15:31
*** ebalduf has quit IRC15:31
jistrccamacho: and then i use this script which uploads the cloned puppet modules to swift https://paste.fedoraproject.org/366061/15347014/raw/15:31
*** absubram has joined #tripleo15:31
pabelangerokay, I see a revert and patch15:32
jistrccamacho: and it generates puppet-modules.yaml environment file, which can be added to deployment command ("-e puppet-modules.yaml") to make sure you're using the cloned puppet modules15:33
*** yamahata has joined #tripleo15:33
jistrccamacho: after every manual change to any of the puppet modules you can re-run the upload to swift, and rerun `overcloud deploy` with the new env file. This is a bunch faster than rebuilding or editing the image.15:34
*** Erming_ has joined #tripleo15:35
*** mikelk has quit IRC15:36
jistrccamacho: also as a side note about "unit testing" the interface between tht and puppet -- much of the interface between the two consists of hiera files, which you can find under /etc/puppet/hieradata on overcloud nodes, there you can inspect what t-h-t generated, which then gets passed to puppet15:37
ccamachojistr, testing it right now :)15:38
ccamachojistr thanks!!!15:38
*** mgould has quit IRC15:38
jistrcool, np :)15:39
*** aufi has quit IRC15:41
*** cwolferh has joined #tripleo15:42
*** panda has quit IRC15:42
*** panda has joined #tripleo15:42
bnemecHmm, I see a bunch of testenv instances in error state.  Is that expected?15:45
bnemec| e2823959-ef42-4a04-be17-96a25da702a7 | testenv33-testenv0-dr3zjvqsxf62                      | ERROR   | deleting   | Shutdown    | ctlplane=10.1.8.91  |15:45
*** absubram has quit IRC15:47
*** jobewan is now known as jobewan_away15:48
bnemecThat one looks like the ipmi issue, but there are a bunch with NoValidHost messages. :-/15:48
*** mbound has joined #tripleo15:48
*** jobewan_away has quit IRC15:48
dtrainoris there ever a point where overcloud systems require a reboot?  i don't believe so because i believe them to be largely immutable but I'd like to hear that from someone else as well15:49
slagleEmilienM: dprince : I think this may have broken us now: https://review.openstack.org/#/c/31577015:49
*** mgould has joined #tripleo15:49
slagleEmilienM: dprince : a couple triple jobs passed, but now I'm seeing failures with: http://paste.openstack.org/show/497098/15:50
*** ifarkas has quit IRC15:51
*** mbound has quit IRC15:53
slaglerevert: https://review.openstack.org/#/c/316167/15:55
*** tesseract has quit IRC15:56
*** absubram has joined #tripleo15:56
*** dbecker has joined #tripleo15:58
*** jcoufal has quit IRC16:00
*** rcernin has quit IRC16:02
*** jobewan has joined #tripleo16:05
*** hjensas has quit IRC16:07
*** dsariel has joined #tripleo16:09
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Update keystone service name for signing keys  https://review.openstack.org/31617716:11
*** mcornea has quit IRC16:15
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles  https://review.openstack.org/31617916:16
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles  https://review.openstack.org/31617916:18
weshaylarsks, I have a new quickstart image that's missing the undercloud-passwords.conf file.  Do you know where that is generated?16:19
weshaylarsks, nvrmind16:22
openstackgerritCarlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles  https://review.openstack.org/30478016:23
weshaytag failure16:23
larsksweshay: oh, good :)16:25
EmilienMslagle: well this one actually fixed our CI16:26
EmilienMslagle: RDO packaging dropped the script16:26
EmilienMslagle: https://github.com/rdo-packages/keystone-distgit/commit/9dbd2556545afff77ec1d33449dd159eeb34be5916:26
EmilienMslagle: https://review.openstack.org/#/c/316177/116:27
*** fragatin_ has joined #tripleo16:29
*** dbecker has quit IRC16:29
EmilienMslagle: the thing is, if we revert the patch, we break puppet CI16:31
EmilienMor make it very unstable16:31
EmilienMI would rather wait for https://review.openstack.org/#/c/316177/16:31
EmilienMdoes it work for you?16:31
*** shardy has quit IRC16:32
*** fragatina has quit IRC16:32
*** apetrich has joined #tripleo16:34
*** athomas has quit IRC16:34
*** trozet has joined #tripleo16:34
*** dbecker has joined #tripleo16:37
*** yamahata has quit IRC16:38
openstackgerritAthlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge.  https://review.openstack.org/31071316:42
*** jistr has quit IRC16:42
*** Guest56726 has quit IRC16:42
*** Goneri has quit IRC16:44
*** lucasagomes is now known as lucas-afk16:45
*** coolsvap has quit IRC16:48
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: use nthost() instead of ipaddr('address') filter  https://review.openstack.org/31618516:49
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: use nthhost() instead of ipaddr('address') filter  https://review.openstack.org/31618516:51
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: use nthhost() instead of ipaddr('address') filter  https://review.openstack.org/31618516:52
larsksSorry for the gerrit spam, apparently I can't type.16:53
*** trozet has quit IRC16:53
Erming_larsks: do you have an idea why introspection doesn't work for me even after re-installed undercloud?16:54
larsksErming_: I'm not sure. In what way is it failing?16:54
*** krickwix has quit IRC16:55
*** devvesa has quit IRC16:55
Erming_larsks: I reinstalled undercloud since adding nodes and failed to do registration and introspection, then now I tried to run "openstack baremetal configure boot" it doesn't change the power state of new nodes.16:55
*** mgould has quit IRC16:56
*** rajinir has quit IRC16:56
*** igorbelikov has quit IRC16:56
larsksAre you able to manually control the power state of new nodes by running "ironic node-set-power-state ..."?16:56
Erming_larsks: no error messages just no change. and " openstack baremetal introspection start bulk" has no effect.16:56
Erming_let me try.16:57
*** electrofelix has quit IRC16:59
Erming_larsks: doesnt work. Do I need clean something in the host? I thougt re-install an undercloud should have cleaned everything16:59
*** egafford has quit IRC17:00
*** egafford1 has joined #tripleo17:00
Erming_Can I just remove the qcow2 files of all the virtual nodes in /var/lib/libvirt/images?17:00
larsksErming_: are you working with the quickstart environment?  Or are you doing things by hand?17:00
Erming_I am using trown's quickstart.17:01
Erming_curl -O https://raw.githubusercontent.com/openstack/tripleo-quickstart/master/quickstart.sh;bash quickstart.sh $VIRTHOST17:01
larsksOkay.  So, what do you mean by "re-installing the undercloud"?  In general, with the quickstart, there is no "reinstall", there is only "tear it all down and redeploy"...17:02
Erming_yes17:02
Erming_but why the new undercloud still seems not good17:02
Erming_the previous one was good but I failed to add nodes later. since only 2 nodes in instackevn.json.17:03
larsksSo now you've redeployed, and all the nodes are in instackenv.json, right?17:03
Erming_yes17:04
Erming_larsks: I pre-added all the nodes into instackenv.json17:04
*** egafford1 has quit IRC17:04
Erming_and then it doesn't work. you may want to have a close look if you want, on the machine. If you have time. I mean17:05
larsksErming_: I have to get a few things out of the way first, but I will try to take a look later today.  I am still a little confused about the state of things: normally, the quickstart populates instackenv.json for you.  Are you manually adding additional nodes?17:06
Erming_larsks: yes I added.17:06
Erming_larsks: as it has only 2 nodes now. before it had 5 nodes.17:07
larsksOkay.  Do the nodes that were populated by the quickstart work correctly?  That is, are you only having problems with the nodes that were added manually?17:07
openstackgerritgreghaynes proposed openstack/diskimage-builder: Install docker for tests  https://review.openstack.org/27802117:07
openstackgerritgreghaynes proposed openstack/diskimage-builder: Fail functests if refusing to run tests  https://review.openstack.org/17700217:07
openstackgerritOpenStack Proposal Bot proposed openstack/os-collect-config: Updated from global requirements  https://review.openstack.org/26444617:07
*** fragatin_ has quit IRC17:07
*** jpena is now known as jpena1off17:07
Erming_larsks: no, this time looks all nodes are inoperable.17:07
Erming_The last time only the added ones not good.17:08
larsksOkay.  I will ping you later today if I have time.  What timezone are you in?17:08
Erming_mountain time17:08
larsksAwesome.17:08
Erming_what time ok for you roughly?17:08
larsksI'm not sure; it depends on how quickly I am able to clear some other things off my plate.17:08
openstackgerritOpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements  https://review.openstack.org/26852817:09
Erming_ok. take your time. thanks for generousity on time17:09
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements  https://review.openstack.org/31377717:10
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB  https://review.openstack.org/31505117:10
*** ohamada has quit IRC17:10
*** rajinir has joined #tripleo17:12
*** ccamacho is now known as ccamacho|hagw17:12
*** yamahata has joined #tripleo17:20
*** _milan_ has quit IRC17:28
*** athomas has joined #tripleo17:28
*** tosky has quit IRC17:30
*** yamahata has quit IRC17:30
*** sambetts is now known as sambetts|afk17:31
*** yamahata has joined #tripleo17:34
*** igorbelikov has joined #tripleo17:36
*** pkovar has quit IRC17:40
*** pblaho has quit IRC17:41
*** florianf has quit IRC17:43
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates: Updates ControlPlaneSubnetCidr to be a string  https://review.openstack.org/31623217:46
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates: Updates ControlPlaneSubnetCidr to be a string  https://review.openstack.org/31623317:46
*** shivrao has joined #tripleo17:51
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB  https://review.openstack.org/31505117:53
bnemecEmilienM: slagle: https://review.openstack.org/#/c/316177 passed nonha.  Shall we just merge it and unblock everything else?17:53
openstackgerritgreghaynes proposed openstack/diskimage-builder: Allow setting per-release mirrors for tests  https://review.openstack.org/27969117:54
*** dbecker has quit IRC17:58
slaglebnemec: probably, yea17:59
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB  https://review.openstack.org/31505118:02
*** shivrao_ has joined #tripleo18:02
*** shivrao has quit IRC18:05
*** shivrao_ is now known as shivrao18:05
rbradydprince, flfuchs, jtomasek: I've been unable to pin down a root cause.  It seems to randomly work http://paste.openstack.org/show/497114/18:17
*** jdob has quit IRC18:18
*** jdob has joined #tripleo18:18
dprincerbrady: can you file a Mistral bug w/ what you are seeing?18:19
rbradydprince: of course18:19
dprincerbrady: I think shardy mentioned he hit similar. I'll rebuild my UC and try to hit it soon enough too18:19
*** cwolferh has quit IRC18:20
*** ccamacho|hagw has quit IRC18:21
*** trozet has joined #tripleo18:21
slaglebnemec: i +A'd it18:27
*** slagle changes topic to "TripleO: http://tripleo.org/ | https://wiki.openstack.org/wiki/TripleO | Meetings On Tuesdays at 14:00 UTC in #openstack-meeting-alt"18:30
slaglei dont remember what all was in the topic, but we'll go with that18:30
*** fragatina has joined #tripleo18:31
EmilienMbnemec, slagle: am I wrong or jobs are faster?18:31
bnemecEmilienM: We doubled the memory, so they should be.18:31
EmilienMgreat18:32
slagleEmilienM: yea, they're looking a little faster18:32
bnemecAnd this is before we even started using the new SSDs.18:32
slaglewe are using the ssd's18:32
bnemecslagle: Oh, you guys were busy last night. :-)18:32
dprincebnemec: we did the rest of the SSD's18:33
slaglebnemec: oh i didnt do it, it was derekh as usual :)18:33
dprincederekh brought up the cloud afterwards18:34
slaglei double checked one of the testenv hosts this morning and confirmed the ssd was in use18:34
dprinceyes, /var/lib/libvirt/images seems to be using the new mount point18:34
bnemecNice18:34
openstackgerritMerged openstack/puppet-tripleo: Update keystone service name for signing keys  https://review.openstack.org/31617718:36
*** jtomasek_ has joined #tripleo18:36
*** cwolferh has joined #tripleo18:37
*** jtomasek_ has quit IRC18:38
*** mbound has joined #tripleo18:38
*** jtomasek_ has joined #tripleo18:39
*** ayoung has quit IRC18:42
*** morazi has quit IRC18:44
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo: deploy keystone at step 3 and 4  https://review.openstack.org/31594318:48
*** gfidente has quit IRC18:52
*** Goneri has joined #tripleo18:56
larsksHey folks: what could result in stack going to CREATE_FAILURE  when there don't appear to be any failed resources?18:59
*** jdob has quit IRC19:09
*** jdob has joined #tripleo19:10
*** fpan has joined #tripleo19:10
rbradydprince, shardy, jtomasek, flfuchs: mistral bug - https://bugs.launchpad.net/mistral/+bug/158164919:11
openstackLaunchpad bug 1581649 in Mistral "Action Execution Response Timeout" [Undecided,New]19:11
larsksThe actual error I'm seeing appears to be "ResourceFailure: resources[0]: Resource CREATE failed: The Referenced Attribute (ManagementPort ip_subnet) is incorrect."19:17
larsksI'm not setting that explicitly anywhere, but I'm wondering if that is dependent on something else.19:18
*** pradk has quit IRC19:19
*** dsariel has quit IRC19:24
*** pradk has joined #tripleo19:26
*** jtomasek_ has quit IRC19:27
*** egafford has joined #tripleo19:29
*** ayoung has joined #tripleo19:30
bnemeclarsks: That sounds like a Heat bug.  When was your undercloud built?19:36
larsksI'm using an undercloud image from May 10.19:38
larsks(mitaka)19:38
bnemecHmm, I wonder if a fix backport might have gotten missed.19:39
*** panda has quit IRC19:42
*** panda has joined #tripleo19:42
larsksI can try an updated image.19:44
larsksThe download will take a while :)19:44
openstackgerritBen Nemec proposed openstack/instack-undercloud: Test overcloud deletion in Mitaka  https://review.openstack.org/31625719:44
bnemeclarsks: ^ is what used to trigger that error for me.19:45
larsksbnemec: is that the right link?  That appears to be a whitespace-only change to the README.19:46
bnemeclarsks: Yes, the depends-on is the important part.19:46
bnemecI just needed a change on the mitaka branch so it would run with the right bits.19:46
larsksbnemec: I am super confused.  There's basically nothing in that review to which you just linked.19:47
bnemeclarsks: Depends-On: I2ba406ec463233387211fc559e169d4b543cefd719:47
bnemecThat's a ci change to delete the overcloud.19:47
larsksOh, see, I though you were talking about a heat depends, and I wasn't seeing any heat config :)19:47
*** diazjf has joined #tripleo19:48
larsksWell, it looks like in 15 minutes or so I will have a more recent undercloud image; I'll see how that works out.19:48
EmilienMslagle, bnemec: before I leave, I wanted to highlight https://review.openstack.org/#/c/316241/ -- we need to be super careful with this one. I want to make sure puppet-keystone will do its job and update the endpoint for the compute service. Please give any feedback in the patch, thanks.19:50
* bnemec makes a note to add puppet module support to his undercloud upgrade job19:52
weshayquick question.. what sets the SaharaPassword in tripleo now that the template does not have a default?19:59
openstackgerritLars Kellogg-Stedman proposed openstack/tripleo-quickstart: fix network configuration  https://review.openstack.org/31626420:00
bnemecweshay: The client generates passwords for all the services.20:02
weshaybnemec, k.. thanks20:02
*** dmk0202 has joined #tripleo20:03
*** paramite has quit IRC20:09
*** jayg is now known as jayg|g0n320:15
*** diazjf1 has joined #tripleo20:16
*** diazjf has quit IRC20:18
* slagle wonders if http://tripleo.org/cistatus.html is hardcoded to show red20:21
*** dmk0202 has quit IRC20:22
slaglethe puppet-tripleo that passed isn't shown20:22
bnemecslagle: I don't think the script has the puppet jobs included yet.20:22
bnemecIt got lost in the transition from jenkins to gerrit.20:22
*** morazi has joined #tripleo20:24
slagleok20:25
slagleand it looks like the last few patches that failed across the board were for valid reasons20:25
*** jdob has quit IRC20:25
bnemechttp://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo-jobs-gerrit.py#n1720:26
*** jdob has joined #tripleo20:26
bnemecIn case anyone feels motivated to add all the puppet modules we use to that list.20:26
dprincebnemec: we should add the puppet jobs there, want me to do it?20:27
dprinceI wish there was a better way to search for them but I couldn't think of anything that was efficient20:27
openstackgerritJames Slagle proposed openstack/instack-undercloud: Consolidate undercloud* elements  https://review.openstack.org/31509620:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Move parts of undercloud-install to puppet  https://review.openstack.org/31293120:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Add timestamps to puppet apply output  https://review.openstack.org/31512820:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Use tripleo::selinux  https://review.openstack.org/31056520:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Minimize element usage  https://review.openstack.org/31509920:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Move 02-undercloud-stack-heat-metadata to python module  https://review.openstack.org/31421920:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Move persistent journal to puppet  https://review.openstack.org/31056420:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Remove truncate nova config workaround  https://review.openstack.org/31518420:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Don't use ironic-conductor element  https://review.openstack.org/31293220:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Don't use os-cloud-config/common-venv elements  https://review.openstack.org/30855020:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Create firewall rules with puppet  https://review.openstack.org/30837420:28
openstackgerritJames Slagle proposed openstack/instack-undercloud: Don't use os-net-config element  https://review.openstack.org/31509520:28
* slagle hogs queue20:28
slagleit was mostly empty anyway20:28
dprinceslagle: I was just waiting until we are sure its all green20:29
* dprince is fine waiting till the weekend to run his patches again20:29
slaglewe had one go all green :)20:29
bnemecdprince: I'm apparently not feeling motivated enough to do it. :-)20:30
bnemecslagle: just DoS'd my local Jenkins :-P20:30
dprincebnemec: I will, then I'll just pester you to review it :)20:30
slaglebnemec: lol20:30
*** trozet has quit IRC20:30
slaglemaybe i should work on a nefarious patch to hack your network20:31
dprincebnemec: https://review.openstack.org/#/c/313189/20:34
bnemecslagle: You could probably find some interesting things to mess with if you looked hard enough. :-)20:35
Erming_larsks: I will be out for about 1.5 hours so sorry if you don't see my response. but you can always access the node (cirrus) at any time. Thanks20:41
*** chem` has joined #tripleo20:47
*** chem has quit IRC20:49
dprincebnemec: I'm going to push a new patch for dereks tripleo-ci fix20:52
dprincebnemec: I think we want to see more than 20 t-h-t patches and I'm not too concerned about that overloading gerrit20:52
*** jdob has quit IRC20:54
bnemecwfm20:57
openstackgerritDan Prince proposed openstack-infra/tripleo-ci: Update the cistatus report  https://review.openstack.org/31318920:58
dprincebnemec: ^^^20:58
dprincebnemec: if you'll +2 I think we can go ahead and land that20:58
dprincebnemec: would be useful to have it posted to watch the queue history over the weekend20:59
bnemecdprince: lgtm21:00
openstackgerritMerged openstack-infra/tripleo-ci: Update the cistatus report  https://review.openstack.org/31318921:01
dprincebnemec: unmolested TripleO!21:01
dprincebnemec: I didn't realize that was a thing man21:02
* dprince reads Ben's latest blog post21:02
bnemecdprince: It's on the internet now, so it must be a thing. ;-)21:02
dprincebnemec: boom, new status page http://tripleo.org/cistatus.html21:05
bnemec\o/21:06
slaglenow with green!21:06
dprinceslagle: yeah green makes me feel better. Even if it is from only the stable branch sometimes21:07
dprincetoo bad derekh isn't around to bask in the glory of the faster running jobs21:08
slaglelooks like upgrades is checking in at right around 2h or a little less21:09
bandiniso I added a new password for a resource agent. new installs work fine, but updates break with ERROR: Failed to validate: Property error: resources[0].properties: Unknown Property MysqlClustercheckPassword21:09
*** lblanchard has quit IRC21:09
openstackgerritAndreas Florath proposed openstack/diskimage-builder: partitioning-parted: extraced and adapted element  https://review.openstack.org/31393821:09
bandinido I need to tweak tripleoclient as well to cover for updates?21:10
dprincebandini: patch link?21:10
bandinisec21:10
*** diazjf1 has quit IRC21:12
bandinidprince: http://people.redhat.com/mbaldess/0001-Tighten-the-access-rules-for-galera.patch (sorry was about to send proper review after I tested the updates :)21:13
dprincebandini: looks reasonable to me. The resources[0].properties implies the resource group. And the error says it is getting an invalid parameter. Did you add MysqlClustercheckPassword to the correct resource group?21:18
dprincebandini: like accidentally add it to the compute resource group in overcloud.yaml instead of the controller one?21:18
bandinidprince: oh let me double check that21:18
dprincebandini: it looks correct to me in the patch I think21:20
dprincebandini: curious if you try the update a second time if it works21:20
dprincebandini: could be a dependency issue in Heat too in that the nested stacks parameters haven't been updated yet.21:20
bandinidprince: trying21:20
bandinidprince: nope still complains (I am running openstack overcloud update stack --template ... -e .. for the record)21:21
bandiniyeah in the patch I put the new variable next to the mysql root one which was already defined. Sounded like the safest bet21:22
dprincebandini: throw up a patch and lets see what upstream CI does w/ it21:22
dprincebandini: it really does look okay to me21:22
bandinidprince: ack, let me open an LP first and link it to the two RH bugzillas21:23
bandinithat way we do the complete dance here21:23
*** egafford has quit IRC21:23
dprincebandini: think of all the LP karma you are getting though. Totally worth it man21:23
bandinidprince: ahahah I know right? /me opens a bottle of champagne to celebrate21:24
bandinidprince: should I flag the "This is a security vulnerability" thingy?21:25
bandiniI mean, it is a bit of one, I guess21:26
bandinidunno21:26
dprincebandini: well, your patch is public now. your call21:26
dprincebandini: I'd just file it21:26
bandiniack21:26
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Tighten the access rules for galera  https://review.openstack.org/31629721:35
larsksbnemec:  (et al) With the latest mitaka undercloud image, I still see "ResourceFailure: resources[0]: resources.Controller.Resource CREATE failed: The Referenced Attribute (ManagementPort ip_subnet) is incorrect."21:41
*** thrash is now known as thrash|wknd21:41
larsksWhat does that error mean?  Does "is incorrect" mean "does not exist"?  Or does it mean something else?21:42
*** fultonj has quit IRC21:43
bnemeclarsks: In my experience it usually means Heat added some validation somewhere that is breaking on our net-iso templates.21:45
*** dprince has quit IRC21:45
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron server  https://review.openstack.org/31512921:47
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron ml2 and ovs agent  https://review.openstack.org/30851421:47
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron metadata service  https://review.openstack.org/30361821:47
openstackgerritDan Prince proposed openstack/tripleo-heat-templates: composable neutron l3 service  https://review.openstack.org/30356221:47
*** dmk0202 has joined #tripleo21:53
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates: Tighten the access rules for galera  https://review.openstack.org/31629722:00
openstackgerritMerged openstack/os-collect-config: Switch to oslo.log again  https://review.openstack.org/29590322:03
*** rlandy has quit IRC22:05
*** dmk0202 has quit IRC22:06
*** fragatina has quit IRC22:11
*** MaxPC has quit IRC22:17
larsksbnemec: thanks. So i should be looking at parameter validation declared in the templates? Or heat code around the site of the traceback?22:22
*** fragatina has joined #tripleo22:26
*** fragatina has quit IRC22:26
*** fragatina has joined #tripleo22:27
openstackgerritJason E. Rist proposed openstack/tripleo-ui: Ignore the .idea folder (JetBrains)  https://review.openstack.org/31630822:31
*** yamahata has quit IRC22:31
*** yamahata has joined #tripleo22:34
*** lblanchard has joined #tripleo22:37
*** apetrich has quit IRC22:40
*** lblanchard has quit IRC22:55
*** lucas-afk has quit IRC23:05
*** jobewan has quit IRC23:10
*** lucasagomes has joined #tripleo23:12
*** cwolferh has quit IRC23:17
*** absubram has quit IRC23:27
*** cwolferh has joined #tripleo23:33
*** saneax_AFK is now known as saneax23:42
*** panda has quit IRC23:42
*** panda has joined #tripleo23:42
*** r-mibu has quit IRC23:45
*** r-mibu has joined #tripleo23:45
*** mbound has quit IRC23:49

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!