Thursday, 2016-01-07

*** absubram has quit IRC00:17
openstackgerritReedip proposed openstack/os-cloud-config: Remove support for remove py33/py26  https://review.openstack.org/26448100:21
openstackgerritReedip proposed openstack/os-cloud-config: Remove support for py33/py26  https://review.openstack.org/26448100:22
openstackgerritJames Slagle proposed openstack/instack-undercloud: Make 98-undercloud-setup rerunnable  https://review.openstack.org/25494600:44
openstackgerritJames Slagle proposed openstack/instack-undercloud: Set nova ironic admin_url correctly  https://review.openstack.org/26449200:44
openstackgerritJames Slagle proposed openstack/instack-undercloud: Manage keystone related configuration via Puppet  https://review.openstack.org/26449300:44
*** dmacpher has joined #tripleo00:48
*** eil397 has quit IRC01:03
*** absubram has joined #tripleo01:42
*** absubram_ has joined #tripleo01:43
*** absubram has quit IRC01:47
*** absubram_ is now known as absubram01:47
*** julim has joined #tripleo02:00
*** sthillma has quit IRC02:09
*** sthillma has joined #tripleo02:14
*** trozet has joined #tripleo02:18
*** absubram has quit IRC02:28
*** sthillma has quit IRC02:59
*** links has joined #tripleo03:10
*** links has quit IRC03:10
*** links has joined #tripleo03:12
*** apetrich_ has joined #tripleo03:16
*** apetrich has quit IRC03:18
*** trozet has quit IRC03:33
*** yamahata has quit IRC03:36
*** thrash is now known as thrash|g0ne03:42
*** sthillma has joined #tripleo03:45
*** tzumainn has quit IRC03:47
*** sthillma_ has joined #tripleo03:48
*** sthillma has quit IRC03:50
*** sthillma_ is now known as sthillma03:50
*** rhallisey has quit IRC03:51
*** coolsvap|away is now known as coolsvap04:21
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Update to pull in rspec-puppet-facts  https://review.openstack.org/26456104:28
*** jaosorior has joined #tripleo04:45
*** dmacpher has quit IRC04:58
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient: Fix chicken and eggs problem with tripleo-heat-templates  https://review.openstack.org/26457105:03
*** absubram has joined #tripleo05:06
*** dmacpher has joined #tripleo05:14
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient: Move more functions to utils.py  https://review.openstack.org/26457405:15
*** masco has joined #tripleo05:41
*** absubram has quit IRC06:05
*** sthillma has quit IRC06:35
*** sthillma has joined #tripleo06:38
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/os-net-config: Use assertTrue/False instead of assertEqual(T/F)  https://review.openstack.org/26460706:45
*** coolsvap is now known as coolsvap|away06:56
*** oshvartz has joined #tripleo07:09
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/python-tripleoclient: Use assertTrue/False instead of assertEqual(T/F)  https://review.openstack.org/26463407:20
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/tuskar: Use assertTrue/False instead of assertEqual(T/F)  https://review.openstack.org/26463907:25
*** jprovazn has joined #tripleo07:26
*** ukalifon1 has joined #tripleo07:36
*** coolsvap|away is now known as coolsvap07:41
*** jtomasek has joined #tripleo07:53
*** dmacpher has quit IRC07:56
*** liverpooler has joined #tripleo07:58
*** stendulker has joined #tripleo08:16
*** bvandenh has joined #tripleo08:17
*** aufi has joined #tripleo08:23
*** sthillma has quit IRC08:28
*** jtomasek has quit IRC08:29
*** jtomasek has joined #tripleo08:30
*** devvesa has joined #tripleo08:33
*** gfidente has joined #tripleo08:35
*** gfidente has quit IRC08:35
*** gfidente has joined #tripleo08:35
*** olap_ has joined #tripleo08:37
*** ifarkas has joined #tripleo08:40
*** mcornea has joined #tripleo08:45
*** jcoufal has joined #tripleo08:57
*** athomas has joined #tripleo09:01
*** jaosorior has quit IRC09:17
*** bvandenh has quit IRC09:21
*** nico_auv has joined #tripleo09:23
openstackgerritJaume Devesa proposed openstack/tripleo-heat-templates: Changes for configuring Nuage  https://review.openstack.org/26039909:25
*** jaosorior has joined #tripleo09:32
*** bvandenh has joined #tripleo09:33
*** derekh has joined #tripleo09:35
*** shardy has joined #tripleo09:35
*** jistr has joined #tripleo09:41
*** bvandenh has quit IRC09:41
* jistr forgot to start irc :)09:41
*** yamahata has joined #tripleo09:41
jistrgfidente: i'll rebase the upgrade patch chain and do a few small tweaks, sounds good?09:50
gfidentejistr, ack go ahead, i'm on call with marios talking about the tht stable/libery branch09:51
jistrgfidente: alright. If you guys think i might be of some use on that, i can join you, otherwise i'm happy doing the upgrades amendments/rebase.09:52
*** yamahata has quit IRC09:53
*** dtantsur|afk is now known as dtantsur10:03
*** mkovacik has joined #tripleo10:04
devvesaHi shardy10:05
shardydevvesa: Hi!10:06
shardythanks for rebasing the stable/liberty patches10:06
devvesaAsking to you because I think you have all the documentation in your mind :)10:06
devvesashardy: Thanks for review them!10:06
*** paramite has joined #tripleo10:07
devvesaI have my virtual tripleo env : http://docs.openstack.org/developer/tripleo-docs/environments/environments.html#virtual-environment10:07
devvesaIs there any doc to follow to add more VMs?10:07
shardydevvesa: I normally just set NODE_COUNT before running instack-virt-setup, but do you need to add VMs to an existing environment?10:09
devvesashardy: exactly10:10
devvesaWell, I don't need it. I can build another environment, but I guess it will take more time10:10
shardydevvesa: Yeah, sorry we don't have that documented atm, would be a good idea tho10:15
*** electrofelix has joined #tripleo10:15
shardyit'll be a case of looking at instack-virt-setup and creating the VMs with virsh in a similar way10:15
shardythen update instackenv.json and register the new nodes10:15
shardytbh I normally just delete everything and re-run instack-virt-setup to get a clean environment10:15
devvesaI've done this... (update the instackenv.json) but the new node in ironic is in maintenance mode and it fails10:16
devvesaI mean:10:16
devvesaI run `tripleo.sh --register-nodes` and it fails10:17
devvesaand then the new node becomes on maintenance10:18
devvesaWell, nevermind. I'll create a new clean env :) With everything merged in master it will be easier than before10:18
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: TEST: this will help testing major upgrades  https://review.openstack.org/26045610:18
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Split pacemaker common check_service function out of _restart.sh  https://review.openstack.org/26044310:18
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Add resources for major upgrade in Pacemaker scenario  https://review.openstack.org/25327610:18
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Use timeout to check for services status  https://review.openstack.org/25957310:18
shardyhttps://github.com/openstack/instack-undercloud/blob/master/scripts/instack-virt-setup#L11310:19
shardydevvesa: I think you'll need to either re-run the tripleo devtest_testenv.sh, or figure out what it's doing with the nodes10:19
jistrgfidente: i only changed this one https://review.openstack.org/#/c/259573/2/extraconfig/tasks/pacemaker_resource_restart.sh, the rest is rebases10:19
shardyI'm not sure on the exact details without looking at the code10:20
jistrgfidente: lemme know if something seems bad about the changes10:20
jistrgfidente: i may go ahead and define an echo_error function in pacemaker_functions to DRY the "| tee /dev/fd2" parts10:22
jistroriginally shardy suggested we echo errors to stderr because that is processed separately by heat i think, and could be used to find the errors more easily than going through the full stdout output10:23
shardydevvesa: you can find it at /usr/libexec/openstack-tripleo/devtest_testenv.sh10:23
shardythere's scope for simplifying things I think, as this stuff is derived from the old TripleO devtest setup10:23
* jistr testing the upgrade locally now10:25
*** jcoufal has quit IRC10:28
*** jaosorior has quit IRC10:34
*** jaosorior has joined #tripleo10:35
*** krickwix has joined #tripleo10:39
*** tremble has joined #tripleo10:40
*** tremble has left #tripleo10:40
*** mgould has joined #tripleo10:53
*** paramite is now known as paramite|afk10:58
*** coolsvap is now known as coolsvap|away11:11
*** regebro has joined #tripleo11:16
*** stendulker has quit IRC11:18
*** lucas-dinner is now known as lucasagomes11:20
cmystershardy: ping11:23
shardycmyster: hi!11:23
cmysteryo11:24
cmysterpm?11:24
*** akrivoka has joined #tripleo11:24
openstackgerritMerged openstack/tripleo-heat-templates: Enable Dell Storage Center iscsi Backends in Cinder  https://review.openstack.org/24492311:25
openstackgerritMerged openstack/tripleo-common: Update the container flag  https://review.openstack.org/26437311:28
openstackgerritMerged openstack/tripleo-heat-templates: Enable configuration of Neutron QoS  https://review.openstack.org/24705611:40
openstackgerritMerged openstack/python-tripleoclient: Fix db error when running python34 unit tests  https://review.openstack.org/26403111:48
openstackgerritMerged openstack/instack-undercloud: Add ip addresses to horizon allowed_hosts  https://review.openstack.org/23493911:50
shardySimple instack-undercloud fix passing CI if someone has a moment:11:53
shardyhttps://review.openstack.org/#/c/224852/11:53
shardychecks for sufficient ram when installing the undercloud11:53
openstackgerritMerged openstack/tripleo-heat-templates: Add TimeZone parameter for all node types  https://review.openstack.org/24988611:56
*** paramite|afk is now known as paramite12:01
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: deploy-ironic: Fix syntax error when checking for root device hints  https://review.openstack.org/26473612:07
*** morazi has quit IRC12:08
*** jprovazn has quit IRC12:18
openstackgerritMerged openstack/instack-undercloud: Add check for sufficient memory to undercloud install  https://review.openstack.org/22485212:20
*** thrash|g0ne is now known as thrash12:20
*** jcoufal has joined #tripleo12:27
jistrhmm i'm still hitting the problem with stacks in UPDATE_IN_PROGRESS despite all resources are *_COMPLETE.. kinda blocks me from testing the upgrades on upstream environment. Is this perhaps fixed in a newer Heat release than what i have?12:29
jistr[stack@instack ~]$ rpm -q openstack-heat-engine12:29
jistropenstack-heat-engine-6.0.0-dev248.el7.centos.noarch12:29
jistri'm searching for more info on this but if someone knows something related off the bat, i'd be grateful for any input12:30
jistrhttp://fpaste.org/308152/14521692/raw/12:30
slaglejistr: there was a heat issue with that, if you restart heat-engine it should move the top level stack to failed12:42
slaglejistr: it's possible the delorean in use is too old to have that fix12:42
jistrslagle: yeah i recall doing that previously (before the holidays break), which got me to UPDATE_FAILED, but when i retried the stack update, it got stuck in exactly the same place...12:43
jistrslagle: i'll try to update to heat from delorean-current12:43
jistrthanks :)12:44
slaglei think the cause of it is an unhandled heat exception somewhere12:44
*** jprovazn has joined #tripleo12:46
openstackgerritMerged openstack/instack-undercloud: Set nova ironic admin_url correctly  https://review.openstack.org/26449212:49
*** gfidente is now known as gfidente|pto12:58
*** dprince has joined #tripleo13:07
*** morazi has joined #tripleo13:07
*** links has quit IRC13:07
shardyjistr: I think the fix may be https://review.openstack.org/#/q/Ic948c2fe5baf23c9c4ced33060f672ca9c278a19,n,z ?13:10
*** weshay_xchat has joined #tripleo13:10
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Enable configuration of Neutron QoS  https://review.openstack.org/26475713:10
*** tzumainn has joined #tripleo13:13
jistrshardy: yeah that could be it. I see the fix landed on Nov 18, while http://trunk.rdoproject.org/centos7/current-tripleo/ has heat from Nov 16, so the heat update could fix my issues indeed. Just finished overcloud deploy, now starting an upgrade to see if i can get further. Thanks :)13:14
shardyjistr: ouch, that's a pretty old current-tripleo pin :(13:15
jistryeah there were some issues with bumping that IIRC, not sure if they're resolved now13:15
*** jayg|g0n3 is now known as jayg13:16
shardyjistr: kk, FWIW I've been running with a local delorean built heat from heat master lately and it seems to work OK13:16
shardyI hit one issue when restarting heat-engine in the middle of an update yesterday, so I wouldn't reccomend doing that unless you have no other option13:17
shardyref https://bugs.launchpad.net/heat/+bug/153154713:17
openstackLaunchpad bug 1531547 in heat "Interrupted stack-update leaves undeletable stack" [High,In progress] - Assigned to Steven Hardy (shardy)13:17
jistrshardy: ack, thanks for the info13:17
*** Madkiss has quit IRC13:17
*** Madkiss has joined #tripleo13:17
*** Madkiss has joined #tripleo13:17
jistrgot further now (to a properly failed upgrade :) ) thanks shardy & slagle13:24
*** rhallisey has joined #tripleo13:25
*** trown|outttypeww is now known as trown13:35
openstackgerritLucas Alvares Gomes proposed openstack/diskimage-builder: Install IPA in a virtual enviroment  https://review.openstack.org/23584313:36
*** masco has quit IRC13:36
dtantsurlucasagomes, btw I've finished https://review.openstack.org/#/c/263309/ could you take a look?13:38
lucasagomesdtantsur, ++13:38
lucasagomeswill do now13:38
dtantsur+504, -55413:38
dtantsurmeh, not an easy patch to review :(13:38
*** rhallisey has quit IRC13:40
lucasagomesyeah it's hard to map what exactly have changed, because it was almost completely refactored (and that's a good thing, cause os-cloud-config is not flexible the way it is right now in tree)13:43
*** rhallisey has joined #tripleo13:43
*** rhallisey has quit IRC13:45
openstackgerritEmilien Macchi proposed openstack/instack-undercloud: Deploy Logging on the undercloud with Puppet  https://review.openstack.org/25865313:45
*** rhallisey has joined #tripleo13:46
dtantsurlucasagomes, yeah, it's essentially a full rewrite13:50
slaglelucasagomes: does this look at familiar at all? http://fpaste.org/308175/14521746/13:51
slaglelucasagomes: seeing that error on our stable/liberty CI jobs13:51
lucasagomesslagle, haven't seem that error before13:52
lucasagomesodd, lemme see if I can find something about it13:52
lucasagomesslagle, seems the nova code in liberty is depending upon some newer code from oslo.serialization?13:53
slaglelucasagomes: this looks suspect: https://review.openstack.org/#/c/246792/13:54
slaglemerged yesterday13:54
slagleor today rather13:54
slagleyea, that's got to be it13:55
lucasagomesslagle, yeah I bet it may be it13:55
lucasagomesdoes oslo.serialization have a stable branch too? If so, we need to make sure https://github.com/openstack/oslo.serialization/commit/92f2111b26d21a30e3dda62a942aa7f28ae41537 was backported to it13:56
lucasagomestho it would be odd because it's not fixing anything13:56
*** absubram has joined #tripleo13:57
*** rhallisey has quit IRC13:57
*** absubram_ has joined #tripleo13:58
lucasagomesdtantsur, reviewed, left a question at L246 of nodes.py13:59
dtantsurlucasagomes, ironicclient does retry on service unavailable: https://github.com/openstack/python-ironicclient/blob/master/ironicclient/common/http.py#L166-L16714:01
lucasagomesslagle, dump_as_bytes is not part of the liberty version of oslo.serialization https://github.com/openstack/oslo.serialization/blob/stable/liberty/oslo_serialization/jsonutils.py14:01
*** absubram has quit IRC14:01
*** absubram_ is now known as absubram14:01
lucasagomesdtantsur, oh14:04
lucasagomesdtantsur, ack, I thought it was only conflict14:04
*** rhallisey has joined #tripleo14:04
dtantsurlucasagomes, we implemented the same thing that nova used to retry14:05
lucasagomesdtantsur, gotcha! Missed that bit14:05
lucasagomeschanged my vote14:05
*** trozet has joined #tripleo14:06
*** trozet_ has joined #tripleo14:08
*** trozet_ has quit IRC14:09
lucasagomesslagle, should re revert that? Since the method is not available in the liberty version of oslo.serialization?14:09
*** trozet_ has joined #tripleo14:09
dtantsurI think revert or quick fix is required14:09
EmilienMshardy, dprince: does heat support versionless for auth_uri & identity_uri ? look my last comment on https://review.openstack.org/261326 for full context14:15
openstackgerritMerged openstack/puppet-tripleo: Upgrade all packages after puppet managed ones  https://review.openstack.org/26104114:15
openstackgerritMerged openstack/puppet-tripleo: Haproxy has non-working Horizon session persistence.  https://review.openstack.org/25848314:16
slaglelucasagomes: sorry, was on a call14:17
shardyEmilienM: Yes, I recently proposed some changes to devstack which switch all auth_uri's except the ec2authtoken one to versionless14:17
slaglelucasagomes: the needed patch isn't in oslo.serialization stable/liberty?14:17
EmilienMshardy: in liberty?14:17
shardyEmilienM: No, master/mitaka14:17
EmilienMshardy: ah14:17
EmilienMpuppet ci is on liberty atm14:17
EmilienMit might explain why14:17
shardyThe same config should work for liberty, but I've not tested it yet14:17
EmilienMshardy: please comment on the patch if we need to set the version14:18
lucasagomesslagle, yeah it's not in stable/liberty14:18
slagleyea i see14:18
lucasagomeshere's the jsonutils.py module for stable/liberty: https://github.com/openstack/oslo.serialization/blob/stable/liberty/oslo_serialization/jsonutils.py14:18
*** bvandenh has joined #tripleo14:19
slagleyea was just looking :)14:19
shardyEmilienM: so, master puppet-heat CI tests with liberty heat?14:19
EmilienMshardy: right.14:19
EmilienMshardy: tl;dr: we wait for trusty packaging to bump our CI to mitaka14:20
EmilienMshardy: should happen very very soon14:20
shardyEmilienM: Ah, OK14:20
shardyAFAIK the config should be the same, I was just confused :)14:20
EmilienMshardy: so you mean we could use versionless? See logs, it does not look like it works14:23
shardyhttp://paste.openstack.org/show/483229/14:23
shardyEmilienM: that's how we configure things on recent devstack14:24
shardyit's all versionless except ec2authtoken14:24
shardyEmilienM: I'm trying to see the difference between that and the puppet config14:24
EmilienMshardy: thanks, all heat config is in logs, you can easily find it14:25
*** lblanchard has joined #tripleo14:26
slaglelucasagomes: i submitted a nova revert, https://review.openstack.org/#/c/264793/14:28
lucasagomesthanks!14:28
shardyEmilienM: I think you're missing the domain configuration, which is required with versionless, because it defaults to the v3 auth plugin14:29
shardyI'll comment on the review14:29
EmilienMshardy: thx a lot14:29
*** trozet_ has quit IRC14:30
*** egafford has joined #tripleo14:30
*** trozet_ has joined #tripleo14:32
*** lucasagomes is now known as lucas-hungry14:32
*** trozet has quit IRC14:35
*** trozet_ is now known as trozet14:35
*** trozet_ has joined #tripleo14:39
*** rpothier has joined #tripleo14:44
openstackgerritMerged openstack/tripleo-heat-templates: Making nova parameters configurable for nuage-metadata-agent  https://review.openstack.org/26377314:48
slagleour stable/liberty CI is failing due to, https://bugs.launchpad.net/tripleo/+bug/153188114:49
openstackLaunchpad bug 1531881 in tripleo "AttributeError: 'module' object has no attribute 'dump_as_bytes'" [Critical,New] - Assigned to James Slagle (james-slagle)14:49
slaglefyi14:49
*** jcoufal_ has joined #tripleo14:50
EmilienMslagle, dprince: monitoring patch pass now CI and addressed all your reviews: https://review.openstack.org/#/c/253709/14:51
EmilienMthere is also its dep on puppet-elements: https://review.openstack.org/#/c/252051/14:52
EmilienMif we could land it soon or so, so we can iterate and get some feedback14:52
dprinceEmilienM: cool, I will review again shortly14:52
EmilienMthanks!14:52
*** liverpooler has quit IRC14:53
*** jcoufal has quit IRC14:53
*** dprince has quit IRC14:58
*** dprince has joined #tripleo14:59
*** pradk has joined #tripleo15:01
rhalliseydprince, morning, what was the reason why we arn't building the heat-docker-agents container with the openstack packages install?15:03
rhalliseyI don't remember15:03
dprincerhallisey: we were trying to avoid installing packages, because it means any package update would potentially require a new heat-docker-agents container as well15:05
dprincerhallisey: sorry, installing is fine, pre-installing was what we tried to avoid15:05
rhalliseydprince, would that be ok though?  swapping out the container15:05
dprincerhallisey: it is okay, as an iterim solution15:05
rhalliseysince that's how a general upgrade would work15:05
*** oshvartz has quit IRC15:06
rhalliseydprince, well what would be the long term solution though, or the ideal one15:06
rhalliseyI'd rather swap out a container for an update then doing a yum update15:06
*** mcornea has quit IRC15:08
*** egafford has quit IRC15:09
*** bvandenh has quit IRC15:09
*** bvandenh has joined #tripleo15:09
*** absubram has quit IRC15:12
*** yamahata has joined #tripleo15:18
*** jtomasek has quit IRC15:20
*** masco has joined #tripleo15:21
*** jtomasek has joined #tripleo15:25
*** nico_auv has quit IRC15:30
*** dprince has quit IRC15:32
*** dprince_ has joined #tripleo15:32
*** lucas-hungry is now known as lucasagomes15:34
*** slagle has quit IRC15:39
*** olap_ has quit IRC15:43
*** electrofelix has quit IRC15:50
*** dprince_ has quit IRC15:51
*** dprince has joined #tripleo15:51
*** electrofelix has joined #tripleo15:51
*** rook has quit IRC15:51
*** rook has joined #tripleo15:52
*** egafford has joined #tripleo16:02
*** absubram has joined #tripleo16:17
*** nico_auv has joined #tripleo16:18
* marios just release tripleo-heat-templates 0.8.7 (stable/liberty) and 1.0.0 (master) http://tarballs.openstack.org/tripleo-heat-templates/16:25
*** weshay_xchat has quit IRC16:32
*** aufi has quit IRC16:33
*** ifarkas has quit IRC16:33
*** weshay_xchat has joined #tripleo16:34
*** gfidente|pto has quit IRC16:36
lucasagomeshi all, someone can take a look at this patch https://review.openstack.org/#/c/239561 please ? It already has a 2+2 (and the failure there seems transient)16:39
lucasagomesthis is a dependency for another patch fixing the ironic-agent element when installing things from source16:39
*** masco has quit IRC16:39
*** sthillma has joined #tripleo16:52
*** ukalifon1 has quit IRC17:03
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Enable Equallogic Backends in Cinder  https://review.openstack.org/26487117:08
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Enable Dell Storage Center iscsi Backends in Cinder  https://review.openstack.org/26487217:08
openstackgerritMerged openstack/tripleo-heat-templates: Wire Neutron ML2 plugin and OVS agent settings as arrays  https://review.openstack.org/26041417:08
*** trown is now known as trown|lunch17:13
*** krickwix has quit IRC17:15
*** dtantsur is now known as dtantsur|afk17:15
*** slagle has joined #tripleo17:17
*** jprovazn has quit IRC17:22
openstackgerritMerged openstack/tripleo-heat-templates: Remove deleted Cinder rows  https://review.openstack.org/20901917:23
openstackgerritMerged openstack/tripleo-heat-templates: Use new heat-docker-agents images  https://review.openstack.org/25351517:30
shardyHey folks, this https://review.openstack.org/#/c/250498/ is passing, adds coverage of overcloud deletes to CI runs17:32
shardyneeds one more +2/A17:32
shardyAlso https://review.openstack.org/#/c/249819/ is another toci_instack patch17:33
openstackgerritMerged openstack-infra/tripleo-ci: Don't pass duplicate projects to tripleo.sh --delorean-build  https://review.openstack.org/24981917:36
jistri guess the overcloud delete adds a few minutes to the CI run time but probably is worth it. Maybe if we're really confident about overcloud deletes in the future, they can be removed again from CI.17:36
shardyjistr: it doesn't add much IME17:37
shardyjistr: and there's a whole class of bug folks get bitten by related to DELETE_FAILED stacks which we never hit in CI17:37
openstackgerritMerged openstack-infra/tripleo-ci: Add CI coverage of deleting the deployed overcloud  https://review.openstack.org/25049817:38
shardyjistr: In a followup patch I'll add some instrumentation to quantify it, but I'd be surprised if it's even one minute17:38
rhalliseyjistr, https://review.openstack.org/#/c/259441/ this backport has merged upstream17:38
rhalliseyjust an fyi17:38
rhalliseyas of like 1 min ago17:38
shardylocally my overcloud deletes take probably 20-30 seconds17:38
openstackgerritMerged openstack/tripleo-heat-templates: Use new heat-docker-agents images  https://review.openstack.org/25944117:40
jistrrhallisey: cool, thanks for letting me know17:40
jistrrhallisey: ^ in stable/liberty too now17:40
rhalliseyjistr, your clever :)17:41
rhalliseyjistr, thanks17:41
*** jcoufal_ has quit IRC17:44
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: TEST: this will help testing major upgrades  https://review.openstack.org/26045617:44
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Add resources for major upgrade in Pacemaker scenario  https://review.openstack.org/25327617:44
*** jistr has quit IRC17:45
*** mkovacik has quit IRC17:46
*** athomas has quit IRC17:51
*** gchamoul has quit IRC17:52
*** bvandenh has quit IRC18:01
*** akrivoka has quit IRC18:06
openstackgerritMerged openstack/tripleo-specs: New Spec: Deploy Puppet modules via Swift  https://review.openstack.org/24530918:06
*** mgould has quit IRC18:08
*** penick has quit IRC18:08
*** trown|lunch is now known as trown18:08
*** eil397 has joined #tripleo18:09
*** paramite is now known as paramite|afk18:11
*** derekh has quit IRC18:11
*** jayg has quit IRC18:13
*** bvandenh has joined #tripleo18:13
*** nico_auv has quit IRC18:13
*** jpeeler has quit IRC18:13
*** lblanchard1 has joined #tripleo18:13
*** masco has joined #tripleo18:14
*** rbrady has quit IRC18:14
*** trozet has quit IRC18:15
*** mburned has quit IRC18:15
*** jayg has joined #tripleo18:15
*** lblanchard has quit IRC18:15
*** trozet_ has quit IRC18:15
*** trown has quit IRC18:15
*** trown|brb has joined #tripleo18:16
*** weshay_xchat has quit IRC18:16
*** rhallisey has quit IRC18:16
*** masco has quit IRC18:19
*** rook has quit IRC18:20
*** rook-desktio has joined #tripleo18:21
*** trown|brb is now known as trown18:21
*** cwolferh has quit IRC18:21
*** cwolferh has joined #tripleo18:21
greghaynesbnemec: we should pin that fedora mirror in dib18:25
greghaynesI think I tried it a while back but realized it didnt 'just work'18:25
*** cwolferh has quit IRC18:26
*** trozet has joined #tripleo18:28
*** rhallisey has joined #tripleo18:28
*** weshay_xchat has joined #tripleo18:30
*** dsneddon has quit IRC18:33
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Build image files from definitions in yaml  https://review.openstack.org/23556918:34
*** dsneddon has joined #tripleo18:34
*** jaosorior has quit IRC18:34
*** jpeeler has joined #tripleo18:34
*** jaosorior has joined #tripleo18:35
*** trozet_ has joined #tripleo18:35
*** eggmaste` has joined #tripleo18:36
*** dsneddon has quit IRC18:36
*** dsneddon has joined #tripleo18:36
bnemecgreghaynes: It looks like DIB_DISTRIBUTION_MIRROR should work: https://github.com/openstack/diskimage-builder/blob/master/elements/fedora/pre-install.d/01-set-fedora-mirror18:36
*** eggmaster has quit IRC18:37
bnemecWe use it in toci and it helped with our mirror failures immensely.18:37
greghayneshrm18:37
*** dsneddon has quit IRC18:37
*** dsneddon has joined #tripleo18:37
*** dsneddon has quit IRC18:37
*** dsneddon has joined #tripleo18:38
greghayneshttp://git.openstack.org/cgit/openstack/diskimage-builder/tree/elements/fedora/test-elements/build-succeeds/environment.d/10-fedora-pin-mirror.bash18:38
greghaynesthat should be pinning it18:38
greghaynesbut it doesn't appear to be working18:38
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Use the new yaml syntax in CI  https://review.openstack.org/25488618:39
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Build image files from definitions in yaml  https://review.openstack.org/23556918:39
*** paramite|afk is now known as paramite18:42
*** trozet_ has quit IRC18:43
bnemecgreghaynes: Oh, the failure in http://logs.openstack.org/61/239561/6/check/gate-dib-dsvm-functests-devstack-centos7/0b750bd/console.html actually happens before the pin script gets run.18:43
bnemecThe other failure on that patch was downloading the base cloud image, so again before the pinning would happen.18:43
*** eggmaste` has quit IRC18:44
*** eggmaste` has joined #tripleo18:44
greghaynesah, right, and IIRC there isnt fedora pinning for that yet18:44
bnemecYou would have to export DIB_CLOUD_IMAGES: https://github.com/openstack/diskimage-builder/blob/master/elements/fedora/root.d/10-fedora-cloud-image#L2718:46
bnemecI'm not sure whether you could do that in an environment file or not.18:46
*** ayoung has quit IRC18:46
*** rbrady has joined #tripleo18:46
*** ayoung has joined #tripleo18:46
greghaynesWe should be able to18:46
*** pradk has quit IRC18:47
*** gchamoul has joined #tripleo18:47
*** trozet has quit IRC18:48
*** rbrady has quit IRC18:50
*** electrofelix has quit IRC18:58
*** trozet has joined #tripleo18:59
*** mburned has joined #tripleo19:04
*** eggmaste` has quit IRC19:07
*** yamahata has quit IRC19:07
*** pradk has joined #tripleo19:14
openstackgerritDan Prince proposed openstack/instack-undercloud: Enable swift's bulk middleware  https://review.openstack.org/26493119:14
*** cwolferh has joined #tripleo19:17
openstackgerritRyan Hallisey proposed openstack/tripleo-common: Deploy Docker compute role with tripleo.sh  https://review.openstack.org/26493319:17
openstackgerritRyan Hallisey proposed openstack/tripleo-common: Update the container flag  https://review.openstack.org/26493419:18
*** paramite is now known as paramite|afk19:19
rhalliseyshardy dprince, ^ those 2 should fix the gate in the stable/liberty branch19:19
dprincerhallisey: cool19:20
rhalliseylots of backports O.o19:21
*** penick has joined #tripleo19:21
slaglec19:22
slagleww19:22
shardyrhallisey: nice, lgtm, just waiting on CI19:23
rhalliseyonly the second one should pass19:24
rhalliseyfor the container job that is19:24
*** yamahata has joined #tripleo19:28
shardyEasy +A for someone, passing CI: https://review.openstack.org/#/c/263669/19:28
shardyand https://review.openstack.org/#/c/26014419:29
egaffordSeeing consistent "No valid host was found. There are not enough hosts available" in my virt env, consistently on my third overcloud test cycle (heat stack-delete, tripleo.sh --overcloud-deploy). Is there a known issue that would reasonably cause this, or should I spelunk? Not asking for research, just "Oh yeah, you have to juggle the quantum reverberators, everyone knows that." if such advice exists.19:33
EmilienMbnemec: I think https://review.openstack.org/#/c/244162 is critical to have it soon, lot of work depends on it I think19:34
*** marios_ has joined #tripleo19:34
openstackgerritSteven Hardy proposed openstack/tripleo-common: tripleo.sh support updating and checking deployed overcloud  https://review.openstack.org/26107719:34
shardyegafford: I'd check the nova hypervisor-stats after each delete, with particular attention on count and running_vms, along with related checks on the ironic node state via ironic node-list19:36
shardyegafford: beyond that, not sure as a few things can cause that error19:37
*** eggmaster has joined #tripleo19:37
egaffordshardy: Yeah, that's pretty much "something is wrong on the internet."19:37
egaffordIt's just the regularity with which it pops up that makes me think it might be a known resource leak.19:37
rhalliseyshardy, can the name field in a structured deployment be an environment variable?19:37
rhalliseylike https://paste.fedoraproject.org/308327/19546314/19:38
shardyegafford: Yeah, that's why you might glean some clues e.g if the stats are decrementing after each create/delete cycle19:38
*** pradk has quit IRC19:38
*** eil397 has quit IRC19:38
*** Goneri has quit IRC19:38
*** kbyrne has quit IRC19:38
*** marios has quit IRC19:38
egaffordYeah, not seeing any leaks in the hypervisor-stats.19:38
shardyrhallisey: you mean "name: {get_resource: DeploymentVersion}" ?19:39
rhalliseyshadower, yes19:39
rhalliseyis that legal?19:39
rhalliseyshardy, ^19:39
*** pradk has joined #tripleo19:40
rhalliseyit's a string19:40
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Enable the ML2 port security extension driver by default  https://review.openstack.org/26494019:40
shardyrhallisey: Yes, you can reference another resource called DeploymentVersion - the get_resource will set the name to the ID of that resource19:40
shardyI'm not sure exactly how that relates to environment variables tho19:40
* shardy going to grab dinner, bbiab19:41
*** shardy is now known as shardy_afk19:41
*** kbyrne has joined #tripleo19:42
*** Goneri has joined #tripleo19:43
bnemecEmilienM: It's merge conflicted and failing CI according to Jirka's last comment.  What are you looking for from me?19:47
EmilienMbnemec: you're the patch owner, I was wondering if we make progress on it, otherwise I'm happy to help19:48
*** lucasagomes is now known as lucas-dinner19:49
bnemecEmilienM: Oh, right.  I only proposed that because I proposed the original revert, and knew we would need to get this back in.  I was under the impression you and dprince were driving it now.19:53
EmilienMbnemec: ok, will continue that way then19:53
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: Manage keystone initialization directly in t-h-t manifests  https://review.openstack.org/24416219:55
EmilienMfor the monitoring, it would be great to land it (undercloud) https://review.openstack.org/#/c/253709/ and https://review.openstack.org/#/c/252051/ - patches already reviewed, CI passing, no blocker.19:55
EmilienMthanks ^19:55
slagleEmilienM: ci isn't passing on those. probably not related to your patches, but just sayin'20:02
EmilienMslagle: it pass non-ha20:02
EmilienMslagle: to me it's enough, since I'm patching undercloud.20:02
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: endpoint_map: bump nova api from 2.0 to 2.1  https://review.openstack.org/26336620:02
slaglelooks like they were both killed due to timeouts20:03
slaglemaybe we're not using a cache somewhere20:03
bnemecslagle: I'm wondering if heat-engine is getting OOM'd on some of those timeouts.  It looks like Heat just randomly stops at some point in the deployment process.20:07
bnemecAnd we do specifically recommend a bigger undercloud than I think we're using in CI.20:07
beaglesrhallisey, did I read right that you are working on something that fixes consistent CI/gate failures on stable/liberty?20:08
slaglebnemec: could be. either that or it's the need 2 vcpu's thing20:09
slagleone of the failures is "ERROR: Timed out waiting for a reply to message ID 3ddc049ccc1243559cb6ac13546998f3"20:09
rhalliseybeagles, sorry that was misleading. It's a fix for the container gate job20:09
slaglewhich the heat folks have said could be due to only 1 vcpu20:09
beaglesrhallisey, ah okay20:09
bnemecslagle: Yeah, that also.  We need to talk to derek about getting the undercloud vms sized up.20:09
*** dhellmann has quit IRC20:11
*** ayoung has quit IRC20:19
beaglesshardy: is there anyone looking into the stable/liberty CI failures?20:21
beaglesoh he's not here .. slagle, same question: ^^^20:22
*** openstackgerrit has quit IRC20:23
*** openstackgerrit has joined #tripleo20:24
*** gchamoul has quit IRC20:30
*** gchamoul has joined #tripleo20:32
slaglebeagles: yes, i submitted a revert to nova this morning20:32
slaglebeagles: https://review.openstack.org/#/c/264793/20:33
*** dprince has quit IRC20:33
*** Marga_ has joined #tripleo20:35
*** Marga_ has quit IRC20:35
*** Marga_ has joined #tripleo20:35
*** ayoung has joined #tripleo20:39
*** CaptTofu___ is now known as CaptTofu20:51
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates: Add Management Network For System Administration.  https://review.openstack.org/26496320:58
*** lblanchard1 has quit IRC21:00
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates: Add Management Network For System Administration.  https://review.openstack.org/26496321:01
*** jayg is now known as jayg|g0n321:02
*** rbrady has joined #tripleo21:04
openstackgerritEthan Gafford proposed openstack/tripleo-heat-templates: Sahara Integration  https://review.openstack.org/22086321:05
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates: Configure keystone public_endpoint  https://review.openstack.org/26497021:14
openstackgerritBen Nemec proposed openstack/python-tripleoclient: Ignore ssl warnings from requests  https://review.openstack.org/26497121:14
bnemecjaosorior: ^Needed to fix a couple of issues with ssl overcloud21:14
bnemecI'd be surprised if there aren't other issues, but since the overcloud is non-functional right now I can't test further. :-(21:15
*** cwolferh has quit IRC21:18
*** penick has quit IRC21:20
shardy_afkdsneddon: Hey I'm working up a quick example for generating a mapping file, do you need something that works on OSP7?21:30
*** shardy_afk is now known as shardy21:30
shardyor can I use new-for-liberty syntax?21:30
dsneddonshardy, Oh, boy, I didn't realize it had changed. I need to document both.21:30
shardydsneddon: It hasn't changed, I can just use a slightly cleaner syntax post-liberty21:30
shardyI'll create an example which works on both21:31
shardyhttps://review.openstack.org/#/c/215116/521:31
shardyit's basically similar to that21:31
shardyI'll tweak the example and post it in a few mins21:31
dsneddonshardy, Great, thanks for your help21:31
*** penick has joined #tripleo21:34
*** paramite|afk is now known as paramite21:43
*** cwolferh has joined #tripleo21:45
*** paramite is now known as paramite|afk21:48
*** eggmaster has quit IRC21:50
*** jaosorior has quit IRC21:50
*** slagle has quit IRC21:50
*** dtantsur|afk has quit IRC21:50
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Use the new yaml syntax in CI  https://review.openstack.org/25488621:51
openstackgerritBrad P. Crochet proposed openstack/tripleo-common: Build image files from definitions in yaml  https://review.openstack.org/23556921:51
*** eggmaster has joined #tripleo21:51
*** jaosorior has joined #tripleo21:51
*** slagle has joined #tripleo21:51
*** dtantsur|afk has joined #tripleo21:51
*** ayoung has quit IRC21:58
*** rpothier has quit IRC22:03
*** trown is now known as trown|outttypeww22:04
*** rhallisey has quit IRC22:06
*** Marga_ has quit IRC22:10
*** penick has quit IRC22:15
*** pradk has quit IRC22:24
*** absubram has quit IRC22:26
*** penick has joined #tripleo22:32
*** sthillma_ has joined #tripleo22:44
*** sthillma has quit IRC22:44
*** sthillma_ is now known as sthillma22:44
*** thrash is now known as thrash|g0ne22:45
*** jtomasek has quit IRC22:45
*** mkovacik has joined #tripleo23:05
*** mkovacik has quit IRC23:12
openstackgerritBen Nemec proposed openstack/tripleo-docs: Document deploying the overcloud with ssl  https://review.openstack.org/26500623:14
*** mkovacik has joined #tripleo23:30
openstackgerritEthan Gafford proposed openstack/tripleo-heat-templates: Sahara Integration  https://review.openstack.org/22086323:35
*** chem has quit IRC23:37
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Add example of creating per-node os-net-config mappings  https://review.openstack.org/26501423:38
shardydsneddon: Hey, I'm going to have to call it a night, but there's a WIP patch23:38
dsneddonshardy, Much appreciated. G'night.23:39
shardyit's partially tested, but I'm having some issues getting the NodeConfigDataLookup parameter data into a format that survives the str_replace and ends up parsable by json.loads23:39
shardydsneddon: so if you feel like fixing that and pushing another revision, feel free, otherwise I'll take another look tomorrow23:40
shardyalso, I think there's a big gotcha with using a firstboot script23:40
shardyin that if you ever change NodeConfigDataLookup (for example to add new nodes), it will replace all the existing nodes, because the user-data changes23:40
dsneddonshardy, OK, I'm doing deployment after deployment testing IPv6, so I'll throw this in and see what pops out the other side.23:40
shardyI'm still thinking about the best way around that23:40
shardydsneddon: I think you'll find it generates an empty mapping file, because the json is malformed when it hits the shell script23:41
dsneddonshardy, So under Liberty you would have used map_merge?23:41
shardycheck out /var/log/cloud-init-output.log and you'll see the script which fails (you can then re-run it)23:42
shardydsneddon: No, I can use a json parameter type, which is then transparently encoded as json via the str_replace23:42
shardywhich is way, way easier than trying to hand-roll the json with all the escapes etc as a string23:43
dsneddonshardy, Is json parameter type really that new? I've gotten so used to it, I take it for granted.23:45
shardyhttp://docs.openstack.org/developer/heat/template_guide/hot_spec.html#str-replace23:45
shardydsneddon: No, it's not the json parameter type, it's that str_replace didn't accept json parameters until the latest verion of HOT (for liberty)23:45
shardybefore liberty, we'd throw a validation error saying the params for str_replace had to be a string23:45
dsneddonshardy, Right. Thanks for this, I'll play with it. I still think it's worth modifying os-net-config so we can give it one giant array of mac addresses and nic names.23:46
shardysince HOT version 2015-10-15 (liberty) we accept list/json parameters and json encode them to create strings23:46
shardydsneddon: Yeah I agree it'd be worth modifying os-net-config, but we still have the problem of how to update the mapping before the network is up, which implies user-data23:47
shardywhich means we have to figure out how to not rebuild the world if the mapping ever changes ;)23:47
shardyI'll give that further thought after some sleep, I have a few ideas23:47
shardythe challenges is finding one which may be backportable23:47
dsneddonshardy, To be continued...23:48
shardydsneddon: yup, good luck, night!23:48
*** ayoung has joined #tripleo23:48
*** shardy has quit IRC23:48
*** weshay_xchat has quit IRC23:49
openstackgerritMerged openstack/tripleo-heat-templates: Adding ManagementIpSubnet to linux bridge net conf  https://review.openstack.org/26366923:58
openstackgerritMerged openstack/tripleo-heat-templates: updating enable_ceph conditions for controller  https://review.openstack.org/26339723:59
openstackgerritMerged openstack-infra/tripleo-ci: Include the branch name in row descriptions  https://review.openstack.org/25950523:59
openstackgerritMerged openstack-infra/tripleo-ci: Tweak the report colors a little  https://review.openstack.org/25950623:59
openstackgerritMerged openstack-infra/tripleo-ci: Remove parse logs  https://review.openstack.org/25950723:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!