Tuesday, 2016-03-29

*** lucasagomes has quit IRC00:10
*** lucasagomes has joined #tripleo00:11
*** shivrao has quit IRC00:38
*** penick has joined #tripleo00:43
*** tiswanso has joined #tripleo00:45
*** tiswanso has quit IRC00:46
*** tiswanso has joined #tripleo00:47
*** akshai has joined #tripleo00:50
*** penick_ has joined #tripleo00:52
*** penick has quit IRC00:52
*** penick_ is now known as penick00:52
*** akshai_ has joined #tripleo00:58
*** NobodyCam has joined #tripleo00:58
*** akshai has quit IRC01:01
*** rhallisey has quit IRC01:05
*** Marga_ has quit IRC01:16
*** penick has quit IRC01:23
*** penick has joined #tripleo01:23
*** thrash is now known as thrash|g0ne01:32
*** penick has quit IRC01:38
*** saneax is now known as saneax_AFK01:41
*** Marga_ has joined #tripleo01:56
*** penick has joined #tripleo01:56
*** Marga_ has quit IRC02:01
*** yamahata has quit IRC02:05
*** penick has quit IRC02:06
*** Marga_ has joined #tripleo02:17
*** penick has joined #tripleo02:18
*** Marga_ has quit IRC02:21
*** saneax_AFK is now known as saneax02:23
openstackgerritJane.zhang proposed openstack/diskimage-builder: Set tgtd not auto-start on OS boot time.  https://review.openstack.org/29754802:31
*** penick has quit IRC02:32
*** ebalduf has quit IRC02:42
*** saneax is now known as saneax_AFK02:43
*** lifeless has quit IRC02:43
*** ccamacho has quit IRC02:44
*** lifeless has joined #tripleo02:44
*** ccamacho has joined #tripleo02:44
*** ebalduf has joined #tripleo02:44
*** yamahata has joined #tripleo02:55
*** mannidi has joined #tripleo03:01
*** rlandy has quit IRC03:02
*** links has joined #tripleo03:13
*** Marga_ has joined #tripleo03:19
*** Marga_ has quit IRC03:24
*** Marga_ has joined #tripleo03:39
*** tiswanso has quit IRC03:40
*** Marga__ has joined #tripleo03:44
*** ayoung has quit IRC03:46
*** Marga_ has quit IRC03:47
*** ramishra has quit IRC03:49
*** ramishra has joined #tripleo03:51
*** masco has joined #tripleo03:52
*** NobodyCam has quit IRC03:54
*** mannidi has quit IRC03:59
openstackgerritMichael Chapman proposed openstack/puppet-tripleo: Add neutron profiles  https://review.openstack.org/29343604:07
*** NobodyCam has joined #tripleo04:18
*** akshai_ has quit IRC04:29
*** mannidi has joined #tripleo04:29
*** shivrao has joined #tripleo04:32
*** shivrao_ has joined #tripleo04:35
*** shivrao has quit IRC04:37
*** shivrao_ is now known as shivrao04:37
*** saneax_AFK is now known as saneax05:01
*** tserong has quit IRC05:08
*** pcaruana has quit IRC05:09
openstackgerritMichael Chapman proposed openstack/puppet-tripleo: Add neutron profiles  https://review.openstack.org/29343605:12
*** tserong has joined #tripleo05:17
*** qasims has joined #tripleo05:25
*** oshvartz has joined #tripleo05:25
*** mannidi has quit IRC05:35
*** jaosorior has joined #tripleo06:08
*** panda has quit IRC06:12
*** panda has joined #tripleo06:13
*** Marga__ has quit IRC06:16
*** ohamada has joined #tripleo06:17
*** tzumainn has quit IRC06:17
*** ohamada has quit IRC06:18
*** tesseract has joined #tripleo06:19
*** tesseract is now known as Guest5878206:19
*** ohamada has joined #tripleo06:20
*** florianf has joined #tripleo06:29
*** akuznetsov has joined #tripleo06:33
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Make compute post-deploy depend on controller's  https://review.openstack.org/29855506:42
*** lmiccini has quit IRC06:46
*** jtomasek has joined #tripleo06:50
*** lmiccini has joined #tripleo06:50
*** jprovazn has joined #tripleo06:56
qasimsFolks, I am hitting this issue on Step4: [2016-03-29 02:57:56,375] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/457942c6-85d6-43ca-b21e-ad237ffd623c.pp. [6]07:02
qasims#033[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock'07:03
*** olap has quit IRC07:03
qasimsI have configured bonds and using vlan os management network on top of that for overcloud nodes .07:04
*** akuznetsov has quit IRC07:08
*** shardy has joined #tripleo07:14
qasims localhost os-collect-config: #033[1;31mError: Execution of '/usr/bin/mysql -NBe drop database if exists `test`' returned 1: ERROR 1213 (40001) at line 1: Deadlock found when trying to get lock; try restarting transaction#033[0m07:17
*** shivrao has quit IRC07:17
*** ebalduf has quit IRC07:21
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates: Configurable enabled plugins list for Sahara  https://review.openstack.org/29843007:23
*** jpena has joined #tripleo07:23
*** sshnaidm has quit IRC07:25
*** aufi has joined #tripleo07:26
*** akuznetsov has joined #tripleo07:32
*** rcernin has joined #tripleo07:36
*** yuanying has quit IRC07:38
*** yuanying has joined #tripleo07:41
*** hewbrocca has joined #tripleo07:45
*** ifarkas has joined #tripleo07:45
*** pcaruana has joined #tripleo07:48
*** rdopiera has joined #tripleo07:50
*** gfidente has joined #tripleo07:50
*** nico_auv has joined #tripleo08:02
*** liverpooler has joined #tripleo08:09
*** derekh has joined #tripleo08:11
*** olap has joined #tripleo08:14
*** jistr has joined #tripleo08:23
openstackgerritMerged openstack/tripleo-ui: Add LICENSE  https://review.openstack.org/29595708:26
*** nijaba has quit IRC08:26
*** nijaba has joined #tripleo08:27
*** yamahata has quit IRC08:27
*** mcornea has joined #tripleo08:27
*** yamahata has joined #tripleo08:28
*** mkovacik has joined #tripleo08:38
*** ohamada_ has joined #tripleo08:44
*** sshnaidm has joined #tripleo08:44
*** ohamada has quit IRC08:48
*** mgould has joined #tripleo09:06
*** tosky has joined #tripleo09:06
*** ohamada__ has joined #tripleo09:07
*** mgagne has quit IRC09:08
*** ohamada has joined #tripleo09:11
*** ohamada_ has quit IRC09:11
*** greghaynes has quit IRC09:12
*** ohamada__ has quit IRC09:13
*** akrivoka has joined #tripleo09:13
*** mgagne has joined #tripleo09:14
*** mgagne is now known as Guest6891009:14
*** greghaynes has joined #tripleo09:14
*** qasims has quit IRC09:17
*** dtantsur|afk is now known as dtantsur09:29
*** ohamada_ has joined #tripleo09:33
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: ssh less  https://review.openstack.org/29597309:35
openstackgerritDerek Higgins proposed openstack-infra/tripleo-ci: Don't execute the output from wait_for  https://review.openstack.org/29529709:37
*** ohamada has quit IRC09:37
openstackgerritMerged openstack-infra/tripleo-ci: Add support for mitaka jobs  https://review.openstack.org/29837009:38
*** ohamada__ has joined #tripleo09:38
*** lifeless has quit IRC09:39
*** lifeless has joined #tripleo09:40
*** ohamada_ has quit IRC09:42
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with non-HA jobs  https://review.openstack.org/29703809:43
*** ohamada_ has joined #tripleo09:52
*** ohamada__ has quit IRC09:55
*** ohamada__ has joined #tripleo10:00
*** ohamada_ has quit IRC10:03
*** ohamada_ has joined #tripleo10:11
*** pblaho has joined #tripleo10:14
*** ohamada__ has quit IRC10:14
*** ccamacho1 has joined #tripleo10:15
*** ccamacho1 has left #tripleo10:15
*** ccamacho1 has joined #tripleo10:16
adarazsderekh: hi! the ipv6 gates all passed! \o/ :)10:19
shadowerwooo10:21
adarazsvery unexpected indeed :D10:22
adarazsderekh: I see the last opinion about it was that we should add ipv6 to the upgrade job, but I have some doubts... it doesn't use pacemaker, and I think we're more interested in that code path for IPv6. probably we should stick to the HA job.10:23
derekhadarazs: nice one, looking10:26
hewbroccahmmph, why doesn't the upgrade job use pacemaker<10:26
hewbroccaThat's where all the hard stuff is...10:26
shadowerI guess that's why :-)10:27
adarazshewbrocca: I agree. though now we had to fix a lot of stuff on the non-pacemaker code path because nobody tested that with IPv6. but I don't think many people care either about it.10:27
adarazspacemaker was actually easier.10:27
derekhgfidente: do I remember you saying you were you doing a test matrix at some stage last week?10:28
gfidentederekh https://review.openstack.org/#/c/294071/10:30
*** qasims has joined #tripleo10:32
derekhgfidente: thanks10:35
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 network isolation on the ha job  https://review.openstack.org/28944510:36
derekhadarazs: see ^ , which suggests we woulf have IPv6 on the upgrades job with pacemaker , we should see if we have agreement on this patch today10:36
adarazsderekh, gfidente: so should I actually add pacemaker to the upgrade job, or should I stick to the HA job?10:37
adarazsI see in that change that upgrades is supposed to be pacemaker10:37
gfidenteyeah I think we wanted at least one job without netiso10:37
gfidenteand considering we have only three jobs10:37
gfidenteI thought of ha netiso/ipv4 and upgrades netiso/ipv610:38
gfidenteupgrades does use pacemaker yes10:38
derekhadarazs: thing is its a group decision, we should make sure most of the projects contributor agree on what we are aiming for, i.e. get gfidente's patch merged,10:40
derekhbut having pacemaker with upgrades lgtm10:40
adarazsgfidente: currently upgrades doesn't use pacemaker.10:40
adarazsderekh: okay, so we can discuss it today on the tripleo meeting maybe, and I make the change to add both pacemaker and netiso v6 to upgrades10:41
derekhadarazs: yup10:41
gfidenteadarazs yeah it's a working in progress, so it currently doesn't even perform an actual upgrade10:43
gfidentebut from the matrix the goal would be to make that pcmk/ipv610:43
adarazsgfidente: yup, I understand that :)10:43
gfidenteI'm not sure if we could use the containers job to cover some of these bits too10:43
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 and pacemaker on the upgrades job  https://review.openstack.org/28944510:46
openstackgerritSteven Hardy proposed openstack/tripleo-docs: Add MidoNet documentation in advanced deployment  https://review.openstack.org/27032010:55
shardymarios: Hey I published a minor edit on ^^ as I think the chmod 755 was wrong, if you're happy now we can just land it10:56
mariosshardy: o/ looking10:58
mariosshardy: +2 i guess wait for the docs build and+A10:59
shardyyup, sounds good, thanks!11:00
mariosshardy: tx11:00
*** lucasagomes is now known as lucas-hungry11:00
*** hewbrocca has quit IRC11:14
*** tserong has quit IRC11:16
openstackgerritMerged openstack/tripleo-puppet-elements: Add rubygems-deep_merge to package list  https://review.openstack.org/29541011:26
*** Marga_ has joined #tripleo11:26
*** hewbrocca has joined #tripleo11:32
*** dprince has joined #tripleo11:32
*** rhallisey has joined #tripleo11:36
openstackgerritMerged openstack/tripleo-heat-templates: Set UpdateIdentifier for upgrade converge, to prevent services down  https://review.openstack.org/29717511:41
*** rodrigods has quit IRC11:42
*** rodrigods has joined #tripleo11:42
openstackgerritRyan Brady proposed openstack/tripleo-common: Adds parameters actions  https://review.openstack.org/29868211:43
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates: Set UpdateIdentifier for upgrade converge, to prevent services down  https://review.openstack.org/29868511:47
*** snecklifter has joined #tripleo11:50
*** snecklifter has quit IRC11:52
*** snecklifter has joined #tripleo11:52
*** snecklifter has quit IRC11:53
*** weshay has joined #tripleo11:53
*** snecklifter has joined #tripleo11:53
*** Marga_ has quit IRC11:54
snecklifterhi, i'm trying to stop config drift across a deployed tripleo cluster, any recommendations?11:58
*** trown|PTO is now known as trown12:01
*** tremble has joined #tripleo12:07
*** tremble has joined #tripleo12:07
*** masco has quit IRC12:08
*** tserong has joined #tripleo12:14
*** jayg|g0n3 is now known as jayg12:15
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29869512:16
*** ccamacho has quit IRC12:18
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29869512:18
*** ccamacho1 has quit IRC12:18
*** ccamacho has joined #tripleo12:18
*** aufi has quit IRC12:20
*** pradk has joined #tripleo12:24
*** ohamada__ has joined #tripleo12:26
*** michchap has quit IRC12:26
shardysnecklifter: can you define config drift?  How is that occuring?12:27
shardysnecklifter: you should apply all config changes via Heat->Puppet, ergo no config drift12:27
sneckliftershardy: testing changes on an individual node, how can you be sure all nodes are running under same config12:28
shardysnecklifter: we re-assert the config state with puppet, on all nodes, every time you update the heat stack12:28
shardysnecklifter: this is done by changing the DeployIdentifier parameter every update, to a timestamp derived number12:29
snecklifterok, so is there an option to run that automatically every 24 hours or some such?12:29
shardythis forces puppet to be re-applied every update, which means updates are a bit slower, but you can be confident of the resulting state12:29
*** ohamada_ has quit IRC12:29
shardysnecklifter: you could run a cron job I suppose which just runs openstack overcloud deploy again (e.g in a script with all the same arguments)12:30
*** qasims has quit IRC12:30
sneckliftershardy: ok, thanks, we are just seeing issues where engineers are testing changes on individual nodes over a large cluster and would like to bring things back in line every so often12:31
shardysnecklifter: Ok, I'd say re-running overcloud deploy should do it, but obviously the main issue is the process problem of folks hacking on the nodes directly12:31
snecklifterso essentially there is no harm in running openstack overcloud deploy <parameters> every so often on a running system?12:31
*** jprovazn has quit IRC12:32
sneckliftershardy: agreed, its more of an issue in earlier dev/testing12:32
shardysnecklifter: in theory it should be fine12:32
snecklifterok, gotcha, thanks very much for responding12:32
sneckliftershardy++12:33
snecklifteri forgot theres no karmabot here :)12:33
shardyhehe :)12:33
*** ohamada_ has joined #tripleo12:37
*** jaosorior has quit IRC12:38
*** jaosorior has joined #tripleo12:39
*** jpena is now known as jpena|lunch12:39
*** jdob has joined #tripleo12:40
*** ohamada__ has quit IRC12:40
*** jaosorior has quit IRC12:42
*** jaosorior has joined #tripleo12:43
*** rhallisey has quit IRC12:44
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29869512:44
*** lucas-hungry is now known as lucasagomes12:44
*** rhallisey has joined #tripleo12:44
*** rlandy has joined #tripleo12:46
*** trozet has quit IRC12:49
*** MaxPC has joined #tripleo12:49
*** morazi has joined #tripleo12:51
*** bvandenh has joined #tripleo12:51
*** michchap has joined #tripleo12:53
*** thrash|g0ne is now known as thrash12:54
*** julim has joined #tripleo12:54
*** ccamacho_ has joined #tripleo12:55
openstackgerritRyan Brady proposed openstack/tripleo-common: Adds Deployment Action & Workflow  https://review.openstack.org/29873212:56
openstackgerritRyan Brady proposed openstack/tripleo-common: Adds parameters actions  https://review.openstack.org/29868212:59
gfidentejistr ack on glance, adding to ceilometer13:00
jistrgfidente: awesome, thanks13:00
gfidenteI was so happy yo see those that immediately went to check all services haha :P13:01
jistr(if it doesn't land soon enough, we can match on 'ceilometer' tag instead, as puppet does auto tagging with class names, incl. the topmost puppet module name)13:01
*** dprince has quit IRC13:01
gfidenteoh ok that I didn't know13:01
*** links has quit IRC13:02
gfidenteand also, for ceilometer it's only agent/notification and collection which misses it13:03
gfidente*collector13:03
openstackgerritDimitri Savineau proposed openstack/puppet-tripleo: Enable HAProxy forwardfor option for Horizon.  https://review.openstack.org/29649713:03
*** dprince has joined #tripleo13:03
gfidentebut we use them from the manifest so ...13:03
derekhdprince: meeting time ?13:03
dprinceisn't it in an hour?13:04
* derekh wonders if he has been the latest victim of DST13:04
michchapI think it's in an hour13:04
shardyYeah it's in an hour isn't it?13:04
trownyep in an hour13:04
* shardy had to double check when his reminder just popped up ;)13:04
ccamachoDST :(13:05
derekhok, carry on13:05
shardytrown: Hey, two quickstart questions for you when you get a sec13:07
shardyfirstly - do you have plans to refactor the docs to quicstart-ify them?13:08
shardyrelated second question - do you have a process to not run the VM pieces, e.g so you can use quickstart to automate initial undercloud configuration on a baremetal box?13:08
shardyI'm assuming it's just a subset of the tags, but wondered if it's been tried yet13:09
trownshardy: ya the first one is a work item on https://github.com/openstack/tripleo-specs/blob/master/specs/mitaka/tripleo-quickstart.rst13:09
*** ayoung has joined #tripleo13:09
trownshardy: I was waiting to have everything done on the first two work items before switching docs, but I could start a WIP for it13:10
shardytrown: yeah a WIP would be useful if you have time, I assume it's mostly the repo paths that will change?13:10
shardytrown: context, I'm wondering about the best format for a TripleO "install mitaka" post/blog/whatever13:11
shardysince we now have branches and folks may want to try them13:11
trownshardy: ya, I guess we would have to have instructions to build your own image for tripleo work, since there is not one being produced yet13:11
*** jprovazn has joined #tripleo13:12
*** tiswanso has joined #tripleo13:12
trownhmm... for install mitaka, I think the RDO delorean image would be the same as we would produce using tripleo instructions13:12
trownsince there is no extra current-tripleo repo in that case13:13
shardyYeah I think that's what derekh is proposing to reuse in https://review.openstack.org/#/c/285259/ right?13:13
*** tiswanso has quit IRC13:13
shardyso if we land that, it'd be reasonable to start basing docs on consuming that artefact I guess13:14
*** tiswanso has joined #tripleo13:14
trownthat is a start of the process, but it the artifact produced would not be usable via tripleo-quickstart yet13:14
trownor at least wouldnt be very quick, since it is basically just the cloud image13:15
derekhshardy: trown ya, its basically just a cloud image, next step is to beef it up a bit to at least have some relevant packages pre installed on it13:16
trownthat patch is more about proving out the mirror server approach, we will need more patches to build up the artifacts we produce13:16
derekhshardy: at the moment files on the mirror server arn't publicly available, I've retricted apache to only listen on the private/test network, see line 25 http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n2513:17
openstackgerritMerged openstack/python-tripleoclient: Turn the orchestration client reference into a property  https://review.openstack.org/29844813:18
shardykk, thanks for the update, sounds like we're still a little early in the process wrt this for Mitaka docs13:18
slagledang, looks like this bug is in mitaka, https://bugs.launchpad.net/heat/+bug/156117213:19
openstackLaunchpad bug 1561172 in tripleo "Overcloud delete never completes" [Critical,Triaged]13:19
derekhshardy: I did this as there is no auth on the upload script, so I wanted to make it less likely a 3rd party uploads fake artifacts13:19
slagleshardy: does that look familiar at all? https://bugs.launchpad.net/heat/+bug/156117213:20
shardyslagle: Hmm, not immediately, let me trawl heat bugs, sec13:21
shardywe should look at re-enabling the overcloud delete CI test13:21
shardyIt'd be good to know if this happened after a failed update13:23
shardybnemec: ^^13:23
trownshardy: I think we could still switch docs to include building your own undercloud.qcow2 (after the project is moved upstream)13:23
slagleshardy: it didnt13:23
slaglei just deployed, and then tried to delete13:23
trownshardy: still an improvement, as you can build it once and reuse it13:23
*** tzumainn has joined #tripleo13:26
*** mcornea has quit IRC13:26
*** mcornea has joined #tripleo13:27
*** mgould has quit IRC13:28
*** lblanchard has joined #tripleo13:29
*** mgould has joined #tripleo13:30
*** egafford has joined #tripleo13:30
*** akuznetsov has quit IRC13:31
shardyslagle: do we have any more information about the state of the stack prior to the delete, or the status of TenantPort?13:33
*** absubram has joined #tripleo13:34
EmilienMchem: can I merge https://review.openstack.org/#/c/294182/ ?13:34
*** dustins has joined #tripleo13:34
*** zaneb has joined #tripleo13:34
openstackgerritJason E. Rist proposed openstack/tripleo-ui: TripleO Logo for Login and Navbar  https://review.openstack.org/29592113:35
*** absubram_ has joined #tripleo13:35
chemEmilienM: Yep, you can, but I don't see the comment you talked me about13:36
EmilienMchem: which comment?13:36
chemEmilienM: you told me there was on comment (on #rdo-puppet)13:36
EmilienMchem: from andreas.13:37
*** absubram has quit IRC13:40
*** absubram_ is now known as absubram13:40
jaygdid that test break tripleo?  /me missed context on that13:42
chemEmilienM: I replied13:42
chemEmilienM: https://review.openstack.org/#/c/294182/ it's ok good to merge.  If jayg wants to have a look, it would be nice though13:44
EmilienMtoo late13:44
chemEmilienM:yes I've just seen :)13:44
*** pradk has quit IRC13:45
michchapEmilienM: In spredzy's profiles, he split glance api and registry into different profiles. What are your thoughts on this? They are different services, but at the same time there's not much benefit from splitting them. If you're thinking about doing container things then maybe they need to be split.13:45
*** pradk has joined #tripleo13:45
*** yamahata has quit IRC13:45
michchapEmilienM: or we can just worry about it later :)13:45
*** yamahata has joined #tripleo13:45
jaygchem: 'Revert revert'?  can't wait to see the next one in this chain!  ;)13:45
jaygmissed a revert when I read it the first time...13:46
dprincemichchap: I did the same thing13:46
chemjayg: revert revert*213:46
dprincemichchap: I had glance-registry and glance-api as separate profiles initially13:46
EmilienMdprince, michchap: yes we can split it now.13:46
*** jpena|lunch is now known as jpena13:47
michchapEmilienM: dprince ok cool. I think I will have the neutron profiles happy by end of week.13:47
slagleshardy: what type of info do you need?13:48
ramishrashardy, slagle: seems somthing like https://bugs.launchpad.net/heat/+bug/1552760, probably not the same, but it would be good to check the neutronclient version.13:48
openstackLaunchpad bug 1552760 in python-neutronclient "DELETE_FAILED for neutron resources with TypeError: Message objects do not support addition." [Critical,Fix released] - Assigned to Miguel Angel Ajo (mangelajo)13:48
slagleshardy: it's just a "normal" overcloud with network isolation, single nic with vlans13:48
openstackgerritMerged openstack/puppet-pacemaker: Revert revert "Basic beaker one node test."  https://review.openstack.org/29418213:48
dprincemichchap: nice13:48
dprinceslagle: I commented on that ticket. I saw a stack delete issue this week related to an OOM memory error in neutron openvswitch agent on the undercloud13:49
*** florianf has quit IRC13:49
dprinceslagle: If I recall there was a cryptic heat stack trace... but in my case the root cause of the issue was neutron openvswitch needed a restart first13:50
*** florianf has joined #tripleo13:50
slagledprince: ok. that's not the issue in my case13:51
dprinceslagle: cool, just an idea13:51
shardyslagle: I was looking to understand the state of the underlying nested stack and neutron port13:52
shardye.g where we're trying to get the "ip_subnet" output from13:52
slagleit's  tht from mitaka, so basically master13:53
slaglei think ben filed the bug when he was using master as well13:53
slagleramishra: i do have that patch13:54
slaglein python-neutronclient13:54
*** akshai has joined #tripleo13:57
*** akshai_ has joined #tripleo13:59
EmilienMmeeting time folks!13:59
*** marios has quit IRC13:59
*** marios has joined #tripleo14:01
*** akshai has quit IRC14:02
*** mcornea has quit IRC14:04
*** absubram has quit IRC14:04
*** ebalduf has joined #tripleo14:07
*** mcornea has joined #tripleo14:13
*** aufi has joined #tripleo14:13
*** Goneri has joined #tripleo14:14
*** mgould has quit IRC14:18
EmilienMif we don't land https://review.openstack.org/#/c/293999/ - tripleo CI will break very soon, I already sent a warning a few days ago, but puppet-oslo is going to be used by default14:20
*** trozet has joined #tripleo14:20
hewbroccatick tock14:21
hewbroccamerge early, merge often!14:21
derekhEmilienM: , also nobody will merge it until is got passing HA job14:22
adarazsderekh: btw I just noticed that the "upgrades" job runs completely without any interesting OVERCLOUD_DEPLOY_ARGS... I was told that the ceph) case applies to it from toci_gate_test.sh, but it actually doesn't.14:22
EmilienMderekh: well, my patch is not related to the failure in HA job14:22
adarazsI thought it tests ceph at least, and I was trying to add IPv6 and pacemaker to ceph) but it looks like it's not used. https://review.openstack.org/#/c/289445/29/toci_gate_test.sh14:23
EmilienMso yeah, we can wait until HA job pass, but puppet-oslo is being used by puppet-nova likely today or tomorrow14:23
derekhEmilienM: doesn't matter as a general rule you need 3xpasses14:23
*** panda has quit IRC14:25
*** panda has joined #tripleo14:25
*** mgould has joined #tripleo14:25
derekhEmilienM: I'm not against merging it without the HA job, but usually a discussion about weither it is ok or not would happen here first, anyways back to the meeting will check back in a bit14:28
EmilienMcool14:28
jaosoriorbnemec: Hey dude, I'm looking into the issue mcornea reported about not being able to update a self-signed cert14:48
jaosoriorbnemec: It seems that the issue is that haproxy needs to be restarted if the certificate is updated14:49
*** ohamada__ has joined #tripleo14:50
bnemecjaosorior: Yeah, it's a known limitation of the tripleo loadbalancer module. :-(14:52
*** ccamacho has quit IRC14:52
bnemecI started looking into it yesterday, but didn't get very far due to unrelated issues setting up my environment.14:52
*** tosky has quit IRC14:53
*** ohamada_ has quit IRC14:54
*** akuznetsov has joined #tripleo14:55
jaosoriorbnemec: Bummer. What is currently managing haproxy? Is it pacemaker or systemd?14:55
*** tosky has joined #tripleo14:57
bnemecjaosorior: I'm not sure, but I think puppet needs to restart it because there are subsequent calls in puppet that depend on the new cert being applied.14:58
jaosoriorbnemec: I guess we could pass the md5, and in puppet see if it changes, and if it does, then trigger that restart14:58
jaosoriorSo we don't restart haproxy on every step of the post-config14:58
bnemecjaosorior: I discussed it with EmilienM on https://bugzilla.redhat.com/show_bug.cgi?id=1301185 and it sounds like there's a way to fix it, I just didn't realize it was going to be such a big problem.14:59
openstackbugzilla.redhat.com bug 1301185 in openstack-puppet-modules "Puppet doesn't restart haproxy when certificate contents change" [Unspecified,Assigned] - Assigned to emacchi14:59
bnemecOn the undercloud you just need to manually restart haproxy and re-run the undercloud installation.14:59
jaosoriorbnemec: That's what I did for the overcloud to get it working; But probably that shouldn't be the solution14:59
bnemecThe same workaround would probably work here, although it's pretty ugly.14:59
bnemecYeah14:59
jaosoriorbnemec: So... what you suggest is to change the path for the certificate15:00
jaosoriorwell, yeah, that can be done for the overcloud easily, there is an option for that15:00
bnemecjaosorior: I think the right thing to do is fix the loadbalancer module so it manages the file, not just the path.15:01
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Ensure cinder-volume will survive reboot on PoC deployments  https://review.openstack.org/29880415:01
jaosoriorbnemec: I agree15:01
bnemecThe current behavior is just wrong and very unintuitive.15:01
bnemecI haven't looked into how  hard it will be to do that though.15:01
jaosoriorbut now I'm thinking, what would be the implications for that though15:01
jaosorioryeah15:01
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Ensure cinder-volume will survive reboot on PoC deployments  https://review.openstack.org/29880415:02
bnemecHopefully it's as simple as wrapping the path with a puppet file resource, but I don't know.15:02
*** qasims has joined #tripleo15:02
*** ohamada_ has joined #tripleo15:03
jaosoriorbnemec: Now that I think about it... the file will be replaced... so we can't really compare that md5 with the new one15:03
*** pblaho has quit IRC15:05
*** egafford has quit IRC15:05
shardyFWIW there is a "digest" heat function which could provide a hash of the cert as an input to puppet15:05
shardydoing it all in puppet sounds probably cleaner tho15:05
*** openstackgerrit has quit IRC15:06
*** openstackgerrit has joined #tripleo15:06
jaosoriorshardy: Yeah, gotta figure out first if we can cleanly do it in puppet first. Gonna try the approach bnemec suggested initially15:07
*** ohamada__ has quit IRC15:07
qasimsHA job I have been failing on this, https://review.openstack.org/#/c/257916/15:08
bnemecYeah, I'm a little concerned about starting another race between puppet and pacemaker.15:08
qasimsError: http://logs.openstack.org/16/257916/8/check-tripleo/gate-tripleo-ci-f22-ha/15e2216/console.html#_2016-03-29_10_22_53_88715:08
hewbroccabnemec: we already have so many -- what difference could one more possibly make15:08
*** aufi has quit IRC15:10
qasimsAny reviews on this failure?15:10
derekhAnybody else around that can look at this (using the mirror server for the instack image), once merged I'll start expanding on how much we use it https://review.openstack.org/#/c/285259/15:11
*** mkovacik has quit IRC15:19
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Ensure cinder-volume will survive reboot on PoC deployments  https://review.openstack.org/29880415:19
bnemecqasims: That job got killed by the gate timeout before it collected logs.  There's basically no way to debug it.15:19
*** egafford has joined #tripleo15:21
derekhshardy: ty15:24
qasimsThanks /bnemec. Should I go with a recheck?15:26
bnemecqasims: Yes15:26
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Deploy ceph in the nonha job  https://review.openstack.org/29305115:26
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the upgrades job  https://review.openstack.org/28944515:26
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use --overcloud-update in the upgrades job  https://review.openstack.org/26046615:26
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Set the defaultbranch to stable/mitaka  https://review.openstack.org/29882015:26
qasimsThanks.15:26
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Remove password default for AodhPassword  https://review.openstack.org/29882115:27
*** rcernin has quit IRC15:27
*** qasims has quit IRC15:28
*** mcornea has quit IRC15:28
openstackgerritMerged openstack/tripleo-heat-templates: Set UpdateIdentifier for upgrade converge, to prevent services down  https://review.openstack.org/29868515:29
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Deploy ceph in the nonha job  https://review.openstack.org/29305115:29
*** links has joined #tripleo15:32
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add quotes around the cinder_iscsi_ip_address value  https://review.openstack.org/29882715:33
*** liverpooler has quit IRC15:34
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Add quotes around the cinder_iscsi_ip_address value  https://review.openstack.org/29882815:34
*** saneax is now known as saneax_AFK15:36
*** mannidi has joined #tripleo15:36
openstackgerritMerged openstack/tripleo-heat-templates: Add quotes around the cinder_iscsi_ip_address value  https://review.openstack.org/29722215:36
openstackgerritMerged openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29869515:38
*** egafford has quit IRC15:42
dtantsurfolks, could you please take a look at a couple of doc patches? https://review.openstack.org/290492 and https://review.openstack.org/29060515:43
dtantsurboth are valid for liberty and mitaka, so it would be nice to get them in asap15:43
*** derekh has quit IRC15:44
*** yamahata has quit IRC15:44
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use --overcloud-update in the upgrades job  https://review.openstack.org/26046615:45
openstackgerritMike Burns proposed openstack/tripleo-heat-templates: change the default satellite tools rpm repo.  https://review.openstack.org/24589515:46
*** dprince has quit IRC15:47
*** dprince has joined #tripleo15:47
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Ensure cinder-volume will survive reboot on PoC deployments  https://review.openstack.org/29880415:48
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29883415:49
*** hewbrocca has quit IRC15:50
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Tempest for tripleo CI  https://review.openstack.org/29584415:51
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with non-HA jobs  https://review.openstack.org/29703815:51
*** olap has quit IRC15:52
*** ccamacho_ has quit IRC15:53
*** dtantsur is now known as dtantsur|afk15:53
dprinceEmilienM: https://review.openstack.org/#/c/236243/ wanna send this :)15:54
EmilienMdprince: let me review it again15:55
EmilienMmarios has a comment15:55
*** mannidi_ has joined #tripleo15:55
*** egafford has joined #tripleo15:56
*** klindgren has left #tripleo15:56
EmilienMdprince: did you look at it?15:57
*** mannidi has quit IRC15:59
dprinceEmilienM: I replied. That comment already exists here I think https://review.openstack.org/#/c/259568/13/puppet/services/README.rst15:59
dprinceEmilienM: the goal is this entire file goes away15:59
EmilienMdprince: +A16:00
dprinceEmilienM: so lets put the comments where they matter16:00
EmilienMdprince: we need https://review.openstack.org/#/c/293999/ soon, otherwise tripleoCI will break.16:00
EmilienMdprince: puppet-nova is about to use puppet-oslo16:00
*** shivrao has joined #tripleo16:01
dprinceEmilienM: If the next CI pass doesn't work I think we can go ahead with this one and +A it16:01
*** hewbrocca has joined #tripleo16:02
openstackgerritMerged openstack/tripleo-heat-templates: Increment step count to include ringbuilder  https://review.openstack.org/23624316:03
EmilienMdprince: my patch is not related at all to the HA failure16:03
*** Guest58782 has quit IRC16:03
*** links has quit IRC16:04
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29884716:04
*** shivrao has quit IRC16:05
*** jtomasek has quit IRC16:07
*** snecklifter has quit IRC16:10
*** snecklifter has joined #tripleo16:11
*** jistr has quit IRC16:14
openstackgerritMike Burns proposed openstack/tripleo-heat-templates: change the default satellite tools rpm repo.  https://review.openstack.org/24589516:14
*** mkovacik has joined #tripleo16:15
*** yamahata has joined #tripleo16:18
gfidentedprince I think it'd be nice to cherry-pick Ibd4f51f65da475bb20a6b08d7bda673f330a5464 to mitaa16:21
gfidentea merge conflict around that in the future might turn bad :P16:21
dprincegfidente: I think it is related to a new feature16:23
*** ibravo has joined #tripleo16:23
gfidenteyeah the roles16:23
*** eil397 has joined #tripleo16:24
gfidentebut it is also generic enough to be worth porting; I was worried about the steps naming becoming confusing if we had to do more cherry-picks for buga16:24
gfidente*bugs16:24
*** Marga_ has joined #tripleo16:25
trownwon't post-composable roles backports be tricky no matter what?16:25
dprincetrown: yep16:29
*** derekh has joined #tripleo16:29
*** Marga_ has quit IRC16:31
*** penick has joined #tripleo16:32
*** ramishra has quit IRC16:32
openstackgerritMerged openstack/tripleo-puppet-elements: Add puppet-oslo to modules elements  https://review.openstack.org/29399916:34
*** sshnaidm has quit IRC16:34
*** Guest68910 is now known as mgagne16:38
*** mgagne has quit IRC16:38
*** mgagne has joined #tripleo16:38
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Adds a PostPuppetConfig for restart of neutron-openvswitch-agent  https://review.openstack.org/29886516:40
*** derekh has quit IRC16:41
*** rasca has quit IRC16:43
openstackgerritAttila Darazs proposed openstack-infra/tripleo-ci: Use IPv6 on the upgrades job  https://review.openstack.org/28944516:43
*** rasca has joined #tripleo16:44
*** nico_auv has quit IRC16:46
*** ifarkas has quit IRC16:48
*** pcaruana has quit IRC16:48
*** jtomasek has joined #tripleo16:49
*** akuznetsov has quit IRC16:53
*** Marga_ has joined #tripleo16:55
*** jpena is now known as jpena|off16:55
*** mgagne_ has joined #tripleo17:06
*** adarazs has quit IRC17:07
*** shadower has quit IRC17:07
*** igorbelikov has quit IRC17:07
*** snecklifter has quit IRC17:07
*** dmsimard has quit IRC17:07
*** larstobi has quit IRC17:07
*** adam_g has quit IRC17:07
*** akshai_ has quit IRC17:07
*** mgagne has quit IRC17:07
*** shakamunyi has quit IRC17:07
*** dhellmann has quit IRC17:07
*** gus has quit IRC17:07
*** dweaver has quit IRC17:07
*** Nakato has quit IRC17:07
*** saneax_AFK has quit IRC17:07
*** vkmc has quit IRC17:07
*** tosky has quit IRC17:07
*** al has quit IRC17:07
*** akshai has joined #tripleo17:07
*** barra204 has joined #tripleo17:07
*** adarazs_ has joined #tripleo17:07
*** trown is now known as trown|lunch17:07
*** tosky_ has joined #tripleo17:07
shardyhttps://review.openstack.org/#/c/274217/ is a docs patch which documents predictable placement17:07
shardyneeds a final review/+A if someone has a moment17:07
*** Nakato_ has joined #tripleo17:07
*** Marga_ has quit IRC17:07
*** Marga_ has joined #tripleo17:07
*** dmsimard1 has joined #tripleo17:07
*** larstobi_ has joined #tripleo17:07
*** adarazs_ is now known as adarazs17:07
*** adam_g` has joined #tripleo17:07
*** al has joined #tripleo17:07
*** dhellmann has joined #tripleo17:07
*** dweaver has joined #tripleo17:07
*** mgagne_ has quit IRC17:07
*** mgagne_ has joined #tripleo17:07
*** larstobi_ is now known as larstobi17:08
*** adam_g` is now known as adam_g17:08
*** adam_g has quit IRC17:08
*** adam_g has joined #tripleo17:08
*** gus has joined #tripleo17:09
*** snecklifter has joined #tripleo17:09
*** vkmc has joined #tripleo17:09
*** lucasagomes is now known as lucas-dinner17:09
openstackgerritMerged openstack/tripleo-docs: Extend the root device selection documentation  https://review.openstack.org/29049217:11
*** mgagne_ is now known as mgagne17:15
*** tiswanso has quit IRC17:15
mgouldhi everyone17:16
mgouldhow can I check the status of the check-tripleo CI pipeline?17:16
jpeelerdprince: could you review this? https://review.openstack.org/#/c/247724/17:16
mgouldI tried to recheck https://review.openstack.org/#/c/272206 7 hours ago, and nothing's happened yet17:16
*** tiswanso has joined #tripleo17:17
mgouldI can't find it on http://status.openstack.org/zuul/ - where else should I be looking?17:17
*** saneax_AFK has joined #tripleo17:17
*** igorbelikov has joined #tripleo17:17
*** saneax_AFK is now known as saneax17:17
*** ccamacho has joined #tripleo17:17
*** dmsimard1 is now known as dmsimard17:21
*** igorbelikov has quit IRC17:22
*** igorbelikov has joined #tripleo17:23
openstackgerritRyan Hallisey proposed openstack/tripleo-docs: Docs for containerized compute node  https://review.openstack.org/25474317:25
*** ohamada_ has quit IRC17:25
*** mannidi__ has joined #tripleo17:25
*** sshnaidm has joined #tripleo17:26
jaosoriordoes someone has time for reviewing this one? https://review.openstack.org/#/c/292799/17:28
*** eil397 has left #tripleo17:29
*** mannidi_ has quit IRC17:29
*** penick has quit IRC17:29
*** Marga__ has joined #tripleo17:34
*** penick has joined #tripleo17:34
*** mannidi__ has quit IRC17:34
mgouldjaosorior, I just left a typo comment, but I don't have enough brain left today for anything else :-(17:35
jaosoriormgould: know the feel :/17:36
mgouldright, hometime17:36
*** Marga__ has quit IRC17:36
mgouldgood night, everybody!17:36
*** Marga_ has quit IRC17:37
*** mgould has quit IRC17:37
*** Marga_ has joined #tripleo17:37
*** Marga_ has quit IRC17:37
*** Marga_ has joined #tripleo17:38
*** Marga_ has quit IRC17:39
*** Marga_ has joined #tripleo17:40
*** ebalduf_ has joined #tripleo17:40
*** rlandy has quit IRC17:50
*** gfidente has quit IRC17:53
*** tosky_ has quit IRC17:53
*** bvandenh has quit IRC17:54
*** shivrao has joined #tripleo17:59
openstackgerritMerged openstack-infra/tripleo-ci: Don't execute the output from wait_for  https://review.openstack.org/29529717:59
*** pcaruana has joined #tripleo18:02
*** hjensas has joined #tripleo18:03
Gonerihttps://review.openstack.org/#/c/288041/ hi, can someone take some time for a review (add ability to adjust the MTU for instack-undercloud)18:04
*** trown|lunch is now known as trown18:12
*** rasca has quit IRC18:13
*** rasca has joined #tripleo18:15
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891918:16
*** jdob has quit IRC18:17
*** jdob has joined #tripleo18:17
*** jaosorior has quit IRC18:18
*** florianf has quit IRC18:19
*** Marga__ has joined #tripleo18:21
openstackgerritMerged openstack/instack-undercloud: add INTERFACE_MTU parameter  https://review.openstack.org/28804118:22
*** panda has quit IRC18:25
*** panda has joined #tripleo18:25
*** Marga_ has quit IRC18:26
*** jdob has quit IRC18:27
*** jdob has joined #tripleo18:28
rookhey shardy - so with the validation script tht... This is the feedback I have from one of our switch vendors "the first packet that is beingn learned will not be forwarded. so if the ping is 1 packet only it might fail"18:39
rookI have seen that across multiple platforms18:39
rookso, the -W doesn't send more than a single packet from what I saw.18:41
shardyrook: Ok, can we simplify the fix by just increasing the count instead of looping on one ping?18:41
rookpossibly18:41
rookshardy: i haven't tried that.18:42
rookthat might be a good test18:42
shardyrook: Ok, I think that might be cleaner so would be good to try if possible18:42
*** david-lyle has quit IRC18:45
*** david-lyle has joined #tripleo18:46
EmilienMmarios: ping https://bugs.launchpad.net/tripleo/+bug/156343718:51
openstackLaunchpad bug 1563437 in tripleo "neutron-openvswitch-agent fails to talk to rabbit after upgrade completes successfully" [Undecided,In progress] - Assigned to Marios Andreou (marios-b)18:51
*** Marga__ has quit IRC18:52
rookhm shardy nope that seems to fial18:52
rookfail*18:52
*** Marga_ has joined #tripleo18:53
*** tiswanso has quit IRC18:55
*** tiswanso has joined #tripleo18:55
rookok, shardy increased the retry to 1018:59
*** Marga__ has joined #tripleo19:00
rookok, going to 10 seems to be the sweet spot shardy19:01
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891919:02
*** Marga_ has quit IRC19:03
*** rdopiera has quit IRC19:03
*** qasims has joined #tripleo19:05
*** akrivoka has quit IRC19:05
openstackgerritKeith Schincke proposed openstack/tripleo-heat-templates: Add support for deploying RGW on controllers.  https://review.openstack.org/28902719:05
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Increase heat rpc_response_timeout in the overcloud  https://review.openstack.org/29894119:12
rookshardy: let me know if that works better for you19:15
openstackgerritMerged openstack/tripleo-heat-templates: PLUMgrid Neutron integration  https://review.openstack.org/25791619:19
shardyrook: thanks, looks better!19:21
shardyjust one nit re the -W option19:21
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891919:22
rookshardy agreed with the comments. thanks19:23
shardyrook: lgtm - if you can raise a bug (launchpad) and reference it in the commit message, we can consider this for stable branches also19:27
* shardy heads off for dinner19:28
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: compute: restart OVS agent on neutron.conf changes  https://review.openstack.org/29894619:28
*** shardy is now known as shardy_afk19:28
EmilienMdprince: see ^19:28
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891919:35
*** qasims has quit IRC19:42
openstackgerritMerged openstack/instack-undercloud: Switch to package-installs  https://review.openstack.org/29136719:44
openstackgerritDmitry Ilyin proposed openstack/puppet-pacemaker: [WIP] Merge with fuel-infra/puppet-pacemaker  https://review.openstack.org/29644019:46
*** ebalduf has quit IRC19:50
*** bvandenh has joined #tripleo19:55
*** Goneri has quit IRC20:07
*** rhallisey has quit IRC20:08
*** Marga__ has quit IRC20:10
*** Marga_ has joined #tripleo20:12
EmilienMslagle: big highlight on https://review.openstack.org/#/c/298866/20:18
*** akuznetsov has joined #tripleo20:24
*** akuznetsov has quit IRC20:28
*** jtomasek_ has joined #tripleo20:30
*** jtomasek_ has quit IRC20:32
*** ccamacho has quit IRC20:38
*** ccamacho has joined #tripleo20:39
*** ccamacho1 has joined #tripleo20:39
*** ccamacho1 has quit IRC20:39
*** ccamacho2 has joined #tripleo20:40
*** rhallisey has joined #tripleo20:41
*** ccamacho has quit IRC20:42
*** tiswanso_ has joined #tripleo20:43
*** tiswanso has quit IRC20:45
*** pradk_ has joined #tripleo20:48
*** pradk has quit IRC20:48
*** pradk has joined #tripleo20:49
*** rlandy has joined #tripleo20:52
*** pradk_ has quit IRC20:52
*** Goneri has joined #tripleo20:53
*** ayoung has quit IRC20:54
*** pradk_ has joined #tripleo20:54
*** pradk_ has quit IRC20:54
*** ebalduf_ has quit IRC20:57
*** shardy_afk has quit IRC20:57
*** pradk has quit IRC20:57
*** bvandenh has quit IRC20:59
*** ebalduf has joined #tripleo21:03
*** jayg is now known as jayg|g0n321:04
*** trown is now known as trown|outtypewww21:07
*** pradk has joined #tripleo21:09
stevebakerbnemec: hey, are you about?21:12
bnemecstevebaker: Yep21:13
stevebakerbnemec: I think I've found a way of running ovb without a patched nova, by booting bm images with an ipxe.iso image instead of an empty image21:14
bnemecstevebaker: Yeah, that should work for CI purposes where we only do one deployment.21:14
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Wait for mysql cluster in HA job  https://review.openstack.org/29898621:16
stevebakerbnemec: it means patching out set_boot_device in openstackbmc - does it need to switch bms to hd for subsequent boots?21:17
bnemecstevebaker: It should be fine as long as you're using localboot.  Does set_boot_device actually cause problems?  All it's doing is setting instance metadata that would be ignored by an unpatched Nova.21:18
*** dprince has quit IRC21:19
*** lblanchard has quit IRC21:20
stevebakerbnemec: I'm using a patched nova, so I just wanted to make sure it behaved like a non patched nova21:21
stevebakerbnemec: can you describe the difference between localboot and not localboot?21:21
bnemecstevebaker: With localboot you only need to pxe boot once for deployment, then after that it boots from HD.21:22
bnemecWithout it the node needs to pxe boot every time, which doesn't work if you're using an ipxe image to do pxe.21:22
stevebakerbnemec: and how is localboot selected?21:23
*** jprovazn has quit IRC21:23
bnemecstevebaker: It's a combination of ironic node capabilities and nova flavor extra-specs.21:24
stevebakerboot_option:local21:24
bnemecThere are some details about how the nodes and flavors need to be configured on http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/profile_matching.html21:24
bnemecWe use localboot by default right now, FWIW.21:25
openstackgerritMerged openstack/python-tripleoclient: Add gnocchi password as a deployment param  https://review.openstack.org/29071021:25
*** akuznetsov has joined #tripleo21:25
stevebakerbnemec: hmm, result from introspection is Introspection completed with errors: The following required parameters are missing: ['local_gb']21:25
stevebakerbnemec: I wonder if that is because its booting ipxe as its main image21:26
bnemecstevebaker: Possibly.  I think when I did this way back when I stuck ipxe into a 50 GB qcow2 image so the root partition still showed up as being large enough.21:27
*** julim has quit IRC21:28
bnemecOr may 40 GB.  Whatever the docs recommend for the empty image.21:28
bnemec*maybe21:28
stevebakerbnemec: lol, should I persevere with this? Having to patch nova is quite a barrier to using ovb21:29
*** akuznetsov has quit IRC21:30
bnemecstevebaker: I know it can work.  I successfully deployed an overcloud node a long time ago with this method, but at the time we didn't have localboot so it was a dead end back then.21:30
*** bvandenh has joined #tripleo21:31
stevebakerbnemec: are you against having ipxe images living in ovb git if that makes things easier for people?21:32
bnemecstevebaker: No, I'd be totally fine with that.  It should be pretty small with a sparse qcow image.21:33
stevebakerbnemec: do you recall how you put it in a qcow?21:33
*** ebalduf has quit IRC21:36
openstackgerritMerged openstack/tripleo-heat-templates: Don't restart Pacemaker-managed services from Puppet  https://review.openstack.org/29883421:36
bnemecstevebaker: I think I actually used larsks's method from http://serverfault.com/questions/469479/does-nova-support-pxe-boot21:37
stevebakerbnemec: cool, I'll have a play21:38
bnemecThat actually doesn't create a qcow, but it's still pretty small.21:38
stevebakeryep21:38
*** egafford has quit IRC21:39
bnemecIt's possible I converted the raw image to a qcow later, since a 4 MB root partition is unlikely to work.21:40
bnemecI'm kind of guessing based on the old image files I have lying around on my hard drive. :-)21:40
*** bvandenh has quit IRC21:41
bnemecAll I see is a raw pxeboot.img, but I may have done more manipulation of it on the (now long-reprovisioned) OpenStack box I was using back then.21:41
*** morazi has quit IRC21:43
*** ayoung has joined #tripleo21:48
*** MaxPC has quit IRC21:49
*** ccamacho2 has quit IRC21:54
*** ccamacho has joined #tripleo21:57
*** trozet has quit IRC22:03
*** dustins has quit IRC22:07
openstackgerritDmitry Ilyin proposed openstack/puppet-pacemaker: [WIP] Merge with fuel-infra/puppet-pacemaker  https://review.openstack.org/29644022:09
*** ccamacho has quit IRC22:10
rookbnemec ping22:12
bnemecrook: pong22:12
rookRE change 29891922:12
rookso, the discussion with one switch vendor is for the first packet, however I have seen where STP eats a couple of packets.22:13
rookbnemec I was giving a single example. I do see your point with the amount of time it takes.22:13
rookbnemec however, the flip side is, if the network IS really ok, it just takes time to converge, it is saving people time since this validation script can have the entire deployment fail due to a missed ping.22:14
rookSo, is it worth the extra couple of minutes VS a failed deployment? I personally think so... but I have been bit by this plenty of times now.22:14
bnemecrook: Yeah, I wouldn't block it solely over that, but I did want to have the discussion because 1 minute times a lot of CI jobs and developer deployments is non-trivial, so if we don't need to add that time then I'd rather not.22:17
rookugh... ok I didn't think about CI..22:17
bnemecActually, this makes me wonder if we should be using wait_for in this script now instead of the built-in ping timeout.22:17
rookbnemec: maybe we could add more intelligence to this? To handle CI vs IRL22:17
bnemecThat way it would end as soon as one ping comes back instead of continuing to do 10 no matter what.22:18
bnemecI don't think we have wait_for available on the overcloud images, but we might be able to use the same strategy in this script.22:19
rookbnemec: well, the first commit i have did something similar..22:19
rookI was using a retry var and had RETRY=522:20
rookif the first ping worked, it just moves on22:20
rooki mean we could implement another ping before this one.22:20
rookif it returns 0, move on, else try the extended ping22:21
bnemecHmm, I kind of like that.  Simpler than the loop, but still accomplishes what we need.22:22
rookbnemec the comment regarding the default gw - I haven't seen this code until now (must be mitaka)...22:24
rookwe could add similar logic.22:24
*** ibravo has quit IRC22:25
bnemecrook: Yeah, that's the one that always bites me in my dev environment because I forget to configure the public interface on my undercloud. :-)22:26
rooklol22:26
rookroger.22:26
bnemecIt seems prudent to make it more robust against strange network glitches too though.22:26
*** ccamacho has joined #tripleo22:27
*** akuznetsov has joined #tripleo22:27
rookalso, withthe code, do we assume the GW is not ipv6?22:27
rookcode/script22:27
* rook can patch that up too22:27
bnemecThat I actually don't know, but I would assume some day we'll support ipv6 for gateways.22:28
rookok, patching because with the ping we use ping vs $ping22:29
*** r-mibu has quit IRC22:31
*** r-mibu has joined #tripleo22:31
*** akuznetsov has quit IRC22:31
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891922:38
rook^ bnemec took in consideration CI based on our above convo... Tested it out. Let me know if you have any other comments! :)22:38
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891922:39
openstackgerritJoe Talerico proposed openstack/tripleo-heat-templates: Ping retry  https://review.openstack.org/29891922:40
rookGah... test code22:40
openstackgerritEmilien Macchi proposed openstack/tripleo-docs: Add MidoNet documentation in advanced deployment  https://review.openstack.org/27032022:49
*** thrash is now known as thrash|g0ne22:53
openstackgerritMerged openstack/tripleo-docs: Document benchmarks and add extra data examples  https://review.openstack.org/29060522:54
openstackgerritMerged openstack/tripleo-docs: Update list of pkgs in includepkgs  https://review.openstack.org/28389922:54
EmilienMslagle: can I +A https://review.openstack.org/298684 ? it's not failing because of the patch22:55
openstackgerritMerged openstack/tripleo-heat-templates: Set the defaultbranch to stable/mitaka  https://review.openstack.org/29882022:56
openstackgerritMerged openstack/tripleo-heat-templates: Remove password default for AodhPassword  https://review.openstack.org/29882122:56
*** ccamacho has quit IRC23:03
*** ccamacho has joined #tripleo23:04
openstackgerritGonĂ©ri Le Bouder proposed openstack/instack-undercloud: ironic: turn on ipxe timout  https://review.openstack.org/29478723:12
openstackgerritMerged openstack/tripleo-docs: Document using node capabilities to control placement  https://review.openstack.org/27421723:15
*** jtomasek has quit IRC23:25
rookbnemec:  do we know if there is being any work done to look at the memory consumption of the heat-engine during a deployment?23:25
rookDepoying ~ 100 nodes, we can consume > 32GB RAM23:25
rookThankfully we have swap.23:25
*** akuznetsov has joined #tripleo23:28
*** akuznetsov has quit IRC23:32
*** jtomasek has joined #tripleo23:38
openstackgerritRyan Hallisey proposed openstack/tripleo-docs: Docs for containerized compute node  https://review.openstack.org/25474323:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!