Wednesday, 2018-04-11

*** yamahata has joined #tripleo00:07
*** itlinux has joined #tripleo00:07
*** jcoufal has joined #tripleo00:20
openstackgerritMerged openstack/puppet-tripleo master: Fix docker debug/mirrors JSON augeas changes  https://review.openstack.org/55979000:31
openstackgerritMerged openstack/tripleo-heat-templates master: Add endpoint map environment without TLS  https://review.openstack.org/55932200:32
openstackgerritMerged openstack/python-tripleoclient master: Use no TLS environment explicitly  https://review.openstack.org/55932300:32
*** dhill_ has quit IRC00:40
*** dparkes has quit IRC00:43
*** artom has joined #tripleo00:45
*** sai_p has quit IRC00:45
*** jcoufal has quit IRC00:49
*** dmacpher has joined #tripleo00:50
*** agopi has quit IRC01:01
*** fzdarsky_ has joined #tripleo01:04
*** fzdarsky has quit IRC01:08
*** psahoo has joined #tripleo01:09
*** fzdarsky_ has quit IRC01:09
*** wolverineav has quit IRC01:13
*** wolverin_ has joined #tripleo01:13
*** fragatina has quit IRC01:13
*** fragatina has joined #tripleo01:14
*** fzdarsky_ has joined #tripleo01:14
*** fragatin_ has joined #tripleo01:16
*** fragatina has quit IRC01:18
*** fragatin_ has quit IRC01:20
*** dprince has joined #tripleo01:28
*** wolverin_ has quit IRC01:32
*** wolverineav has joined #tripleo01:32
*** ihrachys has quit IRC01:33
*** ykarel|away has joined #tripleo01:34
*** eck` is now known as eck`gone01:35
*** wolverin_ has joined #tripleo01:35
*** wolverineav has quit IRC01:36
*** wolverin_ has quit IRC01:40
*** cshastri has joined #tripleo01:44
*** agopi has joined #tripleo01:48
*** wolverineav has joined #tripleo01:54
*** ykarel|away has quit IRC01:54
*** wolverineav has quit IRC01:58
*** wolverineav has joined #tripleo02:01
*** ramishra has joined #tripleo02:29
*** dprince has quit IRC02:33
*** ayoung has quit IRC02:58
*** morazi has joined #tripleo02:58
*** psachin has joined #tripleo03:01
*** chlong has joined #tripleo03:02
*** psachin has quit IRC03:09
*** dmacpher has quit IRC03:10
*** shreshtha has joined #tripleo03:19
*** shreshtha is now known as shreshtha-wfh03:19
*** fragatina has joined #tripleo03:20
*** fragatina has quit IRC03:22
openstackgerritMerged openstack/tripleo-heat-templates master: Use sensu-client healthcheck parameter  https://review.openstack.org/54528003:23
*** fragatina has joined #tripleo03:23
*** psachin has joined #tripleo03:27
openstackgerritNguyen Hai proposed openstack/python-tripleoclient master: Follow the new PTI for document build [Updated]  https://review.openstack.org/55675303:28
*** udesale has joined #tripleo03:34
*** SlickNik has quit IRC03:35
*** agopi has quit IRC03:37
openstackgerritJohn Trowbridge proposed openstack-infra/tripleo-ci master: Add job to test overcloud updates  https://review.openstack.org/55939403:44
openstackgerritMerged openstack/tripleo-heat-templates master: Stop configuring nova_catalog_admin_info for cinder  https://review.openstack.org/55997603:48
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Fix missing allowed network type 'flat' for ODL OVS  https://review.openstack.org/56010103:50
openstackgerritMerged openstack/tripleo-common master: Don't set deployments as hostvars  https://review.openstack.org/55943303:50
openstackgerritMerged openstack/tripleo-heat-templates master: Add environment to enable Designate  https://review.openstack.org/55500603:50
openstackgerritMerged openstack/tripleo-heat-templates master: Don't use keystone admin endpoint for nova placement  https://review.openstack.org/55996903:50
*** fragatina has quit IRC04:06
*** fragatina has joined #tripleo04:06
*** skramaja has joined #tripleo04:10
*** morazi has quit IRC04:16
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: update rdo settings for new registry namespace  https://review.openstack.org/56025604:19
*** jaosorior has joined #tripleo04:21
Tenguhello there04:26
*** udesale has quit IRC04:26
*** rlandy has quit IRC04:31
jaosorioryooo04:31
jaosoriorhow's it going?04:31
Tengutired, gone to bed a bit too late X)04:34
Tenguhmmm. my upgrade is failing… need to dig the error.04:35
Tenguduh. my bad -.-04:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Temporarily disable redirection with TLS  https://review.openstack.org/56025804:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Re-enable redirection based on TLS  https://review.openstack.org/56025904:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: TLS by default for the overcloud  https://review.openstack.org/55492604:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Switch public endpoints to use FQDNs by default  https://review.openstack.org/55992604:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Change default endpoint map entries to use TLS  https://review.openstack.org/55842404:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove empty default for SSLKey  https://review.openstack.org/55927904:40
Tengudidn't dropped the docker_registry file, meaning it didn't get regenerated by my script, meaning it didn't get the new ceph tag -.-'. corrected the script so that it will always generate a fresh registry.04:42
jaosoriordciabrin: do you want to give a try testing this out https://review.openstack.org/#/c/557653/ ? or should I merge it?04:45
*** udesale has joined #tripleo04:45
*** udesale has quit IRC04:46
*** udesale has joined #tripleo04:47
openstackgerritNoam Angel proposed openstack/tripleo-heat-templates master: Fix NeutronCorePlugin path in baremetal-services.yaml  https://review.openstack.org/56026104:47
openstackgerritNoam Angel proposed openstack/tripleo-heat-templates master: Fix NeutronCorePlugin path in baremetal-services.yaml  https://review.openstack.org/56026104:48
*** yprokule has joined #tripleo05:03
*** anilvenkata has joined #tripleo05:11
*** chlong has quit IRC05:12
*** d0ugal has quit IRC05:16
*** pdeore has joined #tripleo05:19
*** marios has joined #tripleo05:20
*** d0ugal has joined #tripleo05:27
*** links has joined #tripleo05:28
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: containerized-undercloud: don't manage stackrc  https://review.openstack.org/55981805:33
*** anande has joined #tripleo05:33
*** ratailor has joined #tripleo05:35
EmilienMweshay: I think we could run fs010 with containerized undercloud but just in periodic jobs, so we can keep good testing coverage05:40
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Deploy container-multinode (fs010) with a containerized undercloud""  https://review.openstack.org/56026405:40
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Revert "Revert "Deploy container-multinode (fs010) with a containerized undercloud""  https://review.openstack.org/56026405:40
*** anande has quit IRC05:41
bandiniEmilienM: if I wanted to install a couple of packages (pcs and pacemaker-cli) before spec tests are run in puppet-pacemaker, how would I go about doing that?05:42
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient master: containerized/undercloud: disable verbosity by default  https://review.openstack.org/55914005:43
bandinibasically in https://review.openstack.org/#/c/559919/ I added some functions and spec tests but they require a couple of packages to be installed to be able to run05:43
jaosoriorbandini: I think you gotta add them to the bindep file05:43
jaosoriorbindep.txt05:43
bandiniooh /me looks05:44
bandinijaosorior: sweet! lol I kep looking around rakefiles and gemfiles and whatnot ;)05:44
bandini*kept05:44
*** wolverineav has quit IRC05:46
EmilienMbandini: yeah bindep05:47
bandinithanks jaosorior EmilienM!05:48
EmilienMbandini: off to bed now, add me as reviewer05:48
bandiniEmilienM: cheers, I will once CI is green ;) good night!05:51
openstackgerritMichele Baldessari proposed openstack/puppet-pacemaker master: Introduce some new functions in pcmk_common  https://review.openstack.org/55991905:53
openstackgerritMichele Baldessari proposed openstack/puppet-pacemaker master: Add a deep_compare option in bundles and resources  https://review.openstack.org/55992005:53
openstackgerritMichele Baldessari proposed openstack/puppet-pacemaker master: Detect when a bundle or a resource needs updating  https://review.openstack.org/55992105:53
* bandini bbiab05:54
*** ykarel|away has joined #tripleo05:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient master: containerized/undercloud: disable verbosity by default  https://review.openstack.org/55914005:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/python-tripleoclient master: containerized/undercloud: disable verbosity by default  https://review.openstack.org/55914005:58
openstackgerrityolanda.robla proposed openstack/tripleo-heat-templates stable/newton: Change ovs user and fix permissions on ovs upgrade  https://review.openstack.org/55847105:59
openstackgerrityolanda.robla proposed openstack/tripleo-heat-templates stable/ocata: Change ovs user and fix permissions on ovs upgrade  https://review.openstack.org/55947406:01
*** rodolof has joined #tripleo06:01
*** jfrancoa has joined #tripleo06:02
*** tzumainn has quit IRC06:02
*** ykarel|away is now known as ykarel06:03
*** kopecmartin has joined #tripleo06:03
openstackgerritMerged openstack-infra/tripleo-ci master: Use same namespace tripleo{{release}} everywhere  https://review.openstack.org/56013306:06
*** trown|outtypewww has quit IRC06:06
*** trown has joined #tripleo06:08
*** rodolof has quit IRC06:08
*** rodolof has joined #tripleo06:09
openstackgerritNoam Angel proposed openstack/tripleo-heat-templates master: Fix NeutronCorePlugin path in baremetal-services.yaml  https://review.openstack.org/56026106:12
openstackgerritNoam Angel proposed openstack/tripleo-heat-templates master: Fix few issues with deploying with baremetal-services.yaml  https://review.openstack.org/56026106:14
openstackgerritMerged openstack/python-tripleoclient master: Add missing unit tests for undercloud_deploy  https://review.openstack.org/55545606:15
*** anande has joined #tripleo06:19
*** nyechiel has joined #tripleo06:22
*** quiquell has quit IRC06:23
*** agurenko has joined #tripleo06:24
*** quiquell has joined #tripleo06:24
*** anande has quit IRC06:28
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Pass env-files and roles-data file to update/upgrade prepare cli.  https://review.openstack.org/55975506:28
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/56028306:31
*** dbecker has quit IRC06:32
*** anande has joined #tripleo06:33
*** radeks has joined #tripleo06:33
*** holser__ has joined #tripleo06:37
*** dsariel has joined #tripleo06:39
*** dsariel has quit IRC06:40
*** dmacpher has joined #tripleo06:40
*** chkumar|off is now known as chandankumar06:41
dciabrinjaosorior, hey. that's ok I think you can merge it06:41
*** dbecker has joined #tripleo06:44
*** slaweq has joined #tripleo06:47
*** cylopez has joined #tripleo06:49
jaosoriordciabrin: cool cool, done; thanks!06:51
*** janki has joined #tripleo06:54
openstackgerritMerged openstack-infra/tripleo-ci master: Add overcloud-deploy-post tag to CI deployment  https://review.openstack.org/55997906:56
*** nyechiel_ has joined #tripleo06:57
*** florianf has joined #tripleo06:58
*** florianf has quit IRC06:58
*** nyechiel has quit IRC06:58
*** florianf has joined #tripleo06:58
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Added network enabled check in multiple nic role rendering file  https://review.openstack.org/56006406:59
openstackgerritMerged openstack/tripleo-upgrade master: Fix a nit in the oooq transformation process.  https://review.openstack.org/56008006:59
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Switch public endpoints to use FQDNs by default  https://review.openstack.org/55992607:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Change default endpoint map entries to use TLS  https://review.openstack.org/55842407:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove empty default for SSLKey  https://review.openstack.org/55927907:02
*** rcernin has quit IRC07:06
*** nyechiel_ has quit IRC07:07
*** nyechiel_ has joined #tripleo07:07
*** amoralej|off is now known as amoralej07:08
*** ccamacho has joined #tripleo07:10
*** dparkes has joined #tripleo07:11
*** shardy has joined #tripleo07:14
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Pass env-files and roles-data file to update/upgrade prepare cli.  https://review.openstack.org/55975507:18
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade stable/queens: Fix a nit in the oooq transformation process.  https://review.openstack.org/56029307:19
*** masco has joined #tripleo07:22
*** tesseract has joined #tripleo07:28
*** ffiore has joined #tripleo07:30
*** ffiore has quit IRC07:30
*** ykarel is now known as ykarel|lunch07:31
*** ffiore has joined #tripleo07:31
*** bogdando has joined #tripleo07:31
openstackgerritNir Magnezi proposed openstack/tripleo-heat-templates master: Containerize Neutron LBaaS service plugin  https://review.openstack.org/55501107:32
*** dmacpher has quit IRC07:33
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Expand jinja variables at one place only  https://review.openstack.org/55836907:44
holser__shardy - concerning https://bugs.launchpad.net/tripleo/+bug/176240307:48
openstackLaunchpad bug 1762403 in tripleo "j2 templates are not rendered during upgrade" [High,Confirmed]07:48
shardyholser__: Hi, I think this is a known issue07:48
holser__Is it documented?07:48
shardythe --templates path must always match the -e path in the case where j2 rendering is involved07:49
shardyholser__: not sure07:49
holser__shardy - the problem is if I am operator I can run whatever I want07:49
shardyholser__: probably there should be better validation in the client to prevent users from doing this07:49
shardyholser__: I understand07:49
holser__1. so framework should warn I do something wrong07:49
shardyholser__: if you look in tripleoclient you'll see there is some path comparison07:49
holser__fail/warn07:49
shardyholser__: yes that's probably the bug here, insufficient validation07:50
shardyI'll comment on the bug07:50
holser__yeah! that's what I am talking about07:50
holser__shardy we are on the same page now07:50
shardyperhaps there's some way to support out-of-tree j2 templates but right now I know it wont work, and never has07:50
shardyholser__: yes07:51
bogdandojaosorior: hi. I'm trying to deploy TLS undercloud, reading http://git.openstack.org/cgit/openstack/tripleo-docs/tree/doc/source/install/advanced_deployment/tls_everywhere.rst . Shall I install ipa-server and prepare it manually? t-h-t and quickstart do not include those steps AFAIK...07:51
*** dsariel has joined #tripleo07:51
bogdandobasically, I have haproxy deploy error, and /etc/pki/tls/private/overcloud_endpoint.pem is empty07:51
holser__thanks shardy07:51
bogdandoand heat installer logs "unable to load Private KeyЭ07:52
bogdando"* :)07:52
*** pkovar has joined #tripleo07:52
bogdandodigging into the undercloud deploy -e command args, I can see there environments/public-tls-undercloud.yaml:, which adds to the resources registry OS::TripleO::Services::HAProxyPublicTLS, but I did not add it to UndercloudServices, shall I ?.. Docs are not very clear on that, sorry07:53
*** jpena|off is now known as jpena07:55
*** tosky has joined #tripleo07:55
bogdandoand OS::TripleO::Services::HAProxyInternalTLS is prolly not that I need... What about OS::TripleO::Services::CertmongerUser, shall I place it into resource registry and undercloud composed services?..07:55
bogdandosigh, that's really needs better examples in tht and docs07:56
bogdandodo we have some dev notes for TLS undercloud?07:56
bogdandovia tht07:57
*** yprokule has quit IRC07:58
*** jpich has joined #tripleo07:58
bogdandoyeah, it seems I missed ::CertmongerUser08:00
bogdandostill not sure for ::HAProxyInternalTLS tho08:00
bogdandoand public one08:00
*** yprokule has joined #tripleo08:01
*** arxcruz|off is now known as arxcruz|ruck08:04
*** gkadam has joined #tripleo08:06
bogdandook I see those are only for in-direct use http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/puppet/services/haproxy.yaml#n10108:07
bogdandoso the only question is either I have to install and configure ipa-server for TLS undercloud?08:07
jaosoriorbogdando: they do08:07
*** nyechiel has joined #tripleo08:07
bogdando...and when may I use environments/fixed-ip-vips.yaml08:07
bogdandolike if I have those VIPs pre-deployed/external?08:08
*** nyechiel_ has quit IRC08:08
jaosoriorbogdando: so, with quickstart you could use config/general_config/deprecated/ipa.yml  for the config and config/nodes/1ctlr_1comp_1supp.yml for the topology08:09
jaosoriorbogdando: you can use environments/public-tls-undercloud.yaml if you need the public certificate to be generated by IPA08:09
jaosoriorbogdando: most deployers will most likely have a publicly signed certificate, so the classic way of doing public TLS still apply08:10
openstackgerritMerged openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/56028308:10
bogdandoauto-generated self-signed works for me08:10
bogdandoso I only need to include free ipa install bits for my UC, right?08:11
*** lucas-afk is now known as lucasagomes08:11
bogdandojaosorior: thanks, looking into config/general_config/deprecated/ipa.yml now08:11
*** nyechiel_ has joined #tripleo08:12
*** nyechiel has quit IRC08:12
*** pcaruana has joined #tripleo08:13
*** suuuper has joined #tripleo08:14
*** nyechiel has joined #tripleo08:18
*** nyechiel_ has quit IRC08:18
*** salmankhan has joined #tripleo08:19
*** mdnadeem has joined #tripleo08:20
*** ykarel|lunch is now known as ykarel08:21
jaosoriorbogdando: did you use environments/ssl/enable-internal-tls.yaml or environments/enable-internal-tls.yaml ?08:22
bogdandonope, for now I want to try only public TLS08:23
jaosoriorbogdando: ok, now I got a bit confused08:23
jaosoriorbogdando: soo... you're trying to do public TLS with a regular undercloud or with a containerized undercloud?08:23
*** ratailor has quit IRC08:24
jaosoriorbogdando: CertmongerUser should be declared by default https://github.com/openstack/tripleo-heat-templates/commit/7414edfee73b827c4b7c343ea1ee388d78848c0408:25
bogdandojaosorior: containerized one08:26
bogdandovia heat, as I noted08:26
openstackgerritSteven Hardy proposed openstack/tripleo-common master: Update swift plan in UpdatePlanEnvironmentAction  https://review.openstack.org/56031108:26
openstackgerritSteven Hardy proposed openstack/tripleo-common master: UpdatePlanEnvironmentAction enable overwrite or merge  https://review.openstack.org/56031208:26
shardyramishra: ^^ those may help wrt the client updating the plan_environment08:26
*** yprokule has quit IRC08:26
jaosoriorbogdando: ok, are you deploying it manually? (manually adding the heat environments) or are you using undercloud_config ?08:26
*** ratailor has joined #tripleo08:27
*** agurenko has quit IRC08:27
ramishrashardy: sure, will check them, thanks!08:27
bogdandoyes, I'm adding that CertmongerUser and Novajoin as well, would it install ipa-server also?08:27
jaosoriorbogdando: novajoin and ipa-server are only needed for the TLS everywhere setup08:27
*** nyechiel_ has joined #tripleo08:27
bogdandojaosorior: here is my underlcoud.conf, it uses extra tht env files08:27
*** nyechiel has quit IRC08:27
*** agurenko has joined #tripleo08:28
jaosoriorbogdando: ok, lets check it out08:29
bogdandojaosorior: https://github.com/bogdando/oooq-warp/commit/af476da70adc77876675ce8dda4c8bcb75da433708:29
jaosoriorbogdando: although, for just doing public TLS in the undercloud, you shouldn't need any extra t-h-t env files08:29
bogdandojaosorior: well, I have a special one, for AIO08:30
bogdandothis helps to learn the deployment framework w/o overclouds overhead :)08:30
*** anilvenkata has quit IRC08:30
bogdandoso I'm learning now net configs and TLS :)08:31
jaosoriorbogdando: OK, if you JUST wanna do public TLS. You don't need novajoin there. You do need  OS::TripleO::Services::CertmongerUser   and you should probably use what's defined in this file https://github.com/openstack/tripleo-heat-templates/blob/master/environments/public-tls-undercloud.yaml08:31
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Expand jinja variables at one place only  https://review.openstack.org/55836908:31
bogdandogot it, thanks! jaosorior08:31
bogdandoso ipa server and novajoin is only needed for internal TLS?08:31
jaosoriorbogdando: correct08:31
*** paramite has joined #tripleo08:32
jaosoriorbogdando: with public TLS you only have one public-facing cert/key set. So we let the deployer manage that.... with internal TLS, we have a bunch of certificates per-host; so we rather use a CA to get those, and certmonger to manage the certs.08:33
bogdandoum, wait, so I still have to create something before start a deploy of undercloud?08:33
bogdandolike that public cert?08:34
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Verify the Swift container exists with a small utility workflow  https://review.openstack.org/52821308:34
jaosoriorbogdando: that was the overcloud.08:34
bogdandoum, I don't have it :)08:34
jaosoriorbogdando: for the undercloud, we assume it's one node. So we can just use certmonger's internal CA08:34
bogdandookay, got iy08:34
bogdandothanks!08:34
jaosoriorbogdando: so you have two options: 1) inject your own cert 2) use certmonger's internal CA08:35
jaosoriorall of this is already set up if you use generate_service_certificate in undercloud.conf08:35
jaosoriorbogdando: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/undercloud_config.py#L728  https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/undercloud_config.py#L75008:36
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Reduce reproducer redudancy  https://review.openstack.org/55836908:36
bogdandojaosorior: yes, I use generate_service_certificate. Just missed the CertmongerUser service in the role file08:37
*** derekh has joined #tripleo08:40
*** radeks has quit IRC08:41
*** radeks has joined #tripleo08:42
*** ratailor is now known as rtailor08:44
Tengujaosorior: btw, any way to tell certmonger to request certs from a freeipa? :)08:47
Tengujaosorior: would that be supported at some level in tripleo, for both under and overcloud?08:47
Tengujaosorior: (meaning: not only for the internal TLS ;))08:48
*** salmankhan has quit IRC08:49
*** nyechiel_ has quit IRC08:52
*** nyechiel_ has joined #tripleo08:53
jaosoriorTengu: yes ther eis08:55
jaosorior* yes there is08:55
jaosoriorfor both08:55
jaosoriorTengu: so, if you're using novajoin already, then there's not much to add but this environment: https://github.com/openstack/tripleo-heat-templates/blob/master/environments/services/haproxy-public-tls-certmonger.yaml08:56
jaosoriorTengu: if you're not using novajoin, you also need to do some more steps08:56
jaosoriorso, you can tell certmonger to use FreeIPA by seeting the CertmongerCA parameter to 'IPA'08:57
jaosorioruhm... and that's it. certmonger will try to contact IPA, assuming your node is enrolled and the haproxy/<fqdn> service principals exist in the IPA database08:58
openstackgerritMerged openstack/tripleo-heat-templates master: Mount the public TLS certificate for HAProxy on up(date|grade) on pacemaker  https://review.openstack.org/55765308:58
Tengujaosorior: hmm ok. Ah, right, the novajoin thingy... will need to work on that. Maybe that would be a nice "first thing" next month for me? :)08:59
jaosoriorTengu: we could sure use some help there :D08:59
Tengujaosorior: I'll check that. That's a small thing I really like to see working properly, even in some… exotic setups like I currently have at work (dedicated realm for IPA NOT matching the cloud domain)09:00
rtailorCan anyone tell, how to test tripleo-puppet-elements changes ?09:00
*** nyechiel has joined #tripleo09:00
*** nyechiel_ has quit IRC09:01
Tengurtailor: puppet rspec ?09:01
jaosoriorTengu: thought we had fixed that realm issue (we added a config parameter to novajoin)09:02
jaosorioroh well09:02
Tengujaosorior: hmm, there are other things. Don't remember, that was last year :D09:02
Tenguwe'll take the time to check that later ;).09:03
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates stable/queens: Mount the public TLS certificate for HAProxy on up(date|grade) on pacemaker  https://review.openstack.org/56032209:03
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Verify the Swift container exists with a small utility workflow  https://review.openstack.org/52821309:03
*** b00tcat has joined #tripleo09:04
*** b00tcat has left #tripleo09:04
rtailorTengu, https://github.com/openstack/tripleo-puppet-elements09:05
openstackgerritCarlos Camacho proposed openstack/tripleo-common master: Allow uploading big files to swift (5GB)  https://review.openstack.org/55902409:05
Tengurtailor: erf… no idea then :/09:06
rtailorjaosorior, Do you have any idea ^^09:06
Tengurtailor: maybe by building an image? But I think EmilienM would be the one for the answer.09:06
rtailorTengu, I think, we can't do like this http://jaormx.github.io/2016/modifying-overcloud-images/09:07
*** pblaho has joined #tripleo09:08
Tengurtailor: hm, this is for the modification of existing image. apparently the repository you point build the images from 009:08
Tengurtailor: and there's an instructin for the image building in the readme, but I don't think this would be a real state of the art test.09:09
*** anande has quit IRC09:13
*** anande has joined #tripleo09:13
*** gfidente has joined #tripleo09:14
*** gfidente has quit IRC09:14
*** gfidente has joined #tripleo09:14
rtailorTengu, Thanks! let me check that.09:16
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Replace message OVB by multinode  https://review.openstack.org/56032509:19
*** salmankhan has joined #tripleo09:19
openstackgerritJulie Pichon proposed openstack/python-tripleoclient master: Remove role commands deprecated in Queens  https://review.openstack.org/55905009:27
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: Run playbooks with custom args  https://review.openstack.org/55347409:29
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: Import gated tripleo-upgrade role.  https://review.openstack.org/55821009:29
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: Use tripleo-upgrade role in undercloud upgrades job.  https://review.openstack.org/54897409:29
*** cylopez has left #tripleo09:29
owalshrtailor, Tengu: yea, building an image is what I do to test - https://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html#get-images09:31
Tenguowalsh: ok :). too easy then :]. thanks for precision09:35
*** akrivoka has joined #tripleo09:37
*** salmankhan1 has joined #tripleo09:37
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Added enable_tempest flag for installing tempest on undercloud  https://review.openstack.org/55851009:38
*** salmankhan has quit IRC09:38
*** salmankhan1 is now known as salmankhan09:38
*** anilvenkata has joined #tripleo09:38
openstackgerritChandan Kumar proposed openstack/python-tripleoclient master: Added fake tempest docker service to undercloud config  https://review.openstack.org/55127109:41
openstackgerritChristian Schwede proposed openstack/tripleo-common stable/queens: Fix parameter indentation on Swift rebalance playbook  https://review.openstack.org/56033709:47
chandankumarsshnaidm: Hello09:49
sshnaidmchandankumar, hi09:49
openstackgerrityolanda.robla proposed openstack/tripleo-heat-templates stable/pike: Add openvswitch/hugetlbfs user and group on update  https://review.openstack.org/56033809:49
chandankumarsshnaidm: https://review.openstack.org/#/c/554522/ in order to test tempest undercloud for stable branches where can i change in tht?09:49
chandankumarsshnaidm: will i pick any file and make random changes?09:49
*** salmankhan has quit IRC09:50
chandankumarsshnaidm: tripleo-ci-centos-7-undercloud-oooq runs on all stable branches09:50
sshnaidmchandankumar, yeah, I think it'll work09:50
chandankumarsshnaidm: white space changes?09:50
sshnaidmchandankumar, just add some comment to one of service yaml files09:51
chandankumarsshnaidm: sure09:51
sshnaidmchandankumar, readme, whitespace, etc could be ignored09:51
jaosoriormandre: are you around?09:52
openstackgerritHamdy Khader proposed openstack/tripleo-heat-templates master: Add support for NVMeOF cinder backend  https://review.openstack.org/55957209:54
jaosoriorbandini: dude, if you have some time could you review these? https://review.openstack.org/#/q/topic:public-tls-default+status:open got it working :D09:55
*** gfidente has quit IRC09:55
jaosoriorshardy, mandre: Is there a way to force a specific container to run every time?09:56
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/queens: [DNM] testing tempest on undercloud for queens  https://review.openstack.org/56034309:57
*** salmankhan has joined #tripleo09:59
openstackgerritMerged openstack/diskimage-builder master: Don't only install python3-virtualenv  https://review.openstack.org/55987110:00
shardyjaosorior: you mean a service container to restart every update, or a bootstrap task that always runs?10:01
rtailorowalsh, was trying for OSP10, it didn't work. Don't know much about image building stuff.10:02
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: install collectd on undercloud  https://review.openstack.org/56004810:02
shardyjaosorior: I think in both cases you'd wire in DeployIdentifier to the container environment, similar to how we add TRIPLEO_CONFIG_HASH10:02
jaosoriorshardy: is there an example around?10:04
jaosoriorshardy: thing is, I would like a bootstrap container to run every time there's an overcloud deploy10:05
jaosoriordciabrin: around?10:05
dciabrinjaosorior, yes10:05
jaosoriordciabrin: seems that mysql_init_bundle only gets ran once in the deployment, is that intended?10:05
dciabrinjaosorior, it runs everytime iiuc, but puppet-pacemaker doesn't change the resource once it's been created10:06
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Include extra_params option in reproducer-quickstart.sh.  https://review.openstack.org/54693210:06
dciabrinjaosorior, it's a regular paunch container, i don't see why it would not re-run every time :/10:06
jaosoriordciabrin: I've noticed it doesn't. It only runs on the first deployment10:07
jaosoriordciabrin: cause the configuration doesn't change10:07
jaosoriordciabrin: so if the configuration of that container stays the same, paunch doesn't need to re-run it10:07
jaosoriordciabrin: which is why I asked shardy how do we get to run a container every deployment10:07
dciabrinjaosorior, oh, makes sense10:07
jaosoriorso... the issue I'm having is that you cannot change mysql passwords :/10:07
jaosoriorbecause that container is the one that does the puppet deployment, and it only runs once10:08
dciabrinjaosorior, but the password is generated via docker-puppet, so the config is still changed on the host. next time you restart the pacemaker container it will be picked up10:09
dciabrinjaosorior, granted, currently nothing restarts pacemaker resource if config changes.10:09
jaosoriordciabrin: but docker-puppet is called manually, instead of using the docker_puppet_tasks section10:10
*** skramaja has quit IRC10:10
dciabrinjaosorior, the only automatic way of restarting it is to rely on the fact that minor updates restart the nodes10:10
*** skramaja_ has joined #tripleo10:10
dciabrinjaosorior, err no. we run puppet manually in docker-init-bundle, but this is different from docker-puppet, which is run during the deploy steps, before executing step1, step2...10:10
dciabrinjaosorior, did I get your point correctly?10:11
jaosoriordciabrin: https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/database/mysql.yaml#L24310:11
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/pike: [DNM] testing tempest on undercloud for pike  https://review.openstack.org/56035110:11
dciabrinjaosorior, right that one doesn't (re)generate the mysql config10:12
dciabrinjaosorior, it only creates a resource in pacemaker10:12
jaosoriordciabrin: it doesn't10:12
jaosoriordciabrin: it runs docker_puppet_apploy.sh manually10:12
openstackgerritMerged openstack/diskimage-builder master: Set the dhclient timeout to match DIB_DHCP_TIMEOUT  https://review.openstack.org/55908410:12
dciabrinjaosorior, which is different from https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/database/mysql.yaml#L12010:13
jaosoriordciabrin: correct10:13
dciabrinthat one ^^^ is executed whenever to do an overcloud deploy, so config should get regerated10:13
jaosoriordciabrin: right10:13
jaosoriorthat's the config10:13
*** psahoo has quit IRC10:13
jaosoriornot the users and passwords10:13
jaosoriorthe config stuff works just fine10:13
dciabrinoh10:13
*** skramaja_ is now known as skramaja10:13
jaosoriorthis https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/database/mysql.yaml#L126 adds the users and passwords to be noops10:14
jaosoriorso that doesn't get executed there10:14
jaosoriorthe container I pointed at does that10:14
jaosoriorand it does so by running the script manually10:14
jaosoriorhere https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/database/mysql.yaml#L25010:14
*** nyechiel has quit IRC10:15
*** nyechiel_ has joined #tripleo10:15
dciabrinjaosorior, ok i see. the side effect of that container is that it creates the mysql user and/or changes the password if it's (re)-executed?10:15
jaosoriordciabrin: correct10:16
dciabrinjaosorior, damned, got you.. :/10:16
jaosoriordciabrin: I think the solution is to move that to use this syntax https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/keystone.yaml#L20310:16
shardysorry got distracted, by design paunch doesn't restart containers unless the json definition changes10:17
shardythat is why we add TRIPLEO_CONFIG_HASH when the configuration changes in the bind mounts10:17
shardyotherwise every update would restart every container10:17
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/ocata: [DNM] testing tempest on undercloud for Ocata  https://review.openstack.org/56035210:17
dciabrinjaosorior, hmm we can't use that, we tried during osp12 and it failed with multi-node. I don't remember the reason right now, but we moved to the manual call to puppet on purpose10:17
shardyas I mentioned passing some other salt like the DeployParameter timestamp could be a workaround10:18
shardyto force the config to change every update10:18
*** cdearborn has joined #tripleo10:18
*** avivgt has joined #tripleo10:19
jaosoriorshardy: I don't understand what that means :(10:19
shardyjaosorior: what part?10:19
jaosoriorshardy: "as I mentioned passing some other salt like the DeployParameter timestamp could be a workaround"10:19
jaosoriorI grep that in t-h-t and don't find an occurence10:19
*** cshastri has quit IRC10:19
*** jaganathan has quit IRC10:20
shardyjaosorior: sorry it's DeployIdentifier10:20
shardyit changes every update, it's a timestamp10:20
*** gfidente has joined #tripleo10:20
*** gfidente has quit IRC10:20
*** gfidente has joined #tripleo10:20
jaosoriorI see10:20
dciabrinjaosorior, this sounds like a bigger problem than mysql though. IIUC, the mysql user password is not part of any config file.10:20
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/ocata: [DNM] testing tempest on undercloud for Ocata  https://review.openstack.org/56035210:20
jaosoriordciabrin: I think we've got a solution10:21
shardyif you have some config which hasn't actually changed, but you want to fake that it always changes, you could include that timestamp in the configuration, e.g the container environment or whatever10:21
shardyeven if the timestamp is ignored10:21
jaosoriorshardy: right, like it's done in the nova-api.yaml10:21
jaosoriorshardy: got it, I'll give it a try10:21
dciabrinjaosorior, how would we tell other container to restart if10:21
dciabrinwoops bad cut'n'paste10:21
dciabrinshardy, makes sense10:21
shardyjaosorior: ack yeah exactly like nova-api.yaml10:21
jaosoriordciabrin: I'll try it out, test it out, raise a bug and put up a fix.10:22
dciabrinjaosorior, ok thx10:22
dciabrinbandini ^^^ fyi10:22
* bandini scrolls back10:22
shardyjaosorior: if the thing that's changing is the password, could you just add that to the environment?10:22
shardyvs doing it every update?10:23
jaosoriorshardy: it applies for every password for every openstack service10:23
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/newton: [DNM] testing tempest on undercloud for Newton  https://review.openstack.org/56035310:23
jaosoriorshardy: we can't rotate passwords currently10:23
shardyjaosorior: ack, but I guess we still only want to mess with users when that password changes, vs running some bootstrap task unconditionally?10:23
shardyI suppose the bootstrap task can do nothing when the password is unchanged10:23
*** nyechiel has joined #tripleo10:24
shardyor re-set the same password10:24
jaosoriorshardy: that's all done here https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/database/mysql.pp#L16310:24
*** nyechiel_ has quit IRC10:24
shardybut my concern is it's a pattern which could cause e.g unneccesary service restarts10:24
shardyjaosorior: ack, Ok maybe DeployIdentifier is enough in this case then10:24
jaosoriorshardy: it won't restart mysql10:25
*** rajinir has quit IRC10:25
*** zoli is now known as zoli|lunch10:29
jaosoriorshardy: I'll give that a try, thanks for the help!10:32
bogdandofolks, when composing networks, What is relation of InternalApi* to Control(Plane)*10:32
bogdandothinking of all-in-one coud10:32
bogdandoshall they merge?10:33
bogdandoI'm trying to use a flat model, but it deploys the same IPs for haproxy frontends and backends :(10:33
* bogdando sigh10:34
jaosoriorbogdando: you probably only need one network10:34
jaosoriorbogdando: but HAProxy still needs to have separate passwords10:34
bogdandoso I'm gonna add environments/fixed-ip-vips.yaml and override all those InternalApi/PublicVirtual/Storage*/Tenant/Control* things10:34
jaosorior* IPs10:35
jaosorior(thinking about passwords right now)10:35
bogdandoright, I cant get that tht language to explain that to my UC tho! :)10:35
*** nyechiel_ has joined #tripleo10:35
jaosoriorbogdando: IIRC, dprince had addressed that somehow... would recommend asking him10:35
*** cdearborn has quit IRC10:35
shardybogdando: the vip is the frontend, and the backend is the bind IP for the service10:36
bogdandowell , one network would work for me as well, thou I wanted to learn composable networks as well10:36
shardyso overriding the vip may be your problem10:36
*** nyechiel has quit IRC10:36
bogdandohere is my messy config WIP folks10:36
shardybogdando: you could use a special network_data.yaml that has only one network in it10:36
shardyto simplify the generated templates10:36
*** shardy is now known as shardy_afk10:36
Tengugfidente: hello! I can confirm: the tag you gave me is OK for ceph containers on pike !10:36
bogdandohttps://github.com/bogdando/oooq-warp/commit/98a5a1c94bf14a3a49acc00d10a8bf18662774f0 shardy_afk jaosorior10:37
openstackgerrityatin proposed openstack/tripleo-heat-templates master: Set ulimit for nova-compute and cinder-volume  https://review.openstack.org/56035910:37
*** pkovar has quit IRC10:38
bogdandoso I'm trying it all-composed multiple networks sitting flat way10:38
bogdandothough would like to know more of that "you could use a special network_data.yaml that has only one network in it" option10:39
bogdandothe issue is that I'm not sure how to end up with network config creating br-ex and br-ctlplane10:39
*** moshele has joined #tripleo10:45
bogdandoshardy_afk: and the role data https://github.com/bogdando/oooq-warp/blob/master/tht/roles/UndercloudAIOOpenshift.yaml10:45
*** ccamacho has quit IRC10:46
*** ccamacho has joined #tripleo10:48
openstackgerritMike Fedosin proposed openstack/tripleo-common master: Optimize mistral baremetal workflows  https://review.openstack.org/56036710:55
openstackgerritbinhong.hua proposed openstack/tripleo-puppet-elements master: Remove heat-api-cloudwatch from package install  https://review.openstack.org/56036810:55
*** anande has quit IRC10:57
gfidenteTengu hey, good :D10:57
d0ugalmfedosin: Nice patch! ^10:58
mfedosind0ugal: thanks :) I hope tests will pass10:59
d0ugalmfedosin: me too :)10:59
d0ugalmfedosin: it is hard to concentrate to review changes like this, but I'll try and do it today11:00
d0ugalor I find it hard at least11:00
shardy_afkbogdando: I think you'll still need the ctlplane network unless we rework the templates, but the network_data could have only e.g the InternalApi network in it11:04
*** shardy_afk is now known as shardy11:04
shardyI expect even in the all-in-one case you may want to separate external and internal traffic on different nics/vlans though11:04
*** pkovar has joined #tripleo11:06
*** dprince has joined #tripleo11:07
*** ansmith has quit IRC11:11
openstackgerritNoam Angel proposed openstack/tripleo-heat-templates master: Fix few issues with deploying with baremetal-services.yaml  https://review.openstack.org/56026111:16
gfidentejistr you happen to have some time to help me figure what is wrong with the external_deploy_task for ceph?11:17
gfidentejistr basically it looks to me that both scenarios passed the ceph install https://review.openstack.org/#/c/546966/11:18
gfidentejistr but then somehow both failed in the same way during following step 2 ansible tasks11:18
*** abishop has joined #tripleo11:18
gfidentejistr and I couldn't find a simple error pointing to the actual issue11:18
jistrgfidente: i happen to have negative amount of time for at least the past 2 weeks but yes i'll help :D11:18
gfidentejistr who can help?11:19
jistrgfidente: just gimme some time to kick off a test of update11:19
gfidentejistr ok thanks, believe me I couldn't figure *what* is failing11:19
gfidentejistr finding that would be a good start11:19
gfidentejistr++11:20
*** panda|off is now known as panda|lunch11:21
*** assassin has joined #tripleo11:26
mfedosinshardy: hi! I'm reviewing your patch https://review.openstack.org/#/c/560312/111:28
mfedosineverything is fine there...11:28
mfedosinBut the code of the action makes me cry11:28
mfedosinand if I had the opportunity to rewrite it from scratch ...11:29
mfedosindo you mind if I take your patch and finish it up to a reasonable state?11:29
mfedosinor I can list or shortcomings there11:31
*** nyechiel_ has quit IRC11:31
ykarelowalsh, hi11:31
owalshykarel: hi11:32
ykarelowalsh, i am setting ulimit for nova compute, can you look: https://review.openstack.org/#/c/560359/11:32
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Added tempest skip list for undercloud for stable branches  https://review.openstack.org/56037411:32
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Verify the Swift container exists with a small utility workflow  https://review.openstack.org/52821311:34
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart master: Run tempest on undercloud on fs003  https://review.openstack.org/55452211:34
shardymfedosin: yeah the code is not great, feel free to push a follow-up which rewrites it if you like11:35
owalshykarel: this is why yea https://bugs.launchpad.net/oslo.rootwrap/+bug/1760471/comments/111:35
openstackLaunchpad bug 1760471 in oslo.rootwrap "oslo-rootwrap-daemon performing badly in docker containers(centos/fedora)" [Undecided,Confirmed]11:35
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/queens: [DNM] testing tempest on undercloud for queens  https://review.openstack.org/56034311:35
mfedosinshardy: thanks!11:35
ykarelowalsh, yes11:36
owalshykarel: and the ssh timeouts too???11:36
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/pike: [DNM] testing tempest on undercloud for pike  https://review.openstack.org/56035111:36
ykarelowalsh, yes that was because of neutron l3 agents11:36
*** zoli|lunch is now known as zoli11:36
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/ocata: [DNM] testing tempest on undercloud for Ocata  https://review.openstack.org/56035211:36
openstackgerritChandan Kumar proposed openstack/tripleo-heat-templates stable/newton: [DNM] testing tempest on undercloud for Newton  https://review.openstack.org/56035311:37
owalshykarel: wow, didn't realise subprocess did this11:37
ykarelowalsh, yes subprocess with python2 has the problem, with python3 doesn't have this issue as victor said11:38
jaosoriorshardy: could you review this backport https://review.openstack.org/#/c/560322/ ?11:41
*** suuuper has quit IRC11:42
shardyjaosorior: ack will do11:42
*** rfolco|off is now known as rfolco|rover11:44
*** jpena is now known as jpena|lunch11:46
EmilienMo/11:48
jaosoriorEmilienM: sup dude, how's it going?11:51
*** morazi has joined #tripleo11:52
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Always run mysql init bundle  https://review.openstack.org/56038711:52
jaosoriordciabrin: that fixed the issue ^^11:52
EmilienMjaosorior: tried to re-enable containerized undercloud on fs010 for pure testing and now it fails on CREATE_FAILED  Resource CREATE failed: Error in EndpointMap output endpoint_map: Invalid URL port "" for make_url called with {u'host': u'', u'scheme': '', u'port': ''} -11:52
EmilienMhttp://logs.openstack.org/64/560264/2/check/tripleo-ci-centos-7-containers-multinode/0671d02/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-04-11_06_18_5511:52
dciabrinjaosorior, ack thx very much!11:54
*** gkadam has quit IRC11:54
jaosoriorEmilienM: any idea where was that invalid call?11:54
jaosoriorEmilienM: I can check it out after the Security Squad meeting.11:55
EmilienMjaosorior: nah, no worries. I just saw it, I'll investigate11:55
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix switching services-docker to services  https://review.openstack.org/55993011:56
openstackgerritMerged openstack/tripleo-heat-templates master: Delete not-used services-docker files  https://review.openstack.org/55679411:56
openstackgerritMerged openstack/tripleo-heat-templates master: Removed unnecessary services from the LiquidioCompute role.  https://review.openstack.org/55969611:56
jaosoriorEmilienM: another option is that an invalid endpoint map was given somehow as a parameter.11:57
*** atoth has joined #tripleo11:58
jaosoriorshouldn't be hard to figure it out. Lets check the command that was used.11:58
*** raildo has joined #tripleo11:58
*** leanderthal has joined #tripleo11:58
*** gkadam has joined #tripleo11:59
jaosoriorEmilienM: /usr/share/openstack-tripleo-heat-templates/environments/no-tls-endpoints-public-ip.yaml was used.11:59
anilvenkataEmilienM, shardy can you please review this https://review.openstack.org/#/c/559806/12:00
jaosoriorEmilienM: seems it went out of sync12:00
anilvenkataEmilienM, shardy it fixing a typo but important for scheduling HA routers12:00
*** amoralej is now known as amoralej|lunch12:01
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add designate to the non-TLS environment  https://review.openstack.org/56039612:01
jaosoriorEmilienM: ^^12:01
*** dsariel has quit IRC12:01
jaosorioralee, lhinds, moguimar, raildo, owalsh: around?12:01
moguimaro/12:01
owalsho/12:01
jaosorior#startmeeting TripleO Security Squad12:02
openstackMeeting started Wed Apr 11 12:02:30 2018 UTC and is due to finish in 60 minutes.  The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot.12:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.12:02
*** openstack changes topic to " (Meeting topic: TripleO Security Squad)"12:02
openstackThe meeting name has been set to 'tripleo_security_squad'12:02
jaosoriorGonna wait a couple of minutes for more folks to log in./12:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart master: Revert "Revert "Deploy container-multinode (fs010) with a containerized undercloud""  https://review.openstack.org/56026412:03
jaosoriorAlright12:05
jaosorior#topic Work progress update12:05
*** openstack changes topic to "Work progress update (Meeting topic: TripleO Security Squad)"12:05
*** nyechiel_ has joined #tripleo12:05
jaosoriorSame as every week, this is just a placeholder to have folks know how the work is going for the different topics.12:05
jaosorior#topic Public TLS by default12:05
*** openstack changes topic to "Public TLS by default (Meeting topic: TripleO Security Squad)"12:05
jaosoriorThis is the one I've been the most active in.12:06
raildoo/12:06
jaosoriorI've got some working patches, and most of the stuff in the chain has already merged12:06
jaosoriorI still need to write proper documentation about that12:06
jaosoriorsince that change does introduce some changes to our defaults (besides the fact that it uses TLS)12:06
*** jcoufal has joined #tripleo12:07
jistrgfidente: looking at the ceph issue now12:07
jaosoriorfor one, the default will be to use FQDNs instead of IPs for the public endpoints.12:07
jaosoriorSo, the user/deployer will need to add those to DNS or /etc/hosts after the deployment12:07
jaosoriorgotta see if we can automate that though12:08
jaosoriorat least the /etc/hosts part12:08
jaosoriorshouldn't be too hard.12:08
jaosoriorAgain, reviews are very much welcome, and the patches are in the etherpad12:08
jaosorior#link https://etherpad.openstack.org/p/tripleo-security-squad12:08
jaosoriorI've also been using the same topic for the patches12:08
jaosorior#link https://review.openstack.org/#/q/topic:public-tls-default+(status:open+OR+status:merged)12:08
*** pdeore has quit IRC12:09
jaosoriorAny questions/feedback?12:09
bogdandojaosorior, shardy: can I debug network data in hiera somehow? I still keep getting the same frontend and backend IPs for haproxy config, and my admin/public VIPs seem ignored12:10
bogdandosorry, it's a meeting here...12:10
jaosoriorbogdando: no worries. I can check that out after the meeting12:10
jaosorior#topic Secret Management for TripleO12:10
*** openstack changes topic to "Secret Management for TripleO (Meeting topic: TripleO Security Squad)"12:10
*** thrash|g0ne is now known as thrash12:10
jaosoriorSo, we started taking the first steps on this task12:11
jaosoriorWe have identified all the potentially sensitive data in this etherpad:12:11
jaosorior#link https://etherpad.openstack.org/p/tripleo-audit-secrets12:11
jaosoriorso, if you are curious about what we identified and how we plan to address it, that's the place to look :)12:11
jaosoriorupon the first stuff to cover is the undercloud's swift12:12
jaosoriorwe plan to use Swift encryption (at rest) and hopefully enable it as a default12:12
jaosoriorthere are several ways of doing that though.12:12
*** eck`gone is now known as eck`12:12
*** liverpooler has joined #tripleo12:12
jaosoriorOne is to have a pre-shared key (or encryption root secret) in the swift configuration12:13
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Update environment files for Q upgrade and ffwd upgrade  https://review.openstack.org/55906112:13
jaosoriorthat will then be used to encrypt the swift containers12:13
jaosoriorThe other, is to use something else (such as barbican)12:13
jaosoriorAnd apparently we already have support for the former one in t-h-t; so that could be a good fit for a feature for the containerized undercloud.12:13
openstackgerritLukas Bezdicka proposed openstack/tripleo-common master: Unite default parameter_resource name  https://review.openstack.org/56011012:14
jaosoriorof course, ideally the best would be that the default configuration wouldn't need to use barbican. We gotta see waht's best to implement12:14
jaosoriorIf folks want to join that effort, help is very appreciated.12:14
moguimarwatching the project update of swift yesterday12:15
moguimarhttps://www.youtube.com/watch?v=rnAtnnE0sQM&feature=youtu.be12:15
moguimarthey mentioned that swift can run stand alone12:15
*** panda|lunch is now known as panda12:16
jaosoriormoguimar: yeah, that's a feature they have. Although we don't really need it as we do have a proper openstack installation on the undercloud.12:16
moguimartrying to get a default option aligned to that would be more popular12:17
jaosoriormoguimar: did you have something in mind regarding the standalone feature?12:17
moguimarnot yet12:17
jaosoriormoguimar: so, we already deploy and use swift by default. We use it to store the overcloud deployment "plan". The plan is merely the set of rendered heat templates, parameters and options to deploy the overcloud.12:17
jaosoriorso, the plan would contain passwords and SSL keys in the form of heat parameters that would be used to deploy the overclod.12:18
jaosorior*overcloud12:18
jaosoriormoguimar: the plan would be to add the encryption option as a default for the swift deployment in the undercloud that we already do.12:19
jistrgfidente: hmm interesting indeed, it looks like it fails in step 1 of docker-puppet.py but i'm having trouble to spot any indication of root cause in the logs12:19
jistrhttp://logs.openstack.org/66/546966/28/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/130db82/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz12:19
jaosoriorNow, the tricky thing are updates/upgrades.12:20
anilvenkataEmilienM, thanks Emilien12:20
jaosoriorThe encryption docs say that only new swift containers would be encrypted. Which would require the deployer to download the plan, delete it, and upload it again.12:20
jaosoriorBut I think that's a reasonable expectation if folks need that feature.12:20
openstackgerritLukas Bezdicka proposed openstack/tripleo-common master: Unite default parameter_resource name  https://review.openstack.org/56011012:21
jaosoriorAnyway, that's all the update from my side. There is still a bunch of research and a POC to do.12:21
jaosoriorany questions.feedback?12:21
jaosoriorquestions/feedback12:21
openstackgerritMike Fedosin proposed openstack/tripleo-common master: UpdatePlanEnvironmentAction enable overwrite or merge  https://review.openstack.org/56031212:21
moguimarI got through the oslo.config docs12:22
moguimarnow I'm watching the project updates in the openstack website to get familiarized with all the projects I heard so far in our discussions12:23
jaosoriormoguimar: if you have questions about any project, reach out, we're always glad to help12:23
moguimarsure12:23
jaosoriorAlright12:24
jaosoriorthat's all for the work updates12:24
jaosorior#topic Migration to Storyboard12:24
*** openstack changes topic to "Migration to Storyboard (Meeting topic: TripleO Security Squad)"12:24
jaosoriorSo... Kendall and EmilienM reached out recently proposing that the Security Squad try out using Storyboard12:25
jaosoriorFor anybody interested12:25
jaosoriorhere's the link12:25
jaosorior#link https://storyboard.openstack.org/12:25
jaosoriorSo, we could track the work progress and user stories there, as well as bugs and other work done by the squad12:26
*** quiquell is now known as quiquell|lunch12:26
EmilienM++12:26
jaosoriorwhich would effectively replace our etherpad which is getting quite filled up12:26
moguimar+112:26
jaosoriorI for one, am quite keen on giving it a try12:26
aleejaosorior, we dont actually have any launchpad items so there is no migration12:26
EmilienMfeel free to reach her on #storyboard, she's diablo_rojo12:26
jaosorioralee: exactly12:26
moguimarscrum feelings12:26
aleeits just a matter of using it12:26
raildoI'm fine with storyboard as well :)12:26
jaosoriorDoes someone have any reservations or comments against it?12:27
aleeand if everyone else is doing so, why not?12:27
aleeI plan to migrate barbican to it next week12:27
jaosorioralee: nice!12:27
jaosorioralright12:27
jaosorior#action TripleO Security Squad will start tracking the projects we're working on in Storyboard12:27
*** nyechiel_ has quit IRC12:28
jaosoriorI still need to give it a better read to know more features that it has, but it seems quite promising12:28
aleejaosorior, has the rest of tripleo migrated too?12:28
jaosoriorEmilienM: ^^12:28
aleedoes it matter?12:28
jaosorioralee: I don't think it matters :)12:28
jaosoriorand I don't know if any other squad from tripleo has migrated12:29
aleejaosorior, fair enough -- it just may get tricky if we have references to launchpad issues12:29
jaosoriorwe do have some12:29
jaosoriorso, we'll need to figure out how that works12:29
aleefor work thatwe depend on that is outside of the security squad12:29
EmilienMonly UI and validations12:29
EmilienMhave migrated12:29
*** rlandy has joined #tripleo12:30
aleeok - so there is precedent then -- no objections from me12:30
jaosorioralright12:30
jaosoriorlets migrate then! :D12:30
EmilienMjtomasek, jrist, honza, florianf, jpich (and others): if you have anything to share on storyboard migration so far12:30
*** eck` is now known as eck`gone12:31
*** nyechiel_ has joined #tripleo12:33
jaosoriorwell, hopefully they'll reach out when they have time. And now we know who to poke if we have some questions regarding how to link Storyboard to our existing TripleO bugs.12:33
jaosorior#topic Any other business12:35
*** openstack changes topic to "Any other business (Meeting topic: TripleO Security Squad)"12:35
jaosoriorDoes anybody have something to bring up to the group?12:35
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Install additional roles for quickstart  https://review.openstack.org/55878612:35
honzaEmilienM: not at the moment12:35
*** liverpooler has quit IRC12:37
*** liverpooler has joined #tripleo12:37
jaosoriorAlright folks12:38
jaosoriorthanks for joining12:38
jaosorior#endmeeting12:38
*** openstack changes topic to "Welcome to Rocky. CI status GREEN but Third Party queue is big. ಠ_ಠ recheck your patches if failed | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"12:38
openstackMeeting ended Wed Apr 11 12:38:20 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)12:38
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tripleo_security_squad/2018/tripleo_security_squad.2018-04-11-12.02.html12:38
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tripleo_security_squad/2018/tripleo_security_squad.2018-04-11-12.02.txt12:38
moguimaro/12:38
openstackLog:            http://eavesdrop.openstack.org/meetings/tripleo_security_squad/2018/tripleo_security_squad.2018-04-11-12.02.log.html12:38
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Install additional roles for quickstart  https://review.openstack.org/55878612:38
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Verify the Swift container exists with a small utility workflow  https://review.openstack.org/52821312:39
florianfEmilienM: Haven't really used it yet. Will know more when validations are migrated!12:39
jistrgfidente: got it12:40
jistrgfidente: it's message size limit in zaqar12:41
gfidenteceph-ansible running with -v12:41
jistrthat's why there's no message informing what went wrong :D12:41
jistrhttp://logs.openstack.org/66/546966/28/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/130db82/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2018-04-11_02_02_14_52512:41
jistrit failed the Mistral workflow which runs Ansible, but Ansible itself could have run totally fine perhaps...12:41
*** pdeore has joined #tripleo12:42
jistrjust gave too big output for the Mistral/Zaqar combo to be able to crunch it12:44
*** quiquell|lunch is now known as quiquell12:46
openstackgerritMike Fedosin proposed openstack/tripleo-common master: Optimize mistral baremetal workflows  https://review.openstack.org/56036712:46
*** salmankhan has quit IRC12:49
*** salmankhan has joined #tripleo12:50
*** jpena|lunch is now known as jpena12:50
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Manage upgrades to a containerized undercloud  https://review.openstack.org/54962412:52
chemjistr: hi, we should land https://review.openstack.org/#/c/559755/ the update is success \o/ and the upgrade fails after .. 3h40 which is longer than ever seen before ... and this is master . omg12:54
*** pchavva has joined #tripleo12:54
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Install changes in additional roles for quickstart  https://review.openstack.org/55878612:54
*** salmankhan has quit IRC12:54
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-upgrade stable/queens: Pass env-files and roles-data file to update/upgrade prepare cli.  https://review.openstack.org/56040412:55
chemjistr: oups sorry, didn't see that we didn't yet land https://review.openstack.org/#/c/559746/12:55
*** ccamacho is now known as ccamacho|lunch12:57
*** ansmith has joined #tripleo12:57
jistrchem: great! let's land them both then! :)12:57
*** toure|gone is now known as toure12:58
chemjistr: I still cannot read the "new log" which is a mixup of dump on one line huge ansible playbook and mistral execution12:59
chemjistr: (fun)12:59
jistrmarios: please consider if you can bump +1->+2 https://review.openstack.org/#/c/55974612:59
jistrmarios: we got green minor update at least12:59
*** gkadam_ has joined #tripleo12:59
chemjistr: so don't know what happen during those 1h30 where the controller upgrade run ... 1h30, man, that's pretty long13:00
chemmarios: and a failure after  3h40 in upgrade (this is master !!)13:00
*** amoralej|lunch is now known as amoralej13:01
chemjistr: marios those mistral logs ... are not friendly (user, dev, whoever)13:01
jistrchem: which one do you mean? (link please?)13:01
chemjistr: https://logs.rdoproject.org/55/559755/6/openstack-check/gate-tripleo-ci-centos-7-container-to-container-upgrades-master-nv/Z36e3f3674069460284522e7397f65ec5/undercloud/home/jenkins/overcloud_upgrade_run_Controller.log.txt.gz13:02
chemjistr: master upgrade13:02
chemon controller13:02
*** gkadam has quit IRC13:02
jistrah yea13:03
*** dsariel has joined #tripleo13:04
*** thrash is now known as thrash|brb13:04
*** psachin has quit IRC13:05
*** rtailor has quit IRC13:07
*** eck`gone is now known as eck`13:08
*** gkadam_ has quit IRC13:08
*** zoli is now known as zoli|afk-TPB13:09
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: containerized-undercloud: don't manage stackrc  https://review.openstack.org/55981813:09
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates master: Upgrade: make bundles use new container image name after upgrade  https://review.openstack.org/56040813:10
*** jmelvin has joined #tripleo13:12
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: DNM - Test fs027 with --debug  https://review.openstack.org/55917913:12
*** cshastri has joined #tripleo13:13
*** Goneri has joined #tripleo13:14
*** salmankhan has joined #tripleo13:16
*** chlong has joined #tripleo13:17
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Reduce reproducer redudancy  https://review.openstack.org/55836913:18
verdurinI'm testing current quickstart.sh and the undercloud VM is hanging on startup, meaning no IP and the script fails13:20
rascamwhahaha, hey, so I'm near to complete this https://review.openstack.org/#/c/555832/ just the upstream repo is missing. Now, we are actually working on this https://github.com/redhat-openstack/tripleo-quickstart-utils but it started as a fork of tripleo-quickstart-extras. Is it ok in any case?13:21
*** radeks has quit IRC13:21
*** radeks has joined #tripleo13:22
*** dhill_ has joined #tripleo13:22
*** dsariel has quit IRC13:22
*** cdearborn has joined #tripleo13:22
*** mcornea has joined #tripleo13:23
*** cshastri has quit IRC13:26
jistrwe'll likely need to merge the backport too as the Depends-On waits for everything proposed, disregarding branch info https://review.openstack.org/#/c/559753/13:26
jistrchem: ^ we probably shouldn't +A that one until the master lands though13:26
*** dhill_ has quit IRC13:26
*** dhill_ has joined #tripleo13:26
*** thrash|brb is now known as thrash13:28
EmilienMdprince: have you ever tried containerized undercloud when heat has convergence enabled? it's failing consistently on https://review.openstack.org/#/c/558354/13:29
*** etingof has quit IRC13:29
*** yprokule has joined #tripleo13:29
chandankumarsshnaidm: https://review.openstack.org/#/c/558510/ is failing13:30
chandankumarhttp://logs.openstack.org/10/558510/6/check/tripleo-ci-centos-7-scenario007-multinode-oooq-container/e4c8f62/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-04-11_10_53_5813:30
chandankumaris it a known issue?13:30
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Remove nat/masquerading config workarounds.  https://review.openstack.org/55785113:30
*** skramaja has quit IRC13:30
chandankumarmore tweaking13:31
*** ansmith has quit IRC13:32
jaosoriorchandankumar: in tripleo-ci-centos-7-scenario001-multinode-oooq-container   is tempest ran from a container?13:32
*** pdeore is now known as pdeore|afk13:33
chandankumarjaosorior: tripleo-ci-centos-7-undercloud-containers is only running tempest from container13:34
chandankumarwhich is fs02713:34
*** pblaho has quit IRC13:35
chandankumarjaosorior: do we want to switch there also?13:35
chandankumarEmilienM: Hello13:35
EmilienMchandankumar: what's up?13:36
chandankumarEmilienM: since fs027 is running tempest from containers I am thinking to use this job against upstream tempest13:36
jaosoriorok13:36
jaosoriorchandankumar: just wondering13:36
EmilienMchandankumar: how tempest is deployed, in a container?13:36
EmilienMchandankumar: if yes, it's useless to run this job in tempest because we have no mechanism to update tempest code when you submit a patch in tempest, since we deploy built containers from RDO. You would have to deploy tempest from source for that project.13:37
*** tzumainn has joined #tripleo13:37
chandankumarEmilienM: tempest and its plugin is installed there from packages13:37
*** suuuper has joined #tripleo13:38
chandankumarEmilienM: that will be doable13:38
EmilienMgo for it then13:38
chandankumarEmilienM: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/validate-tempest/defaults/main.yml#L2713:38
*** suuuper has quit IRC13:38
*** ansmith has joined #tripleo13:38
chandankumarEmilienM: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/validate-tempest/templates/run-tempest.sh.j2#L4313:39
chandankumarI just need to flip the switch13:39
moguimarEmilienM: stories in the storyboard must reference a project?13:39
chandankumarEmilienM: I have also an AI from summit to add kolla-anisble gating against tempest13:39
moguimarand if so, do you know how can I create a project?13:39
EmilienMmoguimar: I think it's a good question for #storyboard channel13:41
moguimaryep, just posted there13:44
moguimarthanks EmilienM13:44
verdurinIs there a way of debugging the undercloud VM, or forcing it to be rebuild in quickstart?13:46
verdurinrebuilt13:46
openstackgerritLukas Bezdicka proposed openstack/python-tripleoclient master: Stop persisting previous configuration on update/upgrade prepare  https://review.openstack.org/55974613:46
openstackgerritLukas Bezdicka proposed openstack/python-tripleoclient master: Add upgrade-[prepare|converge].yaml into upgrade cli  https://review.openstack.org/55886613:46
openstackgerritLukas Bezdicka proposed openstack/python-tripleoclient master: Update prepare using an environment file  https://review.openstack.org/55976013:46
openstackgerritLukas Bezdicka proposed openstack/python-tripleoclient master: WIP - ffwd-upgrade cli adds ffwd-upgrade [prepare|run|converge]  https://review.openstack.org/55793713:46
openstackgerritLukas Bezdicka proposed openstack/python-tripleoclient master: WIP - Introduce ceph upgrade command  https://review.openstack.org/56042313:46
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Added tempest skip list for undercloud for stable branches  https://review.openstack.org/56037413:47
chandankumararxcruz|ruck: ^^ after this review, I will update the skip list13:48
*** fzdarsky_ is now known as fzdarsky13:48
arxcruz|ruckchandankumar: k13:48
mwhahaharasca: yea just import what you have13:48
openstackgerritMarios Andreou proposed openstack/python-tripleoclient master: WIP - ffwd-upgrade cli adds ffwd-upgrade [prepare|run|converge]  https://review.openstack.org/55793713:48
openstackgerritDamien Ciabrini proposed openstack/tripleo-heat-templates stable/queens: Upgrade: make bundles use new container image name after upgrade  https://review.openstack.org/56042613:50
*** udesale has quit IRC13:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Handle undercloud upgrades via host_prep_tasks  https://review.openstack.org/55921013:51
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Manage upgrades to a containerized undercloud  https://review.openstack.org/54962413:52
EmilienMsocial, bogdando: I updated the path and explained in the commit message why we use host_prep_tasks13:52
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: Re-use undercloud upgrades tasks.  https://review.openstack.org/55230313:54
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: WIP - containerized undercloud upgrade  https://review.openstack.org/55362913:55
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: WIP - containerized undercloud upgrade  https://review.openstack.org/55362913:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-common master: TLS by default for the overcloud  https://review.openstack.org/55492613:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Switch public endpoints to use FQDNs by default  https://review.openstack.org/55992613:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Change default endpoint map entries to use TLS  https://review.openstack.org/55842413:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove empty default for SSLKey  https://review.openstack.org/55927913:56
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add CACerts service to all scenario environment files  https://review.openstack.org/56043013:56
dprinceEmilienM: yes, and it failed for me as well13:56
EmilienMdprince: interesting13:57
dprinceEmilienM: keep in mind I started this when convergence was still not our default13:57
EmilienMoh ok13:57
dprinceEmilienM: more work to be done, I think it can be fixed to work13:57
EmilienMI should try again in my env, because upstream CI doesn't give me logs, maybe the machine is down13:57
EmilienMdprince: i aligned the heat config from instack into THT, but same results13:58
EmilienMwill dig13:58
*** links has quit IRC13:58
*** etingof has joined #tripleo13:58
dprinceEmilienM: yeah, its been like a year since I've tried convergence with the undercloud_deploy command13:58
EmilienMok13:59
EmilienMdprince: hey, would you mind look at https://review.openstack.org/#/c/558893/ and maybe move forward?13:59
EmilienMor let me know if you want to change something..13:59
*** udesale has joined #tripleo14:00
*** jistr is now known as jistr|mtg14:02
*** ccamacho|lunch is now known as ccamacho14:02
dprinceEmilienM: its good +A14:04
EmilienMthx!14:04
dprinceEmilienM: np, thank you14:07
*** pkovar has quit IRC14:13
openstackgerritDaniel Alvarez proposed openstack/tripleo-heat-templates master: Adding missing OVN Metadata service to DVR environments  https://review.openstack.org/56043114:13
dalvareznumans: ^14:13
numansdalvarez, thanks.14:13
dalvareznumans: could you pls add the reviewers you feel are relevant for getting this in plz? :)14:14
dalvarezthanks a lot!!14:14
numansdalvarez, ack14:14
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Standardise Workflow messaging and optionally persist messages  https://review.openstack.org/42506014:16
*** dsariel has joined #tripleo14:17
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Verify the Swift container exists with a small utility workflow  https://review.openstack.org/52821314:18
*** jobewan has joined #tripleo14:19
*** agopi has joined #tripleo14:22
*** lblanchard has joined #tripleo14:26
openstackgerritMartin Mágr proposed openstack/tripleo-common master: [WIP] Add health checks for Swift services  https://review.openstack.org/55973714:30
*** lblanchard1 has joined #tripleo14:31
*** lblanchard has quit IRC14:31
*** lblanchard has joined #tripleo14:33
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044514:33
*** cdearborn has quit IRC14:34
*** itlinux has quit IRC14:34
*** anilvenkata has quit IRC14:35
*** lblanchard1 has quit IRC14:36
*** thrash is now known as thrash|biab14:39
*** dsariel has quit IRC14:39
openstackgerritAttila Darazs proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56045214:42
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart-extras master: Add ability  to deploy FreeIPA on OVB  https://review.openstack.org/53443514:42
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Use the build_service_filter from kolla_builder  https://review.openstack.org/55505114:42
*** agurenko has quit IRC14:43
openstackgerrithanish proposed openstack/tripleo-heat-templates stable/queens: Removed unnecessary services from the LiquidioCompute role.  https://review.openstack.org/56045314:43
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Do container image prepare during undercloud deploy  https://review.openstack.org/54602414:46
mwhahahaweshay, arxcruz|ruck, rfolco|rover: ok so we're still seeing ssh failures in ovb even after all the ulimit patches in master. are we still seeing those in the promotion jobs?14:48
mwhahahaexample, https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/10375/14:48
*** cdearborn has joined #tripleo14:48
rfolco|rovermwhahaha, I think its concurrency14:48
arxcruz|ruckmwhahaha: yes, we are seeing that, but isn't related to ulimit anymore, it's the metadata now that is taking too long to response14:48
arxcruz|ruckmwhahaha: https://bugs.launchpad.net/tripleo/+bug/176300914:49
openstackLaunchpad bug 1763009 in tripleo "periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master failing due metadata timeout" [High,Triaged]14:49
mwhahahain ovb it's only running a single test14:49
weshaymwhahaha, fs001 ovb in rdo cloud looks a lot better today imho14:49
mwhahahaso i don't think that's concurrency related?14:49
rfolco|rovermwhahaha, oh, not for this case, sorry.14:50
arxcruz|ruckmwhahaha: maybe, because both failures have the same behavior, we miss the second failure cause14:50
arxcruz|ruckonce we fix the neutron, we are noticing the second cause of failure14:50
arxcruz|ruckmwhahaha: anywah, ykarel is taking a look, and dalvarez will work on it once he has some time, according him, he has a critical bug now to fix14:50
*** cshastri has joined #tripleo14:50
mwhahahak14:50
arxcruz|ruckproblem was, we saw the failure, it was ssh, so it's the neutron ulimit issue, we have a patch, let's wait get merged14:51
*** rajinir has joined #tripleo14:51
arxcruz|ruckwith was understandable in this case14:51
gfidenteslagle so sounds like we want to increase zaqar post size14:52
gfidenteslagle did you pick 1048576 with some theory?14:52
gfidenteslagle can we just double it?14:52
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: Run "overcloud container image prepare" only once  https://review.openstack.org/52862114:54
d0ugalrbrady,d0ugal,apetrich,thrash,toure,jtomasek: Workflow Squad meeting in 10 mins https://etherpad.openstack.org/p/tripleo-workflows-squad-status (probably starting a little late)14:55
*** moshele has quit IRC14:55
bogdandobandini, dciabrin: hi. Do you know if environments/puppet-pacemaker.yaml needs to be switched to docker/services?14:57
bogdandogiven the containers-default moving in progress14:57
slaglegfidente: we would want to check with therve about that14:57
bogdandoor prolly moved to environments/services-barematal othwerise?14:57
therveHi14:58
slaglegfidente: there's code in the workflow to make sure the messages are broken up based on the max message size14:58
slaglei wonder if that did not work correctly14:58
gfidenteslagle pointers?14:58
gfidentetherve reasons to set zaqar size to 1048576 and not, say 4?14:59
gfidente4M14:59
thervegfidente, Performance?14:59
bandinibogdando: let me check15:00
slaglegfidente: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/ansible.py#L37015:00
thervegfidente, Why do you need to increase it?15:00
gfidentetherve ok I am asking because we've hit an error where 1M probably is not enough15:00
d0ugalgfidente: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/ansible.py#L370-L38615:00
openstackgerritSai Ram Peesapati proposed openstack/tripleo-heat-templates master: Parameterizing Puppet Tags  https://review.openstack.org/55759815:00
d0ugaloh, slagle beat me :)15:00
slaglegfidente: i don't think we want to increase the max size, but instead figure out why that code that breaks it up didn't work :)15:01
d0ugalunicode maybe?15:01
gfidenteslagle ok I didn't know that split code existed15:01
bandinibogdando: I guess we can move it since (docker-ha.yaml) is what gives ha in a containerized world15:02
bogdandobandini: ack thanks15:02
ykarelmwhahaha, till now i have seen metadata issue(takes large time > 10 seconds to respond) only in multi controller cases15:02
therveslagle, Is it in a ansible action?15:03
mwhahahaykarel: well ovb is multicontroller, so that'd make sense15:03
ykarelmwhahaha, yes,15:03
slagletherve: it's in the AnsiblePlaybookAction15:03
therveRight wondering where the error happened15:03
slagletherve: http://logs.openstack.org/66/546966/28/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9b09915/logs/undercloud/var/log/mistral/executor.log.txt.gz15:04
ykarelmwhahaha, trying to reproduce multinode locally, will update if find something15:04
slaglei may need to add some debugging for CI to post_message so we can see what happened15:04
*** ykarel is now known as ykarel|away15:05
*** dsariel has joined #tripleo15:07
openstackgerritJames Slagle proposed openstack/tripleo-common master: DNM: post_message debugging  https://review.openstack.org/56046415:08
*** vineet has joined #tripleo15:08
*** ratailor has joined #tripleo15:08
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Migrate Ceph deployment to external_deploy_task  https://review.openstack.org/54696615:08
*** gkadam has joined #tripleo15:09
ykarel|awaymwhahaha, please also look:- https://review.openstack.org/#/c/560359/15:09
*** jistr|mtg is now known as jistr15:10
*** ihrachys has joined #tripleo15:11
openstackgerritJames Slagle proposed openstack/tripleo-common master: DNM: post_message debugging  https://review.openstack.org/56046415:11
*** jfrancoa has quit IRC15:16
therveslagle, json encoding can add quite a bit of size15:17
therveWe should check the size on the encoding bit15:17
thervegfidente, slagle : http://paste.openstack.org/show/718946/15:19
*** paramite has quit IRC15:21
slagletherve: hmm, i hadn't thought about that. yea we'll probably need to fix that15:22
*** sai_p has joined #tripleo15:22
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates master: Deploy Designate in scenario003  https://review.openstack.org/55500715:28
openstackgerritBen Nemec proposed openstack/tripleo-heat-templates master: Mark Designate as experimental  https://review.openstack.org/56047015:28
*** cylopez has joined #tripleo15:30
jistrslagle, therve, gfidente: as the data is just logs, we probably don't need to send it all in a single chunk. We could cut it up into smaller pieces. I didn't look where such code could go though. Perhaps in the tripleo-common actions/workflows or worst case Mistral itself. Not sure if it's feasible or worth doing. Just a thought :)15:31
d0ugaljistr: I think that is what the code is trying to do now?15:31
*** ratailor has quit IRC15:31
slaglejistr: yea we have that part already15:31
*** rodolof has quit IRC15:32
slagleit seems it could be the subsequent json encoding that runs us over the limit still15:32
jistrah i didn't read all the context above :) sorry :)15:32
*** yamahata has quit IRC15:33
*** assassin has quit IRC15:33
*** ykarel|away has quit IRC15:38
*** etingof has quit IRC15:38
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Don't merge repo changes from different branches  https://review.openstack.org/56047315:39
bogdandodoes anyone knows if puppet/services/opendaylight-ovs.yaml expects containerization?15:39
*** thrash|biab is now known as thrash15:39
*** gkadam_ has joined #tripleo15:39
bogdandoputting FIXME around in environments tht15:39
bogdandofor now, looking into existing codebase, I decided do not mark yet a *backend/plugin/host-config* related puppet templates as FIXME15:40
*** gkadam has quit IRC15:40
bogdandothough the fact that we have neutron ml2 plugin containerized spoils the nice picture15:40
*** zoli|afk-TPB is now known as zoli15:40
bogdandoso shall we expect containerized neutron-plugin-ml2-odl15:41
bogdandoanyway, we could set those FIXMEs later...15:41
*** cshastri has quit IRC15:42
*** marios has quit IRC15:48
*** dsariel has quit IRC15:48
*** dparkes has quit IRC15:48
*** dparkes has joined #tripleo15:48
*** itlinux has joined #tripleo15:50
*** pdeore|afk has quit IRC15:50
openstackgerritCarlos Camacho proposed openstack/tripleo-quickstart-extras master: Force json output for openstack software deployment show <deployment>  https://review.openstack.org/56047915:51
*** ramishra has quit IRC15:52
*** etingof has joined #tripleo15:52
*** moshele has joined #tripleo15:53
openstackgerritCarlos Camacho proposed openstack/tripleo-quickstart-extras master: Force json output for openstack software deployment show <deployment>  https://review.openstack.org/56047915:54
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260715:55
*** hamzy has quit IRC15:55
*** salmankhan has quit IRC15:57
slagletherve: since zaqar is what is doing the encoding, we can't exactly chunk that easily15:57
*** shreshtha-wfh has quit IRC15:57
slagletherve: should we just build in more fudge factor into the action size? i have it 90% of max message size right now.15:57
slaglecould go down to something more like 50%15:58
therveslagle, Yeah or check encoding size15:59
therveThough it may be a bit sad for memory15:59
*** salmankhan has joined #tripleo15:59
*** lucasagomes is now known as lucas-afk15:59
slagleyea i could calculate the overhead for each message i guess16:00
*** ykarel|away has joined #tripleo16:01
*** pblaho has joined #tripleo16:01
*** gkadam__ has joined #tripleo16:01
*** cylopez has quit IRC16:03
openstackgerritDougal Matthews proposed openstack/tripleo-common master: WIP Optionally run config download at the end of stack create/update  https://review.openstack.org/55422416:04
*** gkadam_ has quit IRC16:04
*** yamahata has joined #tripleo16:04
*** gkadam has joined #tripleo16:04
*** fragatina has quit IRC16:05
*** fragatina has joined #tripleo16:06
thervestdout is kept in memory, so memory is probably not a big concern :)16:06
*** gkadam__ has quit IRC16:06
openstackgerritJiri Stransky proposed openstack/python-tripleoclient master: Update converge CLI to remove no-ops  https://review.openstack.org/55853616:08
openstackgerritSergii Golovatiuk proposed openstack/tripleo-heat-templates master: Replace LOG.warn with LOG.warning  https://review.openstack.org/55880416:11
*** dprince has quit IRC16:19
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: Document undercloud --verbose option  https://review.openstack.org/56055816:21
*** florianf has quit IRC16:23
*** janki has quit IRC16:23
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Switch remaining env files to containers defaults  https://review.openstack.org/56055916:24
EmilienMjaosorior: I confirm it works now: https://review.openstack.org/#/c/560264/ thanks again man16:25
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Reduce reproducer redudancy  https://review.openstack.org/55836916:25
*** leanderthal has quit IRC16:28
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: Reproduce CI multinode job with libvirt  https://review.openstack.org/54342916:28
openstackgerritJames Slagle proposed openstack/tripleo-common master: Allow for more overhead in zaqar message  https://review.openstack.org/56046416:28
mfedosinrbrady: hi! I've updated my patch with heat capabilities workflows https://review.openstack.org/#/c/559989/16:29
mfedosinwhat do you think?16:29
*** dparkes has quit IRC16:30
*** jpich has quit IRC16:31
bogdandofolks, there is a container-defaults remaining bits PTAL https://review.openstack.org/#/c/560559/16:34
bogdandowould be nice to have +1 from ansible-ceph and networking and partner integration folks...16:34
bogdandoif it works ofc :D16:34
*** udesale has quit IRC16:35
*** dprince has joined #tripleo16:35
jaosoriorEmilienM: nice!16:36
*** zoli is now known as zoli|gone16:37
*** zoli|gone is now known as zoli16:37
*** ihrachys_ has joined #tripleo16:37
*** agopi has quit IRC16:37
*** ffiore has quit IRC16:39
*** ihrachys has quit IRC16:40
*** bogdando has quit IRC16:40
*** wolverineav has joined #tripleo16:40
*** ykarel|away has quit IRC16:43
*** dparkes has joined #tripleo16:46
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: metadata.json: prepare for 9.0.0 release (rocky-m1)  https://review.openstack.org/56056816:51
openstackgerritEmilien Macchi proposed openstack/tripleo-ui master: prepare for 9.0.0 release (rocky-m1)  https://review.openstack.org/56056916:52
EmilienMmwhahaha: ^ let's merge these ones this week in case of gate is super busy next week (which we know it'll be the case)16:53
mwhahahak16:53
*** tesseract has quit IRC16:59
*** gkadam has quit IRC16:59
*** agopi has joined #tripleo17:04
*** kmy has joined #tripleo17:09
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add Ironic Networking Baremetal Templates  https://review.openstack.org/54731817:09
*** alee is now known as alee_afk17:12
*** mdnadeem has quit IRC17:13
*** salmankhan has quit IRC17:13
*** kopecmartin has quit IRC17:16
*** jpena is now known as jpena|off17:16
*** hamzy has joined #tripleo17:18
*** karthiks has quit IRC17:20
*** panda is now known as panda|bbl17:22
*** akrzos has quit IRC17:23
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Remove the os-net-config element config.json template  https://review.openstack.org/55793917:24
openstackgerritMerged openstack/tripleo-common master: Add custom subclass to revert mapping  https://review.openstack.org/55649117:24
openstackgerritMerged openstack-infra/tripleo-ci master: Add job to test overcloud updates  https://review.openstack.org/55939417:24
*** bmurrill has quit IRC17:27
openstackgerritMerged openstack/puppet-tripleo stable/queens: ceilo:base: include ::ceilometer::dispatcher::gnocchi  https://review.openstack.org/56016917:28
openstackgerritMerged openstack/python-tripleoclient master: containerized/undercloud: disable verbosity by default  https://review.openstack.org/55914017:28
openstackgerritMerged openstack/python-tripleoclient master: Use six to import things from urllib  https://review.openstack.org/56000517:28
openstackgerritMerged openstack/tripleo-quickstart master: Allow to clone extras if zuul cloner is not presented  https://review.openstack.org/55777717:28
*** athomas has quit IRC17:29
*** wolverineav has quit IRC17:29
openstackgerritRafael Folco proposed openstack/tripleo-quickstart-extras master: Fix dstat time penalty for failed runs  https://review.openstack.org/55890817:30
*** wolverineav has joined #tripleo17:30
*** chlong has quit IRC17:30
*** akrzos has joined #tripleo17:32
*** yprokule has quit IRC17:33
*** wolverineav has quit IRC17:34
openstackgerritMerged openstack/tripleo-ui master: prepare for 9.0.0 release (rocky-m1)  https://review.openstack.org/56056917:37
*** agopi has quit IRC17:38
*** leanderthal has joined #tripleo17:38
*** trown is now known as trown|lunch17:40
*** ihrachys__ has joined #tripleo17:47
*** ykarel|away has joined #tripleo17:49
*** ihrachys_ has quit IRC17:50
*** arxcruz|ruck is now known as arxcruz17:50
openstackgerritJuan Badia Payno proposed openstack/tripleo-heat-templates master: [WIP]logging: read non-containerized logs  https://review.openstack.org/54584117:52
*** wolverineav has joined #tripleo17:58
openstackgerritMerged openstack/tripleo-quickstart-extras master: Replace message OVB by multinode  https://review.openstack.org/56032518:04
openstackgerritMerged openstack/tripleo-heat-templates master: Allowing Non-IP Traffic in L2 and L3 domains  https://review.openstack.org/53660318:05
*** yprokule has joined #tripleo18:11
*** yprokule has quit IRC18:13
*** ihrachys__ has quit IRC18:14
*** anilvenkata has joined #tripleo18:15
*** ihrachys has joined #tripleo18:16
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: add option to turn on/off non default kernel  https://review.openstack.org/56010018:17
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: Setting the kernel override option to false  https://review.openstack.org/56011218:18
*** orange_juluis has joined #tripleo18:19
orange_juluisHas anybody configured TripleO to deploy the ironic service to the overcloud with separate provisioning and cleaning networks?18:19
*** radeks has quit IRC18:21
*** radeks has joined #tripleo18:22
*** Goneri has quit IRC18:22
openstackgerritMerged openstack/tripleo-quickstart-extras master: Revert "Remove adjust-interface-mtus script"  https://review.openstack.org/55910718:24
*** ykarel|away has quit IRC18:24
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Enable Neutron LBaaS Integration  https://review.openstack.org/55674118:24
openstackgerritMerged openstack/tripleo-upgrade master: Re-use undercloud upgrades tasks.  https://review.openstack.org/55230318:24
openstackgerritMerged openstack/puppet-tripleo master: metadata.json: prepare for 9.0.0 release (rocky-m1)  https://review.openstack.org/56056818:24
*** masco has quit IRC18:29
hjensasorange_juluis: I have not tried it, but it should be possible. But you have to deploy using the default, then create the cleaning and provisioning network in neutron. Then update the parameters and do a stack update.18:29
hjensasorange_juluis: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/ironic-conductor.yaml#L49-L5618:31
*** trown|lunch is now known as trown18:32
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Handle undercloud upgrades via host_prep_tasks  https://review.openstack.org/55921018:33
openstackgerritEmilien Macchi proposed openstack/tripleo-upgrade master: WIP - containerized undercloud upgrade  https://review.openstack.org/55362918:35
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Introduce tripleo-ci-centos-7-containerized-undercloud-upgrades  https://review.openstack.org/55363318:35
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Migrate to the new Mistral context class  https://review.openstack.org/50618618:40
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart-extras master: containerized-undercloud: don't manage stackrc  https://review.openstack.org/55981818:40
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: heat: align config with instack-undercloud  https://review.openstack.org/55835418:40
*** agopi has joined #tripleo18:41
*** shreshtha has joined #tripleo18:55
*** pcaruana has quit IRC18:55
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Enable cleanup by default for undercloud install  https://review.openstack.org/56057818:58
*** moshele has quit IRC18:59
openstackgerritAlex Schultz proposed openstack/python-tripleoclient master: Enable cleanup by default for undercloud install  https://review.openstack.org/56057819:00
*** shreshtha has quit IRC19:04
*** moshele has joined #tripleo19:16
*** akrivoka has quit IRC19:17
*** rpioso|afk is now known as rpioso19:17
*** panda|bbl is now known as panda|off19:19
*** Goneri has joined #tripleo19:24
*** holser__ has quit IRC19:25
*** wolverineav has quit IRC19:25
*** wolverineav has joined #tripleo19:31
*** jmelvin has quit IRC19:34
*** fragatina has quit IRC19:37
*** fragatina has joined #tripleo19:37
trozetdsneddon: you around?19:39
*** moshele has quit IRC19:41
*** fragatina has quit IRC19:42
*** gfidente has quit IRC19:45
openstackgerritTim Rozet proposed openstack/tripleo-common master: Fixes ODL healthcheck URI  https://review.openstack.org/56059819:48
*** jmelvin has joined #tripleo19:49
openstackgerritAna Krivokapic proposed openstack/tripleo-validations master: Extract 'filtered()' function into 'utils'  https://review.openstack.org/56062319:52
*** dbecker has quit IRC19:54
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates master: Removes odl-dlux-gui feature for ODL  https://review.openstack.org/56062419:54
dsneddontrozet, I'm here20:02
*** holser__ has joined #tripleo20:02
trozetdsneddon: so figured out that issue from yesterday20:04
trozetdsneddon: I wanted to confirm with you before I go fix it upstream20:05
trozetdsneddon: https://jira.opnfv.org/browse/APEX-58820:05
*** lblanchard has quit IRC20:06
*** jobewan has quit IRC20:10
*** fragatina has joined #tripleo20:13
*** paramite has joined #tripleo20:20
*** NobodyCam_ has joined #tripleo20:21
*** v1k0d3n_ has joined #tripleo20:21
*** ansmith has quit IRC20:23
*** andreaf_ has joined #tripleo20:27
dsneddontrozet, So was the fix there to modify the dhcp-interfaces-all.sh script?20:27
dsneddontrozet, We've had a lot of bugs caused by that failsafe script.20:28
*** v1k0d3n has quit IRC20:28
*** andreaf has quit IRC20:28
*** NobodyCam has quit IRC20:28
*** tvignaud has quit IRC20:28
*** v1k0d3n_ is now known as v1k0d3n20:28
*** NobodyCam_ is now known as NobodyCam20:28
trozetdsneddon: no, to modify the dhcp-interface@.service systemd file20:28
trozetdsneddon: we redeployed and it all worked after that20:29
dsneddontrozet, So the issue that the script stops when eno3 fails to come up?20:29
*** tvignaud has joined #tripleo20:29
trozetdsneddon: no that part is normal20:29
*** andreaf_ is now known as andreaf20:29
trozetdsneddon: linux networking will just stop when the first interface fails, thats why this dhcp-interface systemd and script exist20:29
trozetdsneddon: to work around that problem20:30
trozetdsneddon: but then the dhcp-interface@.service should fire for each interface, and it doesnt for the ctlplane interface because of20:30
*** alee_afk is now known as alee20:30
trozetConditionPathExists=!/etc/sysconfig/network-scripts/ifcfg-%i20:30
trozetdsneddon: because the ifcfg file for the ctlplane interface was already written at first boot20:31
dsneddontrozet, If your initial os-net-config run succeeds, then you shouldn't be ending up with the DHCP service for all these interfaces.20:31
dsneddontrozet, The dhcp-interfaces-all.sh script runs only if os-net-config fails, I'm pretty sure.20:31
trozetdsneddon: yeah so what we looked at yesterday was not quite right, it had manual intervention during that boot process20:31
trozetdsneddon: we redeploy and let it hit the failure, os-net-config is never fired20:31
*** mrunge has quit IRC20:31
*** bandini has quit IRC20:31
trozetdsneddon: the dhcp-interface systemd process is started on boot, right after network.target so its before os-collect-config even runs20:32
*** bandini has joined #tripleo20:33
dsneddontrozet, Hmm. I'm not sure about that, I honestly have never audited the dhcp-interface script20:33
trozetdsneddon: so the process is, linux networking starts, tries to dhcp interfaces sequentially, no matter what happens there all the dhcp-interface systemd services start which will also try to dhcp each interface that hasnt done so already20:33
*** mrunge has joined #tripleo20:33
trozetdsneddon: at that point the ctlplane interface should have  a dhcp ip and be able to run os-collect-config to get metadata, then fire os-net-config20:33
*** liverpooler has quit IRC20:34
*** holser__ has quit IRC20:34
trozetdsneddon: do you know where this systemd service comes from? dhcp-interface20:35
*** liverpooler has joined #tripleo20:38
dsneddontrozet, I just checked. It appears that the file /etc/udev/rules.d/99-dhcp-all-interfaces.rules exists in the image, and gets deleted if os-net-config succeeds (in the run-os-net-config.sh script that is part of the newer-style NIC configs).20:38
dsneddontrozet, I thought that Heat laid down that file, but it appears it's already part of the image.20:39
*** paramite has quit IRC20:39
trozetdsneddon: ok, it doesnt look like the systemd file belongs to any package... and googling is turning up nothing :/20:39
dsneddontrozet, I'm going to poke around a qcow image20:41
trozetdsneddon: do you see what im saying though for why the fix works?20:41
*** liverpooler has quit IRC20:42
*** raildo has quit IRC20:47
*** orange_juluis has quit IRC20:49
dsneddontrozet, I think so, you removed the condition that checks if the file exists, and doesn't write it or run ifup if the file exists.20:51
dsneddontrozet, Right?20:51
trozetdsneddon: right20:51
*** pchavva has quit IRC20:52
trozetdsneddon: dhcp-all-interfaces.sh has that check in it already, so when taht runs if it sees icfg file, it wont overwrite it20:52
trozetdsneddon: but the ifup part afterwards in the systemd unit file is what we need so the interface actually starts20:52
trozetdsneddon: and that should be just fine to run, even if the interface is already up and the ifcfg file already exists20:52
dsneddontrozet, I agree. I'm just looking up the files that are included in the image so I have an idea what changes we need to make20:53
*** rbowen has quit IRC20:53
*** jtomasek has quit IRC20:54
trozetdsneddon: ok thanks!20:54
*** trown is now known as trown|outtypewww21:01
*** sai_p has quit IRC21:01
dsneddontrozet, The 99-dhcp-all-interfaces.rules file has only: 'SUBSYSTEM=="net", KERNEL!="lo", ACTION=="add", TAG+="systemd", ENV{SYSTEMD_WANTS}+="dhcp-interface@$name.service"'21:01
trozetdsneddon: that should be ok right?21:04
*** slaweq has quit IRC21:08
*** slaweq has joined #tripleo21:09
*** atoth has quit IRC21:10
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Revert "Allow to clone extras if zuul cloner is not presented"  https://review.openstack.org/56064121:12
*** slaweq has quit IRC21:13
*** rfolco|rover is now known as rfolco|off21:13
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM: Setting the kernel override option to false  https://review.openstack.org/56011221:14
*** abishop has quit IRC21:17
dsneddontrozet, The script is in /usr/local/sbin/dhcp-all-interfaces.sh in the image21:17
mwhahahajrist: you around?21:18
trozetdsneddon: yeah, is it part of any package?21:18
EmilienMmwhahaha: honza is around21:19
honzamwhahaha: ohai21:19
mwhahahahonza: what network should the UI be on?21:20
mwhahahait it just the control plane network?21:20
mwhahahaEmilienM: I think on the undercloud the internal_api == control plane right?21:20
mwhahahathere really isn't a distinction21:20
EmilienMI need to check that21:21
honzayes21:21
mwhahahahonza: is there any reason not to always fron the UI with haproxy?21:21
EmilienMmwhahaha: yes21:21
mwhahahas/fron/front21:21
honzamwhahaha: i don't understand all the implications but i don't see why we couldn't always have haproxy21:22
mwhahahaso i'd assume we'd want the UI fronted by HA proxy (for future HAing needs) and i'd assume we'd want it mapped to the 'public' vip21:22
mwhahahabut the question would be on the undercloud upgrade since those concepts don't previously exist how does that work21:22
EmilienMnot only for HA but also for SSL termination21:23
EmilienMwhich is why we have HAproxy on the undercliud.21:23
EmilienMundercloud even.21:23
mwhahaharight21:23
honzathis ^21:23
EmilienMso ++ to have UI in HAproxy21:23
mwhahahaEmilienM: so i think the issue is that we have any other ip address other than the undercloud ip being used21:23
mwhahahaEmilienM: in instack, we only had a single ip for the ctrl plane i thought21:23
*** alee has quit IRC21:24
*** morazi has quit IRC21:24
EmilienMhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L832-L84721:25
mwhahahaso we ahd two ips21:25
mwhahahainternal (ctrlplane) and external (???)21:25
mwhahahahttps://github.com/openstack/instack-undercloud/blob/2633e98788bcf0a02ba6bf20967def902e75e95b/elements/puppet-stack-config/puppet-stack-config.yaml.template#L11-L1221:26
EmilienMpublic?21:26
*** alee has joined #tripleo21:27
*** orange_julius has joined #tripleo21:27
mwhahahahttps://github.com/openstack/instack-undercloud/blob/41f2694d13386a2c533ca300f109afc2fc2f0595/undercloud.conf.sample#L2421:27
mwhahahaso undercloud_public_host21:27
dsneddontrozet, There is also this file in /usr/share/pkg-map/dhcp-all-interfaces: http://paste.openstack.org/show/718993/21:27
mwhahahaEmilienM: so in this case we're not properly listening on the public host for the ui21:27
orange_juliusGetting strange rabbitmq errors after a fairly vanilla installation of TripleO Pike... Channel error on connection <0.19515.5> (172.18.10.10:43876 -> 172.18.10.17:5672, vhost: '/', user: 'guest'), channel 1: operation basic.publish caused a channel exception not_found: "no exchange 'reply_e94d4f4c9edb4c91b98c98fc85ca6835' in vhost '/'"21:27
dsneddontrozet, Does that tell you anything about what package it is part of?21:28
trozetdsneddon: no i run rpm -qf on it and its not part of any package21:29
EmilienMmwhahaha: external then?21:30
EmilienMmwhahaha: in network/service_net_map.j2.yaml21:30
trozetdsneddon: ive got go for now, i'll try to find out where this is coming from tmrw21:30
mwhahahaEmilienM: yea21:30
itlinuxhello all, does anyone have any suggestion on why the fencing gen does not work on 2 of my underclouds and works fine on the 3rd.. here is the output of the one not working http://paste.openstack.org/show/718994/21:31
EmilienMmwhahaha: and none of the endpoints?21:34
EmilienMmwhahaha: I mean the tripleo::ui::endpoint_*21:34
mwhahahaEmilienM: the endpoints were fine21:34
EmilienMkk21:34
EmilienMit was just the binding21:34
mwhahahaEmilienM: it's that the bind host did not match what was being generated for the urls21:35
EmilienMmwhahaha: I'll give a try21:35
EmilienMdtrainor: ^ fyi if you want to read back21:35
dtrainorthx21:36
*** ansmith has joined #tripleo21:37
dsneddontrozet, Thanks for the heads-up about this issue, we can talk more later21:37
*** jmelvin has quit IRC21:40
orange_juliusSorry for the spam, figured it out21:43
*** dprince has quit IRC21:44
*** rcernin has joined #tripleo21:49
dsneddontrozet, dhcp-all-inerfaces is part of tripleo-diskimage-builder, but not in a package.21:50
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Run "overcloud container image prepare" only once  https://review.openstack.org/52862121:54
ihrachysa patch of mine has -1 from RDO Third Party CI. I clicked the link to Jenkins and check console output and it doesn't indicate a specific reason for failure, or tempest output. I can't find logs or other artifacts anywhere. how do I get access to the tempest logs?21:58
ihrachysspecifically, I am looking at https://review.rdoproject.org/jenkins/job/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/10122/21:58
*** itlinux has quit IRC22:03
*** itlinux has joined #tripleo22:06
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: Add Neutron L3 agent failover check during upgrade  https://review.openstack.org/55941022:07
sshnaidmEmilienM, btw, collectd server is installed here: https://review.openstack.org/#/c/560048/ in case you need to test clients22:07
*** hamzy has quit IRC22:07
*** itlinux has quit IRC22:07
*** radeks has quit IRC22:10
*** noslzzp has quit IRC22:10
*** cdearborn has quit IRC22:20
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart master: Add sudo and iproute deps  https://review.openstack.org/56067322:22
*** jcoufal has quit IRC22:27
openstackgerritGonéri Le Bouder proposed openstack/tripleo-quickstart-extras master: tempest: gen the JUNIT out from subunit v2 file  https://review.openstack.org/56067622:28
*** jcoufal has joined #tripleo22:31
-openstackstatus- NOTICE: zuul was restarted to updated to the latest code; you may need to recheck changes uploaded or approvals added between 21:30 and 21:4522:31
*** jcoufal has quit IRC22:33
openstackgerritGonéri Le Bouder proposed openstack/tripleo-quickstart-extras master: tempest: restore Mitaka support  https://review.openstack.org/56067822:34
*** dsariel has joined #tripleo22:36
*** mcornea has quit IRC22:39
*** derekh has quit IRC22:40
*** d0ugal has quit IRC22:43
*** Goneri has quit IRC22:44
*** d0ugal has joined #tripleo22:48
*** ipsecguy has quit IRC23:01
*** ipsecguy has joined #tripleo23:02
*** dparkes has quit IRC23:09
*** slaweq has joined #tripleo23:09
*** dsariel has quit IRC23:09
*** slaweq has quit IRC23:14
*** pchavva has joined #tripleo23:19
EmilienMsshnaidm: ok I'll look,thx23:24
*** pchavva has quit IRC23:31
*** owalsh is now known as owalsh_afk23:47
*** rpioso is now known as rpioso|afk23:54
*** moshele has joined #tripleo23:54
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260723:55
*** moshele has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!