Friday, 2019-09-13

*** d0ugal has quit IRC00:12
*** d0ugal has joined #tripleo00:19
openstackgerritMerged openstack/os-net-config master: Fix usage of oneOf keyword in schema  https://review.opendev.org/68144700:20
*** pkopec has joined #tripleo00:21
*** gbarros has quit IRC00:28
*** saneax has quit IRC00:31
cloudnullweshay|ruck +1 we can add something like https://github.com/openstack/tripleo-ansible/blob/master/zuul.d/layout.yaml#L29-L33 for that new job, which should ensure we're not missing anything when we modify that role.00:36
*** gbarros has joined #tripleo00:49
*** Garyx_ has quit IRC00:49
*** pkopec has quit IRC00:57
*** gbarros has quit IRC00:59
openstackgerritMerged openstack/tripleo-common master: healthcheck_port: run ss with both sudo & root as best effort  https://review.opendev.org/68175201:02
*** Garyx has joined #tripleo01:03
*** ayoung has quit IRC01:24
openstackgerritEmilien Macchi proposed openstack/tripleo-common stable/stein: healthcheck_port: run ss with both sudo & root as best effort  https://review.opendev.org/68195301:33
openstackgerritMerged openstack/tripleo-quickstart-extras master: Fix waiting OC VMs after restored from snapshot  https://review.opendev.org/67997501:35
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/stein: Resolve broken zaqar container caused by logging issues  https://review.opendev.org/68195401:38
*** tkajinam has quit IRC01:54
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Support direct provisioning of bare metal  https://review.opendev.org/66445602:07
*** matbu has quit IRC02:08
*** apetrich has quit IRC02:10
*** tkajinam has joined #tripleo02:11
*** xarlos has quit IRC02:12
openstackgerritMerged openstack/tripleo-heat-templates master: Pass Loader to yaml.load to remove YAMLLoadWarning  https://review.opendev.org/68055702:12
*** matbu has joined #tripleo02:14
openstackgerritSteve Baker proposed openstack/tripleo-quickstart master: Run multinode and undercloud jobs nova-less  https://review.opendev.org/68057302:26
weshay|ruckcloudnull nkinder adding fs039 to catch issues there in the future https://review.rdoproject.org/r/2225902:27
nkinderweshay|ruck, great02:29
weshay|ruck:)02:29
nkinderweshay|ruck, I got that keycloak job working02:29
weshay|ruckah cool /me looks02:29
nkinderCI is still running, but it deployed successfully on the supplemental node02:29
nkinderovercloud deploy will hit this other issue we looked at today02:30
weshay|ruckNICE02:30
weshay|ruckon overcloud_deploy02:30
weshay|rucknkinder actually we can test both02:30
nkinderyep.  Had to lower the memory usage more and increase a timeout for keycloak since it is slower with a small heap02:30
weshay|ruckperfect..  nkinder shall kill the current one and include kevin's change?02:31
nkindertesting both would be awesome02:31
nkindernuke it02:31
weshay|ruckk.. it's restarting now02:31
weshay|ruckcloudnull testing your change here: https://review.rdoproject.org/r/#/c/21672/02:32
weshay|ruckok..  let's all get out of here :)02:32
nkinderI'll check on its progress this02:32
nkinderevening... :)02:32
weshay|rucko/02:33
nkinderlater02:33
*** tkajinam has quit IRC02:36
openstackgerritMerged openstack/tripleo-common stable/stein: healthcheck_port: run ss -ntp with sudo  https://review.opendev.org/68169502:36
*** matbu has quit IRC02:38
openstackgerritMerged openstack/python-tripleoclient stable/stein: Ensure no duplicate IPs are used in undercloud.conf  https://review.opendev.org/68187802:41
*** matbu has joined #tripleo02:44
*** tkajinam has joined #tripleo02:56
*** rf0lc0 has quit IRC03:05
openstackgerritSteve Baker proposed openstack/tripleo-quickstart master: Run multinode and undercloud jobs nova-less  https://review.opendev.org/68057303:12
*** ramishra has joined #tripleo03:16
*** ksambor has joined #tripleo03:48
*** ksambor has joined #tripleo03:49
*** skramaja has joined #tripleo03:57
*** rh-jelabarre has quit IRC03:58
*** ykarel|away has joined #tripleo04:06
*** surpatil has joined #tripleo04:31
*** radeks has quit IRC04:35
*** ykarel|away has quit IRC04:43
*** radek has joined #tripleo04:50
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Support direct provisioning of bare metal  https://review.opendev.org/66445604:51
*** saneax has joined #tripleo04:57
*** surpatil has quit IRC05:01
openstackgerritLuca Miccini proposed openstack/puppet-tripleo master: Enable deep_compare for fencing resource  https://review.opendev.org/68177805:01
*** pcaruana has joined #tripleo05:02
*** saneax has quit IRC05:06
*** marios has joined #tripleo05:07
*** udesale has joined #tripleo05:08
*** ratailor has joined #tripleo05:08
*** surpatil has joined #tripleo05:10
*** radek has quit IRC05:11
*** ykarel|away has joined #tripleo05:13
*** pierreprinetti has quit IRC05:14
*** pierreprinetti has joined #tripleo05:15
*** jtomasek has joined #tripleo05:17
*** radeks has joined #tripleo05:17
*** lmiccini has joined #tripleo05:28
*** pdeore has joined #tripleo05:32
*** yprokule has joined #tripleo05:39
*** pdeore has quit IRC05:39
*** saneax has joined #tripleo05:46
openstackgerritRadoslaw Smigielski proposed openstack/python-tripleoclient stable/queens: [FFU] Fix sensitive hostname in FFU upgrade run command  https://review.opendev.org/68193005:50
*** jfrancoa has joined #tripleo05:51
*** tkajinam has quit IRC05:54
openstackgerritLuca Miccini proposed openstack/puppet-tripleo stable/stein: Implement stonith levels and enable kdump agent.  https://review.opendev.org/68196606:00
openstackgerritLuca Miccini proposed openstack/puppet-tripleo stable/rocky: Implement stonith levels and enable kdump agent.  https://review.opendev.org/68196706:00
*** tkajinam has joined #tripleo06:03
*** xek has joined #tripleo06:03
*** ykarel|away is now known as ykarel06:03
*** jfrancoa has quit IRC06:06
*** rpittau|afk is now known as rpittau06:08
*** xek has quit IRC06:12
*** xek has joined #tripleo06:13
*** avivgts has joined #tripleo06:17
*** tkajinam_ has joined #tripleo06:19
*** tkajinam has quit IRC06:22
*** jfrancoa has joined #tripleo06:23
*** saneax has quit IRC06:23
*** holser has joined #tripleo06:26
*** florianf has joined #tripleo06:33
openstackgerritSergii Golovatiuk proposed openstack/python-tripleoclient master: [FFU] Fix sensitive hostname in FFU upgrade run command  https://review.opendev.org/68197106:42
*** cylopez has joined #tripleo06:45
openstackgerritRadoslaw Smigielski proposed openstack/python-tripleoclient stable/queens: [FFU] Fix sensitive hostname in FFU upgrade run command  https://review.opendev.org/68193006:45
*** slaweq has joined #tripleo06:51
*** xek has quit IRC06:56
*** pkopec has joined #tripleo06:56
*** udesale has quit IRC07:04
*** hberaud|gone is now known as hberaud07:05
*** jbadiapa has joined #tripleo07:06
*** tosky has joined #tripleo07:12
openstackgerritLuca Miccini proposed openstack/puppet-tripleo stable/queens: Make the errors around fencing and IHA clearer  https://review.opendev.org/68197307:14
*** florianf has quit IRC07:14
*** tesseract has joined #tripleo07:15
*** tosky_ has joined #tripleo07:18
*** jpich has joined #tripleo07:21
*** tosky has quit IRC07:21
*** ramishra has quit IRC07:26
*** ramishra has joined #tripleo07:28
*** florianf has joined #tripleo07:33
*** jaosorior has joined #tripleo07:35
*** gfidente has joined #tripleo07:37
*** d0ugal has quit IRC07:38
*** tosky_ is now known as tosky07:38
*** jpena|off is now known as jpena07:40
openstackgerritMerged openstack/tripleo-ci master: Copy repo file in tripleo-repos role  https://review.opendev.org/68130207:46
openstackgerritMerged openstack/diskimage-builder master: Use x86 architeture specific grub2 packages for RHEL  https://review.opendev.org/68188907:52
openstackgerritMerged openstack/diskimage-builder master: Move doc related modules to doc/requirements.txt  https://review.opendev.org/62846607:52
*** apetrich has joined #tripleo07:52
*** bogdando has joined #tripleo07:52
*** d0ugal has joined #tripleo07:54
*** ykarel is now known as ykarel|lunch07:56
*** Tengu has quit IRC07:59
*** tkajinam_ has quit IRC08:01
*** Tengu has joined #tripleo08:03
*** gfidente has quit IRC08:06
openstackgerritRabi Mishra proposed openstack/tripleo-common master: healthcheck: List udp ports with ss  https://review.opendev.org/68198908:09
openstackgerritRadoslaw Smigielski proposed openstack/python-tripleoclient stable/queens: [FFU] Fix sensitive hostname in FFU upgrade run command  https://review.opendev.org/68193008:11
*** ccamacho has joined #tripleo08:14
openstackgerritRabi Mishra proposed openstack/tripleo-heat-templates master: Remove containers before removing associated storage  https://review.opendev.org/68164708:15
openstackgerritAndreas Jaeger proposed openstack/diskimage-builder master: Only install doc requirements if needed  https://review.opendev.org/68199108:17
*** ccamacho has joined #tripleo08:18
*** gfidente has joined #tripleo08:25
*** chem has joined #tripleo08:25
*** gfidente has quit IRC08:39
openstackgerritCarlos Camacho proposed openstack/tripleo-ansible master: Improving ReaR configuration parameters (exclusion path and networking changes)  https://review.opendev.org/68073508:40
*** gfidente has joined #tripleo08:43
ccamachofolks!!! Happy programmers day!!!!!08:43
*** alexmcleod has joined #tripleo08:43
*** suuuper has joined #tripleo08:45
*** derekh has joined #tripleo08:45
jaosorioryay :D08:46
*** jaosorior has quit IRC09:00
*** ykarel|lunch is now known as ykarel09:08
*** suuuper has quit IRC09:09
*** suuuper has joined #tripleo09:10
*** udesale has joined #tripleo09:15
*** suuuper has quit IRC09:19
*** matbu has quit IRC09:36
*** matbu has joined #tripleo09:43
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/rocky: Resolve broken zaqar container caused by logging issues  https://review.opendev.org/68200709:43
*** iurygregory has joined #tripleo09:43
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates stable/queens: Resolve broken zaqar container caused by logging issues  https://review.opendev.org/68200909:44
openstackgerritSaravanan KR proposed openstack/tripleo-ansible master: Add kernelargs and tuned core configuration  https://review.opendev.org/67980909:46
*** udesale has quit IRC09:56
*** udesale has joined #tripleo09:57
*** tkajinam has joined #tripleo10:05
openstackgerritSaravanan KR proposed openstack/tripleo-ansible master: Add kernelargs and tuned core configuration  https://review.opendev.org/67980910:07
*** hberaud is now known as hberaud|lunch10:08
openstackgerritLuca Miccini proposed openstack/tripleo-heat-templates master: Enable deep_compare by default for stonith resources  https://review.opendev.org/68181310:09
*** pcaruana has quit IRC10:11
*** saneax has joined #tripleo10:19
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers  https://review.opendev.org/68201510:22
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add registry authentication parameters to pull ceph related containers  https://review.opendev.org/67769910:33
openstackgerritNatal Ngétal proposed openstack/tripleo-upgrade master: Remove the free-form.  https://review.opendev.org/66353910:38
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers  https://review.opendev.org/68201510:44
openstackgerritCarlos Camacho proposed openstack/tripleo-ansible master: Improving ReaR configuration parameters (exclusion path and networking changes)  https://review.opendev.org/68073510:46
owalshratailor, skramaja: could you hold off merging https://review.opendev.org/681159?11:02
owalsh... until EmilienM and I get the existing validation working11:02
*** surpatil has quit IRC11:02
ratailorowalsh, should I set workflow -1 ?11:02
skramajaowalsh: removed the workflow...11:03
ratailorskramaja, owalsh Thanks!11:03
*** ansmith has joined #tripleo11:04
*** surpatil has joined #tripleo11:08
*** jbadiapa has quit IRC11:08
owalshratailor, skramaja: thanks... now to figure out why nova-conductor healthcheck isn't working...11:13
owalshratailor: once the existing tasks are all ok will need to update that patch I expect11:14
ratailorowalsh, ack.11:14
ratailorowalsh, Thanks!11:14
openstackgerritRajesh Tailor proposed openstack/tripleo-heat-templates master: Add ability to collocate pinned and unpinned instance on same host  https://review.opendev.org/68120911:19
lmiccinio/ mwhahaha EmilienM should I cherry pick https://review.opendev.org/#/c/680751/ to the old branches? puppet-openstack-unit-4.8-centos-7 failing with 'Evaluation Error: Error while evaluating a Function Call, Class[Sensu]: parameter 'client_address' expects a String value, got Undef'11:19
*** iurygregory has quit IRC11:23
*** pcaruana has joined #tripleo11:24
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add registry authentication parameters to pull ceph related containers  https://review.opendev.org/67769911:30
*** hberaud|lunch is now known as hberaud11:32
owalshsshnaidm|rover: hey, what's the best way to reproduce a failing CI job nowadays? README-reproducer.html?11:32
*** lucasagomes has joined #tripleo11:34
*** rh-jelabarre has joined #tripleo11:39
sshnaidm|roverowalsh, yes11:39
*** sshnaidm|rover is now known as sshnaidm|off11:40
*** udesale has quit IRC11:40
owalshack, thanks11:40
*** udesale has joined #tripleo11:41
*** jpena is now known as jpena|lunch11:48
*** zaneb has quit IRC11:48
*** zaneb has joined #tripleo11:49
*** apetrich has quit IRC12:08
openstackgerritRabi Mishra proposed openstack/tripleo-common master: healthcheck: List udp ports with ss  https://review.opendev.org/68198912:08
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Fix nova-conductor healthcheck RPC port  https://review.opendev.org/68203212:14
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Wait for first healthcheck before running validation tasks  https://review.opendev.org/68144212:14
openstackgerritLuca Miccini proposed openstack/puppet-tripleo stable/stein: Fix tripleo::profile::base::monitoring::sensu rspec test  https://review.opendev.org/68203412:15
*** apetrich has joined #tripleo12:16
openstackgerritRajesh Tailor proposed openstack/tripleo-puppet-elements master: Add proper permissions on script  https://review.opendev.org/68203612:19
*** ratailor has quit IRC12:20
*** tkajinam has quit IRC12:25
*** jaosorior has joined #tripleo12:27
pkopecHi, does anybody see this issue with recent tripleo-ansible-inventory ? https://paste.fedoraproject.org/paste/Sx~KXupC3Hi7vwuJ1BuSRw12:31
openstackgerritMerged openstack/tripleo-common stable/stein: healthcheck_port: run ss with both sudo & root as best effort  https://review.opendev.org/68195312:31
openstackgerritMerged openstack/puppet-tripleo master: Enable deep_compare for fencing resource  https://review.opendev.org/68177812:31
openstackgerritMerged openstack/tripleo-heat-templates stable/stein: Resolve broken zaqar container caused by logging issues  https://review.opendev.org/68195412:31
*** Goneri has joined #tripleo12:32
*** rlandy has joined #tripleo12:35
*** abishop has joined #tripleo12:38
owalshpkopec: need to run it to generate the inventory file and pass that to ansible12:39
EmilienMlmiccini: yes12:39
lmicciniEmilienM: thanks! I'll do the cherry dance then12:40
openstackgerritLuca Miccini proposed openstack/puppet-tripleo stable/rocky: Fix tripleo::profile::base::monitoring::sensu rspec test  https://review.opendev.org/68203712:40
openstackgerritLuca Miccini proposed openstack/puppet-tripleo stable/queens: Fix tripleo::profile::base::monitoring::sensu rspec test  https://review.opendev.org/68203812:40
*** ksambor has quit IRC12:41
*** rf0lc0 has joined #tripleo12:44
pkopecowalsh, thanks, works well with static file, but I could swear it worked passing only executable to ansible...12:44
*** jbadiapa has joined #tripleo12:44
ykarelpkopec, yes right it used to work with script12:45
ykarelhttps://review.opendev.org/#/q/topic:bp/split-control-plane-unified-inventory+(status:open+OR+status:merged) seems to have caused issue12:46
EmilienMowalsh: what a mess, hopefully we can land these patches this weekend12:46
EmilienMno need to say we'll need them to stable/stein12:46
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: [WIP] Collectd-sensubility  https://review.opendev.org/68203912:50
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: [WIP] Collectd-sensubility  https://review.opendev.org/68203912:51
openstackgerritArx Cruz proposed openstack/tripleo-quickstart master: Adapt os_tempest in FS001  https://review.opendev.org/67340012:54
*** EmilienM is now known as EvilienM12:55
openstackgerritMartin Mágr proposed openstack/tripleo-common master: Activate health check script for qdrouterd image  https://review.opendev.org/67296912:56
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: Enable health check for metrics_qdr container  https://review.opendev.org/67297012:57
*** mcornea has joined #tripleo12:57
*** jpena|lunch is now known as jpena12:59
weshay|ruckfultonj jistr bootcamp13:00
* jistr joining13:00
openstackgerritMartin Kopec proposed openstack/ansible-role-collect-logs master: WIP: Create infrared plugin  https://review.opendev.org/68204213:02
*** morazi has joined #tripleo13:08
openstackgerritPiotr Kopec proposed openstack/tripleo-heat-templates master: Remove libvirt packaged dependencies  https://review.opendev.org/68109213:10
openstackgerritSaravanan KR proposed openstack/tripleo-ansible master: Add kernelargs and tuned core configuration  https://review.opendev.org/67980913:10
*** Tengu has quit IRC13:11
*** Tengu has joined #tripleo13:11
*** paramite has quit IRC13:14
*** aakarsh|2 has quit IRC13:15
*** iurygregory has joined #tripleo13:16
*** dsneddon has quit IRC13:18
owalshykarel, pkopec: raise a bug then, that patch shouldn't have any effect on the existing behaviour13:24
owalshalthough it looks like ansible is trying to read the python as yaml instead of running it so not convinced it's related13:25
*** florianf_ has joined #tripleo13:26
*** jaosorior has quit IRC13:26
*** Vorrtex has joined #tripleo13:26
*** florianf has quit IRC13:26
owalshykarel, pkopec: need to enable the script inventory plugin, I expect that's the change13:26
ykarelowalsh, that's enabled by default afai checked last13:27
owalshykarel: ah, yea, it just failed "You defined a group 'Undercloud' with bad data for the host list"13:27
*** ade_lee has joined #tripleo13:30
owalshykarel, fultonj: needs to be default=[os.environ.get('TRIPLEO_PLAN_NAME')] here https://review.opendev.org/#/c/664065/12/scripts/tripleo-ansible-inventory@4913:30
owalshpkopec: FYI ^^^13:30
pkopecowalsh, ack13:31
cloudnullmornings13:33
owalshpkopec: if that works could you push a quick fix?13:33
pkopecowalsh, sure, testing this right now13:33
*** ade_lee has quit IRC13:33
ykarelowalsh, ack, hmm seems that's it13:34
*** ade_lee has joined #tripleo13:34
*** chem` has joined #tripleo13:36
*** chem has quit IRC13:37
fultonjowalsh: /me looking13:39
pkopecowalsh, fultonj confirmed, it works when default is changed to list13:40
owalshEvilienM: did the recheck happen for https://review.opendev.org/681869, I don't see any zuul jobs13:42
*** Vorrtex has quit IRC13:42
fultonjpkopec: do you need help from me or are you able to take care of it?13:42
EvilienMowalsh: it's running in check13:43
EvilienMshould finish in 14min13:43
EvilienMhttp://zuul.openstack.org/stream/38e8621482b54d75904de755d59cec51?logfile=console.log13:43
owalshEvilienM: bah, zuul status I had open was just stuck13:43
EvilienMthe tripleo-ci-centos-7-containers-multinode is about to finish with success13:43
pkopecfultonj, well it's a oneliner i think , i'll post patch and let you know13:45
fultonjpkopec: ok, thanks. i'll be happy to review13:46
*** aakarsh|2 has joined #tripleo13:46
owalshEvilienM: for stable/stein healthchecks I assume we need https://review.opendev.org/680115, so we will need to backport everything13:48
*** dsneddon has joined #tripleo13:49
EvilienMowalsh: ah right13:49
EvilienMowalsh: I'm doing it13:50
EvilienMowalsh: oh but there is no validate tasks13:51
EvilienMunless there are being backported13:51
cloudnullanyone around want to give https://review.opendev.org/#/c/681876/ a nudge13:51
cloudnullshould help us with some of our downstream backports for dumb-init13:51
fultonjcloudnull: yep13:52
cloudnullthanks mwhahaha EvilienM fultonj13:52
cloudnullwhile I have your attention https://review.opendev.org/#/c/677237/ :)13:52
EvilienMso demanding13:52
* cloudnull hides 13:52
EvilienMsend BTC13:52
cloudnull1 satoshi coming at you13:53
*** dsneddon has quit IRC13:53
owalshEvilienM: hopefully not, but we'll need to backport the healthcheck sudo fixes and the conductor RPC port fix I think13:54
EvilienMowalsh: ok, i won't submit anything on my side13:54
openstackgerritPiotr Kopec proposed openstack/tripleo-validations master: Fix tripleo-ansible-inventory  https://review.opendev.org/68205513:55
owalshEvilienM: ack, I'll check with ratailor next week re the validation tasks, not sure if he was planning to backport13:56
EvilienMok13:56
*** ksambor has joined #tripleo13:57
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Add IPv6 condition to set to the local_address  https://review.opendev.org/68205714:00
*** ratailor has joined #tripleo14:00
*** ykarel is now known as ykarel|afk14:01
*** dsneddon has joined #tripleo14:04
*** jaganathan has quit IRC14:06
*** gbarros has joined #tripleo14:07
*** raildo has joined #tripleo14:08
*** ykarel|afk has quit IRC14:10
*** dsneddon has quit IRC14:11
*** surpatil has quit IRC14:11
*** dsneddon has joined #tripleo14:17
cloudnullweshay|ruck would you be able to join the #openstack-ansible-sig channel?14:17
weshay|ruckaye14:19
*** radeks has quit IRC14:21
dpeacockEvilienM: How can we get heat reviews showing up in this channel for openstackgerrit bot?14:23
*** ratailor has quit IRC14:24
EvilienMdpeacock: no, there is a #heat channel for that14:24
openstackgerritRonelle Landy proposed openstack/tripleo-ci master: Use the common setting for overcloud_container_cli  https://review.opendev.org/67930814:24
*** dsneddon has quit IRC14:24
dpeacockEvilienM: oh, I thought it was all under this remit now.  Ack - thanks.14:24
*** panda|ruck has quit IRC14:28
*** panda has joined #tripleo14:29
*** hberaud is now known as hberaud|school-r14:29
*** xek has joined #tripleo14:32
openstackgerritJames Slagle proposed openstack/tripleo-common stable/queens: Raise an error if deployment_name cannot be determined  https://review.opendev.org/68206714:33
*** marios has quit IRC14:34
*** dsneddon has joined #tripleo14:34
*** xek_ has joined #tripleo14:35
EvilienMcloudnull: i remember you started to poke at converting the container-puppet.py to ansible, any progress on that?14:36
cloudnullnope. however, i was just talking with Tengu about that before he went on holiday14:36
*** marios has joined #tripleo14:37
cloudnullthe TLDR is I found out I didnt really understand what all was going on behind the scenes14:37
*** xek has quit IRC14:37
cloudnullnow I have a better understand of some of the things, so I think its time we give it another look14:37
cloudnullwhen Tengu is back from PTO we'll be talking about it more14:37
*** marios has quit IRC14:38
*** marios has joined #tripleo14:38
*** hberaud|school-r is now known as hberaud14:39
nkinderweshay|ruck, so there is still something not right with the TLS setup for the ovsdb-server, which is causing those connetion reset errors in the fs039 job - https://logs.rdoproject.org/72/21672/2/check/tripleo-ci-centos-7-ovb-1ctlr_2comp_1supp-featureset039/478d25c/logs/overcloud-controller-0/var/log/containers/openvswitch/ovsdb-server-nb.log.txt.gz14:40
weshay|rucknkinder aye.. /me log diving as well14:40
nkinderweshay|ruck, the errors lead me to believe that we have some enable_ssl flag set to true (which it shoudl be), but the cert and key settings are not provided14:41
*** dsneddon has quit IRC14:41
*** rcernin has quit IRC14:42
*** dsneddon has joined #tripleo14:43
nkinderxek_, do you know if there is anything special about how the certs are handled for OVS?  We're seeing this in CI - https://logs.rdoproject.org/72/21672/2/check/tripleo-ci-centos-7-ovb-1ctlr_2comp_1supp-featureset039/478d25c/logs/overcloud-controller-0/var/log/containers/openvswitch/ovsdb-server-nb.log.txt.gz14:44
weshay|rucknkinder not seeing much here: https://www.diffchecker.com/UpsiMwBf14:45
weshay|ruckmaybe   "OS::TripleO::Undercloud::Net::SoftwareConfig": "{{ undercloud_templates_path }}/net-config-undercloud.yaml"14:45
openstackgerritAlan Bishop proposed openstack/puppet-tripleo master: Support deploying multiple Cinder Pure Storage backends  https://review.opendev.org/68113814:46
xek_nkinder, it looks like the cert and key files are missing14:46
weshay|ruckah .. you added that14:46
openstackgerritAlan Bishop proposed openstack/tripleo-heat-templates master: Support deploying multiple Cinder Pure Storage backends  https://review.opendev.org/68113714:46
xek_nkinder, failed to load client certificates from /etc/openvswitch/cacert.pem14:46
nkinderweshay|ruck, I think this is more likely something that is broken in t-h-t or puppet when TLS is enabled, resulting in the cert files not being in the right place for OVS14:47
weshay|ruckk14:47
nkinderxek_, yeah, CA, key, and cert all seem to not be set14:47
nkinderxek_, this is in master, and we had some other patches break TLS14:48
nkinderxek_, we've fixed some issues, but this is the latest14:48
xek_nkinder, there were 2 patches for ovn14:49
xek_nkinder, I see one of them merged https://review.opendev.org/#/c/674849/14:49
xek_nkinder, but the other one didn't https://review.opendev.org/#/c/680345/14:49
*** cylopez has quit IRC14:49
nkinderxek_, aha, we can add a depends-on to test with that14:50
*** dsneddon has quit IRC14:50
nkinderxek_, that looks like it.  It is the SB and NB db services that are not configured properly, which is what that patch is for14:51
nkinderweshay|ruck, ^^^ want to add the patch dep, or want me to do it?14:51
weshay|rucknkinder this one? https://review.opendev.org/#/c/680345/14:52
nkinderyes14:52
weshay|ruckdoing it now14:52
weshay|ruckjob kicked14:53
*** gbarros has quit IRC14:53
nkinderweshay|ruck++14:53
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: HA: fix <service>_restart_bundle with minor update workflow  https://review.opendev.org/68178214:55
*** yprokule has quit IRC15:01
openstackgerritMerged openstack/tripleo-docs master: fix the spelling mistakes  https://review.opendev.org/68172115:07
*** ykarel|afk has joined #tripleo15:16
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Use the Ansible role for tripleo-packages.  https://review.opendev.org/67862015:21
*** dsneddon has joined #tripleo15:22
*** tesseract has quit IRC15:25
*** jtomasek has quit IRC15:28
*** udesale has quit IRC15:30
*** udesale has joined #tripleo15:31
*** weshay|ruck has quit IRC15:31
*** gbarros has joined #tripleo15:31
*** dsneddon has quit IRC15:36
*** jfrancoa has quit IRC15:38
*** ykarel|afk is now known as ykarel|away15:39
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-ci master: Fallback to public IPv4 when there is no priv. IP  https://review.opendev.org/68209215:40
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: Introduce new module for Paunch  https://review.opendev.org/68209315:40
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: Introduce new module for Paunch  https://review.opendev.org/68209315:42
*** artom has quit IRC15:43
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-ci master: Fallback to public IPv4 when there is no priv. IP  https://review.opendev.org/68209215:47
*** gfidente is now known as gfidente|afk15:50
*** lmiccini has quit IRC15:50
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: Introduce new module for Paunch  https://review.opendev.org/68209315:50
*** ramishra has quit IRC15:51
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - deploy-steps-tasks: replace paunch CLI by ansible paunch module  https://review.opendev.org/68209415:55
*** rlandy is now known as rlandy|brb16:02
*** dsneddon has joined #tripleo16:07
*** florianf_ has quit IRC16:07
*** alexmcleod has quit IRC16:08
*** owalsh has quit IRC16:11
*** dsneddon has quit IRC16:12
*** weshay has joined #tripleo16:13
*** marios has quit IRC16:13
*** weshay is now known as weshay|ruck16:15
*** jpich has quit IRC16:18
*** rpittau is now known as rpittau|afk16:22
*** iurygregory has quit IRC16:24
*** owalsh has joined #tripleo16:29
*** hberaud is now known as hberaud|gone16:32
*** owalsh has quit IRC16:34
*** dsneddon has joined #tripleo16:35
*** bogdando has quit IRC16:37
*** kopecmartin is now known as kopecmartin|off16:38
*** derekh has quit IRC16:40
*** rlandy|brb is now known as rlandy16:42
*** bnemec is now known as beekneemech16:48
*** jpena is now known as jpena|off16:49
*** morazi has quit IRC16:53
*** gbarros has quit IRC16:53
*** owalsh has joined #tripleo16:54
*** gbarros has joined #tripleo17:01
*** artom has joined #tripleo17:03
openstackgerritSergii Golovatiuk proposed openstack/tripleo-upgrade master: Explicitly make a list  https://review.opendev.org/68211517:03
openstackgerritSergii Golovatiuk proposed openstack/tripleo-upgrade master: Explicitly make a list  https://review.opendev.org/68211517:06
*** xek_ has quit IRC17:08
openstackgerritSergii Golovatiuk proposed openstack/tripleo-upgrade master: Explicitly make a list  https://review.opendev.org/68211517:11
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers  https://review.opendev.org/68201517:14
*** holser has quit IRC17:15
*** chem` has quit IRC17:22
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers  https://review.opendev.org/68201517:25
openstackgerritRyan McCabe proposed openstack/tripleo-heat-templates master: [WIP] Disable legacy telemetry by default  https://review.opendev.org/68148517:29
openstackgerritMerged openstack/tripleo-ansible master: Enhance docker check verifying its socket existence.  https://review.opendev.org/68144417:30
openstackgerritRyan McCabe proposed openstack/tripleo-heat-templates master: [WIP] Disable legacy telemetry by default  https://review.opendev.org/68148517:31
openstackgerritRyan McCabe proposed openstack/tripleo-heat-templates master: [WIP] Disable legacy telemetry by default  https://review.opendev.org/68148517:34
*** udesale has quit IRC17:36
openstackgerritRabi Mishra proposed openstack/tripleo-common master: healthcheck: List udp ports with ss  https://review.opendev.org/68198917:53
*** panda has quit IRC18:02
*** panda has joined #tripleo18:03
*** nhicher has quit IRC18:15
*** nhicher has joined #tripleo18:15
nkinderweshay|ruck, looks like we're further, but still broken around the OVN TLS setup - https://logs.rdoproject.org/72/21672/3/check/tripleo-ci-centos-7-ovb-1ctlr_2comp_1supp-featureset039/a303d79/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz18:38
nkinderweshay|ruck, pretty sure this is what is breaking it - https://review.opendev.org/#/c/674849/18:40
nkinder...and that last depends-on you added is for a WIP fix that is not complete18:40
*** saneax has quit IRC18:44
*** holser has joined #tripleo18:46
*** xek_ has joined #tripleo18:54
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Resolve broken zaqar container caused by logging issues  https://review.opendev.org/68200718:56
openstackgerritMerged openstack/puppet-tripleo stable/stein: Fix tripleo::profile::base::monitoring::sensu rspec test  https://review.opendev.org/68203418:56
*** lucasagomes has quit IRC18:59
openstackgerritMerged openstack/tripleo-heat-templates master: Skip systemd healthcheck validation on docker  https://review.opendev.org/68186919:08
openstackgerritMerged openstack/tripleo-common stable/queens: Add ENTRYPOINT to the dumb init install block  https://review.opendev.org/68187619:09
openstackgerritMerged openstack/ansible-role-tripleo-modify-image master: Add docs for updating from local gating repo  https://review.opendev.org/68178119:09
openstackgerritMerged openstack/puppet-tripleo stable/rocky: Fix tripleo::profile::base::monitoring::sensu rspec test  https://review.opendev.org/68203719:09
openstackgerritMerged openstack/puppet-tripleo stable/queens: Fix tripleo::profile::base::monitoring::sensu rspec test  https://review.opendev.org/68203819:09
openstackgerritAlex Schultz proposed openstack/paunch master: Improve volume validation  https://review.opendev.org/68214119:13
openstackgerritNathan Kinder proposed openstack/tripleo-heat-templates master: WIP: Support connecting OVN DB over SSL  https://review.opendev.org/68034519:15
openstackgerritMerged openstack/tripleo-ansible master: Move OVN VIP to all_nodes and consider if OVN is configured for a separate VIP  https://review.opendev.org/67816819:18
* weshay|ruck looks, sorry was at lunch19:21
weshay|rucknkinder, well to test the theory we can put up a revert and add yet another depends-on19:22
nkinderweshay|ruck, nah, I fixed the OVN patch and kicked the job19:22
weshay|ruckk k19:22
* nkinder is hoping to see this pass CI before the weekend19:22
*** artom has quit IRC19:26
openstackgerritMerged openstack/puppet-tripleo master: Add missing response to Redis tcp-check in HAProxy  https://review.opendev.org/68138519:30
openstackgerritMerged openstack/tripleo-common stable/rocky: Use blockinfile for tripleo-ssh-known-hosts  https://review.opendev.org/68108119:30
openstackgerritMerged openstack/tripleo-common stable/queens: Use blockinfile for tripleo-ssh-known-hosts  https://review.opendev.org/68108219:33
openstackgerritMerged openstack/tripleo-heat-templates master: Fix nova-conductor healthcheck RPC port  https://review.opendev.org/68203219:33
*** holser__ has joined #tripleo19:34
*** holser has quit IRC19:34
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates stable/stein: Fix nova-conductor healthcheck RPC port  https://review.opendev.org/68214619:41
EvilienMweshay|ruck: FYI approving https://review.opendev.org/#/c/681442/19:42
weshay|ruckEvilienM,  aye it passed this time :)19:42
*** slaweq has quit IRC19:52
*** lucasagomes has joined #tripleo20:00
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - deploy-steps-tasks: replace paunch CLI by ansible paunch module  https://review.opendev.org/68209420:00
*** panda has quit IRC20:05
*** panda has joined #tripleo20:08
jschlueterhttps://review.opendev.org/#/c/68068820:10
jschlueterhow did this get cherry-picked to queens but skip stein and rocky?20:11
*** aakarsh|2 has quit IRC20:11
*** gfidente|afk has quit IRC20:11
jschlueterEvilienM, apetrich: ^^20:11
EvilienMjschlueter: no idea20:12
openstackgerritEmilien Macchi proposed openstack/tripleo-common stable/stein: Stop sending execution object via Zaqar message  https://review.opendev.org/68215120:12
openstackgerritEmilien Macchi proposed openstack/tripleo-common stable/rocky: Stop sending execution object via Zaqar message  https://review.opendev.org/68215220:12
EvilienMdone20:13
jschlueterEvilienM: thanks20:13
*** ykarel|away has quit IRC20:18
*** weshay|ruck has quit IRC20:19
*** morazi has joined #tripleo20:21
*** ksambor has joined #tripleo20:23
*** hjensas has quit IRC20:24
*** hjensas has joined #tripleo20:28
*** morazi has quit IRC20:30
*** csmart has quit IRC20:41
*** nhicher has quit IRC20:42
*** nhicher has joined #tripleo20:43
*** csmart has joined #tripleo20:50
*** openstackgerrit has quit IRC20:51
*** holser__ has quit IRC21:02
*** rh-jelabarre has quit IRC21:02
*** pcaruana has quit IRC21:03
*** holser has joined #tripleo21:04
*** slaweq has joined #tripleo21:11
*** rf0lc0 has quit IRC21:12
*** raildo has quit IRC21:13
*** slaweq has quit IRC21:16
*** openstackgerrit has joined #tripleo21:21
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: WIP: ensure nova-api is running before starting nova-compute  https://review.opendev.org/68104221:21
*** mcornea has quit IRC21:22
*** fultonj has quit IRC21:25
*** lucasagomes has quit IRC21:30
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - deploy-steps-tasks: replace paunch CLI by ansible paunch module  https://review.opendev.org/68209421:31
openstackgerritAlex Schultz proposed openstack/paunch master: Improve volume validation  https://review.opendev.org/68214121:32
*** holser has quit IRC21:36
mwhahahacloudnull: https://review.opendev.org/#/c/681933/ is in conflict btw21:37
*** jbadiapa has quit IRC21:37
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: Introduce new module for Paunch  https://review.opendev.org/68209321:40
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - deploy-steps-tasks: replace paunch CLI by ansible paunch module  https://review.opendev.org/68209421:40
EvilienMcloudnull: ^ ready for a second review, I got local deployment working with it :)21:41
*** xek_ has quit IRC21:46
*** xek has joined #tripleo21:47
*** xek has quit IRC21:49
*** xek has joined #tripleo21:49
*** EvilienM is now known as EmilienM21:52
*** xek_ has joined #tripleo21:52
*** xek has quit IRC21:55
*** slaweq has joined #tripleo22:11
*** slaweq has quit IRC22:16
*** holser has joined #tripleo22:33
*** rlandy has quit IRC22:38
*** Goneri has quit IRC22:41
*** pkopec has quit IRC22:46
*** holser has quit IRC22:48
*** xek_ has quit IRC22:59
*** tosky has quit IRC23:23
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add local_address to the default hieradata on all hosts  https://review.opendev.org/68193323:23
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: WIP: ensure nova-api is running before starting nova-compute  https://review.opendev.org/68104223:29
*** rcernin has joined #tripleo23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!