Wednesday, 2019-10-09

*** ansmith_ has quit IRC00:23
openstackgerritMerged openstack/tripleo-ansible master: podman_container: command now accepts lists  https://review.opendev.org/68737900:30
*** saneax has quit IRC00:30
*** fultonj has joined #tripleo00:36
openstackgerritMerged openstack/python-tripleoclient stable/stein: Don't look for execution in payload  https://review.opendev.org/68708900:52
openstackgerritMerged openstack/python-tripleoclient stable/rocky: Don't look for execution in payload  https://review.opendev.org/68709000:52
openstackgerritMerged openstack/python-tripleoclient stable/queens: Don't look for execution in payload  https://review.opendev.org/68709100:52
openstackgerritMerged openstack/tripleo-heat-templates master: Collectd-sensubility  https://review.opendev.org/68203900:52
openstackgerritMerged openstack/puppet-tripleo stable/rocky: Initial support for tls_priorities  https://review.opendev.org/68724200:52
openstackgerritMerged openstack/tripleo-heat-templates stable/rocky: Support TLS priorities for pacemaker  https://review.opendev.org/68724600:54
openstackgerritMerged openstack/tripleo-heat-templates master: Add DHCPv6 rules + set ipversion for dhcp rules  https://review.opendev.org/68438500:54
openstackgerritMerged openstack/tripleo-ci master: Revert "move standalone-scenario010 to non-voting"  https://review.opendev.org/68718700:54
*** mschuppert has quit IRC01:04
*** HardCase has joined #tripleo01:15
*** ayoung has quit IRC01:19
*** ayoung has joined #tripleo01:20
*** toure is now known as toure|gone01:22
*** tkajinam_ has joined #tripleo01:26
*** tkajinam has quit IRC01:28
*** ykarel|away has joined #tripleo01:45
*** apetrich has quit IRC02:10
*** psachin has joined #tripleo02:12
*** ayoung has quit IRC02:19
*** ayoung has joined #tripleo02:19
*** ricolin has joined #tripleo02:21
HardCaseAnyone online that might be able to tell me how to reset a corrupt galera root password in my overcloud?  I ctrl+c'd a deployment and it seemed to have messed up the root password02:24
openstackgerritMerged openstack/tripleo-upgrade stable/stein: Disable overcloud image validation  https://review.opendev.org/68729902:30
openstackgerritMerged openstack/tripleo-common master: Use tenancity to backoff when layer is locked  https://review.opendev.org/68619302:30
openstackgerritMerged openstack/tripleo-quickstart master: Use custom ceph-ansible until c8 storage sig ready  https://review.opendev.org/68544902:30
openstackgerritMerged openstack/ansible-role-collect-logs master: add a check for selinux  https://review.opendev.org/68733002:30
openstackgerritMerged openstack/tripleo-quickstart master: RHEL 8 upstream needs to be permissive  https://review.opendev.org/68736102:30
openstackgerritMerged openstack/tripleo-heat-templates master: Fix Manila when run without pacemaker  https://review.opendev.org/68285302:30
openstackgerritMerged openstack/tripleo-common master: Make executor type dynamic  https://review.opendev.org/68618102:36
openstackgerritMerged openstack/paunch master: builder: allow to pass a dict to 'environment'  https://review.opendev.org/68719802:36
*** ramishra has joined #tripleo02:37
*** psachin has quit IRC03:17
*** psachin has joined #tripleo03:19
*** morazi has quit IRC03:27
*** ramishra has quit IRC03:43
*** fultonj has joined #tripleo03:55
*** udesale has joined #tripleo03:55
*** fultonj has quit IRC03:55
*** raukadah is now known as chandankumar03:58
*** ykarel|away has quit IRC04:01
*** fultonj has joined #tripleo04:07
*** fultonj has quit IRC04:07
*** fultonj has joined #tripleo04:08
*** fultonj has joined #tripleo04:08
*** ykarel|away has joined #tripleo04:22
*** ykarel|away is now known as ykarel04:22
*** skramaja has joined #tripleo04:31
*** lmiccini has joined #tripleo04:38
*** saneax has joined #tripleo04:42
*** rh-jelabarre has quit IRC04:51
*** ayoung has quit IRC04:55
*** ayoung has joined #tripleo04:55
*** ramishra has joined #tripleo04:59
*** ayoung has quit IRC05:12
*** ayoung has joined #tripleo05:13
*** marios has joined #tripleo05:14
*** soniya29 has joined #tripleo05:22
*** dsneddon has quit IRC05:27
*** dsneddon has joined #tripleo05:29
*** ksambor has joined #tripleo05:30
*** ksambor has joined #tripleo05:31
*** ksambor has joined #tripleo05:32
*** ratailor has joined #tripleo05:40
*** HardCase has quit IRC05:52
*** matbu has quit IRC06:00
*** kopecmartin|off is now known as kopecmartin06:07
*** slaweq has joined #tripleo06:08
*** jfrancoa has joined #tripleo06:09
*** jbadiapa has joined #tripleo06:29
*** d0ugal has quit IRC06:32
*** jaosorior has quit IRC06:33
*** mschuppert has joined #tripleo06:34
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo stable/queens: Initial support for tls_priorities  https://review.opendev.org/68745906:39
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Move fs020 to os_tempest from train onwards  https://review.opendev.org/68746006:39
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates stable/queens: Support TLS priorities for pacemaker  https://review.opendev.org/68746106:39
*** jtomasek has joined #tripleo06:50
*** jtomasek has quit IRC06:52
triple-oh-noobbeagles: hmmm. Then how do I get it on my overcloud via tripleo? :-/06:53
*** jtomasek has joined #tripleo07:04
*** jaosorior has joined #tripleo07:04
*** xek_ has joined #tripleo07:07
*** pcaruana has joined #tripleo07:10
*** tesseract has joined #tripleo07:13
*** tosky has joined #tripleo07:14
*** saneax has quit IRC07:20
*** bogdando has joined #tripleo07:21
*** iurygregory has joined #tripleo07:21
*** hamdyk has joined #tripleo07:22
*** florianf has joined #tripleo07:23
*** apetrich has joined #tripleo07:24
*** ayoung has quit IRC07:24
*** ccamacho has joined #tripleo07:24
*** ayoung has joined #tripleo07:25
*** shyamb has joined #tripleo07:33
shyambHi07:34
*** holser has joined #tripleo07:34
shyambHow to limit maximum memory usage by an OpenStack service container in RHOSP13?07:34
*** rpittau|afk is now known as rpittau07:35
shyambWe want to implement same for TrilioVault containers in rhosp1307:35
shyambDo we have any options in heat templates? or puppet modules?07:36
*** surpatil has joined #tripleo07:41
*** slaweq has quit IRC07:42
*** ykarel is now known as ykarel|lunch07:44
*** jpena|off is now known as jpena07:46
*** shyamb has quit IRC07:48
*** hberaud|gone is now known as hberaud07:48
*** lucasagomes has joined #tripleo07:49
*** holser has quit IRC07:49
*** dtantsur|afk is now known as dtantsur07:49
*** holser has joined #tripleo07:49
*** slaweq has joined #tripleo07:49
*** ayoung has quit IRC07:52
*** cylopez has joined #tripleo07:53
*** ayoung has joined #tripleo07:54
*** tosky has quit IRC07:58
*** tosky has joined #tripleo07:59
*** soniya29 has quit IRC08:02
*** chem|eod is now known as chem08:02
openstackgerritSergii Golovatiuk proposed openstack/tripleo-common stable/stein: Add support for uploading local image  https://review.opendev.org/67920708:08
*** tkajinam_ has quit IRC08:10
*** cylopez has quit IRC08:12
*** cylopez has joined #tripleo08:13
*** mdbooth has quit IRC08:15
openstackgerritpengyuesheng proposed openstack/diskimage-builder master: Blacklist sphinx 2.1.0 (autodoc bug)  https://review.opendev.org/68747508:16
*** shyamb has joined #tripleo08:19
*** alexmcleod has joined #tripleo08:22
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: WIP - Molecule Ansible lint  https://review.opendev.org/68747908:22
*** pkopec has joined #tripleo08:24
*** ramishra_ has joined #tripleo08:27
*** ramishra has quit IRC08:29
openstackgerritSaravanan KR proposed openstack/tripleo-common master: Consider default service disable state for image builder  https://review.opendev.org/68748408:29
*** saneax has joined #tripleo08:32
*** derekh has joined #tripleo08:33
*** ykarel|lunch is now known as ykarel08:34
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Remove OpenDaylight templates and environments  https://review.opendev.org/68263008:35
*** soniya29 has joined #tripleo08:39
Tenguykarel: heya! apparently a job was put back as voting in the gate, and it's blocking stable/stein: https://github.com/openstack/tripleo-ci/commit/7ff162c7031a52685c8a949f3c4d13d9c54c7c9e // blocking https://review.opendev.org/68678408:41
Tenguykarel: this might be a big issue, preventing new merge in stable/stein :/. Would it be possible to revert asap weshay|ruck patch?08:41
Tengumarios: -^^ as well :)08:42
holserTengu I am here as well08:44
mariosTengu: is scenario10 stein a known one see Tengu ping ^^^ looks borked here http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario010-standalone&branch=stable%2Fstein08:45
mariosarxcruz|rover: |^^^08:45
mariosTengu: ;) sorry should have been for arxcruz|rover ... checking their etherpad not sure when that went voting08:46
mariosTengu: but indeed it is https://github.com/openstack/tripleo-ci/blob/7ff162c7031a52685c8a949f3c4d13d9c54c7c9e/zuul.d/standalone-jobs.yaml#L552-L553 git blame there will tell us anyway08:46
mariosTengu: looks like it happend last night08:47
mariosTengu: https://review.opendev.org/#/c/687187/08:47
mariosarxcruz|rover: we need revert revert and maybe a new job for stein that can be non voting with master voting08:48
mariosTengu: ok i'm gonna propose the revert and we can rever revert revert later if needed ;)08:49
mariosTengu: if its blocking you and also holser ?08:49
marios+ anyone else wanting to merge to stein08:49
holsermarios it's blocking https://review.opendev.org/#/c/686784/08:49
holserlook at chain08:50
holserthere are a lot of patches waiting for merge08:50
shyambTengu: owalsh: how can we limit memory usage by an openstack service container in osp13?08:50
openstackgerritMarios Andreou proposed openstack/tripleo-ci master: Revert "Revert "move standalone-scenario010 to non-voting""  https://review.opendev.org/68749708:50
mariosholser: Tengu arxcruz|rover weshay|ruck ^^08:50
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Replace wget with curl  https://review.opendev.org/68749808:50
Tengumarios: apparently it's needed only in stable/stein08:50
Tengumarios: there's a parallel discussion on #rdo with Yatin08:51
mariosTengu: yeah ok i see... well maybe we need a new job then ... like a child job with voting false only for stein08:51
mariosTengu: but for now revert is ok since it blocks08:51
Tengumarios: revert in master will also affect stein? no branching in tripleo-ci?08:51
mariosTengu: right08:52
Tenguok!08:52
ykarelyes jobs should be voting only if they are in promotion pipeline08:52
mariosykarel: ack did they not add that there yet?08:52
ykareland i think it's only added for Train promotion pipeline08:52
mariosykarel: so there's another reason for revert08:52
mariosykarel: i c08:52
Tengumarios: used my core on that one, just missing a +A08:53
mariosi'll just be sure to be out on an errand when weshay|ruck wakes up. I'll leave him a not 'Tengu did it'08:53
*** jaosorior has quit IRC08:53
marioss/not/note08:53
Tengumarios: meh08:53
TenguI'll be sure to avoid pgm + cix calls ;).08:54
owalshshyamb: looks like you can't - added in stein https://review.opendev.org/61590908:54
mariosi guess this is good practice rumour has it i'm ruck|rovering tomorrow for the new sprint08:54
marios\o/08:54
shyambokay08:54
ykarelmarios, yes it's added to promotion criteria only for train/master:- https://review.rdoproject.org/r/#/c/23019/08:54
mariosthanks ykarel08:54
shyambowalsh: thanks08:54
ykareljob was added for stein also https://review.rdoproject.org/r/#/c/22867/2/zuul.d/standalone-jobs.yaml08:55
ykarelbut good to find what is causing the isue in stein08:55
shyambI want to implement same for out containers08:55
*** d0ugal has joined #tripleo08:56
shyambowalsh: Fix is in paunch08:58
shyambSo, I think we can not implement for containers in queens08:59
shyambI meant for our containers08:59
shyambat-least08:59
owalshshyamb: could backport the paunch patch maybe, mmethot_ WDYT?08:59
openstackgerritRabi Mishra proposed openstack/puppet-tripleo stable/stein: Remove ovn specific logic for octavia  https://review.opendev.org/68750109:00
ramishra_Tengu, holser I hope ^^^ would fix the octavia helathcheck issue09:01
shyambowalsh: that would be great09:01
*** gfidente|afk has quit IRC09:01
ramishra_for stable/stein09:01
shyambowalsh: Anything I can do through heat template?09:01
holserramishra_ looks like you are right09:01
shyambowalsh: https://github.com/trilioData/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover.yaml09:02
shyambThis is my heat template for trilio-datamover service09:02
shyambsitting on compute node09:02
holserramishra_ +1 backport to stable/stein09:02
shyambowalsh: kolla_config section can help?09:03
*** ayoung has quit IRC09:04
*** florianf has quit IRC09:05
arxcruz|rovermarios: Tengu hi, so, scenario010 was failing, so we moved to non-voting09:06
arxcruz|roveri believe the problem was fixed, and wes move it to voting09:06
arxcruz|roversomething happen during this time ?09:06
*** ayoung has joined #tripleo09:07
Tenguarxcruz|rover: apparently octavia healthchecks are failing, making a test in oooq-extras fail because it detects failed systemd units.09:07
mariosarxcruz|rover: see commit message stein is borked https://review.opendev.org/68749709:09
arxcruz|rovermarios: ack09:10
*** xek_ has quit IRC09:11
cgoncalvesoctavia health check fix: https://review.opendev.org/#/q/4c3d5182f4f924e931be8641f3b7df2d31c6892709:12
*** gfidente has joined #tripleo09:14
owalshshyamb: hrmm, well that's run inside the container so I doubt you can change the cgroup limits. I guess you could set a limit via ulimit09:17
ramishra_cgoncalves: I don't think it's healthcheck issue, it's failing to start octavia-api09:18
ramishra_we should merge https://review.opendev.org/#/c/687501/ in stable/stein instead09:19
cgoncalvesramishra_, thanks. agreed. I've just left a comment in https://review.opendev.org/#/c/687497/09:20
*** shyamb has quit IRC09:21
*** shyamb has joined #tripleo09:22
shyambowalsh: okay09:22
shyambDocker does not recommended that09:24
shyambhandling memory limits at container level is the good way I believe09:24
shyambAny plan to backport the paunch fix?09:25
shyambto queens09:25
shyambowalsh: if we backport this fix, is this memory limits configurable?09:25
shyambthrough heat env files like any other parameter09:26
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-quickstart-extras master: Use cache for yum update in containers prepare  https://review.opendev.org/68393609:26
owalshshyamb: yea, the docker_config gets passed straight through to paunch09:27
owalshso just add mem_limit to it...09:27
shyambowalsh: I see, great09:28
shyambthanks09:28
shyambowalsh: any plan to backport the fix to queens?09:28
*** cylopez has quit IRC09:28
owalshshyamb: no idea, but I doubt anybody will object if you really need it (and it works ;-)09:29
shyambgreat09:29
shyambyes, we really need it09:29
shyambowalsh: in production env, our datamover component must consume most of the memory09:30
shyambIt sits on compute nodes09:30
shyambso, we need to have this feature in place09:31
*** jaosorior has joined #tripleo09:31
owalshshyamb: don't the VMs consume most of the memory?09:33
shyambowalsh: yes, but openstack services does also consume memory09:34
shyambany they may go out of limit09:34
owalshshyamb: well maybe due to python being lazy about reclaiming memory, but they shouldn't leak memory09:36
owalshshyamb: but we do tell nova not to allocate some of the memory to the VMs, so are you thinking of setting that as the limit?09:38
shyambowalsh: that is good09:39
shyambWe want to implement something on same line for trilio containers09:39
shyambtrilio containers should use memory beyond a defined limit09:40
owalshshyamb: but if you set a cgroup limit and the service exceeds it then it would just fail with a malloc error09:41
owalshshyamb: so really the limit would just prevent memory leaks growing indefinitely09:43
openstackgerritHarald JensÃ¥s proposed openstack/tripleo-common master: Get undercloud name in get_undercloud_registry()  https://review.opendev.org/68730509:46
openstackgerritJose Luis Franco proposed openstack/tripleo-upgrade stable/queens: Handle when ceph_osds is a dictionary or a list.  https://review.opendev.org/68465609:49
shyambowalsh: okay09:49
shyambwhat would be the best way you are suggesting here09:50
shyambshould we wait for paunch fix or implement something on our side?09:51
owalshshyamb: depends on what's happening... does the memory usage keep growing forever?09:58
openstackgerritSaravanan KR proposed openstack/tripleo-common master: Consider default service disable state for image builder  https://review.opendev.org/68748409:59
*** surpatil has quit IRC09:59
*** shyamb has quit IRC09:59
openstackgerritDougal Matthews proposed openstack/tripleo-ansible master: Add a playbook for the "openstack overcloud delete" CLI Command  https://review.opendev.org/68752410:01
*** d0ugal has quit IRC10:04
openstackgerritTom Barron proposed openstack/tripleo-heat-templates stable/stein: Fix Manila when run without pacemaker  https://review.opendev.org/68509710:11
openstackgerritTom Barron proposed openstack/tripleo-heat-templates stable/stein: Fix Manila when run without pacemaker  https://review.opendev.org/68509710:15
*** shyamb has joined #tripleo10:21
shyambowalsh: It's memory leak10:22
owalshshyamb: I mean is it an actual memory leak in the service e.g if it's python a circular reference or is it the python memory management not releasing memory (which it tends to do for lots of small temporary allocations)10:24
shyambowalsh: not sure in terms of these details10:25
shyamblet me paste you the jira description10:25
shyambowalsh: here it is:10:26
shyambhttp://paste.openstack.org/show/782292/10:26
owalshshyamb: not a bug, cache buffer is controlled by the kernel. It will release it if/when it needs it10:27
shyambWe fixed this by adding MaxMemory parameter in service init file10:29
shyambfor other openstack distros10:29
shyambowalsh: https://github.com/trilioData/triliovault-cfg-scripts/blob/98b621fa13a651917fe8a31be2d5d5cb3620a5ab/ansible/roles/ansible-tvault-contego-extension/templates/contego_service.j2#L710:31
shyambthis was our fix for ansible deployed openstacks10:31
owalshshyamb: unless you are hitting the OOM killer due to this it's not a bug - perfectly normal kernel I/O caching - if something more important needs the memory it gets freed10:35
*** dsneddon has quit IRC10:35
shyambowalsh: Great, thanks for this10:35
shyambI will confirm with our dev team10:36
*** soniya29 has quit IRC10:36
hjensasarxcruz|rover: weshay|ruck: anything I can do to get https://bugs.launchpad.net/tripleo/+bug/1845225 moving? I don't quite understand sshnaidm's comment regarding updating the script. Afict the rpm pulled in point to stein repos. Maby it's the bmc-template image in third party CI cloud that need to be updated?10:37
openstackLaunchpad bug 1845225 in tripleo "OVB master branch job is broken, can't create bmc" [High,Triaged]10:37
hjensasarxcruz|rover: weshay|ruck: ie. image ID c1f9afa1-2f96-4264-8e06-5c111a9966ba in rdocloud?10:38
*** ayoung has quit IRC10:38
*** ayoung has joined #tripleo10:39
*** florianf has joined #tripleo10:41
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Move fs020 to os_tempest from train onwards  https://review.opendev.org/68746010:42
*** tesseract has quit IRC10:44
*** tesseract has joined #tripleo10:46
arxcruz|roverhjensas: checking10:48
*** rcernin has quit IRC10:48
owalshshyamb: ack, could you add a comment to https://bugs.launchpad.net/paunch/+bug/1801949 if it turns out that the large page-cache is causing problems?10:49
openstackLaunchpad bug 1801949 in paunch "containers using too much memory" [Medium,Triaged] - Assigned to Marc Methot (mb-methot)10:49
*** d0ugal has joined #tripleo10:52
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Migrate the openstack overcloud delete command to use Ansible  https://review.opendev.org/68753110:53
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Remove workflows that were only used by openstack overcloud delete  https://review.opendev.org/68753310:55
shyambowalsh: sure10:57
*** dsneddon has joined #tripleo11:04
arxcruz|roverhjensas: i need more time to understand the code :/ i'm not familiar with it11:07
hjensasarxcruz|rover: thanks, I'll keep digging as well. We are in the same situation regarding familiarity. :)11:08
*** ratailor has quit IRC11:18
*** dtantsur is now known as dtantsur|brb11:22
*** udesale has quit IRC11:22
*** morazi has joined #tripleo11:26
*** jpena is now known as jpena|lunch11:29
*** ramishra_ has quit IRC11:30
*** raildo has joined #tripleo11:30
Tenguowalsh: heya :). If you have a spare minute just to (re³)validate https://review.opendev.org/686388 - I've made some more doc, and integrated Tobias comment - even if I really don't like doing that way.. but as long as it can be merged and it's working, I'm fine with it.11:32
*** hberaud is now known as hberaud|lunch11:33
*** lmiccini has quit IRC11:34
*** shyamb has quit IRC11:35
*** d0ugal has quit IRC11:37
*** dsneddon has quit IRC11:41
owalshTengu: ack, looking11:43
Tenguowalsh: thanks :)11:45
*** ramishra has joined #tripleo11:49
*** apetrich has quit IRC11:50
*** ansmith_ has joined #tripleo11:51
*** ekultails has joined #tripleo11:54
*** jaosorior has quit IRC11:58
*** dsneddon has joined #tripleo12:00
*** xek_ has joined #tripleo12:03
*** apetrich has joined #tripleo12:03
*** epoojad1 has joined #tripleo12:04
*** hberaud|lunch is now known as hberaud12:05
holserdalvarez hey12:10
*** ayoung has quit IRC12:12
ramishraEmilienM: Hey, can you approve https://review.opendev.org/#/c/687501/ to unblock stable/stein gate (broken scenario010-standalone fixed with that backport)12:14
EmilienMramishra: looking12:14
*** ayoung has joined #tripleo12:15
*** apetrich has quit IRC12:18
*** moguimar has joined #tripleo12:21
*** moguimar has left #tripleo12:22
*** jpena|lunch is now known as jpena12:23
*** cylopez has joined #tripleo12:25
*** suuuper has joined #tripleo12:26
*** jfrancoa has quit IRC12:26
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Track uploaded layers for workers as global view  https://review.opendev.org/68728812:28
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Never block or wait when workers upload images  https://review.opendev.org/68675612:28
*** mcornea has joined #tripleo12:28
*** jaosorior has joined #tripleo12:28
*** rh-jelabarre has joined #tripleo12:29
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Increase overcloud deploy timeout to 120  https://review.opendev.org/68754612:30
*** psachin has quit IRC12:31
*** jfrancoa has joined #tripleo12:32
*** csmart has quit IRC12:33
cloudnullmornings12:34
*** csmart has joined #tripleo12:36
slaglegood morning12:38
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Convert container environment from a list to a dict  https://review.opendev.org/68729312:38
*** ayoung has quit IRC12:38
*** bcafarel has quit IRC12:39
*** bcafarel has joined #tripleo12:40
*** ayoung has joined #tripleo12:40
dalvarezholser: yo12:42
*** toure|gone is now known as toure12:46
*** aakarsh has quit IRC12:49
*** cfontain_ has joined #tripleo12:56
*** gbarros has joined #tripleo12:59
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Fix dynamic executor appending to objects  https://review.opendev.org/68754812:59
*** epoojad1 has quit IRC13:03
*** chem has quit IRC13:03
*** chem has joined #tripleo13:03
*** dsneddon has quit IRC13:04
*** apetrich has joined #tripleo13:05
*** slaweq has quit IRC13:07
*** slaweq has joined #tripleo13:10
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Tighten up how we add and remove kernel modules  https://review.opendev.org/68741913:10
*** ayoung has quit IRC13:12
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Track uploaded layers for workers as global view  https://review.opendev.org/68728813:12
*** d0ugal has joined #tripleo13:13
*** slaweq_ has joined #tripleo13:13
*** gbarros has quit IRC13:14
*** ayoung has joined #tripleo13:14
*** slaweq has quit IRC13:14
*** gbarros has joined #tripleo13:17
*** jcoufal has joined #tripleo13:19
*** aakarsh has joined #tripleo13:21
*** cfontain_ has quit IRC13:22
openstackgerritArx Cruz proposed openstack/tripleo-heat-templates master: Moving tripleo-ci-centos-7-scenario010-standalone to nv  https://review.opendev.org/68755313:23
*** ayoung has quit IRC13:26
openstackgerritJames Slagle proposed openstack/tripleo-docs master: Update DCN docs.  https://review.opendev.org/68755513:26
*** ayoung has joined #tripleo13:27
*** bnemec has quit IRC13:29
*** slaweq_ is now known as slaweq13:29
*** dsneddon has joined #tripleo13:30
*** bnemec has joined #tripleo13:31
*** pcaruana has quit IRC13:31
*** jaosorior has quit IRC13:33
openstackgerritArx Cruz proposed openstack/tripleo-common stable/stein: Moving tripleo-ci-centos-7-scenario010-standalone to nv  https://review.opendev.org/68755813:33
*** pcaruana has joined #tripleo13:33
openstackgerritLuke Short proposed openstack/tripleo-ansible master: Install packages and load kernel modules before configuring sysctl.  https://review.opendev.org/68708413:33
openstackgerritMerged openstack/puppet-tripleo stable/queens: Initial support for tls_priorities  https://review.opendev.org/68745913:34
*** dsneddon has quit IRC13:36
openstackgerritArx Cruz proposed openstack/tripleo-heat-templates stable/stein: Moving tripleo-ci-centos-7-scenario010-standalone to nv  https://review.opendev.org/68755913:36
openstackgerritArx Cruz proposed openstack/python-tripleoclient stable/stein: Moving tripleo-ci-centos-7-scenario010-standalone to nv  https://review.opendev.org/68756013:42
*** saneax has quit IRC13:43
*** udesale has joined #tripleo13:45
*** bdodd has joined #tripleo13:49
*** rf0lc0 has joined #tripleo13:51
*** otherwiseguy has quit IRC13:51
*** otherwiseguy has joined #tripleo13:52
*** bcrochet has joined #tripleo13:54
*** chem_ has joined #tripleo13:54
*** tonyb has quit IRC13:56
*** elGamal has quit IRC13:56
*** chem has quit IRC13:56
*** bfournie has quit IRC13:56
*** rfolco has quit IRC13:56
*** jidar has quit IRC13:56
*** mrda has quit IRC13:56
*** thrash has quit IRC13:56
*** apetrich has quit IRC13:56
*** cylopez has quit IRC13:56
*** ekultails has quit IRC13:56
*** lucasagomes has quit IRC13:56
*** fultonj has quit IRC13:56
*** rakhmerov has quit IRC13:56
*** migi has quit IRC13:56
*** mjturek has quit IRC13:56
*** bcrochet is now known as thrash13:56
*** fultonj has joined #tripleo13:56
*** cdearborn has joined #tripleo13:56
*** elGamal has joined #tripleo13:56
*** mjturek has joined #tripleo13:56
*** jidar has joined #tripleo13:56
*** gbarros has quit IRC13:58
*** openstackstatus has quit IRC13:58
*** Vorrtex has joined #tripleo13:58
VorrtexHey all.  Still looking to get the reviews in this filter merged.  Been a few months, got one more +W a couple weeks ago and still no traction.  Any way I can get some eyes on them?  https://review.opendev.org/#/q/status:open+branch:stable/queens+topic:queens/multiarch-support13:59
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Update config to support RHEL/CENT8  https://review.opendev.org/68757113:59
*** apetrich has joined #tripleo14:00
*** ekultails has joined #tripleo14:00
*** gbarros has joined #tripleo14:00
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Support TLS priorities for pacemaker  https://review.opendev.org/68746114:01
*** rf0lc0 is now known as rfolco14:02
*** jaosorior has joined #tripleo14:03
*** dtantsur|brb is now known as dtantsur14:05
*** dsneddon has joined #tripleo14:05
*** mcornea has quit IRC14:05
*** Vorrtex has quit IRC14:07
*** slaweq has quit IRC14:07
*** Vorrtex has joined #tripleo14:07
*** rascasoft has quit IRC14:08
*** slaweq has joined #tripleo14:09
*** lucasagomes has joined #tripleo14:11
*** rascasoft has joined #tripleo14:11
*** markllama has joined #tripleo14:12
*** mcornea has joined #tripleo14:12
mwhahahaweshay|ruck: https://review.opendev.org/#/c/686784/ scenario010 nonvoting in check, voting in gate14:16
*** abishop_ has joined #tripleo14:16
weshay|ruckmwhahaha, arxcruz|rover is on that atm ^14:16
mwhahahawait didn't we enable it (should we have limited it to master)14:16
*** Goneri has joined #tripleo14:16
weshay|ruckarxcruz|rover, please respond14:17
*** abishop has quit IRC14:17
arxcruz|roverweshay|ruck: mwhahaha  https://review.opendev.org/#/q/topic:scen10-nv+(status:open+OR+status:merged)14:17
mwhahahawaht won't pull it from the gate14:18
mwhahahas/waht/that14:18
*** apetrich has quit IRC14:18
mwhahahaweshay|ruck: plz remove -2 from https://review.opendev.org/#/c/687497/14:19
mwhahahawe need to land that14:19
arxcruz|rovermwhahaha: sorry, this is the first time i'm doing this in branch, what am i missing ?14:20
mwhahahawe need to revert the voting patch and adjust it14:20
mwhahahathe problem is that has been added to the gate there so your other patches don't remove it from the gate14:20
mwhahahawe would need to limit the gate job to just master14:20
mwhahahaso right now we should revert it and fix the configs (it will take a bunch of patches)14:21
weshay|ruckok.. so remove my -2 on the revert? we can't just fix it?14:22
openstackgerritJames Slagle proposed openstack/tripleo-common master: Fix etcd healthcheck  https://review.opendev.org/68715314:22
mwhahahano we can't14:22
weshay|ruckk14:22
*** asbishop has joined #tripleo14:22
mwhahahaand you're blocking stein14:22
mwhahahaso revert and fix correctly14:22
Tenguowalsh: just addressed your comments.14:22
weshay|ruckarxcruz|rover, please stay on this until we're good14:22
weshay|ruckmwhahaha, master should be voting, stein should be nv correct?14:23
mwhahahayes?14:24
mwhahahai don't think that scenario should be added as voting to the scenarios template14:24
weshay|ruckah.. so break it out of the template14:24
mwhahahai'm looking at it to figur eit out14:24
mwhahahait might just be adding a branch limiter to the gate job14:24
weshay|ruckthat would make it easier and more clear14:24
*** abishop_ has quit IRC14:24
weshay|ruckmwhahaha, so arx was going into tripleo stein repos and marking the job nv there14:25
weshay|ruckwhich also fixes this issue imho14:25
mwhahahathis is the problem with not branching the ci config :/14:25
mwhahahathose non-voting patches should be ok for check, but didn't remove it from gate14:25
weshay|ruckso.. tripleo-ci has always been unbranched, but we could break the zuul config out of tripleo-ci into it's own repo14:25
mwhahahaso we need to limit the gate to master, though it might be better not to add it to that template and controll it selwhere14:25
weshay|ruckin these cases.. my preference is to handle it on the repo branches.. even though that is several patches.. at least how it's setup now14:26
*** apetrich has joined #tripleo14:27
weshay|ruckmwhahaha, my concern is watch scen10 break on master as we do this14:27
mwhahahatough14:28
weshay|ruckok14:28
weshay|ruckarxcruz|rover, so the key here is to get this all done quickly :)14:28
*** asbishop has quit IRC14:30
weshay|ruckmwhahaha, can you please describe for arxcruz|rover how you want to see the zuul patches go14:32
*** ayoung has quit IRC14:36
*** hamdyk has quit IRC14:37
*** ayoung has joined #tripleo14:38
*** udesale has quit IRC14:41
*** artom has joined #tripleo14:42
*** ayoung has quit IRC14:49
*** ayoung has joined #tripleo14:51
*** udesale has joined #tripleo14:54
openstackgerritTom Barron proposed openstack/tripleo-heat-templates stable/stein: DNM - Test Run Manila in scenario004 without pacemaker  https://review.opendev.org/68758015:05
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Quote commands run by "bash" when doing podman exec  https://review.opendev.org/68758115:08
*** jaosorior has quit IRC15:11
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Fix dynamic executor appending to objects  https://review.opendev.org/68754815:11
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Track uploaded layers for workers as global view  https://review.opendev.org/68728815:12
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Never block or wait when workers upload images  https://review.opendev.org/68675615:12
*** ayoung has quit IRC15:12
*** ayoung has joined #tripleo15:13
*** udesale has quit IRC15:14
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Track uploaded layers for workers as global view  https://review.opendev.org/68728815:14
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Never block or wait when workers upload images  https://review.opendev.org/68675615:14
*** iurygregory has quit IRC15:20
*** ramishra has quit IRC15:21
openstackgerritMerged openstack/puppet-tripleo stable/stein: Remove ovn specific logic for octavia  https://review.opendev.org/68750115:23
*** slaweq has quit IRC15:26
*** ykarel is now known as ykarel|afk15:26
*** jfrancoa has quit IRC15:28
*** slaweq has joined #tripleo15:35
*** skramaja has quit IRC15:35
*** slaweq has quit IRC15:44
*** jfrancoa has joined #tripleo15:44
*** abishop has joined #tripleo15:44
*** rpittau is now known as rpittau|afk15:45
*** florianf has quit IRC15:45
*** iurygregory has joined #tripleo15:47
cloudnullif folks can review https://review.opendev.org/#/q/topic:deconstruct/container-puppet+(status:open) I would greatly appreciate it15:51
*** openstackgerrit has quit IRC15:52
*** openstackgerrit has joined #tripleo15:53
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: WIP - Molecule Ansible lint  https://review.opendev.org/68747915:53
*** paramite has quit IRC15:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/tripleo-heat-templates master: Remove all old pcmklatest images  https://review.opendev.org/68760515:54
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Track uploaded layers for workers as global view  https://review.opendev.org/68728815:58
openstackgerritBogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Never block or wait when workers upload images  https://review.opendev.org/68675615:58
*** gbarros has quit IRC15:59
*** ayoung has quit IRC16:01
*** ayoung has joined #tripleo16:02
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Update config to support RHEL/CENT8  https://review.opendev.org/68757116:03
*** ccamacho has quit IRC16:03
*** lucasagomes has quit IRC16:05
*** ayoung has quit IRC16:09
*** ayoung has joined #tripleo16:10
*** ayoung has quit IRC16:14
*** ayoung has joined #tripleo16:17
*** holser has quit IRC16:17
weshay|ruckmwhahaha, so shall I kill https://review.opendev.org/#/c/687497/16:19
weshay|ruckit's in the gate atm16:19
* weshay|ruck is confused w/ what you want to do here16:19
mwhahahayea that's fine16:23
mwhahahasince we fixed stein16:23
mwhahahawe'll see if we need to address it but it's likely ok now16:23
*** derekh has quit IRC16:28
dking_desktopAttempting to build an undercloud with container_images_file gives: NoSuchOptError: no such option container_images_file in group [DEFAULT]16:29
dking_desktopI didn't expect this as container_images_file seems to be in the [DEFAULT] section even in the sample undercloud.conf. Could I be missing something?16:29
openstackgerritMichael Bayer proposed openstack/tripleo-heat-templates master: Add SQLAlchemy-collectd support  https://review.opendev.org/67217316:29
mwhahahadking_desktop: sounds like you have the wrong config for teh version16:30
dking_desktopOkay, I can look into that further. What should the variable be, or where should it be in stein?16:32
mwhahahastein it is probably docker_images_files16:32
mwhahahawe renamed docker_* to container_*16:32
*** ykarel|afk is now known as ykarel16:35
*** marios has quit IRC16:36
*** ramishra has joined #tripleo16:37
dking_desktopThanks. I'll take a look around for that. This is a brand new setup and has only ever had Stein on it, soI might need to find out why that's in the undercloud.conf.sample. It's funny, though, it gets accepted fine in the overcloud deploy, except that there would be nothing listening to pull from.16:39
*** bnemec has quit IRC16:45
*** bnemec has joined #tripleo16:47
dking_desktopmwhahaha: You are fairly certain that this shouldn't be in the config for stien? It looks like that template was updated with that variable back in pike.16:48
*** slaweq has joined #tripleo16:48
mwhahahai'd have to check the code16:49
mwhahahayou sur eit's not contianer_image_file and not files16:49
weshay|ruckk. thanks dude16:51
*** jpena is now known as jpena|off16:53
*** panda is now known as panda|off16:53
dking_desktopYep. But thanks for the sanity check. For a moment, I doubted myself, though. I got it from the online documentation, and then later, double checked from the sample.16:55
mwhahahayea no idea16:56
*** slaweq has quit IRC17:02
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/python-tripleoclient master: Ensure that --output-env-file handles path expansion  https://review.opendev.org/68762617:06
dking_desktopIt looks to me like undercloud_config.py, line 826 expects conf.container_images_file, but that line is actually what causes the error to be thrown from oslo_config/cfg.py, line 2191. I think that's enough to convince me. I'll look a little closer and maybe file a bug report.17:09
mwhahahaso it's a valid config option, we use it in upstream ci17:10
mwhahahaso it's odd that you're getting an error related to that17:10
dking_desktopYeah, I saw that. I'll look a little more into how oslo_config and see if I can figure out what it's doing. It's in the basic deploy documentation, so I would imagine lots of people are doing the same thing, and it's possible I'm doing something wrong, but I'd like to know what.17:13
mwhahahait's not likely an oslo config problem17:14
mwhahahawe've seen this for other reasons17:14
mwhahahai don't suppose you have any selinux related errors17:14
*** sri_ has joined #tripleo17:16
*** alexmcleod has quit IRC17:18
dking_desktopNope, none that I've seen. I moved on without it, and I'm now working on something going on in the Ceph setup, but that's on the overcloud nodes.17:21
openstackgerritTerry Wilson proposed openstack/tripleo-ansible master: Install packages and load kernel modules before configuring sysctl.  https://review.opendev.org/68708417:21
dking_desktopI agree, probably not something with oslo_config itself, but I just don't know enough about that module to know why it's throwing that error.17:22
bnemecGenerally speaking, oslo config would raise that because an opt was referenced in the code without being registered first.17:23
weshay|ruckade_lee, can you take a look at https://bugs.launchpad.net/tripleo/+bug/184738517:26
openstackLaunchpad bug 1847385 in tripleo "rhel 8 master fails to create cm-local-ca.pem, Error: 'openssl pkcs12 " [Critical,Triaged]17:26
openstackgerritMerged openstack/tripleo-docs master: Add a bindep.  https://review.opendev.org/68663717:27
*** ramishra has quit IRC17:27
*** gbarros has joined #tripleo17:27
ade_leeweshay|ruck, ack - will do17:31
weshay|ruckthanks17:31
weshay|ruckade_lee, it's rhel8 only, does not repro on centos717:32
*** ykarel_ has joined #tripleo17:32
*** ykarel_ has quit IRC17:33
*** hberaud is now known as hberaud|gone17:34
*** ykarel has quit IRC17:35
*** ayoung has quit IRC17:37
*** ayoung has joined #tripleo17:38
*** ricolin has quit IRC17:41
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: podman_container: only add --volume if item not empty  https://review.opendev.org/68763717:43
openstackgerritLuke Short proposed openstack/tripleo-heat-templates stable/stein: Lookup pre/post deployment vars with hostvars  https://review.opendev.org/68763817:46
*** gbarros has quit IRC17:51
*** bfournie has joined #tripleo17:55
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: WIP - Introduce tripleo-container-manage role  https://review.opendev.org/68619617:58
EmilienMmwhahaha, cloudnull : thx guys for the help on ^ I'm making good progress. Nearly an undercloud deployed17:59
*** mcornea has quit IRC18:01
EmilienMcloudnull: when I'm done with meetings I need to sync with you on container_puppet.py script18:02
dking_desktopCan the undercloud.conf file contain a shell environment variable, such as $HOME ?18:05
EmilienMdking_desktop: no, it's a file read by Python18:06
dking_desktopThat's probably it, then. The instructions say: "generating the following containers-prepare-parameter.yaml file and including it in undercloud.conf: container_images_file=$HOME/containers-prepare-parameter.yaml:"18:09
*** paramite has joined #tripleo18:10
*** ayoung has quit IRC18:12
*** dtantsur is now known as dtantsur|afk18:12
*** ayoung has joined #tripleo18:13
*** mcornea has joined #tripleo18:15
sri_Hi team, ironic node clean failing in  HP DL360p Gen8 servers, I've only facing this issue very recently, any idea's how to debug this ?18:17
*** slaweq has joined #tripleo18:20
*** suuuper has quit IRC18:21
*** slaweq has quit IRC18:25
*** raildo has quit IRC18:27
cloudnullEmilienM ++18:27
cloudnulllet me know, i'll be around18:27
*** slaweq has joined #tripleo18:29
*** gbarros has joined #tripleo18:31
*** gbarros has quit IRC18:35
*** slaweq has quit IRC18:39
openstackgerritMichael Johnson proposed openstack/tripleo-heat-templates master: Add the Octavia log offload parameters  https://review.opendev.org/68373118:41
*** gbarros has joined #tripleo18:44
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - deploy steps without paunch  https://review.opendev.org/68765118:44
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible master: WIP - Introduce tripleo-container-manage role  https://review.opendev.org/68619618:45
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: WIP - deploy steps without paunch  https://review.opendev.org/68765118:45
sri_bnemec, Hi If you have min, n ode clean failing in Hp gen8 servers I am only seeing this recently in stable/rocky, I was using this servers last month I didn't see this issue at that time18:51
sri_bnemec, the strange thing is i also have hp gen7 serves, i am able clean this nodes without any issue.18:54
triple-oh-noobsri_: Dare I ask the iLo/firmware version is up to date?19:06
triple-oh-noob(see any red screens after attempts?)19:06
*** pcaruana has quit IRC19:07
sri_triple-oh-noob, ilo/firmware is up to date , no there is no error messages in console output19:08
triple-oh-noobOh. Assuming your ipmi is correctly set up then I will be quiet and let someone who knows what they are talking about get back to you :-)  (Was suggesting from mere experience)19:09
*** mgagne has quit IRC19:11
sri_triple-oh-noob, Yes, I was able to run introspection successfully19:12
bnemecsri_: I'm probably not the one to ask about that. I would try #openstack-ironic.19:13
sri_bnemec, ack, I will try that. Thanks :)19:13
*** slaweq has joined #tripleo19:20
*** tesseract has quit IRC19:20
*** jfrancoa has quit IRC19:21
*** cdearborn has quit IRC19:24
dking_desktopWhen I try to deploy an overcloud using the hyperconverged ceph environment (hyperconverged-ceph.yaml), it fails because /dev/vdb does not exist. That device is set directly in ceph-osd.yaml. What should be creating that or where can I find more information?19:27
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Move basic global functions into ansible tasks  https://review.opendev.org/68328619:27
*** holser has joined #tripleo19:28
mwhahahadking_desktop: shouldn't you change that to match the actual devices you have?19:28
dking_desktopmwhahaha: Good question. I suppose that's what I want to know. I didn't know if there was a place with more information about what sorts of things one needs to check or adjust for that sort of thing.19:29
mwhahahathe defaults that are usually there match what's available in ci19:29
mwhahahathough you'll likely need to adjust them to match your actual environment19:29
mwhahahadking_desktop: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/index.html see ceph sections19:31
mwhahahahttps://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/ceph_config.html#configure-osd-settings-with-ceph-ansible19:32
*** zaneb has quit IRC19:33
*** zbitter has joined #tripleo19:33
*** holser has quit IRC19:33
dking_desktopmwhahaha: Thanks! That might be just what I was looking for. Perhaps, should that be added to https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/install_overcloud.html#deploy-the-overcloud in the Ceph block so that folks following the instructions can know about it?19:34
openstackgerritMerged openstack/tripleo-validations master: Fix ost-tox-lower-constraints job indentation  https://review.opendev.org/68728619:35
*** kopecmartin is now known as kopecmartin|off19:35
*** gbarros has quit IRC19:36
*** gbarros has joined #tripleo19:37
*** iurygregory has quit IRC19:37
openstackgerritMerged openstack/python-tripleoclient master: Handle a deployment that was manually cancelled.  https://review.opendev.org/68620219:39
openstackgerritLuke Short proposed openstack/python-tripleoclient stable/stein: Handle a deployment that was manually cancelled.  https://review.opendev.org/68765919:54
openstackgerritLuke Short proposed openstack/python-tripleoclient stable/rocky: Handle a deployment that was manually cancelled.  https://review.opendev.org/68766119:56
*** holser has joined #tripleo19:56
openstackgerritLuke Short proposed openstack/python-tripleoclient stable/rocky: Handle a deployment that was manually cancelled.  https://review.opendev.org/68766119:57
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: WIP - Molecule Ansible lint  https://review.opendev.org/68747919:58
openstackgerritLuke Short proposed openstack/python-tripleoclient stable/queens: Handle a deployment that was manually cancelled.  https://review.opendev.org/68766219:58
*** jcoufal has quit IRC20:00
*** slaweq has quit IRC20:06
*** ayoung has quit IRC20:06
*** ayoung has joined #tripleo20:08
*** slaweq has joined #tripleo20:11
openstackgerritRafael Folco proposed openstack/tripleo-ci master: Retag w/ _arch suffix rhel8 container images  https://review.opendev.org/68766520:12
openstackgerritMerged openstack/tripleo-docs master: builder-extra-args has been removed  https://review.opendev.org/68736820:13
*** artom has quit IRC20:13
*** slaweq has quit IRC20:16
*** gbarros has quit IRC20:20
cgoncalveshow can one run the same (or close to) update workflow to a typical UC+OC env in a standalone env? and how to re-create?20:25
cgoncalves"openstack tripleo deploy --force-stack-update" seems to have run post_update_tasks I manually set, so I'd like to confirm that is the expected option20:26
openstackgerritMerged openstack/tripleo-common stable/stein: Fixup octavia-api healthcheck  https://review.opendev.org/68618620:26
cgoncalvesalthough, --force-stack-create did not recreate the environment as I still see user-created neutron networks and subnets20:26
mwhahahacgoncalves: just rerunning the deploy command should update it, though you can use --force-stack-update to change the heat action to 'UPDATE'20:28
mwhahahahttps://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L16920:29
mwhahahabut we check if the stack_update_mark exists (it's a file) and will trigger an update if it exists20:29
mwhahahawe use this for the undercloud20:29
cgoncalvesmwhahaha, odd. I had to set --force-stack-update for tripleo to run my post-deploy added post_update_tasks20:31
mwhahahahttps://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1245-L125120:31
mwhahahashould create the mark file20:31
mwhahahaso unless you changed teh stack name from the first run, it should say it's an update20:31
mwhahahaof course we have logging around this so you should be able to see some of this in the logs20:32
*** pcaruana has joined #tripleo20:33
cgoncalvesnop, I did not change the stack name. simple standalone deploy, local file changes, re-deploy20:33
cgoncalvesI confirm I have the update mark file20:33
ade_leeweshay|ruck, hey -- looking at that error.20:34
cgoncalveswhat about clean slate re-deployment?20:34
ade_leeweshay|ruck, I have a rhel 8 system here and that command seems to work just fine.20:34
ade_leeweshay|ruck, is this being run in a container?  I would think not ..20:35
ade_leeweshay|ruck, are there selinux logs?  maybe this is prevented by an AVC?20:36
mwhahahacgoncalves: what do you mean redeployment20:36
*** aakarsh|2 has joined #tripleo20:37
weshay|ruckade_lee, selinux is permissive so that's not it20:37
weshay|ruckade_lee, it's logged in the journal on the host.. so I would assume it's not running in a container20:37
ade_leeweshay|ruck, yeah -- so um?20:38
cgoncalvesmwhahaha, how can one redeploy from an existing env with user-created resources to a new deployment without those resources? without having to tear down my whole standalone VM :)20:39
*** aakarsh has quit IRC20:39
mwhahahacgoncalves: so you're asking how do you nuke your deployment so you can start from scratch again?20:39
cgoncalvesmwhahaha, yes20:39
mwhahahacgoncalves: aka removing all the containers and files20:39
mwhahahayou can fake it by removing all the containers, images and nuking /var/lib/config-data20:39
mwhahahabut it's not exactly the same20:39
cgoncalvesand databases20:40
weshay|ruckade_lee, maybe.. this is the real error20:40
weshay|ruck2019-10-08 20:19:40 |         "<13>Oct  9 00:19:19 puppet-user: Warning: ModuleLoader: module 'pacemaker' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules\\n   (file & line not available)",20:40
weshay|ruckI have to go though.. can work on it tomorrow20:40
mwhahahacgoncalves: so standalone doesn't have a persistent heat instance so that's all rebuilt every time20:40
cgoncalvesmwhahaha, reading some docs I had assumed --force-stack-create would do that20:40
*** aakarsh|2 has quit IRC20:40
mwhahahaweshay|ruck: not related20:40
*** pcaruana has quit IRC20:40
ade_leeweshay|ruck, there do seem to be errors about puppet-user: error: Could not connect to cluster (is it running?)"20:40
mwhahahacgoncalves: so --force-stack-create makes the deployment be equivalent of 'CREATE' and not update20:40
ade_leeweshay|ruck, something about not able to start pcsd20:41
mwhahahacgoncalves: but the containers/config would already exist20:41
*** aakarsh has joined #tripleo20:41
cgoncalvesmwhahaha, ok, understood. thank you20:41
ade_leeweshay|ruck, ah -- >Oct  9 00:19:26 puppet-user: Notice: /Stage[main]/Tripleo::Certmonger::Ca::Local/Exec[extract-and-trust-ca]/returns: Can't open /var/lib/certmonger/local/creds for reading, No such file or directory",20:42
cgoncalvesin that case, I wish there would be an "openstack tripleo deploy --nuke" option :)20:42
mwhahahacgoncalves: patches, welcome (but will likely be -2'd) :D20:42
mwhahahait's easier to create disposable vms tbh20:42
*** Goneri has quit IRC20:42
cgoncalvesgreat encouragement with that -2 part :P20:43
mwhahahaemilien can probably share his vagrant config if you are interested20:43
*** sri_ has quit IRC20:43
cgoncalvesyeah, I have it all automated. thought, though, I could futher optimize it and save some bandwidth and time20:43
mwhahahait depends on what you're targeting really20:43
cgoncalvesI have one already :)20:44
triple-oh-nooboctavia deployment via tripleo - I have included environments/services/octavia.yaml in my deploy, Not sure I understand what network config I have to configure. It's rendering my lab stack as returning 503s for the services now.20:44
cgoncalvesFWIW, https://cgoncalves.pt/trash/openstack/box-template-standalone-v0.tar.gz20:44
*** aakarsh has quit IRC20:46
cgoncalvestriple-oh-noob, enabling octavia in tripleo is really just about adding its environment file you mentioned. tripleo will take care of creating all the required resources (neutron lb-mgmt-network, o-hm0 interface, ...)20:48
cgoncalvestriple-oh-noob, if you have logs you could share, that would help20:48
triple-oh-noobcgoncalves: Any advice in the best placement of the template on the deploy? (ie: At the beginning or at the end?)20:55
ade_leeweshay|ruck, is there a journal ?20:55
cgoncalvestriple-oh-noob, order is irrelevant20:56
ade_leeweshay|ruck, that file is supposed to be created when certmonger starts iirc.  so maybe its not there because certmonger failed to start20:56
ade_leeweshay|ruck, journal might tell us why .. dbus maybe ..20:57
*** mcornea has quit IRC20:57
triple-oh-noobcgoncalves: hmm. It's adding to an existing deployment - does that matter?20:57
cgoncalvestriple-oh-noob, if you want to enable the Octavia service on an existing overcloud deployment, you have to prepare the undercloud. if you don't, it results in the overcloud installation being reported as successful yet without Octavia running21:00
openstackgerritMerged openstack/tripleo-heat-templates stable/stein: Fix Manila when run without pacemaker  https://review.opendev.org/68509721:00
openstackgerritMerged openstack/tripleo-validations master: Fix missing closing parenthesis in skeleton role  https://review.opendev.org/68698521:00
openstackgerritMerged openstack/tripleo-common master: Officially deprecate the skopeo image uploader  https://review.opendev.org/68683121:00
triple-oh-noobcgoncalves: I got errors on the deployment. I saw a mention of that, but didn't quite understand what it as suggesting I needed to do.21:00
cgoncalvestriple-oh-noob, IIRC you have to add the environment file to command "openstack overcloud container image prepare"21:00
triple-oh-noobI can completely destroy it and redeploy completely if needed.21:01
triple-oh-noobcgoncalves: Ah, I see.21:01
cgoncalvestriple-oh-noob, could you please share more details about the error you got?21:01
triple-oh-noobcgoncalves: I will do shortly. I have rebooted since I last deployed - so lost what it was saying. I'm attempting again, so will have something of use shortly.21:02
openstackgerritAlex Schultz proposed openstack/tripleo-common stable/stein: Add support for uploading local image  https://review.opendev.org/67920721:03
openstackgerritAlex Schultz proposed openstack/tripleo-common stable/stein: Always close src/dst sessions and raise to retry  https://review.opendev.org/68678721:03
openstackgerritAlex Schultz proposed openstack/tripleo-common stable/stein: Prefix ansible output with timestamps  https://review.opendev.org/68678821:03
cgoncalvestriple-oh-noob, this step-by-step may help you enable Octavia: https://bugzilla.redhat.com/show_bug.cgi?id=1607567#c721:03
openstackgerritAlex Schultz proposed openstack/tripleo-common stable/stein: Implement threading locks around layers  https://review.opendev.org/68678921:03
openstackbugzilla.redhat.com bug 1607567 in rhosp-director "Upgrading existing OSP13 installation to add Octavia provides "update_complete" but Octavia not installed" [Medium,Closed: worksforme] - Assigned to rhos-maint21:03
openstackgerritAlex Schultz proposed openstack/tripleo-common stable/stein: Improving debug logging  https://review.opendev.org/68679221:03
openstackgerritTom Barron proposed openstack/tripleo-heat-templates stable/rocky: Fix Manila when run without pacemaker  https://review.opendev.org/68767421:03
*** xek_ has quit IRC21:04
*** ayoung has quit IRC21:07
triple-oh-noobcgoncalves: You're too kind.21:07
*** ayoung has joined #tripleo21:08
openstackgerritMerged openstack/instack-undercloud stable/rocky: Fix pep8 errors  https://review.opendev.org/68741221:12
openstackgerritMerged openstack/instack-undercloud stable/rocky: Set option:mtu in inspector dnsmasq  https://review.opendev.org/68734221:12
ade_leeweshay|ruck, I wonder if its a timing issue21:12
ade_leeweshay|ruck, certmonger is started right beforehand21:14
ade_leehttps://logs.rdoproject.org/19/687419/4/openstack-check/tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001/b7138c0/logs/overcloud-controller-0/var/log/journal.txt.gz#_Oct_09_00_19_2121:14
ade_leeweshay|ruck, I bet its a timing thing. seen that recently with another certmonger thing21:14
ade_leeweshay|ruck, file is created on server startup, but prob returns before the file is created21:15
triple-oh-noobTIL I can append --log-file to the deploy.21:16
*** morazi has quit IRC21:18
mwhahahatriple-oh-noob: don't. it doesn't work like you think it does21:20
mwhahahai mean you can, but it's not going to capture the actual deployment output21:20
mwhahahait's still recommended to tee the output to a file at the moment21:20
*** ayoung has quit IRC21:21
triple-oh-noob:) thanks21:21
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Permit access to Ceph RGW for 'member' role  https://review.opendev.org/68768021:23
*** ayoung has joined #tripleo21:24
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Permit access to Ceph RGW for 'member' role  https://review.opendev.org/68768021:24
*** gfidente has quit IRC21:25
*** devep has joined #tripleo21:26
*** rfolco has quit IRC21:28
*** radeks_ has quit IRC21:31
*** Vorrtex has quit IRC21:33
*** pkopec has quit IRC21:41
*** cloudnull is now known as cloudnull-afk21:42
*** ayoung has quit IRC21:43
*** ayoung has joined #tripleo21:44
*** holser has quit IRC21:45
*** rh-jelabarre has quit IRC21:48
openstackgerritMerged openstack/instack-undercloud stable/queens: Fix pep8 errors  https://review.opendev.org/68741321:49
*** holser has joined #tripleo21:50
*** ekultails has quit IRC21:55
*** slaweq has joined #tripleo22:11
openstackgerritMerged openstack/instack-undercloud stable/queens: Set option:mtu in inspector dnsmasq  https://review.opendev.org/68734422:13
*** rcernin has joined #tripleo22:14
*** slaweq has quit IRC22:15
abishopmwhahaha, EmilienM: puppet-glance gate failing due to new version of 'signet' gem released today22:20
mwhahahaRecheck we addressed it22:21
abishopmwhahaha: ack, that was quick :-)22:21
*** thrash has quit IRC22:21
*** thrash has joined #tripleo22:24
*** slaweq has joined #tripleo22:25
*** slaweq has quit IRC22:29
*** holser has quit IRC22:34
openstackgerritMerged openstack/tripleo-common stable/stein: Warn when looking up grandparent resource name  https://review.opendev.org/68650022:43
openstackgerritMerged openstack/tripleo-common stable/rocky: Warn when looking up grandparent resource name  https://review.opendev.org/68650122:43
openstackgerritMerged openstack/tripleo-common stable/queens: Warn when looking up grandparent resource name  https://review.opendev.org/68650322:43
*** devep has quit IRC22:58
*** tkajinam has joined #tripleo23:02
triple-oh-noobcgoncalves: So, deployment failed. Let's see if I can paste some of the output to confirm your suspicions.23:07
*** saneax has joined #tripleo23:14
triple-oh-noobcgoncalves: https://pastebin.com/GSuAyM3c23:18
triple-oh-nooblooks like it's failing on creating the network.23:19
*** openstackstatus has joined #tripleo23:19
*** ChanServ sets mode: +v openstackstatus23:19
triple-oh-noobAlthough, when I try to run a simple openstack network list on my overcloud, it has the same error (or similar)23:20
triple-oh-noobneutron api is in a state of "restarting" it appears on the docker list.23:22
triple-oh-noobIt's just in a state of "starting" for up to 10 seconds then attempts a restart over and over.23:24
*** tosky has quit IRC23:30
triple-oh-noobI may well look to delete the overcloud and attempt a fresh deploy.23:30
triple-oh-noob(Given I can hear my bed whispering my name)23:31
triple-oh-noobhahaha. How come sometimes, I do an "openstack stack delete overcloud" and "Y" and it just drops back to my prompt?23:33
hjensastriple-oh-noob: "openstack stack delete overcloud --wait -y" - or "openstack overcloud delete overcloud -y"23:39
triple-oh-noobhjensas: simply, openstack stack delete overcloud23:40
triple-oh-noob... then "are you sure?" Y23:40
triple-oh-noobIt's working. I Can see the cleanin gout of the nodes etc.23:40
triple-oh-noobI just expect it to wait with me and tell me when it's completed.23:40
hjensastriple-oh-noob: without --wait that will drop yo back to prompt and the stack delete happens in the background23:41
triple-oh-noobI'll run it again when the nodes have cleaned.23:41
triple-oh-nooboooooh.23:41
triple-oh-noobThanks hjensas. I learn many things every day :-)23:41
hjensastriple-oh-noob: the "openstack overcloud delete overcloud" have the wait polling stuff built in. :)23:41
triple-oh-noobooh. Yes, I should have run that really.23:42
triple-oh-noobdarn.23:42
triple-oh-noobOkay, that's sorted. Time for bed. And hopefully when I wake i'll have a shiny new deployment.23:45
triple-oh-noobnn all, thanks for the help.23:47
*** aakarsh has joined #tripleo23:54
*** slaweq has joined #tripleo23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!