Friday, 2018-08-17

openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044500:00
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676400:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726800:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)00:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)00:10
openstackgerritSteve Baker proposed openstack/tripleo-common master: Ensure all image parameters have a default set  https://review.openstack.org/59227500:35
*** zaneb has quit IRC01:05
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676401:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726801:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)01:10
*** slaweq has joined #tripleo01:11
*** slaweq has quit IRC01:16
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Add cleanup service for neutron ovs bridges  https://review.openstack.org/59165101:21
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Revert "SSL support for haproxy -> novnc proxy connection"  https://review.openstack.org/59207301:21
*** tzumainn has quit IRC01:29
*** mschuppert has quit IRC01:29
openstackgerritMerged openstack/puppet-tripleo stable/queens: Revert "SSL support for haproxy -> novnc proxy connection"  https://review.openstack.org/59207201:51
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676402:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726802:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)02:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)02:10
openstackgerritMerged openstack/tripleo-quickstart-extras master: Updated tempest role for rocky branch  https://review.openstack.org/59248902:15
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Default bootstrap_server_id  https://review.openstack.org/58975702:15
*** agopi|out has joined #tripleo02:22
*** agopi|out is now known as agopi02:24
*** tbonds has quit IRC02:33
*** matbu has quit IRC02:37
*** rlandy|bbl is now known as rlandy02:41
*** rlandy has quit IRC02:42
*** jhebden is now known as jhebden-lunch02:43
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676403:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726803:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)03:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)03:10
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: add delorean-current to repolist for updates  https://review.openstack.org/59278403:13
weshay|ruckstevebaker, not sure if I have followed everything here, hoping you can take a look https://review.openstack.org/#/c/592784/103:17
*** gkadam_ has joined #tripleo03:24
openstackgerritzhufl proposed openstack/tripleo-quickstart-extras master: Modify test_add_remove_fixed_ip's path  https://review.openstack.org/59278503:26
stevebakerweshay|ruck: I've replied, it will need a change to tripleo-modify-image first03:26
openstackgerritzhufl proposed openstack/tripleo-quickstart-extras master: Modify test_add_remove_fixed_ip's path  https://review.openstack.org/59278503:27
*** ssbarnea has joined #tripleo03:28
*** udesale has joined #tripleo03:47
*** Petersingh has joined #tripleo03:48
openstackgerritSteve Baker proposed openstack/tripleo-common master: Refactor non-docker upload functions into base class  https://review.openstack.org/59008404:00
openstackgerritSteve Baker proposed openstack/tripleo-common master: Skopeo based uploader  https://review.openstack.org/59008604:00
openstackgerritSteve Baker proposed openstack/tripleo-common master: WIP Make skopeo the default uploader  https://review.openstack.org/59008704:00
*** Petersingh is now known as Petersingh|afk04:08
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676404:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726804:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)04:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)04:10
*** ssbarnea has quit IRC04:12
*** ssbarnea has joined #tripleo04:21
*** skramaja has joined #tripleo04:22
jaosoriorstevebaker: I see you modified the patch. That looks a bit better :). need any other reviews?04:24
*** gkadam_ has quit IRC04:30
*** Petersingh|afk is now known as Petersingh04:31
*** ramishra has joined #tripleo04:32
*** janki has joined #tripleo04:39
*** sri_ has quit IRC04:58
*** sri_ has joined #tripleo05:01
*** apetrich has joined #tripleo05:05
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676405:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726805:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)05:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)05:10
Tenguhello there05:10
*** pdeore has joined #tripleo05:11
jaosorioryooo05:15
*** ratailor has joined #tripleo05:16
*** yprokule has joined #tripleo05:41
ramishrajaosorior: Hi, are we ok to merge https://review.openstack.org/#/c/591980/ now as it's critical for the upgrade tests QE is doing? Can you please review/approve it?05:51
jaosoriorramishra: I'm not particularly fond of that, as we only should do that on upgrade. But I guess we can fix that on a further iteration (as part of the work that bogdan is doing.05:53
ramishrajaosorior: reg you question about using upgrade flag, the legacy_env is merged only when that flag is true in that patch. so should be fine I think05:53
ramishrawhat other solution you've?05:53
ramishraI think answered your question;)05:54
jaosoriorramishra: here, bogdan introduces a flag that indicates if an upgrade is going on https://review.openstack.org/#/c/587431/05:54
ramishrayes, as per my suggestion in that patch once that flag moves to L281 it should be fine05:55
*** ksambor has joined #tripleo05:55
*** Petersingh is now known as Petersingh|afk05:55
jaosoriorramishra: lets iterate on this, lets merge yours, let bogdan fix the merge conflict that will occur, and hopefully with that fix he'll address your comment too.05:56
jaosoriormeanwhile, I'm merging your patch, it allows us to move forward05:56
ramishrajaosorior: yes, this a critical blocker bug, so we should land this05:56
*** mschuppert has joined #tripleo05:58
*** Petersingh|afk is now known as Petersingh06:02
*** aufi has joined #tripleo06:03
*** saneax has joined #tripleo06:06
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676406:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726806:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)06:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)06:10
*** hgogada has joined #tripleo06:13
openstackgerritMartin Schuppert proposed openstack/tripleo-heat-templates master: Add role parameter for rx/tx virtio-net queue size  https://review.openstack.org/59245806:17
*** jfrancoa has joined #tripleo06:22
*** Petersingh is now known as Petersingh|afk06:32
openstackgerritMichele Baldessari proposed openstack/instack-undercloud master: WIP Install ansible-pacemaker by default  https://review.openstack.org/59287706:37
*** links has joined #tripleo06:37
openstackgerritOlivier Bourdon proposed openstack/diskimage-builder master: [DNM] Testing CentOS images builds on Ubuntu Xenial  https://review.openstack.org/59136606:40
*** shyamb has joined #tripleo06:45
openstackgerritmelissaml proposed openstack/puppet-pacemaker master: Remove the duplicated word  https://review.openstack.org/59288006:51
*** pcaruana has joined #tripleo06:54
*** rcernin has quit IRC06:55
*** slaweq has joined #tripleo06:57
*** bogdando has joined #tripleo07:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676407:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726807:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)07:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)07:10
*** amoralej|off is now known as amoralej07:11
shyambowalsh: Hi07:18
*** Petersingh|afk is now known as Petersingh07:28
*** ykarel has joined #tripleo07:31
*** rdopiera has joined #tripleo07:33
openstackgerritMerged openstack/puppet-pacemaker master: Remove PE requirement from metadata.json  https://review.openstack.org/59274507:37
*** shardy has joined #tripleo07:39
*** rdopiera has quit IRC07:39
*** rdopiera has joined #tripleo07:40
*** shardy has quit IRC07:40
*** rdopiera has quit IRC07:40
*** rdopiera has joined #tripleo07:40
*** shardy has joined #tripleo07:41
*** jpena|off is now known as jpena07:46
*** ykarel is now known as ykarel|lunch07:46
*** jaosorior has quit IRC07:47
*** shyamb has quit IRC07:47
openstackgerritmelissaml proposed openstack/puppet-tripleo master: Remove the duplicated word  https://review.openstack.org/59289907:50
*** Petersingh is now known as Petersingh|lunch07:56
bogdandodsneddon: hi, around?07:57
bogdandoIIRC, you mentioned there is neutron agents issues with SIGHUP sent over logrotate? perhaps we could extend the logical filters to cover that case like we do here to alter behavior for httpd https://review.openstack.org/#/c/591616/07:58
bogdandof.e. we could send -KILL :D allowing restarting policies to reanimate containers got shot07:59
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722408:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044508:00
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676408:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726808:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)08:10
openstackgerritMarios Andreou proposed openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes add ceph loop device  https://review.openstack.org/58319508:11
openstackgerritOlivier Bourdon proposed openstack/diskimage-builder master: [DNM] Testing CentOS images builds on Ubuntu Xenial  https://review.openstack.org/59136608:18
*** jpich has joined #tripleo08:21
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Enable support for running refstack tests in TQE  https://review.openstack.org/57071908:22
*** derekh has joined #tripleo08:35
*** Petersingh|lunch is now known as Petersingh08:42
*** leanderthal has joined #tripleo08:51
openstackgerritMerged openstack/tripleo-quickstart-extras master: skip mistral action test  https://review.openstack.org/59259108:53
*** shyamb has joined #tripleo08:56
*** ykarel|lunch is now known as ykarel08:56
*** leanderthal has quit IRC08:56
*** mandre is now known as mandre_off09:04
*** panda|off is now known as panda09:08
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676409:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726809:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)09:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)09:10
*** leanderthal has joined #tripleo09:14
*** aevoo has joined #tripleo09:16
openstackgerritCarlos Goncalves proposed openstack/tripleo-heat-templates master: scenario010: enable Ceph RBD  https://review.openstack.org/59293809:18
*** aevoo has quit IRC09:20
*** aevoo has joined #tripleo09:22
*** shyambiradar has joined #tripleo09:25
openstackgerritKamil Sambor proposed openstack/puppet-tripleo master: Use correct parameter for nova_metadata_host  https://review.openstack.org/59293909:28
*** shyamb has quit IRC09:29
*** udesale has quit IRC09:31
openstackgerritKamil Sambor proposed openstack/tripleo-heat-templates master: Remove deprecated param  https://review.openstack.org/59294109:33
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Updated skip list for Tempest  https://review.openstack.org/59198209:34
*** salmankhan has joined #tripleo09:37
owalshshyambiradar: hey09:41
shyambiradarowalsh: Hi09:42
*** shyambiradar has quit IRC09:43
*** shyamb has joined #tripleo09:43
owalshshyamb: did you ping me earlier?09:44
shyambowalsh: yes09:44
shyambshyamb: I spoke with my dev team about mounts09:44
shyambrequired to make snapshots work09:44
shyambthey said, whatever nova-compute container is using, we should mount the same09:45
shyambowalsh: Can we quickly discuss these mounts in a meeting?09:46
owalshshyamb; hmm, that's a bit vague...nova-compute bind mounts a lot https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/nova-compute.yaml#L21309:47
shyambyes, we can discuss these mount's significance and eliminate which are not required09:48
shyambowalsh: We need /etc/ceph09:49
shyambI know that09:49
shyambWe read all ceph info from there09:49
shyambwe need /var/lib/nova09:50
bandiniany takers for a simple one https://review.openstack.org/#/c/592632/ ?09:50
owalshshyamb: ack, expect you'll need /dev and maybe /etc/iscsi too09:53
owalshshyamb: might be safest to switch to kolla-config for these to avoid permissions issues, so they are copied into /etc in the container instead of bind mounting from the host09:54
shyambowalsh: ok09:56
shyambI am testing few fixes09:56
shyambTo move to kolla_config, how much work do you see?09:56
shyambowalsh: Does nova compute using kolla_config to mount these directories?09:57
shyambI don't see that09:57
shyambdocker_config is taking care of mounting volumes09:57
owalshshyamb: just don't set CMD in the Dockerfile and uncomment the kolla_config section in trilio-datamover.yaml I think09:58
owalshshyamb: for nova-compute, with /etc/ceph for example it's bind mounted here /etc/ceph:/var/lib/kolla/config_files/src-ceph:ro09:59
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Reach localhost via ansible local connection  https://review.openstack.org/57674609:59
owalshshyamb: then merged into /etc/ here https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/nova-compute.yaml#L17309:59
*** paramite has joined #tripleo10:01
*** shyamb has quit IRC10:03
*** shyam89 has joined #tripleo10:03
shyam89owalsh: ok10:04
shyam89that seems easy10:04
shyam89will do it after my testing10:04
shyam89ok10:04
shyam89owalsh: I got your point10:05
shyam89will implement it10:06
owalshshyam89: something like this http://paste.openstack.org/show/728291/10:07
shyam89owalsh: thanks10:09
shyam89owalsh: looks good10:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676410:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726810:10
*** ooolpbot has quit IRC10:10
shyam89owalsh: btw, any way to test this heat environment without deploying through overcloud command?10:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)10:10
owalshshyam89: missed the trilio conf file - so http://paste.openstack.org/show/728292/10:10
owalshshyam89: can run some syntax checks via tox10:11
owalshshyam89: if you have a clone of tripeo-heat-templates then tox -e pep8 will pass the templates through jinja2 templating and then run syntax checks. Isn't going to catch everything but will pick up obvious problems10:14
owalshshyam89: if you drop the yaml file in docker/services it should check that too10:14
*** aufi has quit IRC10:14
*** EmilienM is now known as EvilienM10:15
*** ansmith has quit IRC10:16
shyam89owalsh: ok10:19
shyam89owalsh: will use it10:19
*** shyam89 has quit IRC10:30
*** ansmith has joined #tripleo10:31
*** udesale has joined #tripleo10:31
*** Petersingh is now known as Petersingh|afk10:35
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Support virthost localhost via local connection  https://review.openstack.org/57674610:42
*** dxiri has quit IRC10:43
*** shyam89 has joined #tripleo10:51
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Add rocky to emit files  https://review.openstack.org/59236510:55
Tenguheya! for your information, planned maintenance on hub.docker.com end of August: https://success.docker.com/article/planned-downtime-on-hub-cloud-store10:59
Tengumight be good to keep that in mind - even if it's a Saturday.10:59
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Support virthost localhost via local connection  https://review.openstack.org/57674611:01
*** ramishra has quit IRC11:04
*** jpena is now known as jpena|lunch11:04
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Support virthost localhost via local connection  https://review.openstack.org/57674611:05
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Better support for the non-disposable devboxes.  https://review.openstack.org/59299611:05
*** shyam89 has quit IRC11:07
*** shyam89 has joined #tripleo11:07
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676411:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178726811:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)11:10
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)11:10
sshnaidm|offmarios|rover, weshay|ruck seems like scenario007 and scenario000 are broken, gates fail11:10
*** rh-jelabarre has joined #tripleo11:15
*** paramite has quit IRC11:18
*** Petersingh|afk is now known as Petersingh11:20
*** ramishra has joined #tripleo11:23
*** shardy has quit IRC11:24
*** shardy has joined #tripleo11:24
*** udesale has quit IRC11:26
mariossshnaidm|off: o/ i saw some fails on scen 3 but last run for tht is green .. where you looking11:28
sshnaidm|offmarios, http://cistatus.tripleo.org/gates/   http://dashboard-ci.tripleo.org11:28
*** two_oes has joined #tripleo11:28
mariossshnaidm|off: for scen 0 do you mean scen0 multinode updates? if so that one is known (ack on the links yeah i have those  )11:30
mariossshnaidm|off: maybe point to example11:30
openstackgerritMerged openstack/tripleo-heat-templates master: Add BlacklistedIpAddresses stack output  https://review.openstack.org/58925211:30
*** waleedm has joined #tripleo11:31
mariossshnaidm|off: also do youknow why the upstream failing gate jobs in the cockpit at http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1 is showing 591374 as the change for all the failing things11:31
mariossshnaidm|off: i wanted to ask wes about that later i understand this is quique work11:31
mariosbut he is away11:31
mariossshnaidm|off: makes it hard to correlate where that is happening11:32
sshnaidm|offmarios, I fixed dashboard, please refresh it11:33
mariossshnaidm|off: cool thanks checking11:33
sshnaidm|offmarios, for example: https://review.openstack.org/56885611:33
sshnaidm|offtripleo-ci-centos-7-scenario000-multinode-oooq-container-updates timeout11:33
mariossshnaidm|off: ack but anything from yesterday needs recheck i think there were multiple issues at the time11:34
mariossshnaidm|off: like the rdopkg issue:  " https://bugs.launchpad.net/tripleo/+bug/178724411:34
openstackLaunchpad bug 1787244 in tripleo "rdopkg release has broken all ci jobs" [Critical,Fix released] - Assigned to wes hayutin (weshayutin)11:34
mariossshnaidm|off: i think11:34
mariossshnaidm|off: nice11:34
sshnaidm|offmarios, this is from today: https://review.openstack.org/#/c/591980/11:34
mariossshnaidm|off: dashboard works11:34
mariossshnaidm|off: much better !11:34
marios:_11:34
marios:)11:34
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart master: Add featureset 054 - overcloud baremetal+ansible-ml2  https://review.openstack.org/57960111:35
mariossshnaidm|off: ack from 591980 looks like the scen0multinode updates is the known one (https://bugs.launchpad.net/tripleo/+bug/1787226 )... not sure about that 3 nodes multinodes11:35
openstackLaunchpad bug 1786764 in tripleo "duplicate for #1787226 tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)11:35
marioshttp://cistatus.tripleo.org/ from here though i see scen 7 has fails11:36
mariossshnaidm|off: ^ you mentioned 7 as well11:36
sshnaidm|offmarios, http://cistatus.tripleo.org/gates/11:36
sshnaidm|offmarios, and just click on it :)11:37
mariossshnaidm|off: ack :) am doing that now thanks11:37
sshnaidm|offmarios, oh, I see it non-voting in gates, so you can ignore it11:38
mariossshnaidm|off: k stil might see if the error isobv11:38
mariosobvious11:38
mariossshnaidm|off: still, thanks for fixing the dashboard and the pointers :)11:39
sshnaidm|offmarios, np11:39
fultonjtonyb: are you around?11:39
*** morazi has joined #tripleo11:40
*** paramite has joined #tripleo11:41
mariossshnaidm|off: heh well that scen7 job is failing for the known DockerCinderConfigImage not assigned from https://bugs.launchpad.net/tripleo/+bug/178726811:43
openstackLaunchpad bug 1787268 in tripleo "Property DockerCinderConfigImage not assigned" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker)11:43
mariossshnaidm|off: http://logs.openstack.org/91/592591/1/gate/tripleo-ci-centos-7-scenario007-multinode-oooq-container/0ff5c03/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-08-17_07_48_5811:43
mariossshnaidm|off: but as you said both it and 8 are non voting11:43
mariossshnaidm|off: weshay|ruck https://review.openstack.org/#/c/583195/ green tripleo-ci-centos-7-containers-multinode-pike with https://trello.com/c/GvjcnJB2/850-translate-tripleosh-bootstrap-subnodes-into-a-series-of-tasks#comment-5b7682c3e489d64c94c47b2011:45
mariospanda: rfolco myoung ^^^ please if you have some reviews time thanks11:46
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Add scenario 012 - overlcoud baremetal+ansible-ml2  https://review.openstack.org/57960311:47
*** shyam89 has quit IRC11:47
*** pdeore has quit IRC11:51
*** shyam89 has joined #tripleo11:54
weshay|ruckmarios, https://review.openstack.org/#/c/588527/11:56
mariosweshay|ruck: good morning to you too :)11:57
mariosweshay|ruck: thanks :)11:57
weshay|rucklolz11:57
mariossome people say hello by merging stuff11:58
mariosor -1 it sometimes works fine too11:58
weshay|ruckI broke up with an old girlfriend by -2'ing a patch of hers11:59
marioshaha11:59
*** jpena|lunch is now known as jpena12:00
openstackgerritMerged openstack/tripleo-common master: Ensure all image parameters have a default set  https://review.openstack.org/59227512:03
openstackgerritMerged openstack/puppet-tripleo master: Remove PE requirement from metadata.json  https://review.openstack.org/59273712:03
waleedmHi guys, I'm trying to deploy and getting this error "ERROR: Property error: : resources.ComputeSriov<nested_stack>.resources.0<https://192.168.24.2:13808/v1/AUTH_bd79169d677c4f60a5c9dbe965e8bd80/overcloud/puppet/computesriov-role.yaml>.resources.NetworkConfig.properties: : Unknown Property StorageMgmtInterfaceRoutes"12:04
waleedmCan anyone hep with that, please?12:04
openstackgerritMerged openstack/tripleo-common master: Remove container_registry parameter from update workflows  https://review.openstack.org/58948712:08
EvilienMweshay|ruck: wait we broke up?12:08
EvilienMmuahahahahahah12:09
weshay|ruckha12:09
*** shyamb has joined #tripleo12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676412:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to wes hayutin (weshayutin)12:10
*** ooolpbot has quit IRC12:10
weshay|ruckhrm.. INFO:kolla.common.utils.tempest:No package openstack-tempest-all available.12:11
weshay|ruckchandankumar, help please12:11
weshay|ruckhttps://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-centos-7-rocky-containers-build/2311715/logs/kolla/logs/000_FAILED_tempest.log12:11
*** shyam89 has quit IRC12:12
*** raildo has joined #tripleo12:12
ykarelamoralej, ^^12:14
amoralejweshay|ruck, ykarel that should have been fixed now by https://review.rdoproject.org/r/1559512:16
*** waleedm has quit IRC12:17
ykarelamoralej, yes correct, consistent is good now12:18
amoralejyeah12:18
chandankumarweshay|ruck: yes fix is already there ^^12:20
shyambowalsh: My overcloud deployment (with trilio components) is failing with error: http://paste.openstack.org/show/728302/12:21
shyambowalsh: any thoughts?12:21
chandankumarthere is a cyclic dependency -all is dependent on tempest package which generally causes the failure12:21
*** Petersingh is now known as Petersingh|afk12:22
weshay|ruckchandankumar, yup.. thanks12:22
weshay|ruckchandankumar, amoralej nothing gets by you two :)12:22
weshay|ruckthanks12:22
*** ratailor has quit IRC12:23
*** boazel has joined #tripleo12:24
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Enable support for running refstack tests in TQE  https://review.openstack.org/57071912:27
*** aufi has joined #tripleo12:28
*** dbecker has joined #tripleo12:31
chandankumarpanda: EvilienM need +2 and +w on this https://review.openstack.org/#/c/565642/12:31
chandankumarweshay|ruck: ^^12:31
EvilienMchandankumar: looking12:31
EvilienMpanda: EvilienM need +2 and +w on this https://review.openstack.org/#/c/565642/ pest.html.gz12:32
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: add delorean-current to repolist for updates  https://review.openstack.org/59257712:32
EvilienMoops wrong command12:32
EvilienMhttp://logs.openstack.org/42/565642/8/check/tripleo-ci-centos-7-undercloud-containers/68b257d/logs/tempest.html.gz12:32
EvilienMchandankumar: where are the tests?12:32
chandankumarEvilienM: https://review.openstack.org/58952012:33
chandankumarEvilienM: https://review.openstack.org/#/q/topic:tripleo-mistral+(status:open+OR+status:merged)12:33
jistrweshay|ruck: i dared update the dlrn-current patch w/ support for undercloud containers too, lemme know what you think please: https://review.openstack.org/#/c/592577/312:34
EvilienMchandankumar: no I mean the tempest tests12:34
EvilienMchandankumar: I don't see them in logs12:34
EvilienMchandankumar: the mistral tests12:34
weshay|rucksure12:34
* weshay|ruck looks12:34
chandankumarEvilienM: https://review.openstack.org/#/c/565642/8/config/general_config/featureset027.yml@11112:34
jistrweshay|ruck: i removed the conditional fix though... we use that variable in multiple places and AFAIK we do set it. If we want to switch directly to mixed_upgrade we should probably do that in many other places.12:34
chandankumarEvilienM: it will trigger against mistral project only12:34
jistrweshay|ruck: if i misunderstood what that was doing, we can put that part back12:35
chandankumarEvilienM: http://logs.openstack.org/20/589520/3/check/tripleo-ci-centos-7-undercloud-containers/5d365a8/logs/tempest.html.gz12:35
*** trown|outtypewww is now known as trown12:35
chandankumarcoming from this review https://review.openstack.org/#/c/589520/12:36
EvilienMoh ok12:36
EvilienMinteresting12:36
EvilienMwfm12:36
chandankumarfuture will be bright if we replace that with standalone12:36
chandankumarand plugin specific tests only12:36
chandankumarwill be triggered12:37
EvilienM++12:37
EvilienMsoon (tm)12:37
*** rlandy has joined #tripleo12:39
*** Petersingh|afk has quit IRC12:39
weshay|ruckjistr, I don't see the code running in the patches12:40
weshay|ruckjistr, specifically packages_for_update12:40
chandankumarweshay|ruck: EvilienM one more https://review.openstack.org/59198212:40
jistrweshay|ruck: if you mean CI, i just posted it, i bet it barely started12:41
weshay|ruckjistr, in previous patches.. I made a different but similiar patch as well12:41
jistrweshay|ruck: i see, just found it12:42
jistrweshay|ruck: i think in yours it was space-delimited instead of comma-delimited which may make the difference between working and non-working12:43
*** bugzy_ is now known as bugzy12:43
EvilienMchandankumar: commented, not sure why we disabled all these ones12:43
EvilienMthey are in smoke?12:43
weshay|ruckjistr, http://logs.openstack.org/77/592577/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/696dd20/logs/undercloud/home/zuul/overcloud-prep-containers.sh12:43
chandankumarEvilienM: in tempest container all the plugins are there and most of the temoest plugins tests are added in skip list whose support is not there yet12:44
chandankumarlike heat and ironic will come in future12:44
jistrweshay|ruck: ah that, yea i think we simply reuse what was done for undercloud, if we deploy containerized undercloud.12:45
jistrweshay|ruck: so we need to be looking at the UC prepare mainly i think12:45
weshay|ruckI think so too12:45
*** amoralej is now known as amoralej|lunch12:46
weshay|rucktoo many mtgs for me to move on this much TM12:48
weshay|ruckATM12:48
EvilienMchandankumar: +212:48
chandankumarEvilienM: thanks :-)12:48
openstackgerritJulia Kreger proposed openstack/tripleo-heat-templates master: Clarify ironic classic driver removal reno  https://review.openstack.org/59304112:48
jistrweshay|ruck: ack that's fine, i'm hoping the latest patchset i posted could help, we need to wait for CI results now anyway12:49
weshay|ruck:)12:49
*** jcoufal has joined #tripleo12:50
*** skramaja has quit IRC12:53
*** TheJulia is now known as needssleep12:55
*** moguimar has joined #tripleo12:56
openstackgerritMerged openstack/tripleo-quickstart master: Do not manage users/groups/services when chrooted  https://review.openstack.org/59177013:00
*** shyamb has quit IRC13:02
*** shyam89 has joined #tripleo13:03
*** dbecker has quit IRC13:04
*** janki has quit IRC13:08
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676413:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)13:10
*** bfournie has joined #tripleo13:12
*** janki has joined #tripleo13:12
bfourniebogdando: hi, this is a backport of a master patch you approved, could you +w it please? - https://review.openstack.org/#/c/592559/13:15
*** ade_lee has joined #tripleo13:18
*** jcoufal has quit IRC13:18
*** jcoufal has joined #tripleo13:19
shyam89owalsh: I have committed all the changes here13:25
shyam89https://github.com/shyam-biradar/triliovault-cfg-scripts/tree/master/redhat-director-scripts13:25
shyam89Please review13:25
*** dprince has joined #tripleo13:25
jristfultonj: hacking out cleaning worked13:25
jristfultonj: thanks13:25
fultonjjrist: glad i could help13:25
owalshshyam89: I'm in meetings for the rest of the day, then on PTO until end of next week....13:28
shyam89owalsh: ohh13:28
owalshshyam89: but seem to be missing some of the changes discussed e.g CMD is still set in DockerFile13:28
shyam89any backup resource for me?13:28
shyam89I am testing it, not committed yet13:28
owalshshyam89: re the deployment error, looks like puppet failed13:31
shyam89yes13:32
owalshshyam89: on the compute node look in /var/log/messages to see why13:33
*** ramishra has quit IRC13:33
*** dbecker has joined #tripleo13:33
shyam89owalsh: ok13:34
owalshalso useful https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/tips_tricks.html#debugging-docker-puppet-py13:34
*** derekh has quit IRC13:38
*** amoralej|lunch is now known as amoralej13:43
*** jfrancoa has quit IRC13:46
*** jaganathan has quit IRC13:52
*** derekh has joined #tripleo13:53
*** dxiri has joined #tripleo13:56
*** shyam89 has quit IRC13:57
openstackgerritDaniel Alvarez proposed openstack/tripleo-heat-templates master: DNM: OVN test  https://review.openstack.org/59305613:57
*** dansmith is now known as steelydan13:58
*** steelydan is now known as SteelyDan13:58
*** jfrancoa has joined #tripleo13:58
*** ramishra has joined #tripleo14:00
openstackgerritDaniel Alvarez proposed openstack/tripleo-heat-templates master: DNM: OVN test  https://review.openstack.org/59305614:02
*** bugzy_ has joined #tripleo14:02
*** rpioso|afk is now known as rpioso14:03
*** paramite has quit IRC14:05
*** bugzy has quit IRC14:06
*** bugzy_ is now known as bugzy14:06
openstackgerritDaniel Alvarez proposed openstack/tripleo-heat-templates master: DNM: OVN test  https://review.openstack.org/59305614:06
*** gbarros has joined #tripleo14:09
*** derekh has quit IRC14:11
*** ooolpbot has joined #tripleo14:14
*** toure|gone is now known as toure14:14
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:15
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676414:15
*** ooolpbot has quit IRC14:15
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)14:15
openstackgerritJohn Trowbridge proposed openstack/tripleo-heat-templates master: Move to openshift-ansible 3.10  https://review.openstack.org/58249514:17
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Revert "Revert "Move to openshift-ansible 3.10""  https://review.openstack.org/58946914:17
*** dbecker has quit IRC14:17
*** ksambor has quit IRC14:22
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - WIP - Add another level of parents jobs for zuul v3  https://review.openstack.org/59306314:26
*** dbecker has joined #tripleo14:27
*** shardy has quit IRC14:29
*** bugzy_ has joined #tripleo14:30
*** bugzy has quit IRC14:33
*** bugzy has joined #tripleo14:35
*** bugzy_ has quit IRC14:36
*** janki has quit IRC14:46
bfourniebogdando: thank you14:47
*** aufi has quit IRC14:50
*** tzumainn has joined #tripleo14:50
tzumainnhi! question: I'm trying to deploy standalone tripleo with both heat and ironic enabled. The default standalone.yaml environment file has both disabled, so I made a custom one that just removes the lines that disable the Heat and Ironic services14:50
tzumainnthat seemed to work for heat, but not for ironic - does anyone have some hint as to why that might be the case?14:51
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes add ceph loop device  https://review.openstack.org/58319515:01
*** ykarel is now known as ykarel|away15:04
*** yprokule has quit IRC15:04
*** rpioso is now known as rpioso|car15:05
*** dbecker has quit IRC15:07
*** leanderthal has quit IRC15:08
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676415:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)15:10
*** ooolpbot has quit IRC15:10
*** ykarel|away has quit IRC15:12
*** jfrancoa has quit IRC15:16
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM - Testing adding properties to flavors  https://review.openstack.org/57940615:16
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM - Test increasing nova db_sync time for OSP 13  https://review.openstack.org/59307715:16
jistr2018-08-17 13:05:39.821524 | primary | TASK [undercloud-deploy : Install the undercloud] ******************************15:17
jistr2018-08-17 13:05:39.893190 | primary | Friday 17 August 2018  13:05:39 +0000 (0:00:04.505)       0:13:21.474 *********15:17
jistr2018-08-17 14:38:44.620389 | primary | changed: [undercloud]15:17
jistrweshay|ruck: see the time ^ :) 1.5 hours for UC deploy15:17
jistr(looking at http://zuul.openstack.org/stream.html?uuid=ebb643d0b4854fa9b97c7c88eddc8ba7&logfile=console.log )15:17
jistrweshay|ruck: i'm thinking, we shouldn't probably be updating from N days old tripleo-current images to latest delorean-current content in each job. One of the discrepancies we have there is that we use tripleo-current images in CI jobs -- that's not equivalent of what we did before we had containers. We should probably periodically build container images with very latest content.15:21
jistri mean latest RPM content for this whitelist mainly https://github.com/openstack/tripleo-quickstart/blob/47a07824888dddc65db6e63cf4ef7bbbc4a617a1/config/release/tripleo-ci/master.yml#L59-L7015:21
bogdandotzumainn: got deploy logs with errors?15:22
jistrcurrently we run CI on images which passed promotion -- that's wrong IMO. For TripleO content at least, we should be running on latest, even if latest is broken.15:22
jistrweshay|ruck, mwhahaha, EvilienM ^^15:23
tzumainnbogdando, the deploy finishes without errors, and openstack endpoint list shows heat available - just not ironic15:23
tzumainnso I'm wondering if I'm missing something to enable ironic with standalone tripleo15:23
jistrso i think we should have a set of images that get updated every few hours regardless if they're red or green15:23
tzumainnbogdando, actually, now that I think about it, I probably have to follow some of the instructions in https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/baremetal_overcloud.html15:24
mwhahahathe problem is if it fail to deploy the first time it won't work15:24
mwhahahayou have to go from passed promotion containers to current passed promotion containers15:24
bogdandotzumainn: yeah, you all also need to add some -e env file15:24
bogdandoto really enable ironic15:25
mwhahahaso the first install should be the previous promotion results, and then the job would update the current containers with current tripleo bits and run15:25
bogdandowhat was the deployment command? tzumainn15:25
tzumainnbogdando, I followed the steps in https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html15:26
bogdandotzumainn: right. For extra services, in the end you'll need something like https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/baremetal_overcloud.html#deployment15:26
tzumainnbogdando, yep, I'll try that - thanks a lot!15:26
jistrmwhahaha: (note i'm not talking just about update jobs, i'm talking about all jobs). I don't view it that way. What you say is not what we ever did when we had undercloud on RPMs. We always installed latest RPMs, not promoted ones. If the latest was broken, all gate was red, and we ran to fix it. I don't think deploying with containers should be any different. We should deploy what's latest, not what's15:27
jistrworking. Otherwise we defeat the purpose of CI IMO.15:27
*** slaweq has quit IRC15:27
mwhahahajistr: yes it is15:28
mwhahahajistr: we've always had a set of excludedpkgs in CI (Tripleo/puppet) that we updated15:28
mwhahahabut the base packages were the promotion stuff15:28
mwhahahawe do the same thing in CI today where we inject some package updates into the containers15:28
mwhahahawe never had a floating repo that we used in CI15:29
mwhahahaa noop of any of the CI jobs should be passed containers + updated tripleo bits15:29
mwhahahathat's how it was previously and should be how it is today15:29
mwhahahaif it isn't that a bug in the ci15:29
jistrmwhahaha: right, i'm not talking about the base packages. Those are fine, we should use promoted there. But i'm saying we regressed into deploying stale excludedpkgs (Tripleo/puppet) when we moved to containers.15:30
mwhahahafor the update jobs, the first install should be the n-1 promotioned containers/rpms and the second deploy should be current containers + updated tripleo bits15:30
openstackgerritSam Doran proposed openstack/tripleo-heat-templates master: Use ternary filter for configs  https://review.openstack.org/58750615:30
mwhahahajistr: we regressed a lot with containers but that's a story for a different day (preferably with large quantities of alcohol)15:30
jistrmwhahaha: re doing the same thing by injecting packages -- partially, but it's broken. We only inject DLRN gate build, we don't inject delorean-current. So everything what's not in gating-repo is stale.15:31
jistrmwhahaha: we're trying to fix that here -- aside from gating-repo injection, inject also delorean-current https://review.openstack.org/#/c/592577/15:32
openstackgerritCarlos Goncalves proposed openstack/tripleo-heat-templates master: Add OctaviaEventStreamDriver and change default  https://review.openstack.org/59308315:34
jistrmwhahaha: but the problem is the time it takes. 1.5 hours for undercloud deploy. I think that's because there's too much gap between tripleo-current container images and delorean-current RPMs. So what i'm suggesting is -- we shouldn't cover N days of delorean-current gap by updating *in each job*, if we could build a set of images with latest RPMs and then use it in each job. We'd still probably update, but15:35
jistrhopefully much smaller set of packages (perhaps we could go back to injecting just gating-repo if we know the rest is few hours old at max).15:35
jistrmwhahaha: so i'm not really suggesting a change, more like "let's not do this in every job if we can do it just once and then reuse the result"15:35
jistrcc weshay|ruck ^15:35
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: [DNM] Add local network-environment file for fs001  https://review.openstack.org/57723015:35
mwhahahawe shouldn't be injecting all of delorean-current15:35
mwhahahathat's wrong15:35
openstackgerritCarlos Goncalves proposed openstack/tripleo-heat-templates master: Add OctaviaEventStreamerDriver and change default  https://review.openstack.org/59308315:35
*** moguimar has quit IRC15:36
jistrmwhahaha: that delorean-current is still guarded by excludepkgs i think. So IMO it's not wrong :)15:36
mwhahahahttps://github.com/openstack/tripleo-repos/blob/master/tripleo_repos/main.py#L28-L3215:36
jistrmwhahaha: it's how we gated before moving to containers15:36
mwhahahawe should be only updating those packages which don't exist on all the containers15:36
mwhahahaand it's an includpkgs and not a exclude15:36
jistryea sorry includepkgs15:37
jistrmwhahaha: i think that's what we're doing in that patch?15:37
mwhahahai'm not sure15:37
jistri'll try to check15:38
mwhahahawe have no timing in the overcloud-orep-containers.sh do we?15:38
mwhahahaso we don't know how long it takes to run container-check?15:38
jistri'm not sure, i think most of the work is done by undercloud and overcloud-prep then does nothing essentially, just reuses what UC did15:41
jistrbtw:15:41
jistr(undercloud) [stack@undercloud ~]$ repoquery --disablerepo='*' --enablerepo=delorean-current --qf %{NAME} -a | wc -l15:41
jistr11415:41
jistr(undercloud) [stack@undercloud ~]$ sudo vi /etc/yum.repos.d/delorean-current.repo15:41
jistr(undercloud) [stack@undercloud ~]$ repoquery --disablerepo='*' --enablerepo=delorean-current --qf %{NAME} -a | wc -l15:41
jistr76315:41
jistrmwhahaha: the 2nd number is after i commented out includepkgs. So i think the patch is doing the right thing...15:42
mwhahahaso maybe there's a bug in the undercloud work15:42
mwhahahai'm not sure15:42
mwhahahai haven't been keeping up where that interaction occurs. I thought we were still doing the contianer updates externally to the undercloud install15:42
jistrmwhahaha: `undercloud deploy` nowadays has an embedded `image prepare` and image content update, IIUC. The update part is configured between lines 6 and 16 at https://review.openstack.org/#/c/592577/3/roles/undercloud-deploy/templates/containers-prepare-parameter.yaml.j215:44
jistrmwhahaha: IIUC we took several CLI commands and made them into Heat params :)15:44
*** saneax has quit IRC15:45
openstackgerritMarios Andreou proposed openstack/tripleo-quickstart master: Install openstack-heat-agents for Pike subnodes  https://review.openstack.org/58852715:46
*** dbecker has joined #tripleo15:46
jistrmwhahaha, weshay|ruck: anyway i don't mean to be too pushy about this but i still think that having `delorean-current` container images (limited by includepkgs) is the container-world parallel to having the delorean-current RPM repo with includepkgs. It would save a ton of wall time if we didn't have to update RPMs in each job (again, we're talking about non-update jobs too here).15:49
mwhahahayea the problem is where do we build them and the space x number of stable branches15:50
jistryea.. :)15:50
mwhahahathis entire this is an issue with containers that folks just hand wave over15:50
mwhahahapersonally, the containers are too large to CICD15:51
mwhahahaand we don't have the resources to do this15:51
jistryea i understand that angle :/ though we can't really keep doing what we do now either, because our stuck gate due to https://bugs.launchpad.net/tripleo/+bug/1786764 is a direct consequence of it (only updating from gating-repo and not from delorean-current)15:53
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)15:53
mwhahahaas much as i hate it, i'm not sure we can do the updates job anymore15:54
mwhahahabtw is it non-voting yet?15:54
mwhahahaalso does the updates job do an undercloud udpate too or is it just the overcloud15:54
*** hgogada has quit IRC15:54
jistrmwhahaha: it's not only about updates job. Non-updats job can break *exactly* the same way. The only reason updates job is broken this way is that the change went into and updates workflow. If it went to deployment workflow, we may just have a stuck "basic" gate until tripleo-current container images promote, or until we merge the delorean-current container update patch. There's nothing special about updates15:56
jistrjob that would be required to trigger this inconsistency bug.15:56
mwhahahathe updates job is worse because we do this 2x15:56
mwhahahawhen we shouldn't be updating the first pass15:56
mwhahahaso the updates job should not get the updates to the undercloud iniital install15:57
jistrmwhahaha: no, we do update the containers just once in updates job too AFAIK (or if we don't, we should)15:57
jistrthe purpose of updates job is to validate the workflow, not to update to a different set of images15:57
* mwhahaha shrugs15:58
jistrso updates job is not worse in this IMO, it's a matter of time or luck before we hit the same on non-updates job15:58
mwhahahai think this is a conversation for stevebaker15:58
weshay|ruckjistr, agree15:59
weshay|rucksorry I'm not commenting more.. just in too many mtgs15:59
jistrno worries weshay|ruck15:59
*** dbecker has quit IRC15:59
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044516:00
jistranyway this is probably something to circle back on to figure out...16:02
jistrthanks mwhahaha for responding :)16:02
*** gbarros has quit IRC16:05
*** rpioso|car is now known as rpioso16:10
*** salmankhan has quit IRC16:12
openstackgerritCarlos Goncalves proposed openstack/tripleo-heat-templates master: scenario010: enable Ceph RBD  https://review.openstack.org/59293816:12
*** salmankhan has joined #tripleo16:13
*** ooolpbot has joined #tripleo16:13
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:13
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676416:13
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)16:13
*** ooolpbot has quit IRC16:13
*** ramishra has quit IRC16:19
*** ykarel has joined #tripleo16:21
*** tzumainn has quit IRC16:29
*** ykarel has quit IRC16:32
*** salmankhan has quit IRC16:32
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - WIP - Add another level of parents jobs for zuul v3  https://review.openstack.org/59306316:34
*** salmankhan has joined #tripleo16:34
*** jpena is now known as jpena|off16:40
*** links has quit IRC16:43
*** amoralej is now known as amoralej|off16:44
*** jpich has quit IRC16:46
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Fix .bashrc path for XDG exports  https://review.openstack.org/59310316:49
*** rdopiera has quit IRC16:49
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Use sudo when working with root home as non root  https://review.openstack.org/59310416:49
*** bogdando has quit IRC16:52
weshay|ruckmwhahaha, where are you seeing the updates job fail soooo much you need to take action?16:52
*** bogdando has joined #tripleo16:53
bogdandoPTAL https://review.openstack.org/#/c/591616/16:53
bogdandoand https://review.openstack.org/#/c/591591/316:53
mwhahahaweshay|ruck: http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates it doesn't seem to be the most stable thing16:53
mwhahahaso just today i see 2 gate failures on a timeout and what's what maybe 60% pass?16:54
mwhahahathat's just eyeballing it16:54
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Fix host facts for centos  https://review.openstack.org/59310616:55
*** salmankhan1 has joined #tripleo16:55
*** salmankhan has quit IRC16:55
*** salmankhan1 is now known as salmankhan16:55
*** hgogada has joined #tripleo17:06
*** myoung is now known as myoung|bbl17:08
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676417:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)17:10
*** bugzy_ has joined #tripleo17:15
*** bugzy has quit IRC17:17
*** Vorrtex has joined #tripleo17:18
VorrtexHello!  I'm looking for Adam Kimball or Michael Turek?  I'm not sure what timezone they're in, but was hoping someone could point me at their IRC nicks so I can talk to them about a thing or two :)17:19
*** salmankhan has quit IRC17:22
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart master: Support virthost localhost via local connection  https://review.openstack.org/57674617:22
*** trown is now known as trown|lunch17:24
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: Add rocky to emit files  https://review.openstack.org/59236517:34
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: Add rocky to emit files  https://review.openstack.org/59236517:35
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: add rocky as a target release for emit releases  https://review.openstack.org/59311517:36
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: fix full-deploy standalone logic  https://review.openstack.org/59311617:41
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: f28 support for quickstart  https://review.openstack.org/59165217:44
*** bogdando has quit IRC17:48
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3  https://review.openstack.org/59306317:51
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3  https://review.openstack.org/59306317:55
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3  https://review.openstack.org/59306317:58
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3  https://review.openstack.org/59306318:00
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3  https://review.openstack.org/59306318:03
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676418:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)18:10
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: [DNM] Sprint 18 PoC: *TEST* Settings override  https://review.openstack.org/59312918:25
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM - Add another level of parent jobs for zuul v3  https://review.openstack.org/59306318:27
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: [DNM] Sprint 18 PoC: *TEST* Settings override  https://review.openstack.org/59312918:28
*** dxiri has quit IRC18:29
*** dxiri has joined #tripleo18:42
openstackgerritMerged openstack/tripleo-common master: Move python-novajoin package to nova-compute-ironic  https://review.openstack.org/59228018:44
openstackgerritMerged openstack/tripleo-heat-templates master: Add a FreeIPAExtraArgs param to ci/scripts/freeipa_setup.sh  https://review.openstack.org/59263218:44
openstackgerritMerged openstack/tripleo-quickstart master: Enable mistral tempest tests in fs027  https://review.openstack.org/56564218:44
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Use full path to workaround scripts.  https://review.openstack.org/59193018:45
*** rlandy is now known as rlandy|afk18:48
*** trown|lunch is now known as trown18:55
*** ade_lee has quit IRC18:55
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676419:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)19:10
*** aevoo has quit IRC19:13
*** owalsh_ has joined #tripleo19:16
*** owalsh has quit IRC19:20
*** owalsh_ has quit IRC19:20
*** owalsh has joined #tripleo19:21
*** florianf has quit IRC19:26
mwhahahaweshay|ruck: do you know if we capture the container prepare logs anywhere?19:26
mwhahahawhen run in the undercloud19:26
weshay|ruckmwhahaha, heh19:26
*** apetrich has quit IRC19:26
weshay|ruckit's in the undercloud_install log19:26
mwhahahanegative19:26
mwhahahahttp://logs.openstack.org/69/589469/4/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/09b9835/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-08-17_14_52_0519:26
weshay|ruckmwhahaha, so I walked through some of the image update role last night19:27
* weshay|ruck added notes the bug /me looks19:27
weshay|ruckmwhahaha, I think it comes down to https://github.com/openstack/ansible-role-tripleo-modify-image/blob/master/tasks/modify_image.yml#L3419:30
mwhahahaok looks like the container update bits get kicked to the jorunal19:30
weshay|ruckI love the journal19:31
weshay|ruckmwhahaha, tangent.. os-refresh-config is also in the journal right?19:31
mwhahahayea usually19:31
*** owalsh_ has joined #tripleo19:32
mwhahahaso looks like we updated 72 packages * a bunch of containers19:32
mwhahahatrying to see how long it took to update each container19:32
weshay|ruckmwhahaha, tmate?19:32
weshay|ruckah nvrmind19:32
weshay|rucklog file ^19:32
mwhahahacan't tmate a brower :D19:32
mwhahahabut yea19:32
mwhahahai'm going to pull it down and do some greping tho19:32
mwhahahadat 28m log file19:33
weshay|ruckmwhahaha, ?19:33
weshay|ruckI see 12min on the undercloud19:33
mwhahahahttp://logs.openstack.org/69/589469/4/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/09b9835/logs/undercloud/var/log/journal.txt.gz19:33
weshay|ruckhttp://logs.openstack.org/69/589469/4/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/09b9835/logs/ara_oooq/19:33
mwhahahaundercloud-deploy : Install the undercloud19:34
mwhahahatook 1:24:4619:34
*** dbecker has joined #tripleo19:34
mwhahahaof which ~1 hour was doing the container update bits19:34
mwhahahahttp://logs.openstack.org/69/589469/4/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/09b9835/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-08-17_14_52_0519:34
mwhahahathe logs for that are captured (somewhat) in the jorunal19:34
mwhahahaso from 14:52 to 15:35 it was updating containers19:35
*** owalsh has quit IRC19:35
*** owalsh has joined #tripleo19:36
*** owalsh_ has quit IRC19:37
weshay|ruckya19:37
weshay|ruckmwhahaha, so we need to figure this out19:38
weshay|ruckwe can have upstream ci w/o updating19:38
weshay|ruckmwhahaha, or we don't use dlrn-current19:38
weshay|ruckmwhahaha, and tripleo has to promote like everything else19:38
weshay|ruckmwhahaha, was that just w/ the gating repo enabled?19:38
weshay|ruckmwhahaha, blue19:38
weshay|ruckya.. that is just the gating repo19:40
weshay|ruckmwhahaha, it would be really great to get that update process out of the journal19:40
weshay|rucklogs19:40
mwhahahaweshay|ruck: sec19:41
weshay|ruckhttp://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=61&fullscreen19:41
weshay|ruckwe have to move this to pre19:47
mwhahahaweshay|ruck: http://logs.openstack.org/69/589469/4/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/09b9835/logs/undercloud/var/log/journal.txt.gz#_Aug_17_14_54_3119:52
weshay|ruckhttps://github.com/openstack/ansible-role-tripleo-modify-image19:58
weshay|ruckmwhahaha, http://git.openstack.org/cgit/openstack/ansible-role-tripleo-modify-image/tree/files/yum_update.sh19:59
weshay|ruckmwhahaha, https://review.openstack.org/#/c/592577/20:00
weshay|ruckmwhahaha, http://git.openstack.org/cgit/openstack/ansible-role-tripleo-modify-image/tree/files/yum_update.sh20:01
*** hgogada has quit IRC20:02
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676420:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)20:10
*** ooolpbot has quit IRC20:10
*** jcoufal has quit IRC20:12
*** pliu has quit IRC20:14
*** mburned has quit IRC20:15
*** haleyb|away has quit IRC20:17
weshay|ruckmwhahaha, https://review.openstack.org/#/c/592784/20:21
*** Vorrtex has quit IRC20:21
*** dbecker has quit IRC20:31
*** apetrich has joined #tripleo20:33
*** rlandy|afk is now known as rlandy20:36
openstackgerritAlex Schultz proposed openstack/ansible-role-tripleo-modify-image master: Only do yum update when needed  https://review.openstack.org/59316920:37
mwhahahaweshay|ruck: -^20:39
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: remove non-voting jobs from the gate in tripleo  https://review.openstack.org/59317020:39
*** trown is now known as trown|outtypewww21:00
*** gbarros has joined #tripleo21:04
*** dprince has quit IRC21:06
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676421:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)21:10
*** apetrich has quit IRC21:10
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Add scenario 012 - overlcoud baremetal+ansible-ml2  https://review.openstack.org/57960321:10
openstackgerritMerged openstack/tripleo-heat-templates master: Add role parameter for rx/tx virtio-net queue size  https://review.openstack.org/59245821:14
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DNM - Test increasing nova db_sync time for OSP 13  https://review.openstack.org/59307721:15
*** rh-jelabarre has quit IRC21:15
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: [DNM] Add local network-environment file for fs001  https://review.openstack.org/57723021:18
*** mburned has joined #tripleo21:22
*** raildo has quit IRC21:22
*** boazel has quit IRC21:26
*** gbarros_ has joined #tripleo21:43
*** gbarros has quit IRC21:44
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676422:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)22:10
*** ooolpbot has quit IRC22:10
*** gbarros_ has quit IRC22:16
*** gbarros has joined #tripleo22:16
*** gbarros has quit IRC22:17
*** OrionX has joined #tripleo22:18
*** morazi has quit IRC22:22
*** ade_lee has joined #tripleo22:26
*** aevoo has joined #tripleo22:41
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/178676423:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr)23:10
*** rlandy has quit IRC23:27
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Add scenario 012 - overlcoud baremetal+ansible-ml2  https://review.openstack.org/57960323:34
*** toure is now known as toure|gone23:58
OrionXanyone know where to start with a workflow exception on overcloud deploy?: YaqlEvaluationException: Can not evaluate YAQL expression [expression=task(deploy_config).result.deploy_stderr, error=Unknown function \"#property#deploy_stderr\", data={}]\n".  my set of templates worked fine for the original deployement.  this is happening when trying to scale up compute nodes23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!