| *** catintheroof has joined #tripleo | 00:03 | |
| *** bfournie has joined #tripleo | 00:04 | |
| *** jkilpatr has quit IRC | 00:08 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 00:10 |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| *** dparkes_ has quit IRC | 00:11 | |
| *** tosky has quit IRC | 00:15 | |
| openstackgerrit | Merged openstack/paunch master: Log stdout/stderr during error https://review.openstack.org/489640 | 00:17 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient stable/newton: Add --skip-deploy-identifier https://review.openstack.org/489812 | 00:18 |
| openstackgerrit | James Slagle proposed openstack/tripleo-common stable/newton: Add skip_deploy_identifier https://review.openstack.org/489813 | 00:20 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient stable/newton: Add --skip-deploy-identifier https://review.openstack.org/489812 | 00:22 |
| *** itlinux has joined #tripleo | 00:22 | |
| *** psahoo has joined #tripleo | 00:22 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] LVM support for dib-block-device https://review.openstack.org/472065 | 00:26 |
| *** mdnadeem has joined #tripleo | 00:36 | |
| *** fultonj has joined #tripleo | 00:40 | |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Use normal socket file permissions instead of polkit https://review.openstack.org/479816 | 00:43 |
| *** catintheroof has quit IRC | 00:44 | |
| *** psahoo has quit IRC | 00:47 | |
| openstackgerrit | Bob Fournier proposed openstack/os-net-config stable/newton: Use os-net-config mapping file when interface in bond or bridge https://review.openstack.org/489709 | 00:49 |
| *** gbarros has joined #tripleo | 00:51 | |
| *** ramishra has joined #tripleo | 00:51 | |
| *** limao has joined #tripleo | 00:52 | |
| *** tzumainn has quit IRC | 00:53 | |
| *** jkilpatr has joined #tripleo | 01:00 | |
| *** psahoo has joined #tripleo | 01:01 | |
| *** thrash is now known as thrash|g0ne | 01:03 | |
| *** pmannidi has joined #tripleo | 01:05 | |
| *** dmacpher has joined #tripleo | 01:07 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 01:10 |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| *** jkilpatr has quit IRC | 01:17 | |
| *** yamahata has quit IRC | 01:20 | |
| *** itlinux has quit IRC | 01:24 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Adds stop and disable for libvirtd on upgrade to containers https://review.openstack.org/489619 | 01:29 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Add support for custom additional repos in featureset https://review.openstack.org/487742 | 01:29 |
| *** https_GK1wmSU has joined #tripleo | 01:30 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Remove empty metadata_settings from iscsid and multipathd templates https://review.openstack.org/489472 | 01:30 |
| *** https_GK1wmSU has left #tripleo | 01:31 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix network-isolation.j2.yaml to ignore VIPs for disabled networks https://review.openstack.org/485810 | 01:35 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: Support deploying OVN as container services https://review.openstack.org/468860 | 01:42 |
| *** Goneri has quit IRC | 01:43 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Cinder volume/backup containers shouldn't mount two paths at same point https://review.openstack.org/489825 | 01:55 |
| *** nyechiel_mtg has joined #tripleo | 01:59 | |
| *** slagle has quit IRC | 01:59 | |
| *** dixiaoli has joined #tripleo | 02:05 | |
| *** itlinux has joined #tripleo | 02:07 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 02:10 |
| *** bkopilov_ has quit IRC | 02:11 | |
| *** slagle has joined #tripleo | 02:12 | |
| michapma | EmilienM: ack | 02:18 |
| openstackgerrit | Merged openstack/paunch master: paunch apply should exit nonzero for failures https://review.openstack.org/489722 | 02:20 |
| michapma | EmilienM: well at least the overcloud completes now. Is there an easy way to recreate that scenario locally so I can see what causes the validation error? | 02:23 |
| EmilienM | michapma: looking | 02:25 |
| EmilienM | http://logs.openstack.org/05/486905/11/experimental/gate-tripleo-ci-centos-7-scenario008-multinode-oooq-nv/6e8ac3f/logs/undercloud/home/jenkins/overcloud_validate.log.txt.gz#_2017-08-01_21_58_13 | 02:26 |
| EmilienM | michapma: ^ | 02:26 |
| EmilienM | nova couldn't spawn a VM | 02:26 |
| EmilienM | let me dig | 02:26 |
| EmilienM | mechanism_drivers=openvswitch | 02:27 |
| EmilienM | http://logs.openstack.org/05/486905/11/experimental/gate-tripleo-ci-centos-7-scenario008-multinode-oooq-nv/6e8ac3f/logs/subnode-2/etc/neutron/plugin.ini | 02:27 |
| EmilienM | hum | 02:27 |
| EmilienM | we're deploying openvswitch backend, not opendaylight | 02:28 |
| michapma | EmilienM: well that explains it | 02:28 |
| EmilienM | I'm about to eat dinner now | 02:28 |
| michapma | EmilienM: is that because I removed the duplicate parameters? | 02:28 |
| EmilienM | I'll be online after | 02:28 |
| michapma | EmilienM: sure, thanks for the help | 02:28 |
| EmilienM | check that parameters are well loaded | 02:28 |
| EmilienM | openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates --libvirt-type qemu --timeout 80 -e /home/jenkins/cloud-names.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/deployed-server-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/deployed-server-bootstrap-environment-centos.yaml -e | 02:30 |
| EmilienM | /usr/share/openstack-tripleo-heat-templates/ci/environments/scenario008-multinode.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /home/jenkins/neutronl3ha.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/low-memory-usage.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/disable-telemetry.yaml --validation-errors-nonfatal --roles-file | 02:30 |
| EmilienM | /home/jenkins/overcloud_roles.yaml --compute-scale 0 -e /opt/stack/new/tripleo-ci/test-environments/worker-config.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/debug.yaml --overcloud-ssh-user jenkins | 02:30 |
| EmilienM | sorry for spam | 02:30 |
| EmilienM | but I don't see opendaylight environment | 02:30 |
| EmilienM | sounds like quickstart related, the extra args interface doesn't seem to work | 02:31 |
| EmilienM | in the meantime, re-add stuffs in /usr/share/openstack-tripleo-heat-templates/ci/environments/scenario008-multinode.yaml | 02:31 |
| EmilienM | like I did before | 02:31 |
| EmilienM | and continue testing | 02:31 |
| EmilienM | michapma: ^ | 02:31 |
| michapma | EmilienM: I'll check extra-args, thanks. | 02:32 |
| EmilienM | michapma: yeah and in the meantime, add stuffs from the odl env into scenario008-multinode.yaml | 02:34 |
| EmilienM | and check experimental | 02:34 |
| EmilienM | so we can make progress | 02:35 |
| EmilienM | if we could test ODL before pike final release it would be awesome | 02:35 |
| michapma | EmilienM: the things I removed yesterday? | 02:35 |
| EmilienM | michapma: yes | 02:38 |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-heat-templates master: Add OPNFV scenario environment https://review.openstack.org/486905 | 02:43 |
| *** panda has quit IRC | 02:43 | |
| *** dixiaoli has quit IRC | 02:47 | |
| michapma | EmilienM: the extra args should be coming in via this: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-deploy/defaults/main.yml#L118 | 02:48 |
| *** panda has joined #tripleo | 02:48 | |
| *** dixiaoli has joined #tripleo | 02:51 | |
| *** nyechiel_mtg has quit IRC | 02:53 | |
| *** ramishra has quit IRC | 02:53 | |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml https://review.openstack.org/482623 | 03:00 |
| *** itlinux has quit IRC | 03:00 | |
| EmilienM | michapma: indeed :/ | 03:01 |
| *** dsariel has joined #tripleo | 03:01 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 03:10 |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Upload remove defaults for pull_source, push_destination https://review.openstack.org/489837 | 03:11 |
| *** udesale has joined #tripleo | 03:33 | |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart master: Implement scenario008 (opnfv) https://review.openstack.org/486731 | 03:34 |
| EmilienM | michapma: just fyi, when you update a patch that is in Depends-on of another patch, you need to 'recheck' the patch that has Depends-on | 03:35 |
| EmilienM | so this one: https://review.openstack.org/#/c/486905/ | 03:36 |
| EmilienM | See the comment: "This change depends on a change that failed to merge." | 03:36 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add OPNFV scenario environment https://review.openstack.org/486905 | 03:36 |
| EmilienM | I also rebased it ^ | 03:36 |
| michapma | EmilienM: ah, so I add a recheck comment before the check experimental comment | 03:36 |
| EmilienM | yeah or the other way no problem | 03:37 |
| EmilienM | michapma: please check CI results when it's executed | 03:37 |
| EmilienM | michapma: it will probably fail, look at neutron server logs on subnode-2 (overcloud node) | 03:38 |
| michapma | EmilienM: I checked the other scenarios and they seem to use extra_args without any issues. I had some extra stuff in there for controlling the quickstart network creation but I'm not sure that will work in th emultinode tests | 03:38 |
| EmilienM | michapma: let me know on IRC if you found something new | 03:38 |
| EmilienM | michapma: ok, I'm not familiar with this interface but I can look tomorrow | 03:38 |
| EmilienM | I'm about to leave the computer now | 03:38 |
| michapma | EmilienM: yeah will do. trozet and I are meeting tomorrow morning and we'll have a look then as well. | 03:38 |
| *** gbarros has quit IRC | 03:39 | |
| EmilienM | michapma: good | 03:39 |
| EmilienM | michapma: keep in mind that tripleoclient logs are in undercloud/home/jenkins/ | 03:39 |
| EmilienM | they are very helpful if the overcloud stack failed to be started | 03:40 |
| EmilienM | michapma: also if there is a Puppet error, also check in /var/log/messages of the subnode-2 | 03:40 |
| michapma | EmilienM: ok thanks, will give it a go | 03:40 |
| EmilienM | good | 03:40 |
| EmilienM | ttyl | 03:40 |
| michapma | later | 03:40 |
| *** gkadam has joined #tripleo | 03:41 | |
| *** bkopilov_ has joined #tripleo | 03:44 | |
| *** chem has quit IRC | 03:47 | |
| *** morazi has quit IRC | 03:48 | |
| *** links has joined #tripleo | 03:48 | |
| *** dbecker has quit IRC | 03:54 | |
| pabelanger | EmilienM: http://mirror.dfw.rax.openstack.org:8080/buildlogs.centos/ | 03:57 |
| pabelanger | EmilienM: will land https://review.openstack.org/#/c/489673/ tomorrow | 03:58 |
| *** noslzzp has quit IRC | 04:01 | |
| *** itlinux has joined #tripleo | 04:02 | |
| *** shreshtha has joined #tripleo | 04:04 | |
| stevebaker | pabelanger: do you know when node images will be rebuilt to include this change? https://review.openstack.org/#/c/489434/ | 04:08 |
| *** dbecker has joined #tripleo | 04:09 | |
| pabelanger | http://nodepool.openstack.org/dib-image-list | 04:09 |
| pabelanger | should be starting in 23 mins | 04:09 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 04:10 |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| pabelanger | stevebaker: oh wait, that will need a release of diskimage-builder | 04:10 |
| pabelanger | so, won't land in our images until it is tagged and uploaded to pypi | 04:10 |
| stevebaker | pabelanger: ok, its to fix gate-tripleo-buildimage-overcloud-full-centos-7. Maybe we should make it non-voting so python-tripleoclient things can land | 04:11 |
| pabelanger | stevebaker: I thought that job used a clonemap for diskimage builder | 04:13 |
| *** pmannidi has quit IRC | 04:13 | |
| pabelanger | so, if that code landed, then that jobs would use the master branch | 04:13 |
| pabelanger | ya, unless this is on stable branch jobs | 04:14 |
| stevebaker | pabelanger: I'll try another recheck, maybe I did it a little early | 04:14 |
| pabelanger | then it pulls from upper constraints | 04:14 |
| pabelanger | stevebaker: which review? | 04:14 |
| stevebaker | pabelanger: https://review.openstack.org/#/c/489083/ | 04:14 |
| pabelanger | stevebaker: try adding depends-on to 489434 | 04:15 |
| pabelanger | that should fix it | 04:15 |
| stevebaker | pabelanger: ok, I'll give it a crack | 04:15 |
| *** pmannidi has joined #tripleo | 04:16 | |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Add --set argument to prepare command https://review.openstack.org/489083 | 04:16 |
| *** dmacpher has quit IRC | 04:26 | |
| *** udesale__ has joined #tripleo | 04:29 | |
| *** ykarel has joined #tripleo | 04:30 | |
| *** udesale has quit IRC | 04:31 | |
| itlinux | hello all any tips. on this error .. The following properties for node 4a690a1e-b199-4e92-a029-a74ae587bc1d should be non-negative integers, but provided values are: memory_mb= (HTTP 400) | 04:33 |
| *** dsariel has quit IRC | 04:40 | |
| *** dixiaoli has quit IRC | 04:40 | |
| *** dmacpher has joined #tripleo | 04:58 | |
| *** dmacpher has quit IRC | 05:02 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 05:10 |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix iscsid role data's section https://review.openstack.org/487944 | 05:11 |
| *** ramishra has joined #tripleo | 05:12 | |
| *** skramaja has joined #tripleo | 05:13 | |
| *** ratailor has joined #tripleo | 05:15 | |
| *** jtomasek has joined #tripleo | 05:16 | |
| *** psachin has joined #tripleo | 05:21 | |
| *** pdeore has joined #tripleo | 05:25 | |
| *** ramishra has quit IRC | 05:26 | |
| *** dixiaoli has joined #tripleo | 05:28 | |
| *** ramishra has joined #tripleo | 05:28 | |
| *** dixiaoli has quit IRC | 05:28 | |
| *** dixiaoli has joined #tripleo | 05:29 | |
| *** janki has joined #tripleo | 05:41 | |
| *** anshulbehl has joined #tripleo | 05:42 | |
| *** akrivoka has joined #tripleo | 05:43 | |
| *** masco has joined #tripleo | 05:46 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-pacemaker master: Switch requires for wait-for-settle into resource collectors https://review.openstack.org/489534 | 05:46 |
| *** mdnadeem has quit IRC | 05:47 | |
| *** pgadiya has joined #tripleo | 05:48 | |
| *** mdnadeem has joined #tripleo | 05:48 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-pacemaker master: Switch requires for wait-for-settle into resource collectors https://review.openstack.org/489534 | 05:49 |
| *** mdnadeem has quit IRC | 05:49 | |
| *** mdnadeem has joined #tripleo | 05:49 | |
| *** mdnadeem_ has joined #tripleo | 05:50 | |
| *** mdnadeem has quit IRC | 05:51 | |
| *** mdnadeem has joined #tripleo | 05:51 | |
| *** mdnadeem_ has quit IRC | 05:55 | |
| *** marios has joined #tripleo | 05:55 | |
| *** cshastri has joined #tripleo | 05:59 | |
| *** dsariel has joined #tripleo | 06:03 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: Support deploying OVN as container services https://review.openstack.org/468860 | 06:04 |
| *** lmiccini has joined #tripleo | 06:06 | |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart master: Implement scenario008 (opnfv) https://review.openstack.org/486731 | 06:06 |
| jistr | EmilienM: where did you see the upgrade jobs deploy Pike? they should deploy according to "previous_release_from" function, which for "master" as input will produce "ocata" as output currently: https://github.com/openstack-infra/tripleo-ci/blob/f7eeee942428a00af4194e7ad508ea6f8698c99e/scripts/oooq_common_functions.sh#L8 | 06:07 |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-heat-templates master: Add OPNFV scenario environment https://review.openstack.org/486905 | 06:07 |
| jistr | i'm not sure if anything changed but i'm quite sure they used to deploy ocata originally | 06:08 |
| jistr | QUICKSTART_RELEASE=master-undercloud-ocata-overcloud | 06:09 |
| jistr | http://logs.openstack.org/60/468860/16/check/gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv/7114fa4/console.html#_2017-08-02_02_44_25_319182 | 06:09 |
| *** jfrancoa has joined #tripleo | 06:09 | |
| jistr | EmilienM: all looks ok? or is the problem somewhere else? /cc ccamacho | 06:09 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 06:10 |
| openstack | Launchpad bug 1708014 in tripleo "multinode-oooq-container-upgrades deploys master instead of stable" [Critical,Triaged] | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| * jistr updated the bug ^ | 06:17 | |
| *** rlandy has joined #tripleo | 06:18 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-common master: Add pacemaker packages to ovn_northd image https://review.openstack.org/484394 | 06:18 |
| *** mdnadeem has quit IRC | 06:18 | |
| *** janki has left #tripleo | 06:18 | |
| *** janki has joined #tripleo | 06:18 | |
| *** dpawar has joined #tripleo | 06:19 | |
| *** jaosorior has quit IRC | 06:25 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates stable/ocata: Remove docker references from Ocata scenarios https://review.openstack.org/489874 | 06:29 |
| *** rwsu has quit IRC | 06:30 | |
| jistr | that could fix the scenarios for upgrades ^ | 06:32 |
| *** jaosorior has joined #tripleo | 06:34 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-common master: Add neutron-server-ovn image to the overcloud containers https://review.openstack.org/489875 | 06:34 |
| *** pcaruana has joined #tripleo | 06:34 | |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: DO NOT REVIEW: TESTING ONLY https://review.openstack.org/489097 | 06:36 |
| *** ccamacho has quit IRC | 06:38 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make containerized nova-api run with httpd https://review.openstack.org/475366 | 06:43 |
| *** panda has quit IRC | 06:43 | |
| jaosorior | jistr: hey dude | 06:44 |
| *** mcornea has joined #tripleo | 06:44 | |
| *** rcernin has joined #tripleo | 06:46 | |
| *** panda has joined #tripleo | 06:47 | |
| *** mdnadeem has joined #tripleo | 06:48 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Make containerized nova-api run with httpd https://review.openstack.org/475366 | 06:52 |
| *** jaganathan has joined #tripleo | 06:53 | |
| jaosorior | dciabrin: you around? | 06:55 |
| dciabrin | jaosorior, yep o/ | 06:55 |
| jaosorior | dciabrin: hey dude, I saw that your commit adding the MySQLClient resource to almost all the docker/services files merged | 06:56 |
| dciabrin | jaosorior, yeah saw that yesterday evening, w00t | 06:56 |
| jaosorior | dciabrin: I think it's missing from here though https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/heat-api.yaml#L48 | 06:56 |
| dciabrin | looking | 06:56 |
| *** jlinkes has joined #tripleo | 06:56 | |
| jaosorior | dciabrin: or isn't it? | 06:57 |
| therve | jaosorior, Heat API shouldn't connect to the SQL server | 06:57 |
| dciabrin | jaosorior, hmmm, it should have been catched by the new check in yaml-validate.sh, let me check | 06:57 |
| therve | I don't know if it's the reasoning | 06:57 |
| dciabrin | therve, ah that would explain :) | 06:58 |
| jaosorior | therve: oh, that makes sense | 06:58 |
| *** prateek has joined #tripleo | 06:59 | |
| jaosorior | dciabrin: did the mysql container with TLS work? | 06:59 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable TLS for nova api and placement containers https://review.openstack.org/489885 | 06:59 |
| dciabrin | jaosorior, the mysql server starts now, I also needed to chown the cert on the host so it can be read in the container | 07:00 |
| dciabrin | jaosorior, i'm about to retest a full overcloud deploy | 07:00 |
| jaosorior | dciabrin: awesomeness :D | 07:00 |
| *** bogdando has joined #tripleo | 07:02 | |
| *** oidgar has joined #tripleo | 07:05 | |
| *** agurenko has joined #tripleo | 07:07 | |
| *** yprokule has joined #tripleo | 07:07 | |
| *** apetrich has quit IRC | 07:08 | |
| *** bogdando has quit IRC | 07:08 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 07:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| *** janki is now known as janki|lunch | 07:10 | |
| *** apetrich has joined #tripleo | 07:13 | |
| *** milan has joined #tripleo | 07:16 | |
| *** iranzo has joined #tripleo | 07:17 | |
| *** bogdando has joined #tripleo | 07:18 | |
| *** cylopez has joined #tripleo | 07:19 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Ensure directory exists for certificates for haproxy https://review.openstack.org/489891 | 07:20 |
| *** suuuper has joined #tripleo | 07:21 | |
| *** https_GK1wmSU has joined #tripleo | 07:21 | |
| *** https_GK1wmSU has left #tripleo | 07:22 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Change the directory for haproxy certs/keys to be service-specific https://review.openstack.org/489896 | 07:28 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Download logs interface https://review.openstack.org/473933 | 07:31 |
| *** udesale has joined #tripleo | 07:33 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable TLS for containeried haproxy https://review.openstack.org/489900 | 07:34 |
| *** udesale__ has quit IRC | 07:35 | |
| *** amoralej|off is now known as amoralej | 07:44 | |
| *** rlandy is now known as rlandy|ci_mtgs | 07:46 | |
| *** tesseract has joined #tripleo | 07:48 | |
| *** pblaho has joined #tripleo | 07:49 | |
| *** ccamacho has joined #tripleo | 07:52 | |
| *** egonzalez has joined #tripleo | 07:53 | |
| *** gkadam is now known as gkadam-afk | 07:54 | |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart master: Implement scenario008 (opnfv) https://review.openstack.org/486731 | 07:56 |
| *** iranzo has quit IRC | 08:05 | |
| jaosorior | jistr: any idea what I missed from here https://review.openstack.org/#/c/475366/ ? | 08:09 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 08:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| *** chem has joined #tripleo | 08:11 | |
| *** apetrich has quit IRC | 08:12 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Ensure directory exists for certificates for haproxy https://review.openstack.org/489891 | 08:13 |
| jistr | jaosorior: looks like a CI issue rather | 08:14 |
| jaosorior | jistr: I just noticed. Seems like a lot of patches are failing on the undercloud installed O_o | 08:15 |
| *** derekh has joined #tripleo | 08:15 | |
| jaosorior | on the os-refresh-config bit | 08:15 |
| *** apetrich has joined #tripleo | 08:16 | |
| *** udesale__ has joined #tripleo | 08:20 | |
| *** https_GK1wmSU has joined #tripleo | 08:21 | |
| *** https_GK1wmSU has left #tripleo | 08:22 | |
| *** udesale has quit IRC | 08:22 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Drop step_config as top level docker requirement https://review.openstack.org/442716 | 08:24 |
| jistr | jaosorior: repo issue | 08:24 |
| jistr | http://logs.openstack.org/66/475366/8/check/gate-tripleo-ci-centos-7-containers-multinode/bc8ddca/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-08-02_07_25_48 | 08:24 |
| jistr | maybe unsynced mirror | 08:24 |
| *** pgadiya has quit IRC | 08:25 | |
| *** janki|lunch is now known as janki | 08:25 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Ensure openstack-nova-compute stopped during batch compute upgrade. https://review.openstack.org/457811 | 08:26 |
| *** jpena has joined #tripleo | 08:26 | |
| *** paramite has joined #tripleo | 08:27 | |
| owalsh | marios, chem: hey, I guess you're looking at compute upgrades... | 08:32 |
| *** lucas-afk is now known as lucasagomes | 08:32 | |
| *** ebarrera has joined #tripleo | 08:33 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Drop step_config as top level docker requirement https://review.openstack.org/442716 | 08:33 |
| chem | owalsh: hi, yeah, at some point we have to work on the details of the non-controller upgrade | 08:33 |
| *** openstackgerrit has quit IRC | 08:33 | |
| owalsh | marios, chem: hmm, not sure how the upgrade pin will work | 08:34 |
| *** milan has quit IRC | 08:34 | |
| marios | owalsh: o/ yeah ... it seems this part has not been tested/considered well enough yet for P (i was looking at https://review.openstack.org/#/c/489619/ yesterday) | 08:34 |
| marios | chem: i think if we manage to prove the update_tasks + deploy-steps works well (assuming we get it in time) then we could consider doing the same with upgrade_tasks + deploy _steps as a workbook, so we don't have that script... but i don't know if there is enough time for this | 08:36 |
| marios | owalsh: so i need to check (containers team has really been working on the upgrades code for P) but it should be similar to previous cycles, that is pin on upgrade of controllers, pin on compute upgrade, unpin all on converge | 08:37 |
| *** snecklifter has quit IRC | 08:39 | |
| *** snecklifter has joined #tripleo | 08:40 | |
| *** pgadiya has joined #tripleo | 08:41 | |
| owalsh | marios: ok, I'll need to take a look over it all to refresh my memory... but I figure we need to pin the compute container nova.conf, not the BM nova.conf yes? | 08:41 |
| marios | owalsh: indeed there are the expected env params (environments/major-upgrade-converge-docker.yaml UpgradeLevelNovaCompute: auto ) and then on converge UpgradeLevelNovaCompute: '' | 08:41 |
| chem | marios: owalsh shouldn't this pinning/unpinning be unecessary since some time ? | 08:42 |
| owalsh | chem: no, still required AFAIK | 08:42 |
| *** openstackgerrit has joined #tripleo | 08:42 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/488357 | 08:42 |
| marios | owalsh: ah indeed... so the config will be set in the container config... not sure how that UpgradeLevelNovaCompute is wired up so you may want to start by checking that (i mean is it wired up for the container config, not sure) | 08:43 |
| marios | chem: so apparently there was an issue when we even left it as 'auto' and didn't explicitly set it to '' on converge, recently if i recall correctly (so apparently still a thing) | 08:43 |
| owalsh | marios: so environments/major-upgrade-converge-docker.yaml pins the upgrade level on the controllers, but IIRC it's extraconfig/tasks/tripleo_upgrade_node.sh that pins the computes | 08:43 |
| marios | owalsh: right that is what will be executing on the host that is your compute node - so any docker/config will have to go there (until/unless we get a rework, but that is not clear atm) | 08:44 |
| owalsh | marios: but do we deploy the containerized computes at converge? | 08:44 |
| marios | owalsh: well yeah there should be full config run, i.e. the postdeploy steps. it would have been puppet previously, now it would be docker config (and puppet ...i.e. what you get on 'normal deploy' today) | 08:45 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Support for Dell EMC Unity Cinder Driver https://review.openstack.org/487591 | 08:46 |
| chem | marios: owalsh I guess we still need the puppet run for the host and a "docker run" for the rest | 08:46 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Let the pre-upgrade hook stop services in pike+ https://review.openstack.org/484012 | 08:47 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Support keystone wsgi logging to syslog https://review.openstack.org/489218 | 08:47 |
| chem | marios: owalsh that was the way to mimic what the heat was doing https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/tasks/tripleo_upgrade_node.sh#L61-L69 | 08:47 |
| chem | marios: owalsh we need the "new" way as well | 08:47 |
| owalsh | marios: ack, yea, just about to ask about that | 08:47 |
| marios | chem: yeah i think that is right, would be great if we can grab and apply the docker config there too | 08:47 |
| *** nyechiel_mtg has joined #tripleo | 08:48 | |
| owalsh | marios, chem: we may not need a puppet run | 08:48 |
| marios | chem: well except | 08:48 |
| marios | chem: it would be duplicating what will happen on converge :/ | 08:48 |
| marios | chem: that is what the puppet stuff does too though so.. | 08:48 |
| marios | :/ | 08:48 |
| owalsh | marios: not exactly, we would have the pin | 08:48 |
| chem | marios: yeah, but it's necessary to have the node able to work before converge | 08:49 |
| chem | marios: it's required :) | 08:49 |
| marios | chem: ack | 08:49 |
| prateek | chandankumar, arxcruz , can you please review https://review.openstack.org/#/c/477079/ | 08:49 |
| *** ffiore has joined #tripleo | 08:50 | |
| bogdando | what a pitty https://bugs.launchpad.net/tripleo/+bug/1700045/comments/6 . sigh | 08:51 |
| openstack | Launchpad bug 1700045 in tripleo "httpd logs hidden inside containers" [High,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 08:51 |
| bogdando | jistr: ^^ seems we have no choice but using host mounts | 08:51 |
| chem | owalsh: marios I'll do a launchpad bug that capture this like "Ensure non-controller converge the node", wdyt ? | 08:51 |
| bogdando | pity* but pitty as well | 08:51 |
| chem | owalsh: marios with a note about pin/unpin for the nova rpc ? | 08:52 |
| marios | chem: sounds good (maybe avoid 'converge the node' may not be obvious what yu mean, 'usable after upgrade and before converge' | 08:53 |
| owalsh | chem: sound good to me | 08:53 |
| chem | marios: yeah, that's sound better | 08:53 |
| chem | owalsh: ack | 08:53 |
| *** hewbrocca_afk is now known as hewbrocca | 08:53 | |
| owalsh | marios, chem: in the meantime I'll look over this with lyarwood and sven | 08:54 |
| chem | owalsh: thanks | 08:55 |
| jistr | bogdando: ack | 08:55 |
| arxcruz | prateek: done | 08:55 |
| bogdando | jistr: so sad. I hoped we will have smooth experience with sysloggish containers :D | 08:55 |
| bogdando | larsks: ^^ :) | 08:56 |
| *** dpawar has quit IRC | 08:58 | |
| chem | owalsh: marios https://bugs.launchpad.net/tripleo/+bug/1708115 | 08:59 |
| openstack | Launchpad bug 1708115 in tripleo "Ensure non-controller are usable after upgrade and before converge." [Critical,Triaged] | 08:59 |
| marios | ack owalsh chem | 08:59 |
| owalsh | marios, chem: just to confirm... we definitely don't need to support BM -> BM upgrades | 09:02 |
| chem | owalsh: hum ... not sure what you mean | 09:02 |
| owalsh | chem: i.e upgrading to a non-containerized deployment | 09:03 |
| marios | owalsh: well, i don't think that has been declared explicitly yet, but the focus is on containers and will be the 'recommended' way afaik | 09:04 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Add 'discover and enroll nodes' workflow https://review.openstack.org/488526 | 09:04 |
| chem | owalsh: marios I guess that if we keep the puppet step then it will be covered, and I don't currently see why we should removed it | 09:04 |
| chem | remove | 09:04 |
| chem | owalsh: marios by puppet step I mean that https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/tasks/tripleo_upgrade_node.sh#L61-L69 | 09:05 |
| owalsh | chem: ack | 09:05 |
| owalsh | chem: will need a flag for this then, and only stop/disable services when it's a docker env | 09:06 |
| chem | owalsh: marios it just that the manifest should be mostly empty when using docker | 09:06 |
| marios | chem: right, it would be the 'same' upgrade_tasks at least the difference would be what config is applied after | 09:06 |
| marios | chem: oh, except, we'd need to check/make sure the puppet/services upgrade_tasks are in sync/up to date since this cycle has been all about docker/services upgrade_tasks so those may need updating (the puppet ones) | 09:07 |
| owalsh | marios, chem: yea, e.g we can drop the openstack-nova-migration install | 09:09 |
| chem | marios: and now that I think about it, if we have the puppet upgrade_tasks ok, there is no way to apply them blindly, as owalsh said the non-controller script should be aware of which are puppet and which are not ... | 09:10 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 09:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| chem | are not -> docker | 09:10 |
| marios | owalsh: yeah thats the kind of thing i was thinking of | 09:10 |
| marios | chem: yeah i guess we can find a way (e.g. some hiera value that is only on containerized) | 09:11 |
| chem | marios: owalsh then it will be either puppet or docker but we can't run both | 09:12 |
| owalsh | chem, marios: ack, ok. I'll spin up an env to experiment with this asap | 09:12 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Add validation task in docker services https://review.openstack.org/489940 | 09:13 |
| *** nyechiel_mtg is now known as nyechiel | 09:13 | |
| chem | owalsh: marios in fact with this review https://review.openstack.org/#/c/442716/17 I think that running both can work | 09:14 |
| chem | owalsh: make sure to include it in your env. The puppet manifest should then be empty for all containerized services | 09:14 |
| *** dtantsur|afk is now known as dtantsur | 09:17 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Add validation task in docker services https://review.openstack.org/489940 | 09:19 |
| *** limao has quit IRC | 09:23 | |
| *** ckyriakidou has joined #tripleo | 09:24 | |
| *** dsariel has quit IRC | 09:24 | |
| *** salmankhan has joined #tripleo | 09:25 | |
| *** lmiccini has quit IRC | 09:26 | |
| *** psahoo has quit IRC | 09:27 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-common master: Allow tripleo-admin creation both for Nova-managed and split-stack https://review.openstack.org/489613 | 09:35 |
| *** dsariel has joined #tripleo | 09:37 | |
| *** lmiccini has joined #tripleo | 09:38 | |
| *** tosky has joined #tripleo | 09:48 | |
| *** milan has joined #tripleo | 09:49 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: Fix up multipath docker indentation https://review.openstack.org/489952 | 09:57 |
| *** dixiaoli has quit IRC | 10:04 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Persist containerized services httpd logs https://review.openstack.org/489955 | 10:05 |
| *** anshulbehl has quit IRC | 10:06 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 10:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| *** jaosorior has quit IRC | 10:10 | |
| *** psahoo has joined #tripleo | 10:18 | |
| *** anshulbehl has joined #tripleo | 10:18 | |
| *** ramishra has quit IRC | 10:19 | |
| *** ramishra has joined #tripleo | 10:21 | |
| openstackgerrit | Damien Ciabrini proposed openstack/puppet-tripleo master: Enable TLS configuration for containerized Galera https://review.openstack.org/489956 | 10:22 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Add validation task in docker services [Neutron] https://review.openstack.org/489957 | 10:24 |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart master: Implement scenario008 (opnfv) https://review.openstack.org/486731 | 10:25 |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml https://review.openstack.org/482623 | 10:26 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Enable TLS configuration for containerized Galera https://review.openstack.org/489963 | 10:29 |
| *** pmannidi has quit IRC | 10:31 | |
| *** dsariel has quit IRC | 10:32 | |
| *** ansmith has joined #tripleo | 10:40 | |
| *** thrash|g0ne is now known as thrash | 10:41 | |
| *** ansmith_ has joined #tripleo | 10:41 | |
| *** panda has quit IRC | 10:43 | |
| *** panda has joined #tripleo | 10:44 | |
| *** dsariel has joined #tripleo | 10:47 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart master: enable container periodic job https://review.openstack.org/475743 | 10:48 |
| *** bkopilov_ has quit IRC | 10:50 | |
| amoralej | hi, both ovb jobs fail in periodic pipeline | 10:51 |
| amoralej | with 2017-08-02 08:06:32 | Timed out waiting for messages from Execution (ID: 0cd8b935-d268-4297-b2e7-f6c8d6bf45ac, State: RUNNING). The WebSocket timed out before the Workflow completed. | 10:52 |
| amoralej | is it known issue? | 10:52 |
| *** ansmith_ has quit IRC | 10:53 | |
| *** ansmith has quit IRC | 10:53 | |
| *** anshulbehl has quit IRC | 10:54 | |
| openstackgerrit | Or Idgar proposed openstack/python-tripleoclient master: WIP Integrating octavia post deploy to overcloud deploy https://review.openstack.org/489969 | 10:55 |
| *** pgadiya has quit IRC | 10:56 | |
| *** jaosorior has joined #tripleo | 11:02 | |
| *** atoth has quit IRC | 11:05 | |
| *** anshulbehl has joined #tripleo | 11:07 | |
| amoralej | i just reported issue in periodic jobs in https://bugs.launchpad.net/tripleo/+bug/1708146 | 11:07 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Undecided,New] | 11:07 |
| *** pgadiya has joined #tripleo | 11:08 | |
| *** udesale__ has quit IRC | 11:09 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 11:10 |
| openstackgerrit | Or Idgar proposed openstack/tripleo-common master: WIP Add Octavia post deployment steps to tripleo workflows https://review.openstack.org/487450 | 11:21 |
| *** jkilpatr has joined #tripleo | 11:24 | |
| *** psahoo has quit IRC | 11:25 | |
| *** rhallisey has joined #tripleo | 11:26 | |
| *** ffiore has quit IRC | 11:32 | |
| *** jcoufal has joined #tripleo | 11:34 | |
| *** lucasagomes is now known as lucas-hungry | 11:37 | |
| *** janki has quit IRC | 11:41 | |
| *** ramishra has quit IRC | 11:41 | |
| *** iranzo has joined #tripleo | 11:42 | |
| * jistr reported https://bugs.launchpad.net/tripleo/+bug/1708156 | 11:52 | |
| openstack | Launchpad bug 1708156 in tripleo "Undercloud install fails to find openstack-tripleo-ui RPM" [Critical,Triaged] | 11:52 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Derive Params failure message format change https://review.openstack.org/481524 | 11:53 |
| *** dprince has joined #tripleo | 11:54 | |
| *** oidgar has quit IRC | 11:56 | |
| *** atoth has joined #tripleo | 11:58 | |
| *** sbrzozow has joined #tripleo | 11:59 | |
| bandini | jaosorior: ideas as to how fix the puppet-unit-4.5 job ? http://logs.openstack.org/34/489534/5/check/gate-puppet-pacemaker-puppet-unit-4.5-centos-7/3ee7b4d/console.html | 11:59 |
| bandini | am looking at it, but can't see the issue yet | 12:00 |
| jaosorior | bandini: not really. No idea how that one works. But maybe mwhahaha has seen that before. | 12:00 |
| * mwhahaha hides | 12:00 | |
| bandini | eheh | 12:00 |
| mwhahaha | that's a new one | 12:00 |
| mwhahaha | wonder if pupppet bug since it's failing on some puppet type | 12:01 |
| mwhahaha | or some rspec gem update that's breaking things | 12:01 |
| *** mdnadeem has quit IRC | 12:03 | |
| *** rlandy|ci_mtgs has quit IRC | 12:04 | |
| *** morazi has joined #tripleo | 12:05 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-heat-templates master: Update capabilities map to match latest environments https://review.openstack.org/483873 | 12:05 |
| *** jpena is now known as jpena|lunch | 12:06 | |
| mwhahaha | bandini: it passed locally, so maybe a recheck is in order | 12:06 |
| jaosorior | mwhahaha: it's been failing for a couple of days now | 12:07 |
| * mwhahaha tries again | 12:07 | |
| *** bfournie has quit IRC | 12:07 | |
| michapma | jistr: we have that tripleo-ui rpm issue in apex for offline tests - it's one of the few packages that isn't on the undercloud images already and has to come in from the repo | 12:07 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 12:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708156 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1708156 in tripleo "Undercloud install fails to find openstack-tripleo-ui RPM" [Critical,Triaged] | 12:10 |
| *** dsariel has quit IRC | 12:10 | |
| mwhahaha | jaosorior: oh it looks like stdlib updated something | 12:11 |
| *** jcoufal has quit IRC | 12:11 | |
| mwhahaha | jaosorior: http://logs.openstack.org/34/489534/5/check/gate-puppet-pacemaker-puppet-unit-4.5-centos-7/3ee7b4d/console.html#_2017-08-02_05_57_10_709534 so we might need to pin stdlib for testing | 12:11 |
| bandini | that pw_hash sensitive argument addition thing? | 12:11 |
| mwhahaha | yea | 12:11 |
| bandini | ah I see | 12:12 |
| mwhahaha | it worked on my old fixtures, i reran and it's broken now since i just go teh new version | 12:12 |
| mwhahaha | so i assume it's because we test with 4.5 | 12:12 |
| mwhahaha | https://github.com/puppetlabs/puppetlabs-stdlib/commit/42d4ea7af9197f77a3062727eb166c719ca6296a | 12:13 |
| mwhahaha | broke it | 12:13 |
| *** shreshtha has quit IRC | 12:13 | |
| *** dmacpher has joined #tripleo | 12:15 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/487989 | 12:16 |
| *** lblanchard has joined #tripleo | 12:16 | |
| *** noslzzp has joined #tripleo | 12:16 | |
| *** lblanchard has quit IRC | 12:18 | |
| *** amoralej is now known as amoralej|lunch | 12:18 | |
| mwhahaha | so latest stdlib requires pouppet 4.7 | 12:19 |
| bandini | oh | 12:19 |
| mwhahaha | so we need less than 4.17.0 | 12:20 |
| mwhahaha | (stdlib) | 12:20 |
| mwhahaha | or update our jobs to 4.8 | 12:20 |
| mwhahaha | the problem being that tripleo uses 4.6 | 12:20 |
| bandini | let's start with the first option? | 12:21 |
| mwhahaha | so we'll need to pin rdo and puppet to use less < 4.17 | 12:21 |
| mwhahaha | https://review.openstack.org/490004 to fix puppet tests | 12:23 |
| mwhahaha | jaosorior: what's the best way to pin stdlib in rdo to 4.16.0? | 12:24 |
| *** chem has quit IRC | 12:25 | |
| atoth | beagles, have a sec? | 12:25 |
| beagles | atoth, yup | 12:25 |
| atoth | beagles, on the repo fron, I'm not sure I'm including the right ones in the image. which RDO repo(s) would you say I should include? | 12:26 |
| *** chem has joined #tripleo | 12:26 | |
| *** ramishra has joined #tripleo | 12:26 | |
| beagles | atoth: probably the ones that you used when creating the image I think (i.e. ones from the undercloud) - otherwise you'll end upgrading any dependencies that have changed version | 12:27 |
| beagles | atoth, does that make sense to you? | 12:28 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/487398 | 12:29 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/488148 | 12:29 |
| atoth | beagles, all repos from undercloud? | 12:29 |
| *** amoralej|lunch is now known as amoralej | 12:29 | |
| atoth | beagles, I'm assuming the delorean*.repo are the openstack repos | 12:30 |
| atoth | beagles, which is what I added but still does not work | 12:30 |
| beagles | atoth, just the delorean ones I thin | 12:30 |
| beagles | atoth, k... then you need to sort out why :) | 12:30 |
| atoth | beagles, yep, just making sure I started correctly :-) | 12:31 |
| beagles | atoth, by doesn't work do you mean that the puppet outright fails or do you mean "runs, but does not install packages"? | 12:31 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-common master: Allow tripleo-admin creation both for Nova-managed and split-stack https://review.openstack.org/489613 | 12:31 |
| *** pchavva has joined #tripleo | 12:32 | |
| *** iranzo has quit IRC | 12:32 | |
| *** ratailor has quit IRC | 12:32 | |
| *** ebarrera has quit IRC | 12:32 | |
| fultonj | jistr ^ thanks, I think I could use that for my problem with https://review.openstack.org/479288 | 12:32 |
| jistr | d0ugal, thrash: have you ever seen a nova.servers_list mistral action (or any action really) stuck like this? http://paste.openstack.org/show/617254/ | 12:32 |
| jistr | fultonj: yea that's the purpose of that patch :) | 12:33 |
| jaosorior | mwhahaha: I don't know :/ | 12:33 |
| mwhahaha | jaosorior: i checked, the rdo pkg should be ok for now because it hasn't picked up that commit | 12:33 |
| atoth | beagles, same issue as always, doesn't find certain classes and stuff, but works just fine when the 2 repos are actually installed in the image | 12:34 |
| atoth | beagles, rpms | 12:34 |
| jistr | d0ugal, thrash: the stuck action is the one on line 90 here https://review.openstack.org/#/c/489613/4/workbooks/access.yaml | 12:34 |
| *** ebarrera has joined #tripleo | 12:35 | |
| atoth | beagles, openstack-neutron-lbaas.noarch and python-neutron-lbaas.noarch to be exact | 12:35 |
| atoth | beagles, so I'm just trying to figure out how it all works ;-) | 12:36 |
| d0ugal | jistr: the nova.servers_list action is running but not completing? | 12:36 |
| jistr | d0ugal: yea exactly | 12:36 |
| d0ugal | jistr: any errors in the executor log? | 12:36 |
| *** liverpooler has joined #tripleo | 12:36 | |
| d0ugal | jistr: I've not seen it with this action, but I have seen actions fail badly and Mistral doesn't mark them as finished - not seen this for some time tho | 12:36 |
| beagles | atoth, k. so it isn't actually installing the packages.. you could adding some warning messages in the puppet in relevant places and try rerunning the last puppet apply you find in the /var/log/messages - that way you can check if the expected puppet code paths are getting hit | 12:37 |
| thrash | jistr: same here... | 12:37 |
| jistr | d0ugal: no errors, this is tail of executor log http://paste.openstack.org/show/617255/ | 12:38 |
| atoth | beagles, that would be a good option, any quick pointer to something spitting out a log message? | 12:38 |
| jistr | looks like it got the response from nova | 12:38 |
| beagles | atoth: https://stackoverflow.com/questions/4135426/how-to-print-something-when-running-puppet-client | 12:38 |
| jistr | thrash, d0ugal: i managed to find this though https://github.com/StackStorm/st2/issues/2536. It's not exactly the same, but i do use with-items in the action right after the "not finishing one" | 12:39 |
| jistr | so i wonder if the get_servers is actually ok (we do use it elsewhere) | 12:39 |
| *** lucas-hungry is now known as lucasagomes | 12:39 | |
| jistr | and mistral doesn't want to start the next one for some reason | 12:39 |
| * jistr will try reverting to previous state where i didn't get this error yet | 12:40 | |
| d0ugal | jistr: hmm, possibly. I am going to experiment in my env | 12:40 |
| *** bfournie has joined #tripleo | 12:41 | |
| atoth | beagles, thanks, looking through it now but if you know off hand, where will those log messages go? | 12:41 |
| beagles | atoth, /var/log/messages | 12:41 |
| atoth | beagles, ah, cool, thanks | 12:41 |
| *** chem has quit IRC | 12:42 | |
| *** oidgar has joined #tripleo | 12:42 | |
| *** chem has joined #tripleo | 12:43 | |
| *** chem has quit IRC | 12:43 | |
| jistr | d0ugal, thrash: hmm funny, it seems like it gets stuck when i use jinja in the next task. I may need jinja as i need the |tojson(indent=4) filter. Unfortunately for running ansible via os-collect-config i need the tasks as string, while for running it via Mistral i need the tasks as a data structure. I could just write the whole thing twice, but i'm still trying to have it DRY as datastructure and then just | 12:43 |
| jistr | convert it to string for the os-collect-config way. | 12:44 |
| jistr | not sure if there's |tojson(indent=4) equivalent with the <% %> yaql templates (?) | 12:44 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Add Neutron SR-IOV agent container https://review.openstack.org/469066 | 12:45 |
| *** rbowen has quit IRC | 12:45 | |
| *** chem has joined #tripleo | 12:47 | |
| *** amoralej is now known as amoralej|lunch | 12:47 | |
| *** akrivoka has quit IRC | 12:47 | |
| jistr | d0ugal, thrash: so when i change it to http://paste.openstack.org/show/617257/ it doesn't get stuck, but it also doesn't work as the playbook is garbled | 12:48 |
| *** gkadam-afk has quit IRC | 12:48 | |
| d0ugal | jistr: interesting, maybe we have a bug with jinja2 | 12:48 |
| jistr | yea possibly.. | 12:50 |
| jistr | this is how the garbled playbook looks. If i could get it from the pythonish output to proper json, we'd be golden. http://paste.openstack.org/show/617258/ | 12:50 |
| * jistr searches if yaql can do that somehow | 12:51 | |
| d0ugal | jistr: I am trying to find out how to do this with YAQL | 12:51 |
| * jistr too | 12:51 | |
| d0ugal | jistr: <% json_pp($.tasks) %> | 12:52 |
| *** Goneri has joined #tripleo | 12:52 | |
| jistr | d0ugal: whoohoo great thank you very much | 12:52 |
| d0ugal | I think, A custom YAQL function added by Mistral | 12:52 |
| fultonj | wow | 12:53 |
| thrash | jistr: I was going to say <% str($.tasks) %> if it doesn't necessarily need to be pp | 12:53 |
| thrash | but not sure that would quite do it. | 12:53 |
| thrash | d0ugal's is better. :D | 12:53 |
| *** tzumainn has joined #tripleo | 12:55 | |
| jistr | Aug 02 12:54:24 overcloud-controller-0 os-collect-config[1270]: PLAY RECAP ********************************************************************* | 12:55 |
| jistr | Aug 02 12:54:24 overcloud-controller-0 os-collect-config[1270]: localhost : ok=6 changed=5 unreachable=0 failed=0 | 12:55 |
| jistr | [root@overcloud-controller-0 ~]# getent passwd tripleo-admin | 12:55 |
| jistr | tripleo-admin:x:1001:1001::/home/tripleo-admin:/bin/bash | 12:55 |
| jistr | [ | 12:55 |
| jistr | perfect! :)) | 12:56 |
| jistr | d0ugal, thrash thanks a lot | 12:56 |
| d0ugal | yay | 12:56 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: Add timestamper_cmd for tripleo-validations https://review.openstack.org/490013 | 12:56 |
| d0ugal | np | 12:56 |
| fultonj | thanks d0ugal jistr thrash | 12:56 |
| fultonj | jistr: may i add you to https://review.openstack.org/#/c/489825 ? | 12:58 |
| fultonj | jistr: i am about to reply but would appreicate if you could check my understanding | 12:58 |
| bogdando | folks, do you know from which puppet module the cron class comes into puppet-openstack as a dependency? I can't locate it... | 12:59 |
| jistr | fultonj: ack i added myself. Feel free to add me to any review you need. I just have a lot of those incoming (currently 332 unread gerrit messages) so if i miss something and you want my feedback, feel free to ping. | 12:59 |
| bogdando | mwhahaha, EmilienM: ^ ^ perchance? | 12:59 |
| *** lblanchard has joined #tripleo | 13:00 | |
| mwhahaha | bogdando: what do you mean | 13:00 |
| bogdando | mwhahaha: I need to pin it in the rdoinfo | 13:00 |
| mwhahaha | bogdando: cron { } is a puppet resource | 13:00 |
| bogdando | mwhahaha: oops | 13:00 |
| bogdando | mwhahaha: no more questions :) | 13:00 |
| bogdando | thanks! | 13:00 |
| mwhahaha | k :D | 13:00 |
| fultonj | jistr: ok, thanks. it's regarding the generation of ceph.conf and how we map it to containers using kolla_config: and then volumes: that we had talked about previously. | 13:00 |
| *** rbowen has joined #tripleo | 13:02 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Drop step_config as top level docker requirement https://review.openstack.org/442716 | 13:02 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: docker: Add unit tests on service_name https://review.openstack.org/442755 | 13:02 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Add docker templates to configure Ironic inspector https://review.openstack.org/457822 | 13:02 |
| slagle | fultonj: one thing to consider, does /etc/ceph exist if no ceph packages are installed on the host? | 13:03 |
| slagle | fultonj: and if it doesn't, is that going to cause the docker run to fail, since the path does not exist | 13:04 |
| fultonj | slagle: it exists as per the original patch in either case. a line was added to ensure that. i will dig it up. | 13:05 |
| fultonj | slagle: it did cause the docker run to fail when the original patch was going in, but that line was added to address it | 13:05 |
| *** jmelvin has joined #tripleo | 13:05 | |
| slagle | fultonj: i see it. in host_prep_tasks | 13:06 |
| *** jpena|lunch is now known as jpena | 13:06 | |
| *** jcoufal has joined #tripleo | 13:07 | |
| slagle | this just feel like way too tight coupling between ceph and cinder_volume | 13:07 |
| slagle | now we need a /etc/ceph, just to run cinder_volume? | 13:07 |
| slagle | even if you are not using ceph, or dont even want it installed | 13:08 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 13:08 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-docs master: Post deployment installation instructions for Octavia https://review.openstack.org/447496 | 13:08 |
| fultonj | it's an empty directory otherwise, that already exists as per the overcloud image | 13:08 |
| slagle | fultonj: we shouldn't have a dependency on it. split-stack doesn't use images | 13:09 |
| fultonj | i added more details in reply to you and bogdando | 13:09 |
| *** masco has quit IRC | 13:10 | |
| fultonj | in the split stack scenario wouldn't host_prep_tasks take care of it? | 13:10 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 13:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708156 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1708156 in tripleo "Undercloud install fails to find openstack-tripleo-ui RPM" [Critical,Triaged] | 13:10 |
| slagle | fultonj: it would. but cinder_volume requiring /etc/ceph to exist is odd | 13:11 |
| slagle | it's cinder_volume. it supports different backends. it shouldn't require anything ceph specific if not using ceph | 13:11 |
| *** pradk has quit IRC | 13:13 | |
| *** bkopilov_ has joined #tripleo | 13:13 | |
| *** ecerquei has joined #tripleo | 13:13 | |
| slagle | fultonj: is the ceph configuration generated with docker-puppet.py? | 13:14 |
| slagle | fultonj: i'm trying to understand the reasoning here. because no other services that use docker-puppet.py generate config files under /etc on the container host | 13:14 |
| slagle | or does puppet-ceph actually get applied directly on the host? | 13:15 |
| fultonj | the ceph configuration not generated with docker-puppet.py | 13:15 |
| fultonj | the configuration is genereated directly on the host | 13:15 |
| * fultonj gets spec | 13:15 | |
| *** sbrzozow has quit IRC | 13:15 | |
| fultonj | https://review.openstack.org/#/c/387631/ | 13:16 |
| fultonj | i suppose i could generate the configuration in another directory in another patch | 13:16 |
| *** sbrzozow has joined #tripleo | 13:17 | |
| fultonj | but the proposed patch fixes the bug of the container not starting without taking out the work that was done to implement the spec | 13:17 |
| *** athomas has quit IRC | 13:18 | |
| slagle | fultonj: sure, we can go with that. it needs to get passing CI though | 13:19 |
| *** sri__ has quit IRC | 13:20 | |
| fultonj | slagle: ok, thanks very much | 13:20 |
| slagle | i dont know why it faiiled on scenario0002, or if that has anything to do with the patch or not | 13:20 |
| *** jlabarre has joined #tripleo | 13:20 | |
| slagle | the logs didn't get captured it looks like | 13:20 |
| fultonj | i'll get it passing ci next then | 13:20 |
| fultonj | slagle: i'll open another bug for cinder not requring /etc/ceph you can follow | 13:21 |
| fultonj | if you like | 13:21 |
| mwhahaha | cinder requires /etc/ceph/ceph.conf when configured with rbd | 13:24 |
| fultonj | true | 13:24 |
| fultonj | mwhahaha: the bug would be that it not requre /etc/ceph to exist if it is not configured with rbd | 13:25 |
| slagle | mwhahaha: yes. it's the tight coupling between cinder_volume and /etc/ceph when not using ceph that I object to, on stricly nit-picking grounds :) | 13:25 |
| mwhahaha | eh sounds like a personal problem to me :D | 13:26 |
| slagle | one of many | 13:26 |
| *** athomas has joined #tripleo | 13:27 | |
| mwhahaha | seems to be a limitation in our ability to know when it is or isn't configured for inclusion with docker | 13:27 |
| mwhahaha | pesky mount paths | 13:28 |
| slagle | yaql! | 13:28 |
| slagle | it can be done | 13:28 |
| * mwhahaha shudders | 13:28 | |
| *** prateek has quit IRC | 13:28 | |
| *** pgadiya has quit IRC | 13:29 | |
| mwhahaha | it can eb done, but should it be done | 13:29 |
| slagle | yaql is the basis for everything we do in tht | 13:29 |
| slagle | not that you should shudder any less about that :) | 13:29 |
| mwhahaha | :D | 13:29 |
| fultonj | :) | 13:29 |
| * fultonj cant't wait to change it with yaql | 13:30 | |
| zoli | ping all - anyone able to help with this error? http://paste.openstack.org/show/617266/ | 13:31 |
| *** jistr is now known as jistr|mtg | 13:32 | |
| slagle | zoli: i would guess that your overcloud node can't resolve undercloud.localdomain | 13:35 |
| slagle | however, curl return code 7 is actually "Failed to connect() to host or proxy. " | 13:35 |
| slagle | and not couldn't resolve | 13:36 |
| slagle | per https://curl.haxx.se/libcurl/c/libcurl-errors.html | 13:36 |
| slagle | so i'll change my guess to undercloud.localdomain not resolving to the IP where swift is bound | 13:36 |
| zoli | slagle, it resolves hostname/IP: | 13:36 |
| zoli | PING undercloud.localdomain (192.168.101.227) 56(84) bytes of data. | 13:36 |
| zoli | 64 bytes from undercloud.localdomain (192.168.101.227): icmp_seq=1 ttl=64 time=0.071 ms | 13:36 |
| slagle | is that the undercloud IP where swift services are bound? | 13:37 |
| *** amoralej|lunch is now known as amoralej | 13:37 | |
| zoli | slagle, checking | 13:38 |
| *** ansmith_ has joined #tripleo | 13:38 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 13:38 |
| openstackgerrit | Dmitry Tantsur proposed openstack/instack-undercloud master: Run online_data_migrations for Ironic on upgrade https://review.openstack.org/490028 | 13:38 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates master: Add Mistral event engine https://review.openstack.org/462975 | 13:39 |
| *** ansmith has joined #tripleo | 13:39 | |
| *** ecerquei has quit IRC | 13:40 | |
| *** ansmith__ has joined #tripleo | 13:40 | |
| *** ansmith has quit IRC | 13:40 | |
| *** ansmith_ has quit IRC | 13:41 | |
| *** rwsu has joined #tripleo | 13:41 | |
| *** rwsu has quit IRC | 13:41 | |
| *** rwsu has joined #tripleo | 13:41 | |
| *** ansmith__ has quit IRC | 13:41 | |
| *** ansmith has joined #tripleo | 13:44 | |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates master: Adds networking-sfc support https://review.openstack.org/489749 | 13:45 |
| *** jistr|mtg is now known as jistr | 13:52 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Enable repo for python-os-testr https://review.openstack.org/490032 | 13:55 |
| jistr | fultonj: i'll probably just squash this one https://review.openstack.org/#/c/489225 into the bigger patch that follows it | 13:56 |
| jistr | fultonj: it doesn't do much good by itself | 13:56 |
| fultonj | jistr: ok, it gather it's kind of like a POC | 13:57 |
| fultonj | thanks jistr | 13:58 |
| -openstackstatus- NOTICE: We have disable infracloud-vanilla due to the compute host running mirror.regionone.infracloud-vanilla.o.o being offline. Please recheck your failed jobs to schedule them to another cloud. | 13:58 | |
| *** agopi|away has joined #tripleo | 13:59 | |
| jistr | fultonj: yea originally it was meant as the full solution but that was before we found out that none of the actions work with split stack *at all* :)) | 14:00 |
| *** jmelvin has quit IRC | 14:02 | |
| *** fandrieu has joined #tripleo | 14:05 | |
| *** udesale__ has joined #tripleo | 14:06 | |
| *** cdearborn has joined #tripleo | 14:07 | |
| EmilienM | hi | 14:07 |
| *** skramaja has quit IRC | 14:08 | |
| *** rbrady_ has joined #tripleo | 14:09 | |
| *** rbrady_ has quit IRC | 14:09 | |
| *** rbrady_ has joined #tripleo | 14:09 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708014 | 14:10 |
| openstack | Launchpad bug 1708014 in tripleo "scenarios in Ocata reference services introduced in Pike" [Critical,Triaged] - Assigned to Jiřà Stránský (jistr) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708156 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1708156 in tripleo "Undercloud install fails to find openstack-tripleo-ui RPM" [Critical,Triaged] | 14:10 |
| EmilienM | jistr, ccamacho: ok but how do you explain https://review.openstack.org/#/c/489710/ jobs are failing | 14:11 |
| EmilienM | jistr: ah, we need https://review.openstack.org/#/c/489874/ ? | 14:11 |
| jistr | EmilienM: i updated the bug. It's just the scenarios that are wrong. | 14:11 |
| jistr | EmilienM: yea i think so | 14:12 |
| EmilienM | jistr: ahhhh :) I can remove the alert ? | 14:12 |
| jistr | EmilienM: +1 yea | 14:12 |
| EmilienM | ok | 14:12 |
| jistr | EmilienM: so the normal upgrade job goes Ocata->Master as it should have. And the scenario upgrades weren't made to work yet. | 14:12 |
| EmilienM | jistr: ok good | 14:13 |
| EmilienM | jistr: so my patch merged in project-config - let's make the job working ! | 14:13 |
| *** rbrady_ has quit IRC | 14:15 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 14:16 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-common master: Allow tripleo-admin creation both for Nova-managed and split-stack https://review.openstack.org/489613 | 14:16 |
| *** jcoufal has quit IRC | 14:18 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Default $NODEPOOL_PROVIDER https://review.openstack.org/490037 | 14:20 |
| jistr | fultonj: ^ i hope that should be it as far as that patch goes. However, the CI will not be passing yet. We need to execute the workflow differently in CI, i.e. extend tripleo-quickstart and/or tripleo-ci to run the workflow with proper ssh_user, ssh_private_key and ssh_servers. | 14:20 |
| jistr | ..that match the CI environment | 14:20 |
| *** prateek has joined #tripleo | 14:20 | |
| *** mdnadeem has joined #tripleo | 14:20 | |
| openstackgerrit | Sébastien Han proposed openstack/tripleo-heat-templates master: openstack-heat-templates: fix deprecation path https://review.openstack.org/490038 | 14:21 |
| jistr | slagle: ^ fyi eventually i went with creating the new ssh admin user as that will 1) give a consistent environment across split-stack and Nova-managed and 2) we will not have to pass ssh creds to *every* workflow that needs to run Ansible on overcloud, we'll be able to assume those. | 14:21 |
| jistr | slagle: i haven't gone for Heat woodoo when figuring out how to create the user though, i went for providing explicit ssh creds. So the ssh admin creation workflow should be the only workflow where we pass explicit ssh creds in the splitstack case. The rest of the workflows can assume we're using the created user/creds. | 14:22 |
| itlinux | hello all any suggestions on this error Failure caused by error in tasks: fail_workflow\n\n fail_workflow [ | 14:23 |
| itlinux | the workflow timed out.. on my introspect | 14:23 |
| jistr | slagle: if i'm making any sense :) | 14:24 |
| slagle | jistr: yea, i follow, makes sense to me | 14:25 |
| jistr | slagle: cool, thanks | 14:25 |
| slagle | +1 for less Heat voodoo...although at this point we're pretty deep :) | 14:25 |
| *** oidgar has quit IRC | 14:25 | |
| itlinux | hi all | 14:27 |
| *** jmelvin has joined #tripleo | 14:27 | |
| itlinux | any suggestions on introspect with Dell R630 looks like a known issue.. TY | 14:27 |
| * fultonj reads up | 14:27 | |
| *** rwsu has quit IRC | 14:29 | |
| hewbrocca | heh | 14:29 |
| hewbrocca | deep voodoo or deep doodoo? | 14:29 |
| adarazs | EmilienM: hi, are you around? | 14:30 |
| *** jcoufal has joined #tripleo | 14:30 | |
| itlinux | http://paste.openstack.org/raw/617279/ | 14:32 |
| *** thrash is now known as thrash|biab | 14:33 | |
| *** udesale has joined #tripleo | 14:34 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud master: Install pankoclient on undercloud https://review.openstack.org/490044 | 14:34 |
| amoralej | could someone take a look at https://bugs.launchpad.net/tripleo/+bug/1708146 today's ovb jobs in periodic failed with that | 14:34 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Undecided,New] | 14:34 |
| *** pdeore has quit IRC | 14:35 | |
| *** udesale__ has quit IRC | 14:35 | |
| EmilienM | adarazs: in a mtg | 14:36 |
| *** ykarel has quit IRC | 14:36 | |
| adarazs | EmilienM: ack. when you have time, have you seen this/do we have a bug for it? http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/e7e67f0/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-08-02_08_06_32 | 14:36 |
| amoralej | adarazs, https://bugs.launchpad.net/tripleo/+bug/1708146 | 14:37 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Undecided,New] | 14:37 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 14:37 |
| amoralej | i reported it earlier | 14:37 |
| adarazs | amoralej: I should have asked you first :) | 14:37 |
| adarazs | amoralej: thanks. | 14:37 |
| *** ffiore has joined #tripleo | 14:38 | |
| adarazs | hewbrocca: so this blocks upstream promotion: https://bugs.launchpad.net/tripleo/+bug/1708146 -- which blocks RDO CI promotion due to weirdo failing. we should probably have some mistral folks on it. | 14:38 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Undecided,New] | 14:38 |
| * d0ugal looks | 14:39 | |
| d0ugal | That error is usually a sign that something is wrong with Zaqar. | 14:39 |
| *** rwsu has joined #tripleo | 14:39 | |
| *** jlinkes has quit IRC | 14:40 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add pre-deployment negative tests for validations https://review.openstack.org/488495 | 14:40 |
| openstackgerrit | Michael Bayer proposed openstack/puppet-tripleo master: Enable innodb_buffer_pool_size configuration https://review.openstack.org/490046 | 14:40 |
| d0ugal | adarazs: there is no activity in the zaqar logs: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/e7e67f0/logs/undercloud/var/log/zaqar/zaqar.log.txt.gz | 14:40 |
| d0ugal | Only a 404, which is weird. | 14:41 |
| *** pdeore has joined #tripleo | 14:41 | |
| *** ckyriakidou has quit IRC | 14:41 | |
| *** panda has quit IRC | 14:43 | |
| *** panda has joined #tripleo | 14:44 | |
| therve | d0ugal, That's only the websocket logs though | 14:44 |
| therve | The http ones have info | 14:44 |
| d0ugal | ah | 14:45 |
| *** prateek has quit IRC | 14:45 | |
| d0ugal | therve: where do I find those? | 14:45 |
| therve | d0ugal, http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/e7e67f0/logs/undercloud/var/log/httpd/zaqar_wsgi_access.log.txt.gz | 14:45 |
| d0ugal | got it | 14:45 |
| d0ugal | thanks | 14:45 |
| d0ugal | hmm | 14:46 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Add logrotate-crond configuration https://review.openstack.org/490048 | 14:47 |
| therve | It looks like the queue got messages posted accordingly | 14:47 |
| therve | So maybe a failure with the subscription | 14:47 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service https://review.openstack.org/490050 | 14:48 |
| *** ckyriakidou has joined #tripleo | 14:55 | |
| *** ratailor has joined #tripleo | 14:57 | |
| *** paramite has quit IRC | 14:58 | |
| hewbrocca | d0ugal: any more ideas? | 15:00 |
| *** ykarel has joined #tripleo | 15:01 | |
| *** links has quit IRC | 15:01 | |
| d0ugal | hewbrocca: no, not yet - still looking at the logs. | 15:02 |
| d0ugal | I'm not sure what scenarios would cause the websocket to timeout. | 15:03 |
| fultonj | jistr: thanks, i'll try depending on that in my CI change to scenario001 | 15:06 |
| *** jistr is now known as jistr|mtg | 15:06 | |
| karthiks | bnemec, dsneddon Can you please review https://review.openstack.org/#/c/480540/ | 15:08 |
| *** mdnadeem has quit IRC | 15:08 | |
| itlinux | hello guys is this a bug or something that I can find a workaround for it. :http://paste.openstack.org/raw/617279/ | 15:09 |
| itlinux | Thanks | 15:09 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708156 | 15:10 |
| openstack | Launchpad bug 1708156 in tripleo "Undercloud install fails to find openstack-tripleo-ui RPM" [Critical,Triaged] | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| *** ebarrera has quit IRC | 15:10 | |
| d0ugal | therve: so, maybe we need to try and write something that attepts to reconnect on a timeout? | 15:11 |
| d0ugal | I guess websocket-client may raise WebSocketTimeoutException if the connection is interrupted? | 15:11 |
| bogdando | slagle, jistr|mtg: https://bugs.launchpad.net/tripleo/+bug/1708180/comments/1 jfyi | 15:12 |
| openstack | Launchpad bug 1708180 in tripleo "Remote ssh administration user creation not working with split stack" [High,In progress] - Assigned to Jiřà Stránský (jistr) | 15:12 |
| *** nyechiel has quit IRC | 15:12 | |
| d0ugal | therve: they both seemed to timeout after ~8mins - could something be killing connections after that length of time? | 15:12 |
| therve | d0ugal, Probably, but I don't that is the issue. It's not getting messages most likely | 15:13 |
| d0ugal | therve: sorry, I don't follow - what isn't getting messages? | 15:14 |
| d0ugal | do you mean just because it is idle? | 15:14 |
| therve | d0ugal, The websocket connection is not getting the message | 15:14 |
| *** dprince has quit IRC | 15:14 | |
| therve | Looking the previous build, that step finished in about 3 minutes | 15:15 |
| therve | So it should fit in the 8 minutes we see here | 15:15 |
| d0ugal | Right | 15:15 |
| d0ugal | so where are the messages getting lost? :) | 15:15 |
| *** liverpooler has quit IRC | 15:16 | |
| therve | Yeah that's the question | 15:17 |
| therve | It looks like they are correctly posted | 15:19 |
| therve | And zaqar doesn't complain when it gets them | 15:19 |
| *** liverpooler has joined #tripleo | 15:19 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Convert scenario001-multinode-containers job to ceph-ansible https://review.openstack.org/479288 | 15:19 |
| d0ugal | therve: Yup, that was my conclusion too | 15:19 |
| d0ugal | therve: unfortunately we have less information from tripleoclient. | 15:19 |
| d0ugal | We should really have it always log the debug output. | 15:20 |
| *** cshastri has quit IRC | 15:20 | |
| d0ugal | Also, why are workflows started after there is a timeout. | 15:20 |
| therve | Yeah that's another story :) | 15:21 |
| therve | So I just tested websockets subs on master, and it works, so at least that is not completely broken | 15:21 |
| d0ugal | and it doesn't print the "Started Mistral Workflow ..." before the timout | 15:21 |
| d0ugal | so which workflow is actually timing out. | 15:21 |
| d0ugal | (not that the workflow itself seems to be timing out, but you get what I mean I think) | 15:22 |
| *** psachin has quit IRC | 15:22 | |
| d0ugal | The check_pre_deployment_validations workflow is started here: https://github.com/openstack/python-tripleoclient/blob/c6cd190a9f63738deafa4c616e0be8e06afedc96/tripleoclient/v1/overcloud_deploy.py#L856 | 15:23 |
| d0ugal | I can't see a workflow call before that | 15:24 |
| *** ratailor has quit IRC | 15:24 | |
| therve | d0ugal, There is more logs in messages | 15:29 |
| d0ugal | therve: ? | 15:30 |
| d0ugal | oh | 15:30 |
| therve | d0ugal, zaqar websocket logs there for some reasons | 15:30 |
| d0ugal | Interesting | 15:30 |
| d0ugal | Looking | 15:30 |
| therve | OK found the one | 15:31 |
| therve | d0ugal, queue 5205ebec-ee69-401d-99a5-4e36703a3368 | 15:31 |
| therve | takes about 8 mins to get the message | 15:31 |
| therve | 45s on the previous build | 15:32 |
| d0ugal | Interesting | 15:32 |
| d0ugal | Any guesses why? | 15:32 |
| hewbrocca | Mistral went out for a smoke? | 15:33 |
| *** iranzo has joined #tripleo | 15:33 | |
| *** iranzo has quit IRC | 15:33 | |
| *** iranzo has joined #tripleo | 15:33 | |
| *** liverpooler has quit IRC | 15:34 | |
| therve | Well yeah the 8am coffee break | 15:34 |
| dtantsur | even mistral needs coffee | 15:34 |
| *** udesale has quit IRC | 15:34 | |
| * dtantsur had 4 coffees today to just be able to work a bit | 15:35 | |
| therve | dtantsur, And how many workflows did you complete? :) | 15:35 |
| dtantsur | haha, not so many, unfortunately | 15:35 |
| slagle | bnemec: tripleo-repos enables https://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/ for current-tripleo. which is ancient for some reason. is that the right link? | 15:35 |
| *** liverpooler has joined #tripleo | 15:36 | |
| *** yprokule has quit IRC | 15:36 | |
| d0ugal | therve: I see that queue UUID in messages - why don't I see it in the mistral logs? does it have more than one ID? | 15:36 |
| d0ugal | oh wait | 15:36 |
| d0ugal | I see it | 15:36 |
| therve | Yeah it's there | 15:36 |
| d0ugal | I think I was in the wrong file, sorry | 15:36 |
| bnemec | slagle: It may be out of date now. The repos were current as of many months ago, but it looks like we're no longer using buildlogs in the docs. | 15:38 |
| bnemec | slagle: It looks like we should be using https://trunk.rdoproject.org/centos7-master/current-passed-ci/delorean.repo now | 15:39 |
| itlinux | hi bnemec: could you suggest what could be the issue on instack http://paste.openstack.org/raw/617279/ | 15:40 |
| itlinux | ty | 15:40 |
| *** suuuper has quit IRC | 15:42 | |
| *** anshulbehl has quit IRC | 15:42 | |
| *** mcornea has quit IRC | 15:44 | |
| d0ugal | therve: I might be reading the wrong, but wasn't a message sent at 8am and 8:08? | 15:44 |
| d0ugal | i.e. two messages, not one taking that long | 15:45 |
| d0ugal | http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/e7e67f0/logs/undercloud/var/log/mistral/engine.log.txt.gz#_2017-08-02_08_00_32_505 | 15:45 |
| d0ugal | http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/e7e67f0/logs/undercloud/var/log/mistral/engine.log.txt.gz#_2017-08-02_08_08_15_005 | 15:45 |
| therve | d0ugal, Yeah it times out at 8:06 though | 15:45 |
| *** itlinux has quit IRC | 15:45 | |
| therve | d0ugal, Looks like a heat issue | 15:49 |
| hewbrocca | O NOES | 15:50 |
| d0ugal | therve: what. How? | 15:51 |
| therve | d0ugal, It's the one taking 8 minutes | 15:51 |
| d0ugal | ah | 15:52 |
| *** nyechiel has joined #tripleo | 15:52 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Prepare pike.yaml config for TripleO CI https://review.openstack.org/490066 | 15:53 |
| EmilienM | adarazs: can you review this one please? did I miss something? Also, do you know if we can have builds for pike instead of using master? ^ | 15:53 |
| therve | Probably 4a4a172d5b464c1bfc080da15a6a3c2af51a2be5 | 15:53 |
| adarazs | EmilienM: looking | 15:53 |
| *** milan has quit IRC | 15:57 | |
| *** iranzo has quit IRC | 15:57 | |
| *** ykarel has quit IRC | 15:57 | |
| *** links has joined #tripleo | 15:58 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Add support for installing Ceph MDS via ceph-ansible https://review.openstack.org/479426 | 15:59 |
| *** catintheroof has joined #tripleo | 15:59 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: CPU threads range format for derive parameters https://review.openstack.org/480587 | 16:00 |
| *** catintheroof has quit IRC | 16:00 | |
| *** catintheroof has joined #tripleo | 16:00 | |
| adarazs | EmilienM: I think it's good, +2 | 16:01 |
| *** egonzalez has quit IRC | 16:01 | |
| bogdando | stevebaker: hi. Could you please refresh my mind with the source of the images listed, once the environments/docker-centos-tripleoupstream.yaml is under rework. So there should I add a new image? | 16:02 |
| *** jaganathan has quit IRC | 16:03 | |
| *** agurenko has quit IRC | 16:04 | |
| EmilienM | adarazs: also, can we have builds for pike instead of master? | 16:04 |
| EmilienM | adarazs: who controls http://66.187.229.139/ again? | 16:04 |
| EmilienM | adarazs: i'm happy if you make a symblink for now | 16:04 |
| bogdando | stevebaker: is it tripleo-common's container-images/overcloud_containers.yaml ? | 16:04 |
| d0ugal | therve: good detective work - any idea what the fix is? I know very little about Heats code :) | 16:04 |
| adarazs | EmilienM: I don't have access. maybe panda?^ | 16:04 |
| EmilienM | amoralej: ^ | 16:05 |
| therve | d0ugal, Chatting with Zane about it. We can try a revert, because we don't get why it's failing this way | 16:05 |
| michapma | is anyone looking at the tripleo-ui failures in CI? The jobs still seem to be using 8.8.8.8 which can be rate limited, so it will cause failures in that package as it's one of the few that is not part of the undercloud image. | 16:06 |
| bogdando | and which is the place to define parameters defaults for 'Docker.*ConfigImage' ? | 16:06 |
| *** cdearborn_ has joined #tripleo | 16:06 | |
| *** limao has joined #tripleo | 16:06 | |
| amoralej | EmilienM, your plan is to move existing jobs to pike or create new ones? | 16:07 |
| amoralej | you are setting http://66.187.229.139/builds-master/current-tripleo-master/overcloud-full.tar | 16:07 |
| therve | d0ugal, It's really unfortunate that we have to jump through so many hoops to find a small clue :/ | 16:08 |
| therve | Anyway | 16:08 |
| EmilienM | amoralej: my plan is to make stable/pike branch working (tripleoclient was branched) | 16:08 |
| *** ffiore has quit IRC | 16:08 | |
| d0ugal | therve: indeed, I have been trying to figure out how we can make it easier to track down these issues, but I have no idea how we would automate that. | 16:08 |
| EmilienM | amoralej: can you create http://66.187.229.139/builds-pike ? | 16:08 |
| *** rcernin has quit IRC | 16:08 | |
| d0ugal | therve: It would help if the logs were easier to query | 16:08 |
| *** limao has quit IRC | 16:08 | |
| amoralej | EmilienM, nop, i don't have access to that server | 16:08 |
| amoralej | where's that? | 16:08 |
| EmilienM | amoralej: who has access? | 16:08 |
| *** limao has joined #tripleo | 16:09 | |
| therve | d0ugal, Yeah I thought I fixed zaqar logs, but apparently not | 16:09 |
| therve | And swift logs are still broken as well | 16:09 |
| therve | Sigh | 16:09 |
| amoralej | that server is in upstream infra? | 16:09 |
| EmilienM | amoralej: no | 16:09 |
| EmilienM | amoralej: it's hosted by redhat | 16:09 |
| d0ugal | therve: https://forums.mylzh.net/filedata/fetch?id=179289&d=1466117528&type=full | 16:09 |
| amoralej | maybe weshay_pto ? | 16:10 |
| therve | :) | 16:10 |
| EmilienM | amoralej: I'm sure wes knows but he's on pto | 16:10 |
| d0ugal | obligatory ^ | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 16:10 |
| EmilienM | amoralej: let's use master in meantime | 16:10 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common master: Add crond container image https://review.openstack.org/490068 | 16:10 |
| *** marios has quit IRC | 16:10 | |
| EmilienM | trown also might have access but on pto | 16:10 |
| EmilienM | myoung: ^ any thoughts on who has access to http://66.187.229.139/ ? | 16:10 |
| amoralej | EmilienM, currently pike builds are in centos7-pike-bootstrap, but i can change it right now | 16:11 |
| amoralej | other problem is that we are not consistent because of an issue with cloudkittyclient, i need to branch it now | 16:11 |
| myoung | EmilienM: dmsimard? | 16:11 |
| panda | EmilienM: I have access | 16:11 |
| EmilienM | woouhooo | 16:11 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service https://review.openstack.org/490050 | 16:12 |
| EmilienM | panda: how can we get builds for pike? | 16:12 |
| EmilienM | panda: is there a patch to do somewhere? | 16:12 |
| hewbrocca | template validation yay | 16:12 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/puppet-tripleo master: Add logrotate-crond configuration https://review.openstack.org/490048 | 16:12 |
| therve | d0ugal, Do we log the template in CI? | 16:12 |
| d0ugal | therve: no, I don't think so. | 16:13 |
| panda | EmilienM: we have to add the jobs | 16:13 |
| therve | That'd be useful as well | 16:13 |
| EmilienM | panda: in RDO, right? | 16:13 |
| EmilienM | panda: I'm working on the openstack-infra/tripleo-ci and projcet-config bits for pike now | 16:13 |
| *** pcaruana has quit IRC | 16:14 | |
| panda | no, in upstream | 16:14 |
| EmilienM | panda: ok I'm preparing it now and I'll ping you for review | 16:14 |
| panda | EmilienM: that's the right place | 16:14 |
| panda | EmilienM: ok | 16:14 |
| panda | I have to run, I'll be back in ~2 hours | 16:15 |
| *** lmiccini has quit IRC | 16:15 | |
| mwhahaha | is CI clear to recheck or do we have oustanding issues? | 16:15 |
| * mwhahaha thinks we need a good way to indicate this status | 16:16 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service https://review.openstack.org/490050 | 16:16 |
| *** cylopez has quit IRC | 16:16 | |
| EmilienM | mwhahaha: clear to land runway 27 and recheck on taxi alpha, delta | 16:16 |
| panda | EmilienM: pike link created | 16:17 |
| * mwhahaha notes the missing landing gear but rechecks anyway | 16:17 | |
| bogdando | stevebaker: folks, PTAL https://review.openstack.org/#/c/490050 | 16:17 |
| EmilienM | panda: really cool, thx | 16:17 |
| EmilienM | mwhahaha: just pull-up :) | 16:17 |
| *** panda is now known as panda|bbl | 16:17 | |
| therve | d0ugal, Should we propose a temp revert to unblock things? | 16:21 |
| d0ugal | therve: yup, I think that makes sense - it would also verify that patch as the problem. | 16:21 |
| *** cdearborn has quit IRC | 16:22 | |
| *** jistr|mtg is now known as jistr | 16:22 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Improve ansible.cfg ssh args https://review.openstack.org/477663 | 16:24 |
| therve | d0ugal, Can you handle it? I got to go | 16:25 |
| d0ugal | therve: sure | 16:26 |
| d0ugal | I can try | 16:26 |
| therve | Thanks | 16:26 |
| adarazs | therve, d0ugal: it would be cool if we could get in the revert before tomorrow morning so that we could promote upstream. we need that to unblock the CI. | 16:26 |
| jistr | bogdando: thanks, that does look related indeed. From a quick read i'm not sure if it will help us solve the issue we have, but at the very least i'd like to make sure that what we do doesn't collide with the new role. I'll take a closer look at the patch tomorrow. Thanks! | 16:26 |
| adarazs | therve: also please update the bug amoralej opened about your hunches and findings, might be useful later. :) | 16:27 |
| therve | adarazs, I did I think :) | 16:27 |
| adarazs | therve: oh I see, I didn't refresh. thanks! :) | 16:27 |
| *** lucasagomes is now known as lucas-afk | 16:28 | |
| *** itlinux has joined #tripleo | 16:28 | |
| bogdando | stevebaker: oh, got it, it's container-images/overcloud_containers.yaml.j2 | 16:28 |
| jistr | bogdando: my comment was re https://bugs.launchpad.net/tripleo/+bug/1708180/comments/1 -- sorry for late reply, was on meeting until now | 16:29 |
| openstack | Launchpad bug 1708180 in tripleo "Remote ssh administration user creation not working with split stack" [High,In progress] - Assigned to Jiřà Stránský (jistr) | 16:29 |
| d0ugal | adarazs: How do we test CI with https://review.openstack.org/#/c/490075/ ? | 16:29 |
| bogdando | jistr: yeah, got it. Thanks! | 16:29 |
| d0ugal | adarazs: to verify the revert helps | 16:29 |
| *** hewbrocca is now known as hewbrocca_afk | 16:31 | |
| adarazs | d0ugal: you can comment "check experimental" to run some tripleo experimental jobs on it. | 16:31 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 16:32 |
| d0ugal | adarazs: done, thanks. | 16:32 |
| d0ugal | adarazs: unrelated, I added that comment here: https://review.openstack.org/#/c/481947/ | 16:33 |
| d0ugal | adarazs: expecting tripeo jobs, but didn't get any. | 16:33 |
| adarazs | d0ugal: yep. I see the tripleo gates got queued up in zuul according to http://status.openstack.org/zuul/ | 16:34 |
| adarazs | in the "experimental-tripleo" queue. | 16:34 |
| dsneddon | karthiks, Workflow +1 on change 480540 | 16:35 |
| d0ugal | adarazs: thanks - is the mistral repo not setup for it? | 16:35 |
| adarazs | let me check. | 16:35 |
| karthiks | dsneddon, Thanks ! | 16:35 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common master: Add logrotate-crond container images params https://review.openstack.org/490068 | 16:39 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add logrotate with crond service https://review.openstack.org/490050 | 16:40 |
| adarazs | d0ugal: it's not set up there, I will add it. | 16:41 |
| *** dprince has joined #tripleo | 16:42 | |
| *** ccamacho has quit IRC | 16:42 | |
| *** ykarel has joined #tripleo | 16:42 | |
| bnemec | EmilienM: http://66.187.229.139/ is the rh1 mirror server. It's managed by http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp | 16:43 |
| bnemec | You'll need to propose a patch to that to add a builds-pike directory. | 16:44 |
| EmilienM | bnemec: yeah I'm working on it | 16:44 |
| EmilienM | thanks for confirming | 16:44 |
| *** pdeore has quit IRC | 16:44 | |
| bnemec | (ci-admins have direct access to it, but we shouldn't be making one-off changes) | 16:44 |
| d0ugal | adarazs: thanks! | 16:46 |
| EmilienM | bnemec: panda just created builds-pike directory | 16:46 |
| d0ugal | apetrich: FYI ^ | 16:46 |
| bnemec | EmilienM: panda|bbl: Please don't do that. | 16:46 |
| openstackgerrit | Brent Eagles proposed openstack/puppet-tripleo master: Add Octavia API endpoint to haproxy https://review.openstack.org/490082 | 16:46 |
| bnemec | If we have to rebuild the mirror server for some reason (and it has happened) manual changes won't be persisted. | 16:46 |
| * bnemec notes that this is why developers usually don't get access to production systems :-) | 16:47 | |
| EmilienM | bnemec: I agree | 16:47 |
| EmilienM | bnemec: they should be maanged by puppet I guess | 16:47 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-docs master: Post deployment installation instructions for Octavia https://review.openstack.org/447496 | 16:50 |
| *** ykarel has quit IRC | 16:50 | |
| *** limao has quit IRC | 16:51 | |
| adarazs | d0ugal, EmilienM: added 'check experimental' jobs to mistral in case we'll need it: https://review.openstack.org/490086 | 16:52 |
| sai | lol | 16:53 |
| dsneddon | bnemec, EmilienM: Is there any way to generate the undercloud-role.yaml similar to how I can generate the other configs by running ./tools/process_templates.py or ./tools/roles-data-generate-samples.sh | 16:54 |
| EmilienM | bnemec: files are managed by puppet already | 16:55 |
| bnemec | dsneddon: mwhahaha is probably the person to talk to about roles. | 16:55 |
| EmilienM | bnemec: so it's good | 16:55 |
| dsneddon | mwhahaha, Any ideas? ^^^ | 16:55 |
| bnemec | EmilienM: What? There's no builds-pike directory in the puppet manifest. | 16:56 |
| bnemec | Unless something _just_ merged. | 16:56 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 16:56 |
| EmilienM | bnemec: https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/mirror-server/mirror-server.pp#L37-L40 | 16:56 |
| bnemec | EmilienM: That's for ocata... | 16:57 |
| mwhahaha | dsneddon: you mean roles_data_undercloud.yaml? | 16:57 |
| EmilienM | bnemec: right and I'm doing pike ^p | 16:57 |
| EmilienM | amoralej, bnemec : please review https://review.openstack.org/490077 | 16:57 |
| mwhahaha | dsneddon: roles-data-generate-samples.sh will do that based on what is in roles/Undercloud.yaml | 16:57 |
| EmilienM | I'm working on the project-config patch now | 16:57 |
| dsneddon | mwhahaha, No, I mean undercloud-role.yaml. I'm getting a CI error on that file here: http://logs.openstack.org/72/480272/19/check/gate-tripleo-ci-centos-7-undercloud-containers/3264f2d/console.html.gz | 16:58 |
| mwhahaha | i have no idea what undercloud-role.yaml is | 16:58 |
| bnemec | EmilienM: Okay, so you meant in general they're managed by puppet, not that the new pike one was already. Think I got it now. :-) | 16:58 |
| mwhahaha | (and am concerned that that's a thing) | 16:58 |
| dsneddon | mwhahaha, Yeah, me too! | 16:58 |
| EmilienM | mwhahaha, dsneddon: AFIK it's to deploy the heat installer | 16:58 |
| EmilienM | undercloud services using THT | 16:59 |
| EmilienM | dprince wrote that, afik ^ | 16:59 |
| mwhahaha | but where is it coming from | 16:59 |
| *** derekh has quit IRC | 16:59 | |
| mwhahaha | oooq maybe (and theni would say see topic) | 16:59 |
| dsneddon | EmilienM, Yeah, I'm making changes to role.role.j2.yaml, and just when I think I've got it working with all the overcloud roles, it fails in CI because my changes break undercloud-role.yaml. | 16:59 |
| mwhahaha | oh wait doesn't role.role.j2.yaml create that? | 17:00 |
| mwhahaha | or isn't that how it's created | 17:00 |
| EmilienM | lol see topic | 17:00 |
| mwhahaha | ie that's a parse error from the generated role.role.j2.yaml | 17:00 |
| *** dtantsur is now known as dtantsur|afk | 17:00 | |
| *** ramishra has quit IRC | 17:00 | |
| EmilienM | (I should probably add the landing gear stuff from mwhahaha) | 17:01 |
| mwhahaha | it could also be from tripleo-client for the undercloud install command which was experimental but is now being use din voting ci | 17:01 |
| * mwhahaha pulls out the look of disapproval ಠ_ಠ| 17:01 | |
| *** thrash|biab is now known as thrash | 17:02 | |
| EmilienM | adarazs, bnemec: https://review.openstack.org/490093 | 17:02 |
| adarazs | EmilienM: +1 | 17:04 |
| EmilienM | ianw: we need a release of dib in fact (since CI use pip to deploy it for this job) | 17:07 |
| *** tesseract has quit IRC | 17:07 | |
| *** cylopez has joined #tripleo | 17:08 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Better long validation task management https://review.openstack.org/490098 | 17:08 |
| EmilienM | yolanda: you're release manager on DIB, can we get a release now please? | 17:08 |
| *** cylopez has quit IRC | 17:09 | |
| EmilienM | we need https://review.openstack.org/#/c/489434/ to fix tripleoclient gate | 17:09 |
| mwhahaha | dsneddon: undercloud-role.yaml comes from config.role.j2.yaml | 17:09 |
| mwhahaha | dsneddon: http://logs.openstack.org/72/480272/19/check/gate-tripleo-ci-centos-7-undercloud-containers/3264f2d/console.html.gz#_2017-07-28_21_32_55_284504 | 17:09 |
| bnemec | EmilienM: I -1'd the tripleo-ci pike patch. There are a couple of problems with it. | 17:10 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 17:10 |
| *** ckyriakidou has quit IRC | 17:10 | |
| mwhahaha | dsneddon: no that's not right, i think it's roles.role.j2.yaml that does it. this is our glorious jinja2 rendering stuff | 17:11 |
| mwhahaha | dsneddon: so it is coming from role.role.j2.yaml http://logs.openstack.org/72/480272/19/check/gate-tripleo-ci-centos-7-undercloud-containers/3264f2d/console.html.gz#_2017-07-28_21_32_55_284450 | 17:11 |
| dsneddon | mwhahaha, Yeah, that makes more sense, since it's changes to role.role.j2.yaml that's causing the error. | 17:11 |
| mwhahaha | seems to just be a yaml parsing problem | 17:11 |
| dsneddon | mwhahaha, But that doesn't get rendered when you run tools/process_templates.py | 17:12 |
| dsneddon | mwhahaha, So I'm not sure how to do a test render to see what's going wrong. | 17:12 |
| mwhahaha | dsneddon: probably because process_templates.py uses roles_data.yaml and not roles_data_undercloud.yaml | 17:12 |
| mwhahaha | dsneddon: use process_templates.py --roles-data roles_data_undercloud.yaml | 17:13 |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Pass UserKnownHostsFile and deploy ssh key https://review.openstack.org/487650 | 17:13 |
| dsneddon | mwhahaha, Ah, but there is a --roles-data parameter | 17:13 |
| *** bogdando has quit IRC | 17:13 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 17:16 |
| EmilienM | bnemec, amoralej : thank you for your comments, all done ^ | 17:16 |
| openstackgerrit | John Fulton proposed openstack/tripleo-common master: Allow tripleo-admin creation both for Nova-managed and split-stack https://review.openstack.org/489613 | 17:16 |
| dsneddon | mwhahaha, It works! Many thanks. | 17:17 |
| mwhahaha | np | 17:18 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Better long validation task management https://review.openstack.org/490098 | 17:20 |
| *** ebarrera has joined #tripleo | 17:20 | |
| amoralej | EmilienM, it's not needed to update https://github.com/openstack-infra/tripleo-ci/blob/master/toci_gate_test-oooq.sh#L244 ? | 17:22 |
| dprince | mwhahaha: we have gotten some good feedback from the undercloud-heat installer. what issues are you having? | 17:22 |
| *** prateek has joined #tripleo | 17:22 | |
| mwhahaha | dprince: this isn't necessarily an issue with it but more of a commentary on we're using an 'experimeental' feature for our voting ci :D | 17:24 |
| EmilienM | amoralej: ah probably | 17:25 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 17:25 |
| EmilienM | amoralej: thanks! nice catch | 17:25 |
| amoralej | grep catched it :) | 17:26 |
| dprince | mwhahaha: sure, but do keep in mind I added the 'experimental' statement there per comments from you I think :) | 17:27 |
| mwhahaha | dprince: yea because i didn't like what it was doing so it was the only way i wanted to allow it | 17:27 |
| dprince | mwhahaha: and I think it would be worth considering dropping that in Queen | 17:27 |
| *** jlabarre has quit IRC | 17:27 | |
| mwhahaha | possibly | 17:27 |
| dprince | mwhahaha: "didn't like what is was doing"? | 17:27 |
| *** prateek has quit IRC | 17:28 | |
| * mwhahaha goes and digs up the review | 17:28 | |
| mwhahaha | specifically the stuff required to make it work which we put in oooq | 17:31 |
| mwhahaha | https://review.openstack.org/#/c/419040/ | 17:31 |
| mwhahaha | so if we want to unmark it as experimental we need to figure out a way to take the stuff out of oooq | 17:32 |
| mwhahaha | in a semi userfriendly fashion | 17:32 |
| dprince | mwhahaha: fwiw, I never intended the primary use case for this to have anything to do w/ oooq. It was added there as a convenience to oooq users | 17:32 |
| dprince | mwhahaha: I don't use oooq with it | 17:33 |
| dprince | mwhahaha: nor is it a requirement | 17:33 |
| mwhahaha | dprince: right but we need to take that stuff that lives in oooq and make it userfriendly | 17:33 |
| mwhahaha | so yes i know you don't need to use oooq but you do need to do something so it works | 17:33 |
| mwhahaha | that's the something i have issues with | 17:33 |
| *** tosky has quit IRC | 17:36 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: Better long validation task management https://review.openstack.org/490098 | 17:43 |
| adarazs | EmilienM: can I get a review on this? will need it to get a Phase1 promotion: https://review.openstack.org/490098 | 17:50 |
| EmilienM | adarazs: sure | 17:56 |
| *** yamahata has joined #tripleo | 17:57 | |
| adarazs | is there any other tripleo core around for this simple change so that I don't have to merge it with 1 +2? :) | 18:02 |
| amoralej | EmilienM, tripleo-dlrn-promote-pike job is created in ci.centos.org so that we can start doing promotions in pike but we'd like to move to dlrnapi based promotions as soon as we can | 18:03 |
| EmilienM | amoralej++ | 18:03 |
| openstackgerrit | Emilien Macchi proposed openstack/os-net-config master: OvS 2.7 support - dpdk-devargs needs to provided for DPDK devices https://review.openstack.org/480540 | 18:08 |
| *** amoralej is now known as amoralej|off | 18:08 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo master: Use normal socket file permissions instead of polkit https://review.openstack.org/479816 | 18:08 |
| EmilienM | jistr: I love your ASCII charts in commit message | 18:09 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 18:10 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| *** jpena is now known as jpena|off | 18:10 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Remove vars from multinode common config which can be overriden https://review.openstack.org/486154 | 18:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Use stdout_lines instead of stdout for subnode inventory https://review.openstack.org/483396 | 18:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Add node config variables to support deployed server https://review.openstack.org/485689 | 18:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Move common multinode role setup from tripleo-ci https://review.openstack.org/487408 | 18:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Add featureset for 3nodes multinode job https://review.openstack.org/486155 | 18:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Consolidate puppet/docker deployments with one deploy steps workflow https://review.openstack.org/483293 | 18:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Move deploy-steps-playbook to deploy-steps-tasks https://review.openstack.org/485686 | 18:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Default docker_puppet_debug to false https://review.openstack.org/486103 | 18:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add RoleConfig output https://review.openstack.org/485731 | 18:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add support for update_tasks https://review.openstack.org/486104 | 18:11 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Add environment to disable deploy steps https://review.openstack.org/486180 | 18:11 |
| *** salmankhan has quit IRC | 18:12 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Replace references to deprecated controllerExtraConfig https://review.openstack.org/480395 | 18:12 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Enable tempest on scenario001-multinode job https://review.openstack.org/484788 | 18:15 |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Prompt to clear breakpoints when using deployed-server https://review.openstack.org/490125 | 18:16 |
| *** social has quit IRC | 18:18 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Move release notes to the right directory https://review.openstack.org/490127 | 18:19 |
| EmilienM | arxcruz: what is the progress to run tempest in other jobs? | 18:20 |
| arxcruz | EmilienM: requires this https://review.openstack.org/#/c/485252/ | 18:21 |
| arxcruz | then is just alter the featureset00x for each job with the tests we want to run | 18:21 |
| EmilienM | arxcruz: afik we can't have whilelist working WITH blacklist right? | 18:22 |
| arxcruz | EmilienM: we can, but one will anulate the other | 18:23 |
| EmilienM | yeah, I just want to make sure we don't do that | 18:23 |
| arxcruz | EmilienM: if a job is in the blacklist and in the whitelist, it will not be executed, but still, we can set the blacklist file to none | 18:23 |
| EmilienM | chandankumar: can you review https://review.openstack.org/#/c/485252 please? | 18:24 |
| EmilienM | adarazs: ^ also plz | 18:24 |
| arxcruz | EmilienM: on my last comment I show the run with and without the whitelist | 18:24 |
| adarazs | arxcruz: thanks for the fixes on that change. looks good now. | 18:25 |
| EmilienM | cool, let's merge it | 18:25 |
| arxcruz | yay | 18:25 |
| adarazs | EmilienM: submitted. | 18:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Enable pingtest on scenarios container jobs https://review.openstack.org/490129 | 18:26 |
| EmilienM | adarazs: I almost had an heart attack when I did that ^ | 18:26 |
| adarazs | huh, how did we forget about that :) | 18:27 |
| EmilienM | jistr: ^ how come we don't test services when deploying with containers??? | 18:27 |
| adarazs | EmilienM: +2, let's see if they pass. | 18:27 |
| EmilienM | I just hope it will pass | 18:28 |
| EmilienM | adarazs: it's not tested | 18:28 |
| EmilienM | adarazs: we need to run container jobs | 18:28 |
| adarazs | I understand. if it passes we can submit it. | 18:28 |
| EmilienM | I'll use THT + Depends-On for now | 18:28 |
| adarazs | ah right, it won't trigger those. | 18:28 |
| adarazs | good idea. | 18:28 |
| adarazs | EmilienM: you can -W it to be sure, but I guess nobody will accidentally merge it. | 18:29 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Upgrade testing https://review.openstack.org/489710 | 18:29 |
| EmilienM | adarazs: done | 18:30 |
| EmilienM | gosh | 18:30 |
| EmilienM | so we never ran pingtest against container jobs | 18:30 |
| *** links has quit IRC | 18:31 | |
| EmilienM | jistr: sounds like you created the featureset but never enabled pingtest | 18:31 |
| adarazs | d0ugal, therve: https://review.openstack.org/490075 -- looks like the nonha gate managed to deploy the overcloud, that's a good sign. | 18:31 |
| arxcruz | EmilienM: next step now is collect which tests we want to see on each scenario, I already have the tests for scenario001 from pradk today | 18:35 |
| EmilienM | arxcruz: autoscaling for scenario001 | 18:36 |
| EmilienM | arxcruz: I'll take care of other scenarios, I have an idea of what we need to test | 18:36 |
| arxcruz | EmilienM: cool! | 18:37 |
| arxcruz | EmilienM: pradk pointed me for this https://github.com/openstack/puppet-openstack-integration/blob/master/run_tests.sh#L272-L282 for scenario001 | 18:38 |
| EmilienM | arxcruz: yes | 18:38 |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Prompt to clear breakpoints when using deployed-server https://review.openstack.org/490125 | 18:39 |
| EmilienM | arxcruz: test_telemetry_integration is the most important one IMHO | 18:39 |
| EmilienM | arxcruz: don't include Vitrage please | 18:39 |
| EmilienM | arxcruz: I would say, start with test_telemetry_integration only for now and see how it works | 18:39 |
| arxcruz | EmilienM: https://cdn.meme.am/instances/76036706.jpg | 18:40 |
| *** nyechiel has quit IRC | 18:41 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP OpenStack containerized qpid-dispatch-router service https://review.openstack.org/479049 | 18:55 |
| *** atoth has quit IRC | 18:57 | |
| EmilienM | adarazs: see comment on https://review.openstack.org/#/c/490086 | 18:58 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Convert compute-role.yaml to role.role.j2.yaml https://review.openstack.org/482623 | 18:59 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Render IP map and host maps according to network_data.yaml https://review.openstack.org/486280 | 18:59 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Convert controller-role.yaml to role.role.j2.yaml https://review.openstack.org/480272 | 18:59 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Convert objectstorage-role.yaml to role.role.j2.yaml https://review.openstack.org/482624 | 18:59 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Convert blockstorage-role.yaml to role.role.j2.yaml https://review.openstack.org/482625 | 18:59 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-heat-templates master: Convert cephstorage-role.yaml to role.role.j2.yaml https://review.openstack.org/482626 | 18:59 |
| *** jcoufal has quit IRC | 19:00 | |
| *** coolsvap has quit IRC | 19:01 | |
| jtomasek | honza: I think we need more opinions here so we can make the decision https://review.openstack.org/#/c/483039/7 :) | 19:01 |
| honza | jtomasek: i'm good at opinions :) | 19:05 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 19:10 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Prompt to clear breakpoints when using deployed-server https://review.openstack.org/490125 | 19:11 |
| EmilienM | adarazs: https://review.openstack.org/490163 | 19:13 |
| *** jlabarre has joined #tripleo | 19:13 | |
| *** jtomasek has quit IRC | 19:19 | |
| *** panda|bbl is now known as panda | 19:19 | |
| slagle | why do our docs say to use https://trunk.rdoproject.org/centos7-master/current-passed-ci/delorean.repo and not https://trunk.rdoproject.org/centos7/current-tripleo/delorean.repo? | 19:27 |
| slagle | current-passed-ci vs current-tripleo | 19:27 |
| slagle | current-tripleo is what we promoted 2 days ago | 19:27 |
| slagle | current-passed-ci...i have no idea what that is. but it's 11 days old | 19:27 |
| slagle | does anyone know what repos quickstart uses? | 19:29 |
| EmilienM | slagle: current-tripleo in https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml#L25 | 19:30 |
| EmilienM | and consistent for promotion jobs: https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/consistent-master.yml#L25 | 19:31 |
| slagle | EmilienM: ok. tripleo.sh does the same | 19:31 |
| slagle | so the docs are just wrong then | 19:31 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - run tempest in scenario001-container https://review.openstack.org/490167 | 19:31 |
| EmilienM | arxcruz: ^ just experimenting | 19:31 |
| arxcruz | EmilienM: cool, I'll take a look if there's some error | 19:32 |
| openstackgerrit | James Slagle proposed openstack/tripleo-docs master: Document using current-tripleo https://review.openstack.org/490170 | 19:33 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: CI testing: run tempest in container jobs https://review.openstack.org/490172 | 19:33 |
| *** jfrancoa has quit IRC | 19:34 | |
| *** ebarrera_ has joined #tripleo | 19:38 | |
| *** ebarrera has quit IRC | 19:38 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 19:41 |
| vkmc | hey all, I've mentioned this before but couldn't get much information... I'm getting several errors with Ironic on the undercloud | 19:48 |
| *** pchavva has quit IRC | 19:49 | |
| vkmc | precisely this http://paste.openstack.org/show/617315/ | 19:49 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Start redis service after upgrade https://review.openstack.org/490176 | 19:53 |
| *** paramite has joined #tripleo | 19:55 | |
| vkmc | I can always reproduce this, in a minimal deploy | 19:55 |
| vkmc | EmilienM, ^ | 19:55 |
| vkmc | dprince, ^ | 19:55 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Add Telemetry services to scenario002 https://review.openstack.org/474969 | 19:56 |
| EmilienM | vkmc: I'm currently eating lunch, gimme some time :D | 19:57 |
| bnemec | vkmc: That looks like an issue with the liveness checks from haproxy. | 19:59 |
| bnemec | At least that's what used to cause the broken pipe in the api logs. | 19:59 |
| bnemec | I thought it got fixed in Ironic, but maybe not Inspector. | 20:00 |
| vkmc | EmilienM, sharing is caring | 20:00 |
| vkmc | bnemec, not sure if I can provide more logs to confirm this | 20:01 |
| bnemec | vkmc: Is anything actually malfunctioning? In the past I thought this was a cosmetic issue that resulted in spurious log entries, but nothing else. | 20:01 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Add support for installing Ceph MDS via ceph-ansible https://review.openstack.org/479426 | 20:02 |
| vkmc | bnemec, not sure if it's related, but I'm seeing lots of dhcp interfaces being started and failing in systemctl | 20:02 |
| vkmc | and I'm afraid this is causing network issues | 20:02 |
| vkmc | in my late deploy, after a reboot, I wouldn't be able to access the endpoints | 20:03 |
| vkmc | s/late/last/g | 20:03 |
| *** paramite has quit IRC | 20:04 | |
| *** jlabarre has quit IRC | 20:07 | |
| *** sbrzozow has quit IRC | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 20:10 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| *** liverpooler has quit IRC | 20:16 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-docs master: Update Pike storage documentation for ceph-ansible https://review.openstack.org/487155 | 20:20 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 20:20 |
| *** ansmith has quit IRC | 20:25 | |
| *** dparkes has joined #tripleo | 20:32 | |
| *** https_GK1wmSU has joined #tripleo | 20:33 | |
| *** https_GK1wmSU has left #tripleo | 20:34 | |
| *** salmankhan has joined #tripleo | 20:35 | |
| *** salmankhan has quit IRC | 20:39 | |
| stevebaker | morning | 20:39 |
| mwhahaha | afternoon :D | 20:40 |
| panda | 'night | 20:43 |
| mwhahaha | bnemec: do you know what generates cistatus.html? | 20:48 |
| bnemec | mwhahaha: It's a cron job that runs the script in http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/website | 20:49 |
| bnemec | Which I guess actually calls http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo-jobs-gerrit.py to create cistatus.html | 20:50 |
| * mwhahaha wishes we had decent numbers for the gates as cistatus.tripleo.org seems also not to be working right | 20:51 | |
| bnemec | mwhahaha: I've found that often I have to shift+refresh that page a number of times to get the stats to show. | 20:51 |
| bnemec | Not sure why. | 20:51 |
| * mwhahaha shakes fist at his screen | 20:51 | |
| mwhahaha | gate-tripleo-ci-centos-7-ovb-ha-oooq appears to be having a 42% failure rate for infra issues today | 20:51 |
| bnemec | For the past three days we've had a <60% pass rate on all jobs, so that sounds about right. :-/ | 20:53 |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo master: Configure dockerd with --iptables=false https://review.openstack.org/490201 | 20:53 |
| EmilienM | mwhahaha: we need to look at logstash | 20:53 |
| EmilienM | why it fails | 20:53 |
| mwhahaha | probably same reason vanilla cloud was failing | 20:53 |
| mwhahaha | it stopped around 12:00 (whatever TZ gets reported) | 20:53 |
| *** jrist has quit IRC | 20:54 | |
| bnemec | I see a massive number of yum failures in the ovb jobs for some reason. | 20:54 |
| EmilienM | mwhahaha: gate-tripleo-ci-centos-7-ovb-ha-oooq isn't run in vallina, right? | 20:54 |
| bnemec | No, that's all rh1. | 20:54 |
| mwhahaha | EmilienM: no but maybe there was a larger connectivity problem | 20:55 |
| bnemec | Unless someone turned on rdo cloud, which would also explain the massive failure rate. :-) | 20:55 |
| mwhahaha | ie major provider problems | 20:55 |
| *** Goneri has quit IRC | 20:57 | |
| mwhahaha | tripleo-jobs-gerrit.py -> gate-tripleo-ci-centos-7-ovb-ha-oooq | 20:57 |
| mwhahaha | er | 20:57 |
| mwhahaha | ZeroDivisionError: float division by zero | 20:57 |
| mwhahaha | dat math | 20:57 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 21:01 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix ceilometer agent compute service name https://review.openstack.org/486737 | 21:04 |
| *** nyechiel has joined #tripleo | 21:06 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 21:10 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| bnemec | mwhahaha: When I had issues running the script locally, I think it usually had to do with my gerrit username not matching the username I was logged in locally with. | 21:11 |
| bnemec | (I tried it against just the ovb-ha job - about a 35% failure rate since ~July 20th) | 21:12 |
| mwhahaha | yea it needs a username | 21:12 |
| bnemec | I just added this to my ~/.ssh/config: | 21:12 |
| bnemec | Host review.openstack.org | 21:12 |
| bnemec | User=bnemec | 21:12 |
| mwhahaha | or that :D | 21:13 |
| bnemec | Damn, we are merging inconsistent parameter definitions faster than I can fix them. :-P | 21:19 |
| *** lblanchard has quit IRC | 21:21 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Make UpgradeLevelNovaCompute parameters consistent https://review.openstack.org/484041 | 21:21 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Make many networking parameters consistent https://review.openstack.org/483540 | 21:21 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Make RoleParameters and key_name descriptions consistent https://review.openstack.org/483539 | 21:21 |
| *** fandrieu has quit IRC | 21:23 | |
| *** dprince has quit IRC | 21:23 | |
| *** jrist has joined #tripleo | 21:26 | |
| *** cdearborn_ has quit IRC | 21:36 | |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Add --set argument to prepare command https://review.openstack.org/489083 | 21:40 |
| fultonj | does anyone with "the power" want to +W https://review.openstack.org/#/c/489825 ? | 21:41 |
| slagle | fultonj: thanks for fixing this | 21:42 |
| fultonj | slagle: happy to, thanks for accepting this fix | 21:42 |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-heat-templates master: WIP: Stop and disable openstack-nova-compute service on compute nodes https://review.openstack.org/490226 | 21:45 |
| * bnemec gets his recheck button warmed up | 21:45 | |
| bnemec | +W is only the start of merging a patch these days. | 21:45 |
| *** jlabarre has joined #tripleo | 21:49 | |
| fultonj | thanks bnemec | 21:49 |
| fultonj | slagle: fyi https://bugs.launchpad.net/tripleo/+bug/1708302 | 21:54 |
| openstack | Launchpad bug 1708302 in tripleo "openstack clients which can use ceph, should not need to depend on /etc/ceph" [Undecided,New] - Assigned to John Fulton (jfulton-org) | 21:54 |
| *** agopi|away has quit IRC | 21:56 | |
| *** jmelvin has quit IRC | 22:00 | |
| *** jkilpatr has quit IRC | 22:01 | |
| * fultonj back tomorrow | 22:01 | |
| *** panda has quit IRC | 22:02 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Prompt to clear breakpoints when using deployed-server https://review.openstack.org/490125 | 22:05 |
| *** morazi has quit IRC | 22:06 | |
| *** rdopiera has joined #tripleo | 22:07 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 22:10 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| *** ansmith has joined #tripleo | 22:15 | |
| *** https_GK1wmSU has joined #tripleo | 22:17 | |
| *** https_GK1wmSU has left #tripleo | 22:17 | |
| *** jkilpatr has joined #tripleo | 22:38 | |
| ianw | EmilienM: ok, i'll do a dib release in a little | 22:38 |
| *** catintheroof has quit IRC | 22:44 | |
| *** ebarrera_ has quit IRC | 22:47 | |
| *** bfournie has quit IRC | 22:50 | |
| *** jkilpatr has quit IRC | 22:50 | |
| *** jkilpatr has joined #tripleo | 22:50 | |
| *** jrist has quit IRC | 22:55 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1708146 | 23:10 |
| openstack | Launchpad bug 1708146 in tripleo "OVB periodic jobs fail in overcloud deploy task with timeout waiting for messages" [Critical,Triaged] | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| *** itlinux has quit IRC | 23:20 | |
| *** _mmethot_ has joined #tripleo | 23:24 | |
| *** mmethot_ has quit IRC | 23:26 | |
| *** bugzy has quit IRC | 23:30 | |
| *** stee_3 has quit IRC | 23:30 | |
| EmilienM | ianw: thanks | 23:34 |
| *** stee_3 has joined #tripleo | 23:36 | |
| *** bugzy has joined #tripleo | 23:36 | |
| *** ecerquei has joined #tripleo | 23:41 | |
| *** bfournie has joined #tripleo | 23:46 | |
| *** bfournie has quit IRC | 23:47 | |
| *** dparkes has quit IRC | 23:48 | |
| *** bfournie has joined #tripleo | 23:48 | |
| *** ansmith has quit IRC | 23:54 | |
| *** ansmith has joined #tripleo | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
