| *** rascasoft has joined #oooq | 00:24 | |
| *** tosky has quit IRC | 00:34 | |
| *** rascasoft has quit IRC | 00:38 | |
| *** rfolco has quit IRC | 00:40 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (3 more messages) | 00:51 |
|---|---|---|
| *** rfolco has joined #oooq | 00:59 | |
| *** hamzy has joined #oooq | 01:06 | |
| *** rascasoft has joined #oooq | 01:45 | |
| *** rascasoft has quit IRC | 01:50 | |
| *** agopi|off has quit IRC | 01:57 | |
| *** agopi|off has joined #oooq | 02:16 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (3 more messages) | 02:51 |
| *** apetrich has quit IRC | 03:24 | |
| *** ykarel has joined #oooq | 03:36 | |
| *** ccamacho has quit IRC | 03:39 | |
| *** rlandy has quit IRC | 03:50 | |
| *** skramaja has joined #oooq | 03:53 | |
| *** gkadam has joined #oooq | 04:03 | |
| *** udesale has joined #oooq | 04:30 | |
| *** chandankumar has joined #oooq | 04:40 | |
| *** chandankumar has quit IRC | 04:41 | |
| *** chandankumar has joined #oooq | 04:42 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (2 more messages) | 04:51 |
| *** ykarel has quit IRC | 05:09 | |
| *** zbr|ssbarnea has joined #oooq | 05:13 | |
| *** chandankumar is now known as chkumar|ruck | 05:15 | |
| *** zbr has quit IRC | 05:16 | |
| *** ykarel has joined #oooq | 05:26 | |
| *** udesale has quit IRC | 05:45 | |
| *** udesale has joined #oooq | 05:55 | |
| *** jtomasek has joined #oooq | 06:07 | |
| *** ratailor has joined #oooq | 06:13 | |
| *** jtomasek has quit IRC | 06:27 | |
| chkumar|ruck | ykarel, Hello | 06:35 |
| chkumar|ruck | ykarel, centos 7 container build is failing https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/f56417e/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2019-02-06_05_46_14 | 06:39 |
| chkumar|ruck | ykarel, during undercloud deploy | 06:39 |
| ykarel | yup | 06:39 |
| chkumar|ruck | ykarel, I am not sure it is related to this one | 06:40 |
| chkumar|ruck | "PMD: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory", | 06:40 |
| chkumar|ruck | 2019-02-06 05:42:33 | "PMD: net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)", | 06:40 |
| chkumar|ruck | 2019-02-06 05:42:33 | "PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory", | 06:40 |
| chkumar|ruck | 2019-02-06 05:42:33 | "PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4)", | 06:40 |
| chkumar|ruck | 2019-02-06 05:42:33 | "ovs-vsctl: unix:/var/run/openvswitch/db.sock: database connection failed (No such file or directory)", | 06:40 |
| ykarel | chkumar|ruck, nope seems unrelated | 06:40 |
| ykarel | these warnings are from long | 06:41 |
| *** quique|rover|off is now known as quiquell|rover | 06:42 | |
| quiquell|rover | chkumar|ruck: o/ | 06:42 |
| quiquell|rover | ykarel: o/ | 06:42 |
| chkumar|ruck | it is very hard to find what wents wrong in a undercloud_deploy log file | 06:43 |
| ykarel | chkumar|ruck, step 5 failed | 06:45 |
| ykarel | cell v2 discovery task | 06:45 |
| ykarel | chkumar|ruck, quiquell|rover /usr/local/bin/kolla_start: line 18: /usr/lib/rabbitmq/bin/rabbitmq-server: No such file or directory is the issue | 06:48 |
| ykarel | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/f56417e/logs/undercloud/var/log/extra/podman/containers/rabbitmq/stdout.log.txt.gz | 06:48 |
| ykarel | chkumar|ruck, quiquell|rover you can check logs from last periodic run as well there also issues are there related to rabbitmq | 06:50 |
| ykarel | there container-build passed, but others failed | 06:50 |
| chkumar|ruck | ykarel, checking | 06:51 |
| *** saneax has joined #oooq | 06:51 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (2 more messages) | 06:51 |
| quiquell|rover | Yk | 06:54 |
| quiquell|rover | Maybe new version of rabbitmq install it elswhere | 06:56 |
| *** jtomasek has joined #oooq | 06:57 | |
| chkumar|ruck | quiquell|rover, ykarel from passed logs it is working fine there http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/936495c/logs/undercloud/var/log/extra/podman/containers/rabbitmq/stdout.log.txt.gz | 07:02 |
| ykarel | yup | 07:03 |
| chkumar|ruck | quiquell|rover, will i cover this issue in the same bug for rabbitmq-server? | 07:03 |
| ykarel | i am doubting the tripleo-common patch | 07:03 |
| quiquell|rover | chkumar|ruck: yep | 07:03 |
| chkumar|ruck | same bug means https://bugs.launchpad.net/tripleo/+bug/1814763 | 07:03 |
| openstack | Launchpad bug 1814763 in tripleo "[Master][periodic] container build job got failed with No package rabbitmq-server-3.7.10 available" [Critical,Fix released] | 07:03 |
| quiquell|rover | Well no | 07:03 |
| quiquell|rover | Container build is working now | 07:03 |
| chkumar|ruck | quiquell|rover, Let me file a seperate bug | 07:04 |
| *** jfrancoa has joined #oooq | 07:04 | |
| quiquell|rover | ykarel: do you have the tripleo-common review? | 07:04 |
| chkumar|ruck | quiquell|rover, https://review.openstack.org/#/c/634405/ | 07:04 |
| ykarel | https://review.openstack.org/#/c/634405/ | 07:04 |
| chkumar|ruck | quiquell|rover, ykarel https://review.openstack.org/#/c/634405/4/container-images/tripleo_kolla_template_overrides.j2@159 | 07:05 |
| ykarel | yup got it | 07:06 |
| ykarel | it has issues | 07:06 |
| quiquell|rover | jfrancoa: o/ override working fine now | 07:06 |
| ykarel | rabbitmq_packages_append is overridden | 07:06 |
| quiquell|rover | ykarel: ^ | 07:06 |
| ykarel | quiquell|rover, ack | 07:06 |
| jfrancoa | quiquell|rover: great! did our patches fix it then? or did you prepare anything else? | 07:07 |
| quiquell|rover | ykarel: could be that rpm does not get installed at Kolla build? | 07:07 |
| jfrancoa | quiquell|rover: good morning btw | 07:07 |
| ykarel | yes that's the issue with tripleo-common patch | 07:07 |
| ykarel | quiquell|rover, ^^ | 07:07 |
| quiquell|rover | jfrancoa: after the depends on it's working, we have to also wait for rdo | 07:07 |
| quiquell|rover | ykarel: ack, will look at it in | 07:08 |
| quiquell|rover | jfrancoa: good morning ! | 07:09 |
| ykarel | chkumar|ruck, quiquell|rover also check my comment in https://review.openstack.org/#/c/633546/ | 07:11 |
| jfrancoa | quiquell|rover: great news to start the day then | 07:11 |
| ykarel | chkumar|ruck, quiquell|rover needed to get scenario007 fixed | 07:11 |
| ykarel | in review itself it failed, but people missed it as non-voting | 07:11 |
| quiquell|rover | ykarel: will look at it in a few I am at read only mode now | 07:12 |
| ykarel | np just wanted you guys to get noticed, take ur time | 07:12 |
| quiquell|rover | ykarel: thanks mate | 07:13 |
| chkumar|ruck | ykarel, fixing scenario007 | 07:13 |
| ykarel | chkumar|ruck, ack good to have bug for it | 07:14 |
| ykarel | it will soon affect promotions as well | 07:14 |
| chkumar|ruck | ykarel, ok | 07:14 |
| chkumar|ruck | quiquell|rover, ykarel https://bugs.launchpad.net/tripleo/+bug/1814851 | 07:14 |
| openstack | Launchpad bug 1814851 in tripleo "[master][promotion] centos 7 container build job deploying undercloud with usr/lib/rabbitmq/bin/rabbitmq-server: No such file or directory" [High,Triaged] | 07:14 |
| *** udesale has quit IRC | 07:15 | |
| ykarel | ack | 07:15 |
| *** jtomasek has quit IRC | 07:18 | |
| *** jtomasek has joined #oooq | 07:19 | |
| *** udesale has joined #oooq | 07:20 | |
| quiquell|rover | chkumar|ruck: so you fix scenario007 y fix rabbitmq ? | 07:21 |
| quiquell|rover | Ahh is the same | 07:23 |
| chkumar|ruck | quiquell|rover, both are different | 07:23 |
| chkumar|ruck | quiquell|rover, https://bugs.launchpad.net/tripleo/+bug/1814852 | 07:23 |
| openstack | Launchpad bug 1814852 in tripleo "[scenario7] check job failed with Could not fetch contents/tripleo-heat-templates/puppet/services/tacker.yaml during overcloud deploy" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 07:23 |
| quiquell|rover | chkumar|ruck: ack, will take rabbit one then | 07:24 |
| *** dtrainor has joined #oooq | 07:24 | |
| *** rascasoft has joined #oooq | 07:26 | |
| *** dtrainor_ has joined #oooq | 07:26 | |
| quiquell|rover | ykarel: yep confirme rabbitmq-server is missing from yum -y install, will fix | 07:28 |
| quiquell|rover | chkumar|ruck: ^ | 07:28 |
| ykarel | yup | 07:28 |
| *** dtrainor has quit IRC | 07:29 | |
| chkumar|ruck | quiquell|rover, ykarel it means append part is not working | 07:29 |
| ykarel | chkumar|ruck, yup it's overridden | 07:29 |
| ykarel | by other append | 07:29 |
| quiquell|rover | ykarel: yep later on it override | 07:30 |
| ykarel | yup | 07:30 |
| *** apetrich has joined #oooq | 07:50 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:51 | |
| *** ykarel is now known as ykarel|lunch | 07:59 | |
| *** zul has quit IRC | 08:07 | |
| *** kopecmartin|off is now known as kopecmartin | 08:09 | |
| *** quique|rover|brb is now known as quiquell|rover | 08:09 | |
| quiquell|rover | zbr|ssbarnea: I am going to moleculize fetch-images role | 08:11 |
| quiquell|rover | Going to do a refactoring there | 08:11 |
| *** matbu has quit IRC | 08:16 | |
| *** matbu has joined #oooq | 08:19 | |
| *** panda|off is now known as panda | 08:32 | |
| quiquell|rover | panda: o/ how is that eye ? | 08:32 |
| panda | quiquell|rover: better today, hydration problem at night, sometimes it gets so dry while I'm sleeping that is painful in the morning and blurry all day | 08:34 |
| chkumar|ruck | brb after lunch | 08:34 |
| marios | panda: what is it man some infection or something? | 08:34 |
| marios | panda: or is it fight club again (but we don't talk about that, obv) | 08:35 |
| panda | marios: of course infection. (rule number 1 ... rule number 1!) | 08:36 |
| marios | :D | 08:37 |
| panda | the level of spam in my mailbox has reached troubling levels | 08:37 |
| panda | for example, what's this "Weekly report from marios" ? | 08:37 |
| panda | delete! delete! | 08:38 |
| marios | haha i would think you have a filter for that already panda | 08:38 |
| marios | > /dev/null | 08:38 |
| quiquell|rover | panda, marios: To cleanup phase2 from promoter https://review.rdoproject.org/r/#/c/18724/ | 08:40 |
| quiquell|rover | Less logs to see at master.log | 08:41 |
| marios | quiquell|rover: ack will check | 08:42 |
| *** ykarel|lunch is now known as ykarel | 08:47 | |
| *** amoralej|off is now known as amoralej | 08:48 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (2 more messages) | 08:51 |
| *** bogdando has joined #oooq | 08:52 | |
| *** tosky has joined #oooq | 08:56 | |
| quiquell|rover | sshnaidm|afk, marios: fixed https://review.openstack.org/634917 | 09:01 |
| quiquell|rover | sorry about context | 09:01 |
| zbr|ssbarnea | quiquell|rover: cool! ping me with any questions you may have. you already have two poc changes to look at. | 09:04 |
| zbr|ssbarnea | @oooq: wanted to share with you that starting with 18.09 docker has support for ssh, you can do: DOCKER_HOST=ssh://remote docker ps | 09:05 |
| zbr|ssbarnea | and all other functions, teoretically python-docker should also work with that but i didn't test yet. | 09:05 |
| quiquell|rover | zbr|ssbarnea: Hummm looks like using docker-ce is not very welcome around | 09:05 |
| zbr|ssbarnea | ...well so far that's the only option of mac and also the version installed by kubespray.... | 09:08 |
| *** jpena|off is now known as jpena | 09:09 | |
| zbr|ssbarnea | i find it quite cool and useful to point VIRTHOST and DOCKER_HOST to remote machines and keep my desktop less busy. | 09:10 |
| quiquell|rover | zbr|ssbarnea: we usually focus on centos/fedora though | 09:10 |
| zbr|ssbarnea | quiquell|rover: that's what it runs on the "remote" hosts. | 09:11 |
| quiquell|rover | zbr|ssbarnea: ack | 09:11 |
| marios | quiquell|rover: ack will have another look | 09:30 |
| quiquell|rover | marios: thanks | 09:31 |
| quiquell|rover | marios: we will have to do same thing at RDO :-( | 09:31 |
| *** derekh has joined #oooq | 09:34 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:48 | |
| sshnaidm | quiquell|rover, I still don't understand what is purpose of https://review.openstack.org/#/c/634917/ | 09:51 |
| quiquell|rover | sshnaidm: branch_override mechanism is not working for tripleo-upgrade role | 09:52 |
| sshnaidm | quiquell|rover, firstly, which jobs are we talking about? | 09:53 |
| quiquell|rover | sshnaidm: I was investigating failure here https://review.openstack.org/#/c/602248/ | 09:54 |
| quiquell|rover | sshnaidm: I and found the issue and we have a fix for it | 09:54 |
| quiquell|rover | sshnaidm: then I didn't understand why the -pike jobs were not failing | 09:55 |
| quiquell|rover | sshnaidm: here https://review.openstack.org/#/c/560445 | 09:55 |
| quiquell|rover | sshnaidm: So I discover the -pike jobs are using master tripleo-upgrade role | 09:55 |
| quiquell|rover | sshnaidm: and it's beacuase tripleo-upgrade is not at required-projects | 09:55 |
| quiquell|rover | sshnaidm: so toci mechanism of selecting what to clone is old stuff | 09:56 |
| sshnaidm | quiquell|rover, which job in https://review.openstack.org/#/c/602248/ ? | 09:56 |
| sshnaidm | quiquell|rover, talk to me with jobs :) | 09:57 |
| quiquell|rover | Ahh wait we don't have -pike jobs there | 09:58 |
| quiquell|rover | Let me find a unrelated review | 09:58 |
| sshnaidm | quiquell|rover, -pike jobs shouldn't use tripleo-upgrade at all | 09:58 |
| quiquell|rover | sshnaidm: well that's another issue | 09:58 |
| quiquell|rover | sshnaidm: they don't but ansible try to expand them | 09:59 |
| quiquell|rover | sshnaidm: because we are using role to include it | 09:59 |
| sshnaidm | quiquell|rover, what do you mean? do you have logs? | 09:59 |
| quiquell|rover | sshnaidm: yep | 09:59 |
| quiquell|rover | sshnaidm: wait, first this https://review.openstack.org/#/c/634724/ | 09:59 |
| quiquell|rover | sshnaidm: you see -pike job is passing | 09:59 |
| quiquell|rover | sshnaidm: it should not pass if pike noop is not passing | 09:59 |
| quiquell|rover | sshnaidm: so I ran -pike job at the reproducer | 10:01 |
| quiquell|rover | sshnaidm: and even we don't should run tripleo-upgrade we clone it and the role is at master not at pike | 10:02 |
| sshnaidm | quiquell|rover, do you mean this failure in noop? http://logs.openstack.org/48/602248/23/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/39c20d6/job-output.txt.gz#_2019-02-06_02_10_48_172080 | 10:02 |
| quiquell|rover | sshnaidm: I mean the stuff from requirements | 10:02 |
| sshnaidm | quiquell|rover, sorry, let's go step by step | 10:02 |
| quiquell|rover | sshnaidm: yep | 10:02 |
| quiquell|rover | sshnaidm: ok the bug https://bugs.launchpad.net/tripleo/+bug/1814531 | 10:02 |
| openstack | Launchpad bug 1814531 in tripleo "pike: Running tripleo-upgrade at normal job" [Critical,In progress] - Assigned to Jose Luis Franco (jfrancoa) | 10:02 |
| sshnaidm | quiquell|rover, why tripleo-upgrade is running in this job at all? | 10:03 |
| quiquell|rover | sshnaidm: It's not running it's expanding the role I think | 10:03 |
| quiquell|rover | sshnaidm: but we have a fix for that | 10:03 |
| sshnaidm | quiquell|rover, what a fix? | 10:04 |
| quiquell|rover | https://review.openstack.org/#/c/634724/ | 10:04 |
| quiquell|rover | this fix it | 10:04 |
| sshnaidm | quiquell|rover, ok, cool | 10:04 |
| *** sanjayu_ has joined #oooq | 10:04 | |
| *** saneax has quit IRC | 10:04 | |
| sshnaidm | quiquell|rover, and then? | 10:04 |
| quiquell|rover | sshnaidm: Then my issue is that this logic https://github.com/openstack-infra/tripleo-ci/blob/master/roles/run-test/templates/toci_gate_test.sh.j2#L154-L161 | 10:06 |
| quiquell|rover | sshnaidm: Does not work anymore | 10:06 |
| quiquell|rover | sshnaidm: for branch_override | 10:06 |
| quiquell|rover | sshnaidm: -d $TRIPLEO_ROOT/tripleo-upgrade this is always true | 10:06 |
| sshnaidm | quiquell|rover, just a sec | 10:07 |
| sshnaidm | quiquell|rover, with this fix we don't need to worry about tripleo upgrade in pike jobs, right? | 10:07 |
| sshnaidm | quiquell|rover, how is this bash part related? | 10:08 |
| quiquell|rover | sshnaidm: The fix is the main part | 10:09 |
| quiquell|rover | sshnaidm: bug we cannot run upgrade jobs with branch_override | 10:09 |
| quiquell|rover | sshnaidm: they will not work as expected | 10:09 |
| quiquell|rover | sshnaidm: they will not use branch tripleo-upgrade | 10:10 |
| quiquell|rover | sshnaidm: Don't know how big problem is this | 10:10 |
| quiquell|rover | sshnaidm: In general all required-projects are not going to be in the branch_override version | 10:11 |
| quiquell|rover | sshnaidm: At least the clone, taking from DLRN repo will work well | 10:11 |
| sshnaidm | quiquell|rover, how the upgrade jobs related to branch override? I can't understand this | 10:12 |
| sshnaidm | quiquell|rover, let's take an example upgrades job | 10:13 |
| quiquell|rover | sshnaidm: Imaging that we have a this job http://logs.openstack.org/45/560445/240/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/dbe360f/ | 10:15 |
| quiquell|rover | sshnaidm: but tripleo-ci-centos-7-containerized-undercloud-upgrades-Ãpike | 10:15 |
| quiquell|rover | tripleo-ci-centos-7-containerized-undercloud-upgrades-pÃike | 10:15 |
| quiquell|rover | damn.. | 10:15 |
| quiquell|rover | tripleo-ci-centos-7-containerized-undercloud-upgrades-pike | 10:15 |
| quiquell|rover | it means we have branch_override: stable/pike | 10:15 |
| quiquell|rover | So the job should install tripleo-upgrade stable/pike | 10:16 |
| quiquell|rover | Is that correct ? | 10:16 |
| sshnaidm | quiquell|rover, sorry, brb | 10:16 |
| quiquell|rover | ack | 10:16 |
| *** chkumar|ruck has quit IRC | 10:16 | |
| *** chandankumar has joined #oooq | 10:17 | |
| *** chandankumar is now known as chandan|ruck | 10:18 | |
| arxcruz | well, this is really hard | 10:20 |
| arxcruz | 2019-02-05 18:57:13.259684 | primary | vars_files: | 10:20 |
| arxcruz | 2019-02-05 18:57:13.259973 | primary | - "{{ playbook_path | dirname }}/roles/validate-tempest/vars/tempest_skip_master.yml" | 10:20 |
| *** chandan|ruck has quit IRC | 10:20 | |
| arxcruz | 2019-02-05 18:57:13.260074 | primary | ^ here | 10:20 |
| arxcruz | 2019-02-05 18:57:13.260309 | primary | We could be wrong, but this one looks like it might be an issue with | 10:20 |
| arxcruz | 2019-02-05 18:57:13.260567 | primary | missing quotes. Always quote template expression brackets when they | 10:20 |
| arxcruz | the QUOTES IS THERE! | 10:20 |
| Tengu | "We could be wrong" well, yep, you're wrong, ansible :D | 10:22 |
| sshnaidm | quiquell|rover, back | 10:25 |
| sshnaidm | quiquell|rover, ok, let's take tripleo-ci-centos-7-containerized-undercloud-upgrades-pike | 10:25 |
| sshnaidm | quiquell|rover, do you have logs? | 10:25 |
| quiquell|rover | job does not exists | 10:26 |
| quiquell|rover | I am just talking about upgrade job with branch_override | 10:26 |
| sshnaidm | quiquell|rover, here you change for branchless jobs: tripleo-ci-centos-7-containers-multinode-pike | 10:26 |
| quiquell|rover | In case we neeed them | 10:26 |
| quiquell|rover | yep | 10:26 |
| sshnaidm | quiquell|rover, https://review.openstack.org/#/c/634917/4/zuul.d/multinode-jobs.yaml | 10:26 |
| sshnaidm | quiquell|rover, sorry, I don't get - which job does have problem now? | 10:27 |
| sshnaidm | quiquell|rover, tripleo-ci-centos-7-containers-multinode-* jobs in your patch shouldn't use tripleo-upgrades at all | 10:27 |
| quiquell|rover | sshnaidm: I know, but this is not the stuff I discover | 10:28 |
| quiquell|rover | sshnaidm: Imaging that they have to | 10:28 |
| quiquell|rover | sshnaidm: They should have the branch_override version of the role | 10:28 |
| quiquell|rover | sshnaidm: Is that right ? | 10:28 |
| sshnaidm | quiquell|rover, depends | 10:28 |
| sshnaidm | quiquell|rover, depends on which patch/repo/branch they run | 10:29 |
| sshnaidm | quiquell|rover, if it runs on repo tripleo-upgrades branch pike - no need to configure anything | 10:29 |
| quiquell|rover | sshnaidm: well | 10:29 |
| quiquell|rover | wait | 10:29 |
| quiquell|rover | We have RDO | 10:29 |
| sshnaidm | quiquell|rover, yeah? | 10:30 |
| *** chandankumar has joined #oooq | 10:30 | |
| quiquell|rover | sshnaidm: https://softwarefactory-project.io/zuul/t/rdoproject.org/job/periodic-tripleo-ci-centos-7-singlenode-featureset050-upgrades-rocky | 10:30 |
| quiquell|rover | sshnaidm: look at this job | 10:30 |
| sshnaidm | quiquell|rover, yep | 10:30 |
| quiquell|rover | sshnaidm: This is periodic | 10:31 |
| sshnaidm | yeah | 10:33 |
| quiquell|rover | sshnaidm: Let me verify some stuff there | 10:33 |
| quiquell|rover | sshnaidm: this job is going to clone "master" at /home/zuul/src/git.openstack.org/openstack/tripleo-upgrade/ | 10:34 |
| quiquell|rover | sshnaidm: and it does http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-singlenode-featureset050-upgrades-rocky/777d036/zuul-info/inventory.yaml | 10:35 |
| quiquell|rover | sshnaidm: so we are using tripleo-upgrade master for this job instead of stable/rocky | 10:35 |
| quiquell|rover | sshnaidm: that's it | 10:35 |
| quiquell|rover | sshnaidm: https://paste.fedoraproject.org/paste/AR3Vmf3S1Y0KzEp0LDiAxw | 10:36 |
| *** chandankumar has quit IRC | 10:38 | |
| sshnaidm | quiquell|rover, ok, I see it's cloned as master, how did it work before with branch override? | 10:40 |
| quiquell|rover | sshnaidm: so before we moved tripleo-upgrade to required-projects | 10:41 |
| quiquell|rover | sshnaidm: the directory was there only if we have a Depends-On | 10:41 |
| quiquell|rover | sshnaidm: otherwise we just use ZUUL_BRANCH was override with branch_override | 10:41 |
| quiquell|rover | sshnaidm: but now the clone is always there so we never do the fallback to ZUUL_BRANCH | 10:41 |
| sshnaidm | quiquell|rover, ok, so we need to change branc checkout in periodic-tripleo-ci-centos-7-singlenode-featureset050-upgrades-rocky | 10:42 |
| sshnaidm | quiquell|rover, but you change it for not related jobs in your patch | 10:43 |
| quiquell|rover | sshnaidm: I would sahy all branchful jobs have to use zuul mechanism | 10:43 |
| quiquell|rover | sshnaidm: question, tripleo-upgrade is the only one using this ? | 10:44 |
| sshnaidm | quiquell|rover, well, seems like better to do it, yeah. But in case of multinode jobs it won't change somethign | 10:44 |
| quiquell|rover | sshnaidm: So I depcrate my change and change just upgrade jobs at RDO | 10:45 |
| quiquell|rover | ? | 10:45 |
| sshnaidm | quiquell|rover, in required project - yes, afaik, need to ask Chandan if os_tempest has branches too | 10:45 |
| quiquell|rover | sshnaidm: or we should do it at all override_branch ? | 10:45 |
| sshnaidm | where is Chandan | 10:45 |
| sshnaidm | quiquell|rover, now I think it's really better to do in all branchful jobs because we add a lot of stuff in required_projects and some of it can be branchful | 10:46 |
| quiquell|rover | sshnaidm: is at doctor | 10:46 |
| quiquell|rover | sshnaidm: he will come back | 10:46 |
| sshnaidm | arxcruz, ping | 10:46 |
| arxcruz | sshnaidm: pong | 10:46 |
| sshnaidm | arxcruz, is os_tempest branched? | 10:46 |
| quiquell|rover | sshnaidm: yep it is not easy to explain | 10:47 |
| arxcruz | sshnaidm: just the usual stable/{queens, rocky, stein...} | 10:48 |
| sshnaidm | quiquell|rover, well, the key word is required_project.. I use the same mechanism for ovb jobs | 10:48 |
| sshnaidm | arxcruz, ack ^^ quiquell|rover | 10:48 |
| quiquell|rover | sshnaidm: but that role is installed from DLRN | 10:48 |
| quiquell|rover | sshnaidm: so we don't have any problem I think | 10:48 |
| sshnaidm | quiquell|rover, so when we'll have os_tempest running on all jobs we want it to run with correct branch | 10:48 |
| quiquell|rover | or do we ? | 10:49 |
| sshnaidm | quiquell|rover, which role? | 10:49 |
| quiquell|rover | os_tempest | 10:49 |
| sshnaidm | quiquell|rover, it's in required_project | 10:49 |
| quiquell|rover | sshnaidm: yes but we install using RPMs | 10:49 |
| quiquell|rover | sshnaidm: not tripleo-quickstar-tequirements.txt | 10:50 |
| quiquell|rover | sshnaidm: We calculate the DLRN hash with {{ release }} | 10:50 |
| sshnaidm | quiquell|rover, really? | 10:50 |
| quiquell|rover | sshnaidm: and branch_override is working well there | 10:50 |
| quiquell|rover | sshnaidm: I think | 10:50 |
| sshnaidm | arxcruz, how do we use os_tempest? | 10:50 |
| quiquell|rover | sshnaidm: but have to check | 10:50 |
| sshnaidm | arxcruz, do we build rpm? | 10:50 |
| arxcruz | sshnaidm: nope, git | 10:51 |
| sshnaidm | let's look at the job | 10:51 |
| sshnaidm | quiquell|rover, you see ^ | 10:51 |
| arxcruz | sshnaidm: it's on quickstart dependences | 10:51 |
| panda | arxcruz: really? tempest is not branched ... why would they do that ? | 10:51 |
| quiquell|rover | arg | 10:51 |
| quiquell|rover | :-/ | 10:51 |
| quiquell|rover | ok | 10:51 |
| quiquell|rover | then we really ned this at branchful jobs | 10:51 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (2 more messages) | 10:51 |
| quiquell|rover | all of them | 10:51 |
| quiquell|rover | Maybe even replace branch_override but that's next step | 10:52 |
| sshnaidm | arxcruz, quiquell|rover worth to check if we use it from patches and not from master only.. | 10:52 |
| arxcruz | sshnaidm: https://github.com/openstack/tripleo-quickstart/blob/master/quickstart-extras-requirements.txt#L4-L6 | 10:52 |
| sshnaidm | quiquell|rover, yeah | 10:52 |
| sshnaidm | quiquell|rover, https://github.com/openstack-infra/tripleo-ci/blob/master/roles/run-test/templates/toci_gate_test.sh.j2#L165 | 10:52 |
| sshnaidm | arxcruz, really weird why to have it branched while tempest is not | 10:53 |
| arxcruz | sshnaidm: i'm confirming with osa guys | 10:53 |
| panda | arxcruz: yes it's branched | 10:54 |
| arxcruz | panda: well, os_tempest just run the tempest | 10:54 |
| panda | <insert BUT WHY? meme here> | 10:54 |
| arxcruz | it will download install and run tempest | 10:54 |
| arxcruz | so, i don't see a problem here, the tempest remains branchless | 10:54 |
| *** ykarel is now known as ykarel|mtg | 10:57 | |
| sshnaidm | arxcruz, don't osa guys know why it's branched? | 10:58 |
| sshnaidm | quiquell|rover, so let's do it for all branchful jobs, but upgrades and periodic seems like must | 10:59 |
| arxcruz | sshnaidm: i'm pinging the guys, as soon as they answer me, i'll let you know :) | 10:59 |
| quiquell|rover | sshnaidm: but we still keep branch_override ? | 10:59 |
| sshnaidm | quiquell|rover, seems like not needed, but I'm not sure | 11:00 |
| sshnaidm | quiquell|rover, need to be tested/checked as a lot of things tied to it | 11:00 |
| quiquell|rover | sshnaidm: so baby steps | 11:00 |
| quiquell|rover | sshnaidm: first fix then clean | 11:00 |
| sshnaidm | yeah | 11:00 |
| quiquell|rover | sshnaidm: the review on tripleo-ci cover all of them ? | 11:03 |
| quiquell|rover | sshnaidm: upstream ones I mean | 11:03 |
| quiquell|rover | Yep no one except tripleo-ci is using it at tripeo | 11:04 |
| quiquell|rover | http://codesearch.openstack.org/?q=branch_override&i=nope&files=&repos= | 11:04 |
| sshnaidm | quiquell|rover, need to look at upgrades jobs | 11:04 |
| quiquell|rover | sshnaidm: codesearch don't find it there | 11:05 |
| quiquell|rover | marios: let's workflow this to clean pike noop https://review.openstack.org/#/c/634724 | 11:06 |
| marios | quiquell|rover: ack but i want to revisit i saw some of your discussion with sagi here but didn't follow/doing something else right now | 11:11 |
| sshnaidm | marios, we discussed actually a different topic | 11:12 |
| sshnaidm | it's confusing though | 11:12 |
| marios | sshnaidm: (from same bug i thought, one of the other reviews) | 11:12 |
| marios | sshnaidm: ack | 11:12 |
| quiquell|rover | marios: The review just pasted is not related to branch_override | 11:12 |
| sshnaidm | quiquell|rover, I'm not sure what this job runs on: https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/multinode-jobs.yaml#L259 | 11:13 |
| *** ccamacho has joined #oooq | 11:13 | |
| quiquell|rover | marios: is about fixing pike jobs | 11:13 |
| marios | quiquell|rover: k (same bug though right?) | 11:13 |
| marios | quiquell|rover: will check in bit if someone doesn't beat me to it | 11:13 |
| quiquell|rover | marios: ack thanks | 11:14 |
| quiquell|rover | marios: the branch_override thing is a little confused but looks like we need it | 11:14 |
| quiquell|rover | confusing I mean | 11:14 |
| quiquell|rover | sshnaidm: you mean on what branch ? | 11:14 |
| quiquell|rover | sshnaidm: is not a branch_override one we don't care about that | 11:15 |
| quiquell|rover | sshnaidm: well run on whatever the review is | 11:15 |
| quiquell|rover | sshnaidm: except (newton/ocata/pike) | 11:15 |
| sshnaidm | quiquell|rover, yeah.. weird that it's upgrades, but not branch override.. | 11:16 |
| sshnaidm | marios, do you know what does it upgrade to what? tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/multinode-jobs.yaml#L259 | 11:16 |
| marios | sshnaidm: i think there is not even repo switch there, is like no-op | 11:17 |
| quiquell|rover | sshnaidm: But we don't need branch override at all upgrade jobs | 11:17 |
| *** udesale has quit IRC | 11:17 | |
| marios | sshnaidm: but confirm with upgrades team may have changed | 11:17 |
| quiquell|rover | sshnaidm: is just a mechanissm to test tq at other branches | 11:17 |
| sshnaidm | quiquell|rover, yeah, right | 11:18 |
| sshnaidm | I hate upgrades | 11:18 |
| marios | jfrancoa: does tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades even switch repos - i think not right? (noop upgrade jst testing the mechanism?) | 11:18 |
| marios | chem: ? ^ | 11:18 |
| quiquell|rover | upgrades is inception + back to the futureç | 11:19 |
| marios | zbr|ssbarnea: you were right about the fedora 28 issue/beaker (is dns) | 11:19 |
| marios | zbr|ssbarnea: after i re-added the internal server was able to sync, but even then only to clock.redhat.com... don't know if other timeservers are blocked | 11:20 |
| chem | marios: hey, I haven't touched it in a long time. Last one was matbu I think. But it should be as you said, without changing repo | 11:20 |
| marios | chem: ack thanks (also o/ sup !) | 11:20 |
| zbr|ssbarnea | marios: security concerns. now the real question is: isn't this cause by bad DHCP settings? DHCP can inform about which time servers to use. | 11:22 |
| zbr|ssbarnea | and I would say that this is the right place to do it. | 11:23 |
| zbr|ssbarnea | two dhcp options realted to that: ntp-servers and time-servers | 11:23 |
| marios | zbr|ssbarnea: yeah it makes sense to get that from the dhcp server but seems like it isn't done and ther is blanket ban/filter | 11:25 |
| zbr|ssbarnea | marios: raise a ticket, cc me to support it. makes no sense to have invalid default networking setup: filtered-ntp but ntp not advertised by dhcp server. | 11:26 |
| marios | sudo: openstack: command not found | 11:27 |
| marios | really :/ (is openstack-3) :/ | 11:27 |
| * marios shakes fist | 11:28 | |
| jfrancoa | marios: no, at least this job doesn't update repos. it's just a master to master, as I commented in the patch that's why we didn't realize for some time that UpgradeInitiCommand wasn't working in the upstream CI because we have the same repos before the upgrade and after | 11:34 |
| *** ykarel|mtg is now known as ykarel | 11:34 | |
| jfrancoa | sshnaidm:^^^ | 11:35 |
| jfrancoa | sshnaidm: marios: the behaviour of that job was changed some time after last PTG, it was decided to use the standalone-upgrade job to check the ansible tasks and this job to verify the workflow itself, even though it's not a real upgrade | 11:36 |
| marios | jfrancoa: ack thanks | 11:39 |
| jfrancoa | marios: np | 11:40 |
| *** ykarel has quit IRC | 11:43 | |
| *** ykarel has joined #oooq | 11:43 | |
| panda | marios: you have 5 minutes for me ? | 11:44 |
| panda | marios: 50 ? 500 ? | 11:45 |
| zbr|ssbarnea | i got into a weird issue: i modify /etc/sysconfig/docker-storage at a task but later I realise that the file was not updated. is anything that can auto-revert such a change? | 11:49 |
| panda | zbr|ssbarnea: docker-storage-setup ? | 11:53 |
| zbr|ssbarnea | panda: what is with it? i didn't touch that one, only the one without -setup | 11:54 |
| marios | panda: getting some food can we do in bit | 11:57 |
| zbr|ssbarnea | panda: i think I get it, that file is regenerated automatically by the service based on the -setup one. It could have being easier to mention that in a comment. "generated" | 11:58 |
| panda | marios: ok, I'll take my lunch too then | 12:01 |
| *** ratailor has quit IRC | 12:09 | |
| *** jpena is now known as jpena|lunch | 12:28 | |
| *** rf0lc0 has joined #oooq | 12:35 | |
| *** vinaykns has joined #oooq | 12:35 | |
| *** rfolco has quit IRC | 12:37 | |
| quiquell|rover | panda, marios: Let's workflow the override-checkout at tripleo-ci https://review.openstack.org/#/c/634917/ | 12:38 |
| zbr|ssbarnea | panda: do you know a programatic way to detect if "/" is using overlayfs? | 12:38 |
| quiquell|rover | We will have to do the same at RDO | 12:38 |
| zbr|ssbarnea | i would prefer avoiding `mount | grep "on / type overlay"` ... as I am not sure how reliable is. | 12:39 |
| zbr|ssbarnea | i tried using stat by it does not report fs type | 12:40 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario001 (2 more messages) | 12:51 |
| *** gkadam has quit IRC | 12:53 | |
| *** skramaja_ has joined #oooq | 12:56 | |
| weshay | quiquell|rover, you joining the program meeting? | 12:57 |
| *** skramaja has quit IRC | 12:57 | |
| quiquell|rover | weshay: yep | 12:57 |
| weshay | quiquell|rover, k.. I updated the status in the doc | 12:57 |
| weshay | thanks | 12:57 |
| quiquell|rover | weshay: ahh thanks | 12:58 |
| quiquell|rover | weshay: we talk only about stein here ? | 12:58 |
| sshnaidm | quiquell|rover, did you talk with tristan wrt rdo keys? | 12:58 |
| quiquell|rover | sshnaidm: we need to inject rsa keys at /root/known_hosts inside container | 12:59 |
| quiquell|rover | sshnaidm: I am into it with ssh-keyscan | 12:59 |
| sshnaidm | quiquell|rover, k | 12:59 |
| panda | zbr|ssbarnea: lsblk / blkid ? | 13:00 |
| weshay | quiquell|rover, ya.. only stein | 13:00 |
| weshay | quiquell|rover, you can drop after you give status | 13:01 |
| quiquell|rover | weshay: ack | 13:02 |
| quiquell|rover | keykay? | 13:03 |
| quiquell|rover | really ? | 13:03 |
| weshay | I can remove that | 13:04 |
| weshay | lolz | 13:04 |
| quiquell|rover | just kidding I kind of like it | 13:04 |
| weshay | quiquell|rover, bet you 10 bucks they fuck yup your name | 13:04 |
| quiquell|rover | weshay: they are creative people | 13:04 |
| weshay | :) | 13:04 |
| * weshay waits to get paid | 13:04 | |
| quiquell|rover | Let me find the wallet | 13:05 |
| *** rlandy has joined #oooq | 13:06 | |
| panda | marios: did you finish your κουπεπια ? | 13:06 |
| *** amoralej is now known as amoralej|lunch | 13:07 | |
| marios | panda: yeah you want bluejeans? give me few mins, lets say half passed ok for you? | 13:07 |
| marios | panda: finish off what i'm doing | 13:07 |
| marios | and get a coffee | 13:07 |
| marios | and stuff | 13:07 |
| marios | :D | 13:07 |
| panda | marios: ok | 13:08 |
| marios | panda: subject matter? | 13:08 |
| marios | or is it classified | 13:08 |
| panda | marios: rule number 1 | 13:08 |
| rf0lc0 | marios, did you get a successful clean deploy on f28 standalone ? I didn't... | 13:14 |
| *** rf0lc0 is now known as rfolco | 13:14 | |
| marios | rfolco: no i have the same as you | 13:16 |
| marios | rfolco: http://pastebin.test.redhat.com/708798 | 13:16 |
| marios | rfolco: but its still progress ;) i am going to context swithc and try debug that later | 13:17 |
| rfolco | marios, ok thanks, please let me know if you get progress... | 13:18 |
| weshay | quiquell|rover, well done.. I guess I owe you 10 bucks | 13:18 |
| quiquell|rover | weshay: na no one say my name | 13:20 |
| quiquell|rover | weshay: so no one give money | 13:20 |
| * zbr|ssbarnea busy watching our internal brexit primetime... it seems that you can run but you cannot escape it. | 13:25 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| rfolco | ah marios one quick update: I ran some zuul-jobs roles and I went further than last point | 13:32 |
| rfolco | but this is for me running on vm | 13:32 |
| rfolco | marios, is it fair to rename our manual deployment tasks in - vm vs baremetal ? assuming you are running on beaker | 13:34 |
| rfolco | just a thought :) | 13:34 |
| *** trown|outtypewww is now known as trown | 13:35 | |
| rlandy | quiquell|rover: hey still having reproducer ci connection issues? | 13:36 |
| quiquell|rover | rlandy: I have to try the known_host inside container, didn't have time yet | 13:36 |
| rlandy | quiquell|rover:ack - ok - still trying to get connection back on my local system | 13:38 |
| rlandy | quiquell|rover: are you operational from your local zuul? | 13:38 |
| rlandy | I clear all the known hosts | 13:38 |
| quiquell|rover | rlandy: is the known_hosts inside the container | 13:39 |
| quiquell|rover | rlandy: we have to put there the RSA keys for gerrit hosts | 13:39 |
| marios | rfolco: ack yeah i added it on the description (running in beaker) and so some of my notes are kinda beaker specific | 13:42 |
| marios | rfolco: sure feel free to rename if you want dont mind | 13:43 |
| *** tosky has quit IRC | 13:44 | |
| rfolco | marios, nice that we have 2 different scenarios tested, thx | 13:44 |
| marios | rfolco: ack yeah i found it useful to have your notes for comparison - esp the error being the same | 13:46 |
| rlandy | zbr|ssbarnea: weshay: I am not permissioned to view the molecule BJ recording | 13:47 |
| weshay | rlandy, ah k.. sec | 13:47 |
| weshay | rlandy, ok.. try again https://bluejeans.com/s/G60Im | 13:48 |
| rlandy | weshay: thanks | 13:50 |
| weshay | anyone need help figuring out what to do w/ a user story? | 13:50 |
| weshay | marios, rlandy rfolco sshnaidm zbr|ssbarnea panda arxcruz ^ | 13:51 |
| *** ykarel has quit IRC | 13:52 | |
| rfolco | weshay, need to decide what to do about scen9, as f28 is consuming all my energy... will know better later today and talk to rlandy about it | 13:52 |
| panda | rfolco: I'll take it | 13:54 |
| rlandy | rfolco: weshay: I want to start on that today | 13:54 |
| weshay | arxcruz, what user stories are you on? | 13:54 |
| rlandy | panda: I'd like to be involved | 13:54 |
| marios | weshay: the periodic conatiners build will need some discussion / but not quite ready yet (want to work on it a bit more today first ... posted that yesterday for the parenting https://review.rdoproject.org/r/18729 - gonna switch to in a bit | 13:54 |
| rfolco | panda, rlandy my heroes | 13:54 |
| rlandy | panda: you're on scenario12, right? | 13:54 |
| panda | rlandy: yes | 13:57 |
| panda | rlandy: I'll take 9 only if it's orphan | 13:57 |
| rlandy | panda: do you want to talk about the work on scenario009? | 13:57 |
| rlandy | panda: we can work on it together | 13:57 |
| rlandy | let's talk about the split | 13:57 |
| marios | weshay: re that chronyc issue it was both dns and ntpserver added note in https://tree.taiga.io/project/tripleo-ci-board/task/708 (i.e. even if you fix dns you can still only talk to clock.redhat from beaker looks like - at least got passed that bit anyway ) | 13:58 |
| panda | rlandy: ok in 1h ? | 13:58 |
| rlandy | panda: I haven't worked on scenarios so it will be good to work with someone | 13:58 |
| rlandy | panda: sure - pine me when you are ready to talk | 13:58 |
| weshay | marios, did it work w/ a redeploy or on the box running it manually | 13:58 |
| marios | weshay: with a redeploy but specifying the right dns/ntp in the standalone_parameters as in the task | 13:59 |
| weshay | sshnaidm, thanks for the review, updated w/ your suggestions https://review.openstack.org/635023 | 14:00 |
| weshay | marios, ok.. good to know | 14:00 |
| weshay | I may have some ipv6 ipv4 funk on my home network messing me up w/ that | 14:00 |
| weshay | will try again | 14:00 |
| marios | weshay: ack | 14:00 |
| jfrancoa | quiquell|rover: marios: when you have some review cycles, could you please check this? it's an easy fix and it should help make the upgrades job green again https://review.openstack.org/#/c/635130/ | 14:01 |
| marios | jfrancoa: ack added for next reviews | 14:01 |
| jfrancoa | marios: thanks | 14:01 |
| quiquell|rover | jfrancoa: let's upgrades be great again !!! | 14:01 |
| jfrancoa | quiquell|rover: I love that sentence, it's terrific! you'll love it! | 14:02 |
| quiquell|rover | jfrancoa: at RDO we don't have compute node ? | 14:02 |
| jfrancoa | quiquell|rover: nope | 14:03 |
| quiquell|rover | jfrancoa: ack +w | 14:03 |
| weshay | jfrancoa, I'd like to work w/ you or $someone from upgrades on https://review.openstack.org/#/c/630363/ | 14:04 |
| *** holser_ has joined #oooq | 14:04 | |
| weshay | I'm out of context though.. 3 weeks later | 14:04 |
| weshay | so need to refresh | 14:04 |
| *** jtomasek has quit IRC | 14:04 | |
| *** jtomasek has joined #oooq | 14:05 | |
| sshnaidm | weshay, it should depend on install-deps patch, because now it has a wrong link | 14:05 |
| sshnaidm | weshay, should depend on that: https://review.openstack.org/#/c/634301/ | 14:06 |
| weshay | sshnaidm, no.. it's rebased on top of https://review.openstack.org/#/c/634301/ | 14:06 |
| jfrancoa | weshay: sure, I'll have a look at it. or is it better to wait until it gets rebased/updated? | 14:06 |
| weshay | sshnaidm, look at the git log | 14:06 |
| sshnaidm | weshay, oh, right, it's the same repo | 14:06 |
| weshay | :) | 14:07 |
| weshay | if you read planted tripleo.. this was cool https://www.anstack.com/blog/2019/02/05/tripleo-quickstart-deployments.html | 14:07 |
| weshay | ccamacho++ | 14:07 |
| hubbot1 | weshay: ccamacho's karma is now 1 | 14:07 |
| weshay | ccamacho++ | 14:07 |
| weshay | ccamacho++ | 14:07 |
| hubbot1 | weshay: ccamacho's karma is now 2 | 14:07 |
| hubbot1 | weshay: ccamacho's karma is now 3 | 14:08 |
| *** jtomasek has quit IRC | 14:08 | |
| weshay | panda, rfolco noting scenario09 is demonstrated there | 14:08 |
| *** jtomasek has joined #oooq | 14:09 | |
| weshay | sshnaidm, can you vote on https://review.openstack.org/#/c/634301/ as well | 14:10 |
| weshay | again.. thanks :) | 14:10 |
| weshay | since you are right there | 14:10 |
| weshay | thanks | 14:11 |
| ccamacho | weshay o/ :) | 14:14 |
| weshay | nice update! | 14:15 |
| weshay | panda you want to chat? | 14:16 |
| ccamacho | I tested it yesterday so, for master they should work :) | 14:16 |
| weshay | panda, maybe run through the user stories? | 14:16 |
| ccamacho | upgrades++ | 14:16 |
| ccamacho | oooq++ | 14:16 |
| ccamacho | xD | 14:16 |
| * ccamacho going back to my cave | 14:16 | |
| weshay | lol | 14:17 |
| *** ykarel has joined #oooq | 14:19 | |
| *** skramaja_ has quit IRC | 14:22 | |
| weshay | panda, ? | 14:23 |
| panda | weshay: ok | 14:24 |
| quiquell|rover | rlandy, sshnaidm: bad host key for rdo gerrit at repro -> https://review.rdoproject.org/r/18736 | 14:26 |
| quiquell|rover | Let's see what CI says about it | 14:26 |
| rlandy | quiquell|rover++ | 14:26 |
| hubbot1 | rlandy: quiquell|rover's karma is now 5 | 14:26 |
| weshay | zbr|ssbarnea, please start filling this out when you have time https://tree.taiga.io/project/tripleo-ci-board/us/651?milestone=217491 | 14:28 |
| weshay | panda, http://38.145.32.227/20190205.txt | 14:29 |
| zbr|ssbarnea | weshay: yep, i update it today. still in meeting about brexit. | 14:30 |
| *** amoralej|lunch is now known as amoralej | 14:30 | |
| weshay | lolz :) | 14:30 |
| weshay | kk | 14:30 |
| *** zul has joined #oooq | 14:34 | |
| weshay | rlandy, oy.. did you see https://logs.rdoproject.org/64/18664/19/check/tripleo-ci-reproducer-fedora-28-host/1821dde/job-output.txt.gz#_2019-02-06_13_43_27_637284 | 14:40 |
| rlandy | weshay: hmmm ... guess will need to rethink that | 14:42 |
| quiquell|rover | rlandy, weshay: Didn't have much time to look into repro reviews | 14:43 |
| rlandy | tmp/tmp3k7ofzvq/1821dde4e4614ac3a0d663307a107714/untrusted/project_3/review.rdoproject.org/rdo-infra/ansible-role-tripleo-ci-reproducer/playbooks/tripleo-ci-reproducer/../bindep_python2.txt | 14:43 |
| rlandy | weshay: ^^ ci has other objections | 14:43 |
| weshay | rlandy, repaste that | 14:44 |
| rlandy | will need to rebase | 14:44 |
| *** jtomasek has quit IRC | 14:44 | |
| rlandy | yep | 14:44 |
| *** dtrainor_ has quit IRC | 14:45 | |
| weshay | panda, the pass rate on fedora-28 mixed is still much lower than centos7 | 14:45 |
| quiquell|rover | rlandy working locally wity know host at executor and merger too | 14:46 |
| quiquell|rover | :-) | 14:46 |
| quiquell|rover | Let's see know at CI | 14:46 |
| quiquell|rover | sshnaidm: ^ | 14:46 |
| rlandy | weshay: hmmm ... we don;t call a lookup there | 14:50 |
| rlandy | maybe it doesn't like playbook_dir? | 14:51 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario001 (2 more messages) | 14:51 |
| weshay | rlandy, hrm.. and 2019-02-06 13:43:26.566481 | primary | cat: /tmp/tmp3k7ofzvq/1821dde4e4614ac3a0d663307a107714/untrusted/project_3/review.rdoproject.org/rdo-infra/ansible-role-tripleo-ci-reproducer/playbooks/tripleo-ci-reproducer/../bindep_python2.txt: No such file or directory | 14:52 |
| quiquell|rover | rlandy: reproducer fixed http://logs.rdoproject.org/36/18736/2/check/tripleo-ci-reproducer-centos-7-host/49e1278/ | 14:53 |
| rlandy | weshay: that makes sense | 14:53 |
| quiquell|rover | rlandy: let's merge https://review.rdoproject.org/r/#/c/18736/ | 14:53 |
| quiquell|rover | weshay, sshnaidm: ^ | 14:53 |
| rlandy | ok | 14:53 |
| weshay | quiquell|rover, is the ci working? | 14:54 |
| rlandy | weshay: fixing that | 14:54 |
| weshay | quiquell|rover, failing on basic no-op changes | 14:54 |
| weshay | does that review fix the ci? | 14:54 |
| quiquell|rover | weshay: with tha review it will work host ones has just passed | 14:54 |
| weshay | k. cool | 14:55 |
| *** dtantsur has joined #oooq | 14:55 | |
| quiquell|rover | weshay: ti fix CI and non CI | 14:55 |
| quiquell|rover | weshay: for me was not working too | 14:55 |
| weshay | k.. see that | 14:55 |
| rlandy | weshay: we're making the same mistake the ci-focused stuff made - but in the reverse direction | 14:57 |
| weshay | in which particular case? | 14:58 |
| weshay | do we need to blue? | 14:58 |
| rlandy | weshay: I think we need to set a repo_root | 14:58 |
| weshay | rlandy, or is this an inner / outer playbook issue | 15:00 |
| weshay | https://logs.rdoproject.org/64/18664/19/check/tripleo-ci-reproducer-fedora-28-host/1821dde/job-output.txt.gz#_2019-02-06_13_43_24_509451 | 15:00 |
| rlandy | weshay: I don't think so - we copied the playbooks to a playbooks dir - ci dies not do that | 15:00 |
| weshay | so the playbook_dir is actually diff in ci | 15:00 |
| rlandy | we are off a couple dirs | 15:00 |
| rlandy | ack | 15:00 |
| rlandy | we were assuming the setup we created outside ci | 15:01 |
| rlandy | the lookup in yet another problem | 15:01 |
| *** agopi|off is now known as agopi | 15:01 | |
| weshay | rlandy, hrm.. so we should probably have that aligned so both are the same | 15:01 |
| weshay | will lead to headaches in the future if they are not | 15:02 |
| rlandy | weshay: then we will have to revert to the paths originally there | 15:02 |
| rlandy | to run playbooks from cloned git | 15:02 |
| rlandy | but I think we can determine the path | 15:02 |
| weshay | I think the solution for that should be discussed for a minute | 15:03 |
| rlandy | k - in 5 | 15:04 |
| panda | marios: Description in Design field for https://tree.taiga.io/project/tripleo-ci-board/us/652 is accurate. Added a note for the pus part | 15:04 |
| *** quiquell|rover is now known as quique|rover|eat | 15:04 | |
| marios | panda: whats the pus part | 15:06 |
| marios | push you mean? | 15:06 |
| panda | marios: no no, the pus, you know when you have an infection ? :) yes. push | 15:07 |
| rlandy | weshay: k - can bj when you are ready | 15:07 |
| marios | panda: k will do | 15:08 |
| weshay | k.. /me joins | 15:08 |
| weshay | quique|rover|eat, let me know if you have a sec for ^ | 15:08 |
| quique|rover|eat | weshay: connecting | 15:14 |
| quique|rover|eat | Have the fix in a few | 15:15 |
| quique|rover|eat | what blue ? | 15:15 |
| weshay | quique|rover|eat, https://bluejeans.com/u/whayutin/ | 15:15 |
| arxcruz | weshay: hey boss, finally was able to deploy fs021, i'm seeing the error on tempest.scenario.test_network_basic_ops.TestNetworkBasicOps, I'll start to debug now | 15:23 |
| weshay | nice | 15:23 |
| arxcruz | sorry the delay, i was strugling yesterday to fix the delorean hash that was getting one that doesn't exist | 15:23 |
| weshay | arxcruz, ooh.. interesting | 15:23 |
| panda | rlandy: hey, you have time now for the scenario ? | 15:27 |
| rlandy | panda: in meeting - maybe in half hour? | 15:27 |
| panda | rlandy: yes, ok. | 15:28 |
| *** sanjayu_ has quit IRC | 15:35 | |
| *** jfrancoa has quit IRC | 15:38 | |
| *** quique|rover|eat is now known as quiquell|rover | 15:40 | |
| *** zbr has joined #oooq | 15:46 | |
| *** zbr|ssbarnea has quit IRC | 15:47 | |
| *** udesale has joined #oooq | 15:48 | |
| *** jfrancoa has joined #oooq | 15:52 | |
| *** jtomasek has joined #oooq | 15:59 | |
| quiquell|rover | weshay: Going to drop now, did you say you need an update on some bug ? | 15:59 |
| weshay | looking | 16:00 |
| quiquell|rover | ack will wait | 16:01 |
| weshay | quiquell|rover, get out of here, I'll chat w/ you tomorrow | 16:01 |
| *** ykarel is now known as ykarel|away | 16:01 | |
| weshay | quiquell|rover, ping me tomorrow for a reminder | 16:01 |
| quiquell|rover | ack bye | 16:01 |
| *** quiquell|rover is now known as quiquell|off | 16:01 | |
| rlandy | panda: k - let's talk - will finish this later | 16:03 |
| panda | rlandy: your bj ? | 16:07 |
| rlandy | panda: sure | 16:07 |
| rlandy | weshay: will touch base with you after ^^ | 16:07 |
| rlandy | think package install is the issue | 16:08 |
| weshay | k | 16:08 |
| *** zbr|ssbarnea has joined #oooq | 16:10 | |
| *** tosky has joined #oooq | 16:11 | |
| *** zbr has quit IRC | 16:13 | |
| *** ska has joined #oooq | 16:22 | |
| *** zbr has joined #oooq | 16:23 | |
| vinaykns | weshay: my installation is getting failed at http://pastebin.test.redhat.com/708994 | 16:24 |
| weshay | vinaykns, so.. marios and I both hit that issue | 16:24 |
| marios | vinaykns: beaker box? | 16:25 |
| weshay | vinaykns, marios got around it.. | 16:25 |
| vinaykns | yea | 16:25 |
| *** agopi is now known as agopi|brb | 16:25 | |
| ska | Is nic2 always the provisioning network in templates? | 16:25 |
| weshay | vinaykns, I suspect you have update both the undercloud dns and overcloud dns | 16:25 |
| panda | rlandy: forgot to ask, did you modify the --gerrit parameter to pass different gerrit users on the reproducer script ? | 16:25 |
| rlandy | panda: ack | 16:25 |
| marios | vinaykns: if so then use these the dnsservers from /etc/resolv.conf (if you have already run this it may be wiped see https://tree.taiga.io/project/tripleo-ci-board/task/708 for example values | 16:25 |
| rlandy | you can pass upstream-gerrit-user and pass and rdo-gerrit-user and pass | 16:26 |
| marios | vinaykns: and also set clock.redhat.com for the NtpServer | 16:26 |
| *** zbr|ssbarnea has quit IRC | 16:26 | |
| marios | vinaykns: (so you will need to pass into the next deployment your DnsServers: and NtpServer: params | 16:26 |
| panda | rlandy: ok thanks | 16:26 |
| vinaykns | weshay, marios: lemme try again with those changes. | 16:28 |
| *** agopi|brb has quit IRC | 16:32 | |
| panda | marios: when you translated the scenarios, how did you integrate modifications in the featureset ? | 16:34 |
| marios | panda: the only things we carried over were the tempest stuf that goes into featureset_override | 16:35 |
| marios | panda: don't think i came across anything else. what you have in mind | 16:35 |
| *** jtomasek has quit IRC | 16:35 | |
| panda | marios: not in mind, but scenario009 configuration is almost entirely in the featureset file | 16:37 |
| marios | :/ | 16:37 |
| vinaykns | weshay: I haven't updated UC/OC Dnsservers | 16:38 |
| panda | I'll shove everything into featureset override. | 16:38 |
| weshay | vinaykns, k.. not sure why IT blocked public ntp and public dns in some cases | 16:38 |
| vinaykns | weshay, marios: the DnsServers in the undercloud_parameters file and /etc/resolve.conf they are both same | 16:38 |
| weshay | but you'll have to deal w/ that | 16:38 |
| panda | weshay: do you have some time to chat ? | 16:40 |
| marios | vinaykns: yeah if you have already run it once (and it failed on chrony like above) then /etc/resolve.conf has been overwritten with those values which don't work. So you can take the values from another beaker box and use those in the environment file before running it | 16:40 |
| weshay | panda, I do | 16:40 |
| weshay | I'm in | 16:40 |
| vinaykns | marios: okay lemme do that.! | 16:42 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario001 (2 more messages) | 16:51 |
| *** agopi|brb has joined #oooq | 16:52 | |
| weshay | panda, https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/standalone/templates/standalone.sh.j2 | 16:54 |
| *** agopi|brb is now known as agopi | 16:58 | |
| *** udesale has quit IRC | 17:04 | |
| *** gkadam has joined #oooq | 17:09 | |
| rlandy | weshay: let me know when you have time to chat | 17:22 |
| -openstackstatus- NOTICE: Any changes failed around 16:30 UTC today with a review comment from Zuul like "ERROR Unable to find playbook" can be safely rechecked; this was an unanticipated side effect of our work to move base job definitions between configuration repositories. | 17:26 | |
| *** ykarel|away has quit IRC | 17:28 | |
| rlandy | weshay: zuul blocks any access of files outside the zuul work dir. | 17:35 |
| weshay | rlandy, which file was outside? | 17:35 |
| weshay | reqs? | 17:36 |
| *** bogdando has quit IRC | 17:36 | |
| *** kopecmartin is now known as kopecmartin|off | 17:42 | |
| *** trown is now known as trown|lunch | 17:45 | |
| rlandy | yeah - think so | 17:45 |
| rlandy | weshay: paul managed to do this ... https://github.com/openstack/ansible-role-zuul/tree/master/tasks | 17:45 |
| rlandy | maybe times to as his advice | 17:45 |
| * weshay looks | 17:48 | |
| weshay | rlandy, which part of tasks | 17:49 |
| rlandy | weshay; asking on #sf-ops | 17:49 |
| rlandy | weshay: just referring to requirements get file unknown | 17:50 |
| rlandy | weshay: merging https://review.rdoproject.org/r/#/c/18665/ | 17:51 |
| weshay | panda, https://docs.openstack.org/tripleo-quickstart/latest/index.html | 17:51 |
| weshay | rlandy, join my blue in 2min | 17:59 |
| *** derekh has quit IRC | 18:01 | |
| *** jpena is now known as jpena|off | 18:12 | |
| *** gkadam has quit IRC | 18:28 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (3 more messages) | 18:51 |
| *** trown|lunch is now known as trown | 18:55 | |
| rlandy | weshay; playbooks/tripleo-ci-reproducer/bindep_python2.txt: No such file or directory | 19:07 |
| rlandy | weshay: ha - getting somewhere now ... | 19:21 |
| weshay | rlandy, oh cool | 19:40 |
| weshay | hrm.. so python2 rpms are not avail? | 19:40 |
| rlandy | weshay:new patch coming up | 19:40 |
| rlandy | no - the split was wrong | 19:41 |
| *** jfrancoa has quit IRC | 19:58 | |
| rlandy | weshay: woohoo at could not open requirements file | 20:00 |
| rlandy | passed package install | 20:00 |
| weshay | cool | 20:00 |
| * weshay looks | 20:00 | |
| weshay | ah ya | 20:00 |
| weshay | oh no | 20:00 |
| weshay | this thing still 2019-02-06 20:00:43.271789 | primary | "msg": "\n:stderr: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.\nCould not open requirements file: [Errno 2] No such file or directory: 'src/review.rdoproject.org/rdo-infra/ansible-role-tripleo-ci-reproducer/requirements.t | 20:01 |
| weshay | xt'\n" | 20:01 |
| weshay | dear lord | 20:02 |
| rlandy | https://logs.rdoproject.org/64/18664/30/check/tripleo-ci-reproducer-centos-7-host/ee215b6/job-output.txt.gz#_2019-02-06_20_02_54_750306 | 20:03 |
| rlandy | and that | 20:04 |
| rlandy | "msg": "No package matching 'python2-libselinux' found available, installed or updated", | 20:04 |
| * weshay looks | 20:04 | |
| weshay | ah centos | 20:05 |
| weshay | /me looks | 20:05 |
| sshnaidm | rlandy, can you please review in your time? Add extra node support for OVB https://review.rdoproject.org/r/18740 | 20:06 |
| rlandy | f28 does fine with install | 20:06 |
| rlandy | sshnaidm:ack - will look later tonight | 20:06 |
| weshay | rlandy, ugh.. it's the same old shit between centos and fedora | 20:07 |
| rlandy | charming | 20:07 |
| weshay | the source of truth thing .. it's just ugly | 20:07 |
| rlandy | and it finds the bindep | 20:07 |
| weshay | f28 is python2-netaddr.noarch | 20:07 |
| rlandy | but not requirements | 20:07 |
| weshay | centos is python-netaddr.noarch | 20:07 |
| rlandy | weshay: the truth is always ugly | 20:07 |
| weshay | ya.. that is weird | 20:08 |
| rlandy | will have to switch case the install of packages per platform | 20:08 |
| *** ccamacho has quit IRC | 20:08 | |
| weshay | rlandy, k.. let's chat | 20:09 |
| weshay | python3-all-dev [platform:dpkg !platform:ubuntu-precise] | 20:09 |
| weshay | python3-devel [platform:fedora] | 20:09 |
| weshay | python34-devel [platform:centos] | 20:09 |
| rlandy | k - joining | 20:10 |
| weshay | rlandy, https://gist.github.com/MohamedAlaa/2961058 | 20:19 |
| weshay | https://github.com/weshayutin/python-cheatsheet | 20:20 |
| rlandy | for i in `bindep -b -f bindep_$(python_cmd).txt`; do | 20:25 |
| rlandy | sudo $(package_manager) -y install $i; | 20:25 |
| rlandy | done | 20:25 |
| *** sshnaidm is now known as sshnaidm|afk | 20:25 | |
| *** dtantsur is now known as dtantsur|afk | 20:39 | |
| *** panda is now known as panda|off | 20:41 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario001 (3 more messages) | 20:51 |
| *** zbr|ssbarnea has joined #oooq | 22:00 | |
| *** zbr|ssbarnea has quit IRC | 22:01 | |
| *** zbr has quit IRC | 22:02 | |
| weshay | rlandy, | 22:08 |
| weshay | [root@localhost ~]# cat /etc/docker/daemon.json | 22:08 |
| weshay | {} | 22:08 |
| rlandy | weshay: https://github.com/ansible/ansible/issues/40476 | 22:10 |
| *** holser_ has quit IRC | 22:28 | |
| *** agopi is now known as agopi|brb | 22:36 | |
| *** agopi|brb has quit IRC | 22:40 | |
| rlandy | weshay: taking WIP off https://review.rdoproject.org/r/#/c/18664/ | 22:50 |
| rlandy | after ci runs | 22:50 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario001 (3 more messages) | 22:51 |
| *** agopi|brb has joined #oooq | 23:01 | |
| *** agopi|brb is now known as agopi | 23:04 | |
| rlandy | https://review.rdoproject.org/r/#/c/18664/ | 23:28 |
| rlandy | it's a beautiful thing ^^ | 23:28 |
| rlandy | weshay: and script update for docker config check | 23:28 |
| * rlandy back later | 23:28 | |
| *** rlandy is now known as rlandy|bbl | 23:28 | |
| *** holser_ has joined #oooq | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!