| *** tosky has quit IRC | 00:55 | |
| *** rlandy|rover has quit IRC | 01:08 | |
| *** rlandy has joined #oooq | 01:14 | |
| *** rlandy has quit IRC | 01:15 | |
| *** brault has quit IRC | 02:32 | |
| *** brault has joined #oooq | 02:32 | |
| *** sanjayu_ has joined #oooq | 03:08 | |
| *** saneax has quit IRC | 03:09 | |
| *** sanjayu_ has quit IRC | 03:11 | |
| *** sanjayu_ has joined #oooq | 03:11 | |
| *** ccamacho has quit IRC | 03:13 | |
| *** sanjayu_ has quit IRC | 03:19 | |
| *** saneax has joined #oooq | 03:21 | |
| *** bhagyashris has joined #oooq | 03:42 | |
| *** udesale has joined #oooq | 03:42 | |
| *** sanjayu_ has joined #oooq | 03:49 | |
| *** saneax has quit IRC | 03:51 | |
| *** ykarel|away has joined #oooq | 04:35 | |
| *** bhagyashris has quit IRC | 04:44 | |
| *** skramaja has joined #oooq | 04:45 | |
| *** surpatil has joined #oooq | 05:10 | |
| *** bhagyashris has joined #oooq | 05:18 | |
| *** soniya29 has joined #oooq | 05:26 | |
| *** dsneddon_ has quit IRC | 05:30 | |
| *** ykarel|away is now known as ykarel | 05:47 | |
| *** dsneddon_ has joined #oooq | 05:53 | |
| *** marios has joined #oooq | 06:16 | |
| *** SurajPatil has joined #oooq | 06:22 | |
| *** surpatil has quit IRC | 06:24 | |
| *** SurajPatil has quit IRC | 06:28 | |
| *** surpatil has joined #oooq | 06:28 | |
| *** dtantsur|afk is now known as dtantsur | 06:34 | |
| *** akahat has joined #oooq | 06:48 | |
| *** jfrancoa has joined #oooq | 06:54 | |
| *** sanjayu_ has quit IRC | 06:54 | |
| *** dsneddon_ has quit IRC | 07:21 | |
| *** dsneddon_ has joined #oooq | 07:24 | |
| *** bhagyashris has quit IRC | 07:28 | |
| *** dsneddon_ has quit IRC | 07:31 | |
| *** dsneddon_ has joined #oooq | 07:32 | |
| *** jbadiapa has quit IRC | 07:37 | |
| *** jbadiapa has joined #oooq | 07:38 | |
| *** dsneddon_ has quit IRC | 07:38 | |
| *** ksambor has joined #oooq | 07:41 | |
| sshnaidm|afk | zbr, any ideas why no logs in rhel molecule job? https://review.rdoproject.org/r/#/c/23670/ | 07:41 |
|---|---|---|
| *** surpatil has quit IRC | 07:57 | |
| *** dsneddon_ has joined #oooq | 08:00 | |
| *** tesseract has joined #oooq | 08:00 | |
| raukadah | sshnaidm|afk: Hello | 08:04 |
| *** raukadah is now known as chandankumar | 08:04 | |
| chandankumar | sshnaidm|afk: ipv6 fs035 https://review.opendev.org/694000 | 08:05 |
| sshnaidm|afk | chandankumar, looking | 08:05 |
| chandankumar | sshnaidm|afk: I need a undercloud_network_cidr in ipv6 format? | 08:05 |
| sshnaidm|afk | chandankumar, idk, worth a try | 08:07 |
| sshnaidm|afk | chandankumar, but firstly you need https://review.rdoproject.org/r/23682 | 08:11 |
| chandankumar | sshnaidm|afk: yes | 08:12 |
| *** tosky has joined #oooq | 08:13 | |
| *** jtomasek has quit IRC | 08:13 | |
| *** sanjayu_ has joined #oooq | 08:15 | |
| arxcruz | sshnaidm|afk: marios zbr https://review.opendev.org/#/c/693531/ please :) | 08:15 |
| *** bhagyashris has joined #oooq | 08:15 | |
| *** ksambor has quit IRC | 08:17 | |
| marios | arxcruz: make me | 08:17 |
| marios | arxcruz: (done) please add bug /info | 08:17 |
| arxcruz | the bug is in the code as a comment | 08:18 |
| marios | arxcruz: where /me blind | 08:18 |
| marios | arxcruz: please point to it | 08:19 |
| arxcruz | sshnaidm|afk: i didn't get it | 08:23 |
| arxcruz | sshnaidm|afk: so shall i remove the workaround for the lp as well ? | 08:23 |
| arxcruz | marios: https://review.opendev.org/#/c/693531/5/config/release/tripleo-ci/RedHat-8/master.yml@111 | 08:23 |
| arxcruz | marios: sshnaidm|afk done, added the lp in the commit message | 08:25 |
| marios | arxcruz: answered there | 08:25 |
| *** amoralej|off is now known as amoralej | 08:25 | |
| *** jtomasek has joined #oooq | 08:25 | |
| arxcruz | marios: done, added the lp on the commit message | 08:25 |
| chandankumar | sshnaidm|afk: fs035 is now added, thanks :-) | 08:26 |
| *** surpatil has joined #oooq | 08:29 | |
| sshnaidm|afk | arxcruz, from the patch it's unclear why we should install ceph ansible in every rhel8 job. | 08:30 |
| *** sanjayu__ has joined #oooq | 08:30 | |
| sshnaidm|afk | arxcruz, and how is that bug related | 08:31 |
| *** sanjayu_ has quit IRC | 08:33 | |
| *** epoojad1 has joined #oooq | 08:35 | |
| *** ykarel is now known as ykarel|lunch | 08:37 | |
| *** chem has joined #oooq | 08:39 | |
| *** jpena|off is now known as jpena | 08:48 | |
| *** jtomasek has quit IRC | 08:49 | |
| *** sshnaidm|afk is now known as sshnaidm|rover | 08:55 | |
| arxcruz | sshnaidm|afk: so, this is to test the ceph rpm build package from master | 08:55 |
| arxcruz | sshnaidm|rover:^ | 08:55 |
| arxcruz | scenario001 and 004 needs ceph-ansible | 08:55 |
| *** jbadiapa has quit IRC | 08:57 | |
| sshnaidm|rover | arxcruz, scenario001 works well | 08:57 |
| arxcruz | sshnaidm|rover: yes, the problem is we need to install the newest version of ceph-ansible | 08:57 |
| arxcruz | without add the ceph-ansible in undercloud_rpm_dependencies it will always install the 4.0.0 | 08:58 |
| sshnaidm|rover | arxcruz, and which version do we install now? | 08:58 |
| arxcruz | 4.0.0 | 08:58 |
| sshnaidm|rover | arxcruz, how is this related, I fail to understand | 08:58 |
| arxcruz | sshnaidm|rover: so, as it is right now it's working fine | 08:59 |
| arxcruz | however, when we ran the standalone job to test the latest ceph changes, it fails, because the rpm is never installed, because it's fixed to install 4.0.0 | 09:00 |
| sshnaidm|rover | arxcruz, I think you need to provide more details, I don't see any connection between things, "working fine" is not really an argument :) | 09:00 |
| arxcruz | sshnaidm|rover: ok, i'll update the commit message | 09:00 |
| sshnaidm|rover | arxcruz, so far I don't see how this patch is needed | 09:00 |
| arxcruz | sshnaidm|rover: the rpm being build is version 5.0 | 09:01 |
| arxcruz | the repo is added on the job | 09:01 |
| arxcruz | but it never get installed, because there's this https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/tripleo-ci/RedHat-8/master.yml#L112 | 09:01 |
| sshnaidm|rover | arxcruz, you mean ceph-ansible gate job in github? | 09:01 |
| arxcruz | sshnaidm|rover: yes | 09:01 |
| sshnaidm|rover | arxcruz, well, you need to start from this | 09:02 |
| sshnaidm|rover | arxcruz, and why it's not installed? | 09:03 |
| arxcruz | sshnaidm|rover: i update the commit message, hope it's more clear now | 09:04 |
| arxcruz | sshnaidm|rover: the master.yml for rhel have hardcoded to install 4.0.0 always | 09:05 |
| arxcruz | due the lp bug 1841941 | 09:05 |
| openstack | Launchpad bug 1841941 in tripleo "Cannot run scenario4 on rhel8 due to missing pcs required by manila" [Critical,In progress] https://launchpad.net/bugs/1841941 - Assigned to Tom Barron (tpb) | 09:05 |
| arxcruz | sudo dnf install -y ${NODEPOOL_RDO_PROXY}/rhel8-master/deps/latest/Packages/ceph-ansible-4.0.0-0.1.rc16.1.el8.noarch.rpm | 09:05 |
| sshnaidm|rover | arxcruz, yeah, I see | 09:05 |
| marios | comments please when you next have some time https://review.rdoproject.org/r/#/c/23037 (ensure hashes unchanged https://tree.taiga.io/project/tripleo-ci-board/task/1325 ) | 09:05 |
| *** dsneddon_ has quit IRC | 09:05 | |
| arxcruz | sshnaidm|rover: adding the cepn-ansible in undercloud_rpm_dependencies, it will install the updated version that we are gating | 09:06 |
| arxcruz | since the version is higher | 09:06 |
| sshnaidm|rover | arxcruz, which version is higher? | 09:06 |
| arxcruz | sshnaidm|rover: the master version we are building | 09:06 |
| arxcruz | it's 5.0.0alpha1 | 09:06 |
| sshnaidm|rover | arxcruz, so it will override 4.0., what is the problem with this workaround then? | 09:07 |
| sshnaidm|rover | arxcruz, you actually disable this workaround, but leaving it there, I think it's gonna break all rhel8 jobs that use ceph | 09:08 |
| sshnaidm|rover | arxcruz, your job should explicitly install ceph from built repo, all other jobs shouldn't be affected | 09:08 |
| arxcruz | sshnaidm|rover: because it's an specific ceph-ansible package | 09:09 |
| arxcruz | and the 5.0 will be only on my job | 09:09 |
| arxcruz | sshnaidm|rover: and it's passing on rhel-8 jobs | 09:10 |
| sshnaidm|rover | arxcruz, so maybe you don't need this workaround anymore? did you check it? | 09:10 |
| sshnaidm|rover | arxcruz, I see the bug has a few patches merged | 09:11 |
| arxcruz | sshnaidm|rover: i can remove the workaround let's see | 09:11 |
| sshnaidm|rover | arxcruz, I'd like it will be checked with #rdo, not just test patch | 09:12 |
| *** ccamacho has joined #oooq | 09:12 | |
| arxcruz | ok | 09:12 |
| *** dsneddon_ has joined #oooq | 09:16 | |
| *** ksambor has joined #oooq | 09:18 | |
| *** dsneddon_ has quit IRC | 09:21 | |
| *** jbadiapa has joined #oooq | 09:27 | |
| *** ccamacho has quit IRC | 09:32 | |
| *** ccamacho has joined #oooq | 09:36 | |
| *** ksambor has quit IRC | 09:38 | |
| *** bogdando has joined #oooq | 09:41 | |
| *** dsneddon_ has joined #oooq | 09:45 | |
| *** bogdando has quit IRC | 09:49 | |
| *** ykarel|lunch is now known as ykarel | 09:49 | |
| *** ksambor has joined #oooq | 09:59 | |
| *** ksambor has quit IRC | 10:00 | |
| *** bogdando has joined #oooq | 10:09 | |
| *** zbr has quit IRC | 10:21 | |
| *** d0ugal has quit IRC | 10:22 | |
| *** d0ugal has joined #oooq | 10:22 | |
| *** ksambor has joined #oooq | 10:23 | |
| *** ksambor has quit IRC | 10:23 | |
| *** jtomasek has joined #oooq | 10:44 | |
| *** zbr has joined #oooq | 10:47 | |
| *** dsneddon_ has quit IRC | 10:49 | |
| chandankumar | marios: panda regarding component CI poc, jpena will be sharing a diagram soon based on the existing discussion to clear the confusion | 11:00 |
| panda | no single diagram from a single person will clear the confusion at this stage. And it shouldn't. It's the time to ask more questions than give answers. | 11:03 |
| *** dsneddon_ has joined #oooq | 11:03 | |
| *** dsneddon_ has quit IRC | 11:08 | |
| chandankumar | panda: jpena may be taking all the queries on etherpad or hackmd will help? then have a meeting ? | 11:10 |
| marios | chandankumar: panda thanks ... well more diagrams are always good and yah we'll need lots. migi had the idea of joining ours with his like upstream to downstream that would be nice | 11:10 |
| *** udesale has quit IRC | 11:15 | |
| *** epoojad1 has quit IRC | 11:27 | |
| * marios foob biab | 11:46 | |
| *** marios has quit IRC | 11:46 | |
| chandankumar | sshnaidm|rover: http://logs.rdoproject.org/00/694000/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/7581743/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 11:47 |
| chandankumar | sshnaidm|rover: all passed except one | 11:47 |
| *** sanjayu__ has quit IRC | 11:48 | |
| chandankumar | got skipped | 11:48 |
| chandankumar | sshnaidm|rover: from tempest.log, ipv6 addresses are used http://logs.rdoproject.org/00/694000/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/7581743/logs/undercloud/home/zuul/tempest/tempest.log.txt.gz | 11:49 |
| chandankumar | arxcruz: panda please have a look https://review.opendev.org/#/c/694000/ when free, thanks :-) | 11:51 |
| *** saneax has joined #oooq | 11:55 | |
| *** rfolco has joined #oooq | 11:56 | |
| *** dsneddon_ has joined #oooq | 11:59 | |
| *** bhagyashris has quit IRC | 11:59 | |
| panda | matbu: did you extend the invitation for the promoter to michal ? Promoter, is the actual promoter server, not the component CI, for which we have a separate meeting today. | 12:02 |
| matbu | panda: ha nop, maybe you mean marios ? | 12:03 |
| panda | matbu: yes, but he's not here, sorry | 12:03 |
| matbu | panda: hehe no worries :) | 12:03 |
| rfolco | panda, I see Michal accepted the invite btw | 12:04 |
| panda | rfolco: yes, but he has nothing to discuss with us for the promoter | 12:04 |
| *** dsneddon_ has quit IRC | 12:04 | |
| rfolco | panda, right, its the component ci mtg he should go | 12:05 |
| rfolco | where's marios? | 12:05 |
| panda | rfolco: eating | 12:05 |
| rfolco | :) | 12:06 |
| panda | how dare he ? | 12:07 |
| sshnaidm|rover | chandankumar, nice | 12:08 |
| sshnaidm|rover | chandankumar, although I think it's still ipv4 network on overcloud, but not sure if it's solvable now.. | 12:09 |
| chandankumar | sshnaidm|rover: but in tempest.log ipv6 ip address are used | 12:09 |
| chandankumar | that got hitted against overcloud | 12:09 |
| chandankumar | one more thing to check wait in clouds.yaml | 12:09 |
| chandankumar | sshnaidm|rover: http://logs.rdoproject.org/00/694000/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/7581743/logs/undercloud/home/zuul/.config/openstack/clouds.yaml.txt.gz | 12:10 |
| sshnaidm|rover | chandankumar, and tempest network as well? | 12:10 |
| chandankumar | undercloud ipv4 and overcloud4 | 12:10 |
| chandankumar | sshnaidm|rover: yes | 12:10 |
| sshnaidm|rover | chandankumar, cool | 12:10 |
| chandankumar | sshnaidm|rover: it looks for public network id | 12:10 |
| sshnaidm|rover | chandankumar, great | 12:11 |
| chandankumar | ok we sorted out ipv6 stuff one done | 12:11 |
| chandankumar | sshnaidm|rover: another question, in promotion pipeline with fs030 job and standalone-scenario007 job | 12:11 |
| chandankumar | both are overcloud deployment, do we want to run both or remove fs030 job? | 12:12 |
| sshnaidm|rover | chandankumar, lemme check this | 12:14 |
| *** sshnaidm|rover is now known as sshnaidm|bbl | 12:14 | |
| chandankumar | sshnaidm|bbl: ok | 12:14 |
| zbr | sshnaidm|bbl: i got into Error: No matching repo to modify: rhui-*. and i want to know if that is normal on rhel-8 or is something with my job definition. | 12:22 |
| zbr | https://sf.hosted.upshift.rdu2.redhat.com/logs/09/184209/6/check/tripleo-ci-rhel-8-standalone-internal-scenario000/e649bd0/job-output.txt | 12:22 |
| chandankumar | zbr: in downstream we donot use rhui repos | 12:38 |
| chandankumar | zbr: all come from base os or from rhos-release rpm | 12:38 |
| *** dsneddon_ has joined #oooq | 12:39 | |
| *** jpena is now known as jpena|lunch | 12:40 | |
| *** marios has joined #oooq | 12:42 | |
| *** surpatil has quit IRC | 12:44 | |
| *** Goneri has quit IRC | 12:44 | |
| *** chem is now known as chem|lunch | 12:51 | |
| *** rlandy has joined #oooq | 12:52 | |
| *** rlandy is now known as rlandy|rover | 12:52 | |
| chandankumar | rlandy|rover: Hello | 12:55 |
| chandankumar | rlandy|rover: needs your blessing on this https://review.opendev.org/#/c/694000/ as per discussion on ipv6 stuff from yesterday | 12:55 |
| rlandy|rover | chandankumar: hi - looking at hackmd - are you interviewer1? | 12:55 |
| chandankumar | rlandy|rover: /me lowers myself | 12:56 |
| rlandy|rover | We got all the tests passed http://logs.rdoproject.org/00/694000/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/7581743/logs/stestr_results.html except one | 12:57 |
| *** dtantsur is now known as dtantsur|afk | 12:57 | |
| rlandy|rover | ^^ do we plan to fix that one? | 12:57 |
| marios | panda: rfolco we have call in 3 mins? | 12:57 |
| rlandy|rover | chandankumar: ^^ | 12:57 |
| panda | marios: yes and no | 12:58 |
| rlandy|rover | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035SUCCESS in 3h 13m 15s | 12:58 |
| chandankumar | rlandy|rover: fixed | 12:58 |
| rlandy|rover | ^^ passed there | 12:58 |
| rlandy|rover | ok | 12:58 |
| zbr | wow my corporate redhat gmail account is "temporarily unavailable." | 12:58 |
| chandankumar | rlandy|rover: we are now running actual ipv6 tests there | 12:58 |
| rlandy|rover | chandankumar: o - w=? | 12:58 |
| chandankumar | rlandy|rover: yes | 12:59 |
| rlandy|rover | w+? | 12:59 |
| chandankumar | rlandy|rover: yes | 12:59 |
| panda | zbr: dont' you know ? you've been fired. | 12:59 |
| rlandy|rover | dome | 12:59 |
| rlandy|rover | done | 12:59 |
| rlandy|rover | ugh | 12:59 |
| marios | panda: well fantastic then thanks | 12:59 |
| rlandy|rover | bad typing morning | 12:59 |
| chandankumar | rlandy|rover: thanks :-) | 12:59 |
| marios | panda: schroedinger's meeting? | 12:59 |
| marios | it both is and isn't on | 12:59 |
| rfolco | marios, gmeet ?? | 13:00 |
| marios | panda: ah more like there is no invite | 13:00 |
| rlandy|rover | chandankumar: wrt to interviews - I think you shoudl leave you questions on top - as interviewer 1 | 13:00 |
| marios | panda: i mean no link in the invite | 13:00 |
| rfolco | marios, where | 13:00 |
| rfolco | wheeeeere | 13:00 |
| chandankumar | rlandy|rover: ok | 13:00 |
| rlandy|rover | chandankumar: because it's the main knowledge base | 13:00 |
| marios | rfolco: yes do https://meet.google.com/_meet/nmj-kmac-yzb?ijlm=1573650041173&hs=130 panda | 13:00 |
| rlandy|rover | I can go last | 13:00 |
| rfolco | I'm in marios panda | 13:01 |
| chandankumar | rlandy|rover: now check | 13:01 |
| rlandy|rover | chandankumar: great | 13:01 |
| chandankumar | rlandy|rover: sshnaidm|bbl I need to check with each dfg for each fs and scenario to review the tempest tests we are running | 13:02 |
| chandankumar | so that we should have proper coverage | 13:02 |
| chandankumar | running same tests in different scenario does not provide too much value | 13:03 |
| marios | weshay: sshnaidm|bbl rlandy|rover in 1 hour can you folks join us? we wanted to talk about the promoter and moving forward with the new stuff? | 13:05 |
| marios | panda: rfolco ^^^ | 13:05 |
| *** rlandy|rover is now known as rlandy|rover|mtg | 13:05 | |
| rlandy|rover|mtg | marios: ack | 13:05 |
| marios | rlandy|rover|mtg: sshnaidm|bbl weshay just sent invite thanks | 13:06 |
| *** amoralej is now known as amoralej|lunch | 13:09 | |
| chandankumar | marios: panda zbr arxcruz rlandy|rover|mtg: sshnaidm|bbl weshay tristin developed this https://www.softwarefactory-project.io/using-ebpf-programs-to-profile-zuul-ci-builds.html may we can use in our ci, if it does not takes time https://softwarefactory-project.io/logs/71/16971/5/check/dlrn-rpmbuild-tripleo-ci-oooq-rhel8/bb5a187/ci-tracer.html | 13:10 |
| *** epoojad1 has joined #oooq | 13:12 | |
| weshay | rlandy|rover|mtg, ah.. thanks | 13:12 |
| rlandy|rover|mtg | ? | 13:12 |
| *** sanjayu_ has joined #oooq | 13:20 | |
| *** saneax has quit IRC | 13:22 | |
| weshay | rlandy|rover|mtg, don't worry about reporting master | 13:22 |
| weshay | :) | 13:22 |
| weshay | just train in this call | 13:23 |
| rlandy|rover|mtg | matbu: thanks for the update | 13:26 |
| chandankumar | arxcruz: Hello | 13:27 |
| weshay | matbu++ | 13:28 |
| chandankumar | arxcruz: regarding this thread going downstream 'ceph-external job extremely instable', may be we should run more tempest tests related to ceph functionality to find out issues early | 13:28 |
| chandankumar | may be in scenario1 and scenario4 | 13:28 |
| chandankumar | ? | 13:29 |
| *** jpena|lunch is now known as jpena | 13:30 | |
| *** chem|lunch is now known as chem | 13:30 | |
| *** skramaja has quit IRC | 13:32 | |
| weshay | epoojad1++ | 13:33 |
| weshay | congrats epoojad1 on passing the RHCSA :) | 13:33 |
| marios | good job epoojad1 | 13:35 |
| chandankumar | kopecmartin: Hello | 13:36 |
| *** sanjayu__ has joined #oooq | 13:37 | |
| chandankumar | kopecmartin: just two bugs related to packaging issue https://bugzilla.redhat.com/show_bug.cgi?id=1768777 and https://bugzilla.redhat.com/show_bug.cgi?id=1768775 | 13:37 |
| openstack | bugzilla.redhat.com bug 1768777 in python-stestr "python3 stestr trying to use /usr/bin/python2" [Medium,Assigned] - Assigned to chkumar | 13:37 |
| openstack | bugzilla.redhat.com bug 1768775 in openstack-tempest "Depends on python2" [Medium,Assigned] - Assigned to chkumar | 13:37 |
| chandankumar | few days back we fixed stestr bug | 13:37 |
| *** ykarel is now known as ykarel|afk | 13:39 | |
| *** sanjayu_ has quit IRC | 13:39 | |
| *** ksambor has joined #oooq | 13:43 | |
| *** ksambor has quit IRC | 13:43 | |
| zbr | sshnaidm|bbl: please wf https://review.opendev.org/#/c/694041/ and look at https://review.opendev.org/#/c/693882/3 -- looks like bug in podman_container | 13:44 |
| kopecmartin | chandankumar: what was fixed a few days back regarding stestr was renaming stestr-3 to stestr .. this bug is about stestr depending on python2 | 13:46 |
| *** Goneri has joined #oooq | 13:47 | |
| chandankumar | kopecmartin: ok, the package imported downstream quite old, I am importing newer package version for rhos-16, it might fix the issue | 13:50 |
| chandankumar | kopecmartin: give me an hour or two, will get the fix ready | 13:51 |
| tosky | kopecmartin, chandankumar: uh, stestr shouldn't depend on python2 even in any package I know right now - or is the problem in openstack-tempest as dependency? | 13:52 |
| *** rlandy|rover|mtg is now known as rlandy|rover | 13:52 | |
| rlandy|rover | lovely - now all the rhel-8 standlaone jobs are red | 13:52 |
| rlandy|rover | one step forward .... | 13:52 |
| chandankumar | tosky: need to look into that, the downstream packages shipped in osp-16 is also not insync with rdo | 13:53 |
| chandankumar | as it is a manual process | 13:53 |
| tosky | chandankumar: but it should be newer than 2.1.0-4.el8ost | 13:53 |
| tosky | I think you imported a newer one | 13:53 |
| tosky | oh, wrong channel | 13:53 |
| chandankumar | tosky: I will thinking it was automated | 13:53 |
| chandankumar | anyway we will fix it | 13:53 |
| *** epoojad1 has quit IRC | 13:55 | |
| *** migi has joined #oooq | 13:58 | |
| migi | panda: here | 13:58 |
| panda | migi: o/ | 13:58 |
| chandankumar | migi: got some new stuff https://github.com/release-depot/patch_rebaser/ is this used in downstream ? | 13:59 |
| migi | chandankumar: dunno good question to reldel folks, will check | 14:01 |
| *** epoojad1 has joined #oooq | 14:01 | |
| chandankumar | migi: got this from jpena while sharing our ptg conversation | 14:01 |
| *** epoojad1 is now known as epoojad1|afk | 14:01 | |
| jpena | chandankumar, migi: it is used as part of OSP Trunk | 14:02 |
| jpena | to rebase the -patches branch on top of the current branch, called by DLRN | 14:02 |
| chandankumar | migi: jpena we are tracking the notes here https://tree.taiga.io/project/tripleo-ci-board/us/1323?milestone=244669 | 14:02 |
| *** amoralej|lunch is now known as amoralej | 14:05 | |
| *** rlandy|rover is now known as rlandy|rover|mtg | 14:08 | |
| *** akahat has quit IRC | 14:08 | |
| marios | panda: are you putting up taiga? | 14:25 |
| panda | marios: yes | 14:25 |
| marios | panda: rfolco we might as well talk about that stuff too when we meet regarding the component-ci etc | 14:25 |
| panda | migi: who's brian bowen ? | 14:25 |
| marios | panda: thanks | 14:25 |
| *** epoojad1|afk has quit IRC | 14:25 | |
| rfolco | marios, panda I need to chat more about component ci to refine its design before implementing anything | 14:25 |
| *** rlandy|rover|mtg is now known as rlandy|rover | 14:26 | |
| migi | panda: my manager | 14:26 |
| rlandy|rover | weshay: periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-train killing promotions ... if current testproject rerun does not pass, I may remove it | 14:26 |
| migi | panda: why? | 14:26 |
| rfolco | marios, panda if there is task that can be worked in advance, pls let me know | 14:26 |
| migi | panda: ah probably requested doc access, I had 1-1 and was explaining what I'm up to in the current sprint :) | 14:27 |
| weshay | rlandy|rover, sec | 14:28 |
| rfolco | panda, marios time to chat today still ? | 14:29 |
| panda | rfolco: always | 14:30 |
| marios | rfolco: sure gimme | 14:30 |
| rfolco | at least 30 min to define something and get next steps for today/tomorrow | 14:31 |
| marios | rfolco: yes /me runnign for coffe refill while you start meet | 14:31 |
| rfolco | marios, panda https://meet.google.com/bqx-xwht-wky | 14:33 |
| marios | rfolco: panda ? | 14:33 |
| marios | rfolco: joining | 14:33 |
| marios | panda: join us please semaphores this time please | 14:35 |
| rfolco | panda, decide your username and come | 14:35 |
| rfolco | panda, be creative BUT be quick pls | 14:36 |
| panda | rfolco: marios I'm in 1-1, gimme 5 | 14:36 |
| marios | panda: in 25 mins? | 14:37 |
| marios | rfolco: ok with you? ^ | 14:38 |
| rfolco | yep back in 25 | 14:38 |
| *** dsneddon_ has quit IRC | 14:42 | |
| *** ykarel|afk has quit IRC | 14:43 | |
| *** sshnaidm|bbl is now known as sshnaidm|ruck | 14:45 | |
| sshnaidm|ruck | rlandy|rover, hi | 14:46 |
| sshnaidm|ruck | rlandy|rover, sync? | 14:47 |
| rlandy|rover | sshnaidm|ruck: ok | 14:47 |
| sshnaidm|ruck | rlandy|rover, https://meet.google.com/qob-kqnc-ygg | 14:48 |
| *** ksambor has joined #oooq | 14:49 | |
| *** ksambor has quit IRC | 14:49 | |
| *** ykarel has joined #oooq | 14:49 | |
| arxcruz | zbr: i'm taking https://tree.taiga.io/project/tripleo-ci-board/task/1351 if you don't mind ok ? | 14:51 |
| *** dsneddon_ has joined #oooq | 14:51 | |
| *** TrevorV has joined #oooq | 14:52 | |
| *** tesseract has quit IRC | 14:52 | |
| arxcruz | panda: is this done? https://tree.taiga.io/project/tripleo-ci-board/task/1341?kanban-status=1447275 just to not duplicate the work, i see the code were merged... | 14:53 |
| *** tesseract has joined #oooq | 14:55 | |
| *** dsneddon_ has quit IRC | 14:57 | |
| *** ksambor has joined #oooq | 14:58 | |
| marios | rfolco: panda ready when you are (community call link) | 15:00 |
| rlandy|rover | ykarel: thanks for your help on https://bugs.launchpad.net/tripleo/+bug/1851847 | 15:01 |
| openstack | Launchpad bug 1851847 in tripleo "periodic-tripleo-ci-rhel-8-scenario004-standalone-master fails to deploy standalone - pacemaker/mysql" [Critical,Fix released] - Assigned to yatin (yatinkarel) | 15:01 |
| rlandy|rover | ykarel: sshnaidm|ruck and I have a question about https://review.opendev.org/#/c/693992/2/config/release/tripleo-ci/RedHat-8/master.yml | 15:02 |
| panda | marios: link ? | 15:02 |
| rlandy|rover | when/how are the rhel 8.1 containers updated? | 15:02 |
| ykarel | rlandy|rover, np | 15:02 |
| marios | panda: 16:33 #oooq: < rfolco> marios, panda https://meet.google.com/bqx-xwht-wky | 15:02 |
| rlandy|rover | and why don't we keep this change: https://review.opendev.org/#/c/693992/2/config/release/tripleo-ci/RedHat-8/master.yml? | 15:02 |
| rlandy|rover | and always get the updated version? | 15:02 |
| rlandy|rover | ykarel: ^^ | 15:02 |
| ykarel | rlandy|rover, with next promotion containers will be updated | 15:03 |
| rlandy|rover | ykarel: you mean with our promotion? | 15:03 |
| ykarel | containers are build with latest distro packages in promotion pipeline | 15:03 |
| ykarel | rlandy|rover, yes | 15:03 |
| rlandy|rover | oh ok | 15:03 |
| rlandy|rover | ykarel: so - isn't it general good practice though to take the newest rhui when we have a failure | 15:03 |
| rlandy|rover | ie: keep https://review.opendev.org/#/c/693992/2/config/release/tripleo-ci/RedHat-8/master.yml | 15:04 |
| rlandy|rover | unfortunately we failed container build on the last run | 15:04 |
| ykarel | rlandy|rover, u mean updating container in check jobs with distro repos? | 15:05 |
| rlandy|rover | ykarel: yes - when we have a failure in promotions - to fix check | 15:05 |
| ykarel | rlandy|rover, with that we will have side effects, increasing job timing in check | 15:06 |
| ykarel | and we will also have failures in both check/promotion together when issue is there in distro repos | 15:06 |
| rlandy|rover | ykarel: k - something to consider depending on how/when we will get another promotion | 15:06 |
| *** dsneddon_ has joined #oooq | 15:07 | |
| ykarel | rlandy|rover, ack | 15:08 |
| ykarel | rlandy|rover, current failure is real one? | 15:08 |
| ykarel | in container build | 15:08 |
| rlandy|rover | ykarel: possibly - although it's the first container build failure in a long time ... | 15:08 |
| ykarel | anyway the tht patch merged sometime ago, so we need to wait for next run | 15:08 |
| ykarel | ack good to check | 15:08 |
| rlandy|rover | I considered rerunning - but correct - there were merged underway | 15:09 |
| rlandy|rover | so the next run should be the real check | 15:09 |
| rlandy|rover | we will be watching that | 15:09 |
| ykarel | ack | 15:09 |
| * rlandy|rover gets error | 15:09 | |
| rlandy|rover | 2019-11-13 07:49:02 | Command: sudo buildah bud --tls-verify=False --logfile /tmp/kolla-t58aw_tj/docker/swift/swift-object-expirer/swift-object-expirer-build.log -t trunk.registry.rdoproject.org/tripleomaster/rhel-binary-swift-object-expirer:6cd67c01db2056c84a9303663ef6d714d98606f1_638b9ca3 /tmp/kolla-t58aw_tj/docker/swift/swift-object-expirer | 15:10 |
| rlandy|rover | 2019-11-13 07:49:02 | Exit code: 1 | 15:10 |
| rlandy|rover | http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-master-containers-build-push/cc995c3/logs/build.log.txt.gz | 15:11 |
| *** dsneddon_ has quit IRC | 15:12 | |
| rlandy|rover | sshnaidm|ruck: weshay: I put scenario004 ans ovb fs001 back into the promotion criteria for rhel-8 master. We expect those to pass in the next run | 15:13 |
| arxcruz | is gmail off for you guys ? | 15:15 |
| sshnaidm|ruck | rlandy|rover, ack | 15:16 |
| *** skramaja has joined #oooq | 15:17 | |
| *** ksambor has quit IRC | 15:24 | |
| *** skramaja has quit IRC | 15:26 | |
| *** dsneddon_ has joined #oooq | 15:33 | |
| *** soniya29 has quit IRC | 15:33 | |
| marios | https://external-content.duckduckgo.com/iu/?u=http%3A%2F%2Fi0.kym-cdn.com%2Fphotos%2Fimages%2Foriginal%2F000%2F197%2F923%2Fwtf.jpg&f=1&nofb=1 | 15:44 |
| marios | panda: spotted in the wild ^ | 15:45 |
| marios | https://i0.kym-cdn.com/photos/images/original/000/197/923/wtf.jpg | 15:48 |
| *** sanjayu__ has quit IRC | 15:48 | |
| rlandy|rover | oh dear - our gate | 15:49 |
| rlandy|rover | sshnaidm|ruck: ^^ | 15:49 |
| rlandy|rover | "cmd": "set -o pipefail && /home/zuul/install_packages.sh 2>&1 | awk '{ print strftime(\"%Y-%m-%d %H:%M:%S |\"), $0; fflush(); }' > /home/zuul/install_packages.sh.log\n", | 15:50 |
| sshnaidm|ruck | rlandy|rover, what's with it? | 15:50 |
| rlandy|rover | 2019-11-13 15:03:06 | Error: Package: tripleo-ansible-0.5.0-0.20191113132857.7084248.el7.noarch (delorean-current) | 15:50 |
| rlandy|rover | 2019-11-13 15:03:06 | Requires: python2-packaging | 15:50 |
| rlandy|rover | 2019-11-13 15:03:06 | You could try using --skip-broken to work around the problem | 15:50 |
| rlandy|rover | 2019-11-13 15:03:06 | You could try running: rpm -Va --nofiles --nodigest | 15:50 |
| rlandy|rover | sshnaidm|ruck: ^^ | 15:51 |
| rlandy|rover | https://d01cb673c13600630f34-dafd5ea511e702465bad7043bbe0ffd9.ssl.cf1.rackcdn.com/691903/2/gate/tripleo-ci-centos-7-undercloud-containers/14a5eb3/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz | 15:51 |
| sshnaidm|ruck | rlandy|rover, damn.. | 15:51 |
| sshnaidm|ruck | ykarel, don't we have python-packaging in deps? | 15:52 |
| rlandy|rover | checking if all the errors are the same | 15:52 |
| zbr | arxcruz: sure, at the moment I am with rdo on trying to make the job run without any customizations. | 15:52 |
| sshnaidm|ruck | rlandy|rover, it's because of my patch to tripleo-ansible I suppose.. | 15:52 |
| marios | comments please when you next have some time https://review.rdoproject.org/r/#/c/23037 (ensure hashes unchanged https://tree.taiga.io/project/tripleo-ci-board/task/1325 ) panda rfolco weshay rlandy|rover sshnaidm|ruck whenever you have time thank you | 15:52 |
| rlandy|rover | why now though? | 15:52 |
| ykarel | sshnaidm|ruck, we have | 15:52 |
| rlandy|rover | yep - same error everywhere | 15:52 |
| ykarel | atleast in centos and fedora | 15:52 |
| rlandy|rover | 2019-11-13 15:05:41 | Requires: python2-packaging | 15:53 |
| zbr | arxcruz: weshay: one of you could have a brief look at https://code.engineering.redhat.com/gerrit/184209 it would be great, mainly even the reference job is failing. | 15:53 |
| ykarel | sshnaidm|ruck, link for failure? | 15:53 |
| sshnaidm|ruck | ykarel, https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/zuul_opendev_logs_4b8/693993/2/gate/tripleo-ci-centos-7-containers-multinode/4b8ce39/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz | 15:53 |
| sshnaidm|ruck | ykarel, it's from https://review.opendev.org/#/c/688574 and https://review.rdoproject.org/r/#/c/23684/ | 15:54 |
| ykarel | sshnaidm|ruck, ok got it it's not in testing repo, | 15:54 |
| ykarel | amoralej, ^^ | 15:54 |
| ykarel | we can revert rdo one as a quick fix | 15:55 |
| ykarel | sshnaidm|ruck, is that module used somewhere currently? | 15:55 |
| amoralej | mmm | 15:55 |
| sshnaidm|ruck | ykarel, I can revert on tripleo-ansible too | 15:55 |
| amoralej | oh | 15:56 |
| ykarel | sshnaidm|ruck, if that module is not used, i think revert is not need upstream | 15:56 |
| ykarel | until u see some issues | 15:56 |
| rlandy|rover | https://review.rdoproject.org/r/#/c/23684/ is the quicker fix | 15:56 |
| amoralej | it's in el7-build | 15:56 |
| amoralej | we need it in testing | 15:56 |
| sshnaidm|ruck | ykarel, it's used in some new patches and jobs | 15:56 |
| sshnaidm|ruck | ykarel, so it won't work? | 15:56 |
| amoralej | let's promote it quickly | 15:57 |
| sshnaidm|ruck | ykarel, I mean I can revert the part that requires python-packaging | 15:57 |
| amoralej | sshnaidm|ruck, what is it breaking? | 15:57 |
| sshnaidm|ruck | amoralej, well, currently this error break all gates | 15:57 |
| ykarel | ok let's clear gate first, https://review.rdoproject.org/r/#/c/23688/ | 15:58 |
| ykarel | reverted | 15:58 |
| ykarel | and see if tripleo-ansible revert is needed | 15:58 |
| rlandy|rover | upstream one merged 7 hours ago | 15:59 |
| rlandy|rover | trouble with gates only started when the rdo patch merged | 15:59 |
| sshnaidm|ruck | rlandy|rover, yeah, upstream one doesn't affect us | 15:59 |
| ykarel | yup that's why i said possibly that module is not used still | 15:59 |
| sshnaidm|ruck | rlandy|rover, only EmilienM :D | 15:59 |
| ykarel | :D | 15:59 |
| rlandy|rover | sshnaidm|ruck: well - that's a choice for you and EmilienM | 16:00 |
| rlandy|rover | how much it help vs how much it blocks one project | 16:00 |
| ykarel | rdo one should merge soon and available in current repo soon | 16:00 |
| EmilienM | why revert? | 16:00 |
| rlandy|rover | the gate impacts everyone | 16:00 |
| EmilienM | there is no commit message in the revert | 16:00 |
| sshnaidm|ruck | ykarel, amoralej so you need to promote packaging module first? | 16:00 |
| amoralej | ykarel, sshnaidm|ruck but https://review.opendev.org/#/c/688574 is merged | 16:00 |
| EmilienM | ah | 16:00 |
| sshnaidm|ruck | EmilienM, it fails gates | 16:00 |
| EmilienM | "Needs python-packaging in -testing repo." | 16:00 |
| amoralej | so it's going to fail? | 16:00 |
| amoralej | https://review.rdoproject.org/r/#/c/23689/ | 16:01 |
| sshnaidm|ruck | amoralej, most likely, until it's promoted.. | 16:01 |
| ykarel | tripleo projects are used from current, so no promotion is needed | 16:02 |
| amoralej | yes, promotion i meant promotion to -testing | 16:02 |
| amoralej | that review | 16:02 |
| ykarel | okk | 16:02 |
| sshnaidm|ruck | I mean module promotion, yeah | 16:02 |
| amoralej | mmm https://review.rdoproject.org/r/#/c/23684/ passed ci because we are enabling build repo in the job | 16:03 |
| ykarel | and it's likely needed in train-rdo, and train-testing as well as tripleo-ansible is branchless | 16:03 |
| ykarel | but that can be seperate step | 16:03 |
| amoralej | is it unpinned ykarel ? | 16:04 |
| ykarel | amoralej, yes | 16:04 |
| amoralej | i think it's branched | 16:04 |
| amoralej | i see reviews in stable/train | 16:04 |
| sshnaidm|ruck | ykarel, it has branches | 16:04 |
| ykarel | amoralej, sshnaidm|ruck, ohhk, i thought it was not branched | 16:05 |
| ykarel | i remember some discussion around it | 16:05 |
| ykarel | but ok if it's branched | 16:05 |
| sshnaidm|ruck | ykarel, yeah, and I don't know why it was branched.. | 16:05 |
| amoralej | branching is good :) | 16:05 |
| EmilienM | should we land the revert ? | 16:05 |
| sshnaidm|ruck | EmilienM, the spec revert? | 16:06 |
| amoralej | if the code is needing python-packaging, reverting the change in the spec will not help | 16:06 |
| sshnaidm|ruck | yes, please | 16:06 |
| ykarel | yes we should to clear gate quickly | 16:06 |
| *** ccamacho has quit IRC | 16:06 | |
| sshnaidm|ruck | amoralej, it's new code, it can wait a few hours.. | 16:07 |
| sshnaidm|ruck | amoralej, it's not a code from gate jobs | 16:07 |
| amoralej | but https://review.opendev.org/#/c/688574 is merged | 16:07 |
| amoralej | taht's nt used from delorean-current? | 16:07 |
| ykarel | okk tripleo-ansible was branched with other tripleo projects https://github.com/openstack/releases/commit/c3e137bc085869faea2b9a053167e6d9abd5a169#diff-5b2af6e1ec0b1b8b48181ba9c502e08b | 16:07 |
| EmilienM | if we land https://review.rdoproject.org/r/#/c/23689/ it should clear the gate, no? | 16:07 |
| sshnaidm|ruck | amoralej, it should be | 16:08 |
| amoralej | then, i don't understand | 16:08 |
| sshnaidm|ruck | amoralej, it is: Error: Package: tripleo-ansible-0.5.0-0.20191113132857.7084248.el7.noarch (delorean-current) | 16:09 |
| amoralej | then, if we merge https://review.rdoproject.org/r/#/c/23688/ | 16:09 |
| amoralej | it will be able to install | 16:09 |
| amoralej | but | 16:09 |
| amoralej | will it fail because can't import packaging? | 16:09 |
| amoralej | or is that code not used? | 16:09 |
| sshnaidm|ruck | amoralej, but the part that actually uses "packaging" is not used in jobs currently | 16:09 |
| sshnaidm|ruck | amoralej, right | 16:10 |
| amoralej | ok | 16:10 |
| amoralej | so podman_container module is not used at all | 16:10 |
| amoralej | right? | 16:10 |
| sshnaidm|ruck | not in jobs that we run in gates, that's sure | 16:10 |
| sshnaidm|ruck | amoralej, yep | 16:10 |
| amoralej | ok | 16:10 |
| EmilienM | right | 16:10 |
| amoralej | then let's merge the revert | 16:10 |
| EmilienM | it just makes me sad when it breaks | 16:10 |
| EmilienM | but it's another story :P | 16:10 |
| ykarel | EmilienM, 23689 will take some time to get merge, unless we force merge | 16:10 |
| EmilienM | ykarel: ack | 16:10 |
| ykarel | as other tripleo jobs run tere | 16:10 |
| EmilienM | +2 revert then | 16:11 |
| *** Trevor_V has joined #oooq | 16:11 | |
| amoralej | also i'll send a patch to fix gate jobs | 16:11 |
| ykarel | amoralej, ack to not enable build-deps repo during install? | 16:12 |
| amoralej | yep | 16:12 |
| ykarel | okk that's good PA | 16:12 |
| amoralej | althoug that's coming from dlrn i think | 16:13 |
| *** TrevorV has quit IRC | 16:15 | |
| ykarel | ack, good to check | 16:15 |
| *** ykarel is now known as ykarel|away | 16:16 | |
| marios | sshnaidm|ruck: when you next get some time, based on your comments i split that into two reviews https://review.opendev.org/693802 "Move vars/common.yaml to main.yaml for buildimages and containers" and then https://review.opendev.org/#/c/692985/ ontop please check when you next can thanks no fire | 16:21 |
| sshnaidm|ruck | marios, ack, will look | 16:26 |
| chandankumar | rlandy|rover: Did we broke something? all standalone jobs got failed | 16:27 |
| rlandy|rover | chandankumar: yep - see above | 16:27 |
| chandankumar | 2019-11-13 15:02:04 | Requires: python2-packaging | 16:27 |
| chandankumar | got it | 16:27 |
| chandankumar | thanks! | 16:27 |
| rlandy|rover | https://review.rdoproject.org/r/#/c/23688/ should fix it | 16:27 |
| rlandy|rover | chandankumar: you can recheck | 16:28 |
| marios | sshnaidm|ruck: thank you whenever you are next in reviews mode | 16:28 |
| sshnaidm|ruck | rlandy|rover, thanks for heads up, we fixed it really quick | 16:29 |
| *** ksambor has joined #oooq | 16:29 | |
| rlandy|rover | sshnaidm|ruck: do you want to revert the revert https://review.rdoproject.org/r/#/c/23688/ - after https://review.rdoproject.org/r/#/c/23689/ merges? | 16:30 |
| sshnaidm|ruck | rlandy|rover, I think so, if ykarel|away and amoralej tell me that's safe | 16:31 |
| rlandy|rover | rdogerrit> Alfredo Moralejo created rdoinfo master: Promote pytho-packaging to ussuri-testing https://review.rdoproject.org/r/23689 | 16:31 |
| rlandy|rover | <ykarel> sshnaidm|ruck, EmilienM ^^ re revert can be done post ^^ | 16:31 |
| rlandy|rover | looks like it | 16:31 |
| rlandy|rover | we can always revert the revert the revert ..... | 16:32 |
| weshay | chandankumar, you have a minute? | 16:32 |
| *** jtomasek has quit IRC | 16:32 | |
| rlandy|rover | chandankumar: let us know if it's still failing | 16:32 |
| sshnaidm|ruck | let's just ensure it's in the repo | 16:32 |
| rlandy|rover | ack | 16:32 |
| EmilienM | too many reverts I'm lost | 16:32 |
| chandankumar | weshay: yup, let me grab my headphone | 16:32 |
| rlandy|rover | hopefully only one more | 16:33 |
| *** ksambor has quit IRC | 16:33 | |
| sshnaidm|ruck | EmilienM, we'll keep you sad for a few | 16:33 |
| rlandy|rover | we need https://review.rdoproject.org/r/#/c/23689/ to go before the people W+ powers there disappear | 16:34 |
| *** marios is now known as marios|out | 16:37 | |
| EmilienM | sshnaidm|ruck: /me sad face | 16:37 |
| sshnaidm|ruck | rlandy|rover, hmm.. I see a new failure in build image job: http://logs.rdoproject.org/02/693802/1/openstack-check/tripleo-rhel-8-buildimage-overcloud-full/7fb54c3/build.log | 16:40 |
| sshnaidm|ruck | RAM_NEEDED=14 | 16:40 |
| sshnaidm|ruck | '[' 7999188 -lt 14680064 ']' | 16:40 |
| sshnaidm|ruck | '[' 0 == 1 ']' | 16:40 |
| sshnaidm|ruck | return 1 | 16:40 |
| sshnaidm|ruck | rlandy|rover, seems like it has swap, but missed it.. http://logs.rdoproject.org/02/693802/1/openstack-check/tripleo-rhel-8-buildimage-overcloud-full/7fb54c3/zuul-info/host-info.primary.yaml | 16:42 |
| sshnaidm|ruck | ansible_memtotal_mb: 7811 | 16:42 |
| sshnaidm|ruck | marios|out, I rerun rdo jobs on https://review.opendev.org/#/c/693802/ as it failed for unknown reason before, let's see if it's just a occasional failure | 16:43 |
| *** jpena is now known as jpena|brb | 16:45 | |
| *** marios|out has quit IRC | 16:45 | |
| rlandy|rover | sshnaidm|ruck: looking | 16:47 |
| sshnaidm|ruck | rlandy|rover, I reran it, suspect it's just a node failure | 16:47 |
| rlandy|rover | 2019-11-13 11:49:50.679 | failed to glob pattern /etc/rc0.d/[SK][0-9][0-9]network: No such file or directory | 16:48 |
| rlandy|rover | ^^ still there | 16:48 |
| rlandy|rover | 19-11-13 11:49:58.188 | + diskimage_builder/lib/common-functions:tmpfs_check:29 : RAM_NEEDED=14 | 16:48 |
| rlandy|rover | 2019-11-13 11:49:58.191 | + diskimage_builder/lib/common-functions:tmpfs_check:30 : '[' 7999188 -lt 14680064 ']' | 16:48 |
| sshnaidm|ruck | rlandy|rover, ack | 16:48 |
| sshnaidm|ruck | but this ram point is weird as well.. | 16:49 |
| rlandy|rover | sshnaidm|ruck: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 on master is also not looking good | 16:49 |
| ykarel|away | sshnaidm|ruck, rlandy|rover do not hurry for re-revert after -testing patch merges | 16:49 |
| ykarel|away | post merge it takes some time to be in deps repo | 16:50 |
| ykarel|away | and then in infra mirrors | 16:50 |
| ykarel|away | once that merges, it should appear at https://trunk.rdoproject.org/centos7-master/deps/latest/noarch/ in some time | 16:50 |
| ykarel|away | max 30 minutes | 16:50 |
| mjturek | sshnaidm|ruck: Maybe this will help with figuring out what's going on with the ppc builds? https://review.opendev.org/#/c/694111/ | 16:51 |
| rlandy|rover | 2019-11-13 16:37:40.902 | + diskimage_builder/lib/common-functions:tmpfs_check:29 : RAM_NEEDED=14 | 16:51 |
| rlandy|rover | 2019-11-13 16:37:40.905 | + diskimage_builder/lib/common-functions:tmpfs_check:30 : '[' 7999172 -lt 14680064 ']' | 16:51 |
| rlandy|rover | 2019-11-13 16:37:40.907 | + diskimage_builder/lib/common-functions:tmpfs_check:31 : '[' 1 == 1 ']' | 16:51 |
| rlandy|rover | 2019-11-13 16:37:40.910 | + diskimage_builder/lib/common-functions:tmpfs_check:32 : echo 'WARNING: Not enough RAM to use tmpfs for build. Using /tmp. (7999172 < 14G)' | 16:51 |
| rlandy|rover | sshnaidm|ruck: ^^ looks pretty regular | 16:51 |
| sshnaidm|ruck | rlandy|rover, ok | 16:51 |
| weshay | panda, chandankumar tried the key registry and said it doesn't work yet | 16:51 |
| rlandy|rover | checking passing jobs | 16:51 |
| sshnaidm|ruck | rlandy|rover, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 is failing because the same reason of packaging | 16:51 |
| *** jaosorior has joined #oooq | 16:51 | |
| rlandy|rover | ok | 16:51 |
| sshnaidm|ruck | ykarel|away, ack, we'll check this | 16:52 |
| sshnaidm|ruck | mjturek, hope so | 16:52 |
| rlandy|rover | sshnaidm|ruck: yeah - confirmed - the RAM message shows up in passing logs as well http://logs.rdoproject.org/88/687288/65/openstack-check/tripleo-rhel-8-buildimage-overcloud-full/5a98b48/overcloud-full.log | 16:53 |
| rlandy|rover | it's some sort of race condition | 16:53 |
| *** bogdando has quit IRC | 16:53 | |
| chandankumar | weshay: panda if you want to a give a try donot use this https://github.com/quay/quay/blob/master/docs/development-container.md | 16:56 |
| chandankumar | it does not work | 16:56 |
| chandankumar | doc does not mention about quay-config from where to get it | 16:57 |
| chandankumar | panda: https://access.redhat.com/documentation/en-us/red_hat_quay/3/html/deploy_red_hat_quay_-_basic/architecture I have not tried | 16:57 |
| chandankumar | may be a worth trying | 16:57 |
| weshay | rfolco, where is the code for quickstart.sh -c config/general_config/featureset001.yml | 17:03 |
| weshay | https://review.opendev.org/#/c/673005/ | 17:04 |
| weshay | sorry.. that first question was misleading | 17:04 |
| weshay | think I found what I need | 17:05 |
| chandankumar | https://www.mirantis.com/company/press-center/company-news/mirantis-acquires-docker-enterprise/ | 17:06 |
| *** jfrancoa has quit IRC | 17:07 | |
| weshay | rfolco, you there? | 17:08 |
| * weshay grabs a bite | 17:08 | |
| *** jaosorior has quit IRC | 17:12 | |
| *** jpena|brb is now known as jpena | 17:17 | |
| *** apetrich has quit IRC | 17:24 | |
| ykarel|away | rlandy|rover, that build issue is real, caused by https://review.opendev.org/#/c/651204/ | 17:29 |
| ykarel|away | https://review.opendev.org/#/q/Ib15f1ec4766bf4d64a2860422c230e4d514bc224 | 17:29 |
| ykarel|away | i see cherry-pick approved | 17:29 |
| ykarel|away | good to check if there is already fix for it or hold the revies | 17:29 |
| ykarel|away | reviews | 17:29 |
| weshay | rlandy|rover, you have time or too busy? | 17:30 |
| ykarel|away | https://github.com/openstack/tripleo-common/blob/master/container-images/tripleo_kolla_template_overrides.j2#L1016 | 17:30 |
| rlandy|rover | weshay: joining | 17:31 |
| ykarel|away | i don't see anything at https://review.opendev.org/#/q/file:container-images/tripleo_kolla_template_overrides.j2 | 17:31 |
| ykarel|away | EmilienM, can u check ^^ | 17:31 |
| *** rlandy|rover is now known as rlandy|rover|mtg | 17:32 | |
| *** jpena is now known as jpena|brb | 17:35 | |
| *** rlandy|rover|mtg has quit IRC | 17:35 | |
| ykarel|away | EmilienM, more context wrt ^^ https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-master-containers-build-push/cc995c3/logs/buildah-builds/kolla-t58aw_tj/docker/swift/swift-object-expirer/swift-object-expirer-build.log.txt.gz and https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/e9d5b06/log | 17:35 |
| ykarel|away | s/build.log.txt.gz#_2019-11-13_12_44_31 | 17:35 |
| *** rlandy|rover|mtg has joined #oooq | 17:35 | |
| *** rlandy|rover|mtg has quit IRC | 17:38 | |
| *** rlandy has joined #oooq | 17:38 | |
| *** rlandy is now known as rlandy|rover|mtg | 17:39 | |
| rfolco | weshay, did you find it ? | 17:40 |
| rlandy|rover|mtg | ykarel|away: sorry - compuetr carshed | 17:40 |
| rlandy|rover|mtg | crashed | 17:40 |
| rlandy|rover|mtg | looking | 17:40 |
| weshay | sshnaidm|ruck, rlandy|rover|mtg https://review.opendev.org/694120 | 17:46 |
| *** tosky has quit IRC | 17:47 | |
| *** jpena|brb is now known as jpena | 17:51 | |
| *** ykarel|away has quit IRC | 17:51 | |
| *** dtrainor has quit IRC | 17:58 | |
| *** chem is now known as chem|eod | 18:03 | |
| *** jpena is now known as jpena|off | 18:20 | |
| chandankumar | arxcruz: zbr weshay: rfolco rlandy|rover|mtg panda sshnaidm|ruck I will be working from bangalore office from monday, might skip wed retro due to flight | 18:20 |
| weshay | k.. np.. safe travels | 18:21 |
| chandankumar | sshnaidm|ruck: did you get a chance to look at fs030 and scenario7-standalone question? | 18:21 |
| weshay | rfolco, ya.. did you have something started for the tht/ci/environments? | 18:21 |
| sshnaidm|ruck | chandankumar, not yet, will look later | 18:21 |
| rfolco | weshay, something like what | 18:22 |
| weshay | rfolco, something like what you did for the featureset autogen doc | 18:23 |
| weshay | I recall you mentioning it | 18:23 |
| *** rlandy|rover|mtg is now known as rlandy|rover | 18:23 | |
| rfolco | weshay, ah not started yet, no | 18:23 |
| rfolco | weshay, its on my wish list still | 18:23 |
| rfolco | that one would require some socialization, isn't ? weshay | 18:24 |
| weshay | rfolco, you're talking to the ptl :) | 18:26 |
| weshay | rfolco, will ping you this week and run some thoughts by you :) | 18:26 |
| rfolco | weshay, sudo helps ok | 18:27 |
| weshay | rfolco, you had a nice idea there, and I've had some other discussion where that tool would be important | 18:27 |
| rlandy|rover | chandankumar: we're seeing tempest fail on fs039 in train ... http://logs.rdoproject.org/45/23645/4/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-train/cc65ee6/logs/undercloud/home/zuul/tempest/tempest.html.gz -looks like we are we missing something from the fs config | 18:28 |
| rlandy|rover | comparing with fs001, | 18:28 |
| rlandy|rover | there are quite a few different settings | 18:28 |
| rlandy|rover | tempest_tempestconf_profile | 18:28 |
| chandankumar | rlandy|rover: fs039 still screwed up | 18:29 |
| rlandy|rover | tempest_cidr: '10.0.0.0/24' | 18:29 |
| rlandy|rover | chandankumar: it was passing a couple days ago | 18:29 |
| rlandy|rover | was there an additional problem | 18:29 |
| chandankumar | rlandy|rover: checking fs039 | 18:30 |
| *** Goneri has quit IRC | 18:30 | |
| rlandy|rover | chandankumar: I would prefer to fix it than remove from promotion criteria if possible | 18:30 |
| chandankumar | rlandy|rover: fs039 is always a mistry for me from last one month https://review.opendev.org/#/c/639324/ | 18:32 |
| weshay | chandankumar, they just got it passing in master | 18:33 |
| weshay | chandankumar, /me also notes that the security dfg is building a standalone job to test ipa | 18:33 |
| weshay | so it will get better | 18:33 |
| rlandy|rover | chandankumar: hmmm - let's rerun the tests here | 18:34 |
| rlandy|rover | fs039 was not passing on the last run | 18:34 |
| rlandy|rover | fs039 is killing the train promotion. | 18:34 |
| rlandy|rover | I want to remove it to get the promotion going | 18:34 |
| chandankumar | weshay: rlandy|rover the way we are testing the stuff there is wrong, | 18:34 |
| rlandy|rover | chandankumar: let | 18:34 |
| chandankumar | I mean tempest tests | 18:34 |
| rlandy|rover | LP it | 18:34 |
| weshay | srsly | 18:35 |
| chandankumar | rlandy|rover: sshnaidm|ruck and I was working on that | 18:35 |
| *** tesseract has quit IRC | 18:35 | |
| chandankumar | check the patch iteration on this https://review.opendev.org/#/c/639324/ | 18:35 |
| chandankumar | it was suppose to run nova join tempest plugin tests not tempest scenario tests there | 18:35 |
| rlandy|rover | is the patch testable as is? | 18:36 |
| chandankumar | weshay: but standalone without novajoin na | 18:36 |
| chandankumar | ? | 18:36 |
| chandankumar | it was for tls everywhere | 18:37 |
| chandankumar | rlandy|rover: I think yes, we got somestuff not working there, so I depriotrize it | 18:37 |
| rlandy|rover | chandankumar: ok - let me testproject that - and we will see what we get | 18:37 |
| chandankumar | rlandy|rover: https://tree.taiga.io/project/tripleo-ci-board/us/670?milestone=247128 | 18:37 |
| rlandy|rover | in the mean time, train should promote | 18:38 |
| * rlandy|rover tries | 18:38 | |
| *** dtrainor has joined #oooq | 18:39 | |
| chandankumar | rlandy|rover: may be adding a config change will help to test the stuff | 18:39 |
| rlandy|rover | chandankumar: what sort of a config chage? | 18:39 |
| rlandy|rover | change? | 18:40 |
| rlandy|rover | we have many such | 18:40 |
| chandankumar | rlandy|rover: something like this https://review.rdoproject.org/r/#/c/23682/ | 18:40 |
| chandankumar | fs039 third party job auto triggered once a change in fs039 lands | 18:40 |
| rlandy|rover | ack | 18:40 |
| rlandy|rover | agreed | 18:40 |
| * chandankumar rests now in cave see ya | 18:41 | |
| *** chandankumar is now known as raukadah | 18:41 | |
| raukadah | more rrrrr | 18:41 |
| rlandy|rover | https://review.rdoproject.org/r/#/c/23645/ | 18:41 |
| rlandy|rover | raukadah: it's ok - I'm over it | 18:41 |
| sshnaidm|ruck | mjturek, need to fix linters: https://review.opendev.org/#/c/694111/ | 18:42 |
| rlandy|rover | weshay: sshnaidm|ruck: I'm pulling 39 to promote train with reference to this review | 18:42 |
| weshay | rlandy|rover, is there a cix card? | 18:42 |
| rlandy|rover | there can be one | 18:42 |
| sshnaidm|ruck | rlandy|rover, ack | 18:42 |
| weshay | yes please | 18:42 |
| rlandy|rover | on its way | 18:42 |
| sshnaidm|ruck | weshay, it never worked well in train, I think it was out of criteria in previous "master" | 18:43 |
| weshay | sec.. let me get some more context | 18:43 |
| weshay | sshnaidm|ruck, rlandy|rover we're a few weeks away from an upstream tls job | 18:44 |
| weshay | fyi.. | 18:44 |
| weshay | so master is ok'ish https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-master | 18:45 |
| rlandy|rover | damage is done | 18:45 |
| weshay | train is boosted https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-train | 18:45 |
| weshay | but.. I would think that master and train are still close enough | 18:45 |
| weshay | that what ever was done to fix master could be done for train | 18:46 |
| weshay | ack to skip w/ cix card | 18:46 |
| sshnaidm|ruck | weshay, it is 50% rate, better than 0% before, but still | 18:47 |
| weshay | sshnaidm|ruck, understood.. let's cix train, work w/ master as much as possible until we get the upstream job | 18:48 |
| weshay | who likes writing javascript? | 18:48 |
| raukadah | weshay: you mean plain javascript or js framework? | 18:50 |
| weshay | plain | 18:50 |
| sshnaidm|ruck | weshay, such people are not in irc | 18:50 |
| weshay | lolz | 18:50 |
| sshnaidm|ruck | maybe in slack :) | 18:50 |
| weshay | lolz | 18:51 |
| raukadah | weshay: may be tristinc is the your person :-) | 18:51 |
| weshay | I thought a codemonkey / tampermonkey script to remove #code-comments from log files would be awesome | 18:51 |
| raukadah | sshnaidm|ruck: slack is terrible if you are coming from irc land | 18:51 |
| sshnaidm|ruck | raukadah, it pretends to be irc, but can't | 18:51 |
| weshay | when I'm trying to parse openstack log files.. same as the shell command | 18:51 |
| weshay | grep -o '^[^#]*' file | 18:52 |
| weshay | in javascript | 18:52 |
| weshay | maybe I'll give it a go if there are no takers | 18:52 |
| rlandy|rover | raukadah: weshay; sshnaidm|ruck: tracking bug -> CIX https://bugs.launchpad.net/tripleo/+bug/1852477 | 18:53 |
| openstack | Launchpad bug 1852477 in tripleo "periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-train is failing tempest.scenario.test_network_basic_ops.TestNetworkBasicOps" [Critical,Triaged] - Assigned to Ronelle Landy (rlandy) | 18:53 |
| raukadah | weshay: you can check with soniya she knows javascript | 18:53 |
| weshay | heh.. k cool | 18:53 |
| weshay | thanks | 18:53 |
| weshay | thanks rlandy|rover | 18:54 |
| sshnaidm|ruck | weshay, but what can I do with that bug? It was the situation for whole train cycle | 18:54 |
| weshay | sshnaidm|ruck, the only thing that changed is that master is now working | 18:55 |
| sshnaidm|ruck | weshay, and all we got it's increasing from 0% to 50% | 18:55 |
| weshay | heh | 18:55 |
| weshay | sshnaidm|ruck, we need to look at the patches that fixed it for master | 18:55 |
| rlandy|rover | pretty sure tristanC has other things on his plate | 18:55 |
| sshnaidm|ruck | weshay, "fixed" you mean :) https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-master | 18:56 |
| rlandy|rover | periodic-tripleo-rhel-8-buildimage-overcloud-full-master - passed - how nice | 18:56 |
| weshay | nice | 18:56 |
| weshay | it passes at times.. yes | 18:57 |
| rlandy|rover | pity about containers build-push | 18:57 |
| rlandy|rover | I know - I cherish those times | 18:57 |
| rlandy|rover | a little happiness in a sea of red | 18:57 |
| mjturek | thanks sshnaidm|ruck will do! | 19:04 |
| sshnaidm|ruck | rlandy|rover, well, it's same problem with swift-object-expirer | 19:07 |
| sshnaidm|ruck | rlandy|rover, I mean building containers | 19:07 |
| rlandy|rover | yes | 19:07 |
| rlandy|rover | revert still in test | 19:07 |
| sshnaidm|ruck | rlandy|rover, revert? | 19:08 |
| rlandy|rover | sshnaidm|ruck: <openstackgerrit> Alex Schultz proposed openstack/tripleo-common master: Revert "Improve Swift healthchecks" https://review.opendev.org/694118 | 19:08 |
| sshnaidm|ruck | rlandy|rover, ack | 19:09 |
| rlandy|rover | the other cherry picks were -1'ed | 19:09 |
| rlandy|rover | only master merged | 19:09 |
| rlandy|rover | on the upside we have a functional gate again | 19:10 |
| *** sanjayu__ has joined #oooq | 19:23 | |
| *** dtrainor has quit IRC | 19:32 | |
| *** Goneri has joined #oooq | 19:46 | |
| *** amoralej is now known as amoralej|off | 19:47 | |
| rlandy|rover | there's no way to stop a job in gate, right? | 20:05 |
| rlandy|rover | weshay: ^^ | 20:05 |
| weshay | rlandy|rover, you can abandon the change | 20:09 |
| weshay | that will pull it | 20:09 |
| * weshay looks | 20:09 | |
| rlandy|rover | https://review.opendev.org/#/c/693985/ and https://review.opendev.org/#/c/693986/ are still going through gates | 20:09 |
| weshay | one of them has a failing job? | 20:09 |
| weshay | rlandy|rover, what would you like to see happen? | 20:11 |
| weshay | oh.. I see | 20:11 |
| weshay | rlandy|rover, so Alex workflowed and then -1'd both | 20:12 |
| weshay | do you have the context on why? | 20:12 |
| rlandy|rover | weshay: ack - we had to revert master of that change | 20:18 |
| rlandy|rover | it was failing container builds | 20:19 |
| *** jaosorior has joined #oooq | 20:26 | |
| *** dtrainor has joined #oooq | 20:47 | |
| *** jbadiapa has quit IRC | 21:03 | |
| *** jtomasek has joined #oooq | 21:31 | |
| weshay | rlandy|rover, ugh.. memory is not serving.. what was the purpose of scenario012? | 21:31 |
| rlandy|rover | weshay: to torture us | 21:32 |
| weshay | ah yes... evile plan working! | 21:32 |
| * rlandy|rover gets | 21:32 | |
| rlandy|rover | https://tree.taiga.io/project/tripleo-ci-board/us/654 | 21:33 |
| rlandy|rover | weshay: ^^ | 21:33 |
| rlandy|rover | Scenario012 is about testing Ironic on overcloud currently. | 21:34 |
| rlandy|rover | https://github.com/openstack/tripleo-ci/blob/b7979cc18ce766ced781e0876db9373f443c8bcc/zuul.d/standalone-jobs.yaml#L311 | 21:34 |
| rlandy|rover | ^^ what we trigger against | 21:34 |
| weshay | I'm confusing 006 and 12 | 21:35 |
| rlandy|rover | original fs060 https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset060.yml | 21:35 |
| rlandy|rover | because it was fs060? | 21:35 |
| rlandy|rover | you want scenario006? | 21:35 |
| weshay | well.. looking at the docs and config it appeared that scen006 was for ironic | 21:36 |
| weshay | but /me got confused | 21:36 |
| rlandy|rover | that is confusing | 21:36 |
| weshay | rlandy|rover, all the fun I'm having https://docs.google.com/spreadsheets/d/1bGHvAWHPEyZQWvsuGdWCc0Pb87qsDotIwZUr1lI966E/edit#gid=0 | 21:36 |
| rlandy|rover | then we would have two | 21:36 |
| rlandy|rover | where on earth are we calling scenario006? | 21:36 |
| rlandy|rover | - Create featureset026 that will deploy COE on the overcloud instead | 21:36 |
| rlandy|rover | of OpenStack services (scenario006). | 21:36 |
| weshay | well.. it's defined in tht and doc | 21:37 |
| rlandy|rover | yeah - but we haven't touched that in ages | 21:38 |
| *** rfolco has quit IRC | 21:43 | |
| *** sanjayu__ has quit IRC | 22:10 | |
| *** jtomasek has quit IRC | 22:13 | |
| *** Trevor_V has quit IRC | 22:35 | |
| *** tosky has joined #oooq | 22:53 | |
| *** jaosorior has quit IRC | 22:55 | |
| *** rlandy|rover is now known as rlandy|rover|bbl | 23:27 | |
| weshay | crap | 23:30 |
| weshay | rlandy|rover|bbl, sshnaidm|ruck fyi.. infra errors causing gate failures | 23:32 |
| weshay | 2019-11-13 21:34:50 | http://mirror.mtl01.inap.opendev.org:8080/rdo/centos7-train/deps/latest/noarch/GitPython-1.0.1-5.el7.noarch.rpm: [Errno 14] curl#52 - "Empty reply from server" | 23:32 |
| rlandy|rover|bbl | ok - will deal with it when I get back | 23:33 |
| weshay | rlandy|rover|bbl, I got ya | 23:37 |
| weshay | rlandy|rover|bbl, wrote it up, blocker.. ur good | 23:37 |
| *** brault has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
