| *** holser has quit IRC | 00:42 | |
| *** jbadiapa has quit IRC | 00:53 | |
| *** jmasud has quit IRC | 00:53 | |
| *** jmasud has joined #oooq | 00:56 | |
| *** ysandeep|afk is now known as ysandeep | 01:04 | |
| *** rlandy has quit IRC | 01:58 | |
| *** apetrich has quit IRC | 03:09 | |
| *** saneax has joined #oooq | 03:43 | |
| *** ykarel|away is now known as ykarel | 03:45 | |
| *** skramaja has joined #oooq | 04:15 | |
| *** raukadah is now known as chandankumar | 04:37 | |
| *** jmasud has quit IRC | 04:53 | |
| *** udesale has joined #oooq | 04:54 | |
| *** jmasud has joined #oooq | 04:55 | |
| *** ratailor has joined #oooq | 05:13 | |
| *** marios has joined #oooq | 06:32 | |
| *** skramaja has quit IRC | 06:38 | |
| *** skramaja has joined #oooq | 06:46 | |
| *** dpawlik has joined #oooq | 07:10 | |
| *** dpawlik has quit IRC | 07:26 | |
| *** dtantsur|afk is now known as dtantsur | 07:34 | |
| *** pojadhav|away is now known as pojadhav | 07:34 | |
| *** dmsimard|off7 has joined #oooq | 07:48 | |
| *** dmsimard|off has quit IRC | 07:48 | |
| *** dmsimard|off7 is now known as dmsimard|off | 07:48 | |
| *** tesseract has joined #oooq | 07:50 | |
| *** saneax has quit IRC | 07:52 | |
| *** tosky has joined #oooq | 07:57 | |
| *** dpawlik has joined #oooq | 07:58 | |
| *** bogdando has joined #oooq | 08:18 | |
| *** saneax has joined #oooq | 08:22 | |
| *** amoralej|off is now known as amoralej | 08:24 | |
| marios | sshnaidm: chandankumar: do we still need that? shouldn't any more right (i.e. we should have the trunk centos built images now https://review.opendev.org/#/c/711238/ ... goign to remove it from my depends on @ https://review.rdoproject.org/r/#/c/25796/ | 08:26 |
|---|---|---|
| marios | if so abandon it sshnaidm ? | 08:27 |
| *** sshnaidm has quit IRC | 08:29 | |
| marios | ragequit :D | 08:30 |
| chandankumar | marios, checking | 08:31 |
| *** saneax has quit IRC | 08:31 | |
| chandankumar | marios, https://review.opendev.org/#/c/711238/ is no longer needed | 08:32 |
| chandankumar | abandoning it | 08:32 |
| marios | chandankumar: ack thanks for sanity check | 08:32 |
| chandankumar | marios, on fs039, I left a comment on config patch also | 08:33 |
| *** apetrich has joined #oooq | 08:40 | |
| marios | chandankumar: ack thanks checking | 08:41 |
| *** jpena|off is now known as jpena | 08:55 | |
| *** jmasud has quit IRC | 08:55 | |
| *** jmasud has joined #oooq | 08:56 | |
| *** dpawlik has quit IRC | 09:03 | |
| *** dpawlik has joined #oooq | 09:03 | |
| panda | zbr: so to continue to test with centos7 we need to specify the distro ? | 09:04 |
| panda | marios: after we merged everything we found a bug (my fault). This is the fix https://review.rdoproject.org/r/25938 | 09:04 |
| panda | to not make the mistake again I need to change the staging env to include a lot of noise hashes in the mix, and promote more than once. Not a zero amount of work, but I'm working on it | 09:05 |
| marios | panda: ack thanks weshay|ruck updated about that yesterday will check the fix | 09:07 |
| *** dpawlik has quit IRC | 09:08 | |
| *** ykarel is now known as ykarel|away | 09:08 | |
| arxcruz|rover | good morning guys, what do we break today in ooo ? | 09:12 |
| panda | arxcruz|rover: we are waiting for the coronavirus to make the jump from humans to machines. | 09:13 |
| arxcruz|rover | how is it in Ireland? | 09:14 |
| arxcruz|rover | here we are in quarantaine | 09:14 |
| arxcruz|rover | well, not that I have the virus | 09:14 |
| panda | arxcruz|rover: starting slowly. shools closed, st. patrick's canceled. Ready for the hit. | 09:14 |
| arxcruz|rover | although in my build there is a couple confirmed | 09:14 |
| arxcruz|rover | OMG! St. Patrick's canceled?!?!?! | 09:14 |
| zbr | brb, going out to hunt for some eggs. yesterdays it was too late. | 09:14 |
| arxcruz|rover | zbr: been there, done that | 09:15 |
| chandankumar | arxcruz|rover, breakage is coming fyi https://review.rdoproject.org/r/#/c/24462/ and https://review.rdoproject.org/r/#/c/25958/ just landed, may be get caught in neutron component pipeline | 09:15 |
| zbr | panda: tap directly to guiness factory and your should be safe | 09:15 |
| arxcruz|rover | chandankumar: do we have someone to blame? | 09:16 |
| arxcruz|rover | :D | 09:16 |
| chandankumar | arxcruz|rover, not yet, we tested in our gate before merging, results looks good but still | 09:16 |
| arxcruz|rover | marios: we were talking about quota: https://review.opendev.org/#/c/710083/ didn't merge yet :/ | 09:28 |
| arxcruz|rover | I would say to chandankumar +w that :P | 09:28 |
| marios | arxcruz|rover: yeah that can merge but isn't needed for the fix see https://bugs.launchpad.net/tripleo/+bug/1864646/comments/11 | 09:30 |
| openstack | Launchpad bug 1864646 in tripleo "periodic standalone full-tempest-master failing neutron_tempest_plugin.scenario.test_security_groups" [Critical,Fix released] | 09:30 |
| marios | arxcruz|rover: cos they fixed in the plugin itself so we didn't need to override | 09:30 |
| arxcruz|rover | marios: ack | 09:30 |
| arxcruz|rover | apetrich: hey, any update on that mistral bug? https://bugs.launchpad.net/tripleo/+bug/1867744 | 09:31 |
| openstack | Launchpad bug 1867744 in tripleo "Mistral failed command manually execute the following script: /var/lib/mistral/overcloud/ansible-playbook-command.sh" [Critical,Triaged] | 09:31 |
| *** derekh has joined #oooq | 09:33 | |
| apetrich | arxcruz|rover, nope. I'm still on the other one the db-sync | 09:33 |
| *** jbadiapa has joined #oooq | 09:36 | |
| *** jfrancoa has joined #oooq | 09:40 | |
| chandankumar | marios, https://review.opendev.org/#/c/713397/ is good to go | 09:43 |
| marios | chandankumar: ack thx | 09:49 |
| chandankumar | marios, thanks! | 09:50 |
| *** jpena has quit IRC | 10:04 | |
| *** jpena has joined #oooq | 10:05 | |
| *** saneax has joined #oooq | 10:10 | |
| *** danpawlik has joined #oooq | 10:11 | |
| *** danpawlik is now known as dpawlik | 10:12 | |
| *** sshnaidm has joined #oooq | 10:20 | |
| zbr | panda: can you be more specific? | 10:20 |
| marios | chandankumar: green periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-baremetal-master | 10:26 |
| chandankumar | marios, cool | 10:26 |
| chandankumar | jpena, Hello | 10:26 |
| chandankumar | jpena, We can bake current cirros image in rdo c8 nodepool image | 10:26 |
| chandankumar | jpena, how to do that? | 10:27 |
| chandankumar | s/can/want to | 10:27 |
| panda | zbr: nevermind, review this please https://review.rdoproject.org/r/25975 | 10:27 |
| zbr | panda: i will merge it like this but as a note for everyone: try to use parenting more to avoid repetition on jobs, this kind of fix should have touched one line. | 10:29 |
| zbr | shortly: use parenting to group related/similar jobs | 10:30 |
| panda | zbr: any improvement suggestion welcome | 10:30 |
| panda | zbr: well they are not particularly similar ... | 10:30 |
| panda | zbr: also, question about rdo-tox-molecule | 10:30 |
| panda | zbr: has it switched to centos-8 too we need to run some tests on centos-7 still, until we move promoter to centos8 | 10:31 |
| panda | zbr: which will happen next sprint | 10:31 |
| zbr | yep, all tox based jobs are using centos-8 now, unless you manually override the nodeset. | 10:31 |
| zbr | yesterday i realised a design mistake of creating a delegated job that did not had platform in its name. | 10:32 |
| panda | zbr: ok, we need rdo-tox-molecule to run on centos-7 .... | 10:32 |
| zbr | not really, the centos-8 we have is one with docker. | 10:33 |
| zbr | if we need centos-7 in particular, we need to create a new job for that, with platform in its name. | 10:33 |
| panda | zbr: so, what's in rdo-tox-molecule at this time, and what's in delegated-centos-8 ? | 10:33 |
| zbr | in fact that rdo-tox-molecule job run for a very long time using fedora. | 10:34 |
| panda | zbr: yes, but it was running using centos7 containers | 10:34 |
| zbr | rdo-tox-molecule implies running on a machine with docker which is a redhat one (which one is not part of its contract) | 10:35 |
| panda | zbr: so if you moved promoter provisioning in delegated centos-8, did ou also modify the provisioning playbook to run on centos8 ? | 10:35 |
| panda | zbr: if not, taht is good and bad news at the same time | 10:35 |
| zbr | i did not touch any molecule job definitions, lets investigate. clearly the only one affected are delegated ones because docker ones will continue to test the same containers. | 10:36 |
| panda | zbr: good nes because we know the provisioning works on centos8, bad because we are missing cenots7 coverage now | 10:36 |
| zbr | easy to fix, lets do it. | 10:36 |
| zbr | tbh, i was expecting lots of issues with rdo move last evening, i was surprised it went so well | 10:37 |
| panda | zbr: it didn't, all promoter jobs are broken. | 10:37 |
| panda | zbr: well, it didn't for me at least | 10:37 |
| panda | zbr: come over here so I can strangle you. | 10:37 |
| zbr | in fact this is one of the reason why I try to avoid delegated as much as possible, as I do not want to change the outcome based on what machine you run tox -e molecule from. | 10:38 |
| zbr | panda: i would but company/goverment policy forbits me from traveling | 10:38 |
| panda | zbr: inevitable in this case. | 10:38 |
| *** holser has joined #oooq | 10:39 | |
| panda | zbr: this is a critical business trip, you'll get approved. | 10:39 |
| zbr | lessen learn: when platfrom is critical, include it in job name. | 10:39 |
| panda | zbr: when you have only one platform, identifying it is not on top of your mind | 10:39 |
| panda | zbr: anda there's a lot of legacy here. | 10:40 |
| jpena | chandankumar: isn't the cirros image already there? Or do we want to include a newer version? | 10:40 |
| panda | zbr: anyway, All the broken jobs have -centos-7 in their names | 10:40 |
| chandankumar | jpena, I need to check that | 10:40 |
| zbr | panda: join "zbr" on meet if you can, am already in. | 10:41 |
| chandankumar | jpena, https://softwarefactory-project.io/nodepool-log/upstream-centos-8-0000008135.log | 10:41 |
| chandankumar | jpena, cirros-0.3.4-x86_64-disk.img very old one | 10:42 |
| chandankumar | jpena, we need new one http://images.rdoproject.org/cirros/cirros-0.4.0-x86_64-disk.img | 10:42 |
| jpena | chandankumar: ack. So we need to find where is this defined, let me give it a try | 10:43 |
| panda | zbr: link ? | 10:46 |
| *** holser__ has joined #oooq | 10:46 | |
| zbr | you can join with name, user "zbr" instead of id, it will work. | 10:47 |
| zbr | if you are logged in to redhat, if not https://meet.google.com/uux-euzq-cbe | 10:47 |
| *** holser has quit IRC | 10:48 | |
| jpena | chandankumar: aha, got it. So the cirros versions that are cached are defined in https://opendev.org/openstack/project-config/src/branch/master/nodepool/elements/cache-devstack/source-repository-images | 10:48 |
| jpena | chandankumar: that includes cirros 0.3.4, 0.3.5 and 0.4.0. Actually, if you search the nodepool build log for "cirros-0.4.0", you will see it is cached already | 10:48 |
| jpena | it's cached at /opt/cache/files/cirros-0.4.0-x86_64-disk.img | 10:49 |
| chandankumar | jpena, ok, thanks, I wil fix the path in toci | 10:51 |
| *** sshnaidm has quit IRC | 10:54 | |
| *** sshnaidm has joined #oooq | 10:55 | |
| *** jaosorior has quit IRC | 11:06 | |
| *** tesseract-RH has joined #oooq | 11:16 | |
| *** tesseract has quit IRC | 11:18 | |
| *** jaosorior has joined #oooq | 11:20 | |
| panda | zbr: https://review.rdoproject.org/r/2597 | 11:24 |
| panda | zbr: https://review.rdoproject.org/r/25975 | 11:25 |
| *** rlandy has joined #oooq | 11:31 | |
| zbr | panda: there is a feature i used on molecule test itself where I fail the test run when number of run tests does not match a specific number. | 11:35 |
| zbr | the number being defined as env var in each job definition, prevented accidental skips of tests. | 11:36 |
| zbr | sagi did not like it too much but I got 4-5 contributions since where user correctly refreshed the test numbers without ever asking on what is the problem, i guess the error message was clear. | 11:37 |
| zbr | it adds a bit more work when tests are added but eliminates the risk of accidental skips. | 11:38 |
| zbr | as we do not add/remove/disable dests very often, it may work well. | 11:38 |
| rlandy | chandankumar: hey ... https://review.rdoproject.org/r/#/c/24462/ merged :) | 11:40 |
| rlandy | what's the ETA then on getting the new neutron code? | 11:41 |
| chandankumar | rlandy, it will be available now i think | 11:42 |
| rlandy | chandankumar: ok - to close out https://trello.com/c/CAgSZOvD/1392-cixlp1867599tripleociproa-overcloud-deploy-failing-on-fs030-and-fs016-while-pulling-mariadb-container-from-undercloud-registry? | 11:43 |
| chandankumar | rlandy, yes | 11:43 |
| rlandy | chandankumar: one more ... https://bugs.launchpad.net/tripleo/+bug/1867602 - marked incomplete - closing out the CIX card - we can relog if necessary | 11:44 |
| openstack | Launchpad bug 1867602 in tripleo "overcloud deploy failed due to Systemd start for pcsd failed" [Critical,Incomplete] | 11:44 |
| chandankumar | rlandy, we are not able to reproduce it | 11:45 |
| rlandy | chandankumar: ack - I see .. just cleaning up the CIx stuff | 11:45 |
| rlandy | sshnaidm: hi - anything new on the node cleaning issue? https://bugs.launchpad.net/tripleo/+bug/1866204 - just updating CIX card status | 11:48 |
| openstack | Launchpad bug 1866204 in tripleo "OVB is stuck on providing nodes" [Critical,In progress] - Assigned to Luke Short (ekultails) | 11:48 |
| sshnaidm | rlandy, hey, I think DF was on this? | 11:48 |
| rlandy | sshnaidm: ack - it's not assigned to us - just checking as we are also on the card | 11:49 |
| rlandy | not a problem - DF can report | 11:49 |
| chandankumar | rlandy, stepping out for a bit, back in 30 misn | 11:51 |
| rlandy | chandankumar: sure | 11:51 |
| *** holser__ has quit IRC | 11:54 | |
| *** holser has joined #oooq | 11:58 | |
| weshay|ruck | arxcruz|rover, 0/ | 12:00 |
| *** holser has quit IRC | 12:05 | |
| *** holser has joined #oooq | 12:12 | |
| arxcruz|rover | weshay|ruck: o/ | 12:14 |
| arxcruz|rover | weshay|ruck: on the meeting | 12:14 |
| *** Goneri has joined #oooq | 12:16 | |
| *** holser has quit IRC | 12:17 | |
| *** jbadiapa has quit IRC | 12:19 | |
| *** holser has joined #oooq | 12:19 | |
| *** saneax has quit IRC | 12:25 | |
| chandankumar | rlandy, left a comment here https://code.engineering.redhat.com/gerrit/#/c/194604/ | 12:26 |
| rlandy | https://bugs.launchpad.net/tripleo/+bug/1867807 - this needs to go to multinode as well | 12:29 |
| openstack | Launchpad bug 1867807 in tripleo "periodic- centos-8-ovb-3ctlr_1comp-featureset001-baremetal-master fails to download tempest cirros image" [Critical,Triaged] | 12:29 |
| chandankumar | rlandy, all taken care in weshay|ruck patch | 12:29 |
| rlandy | marios left a comment | 12:30 |
| weshay|ruck | chandankumar arxcruz|rover we need a little cleanup re: cirros config | 12:30 |
| marios | rlandy: o/ thanks, which bit? | 12:30 |
| weshay|ruck | I see 3.x defined in a several places | 12:30 |
| marios | brb | 12:30 |
| rlandy | https://bugs.launchpad.net/tripleo/+bug/1867807/comments/3 | 12:30 |
| openstack | Launchpad bug 1867807 in tripleo "periodic- centos-8-ovb-3ctlr_1comp-featureset001-baremetal-master fails to download tempest cirros image" [Critical,Triaged] | 12:30 |
| chandankumar | weshay|ruck, need to replace with 4.x version | 12:30 |
| weshay|ruck | chandankumar, do we have a review up to create a new repo for the skip list? | 12:30 |
| chandankumar | weshay|ruck, nope, I will get it done by today | 12:31 |
| weshay|ruck | thank you | 12:31 |
| rlandy | that override needs to go into multinode | 12:31 |
| rlandy | not just OVB | 12:31 |
| marios | rlandy: ack i thought you were telling me you commented somewhere k | 12:31 |
| rlandy | no - just that you added a valuable comment | 12:31 |
| chandankumar | rlandy, , do we keep these files https://review.opendev.org/#/c/713505/ | 12:31 |
| chandankumar | rlandy, sorry | 12:32 |
| *** ratailor has quit IRC | 12:32 | |
| chandankumar | rlandy, https://review.opendev.org/#/c/713505/ included multinode also | 12:32 |
| rlandy | chandankumar: ack - still in gates - I see now - not merged yet | 12:32 |
| chandankumar | rlandy, time to fix standalone also | 12:32 |
| rlandy | bm might hit it will see | 12:32 |
| rlandy | chandankumar: didn;t show up in standalone | 12:33 |
| chandankumar | rlandy, https://logserver.rdoproject.org/openstack-component-cloudops/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario002-standalone-cloudops-master/21e4ed0/job-output.txt | 12:33 |
| rlandy | ugh - now there too | 12:33 |
| rlandy | oh - go ahead | 12:33 |
| chandankumar | let it merged | 12:34 |
| arxcruz|rover | chandankumar: I see something saying we should move to 5.1 that fix some network issues | 12:37 |
| chandankumar | arxcruz|rover, it is in still in testing phase in devstack | 12:37 |
| arxcruz|rover | chandankumar: regarding the repo, sorry, my bad, completely forgot to ping you on monday :/ | 12:37 |
| chandankumar | rlandy, arxcruz|rover https://review.opendev.org/713642 | 12:39 |
| *** jpena is now known as jpena|lunch | 12:39 | |
| chandankumar | arxcruz|rover, you want to work on that? | 12:39 |
| arxcruz|rover | chandankumar: nah, it's fine, just saying I forgot, sorry for that :) | 12:42 |
| chandankumar | arxcruz|rover, no problem, you are rucking:-) | 12:43 |
| arxcruz|rover | still :) | 12:43 |
| arxcruz|rover | anyway | 12:43 |
| *** udesale_ has joined #oooq | 12:44 | |
| *** jaosorior has quit IRC | 12:45 | |
| chandankumar | arxcruz|rover, can you take care of cirros mess | 12:45 |
| chandankumar | arxcruz|rover, we have images available in nodepool cache also | 12:45 |
| chandankumar | arxcruz|rover, https://opendev.org/openstack/project-config/src/branch/master/nodepool/elements/cache-devstack/source-repository-images | 12:45 |
| chandankumar | arxcruz|rover, https://softwarefactory-project.io/nodepool-log/upstream-centos-8-0000008135.log | 12:46 |
| *** saneax has joined #oooq | 12:46 | |
| arxcruz|rover | chandankumar: the image we are using doesn't have it | 12:46 |
| *** udesale has quit IRC | 12:47 | |
| chandankumar | arxcruz|rover, check the nodepool log | 12:47 |
| arxcruz|rover | chandankumar: so, why not use that ? | 12:47 |
| arxcruz|rover | set image_path to /opt/cache/cirros-4.xxxx | 12:47 |
| chandankumar | arxcruz|rover, i think we moved to ostempest defaults and forgot that | 12:47 |
| arxcruz|rover | chandankumar: this is python-tempestconf, not os_tempest | 12:47 |
| rlandy | arxcruz|rover: https://bugs.launchpad.net/tripleo/+bug/1864652 still an issue? | 12:48 |
| openstack | Launchpad bug 1864652 in tripleo "periodic standalone full-tempest train failing tempest.api.volume.test_volumes_snapshots.VolumesSnapshotTestJSON" [High,In progress] | 12:48 |
| arxcruz|rover | rlandy: it's on skiplist afaik, we can unskip to test | 12:48 |
| arxcruz|rover | giulio patch was merged on tempest | 12:49 |
| *** amoralej is now known as amoralej|lunch | 12:53 | |
| *** jmasud has quit IRC | 12:55 | |
| *** jmasud has joined #oooq | 12:56 | |
| rlandy | periodic-tripleo-ci-centos-8-scenario010-standalone-network-master failure :( | 12:57 |
| chandankumar | rlandy, is it recent? | 12:57 |
| *** holser has quit IRC | 12:57 | |
| rlandy | chandankumar: just failed - investigating | 12:57 |
| rlandy | maybe rerun | 12:57 |
| chandankumar | rlandy, and is it python3-octavai temepst plugin issue | 12:57 |
| rlandy | https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/home/zuul/ | 12:58 |
| rlandy | chandankumar: ^^ | 12:58 |
| rlandy | 2020-03-18 12:31:02 | tripleo_common.image.exception.ImageUploaderException: Modifying image 192.168.24.1:8787/tripleomaster/centos-binary-octavia-api:46b71a6620e3372c998db7a694112fd2 failed | 12:58 |
| rlandy | chandankumar: ^^ let's talk at sync meeting | 12:58 |
| rlandy | until the next successful transaction.\n You can remove cached packages by executing \'dnf clean packages\'.\n stdout_lines: <omitted>\n\x1b[Ke30=\x1b[4D\x1b[K' | 12:59 |
| rlandy | chandankumar: panda: daily centos8 sync | 13:01 |
| chandankumar | rror: Transaction check error:\n file /etc/neutron/plugins/networking-ovn/networking-ovn.ini from install of openstack-neutron-common-1:16.0.0-0.20200318094753.8c7ca77.el8.noarch conflicts with file from package python3-networking-ovn-7.0.0-0.20200109131747.eda5d7f.el8.noarch\n \n level=debug msg="error running [bash -x /tmp/yum_update.sh delorean-current,quickstart-centos-ceph-nautilus,network ] in container \\"centos-binary-octavia-api-work | 13:01 |
| chandankumar | ing-container\\": error while running runtime: exit status 1"\n stderr_lines: | 13:01 |
| rlandy | marios: https://review.rdoproject.org/r/#/c/25919/ | 13:02 |
| chandankumar | amoralej|lunch, https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 13:02 |
| chandankumar | Error: Transaction check error:\n file /etc/neutron/plugins/networking-ovn/networking-ovn.ini from install of openstack-neutron-common-1:16.0.0-0.20200318094753.8c7ca77.el8.noarch conflicts with file from package python3-networking-ovn-7.0.0-0.20200109131747.eda5d7f.el8.noarch\n \n level=debug msg="error running [bash -x /tmp/yum_update.sh delorean-current,quickstart-centos-ceph-nautilus,network ] in container \\"centos-binary-octavia-api-wor | 13:02 |
| chandankumar | king-container\\": error while running runtime: exit status 1"\n stderr_lines: <o | 13:02 |
| marios | * https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-tempest-master/b45b0e3/job-output.txt | 13:04 |
| marios | * 2020-03-17 20:07:22.590829 | primary | TASK [os_tempest : Execute tempest tests] ************************************** | 13:04 |
| marios | 2020-03-17 20:07:22.667818 | primary | Tuesday 17 March 2020 20:07:22 +0000 (0:00:00.286) 1:42:28.284 ********* | 13:04 |
| marios | 2020-03-17 22:09:48.770486 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master] | 13:04 |
| marios | * https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-tempest-master/b45b0e3/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 13:04 |
| *** tesseract has joined #oooq | 13:05 | |
| *** saneax has quit IRC | 13:07 | |
| *** tesseract-RH has quit IRC | 13:08 | |
| marios | https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-security-master/9bafb22/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 13:08 |
| *** holser has joined #oooq | 13:08 | |
| marios | * https://logserver.rdoproject.org/openstack-component-cinder/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario004-standalone-cinder-master/1467efe/logs/undercloud/var/log/tempest/stestr_results.html.gz | 13:09 |
| marios | chandankumar: ^ | 13:10 |
| marios | chandankumar: its come up before for that test * test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke] | 13:10 |
| marios | chandankumar: (mentioned there https://hackmd.io/7MBqFHurTA2e5H8kYRwgag?both#components "failing network-basic-ops") | 13:11 |
| chandankumar | rror: Transaction check error:\n file /etc/neutron/plugins/networking-ovn/networking-ovn.ini from install of openstack-neutron-common-1:16.0.0-0.20200318094753.8c7ca77.el8.noarch conflicts with file from package python3-networking-ovn-7.0.0-0.20200109131747.eda5d7f.el8.noarch\n \n level=debug msg="error running [bash -x /tmp/yum_update.sh delorean-current,quickstart-centos-ceph-nautilus,network ] in container \\"centos-binary-octavia-api-work | 13:11 |
| chandankumar | ing-container\\": error while running runtime: exit status 1"\n stderr_lines: | 13:11 |
| chandankumar | rlandy, marios ^^ | 13:11 |
| rlandy | https://review.rdoproject.org/r/#/c/25919/ | 13:13 |
| *** holser has quit IRC | 13:13 | |
| *** skramaja has quit IRC | 13:14 | |
| rlandy | https://bugs.launchpad.net/tripleo/+bug/1867664 | 13:15 |
| openstack | Launchpad bug 1867664 in tripleo "Master periodic jobs are failing overcloud deploy with ''Container(s) with bad ExitCode: [''container-puppet-neutron''], check logs in /var/log/containers/stdouts/'" [Critical,Triaged] | 13:15 |
| rlandy | marios: ^^ | 13:15 |
| *** ccamacho has quit IRC | 13:16 | |
| zbr | chandankumar: low-hanging https://review.rdoproject.org/r/#/c/24108/ - thanks! | 13:17 |
| marios | rlandy: thx | 13:18 |
| marios | * https://review.rdoproject.org/r/#/c/25938/ | 13:19 |
| *** matbu has quit IRC | 13:22 | |
| marios | rlandy: have link to the prod chain meet ? it disapeared from my calendar at some point weshay|ruck did you remove the team (you added us soem time ago) | 13:24 |
| marios | rlandy: maybe pvt ;) | 13:24 |
| *** jbadiapa has joined #oooq | 13:24 | |
| weshay|ruck | marios, added u | 13:25 |
| marios | weshay|ruck: thx | 13:25 |
| rlandy | marios: https://bluejeans.com/1196265346 | 13:30 |
| marios | rlandy: thank you | 13:30 |
| *** TrevorV has joined #oooq | 13:36 | |
| chandankumar | weshay|ruck, https://bugs.launchpad.net/tripleo/+bug/1867744 | 13:41 |
| openstack | Launchpad bug 1867744 in tripleo "Mistral failed command manually execute the following script: /var/lib/mistral/overcloud/ansible-playbook-command.sh" [Critical,Triaged] | 13:41 |
| *** jpena|lunch is now known as jpena | 13:42 | |
| chandankumar | arxcruz|rover, Hello | 13:42 |
| chandankumar | arxcruz|rover, https://logserver.rdoproject.org/19/25919/4/check/distgit-tripleo-master-testing-centos-8-standalone-scenario003/9148e10/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 13:42 |
| chandankumar | arxcruz|rover, usage: mistral-db-manage [-h] [--config-dir DIR] [--config-file PATH] known? | 13:42 |
| *** amoralej|lunch is now known as amoralej | 13:46 | |
| amoralej | chandankumar, that's in periodic job, right? | 13:46 |
| amoralej | only periodic? | 13:47 |
| chandankumar | amoralej, it is in component pipeline after merging neutron unpinning | 13:47 |
| amoralej | let me check what's happening | 13:47 |
| amoralej | it's rebuilding containers? | 13:47 |
| amoralej | or update on old containers? | 13:47 |
| chandankumar | amoralej, nope, just updating the containers using latest yum repos | 13:47 |
| chandankumar | from changed components | 13:47 |
| amoralej | chandankumar, for my education, you don't build new containers in component promotions? | 13:48 |
| amoralej | just update last promoted? | 13:48 |
| chandankumar | amoralej, yes, we donot build containers, we just use tripleo image prepare and mount the component repo in containers | 13:48 |
| arxcruz|rover | chandankumar: https://bugs.launchpad.net/tripleo/+bug/1866621 | 13:49 |
| openstack | Launchpad bug 1866621 in tripleo "Can't run container mistral_db_sync" [High,Triaged] | 13:49 |
| amoralej | so, new containers are only created in integration pipeline? | 13:49 |
| chandankumar | amoralej, yes | 13:49 |
| amoralej | ok | 13:49 |
| amoralej | let me understand what's the problem | 13:49 |
| arxcruz|rover | chandankumar: afaik apetrich is looking at it | 13:49 |
| chandankumar | arxcruz|rover, aye thanks! | 13:49 |
| apetrich | aye | 13:50 |
| amoralej | python3-networking-ovn should be updated | 13:50 |
| apetrich | arxcruz|rover, ykarel|away I need a check up on priority I'm working on https://bugs.launchpad.net/tripleo/+bug/1866621 and https://bugs.launchpad.net/tripleo/+bug/1867744 I'm more invested on the first one is the second one more urgent? | 13:51 |
| openstack | Launchpad bug 1866621 in tripleo "Can't run container mistral_db_sync" [High,Triaged] | 13:51 |
| openstack | Launchpad bug 1867744 in tripleo "Mistral failed command manually execute the following script: /var/lib/mistral/overcloud/ansible-playbook-command.sh" [Critical,Triaged] | 13:51 |
| chandankumar | apetrich, we need db sync fix for unpinning mistral | 13:52 |
| apetrich | chandankumar, thanks That is what I understood | 13:52 |
| apetrich | just wanted to make sure I'm using my time the best | 13:52 |
| arxcruz|rover | apetrich: yup, thanks :) | 13:52 |
| rlandy | chandankumar: amoralej: we can adjust pipeline criteria if the neutron changes have to hit the integration pipeline to build containers | 14:02 |
| amoralej | i just expected containers rebuild was the first thing in component promotions too | 14:04 |
| amoralej | but anyway, upgrades should work | 14:05 |
| amoralej | i'm starting rdo meeting, i'll be back | 14:05 |
| rlandy | amoralej: chandankumar: https://bugs.launchpad.net/tripleo/+bug/1867931 - for tracking | 14:06 |
| openstack | Launchpad bug 1867931 in tripleo "periodic-tripleo-ci-centos-8-scenario010-standalone-network-master deploy fails with package conflict - "openstack-neutron-common-1:16.0.0-0.20200318094753.8c7ca77.el8.noarch conflicts"" [Critical,Triaged] | 14:06 |
| rlandy | amoralej: we made a decision just to update containers in the pipeline | 14:06 |
| rlandy | we may need to reopen the containers build discussion in the component pipeline | 14:06 |
| rlandy | if we need to build, we'll just ignore criteria and promote | 14:07 |
| rlandy | didn't mark this as promotion-blocker yet | 14:07 |
| rlandy | chandankumar: ^^ | 14:07 |
| chandankumar | rlandy, ack | 14:08 |
| amoralej | my point is that, if there is an issue in some new package that makes container images build to fail, we could catch it in promotion pipeline | 14:10 |
| amoralej | but anyway, i can understand your criteria too | 14:11 |
| *** saneax has joined #oooq | 14:16 | |
| *** ccamacho has joined #oooq | 14:21 | |
| *** holser has joined #oooq | 14:22 | |
| rlandy | periodic-tripleo-ci-centos-8-scenario010-standalone-network-master only failure on network | 14:25 |
| *** holser has quit IRC | 14:26 | |
| *** holser has joined #oooq | 14:28 | |
| chandankumar | weshay|ruck, arxcruz|rover one question related to new repo openstack-tempest-skiplist https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/validate-tempest/vars do I copy all skip list with releazes with git history? | 14:34 |
| weshay|ruck | chandankumar, meh | 14:35 |
| weshay|ruck | ideally yes | 14:35 |
| chandankumar | weshay|ruck, ok, repo coming soon | 14:36 |
| weshay|ruck | I've done it before .. it's possible.. but imho not a requiement for the skip list | 14:36 |
| weshay|ruck | chandankumar, because every skip is documented w/ a bug | 14:36 |
| chandankumar | weshay|ruck, arxcruz|rover I will get the detailed proposal also up for review tomorrow | 14:37 |
| chandankumar | weshay|ruck, https://review.rdoproject.org/r/#/c/25976/ was an interesting finding for tripleo today | 14:37 |
| amoralej | rlandy, chandankumar it'w weird i can't reproduce the issue locally | 14:37 |
| chandankumar | amoralej, rlandy I think we can waive the criteria and let the container build and see in integration pipeline? | 14:38 |
| chandankumar | as a temp measure | 14:38 |
| rlandy | amoralej: will try rekick in testproject | 14:38 |
| amoralej | mmm | 14:39 |
| rlandy | chandankumar: ^^ if that fails - yes | 14:39 |
| chandankumar | rlandy, aye | 14:39 |
| amoralej | wait, i was checking in an ovn container | 14:39 |
| amoralej | only fails in octavia-api one | 14:39 |
| amoralej | right? | 14:39 |
| chandankumar | amoralej, yes | 14:39 |
| rlandy | only scenario10 | 14:39 |
| arxcruz|rover | chandankumar: initially yes, then we can figure out what to do | 14:39 |
| arxcruz|rover | i have an idea to have a tool to detect everything | 14:40 |
| marios | cgoncalves: o/ i had a poke at the standalone-upgrade fail you pointed at... also happens same in the centos8 job too see https://3f2282f1d55b55d11f4f-1fe32fbf22fd99d95a9dcd86fcc536bb.ssl.cf2.rackcdn.com/710473/9/check/tripleo-ci-centos-8-scenario013-standalone-upgrade/3acda3c/job-output.txt ... i am not sure why it fails yet, i wonder if it is because https://review.opendev.org/#/c/713108/ didn't merge yet. | 14:40 |
| marios | weshay|ruck: chandankumar: rlandy: sshnaidm: any idea if build-test-packages fails if the change isn't merge yet? | 14:40 |
| rlandy | link? | 14:40 |
| marios | cgoncalves: having issue with failing like 2020-03-18 12:46:25.929560 | primary | 2020-03-18 12:46:22,154 ERROR:dlrn-build:Build failed. See logs at: /home/zuul/DLRN/data/repos/component/tripleo/3a/da/3ada5f24d0706aa57a45e19f7694ed10c457f9ac_dev/ | 14:40 |
| chandankumar | marios, nope, unless it is a file mention in spec file | 14:40 |
| marios | rlandy: https://3f2282f1d55b55d11f4f-1fe32fbf22fd99d95a9dcd86fcc536bb.ssl.cf2.rackcdn.com/710473/9/check/tripleo-ci-centos-8-scenario013-standalone-upgrade/3acda3c/job-output.txt | 14:41 |
| chandankumar | marios, on c8 jobs somehow py27 contents are coming I am not sure how | 14:41 |
| marios | rlandy: i can see it is trying that "dlrn --config-file projects.ini --head-only --package-name openstack-tripleo-heat-templates --local --info-repo rdoinfo --dev | 14:41 |
| chandankumar | of tht | 14:41 |
| marios | rlandy: and that commit https://github.com/openstack/tripleo-heat-templates/commit/3ada5f24d0706aa57a45e19f7694ed10c457f9ac it is trying to use isn't merged yet (https://review.opendev.org/#/c/713108/ ) | 14:41 |
| *** ysandeep is now known as ysandeep|afk | 14:42 | |
| marios | i think first thing we should do is add /home/zuul/DLRN/ to our collect logs... we don't have that ". See logs at: /home/zuul/DLRN/data/repos/component/tripleo/3a/da/3ada5f24d0706aa57a45e19f7694ed10c457f9ac_dev/ | 14:42 |
| weshay|ruck | marios, it's usually only building unmerged changes | 14:42 |
| weshay|ruck | marios, it's there | 14:42 |
| sshnaidm | marios, https://3f2282f1d55b55d11f4f-1fe32fbf22fd99d95a9dcd86fcc536bb.ssl.cf2.rackcdn.com/710473/9/check/tripleo-ci-centos-8-scenario013-standalone-upgrade/3acda3c/logs/delorean_logs/component/tripleo/3a/da/3ada5f24d0706aa57a45e19f7694ed10c457f9ac_dev/ | 14:42 |
| weshay|ruck | marios, in the /logs | 14:42 |
| marios | weshay|ruck: oh cool, i was looking for DLRN | 14:43 |
| weshay|ruck | sshnaidm++ | 14:43 |
| marios | cgoncalves: so looks like the error is in there https://eb1a256a30bf341096bb-948f82c67d72369e4a57291a7615421b.ssl.cf1.rackcdn.com/710473/9/check/tripleo-ci-centos-7-scenario013-standalone-upgrade/3bae72a/logs/delorean_logs/3a/da/3ada5f24d0706aa57a45e19f7694ed10c457f9ac_dev/build.log.txt.gz | 14:44 |
| marios | error: Installed (but unpackaged) file(s) found: | 14:44 |
| sshnaidm | marios, a lot of cache and *.pyc files | 14:45 |
| marios | chandankumar: and its weird that is py2.7... we should not have that on centos8 | 14:45 |
| cgoncalves | marios, right. I pointed that out in Gerrit before | 14:45 |
| chandankumar | marios, yes, that is what I am figuring out | 14:45 |
| rlandy | scenario010 network rekicked in testproject | 14:45 |
| marios | chandankumar: oh hold on that one i pointed to there was centos7 let me check 8 one | 14:45 |
| chandankumar | marios, I was checking c8 one | 14:46 |
| amoralej | chandankumar, rlandy yum update works fine in my env, i think there is something in that role that is done differently | 14:46 |
| amoralej | probably the list of packages | 14:46 |
| marios | chandankumar: yeah same error | 14:46 |
| amoralej | new packages must be filtered out or something | 14:46 |
| amoralej | could i see the component repo config and the yum_update.sh script? | 14:47 |
| rlandy | amoralej: let's see if it reproduces in the testproject run | 14:47 |
| marios | chandankumar: well actually not exactly same... same error but pointing to py3.6 like error: Installed (but unpackaged) file(s) found: /usr/lib/python3.6/site-packages/tripleo-heat-templates/__pycache__/setup.cpython-36.opt-1.pyc | 14:47 |
| marios | brb needs coffee | 14:48 |
| rlandy | https://review.rdoproject.org/zuul/stream/6e7d28a32d2142f5a5c8df2378a1eb94?logfile=console.log | 14:48 |
| chandankumar | marios, give me some time I need to reproduce locally via dlrn | 14:48 |
| arxcruz|rover | weshay|ruck: periodic-tripleo-rhel-8-train-containers-build-push-vexxhost failing now with a different error | 14:48 |
| *** dtantsur is now known as dtantsur|brb | 14:48 | |
| cgoncalves | thanks for the help, folks! much appreciated | 14:48 |
| weshay|ruck | arxcruz|rover, k.. you know how to debug those jobs? | 14:48 |
| arxcruz|rover | weshay|ruck: not really but i'm trying | 14:49 |
| rlandy | amoralej: search for /tmp/yum_update.sh in https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/home/zuul/ansible.log.txt.gz | 14:49 |
| weshay|ruck | arxcruz|rover, start w/ https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-train-containers-build-push-vexxhost/76e5fc1/logs/containers-failed-to-build.log | 14:49 |
| arxcruz|rover | weshay|ruck: error is /tmp/kolla-w8cswvgq/docker/collectd/collectd-build.log:Error: No matching repo to modify: epel-modular, epel. | 14:50 |
| weshay|ruck | collectd | 14:50 |
| weshay|ruck | then go to https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-train-containers-build-push-vexxhost/76e5fc1/logs/buildah-builds/ | 14:50 |
| weshay|ruck | https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-train-containers-build-push-vexxhost/76e5fc1/logs/buildah-builds/kolla-w8cswvgq/ | 14:50 |
| weshay|ruck | has the actual build for each container | 14:50 |
| weshay|ruck | arxcruz|rover, same issue | 14:50 |
| rlandy | https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/etc/yum.repos.d/network-component.repo.txt.gz component itslef | 14:50 |
| weshay|ruck | arxcruz|rover, https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-train-containers-build-push-vexxhost/76e5fc1/logs/buildah-builds/kolla-w8cswvgq/docker/collectd/collectd-build.log | 14:50 |
| chandankumar | marios, weshay|ruck arxcruz|rover panda sshnaidm rlandy http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013382.html review.opendev.org Downtime March 20, 2020 at 1400UTC | 14:51 |
| weshay|ruck | chandankumar++ | 14:51 |
| chandankumar | weshay|ruck, we can update tripleo channel for the same it is coming | 14:51 |
| weshay|ruck | arxcruz|rover, /me shows you where to work around this | 14:52 |
| rlandy | search of @network https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/var/log/extra/package-list-installed.txt.gz | 14:52 |
| panda | chandankumar: I have local tests to work on. | 14:52 |
| arxcruz|rover | weshay|ruck: ack | 14:52 |
| weshay|ruck | arxcruz|rover, so.. apparently we need to add a epel repo.. that is at least present.. so the disable command does NOT fail | 14:53 |
| weshay|ruck | https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/templates/Dockerfile.j2 | 14:53 |
| weshay|ruck | if we create epel in the tasks.. and copy it out to the base container in the above template | 14:53 |
| chandankumar | weshay|ruck, arxcruz|rover in epel case, make sure it is enabled in the repo file | 14:53 |
| chandankumar | on copying | 14:54 |
| weshay|ruck | arxcruz|rover, you can add a b.s. epel repo in the tasks https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/templates/Dockerfile.j2 | 14:54 |
| weshay|ruck | oops | 14:54 |
| rlandy | Installing: | 14:54 |
| rlandy | openstack-neutron-ovn-metadata-agent noarch1:16.0.0-0.20200318094753.8c7ca77.el8 network 16 k | 14:54 |
| rlandy | replacing python3-networking-ovn-metadata-agent.noarch 7.0.0-0.20200109131747.eda5d7f.el8 | 14:54 |
| rlandy | Upgrading: | 14:54 |
| rlandy | openstack-neutron noarch1:16.0.0-0.20200318094753.8c7ca77.el8 network 25 k | 14:54 |
| rlandy | replacing python3-networking-ovn.noarch 7.0.0-0.20200109131747.eda5d7f.el8 | 14:54 |
| weshay|ruck | arxcruz|rover, sorry wrong link https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/main.yaml#L72 | 14:54 |
| rlandy | openstack-neutron-common noarch1:16.0.0-0.20200318094753.8c7ca77.el8 network 186 k | 14:54 |
| rlandy | openstack-neutron-ml2 noarch1:16.0.0-0.20200318094753.8c7ca77.el8 network 15 k | 14:54 |
| rlandy | python3-neutron noarch1:16.0.0-0.20200318094753.8c7ca77.el8 network 2.5 M | 14:54 |
| rlandy | python3-vmware-nsxlib noarch15.0.7-0.20200317013238.c2f0480.el8 network 247 k | 14:54 |
| rlandy | amoralej: ^^ we should have seen updates | 14:54 |
| weshay|ruck | arxcruz|rover, the job is only failing because it's trying to find an epel.repo defined | 14:54 |
| weshay|ruck | so we can safely add an already disabled epel repo | 14:54 |
| weshay|ruck | and it will pass | 14:54 |
| weshay|ruck | arxcruz|rover, make sense? | 14:54 |
| arxcruz|rover | weshay|ruck: yup | 14:56 |
| weshay|ruck | to bad rfolco is not here, he's done this a couple times | 14:56 |
| weshay|ruck | and could assist.. but I think you get it :) | 14:56 |
| weshay|ruck | arxcruz|rover, send me a review when you have one | 14:56 |
| panda | zbr: https://review.rdoproject.org/r/25975 much better now | 15:01 |
| panda | marios: https://review.rdoproject.org/r/25938 improved commit message and added test to verify that the bug will not happen again. | 15:03 |
| panda | marios: in your queue for tomorrow please :) | 15:03 |
| marios | panda: noone like a smarty pants (ack will do) | 15:04 |
| amoralej | rlandy, yes, i think updates should work, i've double checked in octavia-api container locally | 15:04 |
| amoralej | if it fails is because some issue in the update procedure | 15:04 |
| rlandy | watching the rerun now | 15:05 |
| panda | marios: why do you hate me :( I've shown you nothing but love. If you have problem understanding the patch, tell me and I'll add more comments | 15:07 |
| marios | panda: if you call writing py unit tests 'love' then i never want to make an enemy of you | 15:10 |
| *** matbu has joined #oooq | 15:11 | |
| *** saneax has quit IRC | 15:11 | |
| panda | marios: it's for your own good. If in the future you'll have to write a new feature, all the tests will protect you, and block the patch before it reaches production. All the jobs in the patch will explode, and the you'll remember me ,and maybe thank me. THANKS PANDA! | 15:18 |
| * marios twists needle in panda dool | 15:19 | |
| marios | (doll) | 15:19 |
| panda | no please everywhere but not there! | 15:19 |
| arxcruz|rover | weshay|ruck: https://review.opendev.org/713676 if i understood correct | 15:20 |
| * panda faints. | 15:20 | |
| weshay|ruck | arxcruz|rover, the repo needs to land in https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/templates/Dockerfile.j2#L3 | 15:21 |
| weshay|ruck | that dir.. so it's copied to the container | 15:21 |
| *** holser has quit IRC | 15:22 | |
| weshay|ruck | arxcruz|rover, and you need a when: ansible_distribution and distribution_version clause | 15:23 |
| weshay|ruck | arxcruz|rover, comments on patch | 15:28 |
| arxcruz|rover | weshay|ruck: ack | 15:29 |
| weshay|ruck | arxcruz|rover, I think there is a task that copies all repos to $home/repos | 15:31 |
| arxcruz|rover | weshay|ruck: yes, checking now | 15:31 |
| arxcruz|rover | weshay|ruck: but copies only delorean.repo files | 15:31 |
| *** udesale_ has quit IRC | 15:32 | |
| arxcruz|rover | weshay|ruck: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/main.yaml#L145 | 15:32 |
| weshay|ruck | arxcruz|rover, ack.. so ya.. please account for that :) | 15:32 |
| arxcruz|rover | so, i'll rename the repo file to delorean_epel_dummy.repo | 15:32 |
| arxcruz|rover | so it will be copied | 15:32 |
| rlandy | hmmm .... rerun also failed | 15:33 |
| arxcruz|rover | weshay|ruck: is CIX now? there's nobody in the meeting | 15:33 |
| weshay|ruck | ugh | 15:33 |
| * weshay|ruck checks | 15:33 | |
| rlandy | arxcruz|rover: it was an hour ago | 15:36 |
| arxcruz|rover | seriously ? | 15:36 |
| arxcruz|rover | in my calendar is showing up to now | 15:37 |
| rlandy | arxcruz|rover: no worries - we ewre covered | 15:37 |
| rlandy | arxcruz|rover: maybe a daylight savings issue | 15:37 |
| rlandy | USA changed clocks already | 15:37 |
| weshay|ruck | arxcruz|rover, daylight savings may have screwed that | 15:37 |
| weshay|ruck | already passed | 15:37 |
| arxcruz|rover | :/ | 15:38 |
| *** holser has joined #oooq | 15:39 | |
| marios | rlandy: chandankumar: as discussed https://bugs.launchpad.net/tripleo/+bug/1867945 | 15:50 |
| openstack | Launchpad bug 1867945 in tripleo "periodic centos8 standalone-full-tempest-tempest-master timeout" [Critical,Triaged] | 15:50 |
| rlandy | Modifying image 192.168.24.1:8787/tripleomaster/centos-binary-octavia-api:46b71a6620e3372c998db7a694112fd2 failed | 15:53 |
| rlandy | sam eissue | 15:53 |
| cgoncalves | marios, any update on the dlrn issue seen in c7 octavia upgrade job? maybe there is one but I missed it in the backlog | 16:16 |
| marios | cgoncalves: hi, i had a look in the logs... don't have a solution for you yet but you can see more info on the error in delorean logs... let me fetch | 16:17 |
| cgoncalves | marios, I've seen the drln build.log file. I get it is an issue in the packaging side, or so it is suggested in the log | 16:17 |
| marios | cgoncalves: in there https://eb1a256a30bf341096bb-948f82c67d72369e4a57291a7615421b.ssl.cf1.rackcdn.com/710473/9/check/tripleo-ci-centos-7-scenario013-standalone-upgrade/3bae72a/logs/delorean_logs/3a/da/3ada5f24d0706aa57a45e19f7694ed10c457f9ac_dev/ | 16:18 |
| cgoncalves | ok, just wanted to make sure I had not missed anything since earlier today | 16:18 |
| cgoncalves | yeah, thanks | 16:18 |
| marios | cgoncalves: right not sure what the issue is exactly, like why it is complaining about those files on the second run of build-test-packages | 16:18 |
| *** holser has quit IRC | 16:24 | |
| *** sshnaidm has quit IRC | 16:28 | |
| *** sshnaidm has joined #oooq | 16:28 | |
| *** holser has joined #oooq | 16:30 | |
| *** holser has quit IRC | 16:31 | |
| *** holser has joined #oooq | 16:31 | |
| *** jmasud has quit IRC | 16:34 | |
| *** jmasud has joined #oooq | 16:34 | |
| *** holser has quit IRC | 16:38 | |
| *** holser has joined #oooq | 16:39 | |
| *** holser__ has joined #oooq | 16:43 | |
| *** holser__ has quit IRC | 16:43 | |
| *** holser__ has joined #oooq | 16:44 | |
| *** holser has quit IRC | 16:44 | |
| *** jmasud has quit IRC | 16:54 | |
| *** jmasud has joined #oooq | 16:56 | |
| marios | oooci-jobs.sh now with component promotion criteria search https://github.com/marios/tripleo_ruck_job_tool/commit/df4f3134e72044672fd82686af79a9b904004e38 | 16:56 |
| marios | i know right?! | 16:56 |
| * marios grabs his coat | 16:57 | |
| *** tesseract has quit IRC | 16:59 | |
| rlandy | weshay|ruck: 1-on-1? | 17:01 |
| *** bogdando has quit IRC | 17:02 | |
| weshay|ruck | aye | 17:05 |
| *** dsneddon_ has joined #oooq | 17:05 | |
| *** bogdando has joined #oooq | 17:07 | |
| *** dtantsur|brb is now known as dtantsur | 17:09 | |
| *** marios is now known as marios|out | 17:15 | |
| *** marios|out has quit IRC | 17:23 | |
| panda | zbr: NODE_FAILURE everywhere :( | 17:28 |
| panda | zbr: https://review.rdoproject.org/r/25938 know anything ? | 17:28 |
| *** holser__ has quit IRC | 17:39 | |
| *** bogdando has quit IRC | 17:40 | |
| *** holser has joined #oooq | 17:40 | |
| panda | zbr: nevermind | 17:41 |
| *** holser has quit IRC | 17:58 | |
| *** derekh has quit IRC | 18:00 | |
| *** holser has joined #oooq | 18:07 | |
| panda | weshay|ruck: single promotion command for promoter cli https://review.rdoproject.org/r/25992 | 18:08 |
| weshay|ruck | panda, nice.. did your hotfix re: tripleo-ci-testing vs. the agg hash used.. merge? | 18:09 |
| weshay|ruck | panda, hope you like the st. patricks email :) just trying to get a laugh | 18:10 |
| weshay|ruck | rlandy, bringing up a testnode now | 18:11 |
| *** dtantsur is now known as dtantsur|afk | 18:11 | |
| weshay|ruck | panda, ah.. think that is https://review.rdoproject.org/r/#/c/25938/ | 18:13 |
| panda | weshay|ruck: no, had problems with jobs all day today, it's still here https://review.rdoproject.org/r/25938. Also added test to check this condition in the future. Staging environment is more noisy and less test-convenient now" | 18:13 |
| weshay|ruck | cool.. thank you | 18:13 |
| *** amoralej is now known as amoralej|off | 18:15 | |
| *** jpena is now known as jpena|off | 18:16 | |
| *** holser has quit IRC | 18:16 | |
| *** holser has joined #oooq | 18:17 | |
| rlandy | weshay|ruck: ack - you can check against the held node | 18:37 |
| weshay|ruck | rlandy, it's done? | 18:38 |
| *** sshnaidm is now known as sshnaidm|afk | 18:41 | |
| rlandy | repos are laid | 18:45 |
| weshay|ruck | rlandy, seeing something weird | 18:47 |
| rlandy | I see | 18:50 |
| rlandy | you're on the tmate or your box? | 18:50 |
| weshay|ruck | my own box | 18:51 |
| weshay|ruck | sec.. running a yum update first | 18:51 |
| weshay|ruck | my steps thus far | 18:52 |
| weshay|ruck | rpm -qa | grep openstack | 18:52 |
| weshay|ruck | wget --content-disposition --no-parent -r https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/etc/yum.repos.d/ | 18:52 |
| weshay|ruck | mv logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/5127081/logs/undercloud/etc/yum.repos.d/*.txt.gz . | 18:52 |
| weshay|ruck | for i in `ls *.gz`; do echo $i; gzip -d $i; done | 18:52 |
| weshay|ruck | for i in `ls *.txt`;do echo ${i::(-4)}; mv $i ${i::(-4)}; done | 18:52 |
| weshay|ruck | running an update though... before I try to install anything because installing any of https://logserver.rdoproject.org/openstack-component-network/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-network-master/8760368/logs/undercloud/var/log/extra/podman/containers/octavia_api/podman_info.log.txt.gz | 18:52 |
| weshay|ruck | is not working... I've disabled the component repo for now.. trying to just get current-tripleo installed | 18:53 |
| weshay|ruck | brb | 18:53 |
| rlandy | ok | 18:53 |
| *** holser has quit IRC | 19:18 | |
| *** holser has joined #oooq | 19:36 | |
| rlandy | weshay|ruck: http://pastebin.test.redhat.com/846118 | 19:59 |
| *** jmasud has quit IRC | 20:55 | |
| *** jmasud has joined #oooq | 20:56 | |
| *** chem has quit IRC | 20:58 | |
| *** chem has joined #oooq | 21:00 | |
| arxcruz|rover | weshay|ruck: is there something wrong with the queue? | 21:07 |
| arxcruz|rover | weshay|ruck: 32 min in the dashboard, and only 5 failures | 21:07 |
| weshay|ruck | arxcruz|rover, last time I looked no... /me checks again | 21:08 |
| arxcruz|rover | I mean, it's too good to be true right? | 21:08 |
| weshay|ruck | arxcruz|rover, heh.. you know ur a ruck / rover when | 21:08 |
| weshay|ruck | arxcruz|rover, :) comment on your patch btw.. looks good just needs become: | 21:09 |
| arxcruz|rover | weshay|ruck: already submited a new ps | 21:09 |
| weshay|ruck | arxcruz|rover, cool.. there is a third party rhel8 | 21:09 |
| weshay|ruck | so should see it pass there | 21:09 |
| arxcruz|rover | weshay|ruck: https://review.rdoproject.org/r/#/c/25959/ testproject | 21:10 |
| weshay|ruck | arxcruz|rover, no need man | 21:10 |
| weshay|ruck | ARBY'S WE GOT THE BEEF | 21:10 |
| weshay|ruck | WE GOT THE MEAT | 21:10 |
| weshay|ruck | rather | 21:10 |
| weshay|ruck | arxcruz|rover, think we need to split full tempest on the standalone for c8 btw | 21:11 |
| arxcruz|rover | ok | 21:11 |
| *** jbadiapa has quit IRC | 21:13 | |
| *** openstackstatus has joined #oooq | 21:15 | |
| *** ChanServ sets mode: +v openstackstatus | 21:15 | |
| *** dpawlik has quit IRC | 21:26 | |
| *** rlandy is now known as rlandy|bbl | 22:27 | |
| *** TrevorV has quit IRC | 22:41 | |
| *** jfrancoa has quit IRC | 22:44 | |
| *** holser has quit IRC | 22:55 | |
| *** jmasud has quit IRC | 23:31 | |
| *** jmasud has joined #oooq | 23:39 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!