| weshay|ruck | anbanerj|ruck, merged.. I'll reload the cockpit shortly | 00:55 |
|---|---|---|
| weshay|ruck | rlandy|rover|afk, should be gtg w/ https://review.rdoproject.org/zuul/buildset/f0bb2f75a741453f811ffc68b0c4f789 | 00:56 |
| * weshay|ruck runs a quick errand | 00:56 | |
| weshay|ruck | queue is still too long to promote atm | 00:56 |
| *** jmasud has joined #oooq | 00:58 | |
| *** rlandy|rover|afk is now known as rlandy|rover | 01:00 | |
| weshay|ruck | rlandy|rover, 0/ | 01:00 |
| rlandy|rover | weshay|ruck: hey | 01:00 |
| weshay|ruck | too busy :) | 01:00 |
| rlandy|rover | the queue or me? | 01:00 |
| weshay|ruck | but we have a good run | 01:00 |
| weshay|ruck | rlandy|rover, the queue :) | 01:01 |
| weshay|ruck | how's the food bank | 01:01 |
| rlandy|rover | taking up too much of my time | 01:01 |
| weshay|ruck | heh | 01:01 |
| rlandy|rover | there every night his week | 01:01 |
| weshay|ruck | rlandy|rover oy | 01:01 |
| weshay|ruck | we're almost done w/ master https://review.rdoproject.org/zuul/buildset/f0bb2f75a741453f811ffc68b0c4f789 | 01:01 |
| rlandy|rover | next week hopefully back to two days a week | 01:01 |
| * rlandy|rover looks | 01:01 | |
| rlandy|rover | NICE | 01:02 |
| weshay|ruck | another one is running too | 01:02 |
| rlandy|rover | scenario tempest failed again :( | 01:02 |
| weshay|ruck | ya.. but.. | 01:02 |
| rlandy|rover | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master+ | 01:02 |
| rlandy|rover | only lately | 01:03 |
| weshay|ruck | rlandy|rover, passes here https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/39f5fa2/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 01:03 |
| weshay|ruck | also probably passed in some other jobs | 01:03 |
| rlandy|rover | periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-masteropenstack/tripleo-cimasteropenstack-periodic-integration-main | 01:03 |
| rlandy|rover | and there | 01:03 |
| rlandy|rover | https://logserver.rdoproject.org/86/31586/3/check/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master/4eaa05d/logs/undercloud/var/log/tempest/stestr_results.html.gz | 01:04 |
| rlandy|rover | ^^ that's unfortunate | 01:04 |
| weshay|ruck | if we don't see it in upstream jobs.. we're fine | 01:04 |
| rlandy|rover | it used to be such a nice display | 01:04 |
| weshay|ruck | rlandy|rover, ya.. I asked the tempest folks to look at that | 01:04 |
| weshay|ruck | not sure if I communicated | 01:05 |
| rlandy|rover | } tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario [95.431436s] ... FAILED | 01:05 |
| rlandy|rover | .gz | 01:05 |
| rlandy|rover | queue is still super long | 01:05 |
| rlandy|rover | ugh | 01:06 |
| weshay|ruck | yup | 01:06 |
| rlandy|rover | tomorrow is our lucky day | 01:06 |
| weshay|ruck | rlandy|rover, people try to get their stuff in before we branch | 01:06 |
| rlandy|rover | we kick them off the queue | 01:06 |
| weshay|ruck | slackers | 01:06 |
| rlandy|rover | national holiday - EVRYWHERE | 01:06 |
| rlandy|rover | happy slack off day | 01:06 |
| weshay|ruck | lolz | 01:06 |
| rlandy|rover | except us | 01:07 |
| weshay|ruck | ci has .. you guys are doing great :) take the day off please | 01:07 |
| weshay|ruck | lolz | 01:07 |
| rlandy|rover | by the power invested in me - ci grants you the day off | 01:07 |
| rlandy|rover | weshay|ruck: anything else to take care of tonight? | 01:09 |
| rlandy|rover | ananya is working on checking the if the CIX's in degraded are still needed | 01:09 |
| weshay|ruck | ++ | 01:09 |
| rlandy|rover | gouthamr: sure - all for the better approach | 01:12 |
| rlandy|rover | gouthamr: let's review this tomorrow - then we should do the same in periodics | 01:14 |
| rlandy|rover | thanks for the update | 01:14 |
| *** rlandy|rover has quit IRC | 01:15 | |
| *** spotz has quit IRC | 01:36 | |
| *** jmasud has quit IRC | 02:23 | |
| *** ykarel has joined #oooq | 03:48 | |
| *** skramaja has joined #oooq | 03:54 | |
| *** jmasud has joined #oooq | 04:01 | |
| *** jmasud has quit IRC | 04:12 | |
| *** ratailor has joined #oooq | 04:29 | |
| *** udesale has joined #oooq | 05:02 | |
| *** ratailor_ has joined #oooq | 05:07 | |
| *** marios has joined #oooq | 05:07 | |
| *** jmasud has joined #oooq | 05:07 | |
| *** ratailor has quit IRC | 05:09 | |
| *** jpodivin has joined #oooq | 05:51 | |
| *** udesale has quit IRC | 06:05 | |
| *** ysandeep|away is now known as ysandeep | 06:10 | |
| *** slaweq has joined #oooq | 06:11 | |
| *** udesale has joined #oooq | 06:14 | |
| *** amoralej|off is now known as amoralej | 06:16 | |
| *** ratailor__ has joined #oooq | 06:19 | |
| *** ratailor_ has quit IRC | 06:22 | |
| *** ratailor has joined #oooq | 06:28 | |
| *** ratailor__ has quit IRC | 06:28 | |
| *** jbadiapa has joined #oooq | 06:59 | |
| *** ratailor_ has joined #oooq | 07:04 | |
| *** ratailor has quit IRC | 07:06 | |
| *** jmasud has quit IRC | 07:09 | |
| *** jmasud has joined #oooq | 07:10 | |
| *** jmasud has quit IRC | 07:34 | |
| *** ChanServ has quit IRC | 07:58 | |
| *** skramaja has quit IRC | 07:58 | |
| *** ykarel has quit IRC | 07:58 | |
| *** dviroel has quit IRC | 07:59 | |
| *** dsneddon has quit IRC | 07:59 | |
| *** owalsh has quit IRC | 07:59 | |
| *** odyssey4me has quit IRC | 07:59 | |
| *** soniya29 has quit IRC | 07:59 | |
| *** chem has quit IRC | 07:59 | |
| *** tristanC has quit IRC | 07:59 | |
| *** baha has quit IRC | 07:59 | |
| *** mjturek has quit IRC | 07:59 | |
| *** ratailor_ has quit IRC | 07:59 | |
| *** udesale has quit IRC | 07:59 | |
| *** zbr has quit IRC | 07:59 | |
| *** cgoncalves has quit IRC | 07:59 | |
| *** Tengu has quit IRC | 07:59 | |
| *** jpena|off has quit IRC | 07:59 | |
| *** weshay|ruck has quit IRC | 07:59 | |
| *** tbarron has quit IRC | 07:59 | |
| *** johnsom has quit IRC | 07:59 | |
| *** ysandeep has quit IRC | 07:59 | |
| *** haleyb has quit IRC | 07:59 | |
| *** apetrich has quit IRC | 07:59 | |
| *** amoralej has quit IRC | 07:59 | |
| *** radez has quit IRC | 07:59 | |
| *** gchamoul has quit IRC | 07:59 | |
| *** matbu has quit IRC | 07:59 | |
| *** pojadhav has quit IRC | 07:59 | |
| *** arxcruz has quit IRC | 07:59 | |
| *** chandankumar has quit IRC | 07:59 | |
| *** beagles has quit IRC | 07:59 | |
| *** gouthamr has quit IRC | 07:59 | |
| *** jbadiapa has quit IRC | 07:59 | |
| *** jpodivin has quit IRC | 07:59 | |
| *** akahat has quit IRC | 07:59 | |
| *** dmellado has quit IRC | 07:59 | |
| *** dpawlik has quit IRC | 07:59 | |
| *** bhagyashris has quit IRC | 07:59 | |
| *** anbanerj|ruck has quit IRC | 07:59 | |
| *** soniya29 has joined #oooq | 08:04 | |
| *** owalsh has joined #oooq | 08:04 | |
| *** dsneddon has joined #oooq | 08:04 | |
| *** ChanServ has joined #oooq | 08:04 | |
| *** verne.freenode.net sets mode: +o ChanServ | 08:04 | |
| *** anbanerj|ruck has joined #oooq | 08:05 | |
| *** mjturek has joined #oooq | 08:05 | |
| *** tristanC has joined #oooq | 08:05 | |
| *** baha has joined #oooq | 08:05 | |
| *** jbadiapa has joined #oooq | 08:05 | |
| *** jpodivin has joined #oooq | 08:05 | |
| *** akahat has joined #oooq | 08:05 | |
| *** dmellado has joined #oooq | 08:05 | |
| *** dpawlik has joined #oooq | 08:05 | |
| *** bhagyashris has joined #oooq | 08:05 | |
| *** matbu has joined #oooq | 08:05 | |
| *** pojadhav has joined #oooq | 08:05 | |
| *** arxcruz has joined #oooq | 08:05 | |
| *** chandankumar has joined #oooq | 08:05 | |
| *** skramaja has joined #oooq | 08:05 | |
| *** ykarel has joined #oooq | 08:05 | |
| *** dviroel has joined #oooq | 08:05 | |
| *** gouthamr has joined #oooq | 08:05 | |
| *** gchamoul has joined #oooq | 08:06 | |
| *** radez has joined #oooq | 08:06 | |
| *** haleyb has joined #oooq | 08:06 | |
| *** apetrich has joined #oooq | 08:06 | |
| *** amoralej has joined #oooq | 08:06 | |
| *** ratailor_ has joined #oooq | 08:06 | |
| *** udesale has joined #oooq | 08:06 | |
| *** zbr has joined #oooq | 08:06 | |
| *** cgoncalves has joined #oooq | 08:06 | |
| *** Tengu has joined #oooq | 08:06 | |
| *** jpena has joined #oooq | 08:06 | |
| *** weshay|ruck has joined #oooq | 08:06 | |
| *** tbarron has joined #oooq | 08:06 | |
| *** johnsom has joined #oooq | 08:06 | |
| *** holser has joined #oooq | 08:06 | |
| *** ysandeep has joined #oooq | 08:06 | |
| *** odyssey4me has joined #oooq | 08:06 | |
| *** gouthamr has quit IRC | 08:08 | |
| *** gouthamr has joined #oooq | 08:09 | |
| *** tosky has joined #oooq | 08:09 | |
| *** gouthamr has quit IRC | 08:20 | |
| *** gouthamr has joined #oooq | 08:20 | |
| *** derekh has joined #oooq | 08:31 | |
| *** beagles has joined #oooq | 08:49 | |
| * ysandeep afk for ~1-1.5 hour.. I have a headache.. | 09:28 | |
| *** chem has joined #oooq | 09:34 | |
| chandankumar | amoralej: please have a look at https://review.rdoproject.org/r/c/rdo-jobs/+/32967 and https://review.rdoproject.org/r/c/rdoinfo/+/32936 | 09:36 |
| *** ysandeep is now known as ysandeep|afk | 09:40 | |
| *** ratailor has joined #oooq | 10:11 | |
| *** ratailor_ has quit IRC | 10:11 | |
| chandankumar | slaweq: hello, please have a look at this failure https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_2da/785195/1/check/tripleo-ci-centos-8-standalone-upgrade-victoria/2da7930/logs/undercloud/var/log/tempest/tempest_run.log when free, thans! | 10:33 |
| slaweq | chandankumar: sure, looking | 10:36 |
| slaweq | chandankumar: it seems that it's the same issue like in https://trello.com/c/ZyPpB3K2 | 10:39 |
| slaweq | there are errors while creating interface in ovn-metadata agent | 10:39 |
| chandankumar | slaweq: thanks for confirming was suspecting the same | 10:43 |
| slaweq | chandankumar: yw, that was easy :) | 10:43 |
| *** ykarel has quit IRC | 10:44 | |
| *** ykarel has joined #oooq | 10:45 | |
| *** ykarel has quit IRC | 10:47 | |
| *** ykarel has joined #oooq | 10:48 | |
| *** ykarel_ has joined #oooq | 10:56 | |
| *** ykarel has quit IRC | 10:59 | |
| *** jfrancoa has joined #oooq | 10:59 | |
| *** ratailor_ has joined #oooq | 11:08 | |
| *** ratailor has quit IRC | 11:10 | |
| *** ykarel_ is now known as ykarel | 11:17 | |
| *** jpena is now known as jpena|lunch | 11:30 | |
| *** ysandeep|afk is now known as ysandeep | 11:47 | |
| *** rlandy has joined #oooq | 11:47 | |
| arxcruz | chandankumar: https://review.opendev.org/c/osf/python-tempestconf/+/785450 | 11:48 |
| arxcruz | i was discussing this yesterday with rlandy and gouthamr | 11:48 |
| arxcruz | do you mind if we merge? scen004 is failing because of this | 11:49 |
| arxcruz | gouthamr have it tested locally already | 11:49 |
| arxcruz | Add a bz right now would make the jobs restart and take longer time | 11:49 |
| chandankumar | arxcruz: done, thanks! | 11:50 |
| rlandy | arxcruz++ thanks | 11:50 |
| arxcruz | chandankumar: धन्यवाद् | 11:50 |
| rlandy | merge day ... master is ready | 11:56 |
| chandankumar | arxcruz: you are learning hindi good good :-) | 11:57 |
| chandankumar | arxcruz: can you pass me the patch link for skip list consolidation? | 11:58 |
| rlandy | bhagyashris: are you on the promoter? | 11:58 |
| *** rlandy is now known as rlandy|rover | 11:59 | |
| bhagyashris | yes | 12:00 |
| bhagyashris | rlandy|rover, yes | 12:01 |
| rlandy|rover | bhagyashris: are you editing the criteria? | 12:01 |
| rlandy|rover | for master | 12:01 |
| bhagyashris | rlandy|rover, no | 12:01 |
| rlandy|rover | someone has the file open | 12:01 |
| bhagyashris | no i am not doing anything | 12:01 |
| soniya29 | chandankumar, how to manually enable patrole in tempest? | 12:02 |
| anbanerj|ruck | I am not doing anything as well | 12:02 |
| bhagyashris | rlandy|rover, you can go head | 12:02 |
| chandankumar | weshay|ruck: do you have the link to repoclouse failure from tempest component? | 12:02 |
| chandankumar | soniya29: you mean in Devstack? | 12:05 |
| soniya29 | chandankumar, yes | 12:05 |
| *** Goneri has joined #oooq | 12:05 | |
| soniya29 | chandankumar, I have installed devstack but I haven't enabled it in local.conf | 12:05 |
| chandankumar | soniya29: there might be tempest venv with in devstack deployment, you can clone patrole and install it in tempest venv and run it like just like other tempest tests | 12:06 |
| bhagyashris | rlandy|rover, hey need promoter server for few mins... should i do the changes? | 12:09 |
| rlandy|rover | bhagyashris: will the changes break something :) | 12:09 |
| bhagyashris | rlandy|rover, i see dlrn-promoter service is disable | 12:10 |
| rlandy|rover | atm yes | 12:10 |
| rlandy|rover | we need to edit the criteria | 12:10 |
| bhagyashris | rlandy|rover, i want to check that if it will break then i will revert it back | 12:10 |
| rlandy|rover | bhagyashris: go ahead if it's a safe change | 12:10 |
| bhagyashris | rlandy|rover, is that ok? | 12:10 |
| rlandy|rover | as soon as wes comes on line, we need to start promoting | 12:11 |
| rlandy|rover | bhagyashris: yeah | 12:11 |
| bhagyashris | ok so before that should i test? | 12:11 |
| rlandy|rover | bhagyashris: as long as we can get it back | 12:11 |
| bhagyashris | i mean if the criteria file is ready | 12:11 |
| rlandy|rover | idk how urgent your change is | 12:11 |
| rlandy|rover | bhagyashris: I can edit the criteria file afterwards | 12:11 |
| rlandy|rover | isk who has it open | 12:12 |
| rlandy|rover | weshay|ruck: hey | 12:12 |
| rlandy|rover | weshay|ruck: pls ping when you are ready to promote | 12:12 |
| weshay|ruck | rlandy|rover, one requirement.. at least on sip of coffee | 12:12 |
| bhagyashris | rlandy|rover,we will wait for wes | 12:13 |
| rlandy|rover | go4it | 12:13 |
| bhagyashris | ohh wes is here | 12:13 |
| rlandy|rover | bhagyashris: yeah the last hash is still running | 12:13 |
| rlandy|rover | the previous one ... | 12:13 |
| bhagyashris | rlandy|rover, jfyi i want to test this change https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33161/ | 12:14 |
| weshay|ruck | well.. I'll be.. maybe this is consistent https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master | 12:14 |
| weshay|ruck | let's put in a skip | 12:14 |
| bhagyashris | and if it's not work/breaks then will revert back the change | 12:14 |
| rlandy|rover | weshay|ruck: ack - adding to skip | 12:15 |
| rlandy|rover | fs020 also failed now | 12:16 |
| rlandy|rover | arxcruz: zbr: soniya29: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/36eaa2f/logs/undercloud/var/log/tempest/ - we lost the cool .html display | 12:17 |
| rlandy|rover | harder to see what is going on with these tests | 12:17 |
| rlandy|rover | tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario [104.644828s] ... FAILED | 12:18 |
| rlandy|rover | ^^ fs020 | 12:18 |
| rlandy|rover | so we have a legit issue | 12:18 |
| arxcruz | rlandy|rover: it might be some change in rdo side regarding .gz html | 12:18 |
| rlandy|rover | arxcruz: I figure yes | 12:18 |
| rlandy|rover | - Failed: 1 | 12:18 |
| arxcruz | let me see the failure | 12:19 |
| rlandy|rover | : Failed to find floating IP '10.0.0.148' in server addresses: {'tempest-TestMinimumBasicScenario-1608410280-network': [{'version': 4, 'addr': '10.100.0.6', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:f7:2b:13'}]} | 12:19 |
| arxcruz | testtools.matchers._impl.MismatchError: None matches Is(None): Failed to find floating IP '10.0.0.148' in server addresses: {'tempest-TestMinimumBasicScenario-1608410280-network': [{'version': 4, 'addr': '10.100.0.6', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:f7:2b:13'}]} | 12:19 |
| arxcruz | yeah | 12:19 |
| arxcruz | slaweq: ^ | 12:19 |
| arxcruz | :D | 12:19 |
| rlandy|rover | periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-masteropenstack/tripleo-cimasteropenstack-periodic-integration-main0000000 | 12:20 |
| rlandy|rover | 3 hrs 37 mins 39 secs2021-04-08 21:04:39 | 12:20 |
| rlandy|rover | had a good run the time before | 12:20 |
| rlandy|rover | sporadic | 12:20 |
| arxcruz | maybe it's just sporadic | 12:20 |
| arxcruz | but might worth contact neutron guys | 12:20 |
| slaweq | arxcruz: can You give me link to the failed job? | 12:20 |
| slaweq | I will check, but probably not today | 12:20 |
| arxcruz | slaweq: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/36eaa2f/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 12:20 |
| soniya29 | chandankumar, ack | 12:21 |
| rlandy|rover | slaweq: not urgent ... we are adding to skiplist atm | 12:21 |
| rlandy|rover | and it seems sporadic - at least in fs020 | 12:21 |
| arxcruz | rlandy|rover: it's kind of basic test for us | 12:21 |
| arxcruz | if this is failing, it's bad | 12:21 |
| rlandy|rover | arxcruz: this is a bigger worry ... | 12:22 |
| arxcruz | but since it's sporadic, it might be some concurrency | 12:22 |
| rlandy|rover | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master | 12:22 |
| rlandy|rover | ^^ much more consistent failure | 12:22 |
| arxcruz | yeah, this is bad | 12:23 |
| arxcruz | slaweq: ^ | 12:23 |
| arxcruz | rlandy|rover: we can skip it for that job only if we need promotion, but this is a blocker for sure | 12:23 |
| arxcruz | this is the basic scenario, if it's failing, that's a problem | 12:23 |
| rlandy|rover | testtools.matchers._impl.MismatchError: None matches Is(None): Failed to find floating IP '192.168.24.142' in server addresses: {'tempest-TestMinimumBasicScenario-872779920-network': [{'version': 4, 'addr': '10.100.0.5', 'OS-EXT-IPS:type': 'fixed', 'OS-EXT-IPS-MAC:mac_addr': 'fa:16:3e:45:ae:9b'}]} | 12:23 |
| rlandy|rover | 12:23 | |
| rlandy|rover | ^^ same error | 12:23 |
| rlandy|rover | logging LP | 12:23 |
| arxcruz | i'm afraid this is neutron issue | 12:24 |
| arxcruz | slaweq: | 12:24 |
| weshay|ruck | rlandy|rover, fyi.. we have an inconsistent pattern in tests in periodic vs upstream | 12:25 |
| weshay|ruck | for example | 12:25 |
| weshay|ruck | https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-master/6fd94e4/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 12:25 |
| weshay|ruck | vs.. | 12:25 |
| weshay|ruck | https://codesearch.opendev.org/?q=TestMinimumBasicScenario&i=nope&files=&excludeFiles=&repos=openstack/tripleo-ansible,openstack/tripleo-ci,openstack/tripleo-ci-health-queries,openstack/tripleo-common,openstack/tripleo-common-tempest-plugin,openstack/tripleo-docs,openstack/tripleo-ha-utils,openstack/tripleo-heat-templates,openstack/tripleo-image-elements,openstack/tripleo-ipsec,openstack/tripleo-operator-ansible,openstack/tripleo-puppet-el | 12:25 |
| weshay|ruck | ements,openstack/tripleo-quickstart,openstack/tripleo-quickstart-extras,openstack/tripleo-repos,openstack/tripleo-specs,openstack/tripleo-upgrade,openstack/tripleo-validations | 12:25 |
| rlandy|rover | ugh - we never matched those well | 12:25 |
| * weshay|ruck add basicscenario to periodic | 12:25 | |
| weshay|ruck | rlandy|rover, we run more in periodic | 12:25 |
| weshay|ruck | but this is a miss | 12:26 |
| rlandy|rover | one sec - submitting bug | 12:26 |
| *** jfrancoa has quit IRC | 12:27 | |
| chandankumar | rlandy|rover: anbanerj|ruck we have cleared the octavia tempest issue, now you can see octavia sc10 jobs passing from ussuri onwards | 12:28 |
| *** jpena|lunch is now known as jpena | 12:29 | |
| chandankumar | I am still looking into train case | 12:29 |
| rlandy|rover | chandankumar: ack - thanks | 12:29 |
| rlandy|rover | https://bugs.launchpad.net/tripleo/+bug/1923194 | 12:29 |
| openstack | Launchpad bug 1923194 in tripleo "master promotion: tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario is failing on fs020 and tempest-scenario" [Critical,Triaged] | 12:29 |
| rlandy|rover | logged | 12:29 |
| rlandy|rover | skiplist change coming | 12:30 |
| zbr | rlandy|rover: one more to report: codesearch.rdopoject.org is unsecured and unable to login | 12:30 |
| rlandy|rover | zbr: ongoing outage there | 12:31 |
| rlandy|rover | downstream has been up and down for days | 12:31 |
| rlandy|rover | see outage list - lots of fun | 12:31 |
| chandankumar | zbr: https://review.rdoproject.org/codesearch/ | 12:31 |
| chandankumar | after upgrade, each of them have url | 12:32 |
| chandankumar | *new url | 12:32 |
| rlandy|rover | https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/785631 | 12:39 |
| rlandy|rover | weshay|ruck: anbanerj|ruck: ^^ | 12:40 |
| zbr | and nobody bothered to do a redirect | 12:40 |
| *** jfrancoa has joined #oooq | 12:47 | |
| rlandy|rover | marios: anbanerj|ruck: bhagyashris: pojadhav: ysandeep: want to keep wallaby branching meeting today? | 12:48 |
| marios | rlandy|rover: we could join for quick sync? i don't have something major... planning to branch tripleoclient this afternoon | 12:48 |
| rlandy|rover | marios: ack | 12:48 |
| bhagyashris | bhagyashris, i am free to join | 12:49 |
| weshay|ruck | arxcruz, rlandy|rover minor update on the job list https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/785631/ | 12:49 |
| ysandeep | rlandy|rover, i am free to join | 12:49 |
| weshay|ruck | not 100% sure I got them all | 12:49 |
| weshay|ruck | rlandy|rover, look ok to you? | 12:49 |
| * rlandy|rover looks | 12:49 | |
| ysandeep | bhagyashris, pinging to yourself :) | 12:50 |
| zbr | rlandy|rover: i am curious to which lost nice html you you were referring to, so i would not search for the wrong thing. | 12:50 |
| bhagyashris | :D:D | 12:50 |
| bhagyashris | rlandy|rover, ^ | 12:51 |
| zbr | my guess was that zuul would link to log-server directly instead of its own zuul-build-dashboard view | 12:51 |
| weshay|ruck | zbr, for example https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-master/6fd94e4/logs/undercloud/var/log/tempest/stestr_results.html.gz | 12:51 |
| rlandy|rover | zbr: see the tempest logs in periodics | 12:51 |
| rlandy|rover | .html became .html.gz | 12:52 |
| zbr | ahh, that is another story and i am out of energy. I always advised against compressing html/text files. | 12:52 |
| rlandy|rover | so it no longer resolves to the readable format | 12:52 |
| zbr | you should better find one that opposed me to help fixing this issue | 12:52 |
| rlandy|rover | could you id "someone"? | 12:52 |
| rlandy|rover | pm | 12:52 |
| rlandy|rover | weshay|ruck: other than codesearch - no good way to confirm we have not missed a case | 12:53 |
| rlandy|rover | but we will have to watch | 12:53 |
| rlandy|rover | also - sporadic | 12:53 |
| rlandy|rover | no idea id we will see the same in check | 12:53 |
| rlandy|rover | let's go with that for now | 12:53 |
| rlandy|rover | weshay|ruck: and it passed this round periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master | 12:55 |
| rlandy|rover | SUCCESS | 12:55 |
| rlandy|rover | lol | 12:55 |
| *** jfrancoa has quit IRC | 12:55 | |
| weshay|ruck | hrm.. | 12:55 |
| chandankumar | rlandy|rover: weshay|ruck can we merge this https://review.opendev.org/c/openstack/tripleo-ci/+/784040 ? | 12:55 |
| weshay|ruck | what to do | 12:55 |
| rlandy|rover | weshay|ruck: would merge the skiplist thing | 12:56 |
| rlandy|rover | unmerge on monday and watch | 12:56 |
| rlandy|rover | so we don't kill the weekend for someone working then | 12:56 |
| chandankumar | cgoncalves: please have a look https://review.rdoproject.org/r/q/topic:%22fix_octavia_kvm%22+(status:open%20OR%20status:merged) when free, thanks! | 12:57 |
| rlandy|rover | chandankumar: yep - voted - you can w+ when ready | 12:58 |
| *** amoralej is now known as amoralej|lunch | 12:58 | |
| rlandy|rover | weshay|ruck: have wallaby meeting now - can promote after | 12:58 |
| weshay|ruck | k | 12:58 |
| rlandy|rover | today is a good day on the gate | 12:59 |
| rlandy|rover | so far | 12:59 |
| *** rlandy|rover is now known as rlandy|rover|mtg | 13:00 | |
| weshay|ruck | pojadhav, see my email about the cockpit json? | 13:02 |
| pojadhav | weshay|ruck, yeah | 13:02 |
| weshay|ruck | pojadhav, k | 13:02 |
| weshay|ruck | pojadhav, easy fix can you post one? | 13:02 |
| pojadhav | influxdb doesnt have wallaby jobs in it | 13:03 |
| pojadhav | checked db | 13:03 |
| weshay|ruck | pojadhav, ya.. it hasn't branched yet | 13:03 |
| weshay|ruck | but it will :) | 13:03 |
| pojadhav | then how jobs will appear in that panel ? I am confused | 13:03 |
| weshay|ruck | pojadhav, once opendev branches master -> wallaby.. which is coming... and then tripleo branches master -> wallaby | 13:04 |
| weshay|ruck | the zuul meta on branch will change from master -> stable/wallaby | 13:04 |
| weshay|ruck | and job will start to show up in the cockpit | 13:04 |
| weshay|ruck | pojadhav, make sense? | 13:04 |
| pojadhav | yeah.. got your point | 13:05 |
| pojadhav | can we meet for short.. | 13:05 |
| pojadhav | discussion | 13:05 |
| weshay|ruck | pojadhav, can we try irc? | 13:05 |
| pojadhav | sure | 13:06 |
| chandankumar | rlandy|rover|mtg: can you remove -w on this https://review.opendev.org/c/openstack/tripleo-quickstart/+/785408 so we can merge it? | 13:06 |
| weshay|ruck | pojadhav, put up a change.. to move the wallaby job panel directly below the master on | 13:06 |
| weshay|ruck | one | 13:06 |
| soniya29 | chandankumar, I tried to run the patrole tests as like tempest tests, but it seems i am doing something wrong | 13:06 |
| weshay|ruck | pojadhav, I'll merge your patches.. and we'll take a look at it | 13:06 |
| pojadhav | okay | 13:07 |
| rlandy|rover|mtg | chandankumar: sorry - no can do until we promote | 13:07 |
| rlandy|rover|mtg | hopefully today | 13:07 |
| weshay|ruck | will be today.. by hook or crook | 13:07 |
| pojadhav | weshay|ruck, also I have added for promotions - https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33200 | 13:07 |
| soniya29 | chandankumar, can we meet for a short discussion? | 13:07 |
| chandankumar | rlandy|rover|mtg: ok | 13:09 |
| chandankumar | soniya29: sure | 13:09 |
| soniya29 | chandankumar, https://meet.google.com/ypc-ggxv-wor | 13:10 |
| *** ratailor_ has quit IRC | 13:12 | |
| *** skramaja has quit IRC | 13:15 | |
| rlandy|rover|mtg | anbanerj|ruck: want to join again? | 13:15 |
| rlandy|rover|mtg | promoting | 13:15 |
| bhagyashris | rlandy|rover|mtg, hey you want me to join as well for promotions stuff | 13:15 |
| rlandy|rover|mtg | you can | 13:16 |
| *** ysandeep is now known as ysandeep|away | 13:16 | |
| chandankumar | soniya29: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/post_deployment/tempest/tempest.html | 13:18 |
| chandankumar | soniya29: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/post_deployment/tempest/os_tempest.html | 13:20 |
| chandankumar | soniya29: https://opendev.org/openstack/tripleo-docs | 13:21 |
| soniya29 | chandankumar, network issues | 13:21 |
| pojadhav | weshay|ruck, https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33205 | 13:25 |
| *** ysandeep|away is now known as ysandeep | 13:35 | |
| slaweq | hi, I have a question about tripleo jobs | 13:39 |
| chandankumar | marios: do we want to move the tripleo-ci happy hours call after tripleo-repos call? | 13:39 |
| slaweq | in neutron periodic queue we have tripleo standalone job: https://github.com/openstack/neutron/blob/47dcbc240e3ff9747c23b18a138a0b65bbb7c796/zuul.d/tripleo.yaml#L2 | 13:39 |
| weshay|ruck | pojadhav, k.. let's let these merge.. and then we'll poke at it together | 13:39 |
| slaweq | but I just discovered that it runs neutron victoria release always :/ | 13:40 |
| slaweq | which isn't what we really want :) | 13:40 |
| pojadhav | weshay|ruck, yup | 13:40 |
| slaweq | is there any switch/flag which I should set in that job to run always neutron from master? | 13:40 |
| marios | chandankumar: k but i think it will be a bit late for pune | 13:40 |
| marios | chandankumar: we can cancel happy hour | 13:41 |
| marios | chandankumar: :( | 13:41 |
| *** amoralej|lunch is now known as amoralej | 13:42 | |
| chandankumar | slaweq: may be setting branches flag there? | 13:42 |
| chandankumar | s/flag/zuul var | 13:43 |
| chandankumar | slaweq: https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/standalone-jobs.yaml#L808 | 13:43 |
| weshay|ruck | rlandy|rover|mtg, 35 failed | 13:44 |
| rlandy|rover|mtg | weshay|ruck: looking | 13:44 |
| weshay|ruck | timeout | 13:44 |
| weshay|ruck | anbanerj|ruck, zbr 0/ | 13:44 |
| rlandy|rover|mtg | rerun? | 13:44 |
| weshay|ruck | rlandy|rover|mtg, naw | 13:44 |
| slaweq | chandankumar: but don't this regex mean "don't run it on stable/newton|ocata|pike|queens|stein branches"? How would it help to run on master? | 13:45 |
| zbr | ahh. i was confused by the switch of nicks. | 13:46 |
| weshay|ruck | rlandy|rover|mtg, chandankumar osp-17 failing on build_images.sh: line 49: export: `container-tools:3.0': not a valid identifier | 13:47 |
| rlandy|rover|mtg | https://zuul.openstack.org/builds?job_name=neutron-centos-8-tripleo-standalone .. runs on other branches | 13:47 |
| weshay|ruck | missing dib updates I assume right? | 13:47 |
| *** rlandy|rover|mtg is now known as rlandy|rover | 13:47 | |
| rlandy|rover | probably - checking | 13:48 |
| weshay|ruck | marios, chandankumar you folks have a conflict in repos/happy-hour | 13:48 |
| chandankumar | slaweq: yes, it will not run on those branches, if we call the job in job template it will run on other branches | 13:49 |
| marios | weshay|ruck: yeah | 13:49 |
| marios | weshay|ruck: i guess some of us just won't join happy hour no big problem ;) | 13:49 |
| weshay|ruck | chandankumar, can you push the tripleo-repos meeting 10 min out? | 13:51 |
| rlandy|rover | weshay|ruck: changeprobably has not reach DIB internal | 13:51 |
| chandankumar | slaweq: can I try a fix and see? | 13:51 |
| weshay|ruck | rlandy|rover, ya | 13:51 |
| rlandy|rover | export DIB_DNF_MODULE_STREAMS=""virt:av container-tools:3.0"" | 13:51 |
| slaweq | chandankumar: what fix You are talking about? | 13:51 |
| rlandy|rover | same as before | 13:51 |
| * rlandy|rover checks internal | 13:51 | |
| chandankumar | weshay|ruck: done | 13:52 |
| chandankumar | slaweq: branches: ^master$ putting this as a fix | 13:53 |
| slaweq | chandankumar: ok, I will try that | 13:54 |
| weshay|ruck | pojadhav, http://dashboard-ci.tripleo.org/d/Z4vLSmOGk/cockpit?orgId=1 is missing victoria for "upstream victoria last jobs" | 13:55 |
| chandankumar | marios: weshay|ruck I moved tripleo-repos meeting 15 mins later | 13:55 |
| weshay|ruck | pojadhav, bug here: http://dashboard-ci.tripleo.org/d/HkOLImOMk/upstream-and-rdo-promotions?orgId=1 | 13:55 |
| weshay|ruck | as well | 13:55 |
| marios | chandankumar: ack ;) | 13:55 |
| rlandy|rover | weshay|ruck: shoot - double quotes | 13:56 |
| chandankumar | rlandy|rover: sshnaidm|off let's meet after happy hours | 13:57 |
| pojadhav | weshay|ruck, looking | 13:57 |
| rlandy|rover | ok | 13:57 |
| arxcruz | chandankumar: marios which meeting will we have today ? | 13:57 |
| rlandy|rover | export DIB_DNF_MODULE_STREAMS=""virt:av container-tools:3.0"" - ha - that worked | 13:57 |
| marios | arxcruz: ? | 13:58 |
| arxcruz | there is the happy hours (which i put a beer to cold) and the tripleo-repos | 13:59 |
| marios | arxcruz: there is happy friday in 1 min and in 15 mins tripleo-repos | 13:59 |
| marios | arxcruz: 16:55 < chandankumar> marios: weshay|ruck I moved tripleo-repos meeting 15 mins later | 13:59 |
| arxcruz | ah | 13:59 |
| arxcruz | ok | 13:59 |
| marios | arxcruz: bring beer for me too pls | 13:59 |
| arxcruz | https://people.com/royals/queen-elizabeth-us-presidents-photos/ | 14:13 |
| arxcruz | zbr: marios | 14:13 |
| arxcruz | weshay|ruck: ^ | 14:13 |
| chandankumar | sshnaidm|off: https://meet.google.com/ten-izvy-tjp?authuser=0 | 14:19 |
| anbanerj|ruck | we should play a game next time on happy fridays | 14:20 |
| weshay|ruck | bhagyashris, rlandy|rover we're getting access denied on master containers to the redhat reg | 14:20 |
| rlandy|rover | hit that in downstream | 14:21 |
| rlandy|rover | we should have a login in this promoter | 14:21 |
| rlandy|rover | bhagyashris: when was the last time we accessed RH reg? | 14:22 |
| bhagyashris | i see | 14:22 |
| bhagyashris | rlandy|rover, may be the last promotion | 14:22 |
| rlandy|rover | date? | 14:22 |
| bhagyashris | 26th march | 14:24 |
| bhagyashris | rlandy|rover, ^ | 14:24 |
| bhagyashris | it's specific to master | 14:24 |
| bhagyashris | i mean i am talking about the master | 14:25 |
| *** ysandeep is now known as ysandeep|away | 14:27 | |
| bhagyashris | rlandy|rover, i mean if we see the last master promotion happend on 26 march so we can say it was accessed on 26 march | 14:28 |
| pojadhav | weshay|ruck, added new panel for victoria jobs in cockpit : https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33207 | 14:28 |
| bhagyashris | rlandy|rover, should we try to login the reg from promoter server | 14:35 |
| weshay|ruck | bhagyashris, let's chat | 14:35 |
| bhagyashris | weshay|ruck, yes | 14:36 |
| weshay|ruck | bhagyashris, meet.google.com/gwp-ueud-nxk | 14:36 |
| chandankumar | marios: weshay|ruck rlandy|rover invite sent thanks! | 14:51 |
| marios | ack chandankumar | 14:52 |
| weshay|ruck | rlandy|rover, in tripleo-common where is the ubi8 image defined? | 14:53 |
| rlandy|rover | weshay|ruck: getting | 14:54 |
| weshay|ruck | rlandy|rover, https://meet.google.com/gwp-ueud-nxk?authuser=1 | 14:54 |
| rlandy|rover | https://github.com/openstack/tripleo-ci/blob/d492451c1311e6d83cc7816ca93a5381948dea50/roles/build-containers/templates/tripleo-build.sh.j2#L73 | 14:55 |
| rlandy|rover | joining | 14:55 |
| rlandy|rover | registry.access.redhat.com | 14:59 |
| rlandy|rover | registry.redhat.io | 14:59 |
| rlandy|rover | https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image | 15:00 |
| rlandy|rover | containers_base_image: &16_2_containers_base_image | 15:02 |
| rlandy|rover | rh-osbs/ubi8:8.4 | 15:02 |
| *** ykarel is now known as ykarel|away | 15:07 | |
| *** ykarel|away has quit IRC | 15:13 | |
| *** jmasud has joined #oooq | 15:13 | |
| *** amoralej is now known as amoralej|off | 15:33 | |
| marios | rlandy|rover: weshay|ruck: fyi @ https://review.opendev.org/c/openstack/releases/+/785670 client & common branches for testing | 15:39 |
| *** Goneri has quit IRC | 15:47 | |
| *** ysandeep|away is now known as ysandeep | 15:50 | |
| weshay|ruck | sec.. dealing w/ promoter | 15:52 |
| *** udesale has quit IRC | 15:56 | |
| rlandy|rover | https://github.com/openstack/tripleo-ci/blob/master/roles/build-containers/templates/tripleo-build.sh.j2 | 15:57 |
| rlandy|rover | --base | 15:57 |
| rlandy|rover | https://github.com/openstack/tripleo-ci/blob/master/roles/build-containers/templates/tripleo-build.sh.j2#L73 | 15:57 |
| weshay|ruck | marios++ | 15:58 |
| rlandy|rover | https://github.com/openstack/tripleo-common/blob/master/tripleo_common/image/builder/buildah.py#L50 | 16:01 |
| marios | weshay|ruck: http://lists.openstack.org/pipermail/openstack-discuss/2021-April/021741.html | 16:03 |
| *** jpodivin has quit IRC | 16:14 | |
| rlandy|rover | woohoo slaweq++ https://review.opendev.org/c/openstack/neutron/+/785638 | 16:15 |
| rlandy|rover | wallaby is in plav again | 16:15 |
| soniya29 | chandankumar, how to enable cinder backup feature in tempest? | 16:17 |
| marios | zbr: o/ can you please add to your list for re-visit ;) I moved the bits around for 'make it a module' ... py36 is green now too | 16:20 |
| marios | zbr: i have to work on setup.cfg/py so comments there please | 16:20 |
| marios | zbr: when you next have time | 16:20 |
| marios | zbr: thanks | 16:20 |
| marios | zbr: that ;) https://review.opendev.org/c/openstack/tripleo-ci/+/784392 | 16:20 |
| marios | zbr: "py36 is green now too " didn't report yet but https://zuul.openstack.org/status#784392 | 16:21 |
| marios | weshay|ruck: please add to your reviews 19:20 < marios> zbr: that ;) https://review.opendev.org/c/openstack/tripleo-ci/+/784392 | 16:21 |
| weshay|ruck | k | 16:22 |
| rlandy|rover | weshay|ruck: standalone is on tempest | 16:28 |
| rlandy|rover | you can breathe | 16:28 |
| weshay|ruck | k | 16:28 |
| weshay|ruck | well.. we need to see what it pulled | 16:29 |
| marios | have a good weekend folks o/ | 16:35 |
| *** marios is now known as marios|out | 16:35 | |
| weshay|ruck | rlandy|rover, I still can't promote https://trunk.rdoproject.org/centos8-wallaby/component/common/promoted-components/ | 16:46 |
| weshay|ruck | rlandy|rover, fyi.. I promoted victoria's common | 16:46 |
| rlandy|rover | weshay|ruck: k ... will look in a sec - just testing fix for 17 image builds | 16:47 |
| rlandy|rover | periodic-tripleo-rhel-8-buildimage-overcloud-full-rhos-17 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/None : NODE_FAILURE in 0s | 16:56 |
| rlandy|rover | periodic-tripleo-rhel-8-buildimage-ironic-python-agent-rhos-17 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/None : NODE_FAILURE in 0s | 16:56 |
| rlandy|rover | forget that | 16:56 |
| * rlandy|rover looks at wallaby | 16:57 | |
| *** dviroel has quit IRC | 16:57 | |
| *** jpena is now known as jpena|off | 16:58 | |
| rlandy|rover | weshay|ruck: re: wallaby, you want to move component-ci-testing to promoted components or promoted_component to tripleo-ci-testing? | 17:02 |
| weshay|ruck | rlandy|rover, common from consistent to promoted-components | 17:04 |
| rlandy|rover | weshay|ruck: k - trying step by step | 17:05 |
| ysandeep | rlandy|rover, weshay|ruck Hey, The 8.4 rhel-guest-image which we have in 16.2 is older than what del have in compose and downstream infrared jobs is using. we are using .338 and they have 756. Do you want to handle this kind of update via dependency line - rhel_next_u? | 17:06 |
| rlandy|rover | ysandeep: we shoudl update the node | 17:06 |
| rlandy|rover | and the image build | 17:06 |
| ysandeep | rlandy|rover, yup.. node and image build var here - http://download.eng.bos.redhat.com/rhel-8/rel-eng/RHEL-8/RHEL-8.4.0-InternalSnapshot-1.0/compose/BaseOS/x86_64/images/rhel-guest-image-8.4-338.x86_64.qcow2 | 17:07 |
| rlandy|rover | you can update both that and the node | 17:08 |
| ysandeep | but taking learning from container-tool:3 change.. where weshay|ruck requested us to utilize deps line.. do we want to give this a shot first in rhel_next_u line? or that only for minor release 8.x to to 8.x+1 ? | 17:09 |
| ysandeep | rlandy|rover, ^^ | 17:10 |
| *** jmasud has quit IRC | 17:12 | |
| *** marios|out has quit IRC | 17:13 | |
| rlandy|rover | sec | 17:13 |
| rlandy|rover | weshay|ruck: promoted wallaby common consistent -> component-ci-testing - promoted-components | 17:14 |
| *** jmasud has joined #oooq | 17:14 | |
| rlandy|rover | ysandeep: eveything is so hosed downstream | 17:15 |
| rlandy|rover | I am not sure it matters | 17:15 |
| ysandeep | yup, not doing anything before infra is back.. just putting this on plate for discussion | 17:15 |
| ysandeep | lets chat about this on scrum on Monday. | 17:16 |
| rlandy|rover | ysandeep: ack - thanks | 17:16 |
| ysandeep | rlandy|rover, happy weekend! see you on monday :D | 17:19 |
| *** ysandeep is now known as ysandeep|away | 17:19 | |
| rlandy|rover | you too | 17:25 |
| weshay|ruck | rlandy|rover, all gate failures reviewed :) | 17:36 |
| weshay|ruck | we're good | 17:36 |
| *** dsneddon has quit IRC | 17:36 | |
| rlandy|rover | weshay|ruck: nice - quickstart patch in gate | 17:49 |
| rlandy|rover | looking at victoria now | 17:50 |
| weshay|ruck | pojadhav, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf_py3/telegraf.d/last_promotions.conf | 17:55 |
| rlandy|rover | weshay|ruck: what was the problem with promoting wallaby common? worked ok | 17:57 |
| weshay|ruck | rlandy|rover, extendedhash | 18:02 |
| weshay|ruck | rlandy|rover, dlrn wasn't updated | 18:02 |
| rlandy|rover | extended-hash is ignored upstream | 18:03 |
| weshay|ruck | rlandy|rover, the dlrn server code was not updated | 18:03 |
| weshay|ruck | you run the promotion w/ extendedhash None.. and you get a 404 | 18:04 |
| weshay|ruck | on wallaby only | 18:04 |
| rlandy|rover | never added the extended-hash | 18:04 |
| weshay|ruck | it's in the script we use to promote component | 18:04 |
| rlandy|rover | dlrnapi --url https://trunk.rdoproject.org/api-centos8-wallaby --username $USER repo-promote --commit-hash d6d1b39206972a7ab8e405294699eec7c26f6545 --distro-hash 3d4d4101cd02b32267fc67f67592b0ed9c674764 --promote-name promoted-components | 18:04 |
| rlandy|rover | { | 18:04 |
| weshay|ruck | rlandy|rover, see #rhosops | 18:05 |
| rlandy|rover | passing None is tricky | 18:05 |
| rlandy|rover | sometimes gets intrep as a string and sometimes as a null | 18:06 |
| rlandy|rover | just skipped it upstream | 18:06 |
| rlandy|rover | whatever - it's done | 18:06 |
| weshay|ruck | works fine on other branches | 18:09 |
| rlandy|rover | weshay|ruck: victoria is running now ... running on a new hash ... | 18:13 |
| rlandy|rover | last results were not bad | 18:13 |
| weshay|ruck | ya.. I triggered it.. hoping the new common fixes a bunch | 18:13 |
| rlandy|rover | periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-victoria | 18:13 |
| rlandy|rover | periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria | 18:13 |
| rlandy|rover | periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-victoria | 18:13 |
| rlandy|rover | periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-victoria | 18:13 |
| rlandy|rover | periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-victoria | 18:13 |
| rlandy|rover | ^^ failures | 18:13 |
| weshay|ruck | rlandy|rover, full-tempest-scenario last results may have been a fluke | 18:14 |
| weshay|ruck | not worth debugging yet | 18:14 |
| rlandy|rover | yep - that one passed the run before | 18:14 |
| weshay|ruck | rlandy|rover, we have a problem w/ train | 18:14 |
| rlandy|rover | looking at the others | 18:14 |
| weshay|ruck | emailed u | 18:14 |
| rlandy|rover | 39 ad ipa seem to fail all the time | 18:14 |
| weshay|ruck | that's a big rev.. for dib | 18:14 |
| * rlandy|rover looks | 18:14 | |
| rlandy|rover | weshay|ruck: oh wow - that is an old diskimage-builder | 18:16 |
| rlandy|rover | other branches are also pinned | 18:16 |
| rlandy|rover | why not move those | 18:16 |
| rlandy|rover | - project: diskimage-builder | 18:17 |
| rlandy|rover | tags: | 18:17 |
| rlandy|rover | victoria: | 18:17 |
| rlandy|rover | ah - not pinned in victoria | 18:17 |
| rlandy|rover | or ussuri | 18:17 |
| rlandy|rover | ok | 18:17 |
| *** dviroel has joined #oooq | 18:17 | |
| weshay|ruck | rlandy|rover, the ipa failure in victoria looks legit, maybe missing a backport? | 18:53 |
| rlandy|rover | on it | 18:53 |
| weshay|ruck | rockin | 18:53 |
| weshay|ruck | thanks | 18:53 |
| *** jmasud has quit IRC | 19:16 | |
| *** jmasud has joined #oooq | 19:19 | |
| rlandy|rover | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-victoria | 19:26 |
| *** slaweq has quit IRC | 19:26 | |
| rlandy|rover | this has been failing since 03/30 | 19:26 |
| rlandy|rover | really? | 19:27 |
| *** jmasud has quit IRC | 19:40 | |
| rlandy|rover | what happened on 03/30? | 19:41 |
| rlandy|rover | whole world feel apart | 19:41 |
| rlandy|rover | weshay|ruck: the security component is very old in victoria | 19:48 |
| rlandy|rover | there is another issue showing up in all the lines | 19:48 |
| rlandy|rover | think we should fix that first and see if it gets better | 19:48 |
| weshay|ruck | k | 19:49 |
| * weshay|ruck looks | 19:49 | |
| weshay|ruck | The error was: 'ansible_fqdn' | 19:51 |
| weshay|ruck | https://logserver.rdoproject.org/openstack-component-security/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-security-victoria/59a0b73/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 19:51 |
| rlandy|rover | yep | 19:51 |
| rlandy|rover | 2021-04-09 15:50:36.849095 | fa163ebf-5861-315e-4ede-000000000d7c | FATAL | set main facts | undercloud | error={"msg": "The task includes an option with an undefined variable. The error was: 'ansible_fqdn' is undefined\n\nThe error appears to be in '/usr/share/ansible/roles/tripleo_ipa_registration/tasks/main.yml': line 27, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe | 19:52 |
| rlandy|rover | offending line appears to be:\n\n\n- name: set main facts\n ^ here\n"} | 19:52 |
| rlandy|rover | everywhere | 19:52 |
| rlandy|rover | https://opendev.org/x/tripleo-ipa/src/commit/e65f11baedf6f19ec31dca065c69fa4324f1dd88/tripleo_ipa/playbooks/undercloud-ipa-install.yaml#L105 | 19:54 |
| rlandy|rover | error is there | 19:54 |
| rlandy|rover | ^^ logging that | 19:54 |
| *** jmasud has joined #oooq | 19:54 | |
| *** jmasud has quit IRC | 19:56 | |
| rlandy|rover | 2021-04-09 19:40:32 | 2021-04-09 19:40:32.868116 | fa163e92-5522-6085-d39d-0000000004a2 | FATAL | Install - IPA client test | undercloud | error={"msg": "The task includes an option with an undefined variable. The error was: 'ansible_fqdn' is undefined\n\nThe error appears to be in '/usr/share/ansible/roles/ipaclient/tasks/install.yml': line 30, column 3, but may\nbe elsewhere in the file depending on the exact syntax | 19:57 |
| rlandy|rover | problem.\n\nThe offending line | 19:57 |
| rlandy|rover | taking out fs039 | 19:57 |
| weshay|ruck | rlandy|rover, lot of changes | 20:01 |
| weshay|ruck | https://www.diffchecker.com/1TemA8HF | 20:01 |
| rlandy|rover | weshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1923248 logged CIX | 20:05 |
| openstack | Launchpad bug 1923248 in tripleo "victoria promotion: multinode-ipa and fs039 tests are failing "ansible_fqdn' is undefined"" [Critical,Triaged] | 20:05 |
| rlandy|rover | will ping sec team | 20:05 |
| rlandy|rover | too many changes to follow | 20:05 |
| rlandy|rover | the code throwing the error has been there for a year | 20:05 |
| rlandy|rover | failing since end of march | 20:05 |
| rlandy|rover | nothing new | 20:05 |
| weshay|ruck | ya | 20:05 |
| rlandy|rover | component line has been failing for ages | 20:06 |
| weshay|ruck | rlandy|rover, if everything else passes... I would say let's promote | 20:06 |
| rlandy|rover | ack | 20:06 |
| rlandy|rover | watchimng current run | 20:06 |
| rlandy|rover | weshay|ruck: want to meet on sunday or leave this until monday? | 20:07 |
| weshay|ruck | rlandy|rover, fs020 is in tempest | 20:07 |
| weshay|ruck | :) | 20:07 |
| rlandy|rover | ha Merged openstack/tripleo-quickstart master: Bump to container-tools:3.0 for upstream CI https://review.opendev.org/c/openstack/tripleo-quickstart/+/785408 | 20:08 |
| rlandy|rover | should be in sync now | 20:08 |
| rlandy|rover | I think the fs010 failure is known | 20:09 |
| weshay|ruck | rlandy|rover, scenario010 | 20:10 |
| weshay|ruck | rlandy|rover, you have time to promote today? | 20:10 |
| rlandy|rover | weshay|ruck: sure | 20:11 |
| weshay|ruck | rlandy|rover, can you rebase https://review.opendev.org/c/openstack/tripleo-quickstart/+/785412 | 20:21 |
| rlandy|rover | done | 20:22 |
| rlandy|rover | leaving off -1 - we plan to promote? | 20:22 |
| rlandy|rover | ussuri still does not have a clean ovb run | 20:23 |
| rlandy|rover | so I would leave that | 20:23 |
| rlandy|rover | pinged sec-interop re: ipa bug | 20:23 |
| *** derekh has quit IRC | 20:38 | |
| weshay|ruck | rlandy|rover, I'll trigger ussuri next.. I just updated the common component today | 20:43 |
| rlandy|rover | ok cool | 20:44 |
| rlandy|rover | we'll probably need to rebase ussuri release on victoria file chnage | 20:45 |
| rlandy|rover | will do that on weekend | 20:45 |
| rlandy|rover | periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-victoria sucess | 20:46 |
| rlandy|rover | good | 20:46 |
| rlandy|rover | weshay|ruck: ^^ one ovb passing | 20:46 |
| rlandy|rover | https://review.opendev.org/c/openstack/tripleo-ci/+/784040 should fix 10 | 20:49 |
| weshay|ruck | ah I see.. just merged | 20:50 |
| rlandy|rover | fix for check only | 20:51 |
| weshay|ruck | rlandy|rover, I think I have this ready now https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/785040 | 21:01 |
| weshay|ruck | few new deps keep popping up | 21:01 |
| rlandy|rover | very cool | 21:02 |
| weshay|ruck | that meta: end_play didn't work .. yay ansible | 21:02 |
| rlandy|rover | ugh - gerrit is sooo slow | 21:03 |
| rlandy|rover | clicking through the review takes ages | 21:03 |
| rlandy|rover | surprised the linter didn't doc you | 21:03 |
| rlandy|rover | see extra lines at the end of all files | 21:04 |
| rlandy|rover | weshay|ruck: want to merge that? | 21:05 |
| rlandy|rover | seems minimal damage | 21:05 |
| weshay|ruck | which? | 21:05 |
| rlandy|rover | only runs on roles/repoclosure | 21:05 |
| weshay|ruck | ya.. sure :) | 21:06 |
| rlandy|rover | weshay|ruck: train is also running now | 21:06 |
| rlandy|rover | doing not badly | 21:06 |
| weshay|ruck | rlandy|rover, I should kill it and run ussuri | 21:07 |
| rlandy|rover | can do | 21:07 |
| rlandy|rover | if we want to promote in order | 21:07 |
| weshay|ruck | rlandy|rover, as soon as fs001 victoria passes let's promote | 21:08 |
| weshay|ruck | 35 passed | 21:08 |
| rlandy|rover | yep - just reporetd success | 21:09 |
| rlandy|rover | voted on https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/785040 | 21:10 |
| *** jmasud has joined #oooq | 21:11 | |
| rlandy|rover | like watching a pot boil | 21:14 |
| rlandy|rover | go fs001, go | 21:14 |
| weshay|ruck | k.. let's do this | 21:21 |
| weshay|ruck | you on the promoter? | 21:22 |
| weshay|ruck | rlandy|rover, ^ | 21:22 |
| *** jmasud has quit IRC | 21:25 | |
| *** jmasud_ has joined #oooq | 21:25 | |
| rlandy|rover | ack but getting on tmux | 21:26 |
| rlandy|rover | there | 21:26 |
| weshay|ruck | running | 21:27 |
| weshay|ruck | rlandy|rover, so it will fail after the push... | 21:28 |
| rlandy|rover | same as before? | 21:28 |
| weshay|ruck | I'll remove the registry_client.. and restart | 21:28 |
| weshay|ruck | ya | 21:28 |
| rlandy|rover | why? | 21:28 |
| rlandy|rover | login error? | 21:28 |
| rlandy|rover | expect that? | 21:28 |
| weshay|ruck | ya.. until that script is fixed | 21:28 |
| rlandy|rover | ok- understand | 21:28 |
| weshay|ruck | rlandy|rover, +2 https://review.opendev.org/c/openstack/tripleo-quickstart/+/785412 | 21:34 |
| rlandy|rover | thanks | 21:34 |
| rlandy|rover | what do you want to do about ussuri? | 21:34 |
| weshay|ruck | we'll let it ride over the weekend | 21:34 |
| rlandy|rover | ok | 21:34 |
| rlandy|rover | I'll w+ /785412 once promotion complete | 21:35 |
| weshay|ruck | rlandy|rover, ok.. here they come https://hub.docker.com/u/tripleovictoria | 21:37 |
| rlandy|rover | nice | 21:39 |
| rlandy|rover | getting there | 21:39 |
| rlandy|rover | since when does the licensing print out? | 21:44 |
| weshay|ruck | meh | 21:45 |
| weshay|ruck | don't know | 21:45 |
| *** jmasud_ has quit IRC | 21:53 | |
| weshay|ruck | rlandy|rover, hook me up here on the dep https://review.opendev.org/c/openstack/tripleo-ci/+/785690 | 22:02 |
| rlandy|rover | weshay|ruck: voted | 22:03 |
| rlandy|rover | going to give the 17 patch a run again | 22:03 |
| rlandy|rover | hopefully we will get a node this time | 22:03 |
| weshay|ruck | 2 failed in 20.. we'll circle back around to minor stuff after getting through ussuri and train | 22:04 |
| *** jmasud has joined #oooq | 22:04 | |
| rlandy|rover | periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-victoria passed | 22:04 |
| rlandy|rover | ok | 22:04 |
| rlandy|rover | ok | 22:04 |
| rlandy|rover | ooh - we 're done? | 22:05 |
| rlandy|rover | openstack-base | 22:05 |
| weshay|ruck | ok.. should fail here | 22:05 |
| rlandy|rover | internal node failure agian | 22:06 |
| rlandy|rover | will ahev to try another day | 22:06 |
| rlandy|rover | got the fail | 22:08 |
| rlandy|rover | weshay|ruck: edit the .sh to put victoria before master? | 22:11 |
| weshay|ruck | https://trunk.rdoproject.org/centos8-victoria/current-tripleo/delorean.repo.md5 | 22:12 |
| weshay|ruck | rlandy|rover, we could ya | 22:12 |
| weshay|ruck | http://images.rdoproject.org/centos8/victoria/rdo_trunk/?C=M;O=D | 22:12 |
| rlandy|rover | weshay|ruck: it's ok - went by fast | 22:12 |
| weshay|ruck | https://hub.docker.com/r/tripleovictoria/openstack-base/tags?page=1&ordering=last_updated | 22:12 |
| weshay|ruck | rlandy|rover++ | 22:12 |
| weshay|ruck | rlandy|rover, good shabbos | 22:13 |
| * weshay|ruck shuts the promoter down | 22:13 | |
| rlandy|rover | current-tripleo/ 2021-04-09 17:42 nice | 22:13 |
| rlandy|rover | working patch | 22:13 |
| rlandy|rover | hold on | 22:13 |
| weshay|ruck | k | 22:13 |
| rlandy|rover | https://review.opendev.org/c/openstack/tripleo-quickstart/+/785412 | 22:13 |
| rlandy|rover | w+'ed | 22:13 |
| rlandy|rover | weshay|ruck: ok .. logging off | 22:14 |
| weshay|ruck | rlandy|rover, 0/ | 22:14 |
| weshay|ruck | good job | 22:14 |
| rlandy|rover | have a good shabbos | 22:14 |
| weshay|ruck | you 2 | 22:14 |
| rlandy|rover | will be around on sunday of needed | 22:14 |
| rlandy|rover | manager stuff to work on | 22:14 |
| rlandy|rover | by | 22:14 |
| rlandy|rover | bye | 22:14 |
| *** rlandy|rover has quit IRC | 22:15 | |
| *** jmasud has quit IRC | 22:19 | |
| *** jmasud has joined #oooq | 22:32 | |
| *** jmasud has quit IRC | 22:38 | |
| *** derekh has joined #oooq | 22:47 | |
| *** derekh has quit IRC | 22:56 | |
| *** derekh has joined #oooq | 23:05 | |
| *** apetrich has quit IRC | 23:27 | |
| *** tosky has quit IRC | 23:29 | |
| *** derekh has quit IRC | 23:57 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!