| *** ysandeep|away is now known as ysandeep | 01:19 | |
| *** ysandeep is now known as ysandeep|afk | 02:00 | |
| *** marios is now known as marios|ruck | 05:12 | |
| *** ysandeep|afk is now known as ysandeep | 05:51 | |
| ysandeep | hey marios|ruck good morning o/ ,fyi.. i have logged a bug for queens: https://bugs.launchpad.net/tripleo/+bug/1933922 | 05:53 |
|---|---|---|
| ysandeep | I understand we are talking about eod for queens but reading "[TripleO] moving stable/stein and stable/queens to End of Life" I am not sure if we are there already. | 05:53 |
| marios|ruck | ysandeep: ack thanks will check | 05:55 |
| marios|ruck | ysandeep: for queens we are 'almost' there i think this or next week we will propose it | 05:55 |
| marios|ruck | ysandeep: i mean the actual eol review against releases repo so it should be soon unless something unexpected | 05:56 |
| ysandeep | ack thanks, i wrote the bug since we have plans to add some ci coverage in osp13 and having green upstream jobs will help creating similiar downstream jobs. | 05:57 |
| chandankumar | ysandeep: Do we need tripleo-operator-ansible for queens also? | 05:57 |
| chandankumar | ysandeep: It is packaged till stein | 05:57 |
| chandankumar | *tripleo-operator-ansible rpm | 05:58 |
| ysandeep | chandankumar: hi, is it utilized in upstream queens? | 05:59 |
| chandankumar | ysandeep: currently nope | 05:59 |
| chandankumar | for downstream RHOS-13 job, we might not be needing tripleo-operator-ansible rpm as it is not used anywhere in the deployment, but i need to cross check | 06:00 |
| ysandeep | I can't think of any reasons we would tripleo-operator-ansible rpm in 13/queens if its not utilized in upstream queens currently. | 06:01 |
| ysandeep | may be sshnaidm have better suggestion here. | 06:01 |
| sshnaidm | me neither | 06:02 |
| sshnaidm | in ci we use from ussuri: https://github.com/openstack/tripleo-quickstart-extras/blob/68129460d83fc5c49b014fa4706dd971414f8f1e/roles/overcloud-prep-images/tasks/overcloud-prep-images.yml#L47-L49 | 06:04 |
| ysandeep | sshnaidm: thanks! | 06:04 |
| ysandeep | chandankumar: ^^ | 06:04 |
| chandankumar | ysandeep: sshnaidm ok, thanks, then we are good to remove tripleo-operator-ansible then | 06:06 |
| ysandeep | chandankumar: sorry I didn't get you(remove from where?), when you said It is packaged till stein, i thought its not in queens already. | 06:09 |
| chandankumar | ysandeep: Yesterday during scrum call, wes mentioned you are working on adding downstream RHOS-13 CI job | 06:09 |
| ysandeep | yes I am | 06:10 |
| chandankumar | using TQ and I am working on removing tripleo-operator-ansible from TQE-requirements.txt file | 06:10 |
| chandankumar | Since it is going to affect all release | 06:10 |
| chandankumar | from upstream side, from stein onwards, tripleo-operator-ansible is used so we are replacing the git instllation part with rpm | 06:11 |
| chandankumar | So I thought it might be used for queens also so asked about the rpm | 06:11 |
| chandankumar | as if we remove it will break rhos-13 deployment | 06:11 |
| ysandeep | chandankumar, ack make sense now, thanks! | 06:12 |
| *** jpena|off is now known as jpena | 07:00 | |
| zbr | were permissions issue with the google sheet addresses? | 07:28 |
| anbanerj|rover | marios|ruck, O/ good morning | 07:35 |
| marios|ruck | anbanerj|rover: o/ hello | 07:36 |
| marios|ruck | anbanerj|rover: T - 32 hours and counting | 07:36 |
| marios|ruck | (but whos counting) | 07:36 |
| anbanerj|rover | till tomorrow's retro meeting? :D | 07:58 |
| marios|ruck | anbanerj|rover: yeah end of shift | 07:58 |
| anbanerj|rover | haha | 07:59 |
| anbanerj|rover | marios|ruck, for victoria hash https://trunk.rdoproject.org/api-centos8-victoria/api/civotes_agg_detail.html?ref_hash=4843abdd3c92352cfc910c00b2ca0616 only periodic-tripleo-ci-centos-8-scenario010-kvm-standalone-victoria failed | 08:25 |
| anbanerj|rover | since sc 010 is an ongoing bug should I remove it from criteria and restart the promotion server? | 08:26 |
| marios|ruck | anbanerj|rover: that job is not in the criteria i think looking at http://38.102.83.109/config/CentOS-8/victoria.yaml | 08:31 |
| anbanerj|rover | oh, right forgot about that, so it should be already promoted, right? | 08:32 |
| marios|ruck | anbanerj|rover: no see logs | 08:32 |
| anbanerj|rover | checking | 08:32 |
| marios|ruck | anbanerj|rover: * 2021-06-29 08:27:14,721 3079817 WARNING promoter Candidate hash 'aggregate: 4843abdd3c92352cfc910c00b2ca0616, commit: fb29b03c1f3a4b1e1f89e1ef6a01620376dd3c99, distro: 7ee3df660458411473614409814eb77d1aba0471, extended: None, component: validation, timestamp: 1624940111': missing jobs {'periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-victoria', | 08:32 |
| marios|ruck | 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-victoria', 'periodic-tripleo-centos-8-buildimage-overcloud-full-victoria', 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-victoria', 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039-victoria'} | 08:32 |
| * zbr typo of the day ruin instead of run ... | 08:33 | |
| marios|ruck | anbanerj|rover: might be worth digging a bit there not sure what happened e.g. featureset020 didn't report at all? 'SKIPPED' @ https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-victoria | 08:37 |
| anbanerj|rover | marios|ruck, yes, looks like 5 jobs didnt run or report. And I was so happy seeing only one failure :( Lemme check | 08:37 |
| marios|ruck | anbanerj|rover: before you check | 08:38 |
| marios|ruck | anbanerj|rover: please go post a testproject because tripleo-ci-testing didnt update yet | 08:38 |
| marios|ruck | https://trunk.rdoproject.org/centos8-victoria/tripleo-ci-testing/delorean.repo.md5 | 08:38 |
| marios|ruck | 4843abdd3c92352cfc910c00b2ca0616 | 08:38 |
| marios|ruck | anbanerj|rover: ^^^ | 08:38 |
| marios|ruck | anbanerj|rover: so post a a testprojct with those missing jobs and lets see what happens with them | 08:38 |
| anbanerj|rover | yep | 08:38 |
| *** ykarel is now known as ykarel|lunch | 08:38 | |
| arxcruz | bhagyashris: akahat hey, what's the reviews you want me to review regarding the promoter ? | 08:39 |
| marios|ruck | zbr: "openstack overcloud updgrade ruin --templates... " has a certain ring to it ;) | 08:39 |
| zbr | indeed | 09:14 |
| *** bhagyashris_ is now known as bhagyashris | 09:15 | |
| sshnaidm | chandankumar, ysandeep please take a look in your time: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/796522 | 09:37 |
| chandankumar | checking | 09:37 |
| anbanerj|rover | Hey dpawlik good morning. I am trying to curl elk.review.rdoproject.org:9200 , but it is returning 403. http://paste.openstack.org/show/807015/ I am using the kibana user. Do I need to use some other user? | 09:54 |
| dpawlik | anbanerj|rover: hey, its only allowed from your monitoring host | 09:56 |
| dpawlik | anbanerj|rover: one sec | 09:56 |
| anbanerj|rover | dpawlik, yes, I am trying from the server. I can reach it, but it is returning 403 | 09:56 |
| bhagyashris | arxcruz, this one https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/33815 | 09:57 |
| dpawlik | anbanerj|rover: and the error is true: https://softwarefactory-project.io/r/gitweb?p=software-factory/sf-config.git;a=blob;f=ansible/roles/sf-elasticsearch/templates/roles.yml.j2;h=9939aaae280fca1e5e16d596f127781f7790129b;hb=HEAD#l66 because it does not have "indices:data/read/search" | 10:02 |
| ysandeep | sshnaidm: done, chandankumar already beat me to +w | 10:04 |
| ysandeep | marios|ruck, anbanerj|rover fyi.. https://bugs.launchpad.net/tripleo/+bug/1933962 | 10:07 |
| marios|ruck | thanks ysandeep | 10:15 |
| marios|ruck | ykarel|lunch: o/ hey when you are back can i ask you to help me hold a node with a testproject for debug? | 10:15 |
| marios|ruck | jpena: o/ hey can i ask you for help to hold a node for debug @ https://review.rdoproject.org/r/c/testproject/+/34321 | 10:27 |
| marios|ruck | jpena: are we mean to do that ourselves now /me vaguely remembers some email? | 10:27 |
| marios|ruck | jpena: or maybe just confusing things | 10:28 |
| anbanerj|rover | hey zbr do you have some time today? Need a little help with changing ER's config to RDO | 10:28 |
| jpena | marios|ruck: iirc you should have an API key to do it yourselves, but I can do it quickly | 10:29 |
| marios|ruck | jpena: i would be very grateful if you could do it? I can ask the team later about the api key i think we must have it documented somewhere | 10:30 |
| jpena | marios|ruck: done. Let's see if the cloud behaves and you don't get a node_failure. Vexxhost is having a bumpy day | 10:31 |
| marios|ruck | jpena: do you need my ssh pub key or am i meant to use the 13:29 < jpena> marios|ruck: iirc you should have an API key | 10:31 |
| marios|ruck | jpena: thank you very much | 10:31 |
| jpena | marios|ruck: I think your ssh keys are already injected in those nodes via the oooq playbooks? | 10:31 |
| marios|ruck | jpena: yes i noticed some of those node_failures today blocking master and train lines http://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-main http://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable4 | 10:32 |
| *** ykarel|lunch is now known as ykarel | 10:32 | |
| marios|ruck | jpena: ok thank you i can try it once it is done (can you ping me the ip so i can check it later?) | 10:32 |
| marios|ruck | jpena: sorry and thank you for help | 10:32 |
| marios|ruck | jpena: ah maybe we don't have it yet | 10:32 |
| jpena | marios|ruck: the job has not started yet, I'll ping you with the IP once we get it | 10:32 |
| marios|ruck | jpena: thank you! i will stop harrassing you now | 10:33 |
| marios|ruck | (for now ;) ) | 10:33 |
| marios|ruck | jpena: don't worry i'll watch it from https://review.rdoproject.org/zuul/status#34321 i can probably grep for the ip from the logs | 10:34 |
| zbr | anbanerj|rover: Yes. ping me when ready and give me 5mins to make another coffee | 10:36 |
| anbanerj|rover | zbr, thanks | 10:37 |
| *** jpena is now known as jpena|lunch | 11:34 | |
| chandankumar | akahat: ping, let me know when around, will cherry pick dockerfile code and test all together | 11:43 |
| anbanerj|rover | Hey dpawlik , With the kibana user from the kibana dashboard we can query stuff like "message:"async task did not" AND tags:"console"" But to do the same through API to elasticsearch do we need a different user or different permission? This is how we are trying - http://paste.openstack.org/show/807021/ are we missing something here? | 11:47 |
| dpawlik | anbanerj|rover: looks ok... I can try to add this indices into the elk, but I need to check it first | 11:50 |
| anbanerj|rover | dpawlik, thanks | 11:50 |
| anbanerj|rover | dpawlik, but how are we able to query without any permissions through the dashboard? Does it use some other user in the backend? | 11:52 |
| dpawlik | anbanerj|rover: Thats a good question, but I did not analyze the java code in Elasticsearch in Opendistro so maybe community will know | 11:59 |
| chandankumar | weshay|ruck: rlandy akahat is not around due to power cut at his home from morning itself | 12:14 |
| rlandy | ok | 12:15 |
| rlandy | working on a bug atm | 12:15 |
| rlandy | ysandeep: left comment on https://bugs.launchpad.net/tripleo/+bug/1933962 | 12:34 |
| rlandy | let me know when you have time to chat about it | 12:34 |
| *** jpena|lunch is now known as jpena | 12:36 | |
| ysandeep | rlandy, ack | 12:37 |
| bhagyashris | rlandy, ua/tc/pm sync | 13:01 |
| *** amoralej|off is now known as amoralej | 13:05 | |
| soniya | marios|ruck, anbanerj|rover, Is tempest-multinode-full-py3 job unstable for a while? | 13:11 |
| tosky | soniya: I'm not sure if it's the same issue, but the nova team has reported an issue with the nova live migration test | 13:13 |
| marios|ruck | soniya: looks like it, fyi you can check any job like https://zuul.opendev.org/t/openstack/builds?job_name=tempest-multinode-full-py3 | 13:14 |
| marios|ruck | soniya: not something i'm aware of or tracking (i.e. not aware of bug etc on that) | 13:14 |
| soniya | marios|ruck, I checked, I just wanted to confirm. | 13:16 |
| soniya | marios|ruck, ack | 13:16 |
| soniya | tosky, its non-voting in devstack and voting in tempest and failing continuosly | 13:16 |
| tosky | soniya: but which test? | 13:21 |
| tosky | soniya: only test_live_migration_with_trunk is failing, and that's known to nova | 13:21 |
| bhagyashris | arxcruz, zbr, sshnaidm, rlandy, marios|ruck , ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, weshay|ruck , chandankumar, frenzy_friday, dviroel anbanerj|rover | 13:25 |
| bhagyashris | TripleO CI Community Meeting in 5 mins | 13:25 |
| bhagyashris | https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2021-06-29-Community-call | 13:25 |
| bhagyashris | feel free to add topic at line 32 | 13:26 |
| soniya | tosky, yeah, got it | 13:27 |
| soniya | tosky, ack | 13:28 |
| chandankumar | ysandeep: great work on ruck rover tooling :-) | 13:46 |
| ysandeep | chandankumar: thanks! | 13:47 |
| weshay|ruck | bhagyashris, do we want to promoter sync? | 14:00 |
| * anbanerj|rover lunch | 14:02 | |
| rlandy | chandankumar: weshay|ruck: 34fdb60ce9fa1da3ac65647f3da3bf49 | 14:12 |
| rlandy | chandankumar: http://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp16-2/api/civotes_agg_detail.html?ref_hash=34fdb60ce9fa1da3ac65647f3da3bf49 | 14:13 |
| weshay|ruck | chandankumar, http://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp16-2/api/civotes_agg_detail.html?ref_hash=34fdb60ce9fa1da3ac65647f3da3bf49 | 14:14 |
| sshnaidm | chandankumar, marios|ruck take a look please in your time: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/797463 | 14:20 |
| chandankumar | weshay|ruck: https://sf.hosted.upshift.rdu2.redhat.com/images/redhat8/ | 14:20 |
| marios|ruck | sshnaidm: ack adding to reviews | 14:23 |
| chandankumar | rlandy: http://paste.openstack.org/raw/807027/ | 14:23 |
| chandankumar | rlandy: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32679/31/ci-scripts/dlrnapi_promoter/qcow_client.py | 14:26 |
| rlandy | chandankumar: b8774a8394f57d3f7ed3d2249804c660 | 14:39 |
| rlandy | http://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp17/api/civotes_agg_detail.html?ref_hash=b8774a8394f57d3f7ed3d2249804c660 | 14:42 |
| rlandy | chandankumar: current-tripleo 291dc899e01ebd8eafd5835e065a86f4 | 14:42 |
| pojadhav | weshay|ruck, hey can we chat now? | 14:43 |
| ysandeep | marios|ruck, anbanerj|rover hey have you noticed something similiar in other upstream jobs: https://2593fe2c1676d625fafc-b5f24c1063ad83372f17d0890078a578.ssl.cf5.rackcdn.com/773034/6/gate/tripleo-ci-centos-8-scenario001-standalone/29b8bb2/logs/undercloud/home/zuul/standalone_deploy.log ? | 14:43 |
| rlandy | chandankumar: http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp17/tripleo-ci-testing/delorean.repo.md5 | 14:43 |
| anbanerj|rover | ysandeep, I have seen this a few times (ClientManager' object has no attribute 'sdk_connection) when there is originally some other failure and then in the exception handling or so we see this | 14:45 |
| marios|ruck | ysandeep: i haven't seen that | 14:46 |
| ysandeep | anbanerj|rover, (ClientManager' object has no attribute 'sdk_connection appears in which log file? | 14:46 |
| marios|ruck | anbanerj|rover: i don't know if the sdk_connection is the root | 14:46 |
| marios|ruck | anbanerj|rover: ysandeep: i think more like 2021-06-29 05:38:25.563827 | fa163ef5-cc2b-7982-a6ef-00000000309c | FATAL | Gather podman infos | standalone | error={"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} | 14:46 |
| marios|ruck | ysandeep: anbanerj|rover: afaics there is only one there https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario001-standalone | 14:47 |
| marios|ruck | ysandeep: anbanerj|rover: the one ysandeep pointed to, the other fail there is tempest | 14:47 |
| marios|ruck | ysandeep: anbanerj|rover: so let's see if it happens again we can file a bug | 14:47 |
| marios|ruck | thanks for pointer ysandeep | 14:47 |
| marios|ruck | anbanerj|rover: do the ones you saw 17:45 < anbanerj|rover> ysandeep, I have seen this a few times (ClientManager' object has no attribute | 14:47 |
| marios|ruck | anbanerj|rover: do those ones have the same FATAL line? ^^^ | 14:47 |
| anbanerj|rover | lemme see if I can find them The ones I saw were from 2 weeks back | 14:48 |
| ysandeep | marios|ruck, ack thanks! happened to my patch twice, will wait for more jobs to run, before recheck | 14:48 |
| marios|ruck | anbanerj|rover: ah not to worry then i thought you mean today | 14:48 |
| anbanerj|rover | no no | 14:48 |
| marios|ruck | anbanerj|rover: might be worth keeping an eye on that please i am going within the next hour ^^ otherwise I will file it in the morning | 14:49 |
| ysandeep | arxcruz: when you have time, appreciate some suggestions on https://bugs.launchpad.net/tripleo/+bug/1933922 | 14:54 |
| *** ysandeep is now known as ysandeep|dinner | 14:56 | |
| arxcruz | ysandeep|dinner: kopecmartin we need to backport --exclude-list to queens ^ | 14:57 |
| weshay|ruck | marios|ruck, anbanerj|rover can we chat about master | 15:01 |
| weshay|ruck | storage is still an issue.. obviously https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/b596a61/logs/undercloud/var/log/tempest/stestr_results.html.gz | 15:01 |
| marios|ruck | weshay|ruck: yeah https://bugs.launchpad.net/tripleo/+bug/1933639/comments/9 & posted testproject @ https://review.rdoproject.org/r/c/testproject/+/34321 | 15:02 |
| marios|ruck | weshay|ruck: i asked for a hold on that ^^^ but it hit the node_failure | 15:02 |
| weshay|ruck | ya | 15:02 |
| marios|ruck | weshay|ruck: i went chasing master earlier and found a candidate sec | 15:03 |
| marios|ruck | weshay|ruck: candidate @ aggregate: 6439b21a91a11b464ad5b2cc147e81cd (via http://10.0.148.74/promoter_logs/centos8_master.log) & posted testproject for missing fs1/fs35 @ https://review.rdoproject.org/r/c/testproject/+/34325 (& can use for verify https://bugs.launchpad.net/tripleo/+bug/1933448 ) | 15:03 |
| marios|ruck | weshay|ruck: * 2021-06-29 11:39:09,213 427072 WARNING promoter Candidate hash 'aggregate: 6439b21a91a11b464ad5b2cc147e81cd, commit: c90043f853e34cf3cdbccbba821f6546c9f472fa, distro: 181dc17f38c251200d8c37d56909acb175e0f407, extended: None, component: network, timestamp: 1624897080': missing jobs {'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master', | 15:04 |
| marios|ruck | 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master'} | 15:04 |
| * weshay|ruck looks | 15:04 | |
| marios|ruck | weshay|ruck: lemme put that into the gchat might be better | 15:04 |
| marios|ruck | well fs1 will still hit damn i forgot weshay|ruck | 15:04 |
| marios|ruck | so won't help us cos of fs1 :/ | 15:04 |
| rlandy | chandankumar: hey - how did the 17 promotion go? | 15:04 |
| marios|ruck | assuming they don't just fail on the node_failure anyway weshay|ruck | 15:04 |
| marios|ruck | weshay|ruck: we aren't easily gonna promote anything today | 15:04 |
| marios|ruck | weshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1931226/comments/7 | 15:05 |
| weshay|ruck | overcloud deploy failed on https://trunk.rdoproject.org/api-centos8-master/api/civotes_agg_detail.html?ref_hash=6439b21a91a11b464ad5b2cc147e81cd | 15:06 |
| weshay|ruck | that's not a candidate.. unless a rerun works | 15:06 |
| marios|ruck | weshay|ruck: your url is missing -uc but yeah i saw that too... so did you update the criteria? i saw a master.yaml.bak_wes there | 15:07 |
| marios|ruck | weshay|ruck: and those logs missing jobs {'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master', 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master'} | 15:07 |
| marios|ruck | weshay|ruck: so was that use modification? | 15:07 |
| weshay|ruck | https://trunk.rdoproject.org/api-centos8-master/api/civotes_agg_detail.html?ref_hash=449b329e61f80b511ad8b619077c41b5 | 15:07 |
| weshay|ruck | may be a better candidate | 15:07 |
| marios|ruck | weshay|ruck: otherwise, how do we reconcile https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=6439b21a91a11b464ad5b2cc147e81cd with missing jobs {'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master', 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master'} | 15:08 |
| weshay|ruck | can we chat for 5? | 15:08 |
| marios|ruck | weshay|ruck: k | 15:08 |
| rlandy | eagles: did https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798390 help? | 15:09 |
| rlandy | arxcruz: ^^ pls review | 15:09 |
| marios|ruck | weshay|ruck: https://meet.google.com/coc-mxjc-wpp | 15:09 |
| rlandy | marked as WIP and w-1 until we get your thoughts | 15:09 |
| weshay|ruck | https://meet.google.com/qbd-tcuw-xpd?authuser=1 | 15:09 |
| weshay|ruck | oops | 15:09 |
| weshay|ruck | joining urs | 15:09 |
| eagles | rlandy: yes! thanks! | 15:09 |
| eagles | rlandy: designate tests ran ... and even better... passed ;) | 15:09 |
| rlandy | eagles: k .. good to know ... passing to tempest folks for review | 15:10 |
| eagles | rlandy: cool, thanks again! | 15:10 |
| chandankumar | rlandy: hello | 15:10 |
| chandankumar | rlandy: https://sf.hosted.upshift.rdu2.redhat.com/images/redhat8/rhos-17/?C=M;O=D | 15:11 |
| rlandy | chandankumar:looking | 15:11 |
| chandankumar | rlandy: https://sf.hosted.upshift.rdu2.redhat.com/dockerfiles/redhat8/rhos-17/?C=M;O=D | 15:11 |
| rlandy | chandankumar++ very nice | 15:11 |
| rlandy | chandankumar: what was it? | 15:11 |
| chandankumar | rlandy: commented here https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32679 | 15:13 |
| chandankumar | rlandy: same candidate_hash.full_hash vs c_hash | 15:13 |
| chandankumar | brb | 15:14 |
| rlandy | chandankumar: cool - happy we found the problem | 15:14 |
| weshay|ruck | https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/b596a61/logs/undercloud/var/log/tempest/stestr_results.html.gz | 15:17 |
| marios|ruck | anbanerj|rover: ^^ https://bugs.launchpad.net/tripleo/+bug/1933639/comments/3 | 15:18 |
| dpawlik | anbanerj|rover: I will perform tests tomorrow... Today I did not have enough time ;/ | 15:19 |
| anbanerj|rover | dpawlik, np, thanks :) | 15:20 |
| kopecmartin | arxcruz: why? can't the user use an if-else statement for the arguments? .. the old ones are still supported iirc .. why to rush with the new ones in old releases? | 15:25 |
| *** ykarel is now known as ykarel|away | 15:28 | |
| arxcruz | kopecmartin: this is from os-tempest side | 15:29 |
| marios|ruck | weshay|ruck: * http://10.0.148.74/promoter_logs/centos8_master.log | 15:29 |
| marios|ruck | * 2021-06-29 11:39:09,213 427072 WARNING promoter Candidate hash 'aggregate: 6439b21a91a11b464ad5b2cc147e81cd, commit: c90043f853e34cf3cdbccbba821f6546c9f472fa, distro: 181dc17f38c251200d8c37d56909acb175e0f407, extended: None, component: network, timestamp: 1624897080': missing jobs {'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master', | 15:29 |
| marios|ruck | 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master'} | 15:29 |
| marios|ruck | https://trunk.rdoproject.org/api-centos8-master-uc/api/civotes_agg_detail.html?ref_hash=6439b21a91a11b464ad5b2cc147e81cd | 15:30 |
| rlandy | arxcruz: k - have some time now if you want to meet | 15:34 |
| marios|ruck | weshay|ruck: thank you :) | 15:37 |
| weshay|ruck | thank you too! | 15:37 |
| arxcruz | rlandy: sure | 15:38 |
| kopecmartin | arxcruz: so it's stein, rocky and queens? | 15:38 |
| arxcruz | rlandy: i'm in the meet https://meet.google.com/qiu-eqna-wow | 15:39 |
| kopecmartin | arxcruz: one backport shouldn't hurt .. not sure if we can do whole backport of the patch which deprecates the args .. but at least we can do subset of it | 15:39 |
| rlandy | arxcruz: joining | 15:39 |
| *** ysandeep|dinner is now known as ysandeep | 15:41 | |
| ysandeep | arxcruz, kopecmartin thanks! | 15:42 |
| *** ysandeep is now known as ysandeep|away | 15:44 | |
| *** amoralej is now known as amoralej|off | 15:57 | |
| weshay|ruck | anbanerj|rover, ping me when the skip_list patch is up | 15:58 |
| weshay|ruck | anbanerj|rover, /me notes.. do not try to skip each test | 15:59 |
| weshay|ruck | anbanerj|rover, just skip .. tempest.api.object_storage for that periodic job | 15:59 |
| *** jpena is now known as jpena|off | 15:59 | |
| weshay|ruck | anbanerj|rover, roger that? | 15:59 |
| arxcruz | https://trunk.rdoproject.org/api-centos8-wallaby/api/promotions?promote_name=current-tripleo&limit=1 | 16:00 |
| arxcruz | rlandy: https://trunk.rdoproject.org/api-centos8-wallaby/api/promotions?promote_name=current-tripleo&limit=1 | 16:00 |
| anbanerj|rover | weshay|ruck, yes, checking marios|ruck 's patch if it already has that | 16:00 |
| rlandy | arxcruz: https://hackmd.io/Oqy2XzT3Rl-Jyx4uZ5R2gQ | 16:01 |
| anbanerj|rover | weshay|ruck, ok, it skips for all jobs. Putting up a new one | 16:01 |
| *** dviroel is now known as dviroel|lunch | 16:03 | |
| anbanerj|rover | weshay|ruck, does this look ok https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798709 ? | 16:08 |
| anbanerj|rover | zbr, I think this is why the ER containers are not coming up - https://opendev.org/opendev/elastic-recheck/src/branch/master/elastic_recheck/config.py#L54 this is the default query which produces some com=ntent so that the html files and be built. But if I put it on logstash now it returns nothing | 16:13 |
| anbanerj|rover | Did the query syntax change with some update? | 16:13 |
| weshay|ruck | anbanerj|rover, /me looks | 16:16 |
| anbanerj|rover | weshay|ruck, if you have some time can we check the Victoria promotion as well? There are 4 node failures which I could not get to pass with testP https://review.rdoproject.org/r/c/testproject/+/34312 | 16:16 |
| weshay|ruck | k.. sec | 16:17 |
| *** marios|ruck is now known as marios|out | 16:17 | |
| weshay|ruck | anbanerj|rover, also looks like the storage api tests are passing here: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/6eb1297/logs/undercloud/var/log/tempest/stestr_results.html.gz | 16:19 |
| weshay|ruck | anbanerj|rover, so it's just fs001.. | 16:19 |
| anbanerj|rover | weshay|ruck, https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798709/1/roles/validate-tempest/vars/tempest_skip.yml - this should skip for only fs001 right? | 16:20 |
| weshay|ruck | anbanerj|rover, correct.. those storage tests.. are passing in standalone and fs035, only failing in 001 | 16:21 |
| weshay|ruck | don't think we need to block on that imho | 16:21 |
| arxcruz | rlandy: there is this https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798388 and https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798388 | 16:24 |
| arxcruz | ops | 16:24 |
| arxcruz | rlandy: there's https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798390 and https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798388 | 16:24 |
| weshay|ruck | rlandy, or arxcruz need reviews soon https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798709 | 16:24 |
| arxcruz | my comment was on https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798388 | 16:24 |
| rlandy | cool - thanks | 16:24 |
| weshay|ruck | arxcruz, hook me https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798709 | 16:35 |
| rlandy | ssamal: hello | 16:36 |
| rlandy | weshay|ruck: chandankumar: ^^ say hi! | 16:36 |
| weshay|ruck | ssamal, WELCOME!!! | 16:37 |
| *** dviroel|lunch is now known as dviroel | 17:08 | |
| weshay|ruck | rlandy, https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798714 | 17:16 |
| rlandy | voted | 17:17 |
| weshay|ruck | thanks | 17:19 |
| rlandy | dviroel: hey | 17:28 |
| dviroel | rlandy: o/ | 17:29 |
| rlandy | dviroel: wanted to introduce you to ssamal | 17:29 |
| ssamal_ | dviroel: hello | 17:29 |
| rlandy | dviroel: he's looking at the New Hire Orientation stuff | 17:29 |
| dviroel | hi ssamal_, welcome :) | 17:29 |
| rlandy | I assume you had to go through some of this at a distance | 17:29 |
| dviroel | rlandy: yes, i did | 17:30 |
| rlandy | dviroel: ok - so we might be in touch with you about which course were useful to take first etc. | 17:30 |
| dviroel | rlandy: ack | 17:30 |
| rlandy | dviroel: thanks! | 17:31 |
| dviroel | ssamal_: hey, feel free to ping me if you need anything ;) | 17:31 |
| ssamal_ | dviroel: sure | 17:31 |
| dviroel | ssamal_: which TZ? utc-6? | 17:32 |
| dviroel | or -7? | 17:33 |
| ssamal_ | dviroel: utc-7 | 17:35 |
| dviroel | ssamal_: ack | 17:36 |
| dviroel | por conta do TZ, os 2 gerentes (meu gerente e a outra gerente) ficam mais sobrecarregados, entao eu e o cara novo somos importantes nesse aspecto | 17:40 |
| dviroel | gah | 17:40 |
| dviroel | wrong chat | 17:40 |
| weshay|ruck | lolz | 17:43 |
| *** rlandy_ is now known as rlandy | 17:49 | |
| anbanerj|rover | weshay|ruck, rlandy /me leaving for the day. https://review.rdoproject.org/r/c/testproject/+/34325 this is still running | 17:55 |
| weshay|ruck | k.. thanks I'll watch it | 17:55 |
| rlandy | anbanerj|rover: k - thanks | 18:04 |
| rlandy | lunch - brb | 18:42 |
| rlandy | bhagyashris: do we have a retro tomorrow? | 19:33 |
| weshay|ruck | rlandy, it's on the cal | 19:49 |
| rlandy | weshay|ruck: arxcruz: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798698 please | 19:54 |
| rlandy | bhagyashris, : k - found the invite | 19:58 |
| weshay|ruck | rlandy, ur going to merge conflict w/ https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/798714 | 20:02 |
| weshay|ruck | I think | 20:02 |
| rlandy | ugh - probably | 20:03 |
| rlandy | let me rebase | 20:03 |
| rlandy | it's running a test job now | 20:05 |
| rlandy | will rebase it afterwards | 20:05 |
| dviroel | rlandy: question - when these tests get back to be executed again? when someone fix the issue? | 20:16 |
| rlandy | dviroel: yeah - so when the bug gets fixed we need to remove the test from the skiplist | 20:16 |
| rlandy | there is tracking in the cockpit for tests that are passing and yet still skipped | 20:17 |
| rlandy | some times we are not as on it as we should be | 20:17 |
| rlandy | but generally, when the close the related CIX card, it should not be closed unless the skip is removed | 20:17 |
| dviroel | oh, make sense, is it considered quality degraded? | 20:19 |
| rlandy | dviroel: ack | 20:20 |
| rlandy | because the line goes on running and promoting | 20:21 |
| rlandy | anbanerj|rover: weshay|ruck: question about the queries spreadsheet | 20:23 |
| rlandy | to confirm - we're only checking if those queries are still relevant in upstream | 20:24 |
| rlandy | not RDO | 20:24 |
| weshay|ruck | rlandy, we should talk about that.. needs context.. | 20:28 |
| rlandy | weshay|ruck: so I am working on https://docs.google.com/spreadsheets/d/16rqgaSSoQrYNjsI4q0YInJxrOOJL3xyP3t3D6ZpluFY/edit?ts=60da4093#gid=846893892 | 20:29 |
| weshay|ruck | ya.. I know | 20:30 |
| weshay|ruck | I need to provide some context.. so when you have time.. | 20:30 |
| rlandy | weshay|ruck: I'm around - available | 20:31 |
| weshay|ruck | meet.google.com/pin-fqba-sxw | 20:32 |
| *** dviroel is now known as dviroel|out | 21:22 | |
| weshay|ruck | anbanerj|rover, :) master is promoting http://10.0.148.74/promoter_logs/centos8_master.log https://review.rdoproject.org/r/c/testproject/+/34325 | 21:38 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!