| *** ysandeep- is now known as ysandeep | 06:12 | |
| *** amoralej|off is now known as amoralej | 06:36 | |
| ysandeep | bhagyashris_, hey o/ chandhan is on sick leave, please let me know if you need a hand with ruck/rovering | 07:38 |
|---|---|---|
| bhagyashris_ | ysandeep, sure thanks .. CI pretty much stable. only Victoria is blocked with TIMED_OUT issue on fs035 it's getting timed out at execute tempest test, rerunning the the job in test project patch | 07:40 |
| bhagyashris_ | let see | 07:40 |
| *** bhagyashris_ is now known as bhagyashris|ruck | 07:40 | |
| ysandeep | bhagyashris_, that's good news o/ | 07:41 |
| akahat | bhagyashris_, i'm also here if you need any help :) | 08:34 |
| *** sshnaidm|afk is now known as sshnaidm | 08:35 | |
| *** jpena|off is now known as jpena | 08:35 | |
| *** dmellado_ is now known as dmellado | 08:43 | |
| *** ykarel is now known as ykarel|lunch | 09:05 | |
| frenzy_friday | hey bhagyashris_ , could you pls give me access to https://docs.google.com/spreadsheets/d/16rqgaSSoQrYNjsI4q0YInJxrOOJL3xyP3t3D6ZpluFY/edit?skip_itp2_check=true#gid=846893892 | 09:28 |
| bhagyashris_ | frenzy_friday, Done | 09:39 |
| frenzy_friday | bhagyashris_, thanks | 09:39 |
| bhagyashris_ | akahat, thanks | 09:39 |
| marios | needs votes please when you next have time https://review.opendev.org/c/openstack/tripleo-ci/+/793120 would be good to merge that one to avoid fruther rebases ... just template definitions for new jobs so won't break anything and won't run until it is wired up in different patches https://review.opendev.org/q/topic:wallaby-upgrade-jobs | 09:45 |
| ysandeep | marios, ack checking | 09:49 |
| *** pojadhav- is now known as pojadhav | 10:03 | |
| *** bhagyashris_ is now known as bhagyashris|ruck | 10:08 | |
| bhagyashris|ruck | arxcruz, marios ysandeep plz add in your review list https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/799432 | 10:37 |
| ysandeep | bhagyashris|ruck, already voted | 10:38 |
| bhagyashris|ruck | thanks | 10:38 |
| bhagyashris|ruck | ohh thanks arxcruz ysandeep | 10:38 |
| marios | bhagyashris|ruck: ack | 10:44 |
| *** ykarel|lunch is now known as ykarel | 10:45 | |
| bhagyashris|ruck | marios, that patch got merged | 10:54 |
| marios | bhagyashris|ruck: ? | 10:55 |
| bhagyashris|ruck | marios, this patch got merge https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/799432 | 10:56 |
| marios | bhagyashris|ruck: yeah i had a look and voted before it merged | 10:56 |
| marios | bhagyashris|ruck: whats up | 10:56 |
| bhagyashris|ruck | marios, thanks | 10:56 |
| bhagyashris|ruck | marios, i have one thing for you , master promotion is blocked due to https://bugs.launchpad.net/tripleo/+bug/1933639 . object storage tempest tests are failing consistently on fs001 master periodic | 10:57 |
| bhagyashris|ruck | any updates on it | 10:58 |
| bhagyashris|ruck | i mean see the comments so is Christian working on it or your are going to debug this ^ | 10:59 |
| marios | bhagyashris|ruck: let me add a comment, no christian is waiting for us to give him a node to debug | 10:59 |
| bhagyashris|ruck | marios, ok | 11:00 |
| marios | bhagyashris|ruck: he already checked it and didn't find something in the logs | 11:00 |
| bhagyashris|ruck | ok | 11:00 |
| *** poojajadhav is now known as pojadhav | 11:01 | |
| frenzy_friday | sshnaidm, zbr, we have a problem in tripleo health queries with the length of filename generated by sova. When you get some time pls review https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/799456 | 11:08 |
| frenzy_friday | zbr, pls add https://review.opendev.org/c/openstack/tripleo-ci-health-queries/+/799453 as well to your review list | 11:10 |
| zbr | frenzy_friday: that opens a new question: why we use filenames for text messages? they are not reliable. I am personally not pleased with that design. | 11:11 |
| sshnaidm | frenzy_friday, do you have an example when it's big length? | 11:11 |
| zbr | i prefer simple and predictable: put all errors found inside an errors.txt file, one that is not created if nothing is found. | 11:11 |
| frenzy_friday | sshnaidm, in usual cases no. But in https://opendev.org/openstack/tripleo-ci-health-queries/src/branch/master/samples/errors-testing.err we are trying to have a sample string for all the regex we track | 11:12 |
| sshnaidm | zbr, it creates file so that person that looks in logs will know immediately what is the problem, like "Overcloud_stack__FAILED.log" in https://logserver.rdoproject.org/50/798950/4/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/6156d19/logs/_Overcloud_stack__FAILED.log | 11:12 |
| frenzy_friday | As a unit test for the regexes | 11:12 |
| sshnaidm | without opening files | 11:12 |
| zbr | IMHO, that is achieve by the use of zuul_return | 11:13 |
| sshnaidm | frenzy_friday, is it some regexp that takes a whole line or a lot of reasons for failure? | 11:13 |
| sshnaidm | zbr, probably, if it works - no need to create it | 11:13 |
| frenzy_friday | sshnaidm, lots of failure reasons. | 11:13 |
| sshnaidm | frenzy_friday, any example? | 11:14 |
| frenzy_friday | I think it is good to have the failure reasons in the filename itself - helpful while ruckrovering. But I think there should be a error handling in case the file name gets too long | 11:15 |
| marios | bhagyashris|ruck: https://bugs.launchpad.net/tripleo/+bug/1933639/comments/14 | 11:15 |
| zbr | in fact we could easily make use of zuul warnings or file comments | 11:15 |
| zbr | https://zuul-ci.org/docs/zuul/reference/jobs.html#leaving-warnings | 11:16 |
| frenzy_friday | sshnaidm, example - https://review.opendev.org/c/openstack/tripleo-ci-health-queries/+/796007/3/src/data/queries.yml Here if you add any more items it would go out of length | 11:16 |
| sshnaidm | zbr, yeah, that would be cool | 11:17 |
| zbr | considering that the reason is still inside the filename, you still have to click it, so I am not sure which is the real benefit. Exposing what failed can be done as warning and included directly into the zuul message. | 11:17 |
| zbr | sshnaidm: i do understand the need that caused the original approach, is just that i am not convinced it scales the best. | 11:17 |
| marios | bhagyashris|ruck: let me know if you need any clarification | 11:18 |
| sshnaidm | zuul "warnings:" seems suitable | 11:18 |
| zbr | we could look into, i am not proposing to drop file naming without having an viable alternative | 11:18 |
| bhagyashris|ruck | marios, ack thanks | 11:18 |
| sshnaidm | zbr, the sooner the person can know the failure reason, the better, zuul warnings/comments seems as a best solution for me | 11:19 |
| sshnaidm | I'm not happy about long filenames as well | 11:19 |
| zbr | we should make a card for that and experiment | 11:19 |
| frenzy_friday | yes , ++ that | 11:19 |
| sshnaidm | frenzy_friday, another option - just cut text[:100] | 11:19 |
| zbr | yep, limit size of text for the moment. | 11:20 |
| sshnaidm | if filename is messed up, anyway need to look inside | 11:20 |
| frenzy_friday | sshnaidm, ok, updating the patch | 11:20 |
| frenzy_friday | sshnaidm, zbr Updated https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/799456 | 11:22 |
| sshnaidm | frenzy_friday, just a comment, different patterns can point to same problem, for example ""ReadTimeoutError: HTTPConnectionPool" and "yum download error" both point to Yum download error from mirrors, we need to use same "message" for them | 11:24 |
| marios | bhagyashris|ruck: fyi https://bugs.launchpad.net/tripleo/+bug/1934658/comments/2 cos it will show up on the CIX board | 11:25 |
| frenzy_friday | sshnaidm, yes, the converter which generates the sova patterns json is messed up. Right now everything gets added to "console" with different messages. /me checking how to update it | 11:26 |
| bhagyashris|ruck | marios, ack | 11:29 |
| *** dviroel|out is now known as dviroel | 11:30 | |
| bhagyashris|ruck | ykarel, hey can you please hold node for this job https://review.rdoproject.org/r/c/testproject/+/34321 | 11:30 |
| *** jpena is now known as jpena|lunch | 11:33 | |
| dviroel | marios: o/ hi there, there was a minor update on Readme for this change https://review.opendev.org/c/openstack/tripleo-repos/+/795171 - please take a look when you have some time | 11:33 |
| marios | ack dviroel adding to list | 11:39 |
| *** ykarel_ is now known as ykarel | 11:56 | |
| *** jpena|lunch is now known as jpena | 12:28 | |
| marios | needs reviews please add to your queue tripleo-ci o/ thank you https://review.opendev.org/c/openstack/tripleo-repos/+/795333 | 12:29 |
| *** jpena is now known as jpena|away | 12:36 | |
| *** amoralej is now known as amoralej|lunch | 12:44 | |
| ysandeep | ykarel++ | 12:54 |
| akahat | Review request: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32679 | 13:03 |
| zbr | frenzy_friday: what is the endpoint using wrong certificate? | 13:08 |
| frenzy_friday | zbr, "https://kibana:<passwd>@elk.review.rdoproject.org:9200/logstash-*/_search" <- this is the elasticsearch url which needs cert | 13:10 |
| frenzy_friday | we have another url - the kibana url https://kibana:<passwd>@review.rdoproject.org/elasticsearch/logstash-*/_search - this doesnt need certificate to connect, but pyelasticsearch client doesnt recognize it | 13:10 |
| frenzy_friday | https://hackmd.io/CqKnNTOVRhC6K83sBZrbrQ?both#What-is-failing-now | 13:11 |
| sshnaidm | https://review.rdoproject.org/r/c/rdo-jobs/+/34360 https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/798950 | 13:15 |
| bhagyashris|ruck | Reviews on https://review.opendev.org/c/openstack/tripleo-ci/+/797891 please, this is already 7 days old :p | 13:19 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/tripleo-repos/+/794801 | 13:19 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/tripleo-repos/+/795171 | 13:19 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/tripleo-ci/+/793120 | 13:19 |
| bhagyashris|ruck | https://review.opendev.org/q/topic:wallaby-upgrade-jobs | 13:19 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/tripleo-repos/+/795333 | 13:19 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/tripleo-quickstart/+/791486 | 13:20 |
| bhagyashris|ruck | https://review.rdoproject.org/r/q/topic:%22refactor_job_names%22+(status:open%20OR%20status:merged) | 13:20 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/patrole/+/795548 | 13:20 |
| zbr | marios: tq is using a specific commit from the linter, no wonder you ended up with problems. | 13:20 |
| bhagyashris|ruck | folks plz add in your review list | 13:20 |
| zbr | sorry my language but WTF is quickstart-usb? is anyone using it? | 13:22 |
| marios | zbr: grateful for any comments there and i will update ... but feel free to update if you want to test something | 13:22 |
| marios | zbr: let me know if/when you are poking there so i don't please? | 13:22 |
| marios | zbr: thanks for checking | 13:22 |
| zbr | I see stuff like `ci-scripts/usbkey/RUN_ME.sh:32:28: --playbook quickstart-usb.yml` but the file is not a playbook! is a tasks file! | 13:22 |
| marios | zbr: yeah noticed that before too i think we can probably nuke it | 13:23 |
| marios | zbr: i think it was some kind of idea like the original tripleo 'seed' node so you can deploy from a usb key | 13:23 |
| marios | zbr: wouldn't expect that to be working right now i think | 13:23 |
| zbr | i am sure it never worked, so I prefer to remove it unless someone knows it as working | 13:23 |
| marios | ;) | 13:23 |
| zbr | prevents linter from being upgrades, as newer version is "smarter" | 13:23 |
| sshnaidm | marios, commented https://review.opendev.org/c/openstack/tripleo-ci/+/793393 | 13:26 |
| marios | ack thanks for review sshnaidm will check your comments and reply | 13:29 |
| *** amoralej|lunch is now known as amoralej | 13:33 | |
| * zbr got bit scared observing github autopilot writing the exact code I was supposed to type. | 13:35 | |
| frenzy_friday | zbr, sshnaidm https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/799456 - can we pl merge this one? | 13:59 |
| zbr | sure +2 | 14:00 |
| sshnaidm | frenzy_friday, done | 14:00 |
| zbr | usually i would have asked for tests and using a better filename-sanitizer, but I can understand the time-pressure. | 14:01 |
| *** ysandeep is now known as ysandeep|afk | 14:08 | |
| frenzy_friday | thanks | 14:16 |
| *** jpena|away is now known as jpena | 14:29 | |
| *** ykarel is now known as ykarel|away | 14:34 | |
| *** ysandeep|afk is now known as ysandeep | 14:58 | |
| *** amoralej is now known as amoralej|off | 15:39 | |
| frenzy_friday | rlandy, https://review.opendev.org/c/openstack/tripleo-ci-health-queries/+/798958 will pass when https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/799456 is merged. We are installing the collection from the git repo, I think that is the reason Depends-On is not working | 15:40 |
| ysandeep | sshnaidm: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/252701 | 15:55 |
| *** marios is now known as marios|out | 15:56 | |
| rlandy | frenzy_friday: thank you | 16:09 |
| *** ysandeep is now known as ysandeep|away | 16:17 | |
| *** jpena is now known as jpena|off | 16:33 | |
| * dviroel ☕ | 19:10 | |
| * dviroel 💻 | 19:29 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!