| *** pojadhav|pto is now known as pojadhav | 05:02 | |
| *** akahat|PTO is now known as akahat | 06:02 | |
| *** akahat is now known as akahat|ruck | 06:30 | |
| opendevreview | Dr. Jens Harbott proposed openstack/devstack master: Revert "Add option to set chap algorithms for iscsid for FIPS" https://review.opendev.org/c/openstack/devstack/+/823303 | 08:03 |
|---|---|---|
| opendevreview | Andre Aranha proposed openstack/tempest master: Add support for ecdsa keys https://review.opendev.org/c/openstack/tempest/+/807465 | 08:38 |
| opendevreview | Andre Aranha proposed openstack/tempest master: Add skip for tests that wont work under fips https://review.opendev.org/c/openstack/tempest/+/810808 | 08:38 |
| opendevreview | Andre Aranha proposed openstack/tempest master: WIP - Refactor ssh.Client to allow other clients https://review.opendev.org/c/openstack/tempest/+/820860 | 08:38 |
| *** bhagyashris_ is now known as bhagyashris | 08:45 | |
| opendevreview | Rajat Dhasmana proposed openstack/tempest master: Fix: Schema validation of volume list detail https://review.opendev.org/c/openstack/tempest/+/823375 | 10:53 |
| opendevreview | Andre Aranha proposed openstack/tempest master: WIP - Refactor ssh.Client to allow other clients https://review.opendev.org/c/openstack/tempest/+/820860 | 11:23 |
| opendevreview | Jiri Podivin proposed openstack/tempest master: Patch paramiko https://review.opendev.org/c/openstack/tempest/+/822560 | 11:38 |
| opendevreview | Jiri Podivin proposed openstack/tempest master: WIP - replace paramiko with libssh https://review.opendev.org/c/openstack/tempest/+/806274 | 11:38 |
| opendevreview | Jiri Podivin proposed openstack/tempest master: DNM - Add temporary link to patched paramiko https://review.opendev.org/c/openstack/tempest/+/810809 | 11:39 |
| opendevreview | Jiri Podivin proposed openstack/tempest master: TEST/DNM - add sleep in ceph tests https://review.opendev.org/c/openstack/tempest/+/820270 | 11:39 |
| rpittau | fungi: hi! About https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/821863 do you have any visibility on the availability of python 3.10 for ubuntu focal ? | 13:50 |
| fungi | rpittau: https://packages.ubuntu.com/python3.10 indicates it hasn't made it into focal yet. coreycb (he's not in here but you can find him in #openstack-infra at least) may have details | 14:20 |
| rpittau | fungi: thanks! :) | 14:20 |
| fungi | yw | 14:21 |
| kopecmartin | #startmeeting qa | 15:00 |
| opendevmeet | Meeting started Tue Jan 4 15:00:17 2022 UTC and is due to finish in 60 minutes. The chair is kopecmartin. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'qa' | 15:00 |
| kopecmartin | #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Weekly_QA_Team_meeting | 15:00 |
| kopecmartin | agenda ^^ | 15:00 |
| gmann | o/ | 15:01 |
| vhari | o/ | 15:01 |
| frickler | \o | 15:01 |
| jelabarre-rh | o/ | 15:02 |
| kopecmartin | what an attendance | 15:02 |
| kopecmartin | hi there | 15:02 |
| kopecmartin | happy new year to all :) | 15:03 |
| kopecmartin | let's go through the agenda | 15:03 |
| kopecmartin | #topic Announcement and Action Item (Optional) | 15:03 |
| kopecmartin | nothing here | 15:03 |
| kopecmartin | #topic Yoga Priority Items progress | 15:04 |
| kopecmartin | #link https://etherpad.opendev.org/p/qa-yoga-priority | 15:04 |
| kopecmartin | any updates here? | 15:04 |
| soniya29 | hello | 15:04 |
| kopecmartin | although i don't expect any due to holidays | 15:05 |
| soniya29 | kopecmartin, nothing from my side :) | 15:05 |
| gmann | I have been doing few microversion schema patches | 15:05 |
| gmann | but need to check if they are good to review/passing zuul or not | 15:05 |
| gmann | other than that nothing much to share here | 15:06 |
| kopecmartin | nothing from my side either, i'm trying to catch up and go through mails and reviews | 15:06 |
| jelabarre-rh | still waiting for new patches to get incorporated that I then need to test | 15:06 |
| kopecmartin | ack | 15:07 |
| kopecmartin | #topic OpenStack Events Updates and Planning | 15:07 |
| jelabarre-rh | and then maybe investigate my question on multiple-architecture coverage in Tempest (for what little I understand of the Tempest internals) | 15:07 |
| kopecmartin | i was gonna bring that up in the open discussion | 15:08 |
| jelabarre-rh | ak | 15:08 |
| kopecmartin | good, let's quickly go through the usual and we'll be there in a minute | 15:09 |
| kopecmartin | regarding the events, nothing new | 15:09 |
| kopecmartin | #topic Gate Status Checks | 15:09 |
| kopecmartin | #link https://review.opendev.org/q/label:Review-Priority%253D%252B2+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking) | 15:09 |
| kopecmartin | any gate failures to bring up? | 15:09 |
| frickler | gate was broken last week | 15:09 |
| frickler | paramiko did a new release which broke rsa key login to cirros | 15:10 |
| frickler | and u-c testing didn't catch this, because tempest deployed with committed u-c, not with the patch under test | 15:11 |
| kopecmartin | oh | 15:11 |
| gmann | yeah | 15:11 |
| frickler | ykarel did a fix for the latter and for now paramiko is capped | 15:11 |
| kopecmartin | that's great | 15:11 |
| frickler | I proposed a fix for tempest https://review.opendev.org/c/openstack/tempest/+/823159 | 15:12 |
| frickler | but there's also other options like switching to ecdsa possibly | 15:12 |
| kopecmartin | the DNM in the title confuses me | 15:12 |
| gmann | switching to ecdsa is in progress i think in tempest | 15:12 |
| kopecmartin | gmann: if you mean this | 15:13 |
| kopecmartin | #Link https://review.opendev.org/c/openstack/tempest/+/807465 | 15:13 |
| gmann | yeah | 15:13 |
| kopecmartin | it's more like adding a support, not switching | 15:13 |
| frickler | kopecmartin: I went for DNM because I wanted to discuss whether to do this unconditionally or add a tempest flag | 15:13 |
| gmann | yeah after support we will be able to switch in devstack | 15:13 |
| kopecmartin | and i think it's ready , it's the first piece required in order to move closer to fips goal | 15:13 |
| frickler | medium term I'm also hoping to get a new release of cirros done that fixes the issue properly | 15:14 |
| frickler | or possibly finally get along with forking cirros | 15:14 |
| gmann | frickler: do you have testing patch for 823322 ? | 15:14 |
| frickler | gmann: no, since you don't see the issue in CI. I tested locally | 15:15 |
| gmann | ok | 15:16 |
| kopecmartin | I'd go with https://review.opendev.org/c/openstack/tempest/+/807465 | 15:17 |
| gmann | frickler: kopecmartin lgtm, +2 | 15:17 |
| frickler | then we merged the openeuler patch and almost immediately broke it with the fips patch | 15:17 |
| kopecmartin | it would help us to bypass the gate failure and test the esdsa support even more | 15:18 |
| kopecmartin | as devstack would switch as gmann mentioned | 15:18 |
| frickler | and finally mlavalle has a fix up for fedora which is important for neutron testing https://review.opendev.org/c/openstack/devstack/+/823218 | 15:20 |
| gmann | yeah, slaweq just pinged for review. I will also check after meeting | 15:20 |
| frickler | I added the same swap workaround as we have for other platforms already | 15:21 |
| gmann | +2 | 15:22 |
| gmann | on stable/train Tempest pin. Tempest 28.0.0 pin is reverted to unblock gate. I will continue working on finding the compatible Tempest version for stable/train/ currently master is used and gate is green. | 15:22 |
| kopecmartin | yeah, on that i thought stestr version was the issue | 15:24 |
| gmann | humm. trackback was from tempest run command not stestr so I am not 100% sure | 15:25 |
| kopecmartin | the version of stestr which was used didn't have the new args for sure, i checked the stestr's repo manually | 15:26 |
| gmann | tempest 26.1.0 has exclude-regex arg in tempest run command but it gave error when job passing it and even help message of tempest run does not show new arg | 15:26 |
| kopecmartin | that was strange yes | 15:26 |
| gmann | kopecmartin: yeah but in that case tempest run should accept the new arg and stestr should raise error | 15:26 |
| gmann | yeah, something strange there. I will check this week. | 15:27 |
| kopecmartin | oh, that's right, tempest has a try except block for that situation , hmm | 15:27 |
| kopecmartin | ok, thanks | 15:28 |
| kopecmartin | moving on | 15:28 |
| kopecmartin | #topic Periodic jobs Status Checks | 15:28 |
| kopecmartin | #link https://zuul.openstack.org/builds?job_name=tempest-full-xena-py3&job_name=tempest-full-wallaby-py3&job_name=tempest-full-victoria-py3&job_name=tempest-full-ussuri-py3&job_name=tempest-full-train-py3&pipeline=periodic-stable | 15:28 |
| kopecmartin | #link https://zuul.openstack.org/builds?project=openstack%2Ftempest&project=openstack%2Fdevstack&pipeline=periodic | 15:28 |
| kopecmartin | seems all good here | 15:28 |
| kopecmartin | #topic Sub Teams highlights | 15:28 |
| kopecmartin | Changes with Review-Priority == +1 | 15:29 |
| kopecmartin | #link https://review.opendev.org/q/label:Review-Priority%253D%252B1+status:open+(project:openstack/tempest+OR+project:openstack/patrole+OR+project:openstack/devstack+OR+project:openstack/grenade+OR+project:openstack/hacking) | 15:29 |
| kopecmartin | there are 4 patches, i've reviewed 2 already, the rest I'll try check later today | 15:29 |
| kopecmartin | any changes to bring up? | 15:29 |
| kopecmartin | #topic Open Discussion | 15:32 |
| kopecmartin | (gmann) Turning off the openstack-health service and so does retirement of it? | 15:32 |
| kopecmartin | we've got a volunteer | 15:32 |
| kopecmartin | let me find the email | 15:32 |
| gmann | +1 | 15:33 |
| kopecmartin | I summarized what we talked about on the openstack-health meeting before holidays | 15:33 |
| kopecmartin | #link http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026487.html | 15:33 |
| jelabarre-rh | Arx Cruz? | 15:33 |
| gmann | yeah | 15:33 |
| kopecmartin | so any recommendations where to start? | 15:34 |
| gmann | I think we have repo maintainer now so let's allow them to fix the current issue and once it is up then we will discuss on hosting issue | 15:34 |
| gmann | kopecmartin: it will be good to fix it and bring it up http://status.openstack.org/openstack-health/#/ | 15:35 |
| gmann | #luink http://status.openstack.org/openstack-health/#/ | 15:35 |
| gmann | #link http://status.openstack.org/openstack-health/#/ | 15:35 |
| arxcruz | kopecmartin: yo | 15:35 |
| gmann | arxcruz: hi | 15:35 |
| arxcruz | gmann: congrats on the new heir :) | 15:35 |
| gmann | thanks :) | 15:35 |
| gmann | arxcruz: thanks for volunteering on maintaining the o-h. | 15:35 |
| arxcruz | sure | 15:36 |
| gmann | arxcruz: as you might know we have two issues/things here. 1. maintain repo 2. hosting infra | 15:36 |
| gmann | currently dashboard is broken #link http://status.openstack.org/openstack-health/#/ | 15:36 |
| arxcruz | gmann: yes, as I understood, rdo will provide the infra right ? | 15:36 |
| arxcruz | we can maintain | 15:36 |
| gmann | let's fix that as first step and then we can talk about hosting infra need or so | 15:36 |
| arxcruz | gmann: sure, actually, it's because there is no data being populated | 15:37 |
| gmann | arxcruz: infra is not yet solved. we will see if upstream infra can be used from new aws credits for e-r or so | 15:37 |
| arxcruz | ok | 15:37 |
| arxcruz | frenzy_friday: what's the url for our o-h ? | 15:37 |
| gmann | arxcruz: yeah, I have not checked it but something wrong in code? | 15:37 |
| gmann | frenzy_friday: #link http://status.openstack.org/openstack-health/#/ | 15:38 |
| arxcruz | gmann: so, i can tell in our case, there's a html site, that is the status.o.o/o-h | 15:38 |
| gmann | #link https://opendev.org/openstack/openstack-health | 15:38 |
| gmann | ^^this is source code | 15:38 |
| arxcruz | and also a backgorund job that do some queries on the logs | 15:38 |
| arxcruz | and create a json file | 15:38 |
| arxcruz | this background job runs every x hours | 15:38 |
| arxcruz | and write down a json file that the website reads and shows the results | 15:39 |
| arxcruz | in our case, we create a new branch on openstack-health, because our elasticsearch was newer and we had to make changes in the code for the new api's | 15:39 |
| gmann | so fixes can be in existing repo right? #link https://opendev.org/openstack/openstack-health | 15:40 |
| arxcruz | gmann: yes, in a branch called rdo | 15:40 |
| gmann | did not get? | 15:41 |
| gmann | o-h repo is branchless and we do not tag also. like doing /using it from master version itself | 15:41 |
| arxcruz | gmann: https://opendev.org/opendev/elastic-recheck/src/branch/rdo | 15:41 |
| gmann | ohk you mean e-r one | 15:42 |
| gmann | i got confused with o-h repo | 15:42 |
| arxcruz | gmann: yeah, sorry, my mistake | 15:42 |
| arxcruz | i just come back from vacation, not 100% yet on all the details :) | 15:42 |
| gmann | sure, we can discuss those later. but brining it up will be good first step and to check for infra need | 15:43 |
| gmann | *bringing dashboard up | 15:43 |
| kopecmartin | I'll try to look more into it with arxcruz and let's see what we'll come up with | 15:45 |
| kopecmartin | will share next week | 15:46 |
| gmann | and we can keep this topic in agenda for infra things or so | 15:46 |
| gmann | thanks again arxcruz | 15:46 |
| kopecmartin | +1 | 15:46 |
| kopecmartin | another topic in the agenda is | 15:46 |
| kopecmartin | about expanding devstack team with potential candidates | 15:46 |
| kopecmartin | any updates here? | 15:46 |
| arxcruz | :) | 15:46 |
| gmann | ah, forgot about it, I will do this week | 15:46 |
| gmann | no further updates on this | 15:47 |
| kopecmartin | ok :) | 15:47 |
| kopecmartin | then we have the mixed-architecture stack topic | 15:47 |
| kopecmartin | #link http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026492.html | 15:47 |
| kopecmartin | I was thinking about adding a new opt in tempest which would skip tests which are not meant for different architectures | 15:48 |
| gmann | do we need test changes for that? or just run the tests with different image in separate job? | 15:48 |
| kopecmartin | the same opt could be then used for tests which are specific to those architecture | 15:48 |
| gmann | kopecmartin: skip is not good IMO, if those operation perfoermed by tests cannnot be done in real also then just do not run the test | 15:49 |
| gmann | if they can be done then we need to adjust the tests | 15:49 |
| kopecmartin | yeah, makes sense, .. then maybe let's start from a job, let's create a job with different arch and let's see what's failing and we can take it from there | 15:49 |
| jelabarre-rh | I'm thinking in terms of a system where there might be multiple architectures mixed together. | 15:50 |
| gmann | yeah, and that will be good feedback to projects that this operation did not work in this arch so you will fix it or add as a known limit? | 15:50 |
| gmann | jelabarre-rh: you mean mixed compute node? | 15:51 |
| jelabarre-rh | yes | 15:51 |
| gmann | and if tempest test end up creating VM on differrent arch compute node | 15:51 |
| jelabarre-rh | there is a customer running x86_64 and ppc64le within the same stack | 15:51 |
| frickler | before starting to mix things, how about finishing up the arm devstack job and getting it working? | 15:51 |
| gmann | in that case, challenge is how to detect those mixed arch in tempest/ | 15:52 |
| gmann | Tempest is API driven test suits and not aware much about underlying arch | 15:52 |
| gmann | frickler: +1 | 15:52 |
| jelabarre-rh | I know IBM Power ends up stopping at Train (long story), but I expect the same thing may happen for ARM and maybe RISC-V | 15:52 |
| jelabarre-rh | I'm bringing it up now so planning can happen before such systems need to be tested | 15:53 |
| jelabarre-rh | a discussion topic rather than an urgen current need | 15:53 |
| gmann | yeah, but for tempest it is difficult to know which compute node where test VM is schedule to boot is of what arch | 15:54 |
| kopecmartin | a good ptg topic, i'll make a note of it somewhere | 15:54 |
| gmann | Tempest being as a user of OpenStack, should not aware of compute node arch. for example, if user boot/migrate VM on mixed arch then what happen? what are SLA from provide side | 15:55 |
| gmann | because user operation may fail on mixed arch right? | 15:56 |
| jelabarre-rh | it would be more than just testing one arch then the next, as you'd want to have Affinity and Migration check *how* an attempt to migrate an instance between architectures fails | 15:57 |
| gmann | jelabarre-rh: so that is expected behavior right? I mean if user perform those, it fail. | 15:58 |
| jelabarre-rh | right | 15:58 |
| gmann | so test failing there give the right results. instead skipping test can just hide the things | 15:59 |
| jelabarre-rh | I happened upon that more by chance, because I'm mainly testing mixed-architecture deployment, and not running Tempest against x86_64, just ppc64le/ | 15:59 |
| gmann | k | 16:00 |
| gmann | I think we are running out of time. I am +1 to discuss in PTG more. | 16:00 |
| gmann | or next meeting | 16:00 |
| kopecmartin | yup | 16:01 |
| gmann | but as first step, having job and see what all fail will be good input for discussion | 16:01 |
| kopecmartin | agree | 16:01 |
| jelabarre-rh | sounds good. Mainly to brainstorm for future revisions | 16:01 |
| kopecmartin | #topic Bug Triage | 16:01 |
| kopecmartin | #link https://etherpad.opendev.org/p/qa-bug-triage-yoga | 16:01 |
| kopecmartin | bug numbers are recorded at ^ | 16:01 |
| kopecmartin | i have this one fix to share | 16:02 |
| kopecmartin | #link https://review.opendev.org/c/openstack/tempest/+/823375 | 16:02 |
| kopecmartin | that's all from my side | 16:02 |
| gmann | kopecmartin: ack, will check schema one | 16:02 |
| kopecmartin | thanks | 16:02 |
| kopecmartin | thank you all | 16:03 |
| kopecmartin | see you around | 16:03 |
| gmann | thanks kopecmartin | 16:03 |
| kopecmartin | #endmeeting | 16:03 |
| opendevmeet | Meeting ended Tue Jan 4 16:03:18 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:03 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/qa/2022/qa.2022-01-04-15.00.html | 16:03 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/qa/2022/qa.2022-01-04-15.00.txt | 16:03 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/qa/2022/qa.2022-01-04-15.00.log.html | 16:03 |
| soniya29 | thanks kopecmartin | 16:04 |
| clarkb | arxcruz: gmann: personaly considering how long it hsa been broken with no one noticing or complaining I think you should seriously consider turning it off instead. I think it being unnoticeable for so long is a good indication it isnt well used and that continued maintenance is unlikely | 16:47 |
| clarkb | my takeaway from the call with RDO was basically that there wasn't anothe rgroup that needed this tool and need is a good driver for maintenance | 16:47 |
| clarkb | if there was a shared need then that could have been leveraged but there wasn't | 16:47 |
| arxcruz | clarkb: hmm... we do use, not specifically the part that gmann is interested, but we do use, and help us a lot, that's the reason we are running our own rdo version of it | 16:49 |
| arxcruz | it would be nice to have subunit2sql and tempest data | 16:49 |
| arxcruz | but it helps our work on tripleo | 16:49 |
| clarkb | no you don't use it, that was the result of the call we had | 16:50 |
| clarkb | you use elsatic-recheck which is completely different | 16:50 |
| clarkb | don't confuse the two things | 16:50 |
| arxcruz | clarkb: ok | 16:50 |
| arxcruz | maybe I am indeed confusing :) | 16:50 |
| clarkb | and no one hsa used this for about 6 months or however long it has been broken | 16:51 |
| clarkb | which is why I make this recommendation | 16:51 |
| arxcruz | ok | 16:52 |
| clarkb | arxcruz: the result of th emeeting with RDO was that RDO has its own tooling and doesn't need the per test case resolution. Also if I interpreted correctly the need of subunit was felt to be problematic because not everything uses subunit | 16:56 |
| clarkb | the elastic-recheck and elasticserach stuff is separate and has ongoing maintenance via an opensearch donation from amazon and work from dpawlik to convert ingestion of logs to something that happens outside of the zuul base job | 16:57 |
| arxcruz | clarkb: yeah, right now we don't run too much tempest tests in our tests due the lack of time/resources | 16:57 |
| clarkb | and that is what RDO has been using and helping with and I don't see that changing | 16:57 |
| arxcruz | clarkb: ok I understand now, I wasn't on the RDO meeting | 16:57 |
| arxcruz | thanks for the clarification | 16:58 |
| gmann | clarkb: arxcruz yeah, triopleo does not use it. but if we can bring it up and running then we can discuss on usage vs infra need for this dashboard to keep it up | 17:03 |
| clarkb | gmann: I'm suggeting that the lack of interest for th elast 6 months is a good indication that we shouldn't bother | 17:04 |
| gmann | arxcruz volunteer to maintain it so I am ok to hold the retirement and if he thinks it is not much usable then we can retire | 17:04 |
| clarkb | we can put effort into it but if no one is using it for 6 months is that a good investment | 17:04 |
| clarkb | gmann: I also think arxcruz didn't understand what it was they were volunteering for and confused it with the ELK stuff | 17:04 |
| arxcruz | clarkb: yeah, you're right, but doesn't mean i can't do the o-h as well :) | 17:05 |
| arxcruz | in our case specifically, we don't use because we run only a small set of the tempest tests | 17:05 |
| arxcruz | basically the basicnetworkscenario | 17:05 |
| arxcruz | and specific tests on jobs that have specific services enabled | 17:05 |
| gmann | from ML and today discussion i think arxcruz is going to maintain the o-h. he was confused with o-h being used in tripleo | 17:05 |
| arxcruz | so, when a job fails, it's easy to spot | 17:05 |
| gmann | but yes, I am ok with both option retire or fix but its arxcruz CALL | 17:06 |
| gmann | call | 17:06 |
| arxcruz | gmann: if you need me I'm here to help :) | 17:06 |
| arxcruz | if it is something that will help, i'm more than happy to help | 17:06 |
| gmann | arxcruz: I will say your call :) because i cannot help in that and also do not use much. | 17:07 |
| gmann | last time before you volunteer, we decided to retire it as no one there to maintain it who need it so let's retire repo itself | 17:07 |
| arxcruz | ok, i'll talk with kopecmartin tomorrow and we will discuss this better ok ? | 17:07 |
| gmann | sure. if you or anyone use it or helpful then you or they maintain then we will not force it to retire | 17:08 |
| arxcruz | ok | 17:08 |
| gmann | as repo retirement i mean. infra hosting is another things to discuss with clarkb or in TC | 17:09 |
| clarkb | right from a service perspective I have a strong desire to turn it off because no one is using it | 17:09 |
| clarkb | in my opinion it would be a bda investment ot resurrect something unused for 6 months with no one saying they need the tool | 17:10 |
| gmann | yeah. let's wait for arxcruz call to use/maintain it now. but yes for 6 months it has not been used as it was broken. | 17:10 |
| clarkb | the ELK situation was different. Multiple people said it was important and useful and we figured out a way forward. But I'm not seeing that with o-h | 17:11 |
| opendevreview | Merged openstack/devstack master: Fix stacking without preconfigured DATABASE_PASSWORD https://review.opendev.org/c/openstack/devstack/+/822819 | 18:33 |
| opendevreview | Merged openstack/devstack master: init_cinder() shouldn't always create DEFAULT_VOLUME_GROUP_NAME https://review.opendev.org/c/openstack/devstack/+/555836 | 18:50 |
| mtreinish | gmann: what is needed on o-h? Just saw the ping | 18:58 |
| opendevreview | Merged openstack/tempest master: Fix: Schema validation of volume list detail https://review.opendev.org/c/openstack/tempest/+/823375 | 19:23 |
| opendevreview | Merged openstack/devstack master: Fix mysqladmin failure for Fedora 34 and mariadb https://review.opendev.org/c/openstack/devstack/+/823218 | 19:24 |
| *** tosky_ is now known as tosky | 20:32 | |
| gmann | mtreinish: it is down, not sure if it is code issue or hosting one? so we need two things 1. someone to maintain source code for bugs 2. we need to find infra resources for this services which we can discuss in TC once we are sure that someone is there to maintain the repo. | 21:42 |
| gmann | it been ~ 6 month it is down | 21:42 |
| mtreinish | gmann: it looks like the subunit2sql db is missing data. The runs and test_runs tables are empty | 22:07 |
| mtreinish | o-h itself is pretty minimal it just proxies queries to subunit2sql and es. Keeping those things running is much more important | 22:08 |
| mtreinish | the place we o-h needs the most work is the js side of it, the ui is a bit clunky and I'm sure 50% of the 20k npm dependencies that get pulled in have security issues or something | 22:11 |
| gmann | yeah, js part is something for which we do not have anyone in QA right now who can take care (just keep it up) | 22:14 |
| mtreinish | gmann: I'm about EoD now, but I can check with clarkb and fungi tomorrow to see if there is anything in the logs about subunit2sql. My guess is that a dependency update for sqlalchemy or oslo* or something broke the population of new results (assuming the workers are still running) and the prune job has progressively been removing old results | 22:18 |
| gmann | mtreinish: sure, thanks. arxcruz volunteer to help for maintaining o-h. He can also help in debugging. but thanks for looking | 22:19 |
| mtreinish | yeah, just ran recheck on some open patches on gerrit. It looks like there are definitely some sqlalchemy compat issues with newer versions. Not sure if they're the issue or not, but it definitely needs to be fixed | 22:34 |
| mtreinish | the simple workaround is probably to just pin the sqlalchemy version on the worker nodes while we wait for a fix in subunit2sql | 22:36 |
| fungi | that's probably doable in the configuration management for it, if that's not also bitrotted | 22:47 |
| fungi | or we could temporarily disable configuration management for those two servers temporarily and downgrade sqla on them i guess | 22:47 |
| fungi | but yeah, this week is not great for me, at least not prior to, say, friday | 23:14 |
| fungi | i'm buried under end-of-year/start-of-year paperwork | 23:14 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!