| *** pojadhav|out is now known as pojadhav|ruck | 04:23 | |
| *** frenzy_friday|rover is now known as frenzy_friday|rover|doc | 06:56 | |
| *** frenzy_friday|rover|doc is now known as frenzy_friday|rover | 08:17 | |
| *** dasm|off is now known as dasm | 12:43 | |
| *** frenzy_friday|rover is now known as frenzy_friday|rover|lunch | 12:58 | |
| *** pojadhav|ruck is now known as pojadhav|dr_appt | 13:41 | |
| *** frenzy_friday|rover|lunch is now known as frenzy_friday|rover | 13:50 | |
| gmann | tc-members: meeting time | 16:00 |
|---|---|---|
| gmann | #startmeeting tc | 16:00 |
| opendevmeet | Meeting started Wed Nov 30 16:00:10 2022 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'tc' | 16:00 |
| gmann | #topic Roll call | 16:00 |
| JayF | o/ | 16:00 |
| gmann | o/ | 16:00 |
| slaweq | o/ | 16:00 |
| knikolla[m] | o/ | 16:00 |
| dansmith | o/ | 16:00 |
| jungleboyj | Hello all. | 16:01 |
| ade_lee | o/ | 16:01 |
| gmann | let's wait for a min if rosmaita spotz arne_wiebalck noonedeadpunk join, i cannot see any name in absence section | 16:02 |
| gmann | meanwhile this is today agenda #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda_Suggestions | 16:02 |
| arne_wiebalck | o/ | 16:02 |
| gmann | let's start | 16:04 |
| gmann | #topic Follow up on past action items | 16:04 |
| gmann | gmann to check with foundation about zoom pro account if any and can be shared with TC monthly video call | 16:04 |
| gmann | still did not get the response from foundation staff, I will send the reminder | 16:04 |
| gmann | #action gmann to check with foundation about zoom pro account if any and can be shared with TC monthly video call | 16:04 |
| gmann | other followup is about election things | 16:04 |
| gmann | I have pushed the TC charter change to reflect what we discussed in PTG | 16:05 |
| gmann | #link https://review.opendev.org/c/openstack/governance/+/865367 | 16:05 |
| spotz | o/ here for a few minutes but as mentioned when this time slot was chosen I have a conflict | 16:05 |
| gmann | please review ^^ | 16:05 |
| gmann | spotz: ack | 16:05 |
| gmann | as soon as we can merge it, will be good to plan for next election in advance | 16:05 |
| rosmaita | o/ | 16:06 |
| gmann | I will keep eyes on gerrit if any comment on that | 16:06 |
| gmann | #topic Gate health check | 16:06 |
| gmann | volume detach timeout is happening 100% in lvm job (nova-lvm) and we had to disable those failing test for now | 16:07 |
| dansmith | other than the nova thing, no gate health concerns from me this time, but with the holiday my data is limited | 16:07 |
| gmann | it started when migrated to Jammy | 16:07 |
| gmann | ok | 16:07 |
| dansmith | volume detach seems to be a constant problem | 16:07 |
| slaweq | we had one issue with trunk ports and live-migration on Neutron but it should be fixed already | 16:08 |
| gmann | yeah, even those failing tests are preparing server with ssh-able and make it ready before volume is attached/detached | 16:08 |
| gmann | slaweq: +1 | 16:08 |
| gmann | any other issues/item to highlight for gate health ? | 16:09 |
| gmann | #topic 2023.1 TC tracker checks | 16:10 |
| gmann | #link https://etherpad.opendev.org/p/tc-2023.1-tracker | 16:10 |
| gmann | one update from me is on election things which I already mentioned the patch to review | 16:10 |
| gmann | and other is about i18 SIG discussion in board meeting on Dec 6th. | 16:11 |
| gmann | rosmaita: ^^ can you give brief update on this | 16:11 |
| rosmaita | sure | 16:11 |
| rosmaita | we will propose to the Foundation Board that they fund a hosted version of Weblate | 16:11 |
| rosmaita | which the SIG i18n has idenitified as the best platform to use | 16:12 |
| rosmaita | we also proposed that the Foundation Staff should help us find a volunteer to change the Zanata-to-gerrit plumbing to use weblate | 16:12 |
| rosmaita | this will free up the i18n team to do the content migration and ultimately focus on doing translations | 16:13 |
| gmann | yeah, it will be discussed in Dec 6th board meeting and let's see how it goes | 16:15 |
| gmann | rosmaita: anything else on this? or any query form any other tc-members ? | 16:16 |
| gmann | *from | 16:16 |
| rosmaita | nothing from me | 16:16 |
| noonedeadpunk | o/ | 16:16 |
| noonedeadpunk | sorry, was side-pinged last moment :( | 16:17 |
| gmann | np! | 16:17 |
| gmann | ok, let's move | 16:17 |
| gmann | any other updates from anyone on the tracker items ? | 16:17 |
| gmann | #topic FIPS testing on ubuntu paid subscription | 16:18 |
| gmann | #link https://review.opendev.org/c/openstack/project-config/+/861457 | 16:18 |
| gmann | as we discussed it in PTG also, with currently available options to test FIPS, using ubuntu paid subscription is one of the option where we can make FIPS jobs voting | 16:19 |
| gmann | centos stream is not stable enough to test FIPS on gate | 16:19 |
| gmann | opendev is looking for TC official agreement on this. | 16:19 |
| fungi | note that tinwood, the openstack charms ptl, was able to negotiate a free subscription for our test jobs to use | 16:20 |
| gmann | I would like to start the voting on this but before that if we need more discussion/questions, we have ade_lee here to explain the situation | 16:20 |
| fungi | so while people normally pay for access to ubuntu's fips-enabled packages, we're getting gratis access to them | 16:20 |
| JayF | I'll note I had a conversation with smoe of the opendev folks about this when it surfaced, and my impression is that a developer would be able to access the actual-fips-packages to do local testing of a failing job, just without support. As long as this is accurate (developers can troubleshoot broken fips jobs locally), I am on board. | 16:21 |
| noonedeadpunk | I wonder if they would need to have this free subscribtion that covers 5 machines for that? | 16:21 |
| ade_lee | thats correct. the agreement was that the subscrption is free for use for fips testing , but with no support | 16:22 |
| fungi | note that the canonical folks didn't say they were providing free access to developers, just to our ci jobs | 16:22 |
| frickler | can that agreement be made public? | 16:23 |
| fungi | and they acknowledge that we can't reasonably *secure* that access due to the way our jobs are run | 16:23 |
| JayF | How could we expect developers to be able to maintain FIPS support if they cannot test that outside of zuul? | 16:23 |
| gmann | yeah I was wondering on that we will not be able to test it locally right? its is just in CI | 16:23 |
| fungi | do we expect developers to maintain osc support for rackspace's api without a paid account in rackspace? | 16:23 |
| dansmith | I feel like most of the things you'd need to 'debug' from a fips job failure is not something you'd really necessarily have to reproduce locally. It'd be like an exception complaining that md5 isn't available or something | 16:24 |
| fungi | or is the free rackspace account we arranged for osc testing not acceptable practice either? | 16:24 |
| dansmith | which is pretty straightforward and once we're gating on these, would be when you add a new feature that is using something like that | 16:24 |
| dansmith | fungi: the same goes for all the special hardware support we have in lots of places | 16:24 |
| dansmith | fungi: I have zero $10k nvidia GPUs locally | 16:25 |
| gmann | true, all backends | 16:25 |
| fungi | correct, though for the most part we don't currently test those upstream because they're not available to us | 16:25 |
| noonedeadpunk | do we test gpus for reall? huh... | 16:25 |
| dansmith | I'm still waiting for my s/390 machine in the mail | 16:25 |
| JayF | I'll note that many of those special-hardware-support tests are non-blocking, too (at least in Ironic they don't get to vote) | 16:25 |
| slaweq | dansmith but isn't it like things which requires e.g. special hardware for testing are in 3rd party jobs? | 16:25 |
| noonedeadpunk | but are we making fips as required to pass for all projects? | 16:26 |
| slaweq | at least in neutron it is like that - if something require special hardware, we don't gate with such job | 16:26 |
| dansmith | slaweq: yeah, but I think in most projects if a patch causes a particular driver to fail, a core team would expect that to be fixed before merge, no? | 16:26 |
| noonedeadpunk | I guess it's up to projects to decide if add them or not after all? | 16:26 |
| fungi | that's why i brought up osc's testing. the client/sdk team has arranged gratis accounts with public cloud providers for use in upstream testing, something which an individual developer would not have access to without similar negotiations | 16:26 |
| gmann | but will it be hard to fix the FIPS things just from CI failure and even we are not able to produce them locally? | 16:26 |
| slaweq | dansmith sure | 16:27 |
| dansmith | the FIPS thing seems much easier to debug than hardware | 16:27 |
| gmann | yeah | 16:27 |
| dansmith | gmann: right, I think it's much easier | 16:27 |
| *** pojadhav|dr_appt is now known as pojadhav|out | 16:27 | |
| dansmith | "sorry MD5 not allowed, *sad trombone*" is pretty straightforward | 16:27 |
| slaweq | we can always try to add it and if there will be too many issues with it and it will be hard to debug without access to the FIPS env for developers, we can always remove that job(s) from gate | 16:28 |
| * jungleboyj is happy to see someone else use sad trombone. :-) | 16:28 | |
| dansmith | right, well, that's what we've done up until now.. we've added them and then when centos breaks, we make them n-v | 16:28 |
| slaweq | but probably it will be as dansmith says - it will be pretty easy to fix issues related to FIPS | 16:28 |
| gmann | true, we can re-iterate it based on future situations | 16:29 |
| dansmith | so if this becomes a problem, it's easy to do the same | 16:29 |
| gmann | ok, so this does not seems blocking to allow CI testing on it | 16:29 |
| dansmith | AFAIK, none of the times we've had to do that have been due to fips problems, just CS problems | 16:29 |
| gmann | and if any other distro (free version) become stable/available then we can always move FIPS jobs to that | 16:29 |
| dansmith | also that ^ :) | 16:30 |
| JayF | I'll note that in the review posted here earlier; debian was listed as an alternative | 16:30 |
| JayF | but a nonviable one because we don't use it for many other things (yet) | 16:30 |
| fungi | though it is in the pti for 2023.1 | 16:30 |
| JayF | So; I don't think it's resonable to say "if a free version becomes available" -- it is aavailable | 16:30 |
| dansmith | JayF: becomes "viable" | 16:30 |
| gmann | also *stable* enough to test everywhere | 16:30 |
| gmann | FIPS goal is to add jops in every project as voting | 16:31 |
| dansmith | part of the problem has been that CS9 breaks or behaves differently than *all* the other tests, and so it becomes a problem for people who are set up to test on ubuntu | 16:31 |
| JayF | I mean, viability is all in a matter of if we want to spend time on it. And debian is incredibly stable. I'm not suggesting we change it, I just don't want us to couch this as us having no choice; we do have a choice | 16:31 |
| JayF | we're just prioritizing using existing infra + a token over revamping the infra to use a more free solution | 16:31 |
| fungi | right, someone needs to work out the logistics for it and make sure the baseline testing on that platform without fips is also running and in good shape so that fips-specific issues can be differentiated from general platform-related issues | 16:31 |
| dansmith | debian is similar in that it becomes some work if someone has to bootstrap the debian environment to see if it's a debian problem or a FIPS on | 16:31 |
| dansmith | *one | 16:31 |
| gmann | question is what are current best and stable/viable options to start it | 16:31 |
| dansmith | debian being stable isn't the concern, it's debian being different than all the other jobs | 16:31 |
| dansmith | we had a debian failure for a few weeks recently | 16:32 |
| dansmith | which wasn't a debian problem (AFAIK) but just a "different than focal" problem | 16:32 |
| JayF | Which I'm fine with; I just don't want us to pretend like we don't have fully-foss options when we would if we prioritized debian CI support. | 16:32 |
| dansmith | I'm prioritizing this purely as a "minimal change from our other jobs" perspective... right. | 16:32 |
| fungi | note that nobody has said ubuntu's packages for this aren't fully foss, they just charge money for access to them (that doesn't make then not free/libre open source) | 16:33 |
| noonedeadpunk | to be fair, we have debian and ubuntu jobs running in osa and we haven't seen debian being much different or failing differently rather then ubuntu for quite a while | 16:33 |
| dansmith | noonedeadpunk: we had an example in the devstack gate just a couple weeks ago | 16:33 |
| noonedeadpunk | but yes, that we're currently running ubuntu we can't use debian only for fips | 16:33 |
| fungi | open source licenses don't preclude someone from charging money when distributing the software | 16:34 |
| JayF | that's fair; I should've been more precise with my choice of words | 16:34 |
| noonedeadpunk | Well, I used devstack last time 3 years ago or so, so hard to judge on it | 16:34 |
| dansmith | noonedeadpunk: https://review.opendev.org/c/openstack/devstack/+/864135 | 16:34 |
| dansmith | (turned out to be within our control, we just didn't know it, because it was different) | 16:35 |
| gmann | at least in current situation, if debian become much tested in projects side also and have voting jobs then nobody stop us to move FIPS jobs on that | 16:35 |
| dansmith | sounds like we're circling the drain of agreement, shall we vote? | 16:35 |
| frickler | getting charged for something to me sounds like the opposite of free | 16:35 |
| dansmith | frickler: beer and speech :) | 16:35 |
| jungleboyj | :-) | 16:35 |
| gmann | key part is we want to wait for FIPS testing or can start with ubuntu paid(free for us) for now? | 16:35 |
| gmann | any other point/discussion ? otherwise let's vote | 16:36 |
| noonedeadpunk | gmann: Can you post question for vote before starting vote ? :) | 16:37 |
| gmann | seems we are good to vote, just to make sure and to avoid invalid voting :) this is wording of vote "Considering the currently available options, Is it ok to use the Ubuntu paid subscription for FIPS testing?" | 16:37 |
| gmann | noonedeadpunk: yeah ^^ | 16:37 |
| dansmith | fine with me | 16:37 |
| noonedeadpunk | silence means no objections I guess? | 16:38 |
| rosmaita | i am ready | 16:38 |
| slaweq | ++ | 16:38 |
| gmann | yeah, let's start | 16:38 |
| gmann | #startvote Considering the currently available options, Is it ok to use the Ubuntu paid subscription for FIPS testing? Yes, No | 16:38 |
| opendevmeet | Begin voting on: Considering the currently available options, Is it ok to use the Ubuntu paid subscription for FIPS testing? Valid vote options are Yes, No. | 16:38 |
| opendevmeet | Vote using '#vote OPTION'. Only your last vote counts. | 16:38 |
| noonedeadpunk | #vote Yes | 16:39 |
| dansmith | #vote Yes | 16:39 |
| slaweq | #vote Yes | 16:39 |
| gmann | #vote Yes | 16:39 |
| arne_wiebalck | #vote Yes | 16:39 |
| JayF | #vote yes | 16:39 |
| rosmaita | #vote yes | 16:39 |
| spotz_ | #vote yes | 16:40 |
| gmann | I think knikolla[m] spotz left to vote, waiting for a min if they will. | 16:40 |
| knikolla[m] | #vote Yes | 16:40 |
| gmann | cool | 16:40 |
| gmann | #endvote | 16:40 |
| opendevmeet | Voted on "Considering the currently available options, Is it ok to use the Ubuntu paid subscription for FIPS testing?" Results are | 16:40 |
| opendevmeet | Yes (9): spotz_, knikolla[m], gmann, slaweq, arne_wiebalck, rosmaita, dansmith, JayF, noonedeadpunk | 16:40 |
| knikolla[m] | sorry, am in two meetings at the same time. | 16:40 |
| gmann | knikolla[m]: ack, np | 16:40 |
| gmann | ok so we have the agreement now, I will post the link to project-config patch | 16:41 |
| gmann | moving to next topic | 16:41 |
| dansmith | fungi: can you send it? | 16:41 |
| gmann | #topic Adjutant situation (not active) | 16:41 |
| gmann | Last change merged Oct 26, 2021 (more than 1 year back) | 16:41 |
| gmann | Gate is broken | 16:41 |
| fungi | dansmith: send what" | 16:42 |
| fungi | ? | 16:42 |
| dansmith | fungi: merge the fips patch | 16:42 |
| fungi | oh, sure | 16:42 |
| ade_lee | thanks all | 16:42 |
| gmann | it seems Adjutant situation is not improved since we abandon the proposal of marking it 'inactive' | 16:42 |
| fungi | after the meeting adjourns i'll include a link to the minutes in the change approval | 16:42 |
| gmann | #link https://review.opendev.org/c/openstack/governance/+/849153 | 16:42 |
| gmann | ade_lee: thanks for all effort on this goal | 16:42 |
| gmann | fungi: thanks | 16:43 |
| gmann | and I pinged PTL on IRC I think couple of times but no response | 16:43 |
| gmann | even no response on gerrit | 16:43 |
| gmann | I feel we should mark it as 'Inactive' (restoring the above patch) and then we will see if we can retire it if no maintainer or not ? | 16:44 |
| JayF | I would vote +1 to such a patch given the current situation | 16:44 |
| knikolla[m] | ++ | 16:44 |
| dansmith | sounds fine to me | 16:45 |
| fungi | out of curiosity, has anyone sent e-mail to the ptl? i suppose it's possible they're just unaware of the responsibilities they signed up for or what they should be paying attention to | 16:45 |
| knikolla[m] | Unfortunately we have stopped using Adjutant in our cloud in favor of a different solution so I can't justify the time to track its state as I used to a few years ago. | 16:45 |
| slaweq | overall stats of the Adjutant are like https://paste.opendev.org/show/b4OWBZ74G8t6QKKs4Fqq/ | 16:45 |
| slaweq | I'm +1 for marking it as "inactive" | 16:45 |
| gmann | fungi: I do not think they are unware, they ack in the governance patch (marking inactive) and we abandon the same | 16:46 |
| noonedeadpunk | according paste it jobs are not really broken :D | 16:46 |
| fungi | got it | 16:46 |
| gmann | but anyways let me propose it for inactive and will send email also to PTL to ack it | 16:46 |
| noonedeadpunk | Last time I looked at adjutant it was broken mainly due to django version | 16:46 |
| slaweq | yeah, and I think that it was someone who was ptl for many cycles already, not someone pretty new | 16:46 |
| gmann | yeah | 16:46 |
| fungi | oh, thanks, for some reason i thought adjutant ended up with a volunteer new ptl eventually | 16:47 |
| gmann | #action gmann to mark Adjutant a 'inactive' and send notification to PTL | 16:47 |
| gmann | ok, moving next | 16:47 |
| gmann | #topic Recurring tasks check | 16:47 |
| gmann | Bare 'recheck' state | 16:47 |
| gmann | #link https://etherpad.opendev.org/p/recheck-weekly-summary | 16:47 |
| gmann | slaweq: over to you | 16:47 |
| slaweq | all good there I think | 16:47 |
| slaweq | average number of recheckes before merge is a bit higher this last week | 16:48 |
| slaweq | but as we had issue in neutron and there were some other issues also, I think it's just related | 16:48 |
| slaweq | and should be better next week | 16:48 |
| gmann | yeah | 16:48 |
| slaweq | bare rechecks are good | 16:48 |
| gmann | thanks for monitoring | 16:48 |
| slaweq | especially for projects where there is more rechecks - most of them aren't bare | 16:49 |
| slaweq | that's all | 16:49 |
| gmann | that is nice | 16:49 |
| jungleboyj | That is a good improvement. | 16:49 |
| gmann | #topic Open Reviews | 16:49 |
| dansmith | sweet | 16:49 |
| gmann | #link https://review.opendev.org/q/projects:openstack/governance+is:open | 16:49 |
| gmann | tc charter change we already talked | 16:49 |
| noonedeadpunk | sounds like 865367 ready for merge? | 16:50 |
| gmann | fungi: for all other project updates, we are waiting for project-config patches to merge, can you please check | 16:50 |
| noonedeadpunk | or we need 2/3 of tc for this one? | 16:50 |
| gmann | noonedeadpunk: yeah | 16:50 |
| gmann | ok that is all from today agenda, anything anyone would like to bring ? | 16:50 |
| fungi | gmann: i can take a look | 16:50 |
| gmann | fungi: thanks | 16:50 |
| gmann | FYI, next meeting is video call on Dec 7. | 16:51 |
| dansmith | so, | 16:51 |
| dansmith | I will be out the rest of the year before our next meeting, just FYI so I shan't be around for lively discussion or to open a video call on our gmeet, if we still do that | 16:51 |
| dansmith | hopefully rosmaita slaweq or spotz can do it, if we still need it pending the zoom stuff | 16:52 |
| gmann | dansmith: ack | 16:52 |
| gmann | sure | 16:52 |
| slaweq | sure | 16:52 |
| rosmaita | i should be around | 16:52 |
| gmann | if nothing else, let's close the meeting. Thanks everyone for joining. | 16:53 |
| gmann | #endmeeting | 16:53 |
| opendevmeet | Meeting ended Wed Nov 30 16:53:39 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:53 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/tc/2022/tc.2022-11-30-16.00.html | 16:53 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/tc/2022/tc.2022-11-30-16.00.txt | 16:53 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/tc/2022/tc.2022-11-30-16.00.log.html | 16:53 |
| slaweq | o/ | 16:53 |
| rosmaita | bye | 16:53 |
| noonedeadpunk | we also have quite bad situation with sahara. Gates are broken for Zed and master (and they never passed for Zed actually). There're patches to fix main issues, but there're couples of them that needs to be merged. I tried to invest some time fixing and sorting things out and came to the point when only one tempest job is not passing | 16:54 |
| noonedeadpunk | didn't have time to dig more. | 16:54 |
| noonedeadpunk | And didn't mail PTL yet | 16:54 |
| noonedeadpunk | Will try to invest some time into that, otherwise it has quite big chances to follow adjutant | 16:55 |
| noonedeadpunk | Btw tooz gates are also broken | 16:55 |
| frickler | there's also some other candidates if you look at e.g. https://review.opendev.org/q/topic:reno-zed+status:open or https://review.opendev.org/q/topic:add-antelope-python-jobtemplates+status:open | 16:59 |
| noonedeadpunk | yeah. freezer is another good example | 17:03 |
| noonedeadpunk | (not limited to it ofc) | 17:05 |
| gmann | noonedeadpunk: good point, I have also observed sahara case. let's discuss it in next meeting. | 17:12 |
| gmann | freezer seems merging the patch when I add ptl in review | 17:12 |
| gmann | frickler: ack, those are good starting point to know about such projects | 17:13 |
| opendevreview | Ghanshyam proposed openstack/governance master: Add Adjutant in inactive project list https://review.opendev.org/c/openstack/governance/+/849153 | 19:41 |
| opendevreview | Merged openstack/governance master: Add Skyline repository for OpenStack-Ansible https://review.opendev.org/c/openstack/governance/+/863166 | 20:53 |
| opendevreview | Merged openstack/governance master: Add the manila-infinidat charm to Openstack charms https://review.opendev.org/c/openstack/governance/+/864068 | 20:53 |
| opendevreview | Merged openstack/governance master: Add the cinder-infinidat charm to Openstack charms https://review.opendev.org/c/openstack/governance/+/863958 | 20:53 |
| *** dasm is now known as dasm|off | 22:23 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!