| *** slaweq_ has joined #openstack-meeting-3 | 00:10 | |
| *** slaweq has quit IRC | 00:12 | |
| *** yamamoto has quit IRC | 00:14 | |
| *** apetrich has quit IRC | 02:14 | |
| *** yamamoto has joined #openstack-meeting-3 | 02:43 | |
| *** psahoo has joined #openstack-meeting-3 | 03:31 | |
| *** psachin has joined #openstack-meeting-3 | 03:57 | |
| *** psahoo has quit IRC | 04:35 | |
| *** psahoo has joined #openstack-meeting-3 | 05:00 | |
| *** psahoo has quit IRC | 05:11 | |
| *** psahoo has joined #openstack-meeting-3 | 05:27 | |
| *** belmoreira has joined #openstack-meeting-3 | 05:46 | |
| *** belmoreira has quit IRC | 05:47 | |
| *** psahoo has quit IRC | 05:55 | |
| *** psahoo has joined #openstack-meeting-3 | 06:12 | |
| *** slaweq_ has quit IRC | 06:14 | |
| *** slaweq_ has joined #openstack-meeting-3 | 06:17 | |
| *** slaweq_ has quit IRC | 06:21 | |
| *** slaweq_ has joined #openstack-meeting-3 | 06:34 | |
| *** tsmith2 has quit IRC | 06:40 | |
| *** tsmith2 has joined #openstack-meeting-3 | 06:40 | |
| *** ralonsoh has joined #openstack-meeting-3 | 06:42 | |
| *** slaweq_ is now known as slaweq | 06:48 | |
| *** belmoreira has joined #openstack-meeting-3 | 07:13 | |
| *** tosky has joined #openstack-meeting-3 | 07:40 | |
| *** Luzi has joined #openstack-meeting-3 | 07:55 | |
| ttx | #startmeeting large_scale_sig | 08:00 |
|---|---|---|
| openstack | Meeting started Wed Aug 26 08:00:04 2020 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:00 |
| *** openstack changes topic to " (Meeting topic: large_scale_sig)" | 08:00 | |
| openstack | The meeting name has been set to 'large_scale_sig' | 08:00 |
| ttx | #topic Rollcall | 08:00 |
| *** openstack changes topic to "Rollcall (Meeting topic: large_scale_sig)" | 08:00 | |
| ttx | Who is here for the Large Scale SIG meeting ? | 08:00 |
| ttx | amorin, belmoreira, tobberydberg maybe | 08:01 |
| belmoreira | o/ | 08:01 |
| belmoreira | thanks for the ping | 08:02 |
| *** masahito has joined #openstack-meeting-3 | 08:02 | |
| masahito | o/ | 08:02 |
| tobberydberg | I'm here, but haven't been much part in the Large Scale SIG though ;-) | 08:02 |
| ttx | heh | 08:02 |
| ttx | Alright, let's start and see if anyone else joins | 08:02 |
| ttx | Our agenda for today is at: | 08:03 |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-meeting | 08:03 |
| ttx | #topic Discuss fail of first EU-US meeting | 08:03 |
| *** openstack changes topic to "Discuss fail of first EU-US meeting (Meeting topic: large_scale_sig)" | 08:03 | |
| ttx | As you know we recently decided to create a meeting rotation, to create a US-friendly meeting time | 08:03 |
| ttx | Two weeks ago we had our first EU-US friendly meeting, but then we did not get anyone new showing up | 08:03 |
| ttx | Obviously creating a new meeting in the middle of August is not optimal, but still that was disappointing | 08:03 |
| ttx | We will need to make a bit more noise for the next one, and if nobody shows up again, reconsider our decision | 08:04 |
| ttx | (There is no point in making it harder for APAC folks if we do not really get new members) | 08:04 |
| ttx | Thoughts on that? | 08:04 |
| belmoreira | +1 | 08:04 |
| belmoreira | there are large deployments in the US. Should we try to contact people directly? | 08:05 |
| ttx | I had a few direct contacts based on https://etherpad.opendev.org/p/large-scale-sig | 08:05 |
| ttx | and tried to send them direct email | 08:05 |
| ttx | Beth said she can;t do IRC, the others said IRC was fine but did not show up at that first meeting | 08:06 |
| ttx | But I agree, we should do personal contacts where we can | 08:06 |
| ttx | in advance of the next one | 08:06 |
| ttx | like people who participated actively to the opendev on large scale infra | 08:07 |
| ttx | (James Penick, Erik Andersson...) | 08:07 |
| ttx | If you have a personal contact that could be interested, don;t hesitate to ping them... like the week before | 08:08 |
| *** mdelavergne has joined #openstack-meeting-3 | 08:08 | |
| belmoreira | sure | 08:08 |
| *** mdelavergne has left #openstack-meeting-3 | 08:09 | |
| ttx | #action all to contact US large deployment friends to invite them to next EU-US meeting | 08:09 |
| ttx | ok, moving on | 08:09 |
| ttx | #topic PTG/Summit plans | 08:09 |
| *** openstack changes topic to "PTG/Summit plans (Meeting topic: large_scale_sig)" | 08:09 | |
| ttx | We need to discuss if we want to do anything around the Summit and PTG in October | 08:09 |
| *** mdelavergne has joined #openstack-meeting-3 | 08:09 | |
| ttx | As a reminder, our next summit will be virtual and happen on Oct 19-23, 2020 | 08:09 |
| amorin | hello | 08:09 |
| ttx | There will be "forum sessions" there that we could leverage to get more input | 08:09 |
| amorin | sorry I am late | 08:09 |
| ttx | amorin: hi! | 08:09 |
| mdelavergne | Hi, sorry I am late too | 08:10 |
| ttx | There will also be a PTG (project teams gathering) on Oct 26-30, 2020 | 08:10 |
| ttx | This is more of a work event, where we could have a video meeting to discuss future SIG work if we wanted | 08:10 |
| ttx | In both cases if we are interested we should engage quickly to book slots | 08:10 |
| ttx | Personally I think we should definitely ask for a Forum session, for example around gathering scaling stories | 08:10 |
| ttx | To act as a recruitment function but also to learn new things | 08:10 |
| ttx | I'm less convinced that with the current level of activity/size of the group we need a PTG group meeting. | 08:11 |
| ttx | IRC is plenty enough to coordinate our slow work, so unless there is a particular activity we'd like to book time to synchronously work on... | 08:11 |
| ttx | Thoughts? | 08:11 |
| belmoreira | having a video meeting will be definitely more engaging, I think | 08:11 |
| ttx | belmoreira: just for PTG or all the time? | 08:12 |
| masahito | If there is chance to talk each other, the short video meeting seems to be nice. | 08:12 |
| belmoreira | I was thinking for the PTG | 08:12 |
| ttx | OK, so maybe a short one | 08:12 |
| ttx | or two to match the two timezones | 08:13 |
| amorin | how are working forum sessions, is it going to be with video meeting as well? | 08:13 |
| ttx | I'm not sure yet... I assumed it would be opendev-style | 08:13 |
| ttx | (opendev-the-event style) | 08:14 |
| amorin | so with video on zoom, right? | 08:14 |
| ttx | like a zoom meeting with a couple of moderators | 08:14 |
| ttx | or meetpad | 08:14 |
| ttx | right | 08:14 |
| amorin | so if it is like that, maybe forum session is enough, we can meet over there and recruit new members / find new stories | 08:14 |
| ttx | OK, so how about... a Forum session specifically to find new stories, a bit like that session at opendev-the-event | 08:15 |
| amorin | +1 | 08:15 |
| belmoreira | I think it will be good for the group and it's also an opportunity to fin/interact with other people | 08:15 |
| ttx | + one hour of video meeting to replace our regular meeting | 08:15 |
| ttx | (at the PTG) | 08:15 |
| belmoreira | +1 | 08:15 |
| amorin | works for me | 08:15 |
| ttx | then if we identify potential new mebers in the Forum session we can follow up at the PTG | 08:16 |
| ttx | OK, I'll fill the forms and file then with the required authorities | 08:16 |
| ttx | I'll try to be close to our usual meeting times | 08:17 |
| ttx | if those slots are still available | 08:17 |
| *** dosaboy has quit IRC | 08:17 | |
| *** dosaboy has joined #openstack-meeting-3 | 08:17 | |
| ttx | #agreed One Forum session on scaling stories, One PTG short meeting to replace our regular meeting that week | 08:18 |
| ttx | #action ttx to request Forum/PTG sessions | 08:18 |
| ttx | ok, moving on to our regular topics | 08:18 |
| ttx | #topic Progress on "Documenting large scale operations" goal | 08:18 |
| *** openstack changes topic to "Progress on "Documenting large scale operations" goal (Meeting topic: large_scale_sig)" | 08:18 | |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-documentation | 08:18 |
| ttx | First, osarchiver... | 08:18 |
| ttx | (which was just mentioned again on openstack-discuss) | 08:18 |
| ttx | What's the status of moving that upstream? I think it's stalled waiting on progress on OSops resurrection? | 08:19 |
| amorin | on OVH side, we worked on applying some changes based on comments we receive on the ml | 08:19 |
| amorin | we removed debian folder for example | 08:19 |
| ttx | hah yes that will please zigo for sure | 08:19 |
| amorin | yes | 08:20 |
| amorin | so, for us it's good | 08:20 |
| ttx | amorin: so next step is to make progress on resurrecting OSops | 08:20 |
| amorin | yes | 08:20 |
| ttx | belmoreira: we'll need to work with smcginnis to make progress on the governance side of it | 08:21 |
| ttx | I might raise it on the TC channel | 08:21 |
| belmoreira | great | 08:21 |
| belmoreira | I'm happy to help on that | 08:21 |
| ttx | #action ttx to push for OSops resurrection | 08:21 |
| ttx | belmoreira: thanks | 08:21 |
| *** apetrich has joined #openstack-meeting-3 | 08:22 | |
| ttx | #action belmoreira, ttx to push for OSops resurrection | 08:22 |
| ttx | Second, we still need more config options on the Large scale configuration wiki page before we can push for a link in Nova doc | 08:22 |
| ttx | amorin: do you have enough matter to work on, or should we just pause that until we have more participants working on it? | 08:22 |
| amorin | yes, unfortunately I am not able to find enough time | 08:22 |
| amorin | maybe we can pause that for now | 08:22 |
| amorin | and stay focus on resurrecting osops? | 08:23 |
| ttx | If the issue is more time than data, I'm fine carrying the work item over | 08:23 |
| ttx | If you miss data to do it, then we should just pause it | 08:23 |
| amorin | on the other end, we recently disccuss about rabbit binding issues on the ml | 08:23 |
| amorin | and things in this topic could also be copied back to documentation | 08:23 |
| ttx | amorin: your call, I can just carry over the #action, or just stop mentioning it every meeting :) | 08:24 |
| amorin | ok, about nova, we can pause, we dont have enough data | 08:24 |
| ttx | ok, done | 08:24 |
| ttx | And finally our last workstream is around collecting metrics/billing stories | 08:25 |
| ttx | Last I looked we only had OVH entry on that one | 08:25 |
| ttx | belmoreira, masahito: do you think you could contribute CERN/LINE take on that? | 08:26 |
| ttx | (line 36 of https://etherpad.opendev.org/p/large-scale-sig-documentation) | 08:26 |
| belmoreira | yes, I can. The problem is committing when I'm going to do it. | 08:27 |
| ttx | No commitment really. I just will continue to pester you for it regularly :) | 08:27 |
| belmoreira | :) | 08:27 |
| ttx | #action all to describe briefly how you solved metrics/billing in your deployment in https://etherpad.openstack.org/p/large-scale-sig-documentation | 08:27 |
| ttx | Anything else on this topic? | 08:27 |
| masahito | yes. | 08:28 |
| ttx | masahito: yes you can contribute story, or yes you have something else on this topic? | 08:28 |
| ttx | Assuming that was yes you can contribute story, and moving on to next topic | 08:29 |
| masahito | ah, sorry. yes I can contribute story. | 08:30 |
| ttx | #topic Progress on "Scaling within one cluster" goal | 08:30 |
| *** openstack changes topic to "Progress on "Scaling within one cluster" goal (Meeting topic: large_scale_sig)" | 08:30 | |
| ttx | #link https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling | 08:30 |
| ttx | First, update on the healthcheck/middleware ping discussion | 08:30 |
| ttx | #link https://review.opendev.org/#/c/735385/ | 08:30 |
| ttx | I think we collectively made good progress to get it merged in oslo.messaging... | 08:30 |
| ttx | I was hoping we'd have a go-ahead from sean-k-mooney or dansmith on your last reply: | 08:30 |
| ttx | #link http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016697.html | 08:30 |
| ttx | but I'll leave it to bnemec to get the approval timing right | 08:31 |
| amorin | ok | 08:31 |
| ttx | Second workstream is on oslo.metrics | 08:31 |
| ttx | https://opendev.org/openstack/oslo.metrics | 08:32 |
| amorin | should we ping back bnemec for this? | 08:32 |
| ttx | amorin: I think he'll get to it. I think he's leaving some time for last minute objections | 08:32 |
| amorin | ok | 08:32 |
| *** tosky has quit IRC | 08:32 | |
| amorin | so we wait for now, right? | 08:32 |
| ttx | yes | 08:32 |
| amorin | ack | 08:32 |
| *** tosky has joined #openstack-meeting-3 | 08:33 | |
| ttx | So on the oslo.metrics side, we need to make progress on two sides: try to find other users interested in running it, and improve the code base to make it more openstack-y | 08:33 |
| amorin | I will ask someone at OVH if we can test this | 08:33 |
| ttx | I'll try to work on adding basic tests, unless someone beats me to it | 08:34 |
| ttx | so that we feel more comfortable with proposed changes | 08:34 |
| ttx | masahito: the code we have in https://opendev.org/openstack/oslo.metrics is what you run at LINE currently? Or is there a difference? | 08:34 |
| masahito | Basically same. But we have some improvement since I pushed the patch... | 08:35 |
| ttx | masahito: ok, would be great to push it to oslo.metrics as well, when you get the chance | 08:36 |
| ttx | I'd like to make a release with Victoria release | 08:36 |
| masahito | I try to. | 08:36 |
| ttx | #action masahito to push latest patches to oslo.metrics | 08:37 |
| ttx | #action ttx to look into a basic test framework for oslo,metrics | 08:37 |
| ttx | #action amorin to see if oslo.metrics could be tested at OVH | 08:37 |
| ttx | Last workstream... we still need more scaling stories | 08:37 |
| ttx | https://etherpad.openstack.org/p/scaling-stories | 08:38 |
| ttx | we'll likely need to do a campaign to collect those, as our current outreach is not very efficient | 08:38 |
| ttx | A forum session at next summit centered on that will help imho | 08:38 |
| ttx | Anything else on this topic/goal? | 08:38 |
| mdelavergne | not from my side | 08:39 |
| ttx | #topic Next meeting | 08:39 |
| *** openstack changes topic to "Next meeting (Meeting topic: large_scale_sig)" | 08:39 | |
| ttx | Next meeting will be US-EU on Sept 9, 16utc. | 08:39 |
| ttx | That's the one we'll need to make more noise about | 08:39 |
| ttx | Who from this meeting will likely be able to also participate to this one? | 08:39 |
| amorin | I do | 08:40 |
| ttx | Thanks! | 08:40 |
| ttx | Then next EU-APAC meeting will be Sept 23, 8utc. | 08:40 |
| ttx | How does that sound? | 08:40 |
| amorin | good | 08:41 |
| belmoreira | good | 08:41 |
| ttx | #info next meetings: Sep 9, 16:00UTC; Sep 23, 8:00UTC | 08:41 |
| ttx | That's all I had... Any last thoughts? requests? | 08:41 |
| ttx | OK then... thanks everyone! | 08:42 |
| ttx | #endmeeting | 08:42 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 08:42 | |
| openstack | Meeting ended Wed Aug 26 08:42:26 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 08:42 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-08-26-08.00.html | 08:42 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-08-26-08.00.txt | 08:42 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-08-26-08.00.log.html | 08:42 |
| mdelavergne | Thanks, see you! | 08:42 |
| masahito | thanks | 08:43 |
| *** mdelavergne has quit IRC | 08:43 | |
| amorin | thanks | 08:44 |
| belmoreira | thanks | 08:45 |
| *** yamamoto has quit IRC | 09:27 | |
| *** yamamoto has joined #openstack-meeting-3 | 09:49 | |
| *** baojg has quit IRC | 09:52 | |
| *** baojg has joined #openstack-meeting-3 | 09:53 | |
| *** apetrich has quit IRC | 10:31 | |
| *** Luzi has quit IRC | 10:42 | |
| *** apetrich has joined #openstack-meeting-3 | 10:51 | |
| *** masahito has quit IRC | 11:03 | |
| *** yamamoto has quit IRC | 11:10 | |
| *** yamamoto has joined #openstack-meeting-3 | 11:44 | |
| *** raildo has joined #openstack-meeting-3 | 11:44 | |
| *** yamamoto has quit IRC | 11:51 | |
| *** yangyapeng has left #openstack-meeting-3 | 12:19 | |
| *** _erlon_ has joined #openstack-meeting-3 | 12:22 | |
| *** bnemec has joined #openstack-meeting-3 | 12:59 | |
| *** Adri2000 has joined #openstack-meeting-3 | 13:05 | |
| *** yamamoto has joined #openstack-meeting-3 | 13:48 | |
| *** yamamoto has quit IRC | 13:58 | |
| *** dustinc has joined #openstack-meeting-3 | 14:37 | |
| *** lpetrut has joined #openstack-meeting-3 | 14:46 | |
| *** lpetrut has quit IRC | 14:58 | |
| slaweq | #startmeeting neutron_ci | 15:00 |
| slaweq | hi | 15:00 |
| openstack | Meeting started Wed Aug 26 15:00:21 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 15:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 15:00 |
| bcafarel | o/ | 15:00 |
| slaweq | and please give me 3 minutes before we start | 15:00 |
| ralonsoh | hi | 15:01 |
| slaweq | ok, I'm back | 15:02 |
| slaweq | and I think we can start | 15:02 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:02 |
| slaweq | #topic Actions from previous meetings | 15:02 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 15:02 | |
| slaweq | ralonsoh to check timing out neutron-ovn-tempest-full-multinode-ovs-master jobs | 15:03 |
| ralonsoh | slaweq, sorry but I didn't spend time on this | 15:03 |
| ralonsoh | (and at this point I barely remember it) | 15:03 |
| slaweq | ralonsoh: bug is here https://bugs.launchpad.net/neutron/+bug/1886807 | 15:04 |
| openstack | Launchpad bug 1886807 in neutron "neutron-ovn-tempest-full-multinode-ovs-master job is failing 100% times" [High,Confirmed] - Assigned to Maciej Jozefczyk (maciejjozefczyk) | 15:04 |
| *** lajoskatona has joined #openstack-meeting-3 | 15:04 | |
| slaweq | I think it is failing like that still | 15:04 |
| lajoskatona | o/ | 15:04 |
| ralonsoh | slaweq, I'll check it this week | 15:04 |
| slaweq | thx ralonsoh | 15:04 |
| slaweq | #action ralonsoh to check timing out neutron-ovn-tempest-full-multinode-ovs-master jobs - bug https://bugs.launchpad.net/neutron/+bug/1886807 | 15:05 |
| slaweq | next one | 15:05 |
| slaweq | slaweq to move uwsgi jobs to periodic queue and promote -uwsgi ones to be gating | 15:05 |
| slaweq | Patch https://review.opendev.org/745822 | 15:05 |
| *** psahoo has quit IRC | 15:05 | |
| slaweq | please review it :) | 15:05 |
| slaweq | and the last one on my list: | 15:05 |
| slaweq | maciejjozefczyk to check https://bugs.launchpad.net/neutron/+bug/1890445 | 15:05 |
| openstack | Launchpad bug 1890445 in neutron "[ovn] Tempest test test_update_router_admin_state failing very often" [Critical,Confirmed] | 15:05 |
| slaweq | I don't think maciek will look into this so we will need new volunteer | 15:06 |
| slaweq | I will ask jlibosva and lucasgomes if they can take a look into that one | 15:06 |
| ralonsoh | (thanks) | 15:07 |
| slaweq | #action slaweq to ask jlibosva and lucasgomes if they can check https://bugs.launchpad.net/neutron/+bug/1890445 | 15:07 |
| openstack | Launchpad bug 1890445 in neutron "[ovn] Tempest test test_update_router_admin_state failing very often" [Critical,Confirmed] | 15:07 |
| slaweq | that are all actions from last meeting | 15:07 |
| slaweq | so lets move to the next topic | 15:07 |
| slaweq | #topic Switch to Ubuntu Focal | 15:07 |
| *** openstack changes topic to "Switch to Ubuntu Focal (Meeting topic: neutron_ci)" | 15:07 | |
| slaweq | any updates about that one? | 15:07 |
| ralonsoh | bcafarel, pushed a change to update the lower contrains | 15:08 |
| ralonsoh | https://review.opendev.org/#/c/748168/ | 15:08 |
| bcafarel | sorry it took some trials to get this one passing, it is OK locally now (and testing in progress in https://review.opendev.org/#/c/734304/ ) | 15:09 |
| slaweq | ok, so with that we should be good with functional/fullstack jobs to be moved to focal | 15:10 |
| slaweq | right? | 15:10 |
| ralonsoh | I think so | 15:10 |
| bcafarel | yes they work fine with ncat installed, just need to pass that pesky cI | 15:10 |
| slaweq | good | 15:10 |
| bcafarel | although we could just drop depends-on in https://review.opendev.org/#/c/734304/ and merge the change, we would be ready then | 15:11 |
| slaweq | for other jobs we need this https://review.opendev.org/#/c/731207/ to be merged, right? | 15:11 |
| lajoskatona | not much activity on it recently | 15:12 |
| slaweq | or can we simply change nodeset in e.g. neutron-tempest-plugin jobs now and move on with that? | 15:12 |
| slaweq | wdyt? | 15:12 |
| ralonsoh | I don't think this will work | 15:12 |
| ralonsoh | we need this devstack change | 15:12 |
| slaweq | why? | 15:13 |
| bcafarel | https://review.opendev.org/#/c/738163/ still has a few red jobs (in addition to ft/functional/lower constraints) | 15:13 |
| ralonsoh | hmmmm you are right | 15:13 |
| ralonsoh | just changing the node set could work | 15:13 |
| ralonsoh | yes, you are right | 15:13 |
| ralonsoh | we can try it in a DNM patch | 15:13 |
| lajoskatona | +1 | 15:13 |
| slaweq | let me try to propose patch for neutron-tempest-plugin to change that and see | 15:13 |
| ralonsoh | sure! | 15:13 |
| slaweq | #action slaweq to propose neutron-tempest-plugin switch to focal nodes | 15:14 |
| lajoskatona | if devstack change wil be merged we can remove the nodesets | 15:14 |
| slaweq | lajoskatona: right | 15:14 |
| ralonsoh | yeah, correct | 15:14 |
| slaweq | lajoskatona: what about https://review.opendev.org/#/c/736703/ - is that ready to go? | 15:15 |
| lajoskatona | I check again rally | 15:16 |
| lajoskatona | but as I see it is not worse than with bionic | 15:16 |
| slaweq | lajoskatona: great | 15:16 |
| slaweq | so lets review this patch and move on with it | 15:17 |
| slaweq | thx | 15:17 |
| slaweq | I think that's all regarding migration to Focal | 15:17 |
| slaweq | so lets move on to the next topic | 15:17 |
| slaweq | #topic Stadium projects | 15:17 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 15:17 | |
| slaweq | anything You want to discuss regarding stadium's ci? | 15:17 |
| ralonsoh | no | 15:18 |
| lajoskatona | not really | 15:18 |
| bcafarel | nothign here | 15:18 |
| slaweq | ok, that's fast :) | 15:18 |
| slaweq | lets move to the next topic then | 15:18 |
| slaweq | #topic Stable branches | 15:18 |
| *** openstack changes topic to "Stable branches (Meeting topic: neutron_ci)" | 15:18 | |
| slaweq | anything here? | 15:18 |
| slaweq | IMO ci of stable branches is pretty ok still | 15:18 |
| bcafarel | not much either :) (still catching up from backlog) | 15:18 |
| bcafarel | yep from what I saw everything seems to be in good shape | 15:19 |
| slaweq | great, at least ci of stable branches is stable :) | 15:19 |
| slaweq | ok, so lets move on | 15:20 |
| slaweq | #topic Grafana | 15:20 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 15:20 | |
| slaweq | today I added neutron-tempest-plugin-scenario-ovn job to the grafana https://review.opendev.org/748223 | 15:20 |
| slaweq | as we are missing it there | 15:20 |
| slaweq | except that, it looks pretty good now | 15:21 |
| slaweq | I don't see any major problems with any of the jobs | 15:22 |
| slaweq | maybe except pep8 failures like: | 15:22 |
| slaweq | https://zuul.opendev.org/t/openstack/build/6c8fbf9b97b44139bf1d70b9c85455bb | 15:22 |
| slaweq | did You saw something similar recently? | 15:22 |
| bcafarel | it may be caused by recent pylint bump (so existing reviews can start to fail pep8) | 15:23 |
| ralonsoh | slaweq, if I'm not wrong | 15:24 |
| ralonsoh | this is when you implement a DictModel | 15:24 |
| ralonsoh | you need to define the parameters passed to this dict | 15:24 |
| ralonsoh | (very bad description) | 15:25 |
| slaweq | ok, so we will need to fix that issue | 15:25 |
| ralonsoh | let me check this patch and I'll try to find what is happening | 15:25 |
| ralonsoh | do you have a patch? | 15:25 |
| slaweq | https://review.opendev.org/#/c/715482/21/neutron/agent/linux/dhcp.py | 15:26 |
| slaweq | it failed here | 15:26 |
| slaweq | those lines were not touched by the patch but file was changed | 15:26 |
| ralonsoh | slaweq, ok, I'll debug this today or tomorrow morning | 15:26 |
| slaweq | thx ralonsoh | 15:27 |
| slaweq | #action ralonsoh to check issue with pep8 failures like https://zuul.opendev.org/t/openstack/build/6c8fbf9b97b44139bf1d70b9c85455bb | 15:27 |
| slaweq | that's all what I have regarding grafana | 15:27 |
| slaweq | do You have anything else or can we move on? | 15:28 |
| ralonsoh | I'm ok | 15:28 |
| bcafarel | all good | 15:28 |
| slaweq | ok, lets move on | 15:28 |
| slaweq | #topic Tempest/Scenario | 15:28 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 15:28 | |
| slaweq | Due to recent issue https://bugs.launchpad.net/neutron/+bug/1890842 I would like to add neutron-tempest-plugin-api to be running on neutron-lib patches, | 15:29 |
| openstack | Launchpad bug 1890842 in neutron "API test test_create_port_without_propagate_uplink_status fails with neutron-lib 2.5.0" [Critical,Fix released] - Assigned to Slawek Kaplonski (slaweq) | 15:29 |
| slaweq | Patch proposed https://review.opendev.org/745830 | 15:29 |
| slaweq | this patch has already 3x +2 | 15:29 |
| slaweq | but needs +W :) | 15:29 |
| slaweq | can one of You +W it? | 15:29 |
| ralonsoh | I'll do it | 15:29 |
| slaweq | thx a lot | 15:29 |
| ralonsoh | and thanks for this patch | 15:29 |
| slaweq | the other issue which we have is | 15:29 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1892017 | 15:29 |
| openstack | Launchpad bug 1892017 in neutron "Neutron server logs are too big in the gate jobs" [High,Fix released] - Assigned to Slawek Kaplonski (slaweq) | 15:29 |
| slaweq | melwitt reported us that q-svc logs are very big | 15:30 |
| slaweq | I already proposed https://review.opendev.org/#/c/746714/ and it's merged | 15:30 |
| slaweq | this reduces log file for about 50% | 15:30 |
| slaweq | but it's still around 40MB | 15:30 |
| slaweq | so infra guys for now disabled indexing of q-svc logs in logstash | 15:31 |
| ralonsoh | yeah, the resource extend logging was excesive in this case | 15:31 |
| slaweq | I will try to check there what else we can reduce there | 15:31 |
| slaweq | but if You have any ideas, feel free to propose patches | 15:31 |
| ralonsoh | (remove debug logs!!) | 15:31 |
| slaweq | ralonsoh: yeah | 15:32 |
| slaweq | but that may make our life harder :P | 15:32 |
| slaweq | next thing | 15:32 |
| slaweq | ralonsoh found and reported today issue https://bugs.launchpad.net/neutron/+bug/1893031 | 15:33 |
| openstack | Launchpad bug 1893031 in neutron "[tempest] "test_dns_domain_and_name" is failing 100% of times" [Undecided,New] | 15:33 |
| slaweq | it should be fixed with https://review.opendev.org/#/c/748140/ which is now in the gate | 15:33 |
| ralonsoh | correct | 15:33 |
| slaweq | from other issues I found today one failure of qos test in linuxbridge job | 15:34 |
| slaweq | https://fb1e48505d4c8944956b-dc45e57a445c28f373928001a4323ec4.ssl.cf5.rackcdn.com/696926/23/check/neutron-tempest-plugin-scenario-linuxbridge/086d79c/testr_results.html | 15:34 |
| slaweq | did You saw something similar before? | 15:34 |
| ralonsoh | let me check | 15:35 |
| ralonsoh | bytes_per_second = 385961, expected_bw = 192000 | 15:35 |
| ralonsoh | hmmmm twice the speed | 15:35 |
| slaweq | yes, that's why I raised it here | 15:36 |
| ralonsoh | no sorry, I never saw this | 15:36 |
| slaweq | because it seems "interesting" for me :) | 15:36 |
| *** belmoreira has quit IRC | 15:36 | |
| ralonsoh | and LB is the most stable backend | 15:36 |
| ralonsoh | (I never said that) | 15:36 |
| ralonsoh | in QoS | 15:36 |
| slaweq | xD | 15:37 |
| ralonsoh | is this the first time you see it? | 15:37 |
| ralonsoh | or is this something recurrent? | 15:37 |
| slaweq | yes | 15:37 |
| ralonsoh | ok | 15:37 |
| ralonsoh | I'll save it, just in case | 15:37 |
| slaweq | ok | 15:38 |
| slaweq | I will try to look deeper into logs, maybe I will find something interesting there | 15:39 |
| slaweq | #slaweq to check logs of qos failure in LB job https://fb1e48505d4c8944956b-dc45e57a445c28f373928001a4323ec4.ssl.cf5.rackcdn.com/696926/23/check/neutron-tempest-plugin-scenario-linuxbridge/086d79c/testr_results.html | 15:39 |
| slaweq | speaking about qos and scenario tests | 15:40 |
| slaweq | long time ago I proposed patch https://review.opendev.org/#/c/721235/ | 15:40 |
| slaweq | please review it when You will have some time | 15:40 |
| slaweq | and that's all from me for today | 15:40 |
| slaweq | periodic jobs seems to be working fine | 15:40 |
| slaweq | anything else You want to talk about today? | 15:41 |
| ralonsoh | one quick question | 15:41 |
| ralonsoh | during those two weeks | 15:41 |
| ralonsoh | with the oslo.privsep change (un-monkey patching the daemon( | 15:41 |
| ralonsoh | and the latests pyroute2 lib version | 15:41 |
| ralonsoh | did you notice the functional/fullstack tests more stable? | 15:42 |
| ralonsoh | in other works | 15:42 |
| ralonsoh | words | 15:42 |
| ralonsoh | did you see timeouts? | 15:42 |
| slaweq | looking at http://grafana.openstack.org/d/Hj5IHcSmz/neutron-failure-rate?viewPanel=24&orgId=1&from=now-30d&to=now I don't think it is more stable than it was | 15:42 |
| slaweq | maybe slightly better | 15:43 |
| slaweq | fullstack seems to be pretty good recently | 15:43 |
| ralonsoh | yes, a 0.001% better | 15:43 |
| ralonsoh | hehehehe | 15:43 |
| slaweq | :) | 15:43 |
| bcafarel | :) | 15:43 |
| ralonsoh | once we have George | 15:44 |
| slaweq | but even if it's 0.001% better, it's step in good direction | 15:44 |
| ralonsoh | we can think about moving some fullstack tests to George | 15:44 |
| slaweq | yes, I was reviewing George patches today | 15:44 |
| ralonsoh | using containers will help to isolate some noisy tests | 15:44 |
| slaweq | looks good and very interesting | 15:44 |
| ralonsoh | a lot!! | 15:44 |
| ralonsoh | that's what fullstack test should have been implemented | 15:44 |
| ralonsoh | but this is easy to say this now | 15:44 |
| slaweq | yes | 15:44 |
| slaweq | ok, if You don't have anything else for today, lets finish this meeting earlier | 15:46 |
| slaweq | thx for attending | 15:46 |
| ralonsoh | bye! | 15:46 |
| slaweq | #endmeeting | 15:46 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:46 | |
| slaweq | o/ | 15:46 |
| openstack | Meeting ended Wed Aug 26 15:46:44 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:46 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-08-26-15.00.html | 15:46 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-08-26-15.00.txt | 15:46 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-08-26-15.00.log.html | 15:46 |
| bcafarel | o/ | 15:47 |
| lajoskatona | Bye | 15:47 |
| tosky | ups, I missed the last minute | 15:47 |
| slaweq | tosky: hi | 15:48 |
| slaweq | tosky: do You want to discuss something specific? | 15:48 |
| slaweq | tosky: regarding migration to zuulv3, I'm still working on neutron-grenade-ovn job but I hope it will be ready soon | 15:48 |
| slaweq | and regaring networking-midonet, I'm still waiting for info from someone who wanted to maintain the project but it doesn't seems like he will be able to do that | 15:49 |
| slaweq | so probably we will simply drop those midonet legacy jobs soon | 15:49 |
| slaweq | tosky: and that's all update which I have regarding this | 15:50 |
| tosky | and the 3 jobs in openstack-zuul-jobs? | 15:50 |
| slaweq | tosky: those I didn't touch yet | 15:50 |
| slaweq | sorry | 15:50 |
| tosky | let's jump on the neutron channel maybe | 15:50 |
| slaweq | sure | 15:50 |
| tosky | I have a question about 2 of them which may help | 15:50 |
| *** yamamoto has joined #openstack-meeting-3 | 15:55 | |
| *** yamamoto has quit IRC | 16:05 | |
| *** artom has quit IRC | 16:10 | |
| *** dosaboy has quit IRC | 16:19 | |
| *** dosaboy has joined #openstack-meeting-3 | 16:19 | |
| *** psachin has quit IRC | 16:21 | |
| *** yamamoto has joined #openstack-meeting-3 | 16:38 | |
| *** yamamoto has quit IRC | 16:47 | |
| *** lpetrut has joined #openstack-meeting-3 | 16:49 | |
| *** lpetrut has quit IRC | 16:49 | |
| *** ralonsoh has quit IRC | 16:51 | |
| *** artom has joined #openstack-meeting-3 | 16:56 | |
| *** bnemec has quit IRC | 17:18 | |
| *** lajoskatona has quit IRC | 17:27 | |
| *** artom has quit IRC | 17:45 | |
| *** belmoreira has joined #openstack-meeting-3 | 17:57 | |
| *** belmoreira has quit IRC | 18:30 | |
| *** belmoreira has joined #openstack-meeting-3 | 18:32 | |
| *** e0ne has quit IRC | 18:34 | |
| *** baojg has quit IRC | 18:52 | |
| *** baojg has joined #openstack-meeting-3 | 18:53 | |
| *** belmoreira has quit IRC | 19:12 | |
| *** e0ne has joined #openstack-meeting-3 | 19:18 | |
| *** tosky has quit IRC | 19:38 | |
| *** artom has joined #openstack-meeting-3 | 19:55 | |
| *** e0ne has quit IRC | 19:58 | |
| *** bnemec has joined #openstack-meeting-3 | 20:40 | |
| *** yamamoto has joined #openstack-meeting-3 | 20:44 | |
| *** yamamoto has quit IRC | 20:51 | |
| *** slaweq has quit IRC | 21:18 | |
| *** bnemec has quit IRC | 21:42 | |
| *** yamamoto has joined #openstack-meeting-3 | 21:45 | |
| *** bnemec has joined #openstack-meeting-3 | 21:52 | |
| *** yamamoto has quit IRC | 22:00 | |
| *** raildo_ has joined #openstack-meeting-3 | 22:01 | |
| *** raildo has quit IRC | 22:04 | |
| *** bnemec has quit IRC | 22:09 | |
| *** bnemec has joined #openstack-meeting-3 | 22:21 | |
| *** bnemec has quit IRC | 22:44 | |
| *** bnemec has joined #openstack-meeting-3 | 22:48 | |
| *** bnemec has quit IRC | 22:54 | |
| *** yamamoto has joined #openstack-meeting-3 | 23:16 | |
| *** baojg has quit IRC | 23:21 | |
| *** bnemec has joined #openstack-meeting-3 | 23:21 | |
| *** baojg has joined #openstack-meeting-3 | 23:22 | |
| *** bnemec has quit IRC | 23:28 | |
| *** bnemec has joined #openstack-meeting-3 | 23:50 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!