16:00:04 <slaweq> #startmeeting neutron_ci
16:00:05 <openstack> Meeting started Tue Nov 26 16:00:04 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:08 <openstack> The meeting name has been set to 'neutron_ci'
16:00:18 <njohnston> o/
16:00:41 <slaweq> hi
16:01:49 <bcafarel> o/
16:02:52 <slaweq> liuyulong: ralonsoh: CI meeting, are You around?
16:03:02 <ralonsoh> hi sorry
16:03:25 <slaweq> ok, lets start
16:03:27 <slaweq> Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate
16:03:31 <slaweq> please open the link first
16:04:26 <slaweq> #topic Actions from previous meetings
16:04:36 <slaweq> first one is
16:04:38 <slaweq> njohnston prepare etherpad to track stadium progress for zuul v3 job definition and py2 support drop
16:04:56 <njohnston> Done!  Do you want to go over it now or in the stadium section
16:05:09 <njohnston> ?
16:05:41 <njohnston> #link https://etherpad.openstack.org/p/neutron-train-zuulv3-py27drop
16:06:30 <slaweq> thx njohnston
16:06:38 <slaweq> we can go over this list in stadium section
16:06:44 <njohnston> sounds good
16:07:11 <slaweq> we have also second action item
16:07:13 <slaweq> njohnston to check failing NetworkMigrationFromHA in multinode dvr job
16:07:44 <njohnston> working on that this morning, haven't gotten to the bottom of it yet
16:08:09 <slaweq> ok, sure
16:08:16 <njohnston> I didn't get to it until about 30 minutes ago :(
16:08:49 <slaweq> ok
16:09:03 <slaweq> can I assign it to You for next week also as a reminder?
16:09:10 <njohnston> sure
16:09:28 <slaweq> #action njohnston to check failing NetworkMigrationFromHA in multinode dvr job
16:09:30 <slaweq> thx
16:09:40 <slaweq> and that are all actions from last week for today
16:10:00 <slaweq> #topic Stadium projects
16:11:02 <slaweq> tempest-plugins migration
16:11:27 <slaweq> njohnston: recently pushed phase 2 patch for neutron-dynamic-routing: https://review.opendev.org/#/c/695014/
16:11:34 <slaweq> I already +2 it
16:11:39 <slaweq> please also review this one
16:11:55 <slaweq> after this will be merged we will only need vpnaas to do still
16:12:08 <slaweq> and I know mlavalle is making some progress with it
16:12:14 <bcafarel> the removal step is usually easier yes
16:13:30 <slaweq> bcafarel: exactly
16:13:48 <njohnston> So just to show the link again: https://etherpad.openstack.org/p/neutron-train-zuulv3-py27drop
16:13:51 <slaweq> so this should be basically all about this migration of plugins
16:14:00 <slaweq> yes, lets go with Your link njohnston
16:14:05 <njohnston> I went through all the projects looking for zuul legacy jobs for zuulv3
16:14:52 <njohnston> For py27, I wanted to note that the goal is just to drop testing and other flags (like in setup.cfg), not necessarily to merge code hostile to py27 at this time
16:15:16 <njohnston> Also there is agreed upon a phased approach, so I designate each project in a phase for when to remove support
16:15:31 <njohnston> We are in phase 1, phase 2 starts at week R-22
16:16:15 <njohnston> Obviously there is a lot of content in the etherpad, are there specific points or projects people would like to discuss?
16:16:37 <njohnston> I specifically exempted networking-ovn since master development is going to cease there
16:17:05 <njohnston> and networking-tempest-plugin because the branchless tempest makes py27 something dependent on tempest and I need to check exactly where we are on that
16:17:09 <slaweq> one thing about legacy to zuulv3 convertion
16:17:19 <slaweq> all grenade jobs are sill legacy
16:17:35 <slaweq> so we will need to convert them but it has to be done first in grenade repo
16:17:56 <njohnston> What work do we need to do in order to convert them, if the job definition comes from the grenade repo?
16:18:42 <slaweq> for jobs which are defined in grenade repo I think we don't need anything to do
16:18:56 <slaweq> but we have e.g. neutron-grenade-multinode job which is defined in neutron repo
16:18:59 <njohnston> For this survey I only examined jobs that are natively defined in the repo, so I did not consider grenade at all except where jobs were defined natively for grenade like networking-midonet-grenade-ml2 or networking-odl-grenade
16:19:05 <slaweq> and we will need to convert it at some point
16:19:26 <bcafarel> any specific work items we should start looking into first? or just pick one/add our name/mark as done steps
16:19:42 <njohnston> pick anything in a phase 1 project
16:20:13 <slaweq> ok, I will try to pick some of those soon too
16:20:53 <njohnston> thanks!
16:21:00 <bcafarel> ack I'll clean sfc py2 stuff as a start
16:21:14 <slaweq> thanks for preparing this etherpad - it's great list of things todo
16:21:16 <slaweq> :)
16:21:34 <njohnston> this is a good example of a py27 change: https://review.opendev.org/#/c/692031/
16:21:40 <njohnston> I'll link that at the top
16:22:42 <bcafarel> +1 I was looking at neutron removal, but better if we have a sample for stadium project
16:22:42 <slaweq> thx
16:24:15 <slaweq> ok, do You have anything else regarding stadium projects?
16:24:21 <slaweq> or can we move on?
16:24:26 <njohnston> no, I think that is all.
16:25:00 <slaweq> ok, so lets move on
16:25:07 <slaweq> #topic Grafana
16:25:21 <slaweq> #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate
16:26:18 <slaweq> neutron-grenade job is going to high failure rate recently
16:26:32 <slaweq> but this is now removed from queues as part of dropping of py27 support
16:29:09 <njohnston> we'll have to see if the spike in e.g. check queue unit test jobs continues today
16:29:44 <slaweq> yes, and also functiona/fullstack tests
16:30:04 <slaweq> but for functional and fullstack I saw at least couple of changes today where job failed due to patch on which it was run
16:30:14 <slaweq> so those were not an issues with job itself
16:30:42 <njohnston> right, so hopefully we can wait and see it revert to mean
16:30:54 <slaweq> yes
16:31:00 <slaweq> from what I was checking today
16:31:15 <slaweq> I noticed only one issue which hits as quite often
16:31:22 <slaweq> https://bugs.launchpad.net/neutron/+bug/1850557
16:31:22 <openstack> Launchpad bug 1850557 in neutron "DHCP connectivity after migration/resize not working" [Medium,Confirmed] - Assigned to Slawek Kaplonski (slaweq)
16:31:38 <slaweq> I just raised Importance of this bug to High
16:31:58 <slaweq> I was trying to reproduce it locally but I wasn't able to do that
16:32:16 <slaweq> but I will continue investigating this issue
16:33:18 <slaweq> and that's all what I have about grafana for today
16:33:26 <slaweq> anything else You want to add/ask?
16:33:48 <ralonsoh> in https://bugs.launchpad.net/neutron/+bug/1850557
16:33:48 <openstack> Launchpad bug 1850557 in neutron "DHCP connectivity after migration/resize not working" [High,Confirmed] - Assigned to Slawek Kaplonski (slaweq)
16:33:56 <ralonsoh> we have only one compute node
16:34:04 <ralonsoh> actually this is functional
16:34:19 <slaweq> there is one compute and one "all-in-one" node
16:34:43 <ralonsoh> ok so the OF rules should be the same
16:35:19 <slaweq> yes, and as I wrote in comment #2 DHCP request comes from VM to the DHCP server after migration
16:35:29 <slaweq> so connectivity at least in this direction works fine
16:35:45 <slaweq> I can't say where this response is lost exactly
16:38:38 <slaweq> #action slaweq to continue investigating issue https://bugs.launchpad.net/neutron/+bug/1850557
16:38:39 <openstack> Launchpad bug 1850557 in neutron "DHCP connectivity after migration/resize not working" [High,Confirmed] - Assigned to Slawek Kaplonski (slaweq)
16:38:42 <slaweq> ok, lets move on
16:38:50 <slaweq> #topic Tempest/Scenario
16:39:12 <slaweq> as I said already I'm aware only about this one issue which imacts us now
16:39:41 <slaweq> but ralonsoh did You maybe had chance to check my proposal of merging/dropping some tempest/neutron-tempest-plugin jobs?
16:39:44 <njohnston> cool
16:40:01 <ralonsoh> slaweq, yes but not enough time
16:40:16 <ralonsoh> slaweq, I created a list of tests, from both jobs
16:40:29 <ralonsoh> to see how many of them can be merged
16:40:36 <ralonsoh> but I didn't finish sorry
16:41:15 <slaweq> ralonsoh: no problem, if You will have some time, please reply to my email about it and we will continue this discussion there
16:41:23 <ralonsoh> sure
16:41:35 <slaweq> thx a lot
16:41:41 <slaweq> and one last thing from me
16:41:50 <slaweq> like a heads-up
16:42:02 <slaweq> we are starting process of merging networking-ovn driver to neutron
16:42:12 <njohnston> +100
16:42:13 <slaweq> so I pushed patch https://review.opendev.org/#/c/696094/ to move ovn jobs to neutron repo
16:42:39 <slaweq> it's not ready yet but please be aware of it and take a look at this patch when it will be not WIP anymore
16:42:49 <njohnston> ok
16:42:58 <slaweq> also I'm not sure if we shouldn't move from .zuul.yaml file to zuul.d/ directory
16:43:08 <slaweq> and add definition of different jobs there
16:43:21 <slaweq> as this .zuul.yaml file is getting to be nightmare now
16:43:31 <slaweq> what do You think about such potential change?
16:43:37 <ralonsoh> slaweq, we can move it once we finish the migration
16:43:47 <njohnston> I think it's a good idea to do
16:43:51 <ralonsoh> but yes, it's better to have several files in a directory
16:44:05 <slaweq> ok, thx for supporting this idea
16:44:21 <slaweq> ralonsoh: I will try to do it "in parallel" to ovn-merge work
16:44:28 <ralonsoh> sure
16:44:38 <slaweq> if this will be merged first I will rebase my networking-ovn jobs patch
16:44:58 <slaweq> or I will rebase this "zuul.d dir" patch if needed :)
16:45:13 <slaweq> #action slaweq to move job definitions to zuul.d directory
16:45:15 <bcafarel> +1 to move to split files in zuul.d later it is easier to read
16:45:44 <slaweq> ok, that's all what I have for today
16:46:03 <slaweq> periodic jobs are working very well since few days at least
16:46:08 <slaweq> so we are good there
16:46:15 <slaweq> do You have anything else to talk about now?
16:46:25 <slaweq> or if not, we can finish a bit earlier today
16:46:27 <ralonsoh> just 10 secs, to "sell" my patches, related to bugs in Neutron
16:46:29 <ralonsoh> https://review.opendev.org/#/c/695060/
16:46:36 <ralonsoh> njohnston, ^^ as an expert in DBs
16:46:39 <ralonsoh> that's all
16:46:58 <njohnston> ralonsoh: I'll take a look right away
16:47:06 <slaweq> I even set it as "important change" to review :)
16:47:07 <ralonsoh> thanks!!
16:47:56 <njohnston> thanks for a great meeting all
16:48:07 <slaweq> ok, so I think we can finish earlier today
16:48:10 <slaweq> thx for attending
16:48:14 <slaweq> and see You online
16:48:15 <slaweq> o/
16:48:15 <njohnston> o/
16:48:18 <slaweq> #endmeeting