| *** igordc has quit IRC | 00:02 | |
| *** gyee has quit IRC | 00:13 | |
| *** mdbooth_ has joined #openstack-meeting | 00:16 | |
| *** mdbooth has quit IRC | 00:17 | |
| *** lbragstad has quit IRC | 00:18 | |
| *** mdbooth_ has quit IRC | 00:27 | |
| *** diablo_rojo has quit IRC | 00:35 | |
| *** lseki has quit IRC | 00:36 | |
| *** yamamoto has joined #openstack-meeting | 00:46 | |
| *** yamamoto has quit IRC | 00:50 | |
| *** wwriverrat has joined #openstack-meeting | 00:53 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:53 | |
| *** bbowen has quit IRC | 00:59 | |
| *** hongbin has quit IRC | 01:00 | |
| *** cheng1 has joined #openstack-meeting | 01:03 | |
| *** mdbooth has joined #openstack-meeting | 01:03 | |
| *** yamamoto has joined #openstack-meeting | 01:17 | |
| *** takashin has joined #openstack-meeting | 01:18 | |
| *** takashin has left #openstack-meeting | 01:18 | |
| *** yamamoto has quit IRC | 01:22 | |
| *** bbowen has joined #openstack-meeting | 01:27 | |
| *** ekcs has joined #openstack-meeting | 01:27 | |
| *** jamesmcarthur has quit IRC | 01:30 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:38 | |
| *** mdbooth_ has joined #openstack-meeting | 01:40 | |
| *** mdbooth has quit IRC | 01:42 | |
| *** whoami-rajat has joined #openstack-meeting | 01:57 | |
| *** njohnston has quit IRC | 02:05 | |
| *** erlon has quit IRC | 02:11 | |
| *** enriquetaso has quit IRC | 02:13 | |
| *** jamesmcarthur has quit IRC | 02:13 | |
| *** iyamahat__ has quit IRC | 02:28 | |
| *** yamahata has quit IRC | 02:28 | |
| *** cheng1 has quit IRC | 02:29 | |
| *** lbragstad has joined #openstack-meeting | 02:29 | |
| *** hongbin has joined #openstack-meeting | 02:34 | |
| *** cheng1 has joined #openstack-meeting | 02:36 | |
| *** yamamoto has joined #openstack-meeting | 02:37 | |
| *** yamamoto has quit IRC | 02:42 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:48 | |
| *** armax has quit IRC | 02:51 | |
| *** jamesmcarthur has quit IRC | 02:51 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:52 | |
| *** Liang__ has joined #openstack-meeting | 03:03 | |
| *** Liang__ is now known as LiangFang | 03:07 | |
| *** jamesmcarthur has quit IRC | 03:14 | |
| *** apetrich has quit IRC | 03:14 | |
| *** cheng1 has quit IRC | 03:16 | |
| *** yamamoto has joined #openstack-meeting | 03:23 | |
| *** yamamoto has quit IRC | 03:27 | |
| *** igordc has joined #openstack-meeting | 03:30 | |
| *** yamamoto has joined #openstack-meeting | 03:30 | |
| *** yamahata has joined #openstack-meeting | 03:40 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:44 | |
| *** jamesmcarthur has quit IRC | 03:48 | |
| *** igordc has quit IRC | 03:54 | |
| *** janki has joined #openstack-meeting | 03:59 | |
| *** cheng1 has joined #openstack-meeting | 03:59 | |
| *** imsurit_ofc has joined #openstack-meeting | 03:59 | |
| samP | Hi all for masakari | 04:00 |
|---|---|---|
| samP | #startmeeting masakari | 04:00 |
| openstack | Meeting started Tue Mar 26 04:00:39 2019 UTC and is due to finish in 60 minutes. The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot. | 04:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 04:00 |
| *** openstack changes topic to " (Meeting topic: masakari)" | 04:00 | |
| openstack | The meeting name has been set to 'masakari' | 04:00 |
| samP | Hi all! | 04:00 |
| samP | #topic Stein Release | 04:01 |
| *** openstack changes topic to "Stein Release (Meeting topic: masakari)" | 04:01 | |
| samP | Masakari RC1 is released with tag 7.0.0.0rc1 for masakari and masakari-monitors | 04:02 |
| *** shilpasd has joined #openstack-meeting | 04:03 | |
| samP | related python-masakariclient version is 5.4.0 | 04:03 |
| samP | I am planning to release a new version for masakari-dashborad soon. | 04:04 |
| samP | I have just approved release bot patches for python-masakariclient | 04:06 |
| samP | once those are merged, all the masakari repos are ready to accept changes for next release. | 04:07 |
| samP | #topic Stein Backports | 04:07 |
| *** openstack changes topic to "Stein Backports (Meeting topic: masakari)" | 04:07 | |
| samP | There are some patches still in the review state. Which are not critical but good to have. | 04:08 |
| *** tashiromt has joined #openstack-meeting | 04:09 | |
| samP | Once they merged to current master, please consider proposing the backport to stable/stain | 04:09 |
| samP | #topic AOB | 04:09 |
| *** openstack changes topic to "AOB (Meeting topic: masakari)" | 04:09 | |
| shilpasd | yes, sure | 04:09 |
| samP | shilpasd: hi | 04:09 |
| *** yamamoto has quit IRC | 04:10 | |
| shilpasd | samP: hi, one request | 04:10 |
| shilpasd | During devstack host-monitor testing, unknowingly 192.168.10.155 is shutdown and there is no way we can bring that machine back. | 04:10 |
| *** yamamoto has joined #openstack-meeting | 04:10 | |
| shilpasd | can you please bring this back for us | 04:10 |
| *** hongbin has quit IRC | 04:10 | |
| samP | shilpasd: sure, I will fix it. | 04:10 |
| shilpasd | thank you | 04:10 |
| samP | is it OK to reset the node? or you need the data? | 04:11 |
| shilpasd | is it okay to reset | 04:11 |
| samP | OK then, I will put a fresh OS on it. | 04:12 |
| samP | I will let you know IRC #openstack-masakari when its done | 04:12 |
| shilpasd | sure, thnaks | 04:13 |
| samP | I dont have any other topics to share. | 04:13 |
| samP | shilpasd: do you have any items to share? | 04:13 |
| shilpasd | samP: no | 04:14 |
| samP | shilpasd: thans | 04:14 |
| samP | /thans/thanks | 04:14 |
| shilpasd | samP: thanks | 04:14 |
| samP | s/thans/thanks | 04:14 |
| samP | sorry :) | 04:14 |
| samP | Let's finish today's meeting | 04:14 |
| samP | Please use #openstack-masakari IRC channel @freenode or openstack ML for further discussions | 04:15 |
| samP | Thank you all! | 04:15 |
| samP | #endmeeting | 04:15 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 04:15 | |
| openstack | Meeting ended Tue Mar 26 04:15:45 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 04:15 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-03-26-04.00.html | 04:15 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-03-26-04.00.txt | 04:15 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-03-26-04.00.log.html | 04:15 |
| *** janki has quit IRC | 04:16 | |
| *** janki has joined #openstack-meeting | 04:17 | |
| *** shilpasd has left #openstack-meeting | 04:18 | |
| *** sridharg has joined #openstack-meeting | 04:27 | |
| *** lbragstad has quit IRC | 04:34 | |
| *** rbudden has quit IRC | 04:44 | |
| *** imsurit_ofc has quit IRC | 04:56 | |
| *** imsurit_ofc has joined #openstack-meeting | 04:57 | |
| *** ekcs has quit IRC | 04:57 | |
| *** ekcs has joined #openstack-meeting | 05:03 | |
| *** ekcs has quit IRC | 05:06 | |
| *** yamamoto has quit IRC | 05:09 | |
| *** markvoelker has joined #openstack-meeting | 05:17 | |
| *** cheng1_ has joined #openstack-meeting | 05:41 | |
| *** cheng1 has quit IRC | 05:43 | |
| *** diablo_rojo has joined #openstack-meeting | 05:54 | |
| *** diablo_rojo has quit IRC | 06:27 | |
| *** Luzi has joined #openstack-meeting | 06:43 | |
| *** cheng1__ has joined #openstack-meeting | 06:43 | |
| *** cheng1_ has quit IRC | 06:46 | |
| *** kopecmartin|off is now known as kopecmartin | 07:00 | |
| *** cheng1__ has quit IRC | 07:04 | |
| *** cheng1 has joined #openstack-meeting | 07:09 | |
| *** kaisers has joined #openstack-meeting | 07:09 | |
| *** pcaruana has joined #openstack-meeting | 07:10 | |
| *** apetrich has joined #openstack-meeting | 07:12 | |
| *** slaweq has quit IRC | 07:15 | |
| *** joxyuki has joined #openstack-meeting | 07:16 | |
| *** slaweq has joined #openstack-meeting | 07:17 | |
| *** e0ne has joined #openstack-meeting | 07:25 | |
| *** ivy has joined #openstack-meeting | 07:54 | |
| *** iyamahat has joined #openstack-meeting | 07:56 | |
| *** joxyuki has left #openstack-meeting | 08:03 | |
| *** iyamahat has quit IRC | 08:06 | |
| *** LiangFang has quit IRC | 08:18 | |
| *** jrbalderrama has joined #openstack-meeting | 08:22 | |
| *** tssurya has joined #openstack-meeting | 08:28 | |
| *** ttsiouts has joined #openstack-meeting | 08:30 | |
| *** rcernin has quit IRC | 08:36 | |
| *** electrofelix has joined #openstack-meeting | 08:40 | |
| *** ttsiouts has quit IRC | 08:43 | |
| *** ttsiouts has joined #openstack-meeting | 08:43 | |
| *** ttsiouts has quit IRC | 08:47 | |
| *** ralonsoh has joined #openstack-meeting | 08:54 | |
| *** tetsuro has joined #openstack-meeting | 08:57 | |
| *** priteau has joined #openstack-meeting | 08:58 | |
| *** Lucas_Gray has joined #openstack-meeting | 09:00 | |
| *** priteau has quit IRC | 09:01 | |
| *** ttsiouts has joined #openstack-meeting | 09:02 | |
| *** priteau has joined #openstack-meeting | 09:02 | |
| *** rossella_s has quit IRC | 09:06 | |
| *** rossella_s has joined #openstack-meeting | 09:11 | |
| *** mdbooth has joined #openstack-meeting | 09:19 | |
| *** mdbooth_ has quit IRC | 09:21 | |
| *** a-pugachev has joined #openstack-meeting | 09:39 | |
| *** cheng1 has quit IRC | 09:44 | |
| *** rcernin has joined #openstack-meeting | 09:45 | |
| *** e0ne has quit IRC | 09:49 | |
| *** imsurit_ofc has quit IRC | 09:50 | |
| *** Luzi has quit IRC | 09:52 | |
| *** Lucas_Gray has quit IRC | 09:55 | |
| *** e0ne has joined #openstack-meeting | 09:56 | |
| *** Lucas_Gray has joined #openstack-meeting | 09:56 | |
| *** Lucas_Gray has quit IRC | 10:00 | |
| *** Lucas_Gray has joined #openstack-meeting | 10:01 | |
| *** zhubx has quit IRC | 10:06 | |
| *** zhubx has joined #openstack-meeting | 10:07 | |
| *** zhubx has quit IRC | 10:08 | |
| *** zhubx has joined #openstack-meeting | 10:08 | |
| *** zhubx has quit IRC | 10:09 | |
| *** zhubx has joined #openstack-meeting | 10:10 | |
| *** rcernin has quit IRC | 10:24 | |
| *** bbowen has quit IRC | 10:48 | |
| *** psachin has joined #openstack-meeting | 10:50 | |
| *** e0ne has quit IRC | 10:59 | |
| *** a-pugachev has quit IRC | 11:03 | |
| *** a-pugachev has joined #openstack-meeting | 11:06 | |
| *** njohnston has joined #openstack-meeting | 11:06 | |
| *** a-pugachev has quit IRC | 11:06 | |
| *** a-pugachev has joined #openstack-meeting | 11:07 | |
| *** tetsuro has quit IRC | 11:19 | |
| *** cheng1 has joined #openstack-meeting | 11:26 | |
| *** a-pugachev has quit IRC | 11:28 | |
| *** priteau has quit IRC | 11:29 | |
| *** a-pugachev has joined #openstack-meeting | 11:29 | |
| *** e0ne has joined #openstack-meeting | 11:37 | |
| *** bbowen has joined #openstack-meeting | 11:58 | |
| *** ttsiouts has quit IRC | 12:00 | |
| *** ttsiouts has joined #openstack-meeting | 12:00 | |
| *** ttsiouts has quit IRC | 12:05 | |
| *** tashiromt has quit IRC | 12:05 | |
| *** e0ne has quit IRC | 12:22 | |
| *** ttsiouts has joined #openstack-meeting | 12:26 | |
| *** e0ne has joined #openstack-meeting | 12:27 | |
| *** markvoelker has quit IRC | 12:31 | |
| *** liuyulong has joined #openstack-meeting | 12:32 | |
| *** radez has joined #openstack-meeting | 12:41 | |
| *** janki has quit IRC | 12:45 | |
| *** janki has joined #openstack-meeting | 12:45 | |
| *** lbragstad has joined #openstack-meeting | 12:46 | |
| *** jamesmcarthur has joined #openstack-meeting | 12:46 | |
| *** janki has quit IRC | 12:46 | |
| *** janki has joined #openstack-meeting | 12:46 | |
| *** lseki has joined #openstack-meeting | 12:57 | |
| *** enriquetaso has joined #openstack-meeting | 13:07 | |
| *** rubasov has joined #openstack-meeting | 13:08 | |
| *** e0ne has quit IRC | 13:08 | |
| *** mriedem has joined #openstack-meeting | 13:09 | |
| *** e0ne has joined #openstack-meeting | 13:11 | |
| *** Luzi has joined #openstack-meeting | 13:15 | |
| *** e0ne has quit IRC | 13:18 | |
| *** davidsha has joined #openstack-meeting | 13:22 | |
| *** e0ne has joined #openstack-meeting | 13:26 | |
| *** janki has quit IRC | 13:42 | |
| *** eharney has joined #openstack-meeting | 13:43 | |
| *** zhubx has quit IRC | 13:46 | |
| *** zhubx has joined #openstack-meeting | 13:46 | |
| *** mjturek has joined #openstack-meeting | 13:55 | |
| *** awaugama has joined #openstack-meeting | 13:56 | |
| *** ircuser-1 has quit IRC | 13:57 | |
| *** cheng1_ has joined #openstack-meeting | 13:59 | |
| *** lajoskatona has joined #openstack-meeting | 13:59 | |
| *** tidwellr has joined #openstack-meeting | 14:00 | |
| *** boden has joined #openstack-meeting | 14:00 | |
| *** njohnston is now known as njohnston_ | 14:00 | |
| *** njohnsto_ has joined #openstack-meeting | 14:01 | |
| *** mlavalle has joined #openstack-meeting | 14:01 | |
| mlavalle | #startmeeting networking | 14:01 |
| openstack | Meeting started Tue Mar 26 14:01:40 2019 UTC and is due to finish in 60 minutes. The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| *** openstack changes topic to " (Meeting topic: networking)" | 14:01 | |
| openstack | The meeting name has been set to 'networking' | 14:01 |
| liuyulong | hi | 14:01 |
| tidwellr | hi | 14:02 |
| haleyb | hi | 14:02 |
| lajoskatona | Hi | 14:02 |
| njohnsto_ | o/ | 14:02 |
| rubasov | o/ | 14:02 |
| ralonsoh | hi | 14:02 |
| mlavalle | how are you all! | 14:02 |
| *** njohnsto_ is now known as njohnston | 14:02 | |
| boden | howdy | 14:02 |
| njohnston | splendid, and you? | 14:02 |
| *** cheng1 has quit IRC | 14:02 | |
| mlavalle | as usual, you will find updated agenda here: https://wiki.openstack.org/wiki/Network/Meetings | 14:03 |
| mlavalle | #topic Announcements | 14:03 |
| *** openstack changes topic to "Announcements (Meeting topic: networking)" | 14:03 | |
| bcafarel | late o/ | 14:03 |
| mlavalle | First of all, we cut RC-1 over the weekend | 14:05 |
| *** kaisers has quit IRC | 14:05 | |
| mlavalle | https://review.openstack.org/#/c/644339/ | 14:05 |
| mlavalle | We have until April 5th in case we need to cut another RC | 14:06 |
| mlavalle | opr several | 14:06 |
| *** kaisers has joined #openstack-meeting | 14:06 | |
| mlavalle | And please remember that the etherpad for the PTG is up for topics suggestions: https://etherpad.openstack.org/p/openstack-networking-train-ptg | 14:07 |
| mlavalle | I can see that a healthy number of you all will be in attendance | 14:07 |
| mlavalle | Looking forward to see you there | 14:07 |
| mlavalle | \o/ | 14:07 |
| bcafarel | and without train horns in the night :) | 14:08 |
| mlavalle | that's right :-) | 14:08 |
| mlavalle | any other annoucements I might have missed? | 14:09 |
| mlavalle | ok, let's move on | 14:10 |
| mlavalle | #topic Blueprints | 14:10 |
| *** openstack changes topic to "Blueprints (Meeting topic: networking)" | 14:10 | |
| mlavalle | These were the blueprints we were targeting for our RC-1: https://launchpad.net/neutron/+milestone/stein-rc1 | 14:10 |
| *** jamesmcarthur has quit IRC | 14:11 | |
| mlavalle | As you can all see, we merged almost everything we targeted for RC-1 | 14:11 |
| njohnston | \o/ | 14:11 |
| bcafarel | yeah, lots of green in that list | 14:12 |
| mlavalle | liuyulong: this was assigned to you: https://bugs.launchpad.net/neutron/+bug/1813787 | 14:12 |
| openstack | Launchpad bug 1813787 in neutron "[L3] DVR router in compute node was not up but nova port needs its functionality" [Medium,In progress] - Assigned to LIU Yulong (dragon889) | 14:12 |
| mlavalle | do you have a status report on it? | 14:13 |
| *** jamesmcarthur has joined #openstack-meeting | 14:13 | |
| liuyulong | Yes | 14:13 |
| *** jamesmcarthur has quit IRC | 14:14 | |
| liuyulong | For the new approach slaweq and me mentioned in the patch set, I met some refactor problem. | 14:14 |
| liuyulong | It will become much more complicated. | 14:14 |
| mlavalle | liuyulong: thanks for the follow up! | 14:15 |
| liuyulong | But, FYI, our local environment is running the current approach, everything works fine. | 14:15 |
| liuyulong | We've tested concurrently boot 10/20/30 VM under different DVR router. | 14:16 |
| liuyulong | It can work, all VM get the metadata. But indeed, some retry can be seen during the boot time. | 14:16 |
| *** cheng1_ has quit IRC | 14:16 | |
| mlavalle | ok. here's what I propose then. when it is ready, we can merge it in master and then backport to Stein | 14:17 |
| mlavalle | that way we don't have to rush against the calendar | 14:17 |
| liuyulong | OK | 14:17 |
| mlavalle | and we have time to review it properly | 14:17 |
| mlavalle | does that work for everybody? | 14:17 |
| *** e0ne has quit IRC | 14:18 | |
| liuyulong | The current approach is here: https://review.openstack.org/#/c/633871/, maybe we can get it in first then, I'll try refactor it to the new style. | 14:19 |
| mlavalle | ok, I'll take that as a yes | 14:19 |
| mlavalle | liuyulong: that works | 14:19 |
| mlavalle | so next question..... | 14:20 |
| mlavalle | to the team | 14:20 |
| mlavalle | is there anything critical that we should include in a potential RC-2? | 14:20 |
| bcafarel | not that I know of, but better check with slaweq | 14:21 |
| *** njohnston is now known as njohnston_phone | 14:22 | |
| *** njohnston_ is now known as njohnston | 14:22 | |
| mlavalle | I also think we don't need another RC | 14:23 |
| *** Luzi has quit IRC | 14:23 | |
| mlavalle | if there are bugs that show up over the next few weeks, we can always backport to Stein | 14:23 |
| mlavalle | right? | 14:24 |
| njohnston | makes sense to me | 14:24 |
| bcafarel | and to me too | 14:24 |
| *** e0ne has joined #openstack-meeting | 14:25 | |
| mlavalle | so let's take this as good and we can start merging code into master freely again | 14:25 |
| mlavalle | so merge away! | 14:25 |
| boden | mlavalle and neutron-lib master too right? | 14:25 |
| mlavalle | boden: correct | 14:25 |
| boden | flood gates open | 14:25 |
| bcafarel | :) | 14:25 |
| mlavalle | ok, let's move on | 14:26 |
| mlavalle | #topic Community Goals | 14:26 |
| *** openstack changes topic to "Community Goals (Meeting topic: networking)" | 14:26 | |
| mlavalle | is there anything to be said about py3? | 14:26 |
| njohnston | We remain in compliance with the community py3 goal | 14:27 |
| bcafarel | yes all good for this cycle, maybe some minor changes here and there but I think we are in great shape py3-wise | 14:27 |
| mlavalle | thank so much! | 14:28 |
| njohnston | and I will keep slowly plugging away at the stadium jobs that are above and beyond the basic py3 Stein goal so we are well set up for T and U | 14:28 |
| mlavalle | Thanks to the hard work of several of you, this cycle we have caught up with all the community goals | 14:28 |
| mlavalle | \o/ | 14:28 |
| * mlavalle taking a pause to let that sink in | 14:29 | |
| njohnston | \o/ | 14:29 |
| mlavalle | ok, let's move on | 14:30 |
| mlavalle | #topic Bugs | 14:30 |
| *** openstack changes topic to "Bugs (Meeting topic: networking)" | 14:30 | |
| mlavalle | This past week yours truly was the bugs deputy | 14:31 |
| mlavalle | Here's the report http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004235.html | 14:32 |
| mlavalle | We have two critical bugs: https://bugs.launchpad.net/neutron/+bug/1821384 | 14:33 |
| openstack | Launchpad bug 1821384 in neutron "Customization of ubuntu image fails before upload to devstack" [Critical,Fix released] - Assigned to Slawek Kaplonski (slaweq) | 14:33 |
| *** armax has joined #openstack-meeting | 14:33 | |
| mlavalle | The fix was merged yesterday: https://review.openstack.org/#/c/645883/ | 14:33 |
| mlavalle | and the other critical was https://bugs.launchpad.net/neutron/+bug/1820865 | 14:34 |
| openstack | Launchpad bug 1820865 in neutron "Fullstack tests are failing because of "OSError: [Errno 22] failed to open netns"" [Critical,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 14:34 |
| mlavalle | that I was working on | 14:34 |
| mlavalle | but I see ralonsoh proposed a fix this morning | 14:34 |
| ralonsoh | I have a possible solution | 14:34 |
| ralonsoh | yes, one sec | 14:34 |
| ralonsoh | #link https://review.openstack.org/#/c/647721/ | 14:35 |
| ralonsoh | in one line: to check if the namespace is accesible (aka: we can open it) | 14:35 |
| mlavalle | yeap | 14:36 |
| *** jamesmcarthur has joined #openstack-meeting | 14:36 | |
| mlavalle | so the name space can exist but not ready to be opened, is that it? | 14:36 |
| ralonsoh | exactly, I saw it a couple of times in the CI: the namespace was still open when the test tried to open it | 14:37 |
| njohnston | yes, I saw that this weekend as well | 14:37 |
| mlavalle | I took network_namespace_exists as goof, that is why I was trying to change pyroutes2 version | 14:38 |
| mlavalle | cool, | 14:38 |
| mlavalle | thanks for the fix ralonsoh | 14:38 |
| ralonsoh | np! | 14:38 |
| mlavalle | haleyb: would you have a chance to take a look at https://bugs.launchpad.net/neutron/+bug/1821299? | 14:39 |
| openstack | Launchpad bug 1821299 in neutron "Associating Floating IP with a port can result in duplicate Floating IPs, due to the original FIP not being removed from the SNAT namespace." [Undecided,New] | 14:39 |
| haleyb | mlavalle: yes, will look. did your downgrade of pyroute2 show anything? | 14:39 |
| haleyb | oh, sorry, still on old topic | 14:39 |
| haleyb | yes, i'll look at that too | 14:39 |
| mlavalle | haleyb: the downgrade has been succesful so far. but let's see if ralonsoh fix is the good one | 14:40 |
| mlavalle | it looks like it | 14:40 |
| mlavalle | it makes sense | 14:40 |
| mlavalle | I would also like someone to take a look at https://bugs.launchpad.net/neutron/+bug/1821357 | 14:41 |
| openstack | Launchpad bug 1821357 in neutron "VRRP vip on VM not reachable from other network on DVR setup" [Undecided,New] | 14:41 |
| mlavalle | any other bugs we should discuss today? | 14:41 |
| liuyulong | hi, one small thing | 14:42 |
| liuyulong | https://bugs.launchpad.net/neutron/+bug/1813703 | 14:43 |
| openstack | Launchpad bug 1813703 in neutron "[L2] [summary] ovs-agent issues at large scale" [High,Fix released] - Assigned to LIU Yulong (dragon889) | 14:43 |
| liuyulong | https://review.openstack.org/#/c/638641/ this one change for it. | 14:43 |
| liuyulong | I'd like to raise this change to our meeting as slaweq suggested. | 14:43 |
| liuyulong | It changes ovs-agent iteration log level from DEBUG to INFO. | 14:43 |
| ralonsoh | (I would like to have this patch merged) | 14:44 |
| liuyulong | The reason is obvious. Operators may need to detect the ovs-agent LOG status. | 14:44 |
| liuyulong | And more useful thing is it can be used to estimate port processing time. My OP colleagues love this. | 14:44 |
| liuyulong | But the most noteworthy concern is the log size now. | 14:45 |
| tidwellr | I'm in favor of this, balancing log verbosity can be an art. I think this is useful | 14:45 |
| njohnston | I think it is very valuable, my only thought is that perhaps a release note warning operators that provide very little extra space on the partition where they store their logs that they may need to increase that space as an upgrade concern. | 14:45 |
| liuyulong | After some estimating and local debug, it's about 26Mb or compressed size 1Mb per day. | 14:45 |
| *** zbr has quit IRC | 14:47 | |
| njohnston | If that is the size differential then I think there is no concern, but when in use in an active environment with a lot of ephemeral instances my gut feeling is that the size impact would be greater. | 14:47 |
| mlavalle | liuyulong: thanks. for the sake of time let's continue the conversation in gerrit | 14:47 |
| haleyb | liuyulong: are operators really looking at this level of detail in the agent? it still seems like more of a debugging thing to me | 14:47 |
| *** jrbalderrama has quit IRC | 14:48 | |
| haleyb | i know in the past we had reduced log levels because infra said our logs were too big (and they still are) | 14:48 |
| liuyulong | haleyb, some LOG monitor can be set on these log | 14:48 |
| mlavalle | please post your comment in the patch | 14:49 |
| mlavalle | The deputy for this week is rubasov | 14:49 |
| liuyulong | OK, let's move to the gerrit. | 14:49 |
| mlavalle | who has started triaging already | 14:49 |
| rubasov | ack, started triaging yesterday | 14:49 |
| liuyulong | tidwellr, njohnston, haleyb, thanks for the comments. | 14:49 |
| mlavalle | and also please note that I updated the calendar | 14:49 |
| mlavalle | please take note of your next duty week | 14:50 |
| *** zbr has joined #openstack-meeting | 14:50 | |
| mlavalle | #topic neutron-lib | 14:50 |
| *** openstack changes topic to "neutron-lib (Meeting topic: networking)" | 14:50 | |
| mlavalle | boden: you are up | 14:50 |
| boden | hi.. now that master branches are back open, pls be looking for neutron-lib and consumption patches... I have a few (read 10s) ready | 14:51 |
| boden | that's it | 14:51 |
| mlavalle | boden: LOL | 14:51 |
| mlavalle | great! | 14:51 |
| *** munimeha1 has joined #openstack-meeting | 14:52 | |
| mlavalle | #topic On demand agenda | 14:53 |
| *** openstack changes topic to "On demand agenda (Meeting topic: networking)" | 14:53 | |
| mlavalle | anything else we should discuss today? | 14:53 |
| * mlavalle points out that he just had an out of bad chat with tidwellr, who will be added to the bugs deputy rotation | 14:54 | |
| mlavalle | out of band^^^ | 14:54 |
| njohnston | tidwellr++ | 14:54 |
| tidwellr | mlavalle: never a bad chat with you :) | 14:54 |
| bcafarel | welcome in the band then :) | 14:54 |
| *** diablo_rojo has joined #openstack-meeting | 14:54 | |
| mlavalle | ok team. Thanks for attending | 14:54 |
| mlavalle | enjoy the rest of your week! | 14:55 |
| *** boden has left #openstack-meeting | 14:55 | |
| mlavalle | #endmeeting | 14:55 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 14:55 | |
| openstack | Meeting ended Tue Mar 26 14:55:10 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:55 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-03-26-14.01.html | 14:55 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-03-26-14.01.txt | 14:55 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-03-26-14.01.log.html | 14:55 |
| ralonsoh | bye | 14:55 |
| njohnston | thanks! | 14:55 |
| bcafarel | o/ | 14:55 |
| lajoskatona | bye | 14:55 |
| ralonsoh | (5 mins for the QoS meeting) | 14:55 |
| ralonsoh | #startmeeting neutron_qos | 15:00 |
| openstack | Meeting started Tue Mar 26 15:00:37 2019 UTC and is due to finish in 60 minutes. The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_qos)" | 15:00 | |
| openstack | The meeting name has been set to 'neutron_qos' | 15:00 |
| ralonsoh | Hi | 15:00 |
| davidsha | Hi! | 15:00 |
| *** cheng1 has joined #openstack-meeting | 15:00 | |
| mlavalle | o/ | 15:00 |
| rubasov | hi | 15:01 |
| ralonsoh | ok, let's start | 15:01 |
| ralonsoh | #topic RFEs | 15:01 |
| *** openstack changes topic to "RFEs (Meeting topic: neutron_qos)" | 15:01 | |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1578989 | 15:02 |
| openstack | Launchpad bug 1578989 in neutron "[RFE] Strict minimum bandwidth support (egress)" [Wishlist,Fix released] - Assigned to Bence Romsics (bence-romsics) | 15:02 |
| lajoskatona | o/ | 15:02 |
| ralonsoh | except for some doc patches, almost everything is done | 15:02 |
| ralonsoh | is that right? | 15:02 |
| rubasov | we have some tests remaining | 15:02 |
| rubasov | the doc was merged completely | 15:03 |
| ralonsoh | that's right, the tempest tests | 15:03 |
| rubasov | and a followup rfe to decouple the placement reporter service plugin from ml2 | 15:03 |
| mlavalle | yeap, the doc is in | 15:03 |
| mlavalle | the nova guys were on top of me for that | 15:03 |
| ralonsoh | yes, the rfe is in this list | 15:03 |
| ralonsoh | ok, so keep reviewing the docs and the tests | 15:04 |
| rubasov | lajoskatona: anyhting else I missed? | 15:04 |
| ralonsoh | for the records | 15:04 |
| lajoskatona | RFE for for decoupling: https://bugs.launchpad.net/neutron/+bug/1818479 | 15:04 |
| openstack | Launchpad bug 1818479 in neutron "RFE Decouple placement reporting service plugin from ML2" [Wishlist,New] - Assigned to Bence Romsics (bence-romsics) | 15:04 |
| ralonsoh | tempest test: #link https://review.openstack.org/629142, #link https://review.openstack.org/629253 | 15:04 |
| ralonsoh | fullstack: #link https://review.openstack.org/638666, #link https://review.openstack.org/642527 | 15:04 |
| ralonsoh | yes, we have this RFE in the list | 15:04 |
| ralonsoh | just let's finish this one first | 15:04 |
| lajoskatona | rubasov: I think that's it | 15:05 |
| ralonsoh | Thank you very much | 15:05 |
| ralonsoh | next one then | 15:05 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1818479 | 15:05 |
| openstack | Launchpad bug 1818479 in neutron "RFE Decouple placement reporting service plugin from ML2" [Wishlist,New] - Assigned to Bence Romsics (bence-romsics) | 15:05 |
| ralonsoh | please, lajoskatona | 15:05 |
| davidsha | ralonsoh: Is that the right link? | 15:06 |
| ralonsoh | for this RFE? yes | 15:06 |
| rubasov | slaweq had a code comment on the placement reporter service plugin just before merge we could not fix right there | 15:06 |
| davidsha | kk | 15:06 |
| rubasov | so I have the task of investigating | 15:07 |
| mlavalle | we haven't discussed it in the drivers meeting | 15:07 |
| rubasov | summarize the outcome of that in the rfe and then take it to the drivers meeting | 15:07 |
| mlavalle | please update the RFE with the results of your research and at that point I will schedule it for discussion | 15:08 |
| ralonsoh | rubasov, perfect, do you want me to remove it from the QoS agenda? | 15:08 |
| rubasov | mlavalle: I'll do that | 15:08 |
| rubasov | ralonsoh: yeah, maybe hide it until the drivers discuss it | 15:09 |
| ralonsoh | perfect, thanks! | 15:09 |
| ralonsoh | next one | 15:10 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1777627 | 15:10 |
| openstack | Launchpad bug 1777627 in neutron "RFE: QoS – rule's management (set,delete, show…) requires <qos-policy> parameter in addition to <rule-id>" [Wishlist,Fix committed] - Assigned to Miguel Lavalle (minsel) | 15:10 |
| ralonsoh | Merged! thanks mlavalle | 15:10 |
| mlavalle | \o/ | 15:10 |
| ralonsoh | I'll remove it from the backlog next meeting | 15:10 |
| davidsha | This was the one that only the rule id should be needed right? | 15:10 |
| mlavalle | I have a question, thoiugh | 15:10 |
| ralonsoh | sure | 15:10 |
| mlavalle | davidsha: right | 15:10 |
| mlavalle | https://docs.openstack.org/neutron/latest/admin/config-qos.html#trusted-projects-policy-json-configuration | 15:11 |
| *** e0ne has quit IRC | 15:12 | |
| mlavalle | shouldn't we update this section of the docs, now that we have policy in code? | 15:12 |
| ralonsoh | I don't understand the question | 15:13 |
| mlavalle | I need to add a paragraph indicatin that the alias policies are setup to just follow the other policy | 15:13 |
| ralonsoh | ah, you mean the rule permissions will depend on the policy one | 15:13 |
| ralonsoh | can you submit it? | 15:14 |
| ralonsoh | or do you want me to do it? | 15:14 |
| mlavalle | like here: https://review.openstack.org/#/c/613820/11/neutron/conf/policies/qos.py@241 | 15:15 |
| *** dtrainor_ has joined #openstack-meeting | 15:15 | |
| mlavalle | no, I'll submit it | 15:15 |
| *** simon-AS559 has joined #openstack-meeting | 15:15 | |
| ralonsoh | cool, thanks! | 15:15 |
| mlavalle | I just wanted to point it out | 15:15 |
| *** awaugama has quit IRC | 15:15 | |
| ralonsoh | ok, and the last RFE I have is | 15:16 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1560963 | 15:16 |
| openstack | Launchpad bug 1560963 in neutron "[RFE] Minimum bandwidth support (egress)" [Wishlist,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 15:16 |
| ralonsoh | sorry but I was focused on the CI bugs | 15:16 |
| ralonsoh | and I didn't start the tunnelled traffic patch | 15:17 |
| ralonsoh | I'm planning to start it next week | 15:17 |
| mlavalle | thanks for your hard work! | 15:17 |
| ralonsoh | (I still have a couple of them tracked and prepared) | 15:17 |
| ralonsoh | no more RFEs in the list, next time will be in Train! | 15:18 |
| ralonsoh | let's move to bugs | 15:18 |
| ralonsoh | #topic Bugs | 15:18 |
| *** openstack changes topic to "Bugs (Meeting topic: neutron_qos)" | 15:18 | |
| ralonsoh | a list of merged and closed ones: | 15:19 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1815618 | 15:19 |
| openstack | Launchpad bug 1815618 in neutron "cannot update qos rule" [High,Fix released] - Assigned to Lajos Katona (lajos-katona) | 15:19 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1818683 | 15:19 |
| openstack | Launchpad bug 1818683 in neutron "Placement reporter service plugin sometimes creates orphaned resource providers" [Undecided,Fix released] - Assigned to Bence Romsics (bence-romsics) | 15:19 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1819029 | 15:19 |
| openstack | Launchpad bug 1819029 in neutron "QoS policies with minimum-bandwidth rule should be rejected on non-physnet ports/networks" [Undecided,Fix released] - Assigned to Slawek Kaplonski (slaweq) | 15:19 |
| ralonsoh | thanks lajoskatona rubasov | 15:19 |
| ralonsoh | the next one | 15:20 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1819125 | 15:20 |
| openstack | Launchpad bug 1819125 in neutron "Fullstack test test_min_bw_qos_policy_rule_lifecycle failing often" [Critical,Fix committed] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 15:20 |
| ralonsoh | #link https://review.openstack.org/#/c/646082/ | 15:20 |
| ralonsoh | I have proposed this patch to avoid cleaning the min QoS rules when the agent is started | 15:21 |
| ralonsoh | I did it in functional tests but I missed this one for fullstack | 15:21 |
| ralonsoh | PS3, as commented, is working | 15:21 |
| ralonsoh | PS4 still in the CI | 15:21 |
| ralonsoh | and the last one in the etherpad list | 15:22 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1818696 | 15:22 |
| openstack | Launchpad bug 1818696 in neutron "frequent ci failures trying to delete qos port" [Undecided,Incomplete] | 15:22 |
| ralonsoh | IMO we can close it | 15:22 |
| ralonsoh | this problem was related to a specific patch | 15:22 |
| ralonsoh | and this problem is not happening again | 15:22 |
| mlavalle | cool | 15:23 |
| ralonsoh | If we don't see this problem during the next two weeks (next meeting), we'll close it | 15:23 |
| *** janki has joined #openstack-meeting | 15:23 | |
| ralonsoh | and I don't have more bugs in the list (because QoS doesn't have bugs.....) | 15:24 |
| ralonsoh | anyone I'm missing? | 15:24 |
| mlavalle | nope | 15:24 |
| *** iyamahat has joined #openstack-meeting | 15:24 | |
| ralonsoh | cool | 15:24 |
| ralonsoh | #topic Open Discussion | 15:25 |
| *** openstack changes topic to "Open Discussion (Meeting topic: neutron_qos)" | 15:25 | |
| ralonsoh | davidsha, ? | 15:25 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1476527 | 15:25 |
| openstack | Launchpad bug 1476527 in neutron "[RFE] Add common classifier resource" [Wishlist,Triaged] - Assigned to Igor D.C. (igordcard) | 15:25 |
| davidsha | Yup, I've fixed that DB issue, going to move onto extending the OvS backend this week | 15:25 |
| ralonsoh | for the records: #link https://review.openstack.org/#/c/636333 | 15:26 |
| *** dtrainor_ has quit IRC | 15:27 | |
| ralonsoh | davidsha, do you have any blocker or question? | 15:27 |
| davidsha | There is someone working on Neutron Classifier at the moment thats adding extensions to store the classifications in the agent extensions API of the L2 and L3 agents atm. | 15:27 |
| *** dtrainor has joined #openstack-meeting | 15:27 | |
| davidsha | None from me, but does anyone have questions for me? | 15:27 |
| *** cheng1 has quit IRC | 15:27 | |
| *** e0ne has joined #openstack-meeting | 15:27 | |
| ralonsoh | who's working on this? | 15:28 |
| *** cheng1 has joined #openstack-meeting | 15:28 | |
| ralonsoh | do you have patches? | 15:28 |
| davidsha | Myself on the Neutron side and snierodz is working on the neutron classifier side | 15:28 |
| *** dtrainor has quit IRC | 15:29 | |
| *** dtrainor has joined #openstack-meeting | 15:29 | |
| davidsha | As far as patches I have 2 up, 1 for neutron_lib and one for neutron | 15:29 |
| ralonsoh | btw, do you have some kind of fullstack or tempest test for this feature? | 15:29 |
| ralonsoh | to see how it's working | 15:29 |
| davidsha | Not added yet, I've been manually testing it. | 15:30 |
| ralonsoh | ok | 15:30 |
| davidsha | Neutron: https://review.openstack.org/#/c/636333/ Neutron-lib: https://review.openstack.org/#/c/636330/5 | 15:30 |
| davidsha | They're all on the same topic | 15:31 |
| ralonsoh | yes, I added those two patches to the etherpad | 15:31 |
| davidsha | thanks! | 15:31 |
| *** dtrainor has quit IRC | 15:31 | |
| ralonsoh | thank you! | 15:31 |
| ralonsoh | any other topic to discuss? | 15:31 |
| mlavalle | not from me | 15:32 |
| *** e0ne has quit IRC | 15:32 | |
| ralonsoh | thank you all and see you in two weeks! And remember to add your topics to the PTG agenda!! | 15:32 |
| ralonsoh | #endmeeting | 15:33 |
| davidsha | see ya! | 15:33 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:33 | |
| openstack | Meeting ended Tue Mar 26 15:33:05 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:33 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-03-26-15.00.html | 15:33 |
| *** e0ne has joined #openstack-meeting | 15:33 | |
| ralonsoh | bye | 15:33 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-03-26-15.00.txt | 15:33 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-03-26-15.00.log.html | 15:33 |
| *** wwriverrat has quit IRC | 15:33 | |
| lajoskatona | bye | 15:36 |
| *** _pewp_ has quit IRC | 15:38 | |
| *** _pewp_ has joined #openstack-meeting | 15:40 | |
| *** hemna has quit IRC | 15:40 | |
| *** hemna has joined #openstack-meeting | 15:42 | |
| *** a-pugachev has quit IRC | 15:43 | |
| *** hongbin has joined #openstack-meeting | 15:43 | |
| *** a-pugachev has joined #openstack-meeting | 15:44 | |
| *** dtrainor has joined #openstack-meeting | 15:47 | |
| *** e0ne has quit IRC | 15:50 | |
| *** cheng1 has quit IRC | 15:55 | |
| *** e0ne has joined #openstack-meeting | 15:56 | |
| *** liuyulong is now known as liuyulong_zzzZ | 15:59 | |
| slaweq | #startmeeting neutron_ci | 16:00 |
| openstack | Meeting started Tue Mar 26 16:00:35 2019 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 16:00 | |
| slaweq | hi | 16:00 |
| openstack | The meeting name has been set to 'neutron_ci' | 16:00 |
| *** e0ne has quit IRC | 16:00 | |
| mlavalle | o/ | 16:00 |
| bcafarel | o/ | 16:01 |
| haleyb | hi | 16:01 |
| njohnston | here | 16:02 |
| slaweq | ok, lets start then | 16:02 |
| slaweq | #topic Actions from previous meetings | 16:02 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 16:02 | |
| slaweq | bcafarel to switch neutron-grenade multinode jobs to bionic nodes | 16:02 |
| bcafarel | should be done (I am looking for the link) | 16:02 |
| slaweq | I think that it's done even by my patch https://review.openstack.org/#/c/639361/ | 16:03 |
| bcafarel | in fact it was in the original review to switch to bionic | 16:03 |
| bcafarel | ^ that is the one :) | 16:03 |
| slaweq | yeah, I had many such "DNM" test patches and I somehow missed that in this one someone changed description and removed "DNM" from title | 16:03 |
| slaweq | :) | 16:03 |
| bcafarel | :) | 16:04 |
| slaweq | thx bcafarel for checking that | 16:04 |
| bcafarel | just dropping the override was enough in the end | 16:04 |
| slaweq | ok, next one | 16:04 |
| slaweq | mlavalle to check https://bugs.launchpad.net/neutron/+bug/1820865 | 16:04 |
| openstack | Launchpad bug 1820865 in neutron "Fullstack tests are failing because of "OSError: [Errno 22] failed to open netns"" [Critical,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 16:04 |
| mlavalle | I did checked on it | 16:04 |
| mlavalle | created a patch | 16:04 |
| mlavalle | but seems ralonsoh proposed a better solution | 16:05 |
| mlavalle | https://review.openstack.org/#/c/647721 | 16:05 |
| slaweq | I saw that You tried to downgrade pyroute2 to check if that isn't issue in nevest version of it, right mlavalle? | 16:05 |
| mlavalle | right | 16:06 |
| slaweq | ok, I hope that ralonsoh's solution would fix this problem once and for all :) | 16:06 |
| mlavalle | I was taking network_namespace_exists as proof that the namespace existed | 16:06 |
| mlavalle | but it seems that it can exist but might not be ready to be opened | 16:06 |
| lajoskatona | Hi | 16:08 |
| slaweq | yes, I also was thinking that if namespace exists than it should be ready always :) | 16:08 |
| slaweq | hi lajoskatona | 16:08 |
| rubasov | late o/ | 16:08 |
| slaweq | hi rubasov | 16:08 |
| slaweq | ok, lets review ralonsoh's patch and lets hope it will help | 16:08 |
| *** Lucas_Gray has quit IRC | 16:08 | |
| slaweq | next action | 16:08 |
| slaweq | slaweq/mlavalle to check https://bugs.launchpad.net/neutron/+bug/1820870 | 16:08 |
| openstack | Launchpad bug 1820870 in neutron "Fullstack tests are failing because async_process is not started properly" [High,In progress] - Assigned to Slawek Kaplonski (slaweq) | 16:08 |
| slaweq | I send some dnm patch with extra logs added | 16:09 |
| slaweq | it is for sure no issue with rabbitmq | 16:09 |
| slaweq | a lot of errors related to rabbit were caused by that rabbitmq user/vhost was already deleted | 16:09 |
| slaweq | what happend there is that sometimes AsyncProcess.is_active() was returning False always | 16:10 |
| slaweq | and after 1 minute test was failing | 16:10 |
| *** njohnston_phone has quit IRC | 16:10 | |
| slaweq | so test worker was cleaning everything, removed rabbitmq user/vhost, killed spawned processes and so on | 16:10 |
| slaweq | but process which was "not active" (and caused failure) was in fact running fine | 16:11 |
| slaweq | it wasn't killed as test runner didn't know its pid | 16:11 |
| slaweq | so it was keep running and logging that it can't connect to rabbitmq | 16:11 |
| slaweq | in one case such agent's log got about 1,7 GB :) | 16:11 |
| slaweq | all my findings should be described in comments to bug in launchpad | 16:12 |
| slaweq | and I did patch https://review.openstack.org/#/c/647605/ | 16:12 |
| slaweq | please review it | 16:12 |
| * mlavalle thinks we forgot the grafana url at the beginning of the meeting | 16:12 | |
| slaweq | mlavalle: right, sorry | 16:13 |
| slaweq | #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 16:13 |
| slaweq | please open it now :) | 16:13 |
| njohnston | :-) | 16:13 |
| slaweq | and that is basically all about this action :) | 16:14 |
| slaweq | next one from last week was: | 16:14 |
| slaweq | mlavalle to debug reasons of neutron-tempest-plugin-dvr-multinode-scenario failures | 16:14 |
| mlavalle | I didn't make much progress there | 16:14 |
| slaweq | ok, that's fine :) | 16:16 |
| slaweq | we all had more important things last week | 16:16 |
| mlavalle | I think I sais I was going to do this slowly | 16:16 |
| slaweq | yes :) | 16:16 |
| mlavalle | :-) | 16:16 |
| slaweq | should I assign it to You for next week to keep it in mind? | 16:16 |
| *** simon-AS559 has left #openstack-meeting | 16:16 | |
| mlavalle | but it's a good idea to ask me every week | 16:16 |
| slaweq | ok, I will :) | 16:17 |
| mlavalle | that way the fear of facinf the Hulk will push me to make progress | 16:17 |
| slaweq | LOL | 16:17 |
| slaweq | #action mlavalle to debug reasons of neutron-tempest-plugin-dvr-multinode-scenario failures | 16:17 |
| slaweq | and that was all actions from last week | 16:17 |
| slaweq | anything You want to add/ask here? | 16:17 |
| njohnston | nope | 16:18 |
| slaweq | ok, so lets move on to the next topic | 16:19 |
| slaweq | #topic Python 3 | 16:19 |
| *** openstack changes topic to "Python 3 (Meeting topic: neutron_ci)" | 16:19 | |
| slaweq | Stadium projects etherpad: https://etherpad.openstack.org/p/neutron_stadium_python3_status | 16:19 |
| slaweq | njohnston: any updates? | 16:19 |
| njohnston | I started looking at the vpnaas jobs and tinkering with them | 16:20 |
| *** psachin has quit IRC | 16:20 | |
| njohnston | I am trying to learn more from my failures in defining zuul jobs | 16:20 |
| njohnston | But zuul is a tough teacher and slaps my hand a lot | 16:20 |
| njohnston | Progress is slow but the timeline is long so I think things are OK | 16:20 |
| slaweq | if You need any help with zuul I can try to help with it :) | 16:21 |
| njohnston | thanks! | 16:21 |
| slaweq | just ping me if You will need anything | 16:21 |
| njohnston | will do | 16:21 |
| slaweq | thx | 16:21 |
| slaweq | so next topic is | 16:22 |
| slaweq | #topic Ubuntu Bionic in CI jobs | 16:22 |
| *** openstack changes topic to "Ubuntu Bionic in CI jobs (Meeting topic: neutron_ci)" | 16:22 | |
| slaweq | as grenade jobs are switched to bionic I think we are done with all of jobs here | 16:22 |
| slaweq | most of them are already zuulv3 syntax so switched some time ago | 16:22 |
| slaweq | and legacy jobs were switched recently | 16:22 |
| bcafarel | so all done for the topic? | 16:23 |
| slaweq | if there are no objections I think we can remove this topic from agenda for next meetings | 16:23 |
| slaweq | bcafarel: yes, IMO all is done for us | 16:23 |
| njohnston | +1 for being done with it, good job! | 16:24 |
| *** diablo_rojo has quit IRC | 16:24 | |
| slaweq | also stadium projects are switched to bionic as infra switched legacy jobs to bionic too | 16:24 |
| njohnston | is midonet still the lone holdout? | 16:24 |
| mlavalle | yes, let's remove this topic from the agenda | 16:24 |
| *** gyee has joined #openstack-meeting | 16:25 | |
| slaweq | njohnston: yes, for midonet there is patch https://review.openstack.org/#/c/639990/ | 16:25 |
| slaweq | to switch them too | 16:25 |
| slaweq | but jobs are non-voting already | 16:25 |
| njohnston | ok | 16:25 |
| slaweq | and midonet team will work on that I hope | 16:25 |
| *** e0ne has joined #openstack-meeting | 16:25 | |
| slaweq | ok mlavalle, thx for confirmation :) | 16:26 |
| slaweq | #topic tempest-plugins migration | 16:26 |
| *** openstack changes topic to "tempest-plugins migration (Meeting topic: neutron_ci)" | 16:26 | |
| slaweq | any updates? | 16:26 |
| slaweq | I didn't have time to work on that | 16:26 |
| mlavalle | I will work on this this week | 16:26 |
| slaweq | tmorin didn't replied to me | 16:26 |
| njohnston | I hope to get back to the fwaas migration as soon as my urgent matters clear up | 16:26 |
| slaweq | I will also try to do this migration for bgpvpn this or next week | 16:27 |
| bcafarel | no progress here either (hope to send at least WIP patch by next meeting) | 16:27 |
| *** e0ne has quit IRC | 16:28 | |
| slaweq | #topic Grafana | 16:28 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 16:28 | |
| *** iyamahat has quit IRC | 16:28 | |
| slaweq | http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 16:28 |
| slaweq | ^^ just as a reminder :) | 16:28 |
| *** yamahata has quit IRC | 16:28 | |
| slaweq | do You see anything which You want to talk about on grafana? | 16:29 |
| slaweq | I don't see anything new and very urgent there | 16:30 |
| mlavalle | looks much better this week | 16:30 |
| slaweq | and I think that we finally more or less make gates working | 16:30 |
| njohnston | \o/ | 16:30 |
| slaweq | there are still some issues but nothing new, at least I'm not aware of anything new from this week | 16:31 |
| mlavalle | agree | 16:31 |
| slaweq | so lets talk about some specific jobs now | 16:32 |
| slaweq | #topic fullstack/functional | 16:32 |
| *** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)" | 16:32 | |
| slaweq | we have still those 2 issues with fullstack jobs: | 16:32 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1820865 and | 16:33 |
| openstack | Launchpad bug 1820865 in neutron "Fullstack tests are failing because of "OSError: [Errno 22] failed to open netns"" [Critical,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 16:33 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1820870 | 16:33 |
| openstack | Launchpad bug 1820870 in neutron "Fullstack tests are failing because async_process is not started properly" [High,In progress] - Assigned to Slawek Kaplonski (slaweq) | 16:33 |
| slaweq | but patches for both are already proposed | 16:33 |
| slaweq | when both will be merged, we can maybe next week make it voting again if it will be better on grafana | 16:33 |
| slaweq | anything else You want to talk about, regarding to functional or fullstack jobs? | 16:34 |
| mlavalle | agree | 16:34 |
| mlavalle | I was going to ask about making the job voting again | 16:34 |
| slaweq | :) | 16:35 |
| slaweq | I remember about that | 16:35 |
| slaweq | but lets fix those two issues and then check how it will be for few days at least | 16:35 |
| slaweq | what do You think about such plan? | 16:35 |
| njohnston | +1 | 16:36 |
| slaweq | ok, next topic | 16:37 |
| slaweq | #topic Tempest/Scenario | 16:37 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 16:37 | |
| slaweq | I want to raise here 2 issues | 16:37 |
| slaweq | first is https://bugs.launchpad.net/neutron/+bug/1815585 | 16:37 |
| openstack | Launchpad bug 1815585 in neutron "Floating IP status failed to transition to DOWN in neutron-tempest-plugin-scenario-linuxbridge" [High,Confirmed] | 16:37 |
| slaweq | I was debugging it a bit | 16:37 |
| slaweq | and what is strange for me is fact that port is unbound and active - and because of that active status test is failing | 16:38 |
| slaweq | do You know if that should be possible somehow to have unbound port active? | 16:38 |
| slaweq | for me this looks odd | 16:38 |
| slaweq | ahh, example of such failure is e.g. here: http://logs.openstack.org/93/631793/6/check/neutron-tempest-plugin-scenario-linuxbridge/1c3c083/testr_results.html.gz | 16:39 |
| mlavalle | yes, it looks od | 16:39 |
| mlavalle | odd | 16:39 |
| slaweq | maybe this is somehow related to https://bugs.launchpad.net/neutron/+bug/1819446 | 16:39 |
| openstack | Launchpad bug 1819446 in neutron "After the vm's port name is modified, the port status changes from down to active. " [Low,Confirmed] | 16:39 |
| slaweq | but in this bug they reported that changing name of port will switch it to active | 16:40 |
| mlavalle | even stranger | 16:40 |
| slaweq | in test there is no such port updates before this place where test is failing but maybe it can happen not only on name update - I don't know | 16:40 |
| slaweq | I have it in my backlog still but if someone would have time, please feel free to take it :) | 16:41 |
| slaweq | in the meantime I can mark this test as unstable | 16:41 |
| slaweq | what You think about it? | 16:42 |
| slaweq | according to logstash it happend 9 times in last 7 days | 16:42 |
| mlavalle | yep | 16:42 |
| slaweq | ok, thx mlavalle, I will do it today :) | 16:43 |
| slaweq | #action slaweq to mark test_floatingip_port_details test as unstable | 16:43 |
| slaweq | and second problem which we have for some time already is problem with intermittent ssh failures in various tempest tests | 16:43 |
| slaweq | example: http://logs.openstack.org/46/638646/14/check/neutron-tempest-plugin-scenario-linuxbridge/1ff70f5/testr_results.html.gz | 16:44 |
| slaweq | here it is linuxbridge job | 16:44 |
| slaweq | but it may happend in any tempest job in fact | 16:44 |
| slaweq | I didn't saw any "pattern" of those failures | 16:44 |
| mlavalle | do we have logstash query for them? | 16:45 |
| slaweq | only thing which I think is quite common is fact that those instances can connect to metadata service but ssh to FIP is not working | 16:45 |
| slaweq | mlavalle: no, I don't have logstash query for that | 16:45 |
| slaweq | and I didn't reported bug for this also | 16:45 |
| slaweq | I will report it to be able to track progress there | 16:45 |
| mlavalle | if we created a logstash query and attach that for a bug report, we might cooperate in trying to isolate the issue | 16:45 |
| slaweq | and I will also try to prepare some query | 16:46 |
| slaweq | mlavalle++ I will do | 16:46 |
| mlavalle | slaweq: I'll help | 16:46 |
| slaweq | #action slaweq to report a bug related to intermittent ssh failures in various tests | 16:46 |
| slaweq | thx mlavalle | 16:46 |
| *** iyamahat has joined #openstack-meeting | 16:46 | |
| slaweq | and that's all from my side for today | 16:47 |
| slaweq | anything else You want to talk about regarding to scenario jobs or anything else related to CI? | 16:47 |
| slaweq | #topic Open discussion | 16:47 |
| *** openstack changes topic to "Open discussion (Meeting topic: neutron_ci)" | 16:47 | |
| bcafarel | quick comment on stable branches, ocata branch gets -1 in recent backport candidates | 16:48 |
| bcafarel | https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:stable/ocata | 16:48 |
| *** eharney has quit IRC | 16:49 | |
| bcafarel | I need to check if there's an easy fix, or if it's just getting old | 16:49 |
| mlavalle | probably the latter | 16:49 |
| *** ttsiouts has quit IRC | 16:49 | |
| *** ttsiouts has joined #openstack-meeting | 16:49 | |
| bcafarel | pike branch needs some rechecks from time to time, but not as bad | 16:50 |
| slaweq | bcafarel: looking at one random test failure: http://logs.openstack.org/51/646651/1/check/openstack-tox-cover/2a21a99/job-output.txt.gz#_2019-03-25_10_50_42_620773 | 16:50 |
| slaweq | first question is: why it is tested on bionic? | 16:50 |
| slaweq | and it's not job defined in our repo I think | 16:51 |
| slaweq | so maybe we should ask e.g. gmann about it | 16:51 |
| slaweq | or someone else from qa team | 16:51 |
| bcafarel | oooh very good point, so that would mostly be a fallout from bionic switch? | 16:51 |
| slaweq | I just checked this one job for now :) | 16:52 |
| slaweq | I don't know about other failures | 16:52 |
| gmann | ocata should not do bionic. i will check the job | 16:52 |
| bcafarel | gmann++ thanks | 16:52 |
| slaweq | but yeah, looking at 3 other patches, it also failed on openstack-tox-cover job | 16:52 |
| slaweq | hi gmann | 16:53 |
| slaweq | thx for help with it :) | 16:53 |
| *** tssurya has quit IRC | 16:53 | |
| slaweq | and thx bcafarel for taking care of those stable branches :) | 16:54 |
| *** ttsiouts has quit IRC | 16:54 | |
| slaweq | ok, anything else for today? | 16:55 |
| slaweq | if not I think we can finish few minutes earlier :) | 16:55 |
| bcafarel | sounds good :) | 16:55 |
| mlavalle | o/ | 16:55 |
| slaweq | ok, thx for attending the meeting | 16:56 |
| njohnston | +1 thanks! | 16:56 |
| slaweq | #endmeeting | 16:56 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 16:56 | |
| slaweq | o/ | 16:56 |
| openstack | Meeting ended Tue Mar 26 16:56:09 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:56 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-03-26-16.00.html | 16:56 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-03-26-16.00.txt | 16:56 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-03-26-16.00.log.html | 16:56 |
| rubasov | o/ | 16:56 |
| lajoskatona | o/ | 16:56 |
| bcafarel | o/ | 16:56 |
| *** mriedem has quit IRC | 16:57 | |
| *** rubasov has left #openstack-meeting | 16:57 | |
| *** yamahata has joined #openstack-meeting | 17:05 | |
| *** janki has quit IRC | 17:09 | |
| *** igordc has joined #openstack-meeting | 17:13 | |
| *** igordc has quit IRC | 17:13 | |
| *** igordc has joined #openstack-meeting | 17:13 | |
| *** mlavalle has left #openstack-meeting | 17:18 | |
| *** kopecmartin is now known as kopecmartin|off | 17:21 | |
| *** jamesmcarthur has quit IRC | 17:25 | |
| *** zbr|ssbarnea has joined #openstack-meeting | 17:29 | |
| *** zbr has quit IRC | 17:31 | |
| *** janki has joined #openstack-meeting | 17:33 | |
| *** a-pugachev has quit IRC | 17:40 | |
| *** eharney has joined #openstack-meeting | 17:47 | |
| *** janki has quit IRC | 17:49 | |
| *** ttsiouts has joined #openstack-meeting | 17:54 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:56 | |
| *** stevebot has joined #openstack-meeting | 18:00 | |
| *** davidsha has quit IRC | 18:01 | |
| *** manjeets_ has joined #openstack-meeting | 18:01 | |
| *** manjeets_ has quit IRC | 18:02 | |
| *** dolphm has joined #openstack-meeting | 18:06 | |
| *** stevebot has left #openstack-meeting | 18:12 | |
| *** armax has quit IRC | 18:14 | |
| *** ttsiouts has quit IRC | 18:17 | |
| *** ttsiouts has joined #openstack-meeting | 18:17 | |
| *** ttsiouts has quit IRC | 18:22 | |
| *** tidwellr has left #openstack-meeting | 18:32 | |
| *** armax has joined #openstack-meeting | 18:48 | |
| *** armax has quit IRC | 18:52 | |
| * fungi peers around the room looking for infra-ites | 19:00 | |
| frickler | o/ | 19:01 |
| ianw | #startmeeting infra | 19:01 |
| openstack | Meeting started Tue Mar 26 19:01:09 2019 UTC and is due to finish in 60 minutes. The chair is ianw. Information about MeetBot at http://wiki.debian.org/MeetBot. | 19:01 |
| jroll | \o | 19:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 19:01 |
| *** openstack changes topic to " (Meeting topic: infra)" | 19:01 | |
| openstack | The meeting name has been set to 'infra' | 19:01 |
| dtroyer | o/ | 19:01 |
| ianw | #link http://lists.openstack.org/pipermail/openstack-infra/2019-March/006298.html | 19:01 |
| ianw | #topic Announcements | 19:01 |
| *** openstack changes topic to "Announcements (Meeting topic: infra)" | 19:01 | |
| ianw | clarkb remains AFK, hence me # tagging this one | 19:02 |
| cmurphy | o/ | 19:02 |
| ianw | no other announcements | 19:02 |
| ianw | #topic Actions from last meeting | 19:02 |
| *** openstack changes topic to "Actions from last meeting (Meeting topic: infra)" | 19:02 | |
| ianw | #link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-19-19.01.html | 19:03 |
| ianw | clarkb was to ask about infra ptg days being thursday friday | 19:03 |
| ianw | anyone know if that happened? | 19:03 |
| corvus | i think i remember an irc message suggesting that it did | 19:03 |
| frickler | I seem to remember that that was confirmed, yes | 19:04 |
| fungi | i believe an updated ptg schedule has been published for feedback which has our time moved | 19:04 |
| * fungi looks | 19:04 | |
| corvus | the published published one doesn't appear to have been updated: https://www.openstack.org/ptg/#tab_schedule | 19:04 |
| fungi | #link http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190321/3a3989bb/attachment-0001.png | 19:05 |
| fungi | from | 19:05 |
| fungi | #link http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004122.html Strawman Schedule | 19:05 |
| fungi | that seems to show infra/qa sharing a room thursday and friday | 19:05 |
| ianw | ok, i think let's leave an action item to just make sure everything published comes into sync? | 19:06 |
| fungi | diablo_rojo_phon probably has to let jamesmcarthur know when it's settled so the official one can be updated | 19:06 |
| ianw | #action clarkb double check infra ptg days published as thu/fri | 19:07 |
| jamesmcarthur | correct :) | 19:07 |
| ianw | ok, all action items accounted for | 19:08 |
| ianw | #topic Specs Approval | 19:08 |
| *** openstack changes topic to "Specs Approval (Meeting topic: infra)" | 19:08 | |
| ianw | I don't think there are any specs for approval at this point? | 19:08 |
| *** ttsiouts has joined #openstack-meeting | 19:08 | |
| ianw | #topic Priority Efforts | 19:09 |
| *** openstack changes topic to "Priority Efforts (Meeting topic: infra)" | 19:09 | |
| ianw | cmurphy: i saw a few more puppet4 changes going out, just a matter of review at this point? | 19:09 |
| cmurphy | ianw: pretty much | 19:10 |
| cmurphy | clarkb caught up with me already so just proposed a few more | 19:10 |
| fungi | there was some fiddling over the weekend with the pip provider to work out getting gear upgraded on logstash01 | 19:10 |
| fungi | but cmurphy got that sorted out | 19:10 |
| cmurphy | yay | 19:10 |
| ianw | oh again, more things related to the change in output of pip? | 19:11 |
| cmurphy | the same issue as before | 19:11 |
| cmurphy | we just hadn't merged that fix | 19:11 |
| cmurphy | till last weekend | 19:11 |
| ianw | oh right ... heh plenty of time for pip to have changed itself again in between :) | 19:12 |
| cmurphy | heh | 19:12 |
| ianw | #topic Opendev | 19:12 |
| *** openstack changes topic to "Opendev (Meeting topic: infra)" | 19:12 | |
| ianw | i did send out the git -> https changes as discussed in last few meetings. no real issues | 19:12 |
| ianw | a few weird things with some repos not having .gitreview, and some missing zuul branch configs, both of which i didn't code for, but otherwise ok | 19:13 |
| ianw | jroll: want to update on where the renaming thread is at? | 19:14 |
| fungi | (for openstack) | 19:14 |
| jroll | ianw: it seems we have consensus on the plan, I need to spin a new version of the governance patch | 19:14 |
| jroll | it should be straightforward to make a list of the renames from there | 19:14 |
| ianw | what's that review # for a link? | 19:15 |
| jroll | (spoiler: we're going with option #2, all unofficial projects in one namespace, all official projects in the openstack namespace) | 19:15 |
| jroll | #link https://review.openstack.org/#/c/645601/ | 19:15 |
| corvus | sounds good (though unofficial projects can still self-select their own org at https://ethercalc.openstack.org/opendev-transition ) | 19:16 |
| fungi | well, all unofficial projects who don't request their own namespace from opendev before the maintenance in one namespace anyway | 19:16 |
| fungi | er, or what corvus said | 19:16 |
| jroll | sorry, yes, that | 19:16 |
| corvus | we have some takers there now (3 people!) | 19:16 |
| jroll | we'll make an attempt to tell them about it :) | 19:16 |
| corvus | jroll: much obliged! :) | 19:16 |
| ianw | #link https://storyboard.openstack.org/#!/story/2004627 | 19:17 |
| corvus | but yeah, happy to stick anyone who doesn't respond into "unknown/" | 19:17 |
| corvus | or whatever (or whatever/) :) | 19:17 |
| ianw | so it's fungi signed up to do the modrewrite & gitreview update implementation of the renames once this is finalised? | 19:17 |
| *** ttsiouts has quit IRC | 19:18 | |
| fungi | yes, an initial stab at incorporating corvus's test redirects is already in place on files.o.o and covers git.openstack.org urls | 19:18 |
| fungi | i'm still working on the variant for the other git sites | 19:18 |
| fungi | but all the vhosts are configured there now with their ssl certs/keys installed | 19:19 |
| *** ttsiouts has joined #openstack-meeting | 19:19 | |
| ianw | cool, well that all seems like it's moving along | 19:19 |
| corvus | i'm planning on testing whether zuul itself follows those redirects for the git driver later today/tomorrow | 19:19 |
| fungi | for the git-review and zuul configuration patching, i had a quick question... | 19:19 |
| corvus | ianw: re task 29707 -- i asked fungi for a cert for docs.opendev.org earlier; he went ahead and got review.opendev.org too. if the LE stuff is ready, great, and if it's not, we can use the comodo. | 19:20 |
| fungi | particularly on the zuul configuration itself | 19:20 |
| ianw | corvus: it's very close to reviewable; i just finished yesterday all the implementation bits i can think of. will beat into shape with docs etc and request reviews in next day or so | 19:20 |
| fungi | is it sufficient to just patch the contents of {.,}zuul{,.d/*}.yaml or do we also need to try to update any references to checked-out paths in playbooks and roles? | 19:21 |
| corvus | ianw: w00t | 19:21 |
| *** enriquetaso has quit IRC | 19:21 | |
| frickler | fungi: well, there are things like this https://opendev.org/openstack/openstack-chef/src/branch/master/playbooks/integration.yaml#L6 | 19:22 |
| corvus | fungi: that's a good question ... if we don't update the playbooks/roles, i feel certain that some things will break; however it's understandable we may want to reduce the complexity there and leave that for humans... | 19:22 |
| frickler | can we make zuul create symlinks for the old path? | 19:22 |
| frickler | similar to the redirects? | 19:22 |
| ianw | fungi: you mean things like "upper_constraints: "{{ ansible_user_dir }}/{{ zuul.projects['git.openstack.org/openstack/requirements'].src_dir }}/upper-constraints.txt" | 19:23 |
| corvus | frickler: well, there's also the more 'zuulish' way of doing it which is using {{ zuul.projects['git.openstack.org/openstack/openstack-chef'].src_dir }} | 19:23 |
| fungi | ianw: frickler: yeah, those examples | 19:23 |
| *** ttsiouts has quit IRC | 19:23 | |
| frickler | corvus: will that work better when we s/git.o.o/opendev.org/ ? | 19:24 |
| corvus | frickler: no, but it won't be affected by symlinks, so jobs that use that would need a different workaround | 19:24 |
| frickler | corvus: ah, right | 19:24 |
| ianw | fungi: i feel like that will have to be updated, it will just break otherwise? if that's zuul.projects keeping an alias or a direct code change ... | 19:24 |
| fungi | so i have a feeling there are enough of these cases that my script should also recurse top-level playbooks and roles directories | 19:25 |
| fungi | i know zuul itself just expects its configs to have .yaml extensions, but ansible is likely to be more flexible in that regard. does also looking for *.yml in (any child of) those directories cover it? | 19:26 |
| corvus | fungi: that's probably not a bad idea. though playbooks can be anywhere (zuul.yaml will tell you where they are, but of course they could include_tasks in other directories) | 19:26 |
| corvus | fungi: generally yes. | 19:26 |
| fungi | oh, interesting, so the zuul configuration can set explicit paths to playbooks and roles? | 19:26 |
| corvus | fungi: playbooks, yes, roles no | 19:26 |
| corvus | roles are either at the top level of the repo or in roles/ | 19:27 |
| fungi | oh, so they can also be in the top level directory. okay | 19:27 |
| corvus | i bet if you just did roles/* and each of the directories holding a playbook, you'd get 99.99% | 19:27 |
| fungi | what's the configuration directive i should look for which sets the path to a playbook tree? | 19:28 |
| ianw | codesearch for "zuul.projects\['git.openstack.org" actually showed up way less than i thought it would be ... only around 60 matches | 19:29 |
| corvus | fungi: the values under 'pre-run', 'run', and 'post-run' are (lists of) paths to playbook files relative to repo root | 19:29 |
| fungi | thanks, that ought to be manageable | 19:29 |
| corvus | fungi: eg "pre-run: playbooks/my-playbook.yaml" is typical, but could be "pre-run: some/weird/dir/whatever.yaml" | 19:29 |
| fungi | and those are relative to the top-level directory of the repository? | 19:30 |
| corvus | yep | 19:30 |
| fungi | perfect | 19:30 |
| fungi | and they have to end in .yaml or .yml? | 19:30 |
| corvus | fungi: feel free to ping me with questions or even requests to help write code chunks | 19:30 |
| corvus | fungi: .yaml for zuul | 19:30 |
| corvus | er | 19:30 |
| corvus | 1 sec | 19:30 |
| fungi | ansible will let them be anything though, i guess | 19:31 |
| corvus | i think zuul.yaml has to be .yaml i think playbooks (even those run by zuul) can be anything | 19:31 |
| fungi | yeah, didn't want to monopolize the meeting. i've got enough for a first stab at it | 19:31 |
| fungi | thanks! | 19:31 |
| ianw | any other updates? i think dtroyer was all good with your bits last week? | 19:32 |
| dtroyer | yup | 19:32 |
| ianw | #topic Storyboard | 19:32 |
| *** openstack changes topic to "Storyboard (Meeting topic: infra)" | 19:32 | |
| ianw | any updates? | 19:33 |
| *** ttsiouts has joined #openstack-meeting | 19:33 | |
| fungi | the missing support for swift form-post signatures in openstacksdk has been proposed | 19:33 |
| fungi | #link https://review.openstack.org/639760 (openstack/openstacksdk) Add support for generating form-post signatures | 19:34 |
| fungi | this was a blocker for secure implementation of the story attachments feature | 19:34 |
| fungi | other than that, lots of pending changes to review | 19:35 |
| fungi | yesterday was the proposal deadline for outreachy candidates, so there was quite the flurry of last-minute activity | 19:36 |
| fungi | (or maybe the deadline is today) | 19:36 |
| *** electrofelix has quit IRC | 19:37 | |
| ianw | cool, do the reviews require a lot of web-architecture pre-knowledge? or is it pretty standalone? | 19:37 |
| fungi | there's a bit of a split. the storyboard repo holds the api implementation in python | 19:38 |
| fungi | the storyboard-webclient repo holds the web front-end and is mostly javascript, using an oldish angular framework | 19:38 |
| fungi | luckily we also have draft builds of the webclient rendered in ci jobs, so you can see whether or not they work and/or if they introduce unexpected behavior | 19:39 |
| fungi | i don't know whether diablo_rojo_phon or SotK are around and have anything to add | 19:40 |
| diablo_rojo_phon | Nothing from me I don't think. | 19:41 |
| diablo_rojo_phon | Oh. | 19:41 |
| diablo_rojo_phon | Outreachy application deadline was today | 19:41 |
| *** mriedem has joined #openstack-meeting | 19:41 | |
| fungi | though we also have a storyboard irc meeting most weeks this time on wednesdays, if anyone wants to continue more in-depth discussions about such things (as well as any time in #storyboard) | 19:41 |
| diablo_rojo_phon | So the flow of potential new contributors will be ebbing now | 19:41 |
| fungi | yeah, the channel was full of folks trying to figure out how to install storyboard and run local tests for the past month or more | 19:42 |
| fungi | but a big plus is that helped to fine-tune some of our developer docs and workflow | 19:42 |
| fungi | anyway, that's all i've got for now | 19:43 |
| ianw | #topic General Topics | 19:43 |
| *** openstack changes topic to "General Topics (Meeting topic: infra)" | 19:43 | |
| ianw | #topic PTG planning | 19:44 |
| *** openstack changes topic to "PTG planning (Meeting topic: infra)" | 19:44 | |
| ianw | just wanted to stamp the link in again | 19:44 |
| ianw | #link https://etherpad.openstack.org/2019-denver-ptg-infra-planning | 19:44 |
| ianw | i'm sure ideas, questions and comments are welcome | 19:45 |
| ianw | we seem to be free of other general topics, which must mean that everything is perfect and we can all just go to the beach | 19:45 |
| *** jamesmcarthur has quit IRC | 19:45 | |
| *** diablo_rojo has joined #openstack-meeting | 19:45 | |
| frickler | we had nb nodes run full again | 19:46 |
| frickler | possibly related to suse breaking their repos | 19:46 |
| ianw | #topic Open Discussion | 19:46 |
| *** openstack changes topic to "Open Discussion (Meeting topic: infra)" | 19:46 | |
| ianw | frickler: oh right, is the short version of the unbound issues that it was a super old package without the latest dnssec keys due to the mirror going away? | 19:47 |
| frickler | ianw: yes, except that it is still broken, mirror updates hopefully happening soon | 19:47 |
| frickler | also it wasn't mirror going down, but repo actively being published in an empty state iiuc | 19:48 |
| ianw | cool ... that explains why it looked so similar to the actual brokenness we had due to dnssec being stripped by opendns ... red herrings everywhere! | 19:49 |
| * fungi wants a pickled herring sandwich now | 19:49 | |
| frickler | yes, the effect was the same, unbound failing to validate any dnssec response it receives | 19:49 |
| frickler | returning servfail to the clients | 19:50 |
| ianw | i think that a cron job on the nb* to clear out +x day files on /opt/dib_tmp might help slightly | 19:50 |
| ianw | but also, i had to clear out a few builds that were on disk, but not in "nodepool image-list" | 19:50 |
| frickler | how much quota do we have for volumes? maybe adding like 1tb to each would also help | 19:50 |
| frickler | so we don't fill it up within a few days | 19:51 |
| fungi | nb01 and nb02 have a 1tb volume each | 19:51 |
| fungi | nb03 is a harder one to solve since there's no cinder there, and instead we need to ask for a flavor with a larger rootfs | 19:51 |
| fungi | though cleanup got utilization on those volumes down to something like 50% anyway | 19:52 |
| fungi | not sure how much headroom we really want to provide | 19:52 |
| frickler | I think if we had space for like 1-2 weeks of builds, that would allow us to easier react on issues | 19:53 |
| corvus | aiui Shrews is looking into nodepool leaks | 19:53 |
| ianw | frickler: i think that a background cleanup job might provide a little more breathing space, for a small error | 19:54 |
| fungi | yeah, there was something about making sure nodepool removes the on-disk files corresponding to failed builds? | 19:54 |
| fungi | rather than just leaving them lying around | 19:55 |
| ianw | ok, we're just about at time, as usual discussion can continue in #openstack-infra | 19:56 |
| frickler | does anyone have a simple recipe to monitor /opt space in cacti? otherwise I can look into that | 19:56 |
| ianw | frickler: as in something to alert us? | 19:57 |
| frickler | ianw: as in being able to track usage. currently we only have / unless I looked wrong | 19:58 |
| fungi | as in right now /opt isn't tracked in cacti for the nodepool builders | 19:58 |
| ianw | oh right, yeah that would be a start | 19:58 |
| corvus | i thought the script did that. but if it's not working, it's just some clicking | 19:58 |
| fungi | i think it will add any filesystems which are present when the server is first added | 19:59 |
| fungi | but if new filesystems are added after the config has already been generated, that doesn't get updated | 19:59 |
| corvus | i guess we stopped running the script all the time? | 19:59 |
| corvus | or, rather, we have it ignore hosts that are already there? | 20:00 |
| fungi | sounds likely, but i haven't looked deeply | 20:00 |
| ianw | what's "the script"? | 20:00 |
| *** ttsiouts has quit IRC | 20:00 | |
| corvus | maybe there's an option to run it once for that host manually | 20:00 |
| ianw | i think that tracking it explicitly would be great, and we can then correlate when it starts to climb over the long term | 20:01 |
| ianw | frickler, want an action item to look into this? | 20:01 |
| corvus | i have no idea where the script ended up :) | 20:01 |
| *** ttsiouts has joined #openstack-meeting | 20:01 | |
| frickler | ianw: yeah | 20:01 |
| corvus | https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/cacti.pp#n64 | 20:01 |
| fungi | #link https://opendev.org/openstack-infra/system-config/src/branch/master/modules/openstack_project/manifests/cacti.pp#L72-L76 Exec[cacti_import_xml] | 20:01 |
| ianw | #action frickler look into disk usage tracking, particularly of /opt on nodepool builders | 20:02 |
| corvus | it's the create_graphs.sh script | 20:02 |
| fungi | oh, nevermind, yeah it's what corvus linked | 20:02 |
| corvus | https://opendev.org/openstack-infra/system-config/src/branch/master/modules/openstack_project/files/cacti/create_graphs.sh | 20:02 |
| corvus | there it is | 20:02 |
| corvus | #link https://opendev.org/openstack-infra/system-config/src/branch/master/modules/openstack_project/files/cacti/create_graphs.sh | 20:02 |
| fungi | and the cron at line 87 in the manifest runs it | 20:03 |
| frickler | o.k., I'll take a closer look tomorrow | 20:03 |
| corvus | hrm. that sort of looks like it should be running and idempotent | 20:03 |
| corvus | so there maybe a bug there | 20:03 |
| corvus | apparently, there's a log file :) | 20:03 |
| ianw | ok, i gotta dash. i'll end meeting here and we can pick this up later | 20:04 |
| ianw | #endmeeting | 20:04 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 20:04 | |
| openstack | Meeting ended Tue Mar 26 20:04:02 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 20:04 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-26-19.01.html | 20:04 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-26-19.01.txt | 20:04 |
| ianw | thanks everyone! | 20:04 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-26-19.01.log.html | 20:04 |
| corvus | ianw: thanks! | 20:04 |
| *** ttsiouts has quit IRC | 20:05 | |
| *** enriquetaso has joined #openstack-meeting | 20:20 | |
| *** bbowen has quit IRC | 20:26 | |
| *** senrique_ has joined #openstack-meeting | 20:32 | |
| *** fatema_ has joined #openstack-meeting | 20:34 | |
| *** enriquetaso has quit IRC | 20:34 | |
| *** mriedem has quit IRC | 20:36 | |
| *** ttsiouts has joined #openstack-meeting | 20:41 | |
| *** eharney has quit IRC | 20:48 | |
| *** senrique_ has quit IRC | 20:51 | |
| *** lbragstad has quit IRC | 21:01 | |
| *** ralonsoh has quit IRC | 21:02 | |
| *** munimeha1 has quit IRC | 21:05 | |
| *** maxhub_ has quit IRC | 21:10 | |
| *** hongbin has quit IRC | 21:19 | |
| *** mdbooth_ has joined #openstack-meeting | 21:19 | |
| *** mdbooth has quit IRC | 21:23 | |
| *** bbowen has joined #openstack-meeting | 21:31 | |
| *** pcaruana has quit IRC | 21:33 | |
| *** whoami-rajat has quit IRC | 21:37 | |
| *** ttsiouts has quit IRC | 22:07 | |
| *** ivy has quit IRC | 22:11 | |
| *** ttsiouts has joined #openstack-meeting | 22:14 | |
| *** ttsiouts has quit IRC | 22:18 | |
| *** ircuser-1 has joined #openstack-meeting | 22:31 | |
| *** slaweq has quit IRC | 22:37 | |
| *** rcernin has joined #openstack-meeting | 22:38 | |
| *** gyee has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
