Tuesday, 2019-03-12

*** gyee has quit IRC00:16
*** erlon has joined #openstack-meeting00:41
*** lbragstad has quit IRC00:49
*** enriquetaso has quit IRC00:55
*** erlon has quit IRC00:57
*** whoami-rajat has joined #openstack-meeting01:12
*** erlon has joined #openstack-meeting01:48
*** hongbin has joined #openstack-meeting01:53
*** tetsuro has joined #openstack-meeting02:10
*** armax has quit IRC02:12
*** armax has joined #openstack-meeting02:19
*** openstack has joined #openstack-meeting02:23
*** ChanServ sets mode: +o openstack02:23
*** iyamahat__ has quit IRC02:31
*** yamahata has quit IRC02:31
*** lbragstad has joined #openstack-meeting02:58
*** lbragstad has quit IRC03:07
*** apetrich has quit IRC03:16
*** yamamoto has joined #openstack-meeting03:30
*** hongbin has quit IRC03:33
*** erlon has quit IRC03:35
*** munimeha1 has quit IRC03:43
*** tpatil has joined #openstack-meeting03:45
*** imsurit_ofc has joined #openstack-meeting03:46
*** liuyulong has quit IRC03:47
samPhi all for masakari04:00
samP#startmeeting masakari04:00
openstackMeeting started Tue Mar 12 04:00:15 2019 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:00
*** openstack changes topic to " (Meeting topic: masakari)"04:00
openstackThe meeting name has been set to 'masakari'04:00
samPHi all04:00
samP#topic High priority items04:00
*** openstack changes topic to "High priority items (Meeting topic: masakari)"04:00
*** shilpasd has joined #openstack-meeting04:01
samPpython-masakariclient 5.4.0 released04:01
samPThis release is for the stein. Branch will created later04:02
samPWe have about 2 weeks for other projects for their RCs04:03
samP#topic Critical bugs and patches04:04
*** openstack changes topic to "Critical bugs and patches (Meeting topic: masakari)"04:04
samPAny critical bugs or patches need to discuss here?04:04
tpatil# link : https://bugs.launchpad.net/masakari/+bug/181942204:04
openstackLaunchpad bug 1819422 in masakari "Get Notification for API version 1.1 returns KeyError: 'progress_details'" [Undecided,In progress] - Assigned to Shilpa Devharakar (shilpasd)04:04
tpatilPatch is uploaded : https://review.openstack.org/64246204:05
tpatilThere are few minor comments on this patch which will be fixed today04:05
tpatilWe should merge this patch soon04:05
tpatilAlso during regression testing, we have found one more bug which Shilpa will report in LP soon04:06
tpatilIf the recovery_flow is reserved, then it fails to execute task as it attempts to persists Host object and it fails04:06
tpatilShilpa is working on fixing this issue04:06
*** tashiromt has joined #openstack-meeting04:07
samPtpatil: got it. I will review first patch as soon as it updates04:08
samPsecond problem you mentioned above, is it happen every time where recovery_flow=reserved04:09
samPthat means, recovery_flow=reserved does not work at all, correct.04:11
samPam I correct?04:11
tpatilYes, We will fix this issue soon04:12
samPIn that case, we must fix that before release. and need to backport as well.04:12
tpatilMaybe today or tomorrow04:12
samPtpatil: Sure, thanks.04:12
tpatilWhen are you planning to cut stable/stein for masakari project?04:13
*** mordred has quit IRC04:14
samPCurrently, I am planning to do it on 3/2204:14
samPwith the RC104:15
tpatilok, we will fix this issue before that for sure04:15
samPtpatil: if time is not enough, we can fix the master and backport to Stein. Because, this it critical issue04:15
samPs/it/'is a'04:16
tpatilWe have already found out a solution to fix this issue04:16
samPtpatil: great. then no problem at all04:17
samPthank you.04:17
samPAny other critical issues?04:18
*** imsurit_ofc has quit IRC04:18
tpatilStill working on04:19
tpatilregression testing04:19
samPtpatil: no problem04:19
*** mordred has joined #openstack-meeting04:21
samPif any, please bring them up anytime in this meeting04:21
samP#topic Stein Work Items04:21
*** openstack changes topic to "Stein Work Items (Meeting topic: masakari)"04:21
samPPlease share if you have any update on this04:21
tpatil1) Add progress details for recovery workflow04:22
*** yamamoto has quit IRC04:22
tpatilDependency on openstacksdk, now 0.26.0 is released so we should merge dashboard patch soon04:22
tpatil#link : https://review.openstack.org/#/c/640755/04:22
*** yamamoto has joined #openstack-meeting04:22
tpatilI would like to thanks openstacksdk team to accept our patch at the last minute and release a newer version of openstacksdk.04:23
samPtpatil: done.04:24
tpatilsamP: Thanks04:24
tpatil2)  Functional tests04:24
samPtpatil: Thanks openstacksdk team!04:25
samPThanks openstacksdk team!04:25
tpatilI will update patch to add functional tests to check recovery workflow details04:25
tpatil#link : https://review.openstack.org/#/c/64222304:25
tpatilFrom functional tests point of view, segments, hosts and notifications functional tests (Process/VM) are covered04:27
samPtpatil: got it.04:27
tpatil3) Add devstack support for masakari-monitors04:29
tpatil#link : https://review.openstack.org/#/c/638577/04:29
tpatilGate job is failing, working on fixing this issue04:29
tpatilif we can find a fix for gate job, then we can install and deploy all monitors except host monitor04:30
samPI think same error occur in Masakari 3rd party CI and Honjo-san is re-run the CI for fix this.04:32
*** diablo_rojo has quit IRC04:32
samPI will ask Honjo-san share if he have any info about his04:33
tpatilsamP: Yes, that will help04:33
samPAs you said, if can't fix this issue before release, then it is OK to remove host monitor04:34
tpatilIn the above patch, host monitor is not included at all04:34
tpatilbut still the gate job is failing04:35
tpatilwe want to install monitors only on nodes where compute service is running04:35
tpatilwe have made required changes in the masakari plugin but still it fails to install some neutron agent04:36
tpatilon the compute node04:36
tpatillooking into the sissue04:36
samPtpatil: got it.04:37
*** mordred has quit IRC04:37
*** janki has joined #openstack-meeting04:38
tpatilI have uploaded a new PS for checking  recovery workflow details04:38
tpatilOnce the gate job passes, please review and approve this patch : https://review.openstack.org/#/c/642223/04:38
samPtpatil: sure, I will do that04:39
tpatilsamP: Thank you04:39
samPAny other updates?04:43
samPif not we can finish today's meeting04:44
tpatilsamP: No, That's all from my end04:44
*** mordred has joined #openstack-meeting04:45
samPtpatil: thank you!04:45
samPnoting from my end.04:45
samPLet's finish today's meeting04:46
samPPlease user #openstack-masakari IRC@free node or04:47
samPopenstack-discuss@lists.openstack.org for further discussions.04:47
samPThank you all!04:47
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:47
openstackMeeting ended Tue Mar 12 04:47:52 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:47
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-03-12-04.00.html04:47
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-03-12-04.00.txt04:47
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-03-12-04.00.log.html04:47
*** markvoelker has joined #openstack-meeting05:06
*** armax has quit IRC05:11
*** yamahata has joined #openstack-meeting05:12
*** sridharg has joined #openstack-meeting05:22
*** larainema has joined #openstack-meeting05:26
*** cheng1 has joined #openstack-meeting05:48
*** cheng1 has quit IRC05:59
*** cheng1 has joined #openstack-meeting05:59
*** slaweq_ has joined #openstack-meeting06:00
*** rf0lc0 has joined #openstack-meeting06:00
*** snierodz_ has joined #openstack-meeting06:04
*** joxyuki has joined #openstack-meeting06:05
*** jbadiapa has quit IRC06:05
*** slaweq has quit IRC06:07
*** rfolco|ruck|off has quit IRC06:07
*** mahatic has quit IRC06:07
*** snierodz has quit IRC06:07
*** gary_perkins has quit IRC06:08
*** andreaf has quit IRC06:09
*** flaper87 has quit IRC06:10
*** gary_perkins has joined #openstack-meeting06:10
*** andreaf has joined #openstack-meeting06:12
*** flaper87 has joined #openstack-meeting06:13
*** e0ne has joined #openstack-meeting06:18
*** tpatil has quit IRC06:43
*** cheng1 has quit IRC06:51
*** cheng1 has joined #openstack-meeting06:52
*** Luzi has joined #openstack-meeting06:53
*** apetrich has joined #openstack-meeting07:13
*** dkushwaha has joined #openstack-meeting07:37
*** pcaruana has joined #openstack-meeting07:39
*** pcaruana has quit IRC07:43
*** pcaruana has joined #openstack-meeting07:43
*** kopecmartin|off is now known as kopecmartin07:53
*** jbadiapa has joined #openstack-meeting07:54
*** iyamahat has joined #openstack-meeting07:55
dkushwaha#startmeeting tacker08:03
openstackMeeting started Tue Mar 12 08:03:32 2019 UTC and is due to finish in 60 minutes.  The chair is dkushwaha. Information about MeetBot at http://wiki.debian.org/MeetBot.08:03
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:03
*** openstack changes topic to " (Meeting topic: tacker)"08:03
openstackThe meeting name has been set to 'tacker'08:03
dkushwaha#topic Roll Call08:03
*** openstack changes topic to "Roll Call (Meeting topic: tacker)"08:03
dkushwahawho is here for Tacker weekly meeting?08:03
hyunsikyangi am late08:06
dkushwahahello hyunsikyang08:06
hyunsikyangthis week little bit busy08:07
dkushwahanp :)08:07
*** hadai has joined #openstack-meeting08:07
*** tssurya has joined #openstack-meeting08:08
dkushwahalets wait if some more members join, otherwise we can skip.08:08
dkushwahaseems no more members available today08:12
dkushwahahyunsikyang, do you have something to discuss?08:12
hyunsikyangActually, I submitted one blueprint and I am preparing one more08:13
hyunsikyangfor tacker integration with Fenix project08:13
hyunsikyangIf we don't have any special issues, I want to discuss what we will do in tacker and summit presentation.08:14
dkushwahahyunsikyang, go ahead08:15
hyunsikyangAt first, Please review, https://review.openstack.org/#/c/636272/08:15
hyunsikyangthis is a spec for network function extention for C-VNF08:15
hyunsikyangNow, Tacker uses a kubernetes as a VIM. But, It only support basic things for C-VNF.08:16
*** e0ne has quit IRC08:17
hyunsikyangI think we should provide more functionality to improve the utilization of C-VNF.08:17
hyunsikyangthat spec is a first step of function extention for C-VNF.08:19
*** jiaopengju has joined #openstack-meeting08:19
hyunsikyangBut, I want to hear your opinion for next step of tacker, first.08:19
dkushwahahyunsikyang, is this above spec have any relation with Fenix integration08:20
hyunsikyangno, that is different one.08:20
dkushwahaIMO both are independent08:20
*** jaewook_oh has joined #openstack-meeting08:20
hyunsikyangI will upload it as soon as possible. I has been discussing with fenix PTL.08:21
dkushwahahyunsikyang, cool08:22
*** hjwon has joined #openstack-meeting08:23
*** hokeeeeun has joined #openstack-meeting08:23
*** e0ne has joined #openstack-meeting08:23
dkushwahahyunsikyang, Regarding https://review.openstack.org/#/c/636272/ . Sorry I could not looked on this, Please give me sometimes(couple of days), i will review and will provide my feedbacks08:23
hyunsikyangthanks :)08:23
hyunsikyangwe can discuss more detail thing in the summit too.08:23
dkushwahahyunsikyang, yup08:24
dkushwahahyunsikyang, will you have time in this week for some other works?08:24
hyunsikyangI am trying to finish the fenix blueprint and review in this week.08:25
dkushwahaI was looking volunteer for "Migrating legacy jobs to Bionic"08:26
dkushwaha http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003614.html08:26
hyunsikyangwait, I will check it.08:28
dkushwahahyunsikyang, ok08:28
*** iyamahat has quit IRC08:29
hyunsikyangI am reading it now. but, I need a time to check it. I am not familiar with this.08:34
dkushwahahyunsikyang, ok, so please check it, if it looks feasible to you otherwise i will work on that.08:36
*** ralonsoh has joined #openstack-meeting08:36
dkushwahahyunsikyang, lets close this meeting. We can continue on Tacker channel.08:36
hyunsikyangok thanks08:36
dkushwahathanks hyunsik for joining this meeting.08:37
dkushwahaClosing now08:37
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:37
openstackMeeting ended Tue Mar 12 08:37:28 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:37
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-03-12-08.03.html08:37
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-03-12-08.03.txt08:37
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-03-12-08.03.log.html08:37
*** e0ne has quit IRC08:39
*** hjwon has quit IRC08:44
*** ttsiouts has joined #openstack-meeting08:49
*** a-pugachev has joined #openstack-meeting08:53
*** rubasov has joined #openstack-meeting08:58
*** tetsuro has quit IRC08:59
*** bertys has joined #openstack-meeting08:59
*** bertys has left #openstack-meeting09:01
jiaopengju#startmeeting karbor09:01
openstackMeeting started Tue Mar 12 09:01:46 2019 UTC and is due to finish in 60 minutes.  The chair is jiaopengju. Information about MeetBot at http://wiki.debian.org/MeetBot.09:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.09:01
*** openstack changes topic to " (Meeting topic: karbor)"09:01
openstackThe meeting name has been set to 'karbor'09:01
jiaopengjuping liushuai09:02
*** cheng1 has quit IRC09:02
jiaopengjulet’s wait a few minutes09:02
*** cheng1 has joined #openstack-meeting09:02
*** priteau has joined #openstack-meeting09:02
*** ttsiouts has quit IRC09:04
*** ttsiouts has joined #openstack-meeting09:05
*** liushuai has joined #openstack-meeting09:06
jiaopengjuhi liushuai09:07
*** dmacpher has joined #openstack-meeting09:07
liushuaihi jiaopengju09:07
*** tetsuro has joined #openstack-meeting09:08
jiaopengjulet’s start the meeting09:08
jiaopengju#topic stein release09:08
*** openstack changes topic to "stein release (Meeting topic: karbor)"09:08
*** hadai has quit IRC09:09
jiaopengjuas you now, the stein release will be finished soon, we have freezed the feature submmit,09:09
*** tetsuro has quit IRC09:10
jiaopengjubug fixes will keep open until stein tag is added09:10
jiaopengjuplease continue to review patches in the reviewboard09:11
jiaopengjuthis patch has no unit test, if it’s necessary, we can add unit test on it directly09:12
jiaopengjuthis patch need another +2 https://review.openstack.org/#/c/633159/09:13
jiaopengjufeel free to merge it :)09:13
jiaopengjulet’s go on09:14
liushuaii will review soon09:14
jiaopengju#topic new features in train cycle09:14
*** openstack changes topic to "new features in train cycle (Meeting topic: karbor)"09:14
jiaopengjuI have some new features, for example, database bank driver, rbd bank, etc. If you have some good ideas, please told me, we can discuss it in next meeting09:16
jiaopengjuI will create a ethpad link soon, so you can write it there09:17
jiaopengjuthat’s all the topics for today’s meeting, do you have any thing to talk about?09:17
jiaopengjuok, I will end the meeting soon09:18
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"09:18
openstackMeeting ended Tue Mar 12 09:18:52 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)09:18
openstackMinutes:        http://eavesdrop.openstack.org/meetings/karbor/2019/karbor.2019-03-12-09.01.html09:18
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/karbor/2019/karbor.2019-03-12-09.01.txt09:18
openstackLog:            http://eavesdrop.openstack.org/meetings/karbor/2019/karbor.2019-03-12-09.01.log.html09:18
*** liushuai has quit IRC09:19
*** joxyuki has quit IRC09:20
*** rf0lc0 is now known as rfolco|ruck09:34
*** tashiromt has quit IRC09:36
*** cheng1 has quit IRC09:38
*** dkushwaha has quit IRC09:39
*** hyunsikyang__ has joined #openstack-meeting09:44
*** hyunsikyang__ has quit IRC09:46
*** hyunsikyang has quit IRC09:48
*** cheng1 has joined #openstack-meeting09:50
*** vishalmanchanda has joined #openstack-meeting10:06
*** rubasov has quit IRC10:07
*** e0ne has joined #openstack-meeting10:16
*** cheng1 has quit IRC10:20
*** cheng1 has joined #openstack-meeting10:21
*** lpetrut has joined #openstack-meeting10:33
*** hokeen has joined #openstack-meeting10:37
*** hokeeeeun has quit IRC10:40
*** hokeen has quit IRC10:48
*** electrofelix has joined #openstack-meeting10:49
*** njohnston has joined #openstack-meeting11:05
*** carlos_silva has joined #openstack-meeting11:11
*** liuyulong has joined #openstack-meeting11:17
*** imsurit_ofc has joined #openstack-meeting11:25
*** jaewook_oh has quit IRC11:35
*** ttsiouts has quit IRC11:45
*** ttsiouts has joined #openstack-meeting11:45
*** imsurit_ofc has quit IRC11:46
*** ttsiouts has quit IRC11:50
*** cheng1 has quit IRC11:56
*** rubasov has joined #openstack-meeting11:57
*** hyunsikyang has joined #openstack-meeting12:02
*** ttsiouts has joined #openstack-meeting12:09
*** markvoelker has quit IRC12:18
*** raildo has joined #openstack-meeting12:29
*** liuyulong_ has joined #openstack-meeting12:31
*** e0ne has quit IRC12:36
*** rbudden has joined #openstack-meeting12:39
*** mjturek has joined #openstack-meeting12:40
*** mjturek has quit IRC12:45
*** yamamoto has quit IRC12:47
*** yamamoto has joined #openstack-meeting12:48
*** mriedem has joined #openstack-meeting12:53
*** yamamoto has quit IRC12:53
*** rbudden has quit IRC12:58
*** janki has quit IRC12:59
*** mlavalle has joined #openstack-meeting12:59
*** janki has joined #openstack-meeting12:59
mlavalle#startmeeting networking13:00
openstackMeeting started Tue Mar 12 13:00:20 2019 UTC and is due to finish in 60 minutes.  The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
*** rbudden has joined #openstack-meeting13:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
*** openstack changes topic to " (Meeting topic: networking)"13:00
openstackThe meeting name has been set to 'networking'13:00
mlavalleHi there!13:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"13:02
openstackMeeting ended Tue Mar 12 13:02:30 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:02
openstackMinutes:        http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-03-12-13.00.html13:02
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-03-12-13.00.txt13:02
openstackLog:            http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-03-12-13.00.log.html13:02
*** enriquetaso has joined #openstack-meeting13:02
*** slaweq_ is now known as slaweq13:08
*** yamamoto has joined #openstack-meeting13:12
*** lbragstad has joined #openstack-meeting13:17
*** vishalmanchanda has quit IRC13:25
*** irclogbot_1 has quit IRC13:26
*** e0ne has joined #openstack-meeting13:28
*** jamesmcarthur has joined #openstack-meeting13:29
*** irclogbot_1 has joined #openstack-meeting13:30
*** munimeha1 has joined #openstack-meeting13:46
*** enriquetaso has quit IRC13:48
*** jamesmcarthur has quit IRC13:48
*** jamesmcarthur has joined #openstack-meeting13:48
*** davidsha has joined #openstack-meeting13:50
*** mlavalle has quit IRC13:52
*** jbadiapa has quit IRC13:55
*** mlavalle has joined #openstack-meeting13:55
*** awaugama has joined #openstack-meeting13:56
*** njohnston is now known as njohnston_13:57
*** njohnsto_ has joined #openstack-meeting13:58
*** njohnsto_ is now known as njohnston13:58
*** openstack has joined #openstack-meeting15:38
*** ChanServ sets mode: +o openstack15:38
ralonsoh(I think davidsha is experiencing network problems)15:38
davidshaNvm we have things modeled a little differently in neutron classifier and I was expecting a BaseQoSRule table or some something similar for all the BW limit, DSCP and BW assurance rules to inherit from.15:38
njohnstonnetsplit just unsplit15:38
ralonsohaaah ok15:38
ralonsohnjohnston, it's the wind here in Ireland now!! it's affecting internet15:39
ralonsohok, thansk davidsha15:39
ralonsohany other topic?15:39
ralonsohthank you all and "see" you in two weeks15:40
*** jbadiapa has quit IRC15:41
*** e0ne has quit IRC15:45
*** cyberpear has joined #openstack-meeting15:45
*** enriquetaso has quit IRC15:46
*** jchhatbar has quit IRC15:47
*** enriquetaso has joined #openstack-meeting15:47
*** ttsiouts has joined #openstack-meeting15:48
cyberpearThe topic link doesn't work w/ the trailing slash.  Stripping the slash made it work for me.15:48
*** vishakha has joined #openstack-meeting15:49
*** anteaya has joined #openstack-meeting15:56
*** e0ne has joined #openstack-meeting15:58
*** njohnston_ has joined #openstack-meeting16:01
slaweq#startmeeting neutron_ci16:01
openstackMeeting started Tue Mar 12 16:01:41 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:01
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:01
openstackThe meeting name has been set to 'neutron_ci'16:01
*** njohnston is now known as njohnston216:01
*** njohnston_ is now known as njohnston16:02
slaweqhi njohnston16:02
slaweqlets wait a bit more for others like haleyb, hongbin, bcafarel16:02
slaweqI know that mlavalle will not be here today16:02
bcafarelbot is back in order?16:03
bcafarel(also o/ )16:03
slaweqand also I need to finish this meeting in about 45 minutes16:03
slaweqbcafarel: seems so16:03
slaweqok, lets go then16:03
slaweqfirst of all16:03
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:04
slaweqplease open it now :)16:04
slaweqit will be ready for later :)16:04
*** jrbalderrama has quit IRC16:04
*** diablo_rojo has joined #openstack-meeting16:04
slaweq#topic Actions from previous meetings16:04
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:04
slaweqlets go then16:04
slaweqfirst action16:04
slaweq* njohnston Debug fullstack DSCP issue16:05
*** enriquetaso has quit IRC16:05
slaweqI know that njohnston didn't look into it16:05
slaweqbecause I did :)16:05
slaweqand I found an issue16:05
slaweqfix is here: https://review.openstack.org/#/c/642186/16:05
*** a-pugachev has quit IRC16:06
slaweqbasically it looks that sometimes tcpdump was starting too slow, we send one icmp packet which wasn't captured by tcpdump and test failed16:06
*** a-pugachev has joined #openstack-meeting16:06
slaweqso I proposed to send at least 10 packets always16:06
ralonsohgood catch!16:06
slaweqthen some should be always captured by tcpdump16:06
slaweqthx ralonsoh :)16:07
slaweqok, next one16:07
slaweqmlavalle to take a look at fullstack dhcp rescheduling issue https://bugs.launchpad.net/neutron/+bug/179955516:07
openstackLaunchpad bug 1799555 in neutron "Fullstack test neutron.tests.fullstack.test_dhcp_agent.TestDhcpAgentHA.test_reschedule_network_on_new_agent timeout" [High,Confirmed]16:07
slaweqI don't think that mlavalle was looking into this one16:07
ralonsohslaweq, I can try it16:08
slaweqthx ralonsoh :)16:08
slaweq#action ralonsoh to take a look at fullstack dhcp rescheduling issue https://bugs.launchpad.net/neutron/+bug/179955516:08
openstackLaunchpad bug 1799555 in neutron "Fullstack test neutron.tests.fullstack.test_dhcp_agent.TestDhcpAgentHA.test_reschedule_network_on_new_agent timeout" [High,Confirmed]16:08
slaweqand the last one from last week was:16:08
slaweqslaweq to talk with tmorin about networking-bagpipe16:08
slaweqand I totally forgot about this16:09
slaweqsorry for that16:09
slaweqI will add it to myself for this week16:09
slaweq#action slaweq to talk with tmorin about networking-bagpipe16:09
slaweqanything else You want to add/ask?16:10
slaweqok, lets move on then16:11
slaweq#topic Python 316:11
*** openstack changes topic to "Python 3 (Meeting topic: neutron_ci)"16:11
slaweqnjohnston: any updates about stadium projects?16:11
slaweqetherpad is here: https://etherpad.openstack.org/p/neutron_stadium_python3_status16:11
njohnston2Nothing yet, but I will note that as of now we have satisfied the basic requirements for the community goal16:12
njohnston2The community goal doesn't specify all the kinds of testing we're handling at this point, it establishes a baseline16:12
slaweqgreat then16:13
njohnston2We've met that baseline across the stadium AFAICT16:13
slaweqthat is very good news njohnston2 :)16:13
njohnston2I'll continue and I think we should keep this topic, but this is more forward-looking, getting us ready for the python2 removal in the U cycle16:13
slaweqbut I will keep this topic in agenda for now, to track our (slow) progress on that in next cycle too16:13
slaweqthx njohnston216:14
*** ttsiouts has quit IRC16:14
slaweqok, lets move on then16:14
slaweqnext topic16:14
slaweq#topic Ubuntu Bionic in CI jobs16:14
*** openstack changes topic to "Ubuntu Bionic in CI jobs (Meeting topic: neutron_ci)"16:14
*** ttsiouts has joined #openstack-meeting16:14
slaweqWe are almost good with it16:16
slaweqwe have for now one issue with fullstack tests in neutron16:16
slaweqbecause on Bionic node we shouldn't compile ovs kernel module anymore, it's not necessary16:16
slaweqI will try to push patch for that tomorrow morning, unless there is someone else who want's to take care of it today16:17
slaweqI tried today with https://review.openstack.org/#/c/642461/ and it looks good16:17
slaweqbut it should be done in the way which will work on both, xenial and bionic16:18
njohnstonI’ll take a look16:18
slaweqfor networking-dynamic-routing we should be good to go with patch https://review.openstack.org/#/c/642433/216:18
slaweqnjohnston: thx :)16:18
slaweqif some of You have +2 power in networking-dynamic-routing, please check this patch :)16:19
slaweqit isn't complicated :)16:19
slaweqand works fine for Bionic too: https://review.openstack.org/#/c/639675/216:19
slaweqthx njohnston2 :)16:19
slaweqand third problem (but not very urgent) is with networking-bagpipe: https://review.openstack.org/#/c/642456/1 fullstack job is running but there is some fail still: http://logs.openstack.org/87/639987/3/check/legacy-networking-bagpipe-dsvm-fullstack/8d5af6c/job-output.txt.gz16:20
slaweqhere we should have for sure same patch as for neutron to not compile ovs kernel module on Bionic16:20
slaweqbut even with this tests are failing16:21
njohnston2"ImportError: cannot import name async_process" perhaps a dependency changed names?16:21
slaweqI'm not familiar with bagpipe so I'm not sure if that is somehow related to switch to Bionic16:21
slaweqnjohnston2: maybe16:21
slaweqbut problem isn't very urgent as it is non-voting job there :)16:21
slaweqand last one with some problems is networking-midonet, but here yamamoto is aware of those issues so I hope they will fix them16:22
slaweqok, any questions/something to add regarding bionic?16:23
njohnstonlooks good, great work16:23
slaweqthx njohnston :)16:24
*** njohnston is now known as njohnston_16:24
*** njohnston2 is now known as njohnston16:24
slaweqok, so moving to next topic16:24
slaweq#topic tempest-plugins migration16:24
*** openstack changes topic to "tempest-plugins migration (Meeting topic: neutron_ci)"16:24
slaweqnjohnston_: how many of You are in the meeting? :D16:24
*** njohnston_ is now known as njohnston216:24
slaweqEtherpad: https://etherpad.openstack.org/p/neutron_stadium_move_to_tempest_plugin_repo16:24
slaweqany updates here?16:24
slaweqI don't have anything16:24
njohnstonHey, I've been saying that cloning myself was the only way to get ahead of these trello cards...16:25
njohnstonnothing from me16:25
slaweqnjohnston: LOL16:25
slaweqgreat idea16:25
slaweqbut I don't know if my wife will handle that when I will clone myself :P16:25
* haleyb wanders in late16:25
njohnstonyou're not allowed to clone yourself, because an army of Hulks would be unstoppable16:25
haleybany bugs for me? :)16:25
njohnstonhaleyb: all of them16:25
slaweqhaleyb: hi, sure, if You want some :P16:26
slaweqok, lets move on16:26
slaweq#topic Grafana16:26
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:26
slaweqreminder link: http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:26
haleybthat reminds me to update the ovn dashboard16:27
njohnstonis it weird to anyone else how for the check queue the number of runs is on a steady downward slope16:27
*** gyee has joined #openstack-meeting16:27
njohnstonexample "Number of Unit Tests jobs runs (Check queue)" has a high around 78 on 3/6 steadily down to 45 for 3/1116:28
*** yamahata has quit IRC16:28
slaweqnjohnston: yes, looks strange but lets see how it will be in next days maybe16:29
slaweqIMHO last week we had some "spike" there16:29
*** iyamahat has quit IRC16:29
slaweqnjohnston: if You will get data from 30 days, it doesn't look odd16:30
njohnstonoh good16:30
njohnstonI set it to the 30 day view, that should render sometime on Thursday16:30
*** dmacpher has quit IRC16:31
njohnstonactually that was very fast.  That really helps show what is normal skitter and what isn't.16:31
slaweqfrom other things it generally looks much better than in last week16:32
slaweqI see that unit tests are rising, and that isn't good IMO16:32
bcafarela few fixes got in yeah16:32
slaweqand in last couple of days I saw issue like http://logs.openstack.org/79/633979/26/check/openstack-tox-py37/e7878ff/testr_results.html.gz in various unit tests16:32
slaweqdid You saw it too?16:32
bcafareltest_port_ip_update_revises? I don't think so16:33
haleybi saw this update_revises somewhere16:33
slaweqlooking at logstash: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%20%5C%22line%20170%2C%20in%20test_port_ip_update_revises%5C%2216:33
slaweqit happend at least couple of times in last week16:34
njohnstonI managed to miss that one16:34
slaweqanyone wants to look at this?16:34
slaweqralonsoh: thx a lot16:35
slaweqplease report a bug for it so we can track it there16:35
slaweq#action ralonsoh to take a look at update_revises unit test failures16:35
slaweqok, lets move on then16:36
slaweqor You want to talk about something else related to grafana?16:36
njohnstonWhile we have grafana up... What do people think about marking the neutron-tempest-iptables_hybrid-fedora job voting?  It looks very stable as far as error rate, and will be helpful as the next version of RHEL comes down the pike.16:36
slaweqnjohnston: that is very good idea IMO16:37
njohnstonOK, I'll push a change for it16:38
slaweqit's under 25% in last 30 days at least16:38
slaweqthere wasn't any real problem with it before IIRC16:38
njohnstonit only deviates from neutron-tempest-iptables_hybrid by a percent or two16:38
slaweqso I think it's very good idea to make it voting (and maybe gating in few weeks too)16:39
haleybis the fedora one on that page?16:39
njohnstonwhich seems to be within the margin of error16:39
njohnstonhaleyb: Yes, look in " Integrated Tempest Failure Rates (Check queue)"16:39
njohnstonwhich has a lot of lines but that is one of them16:39
haleybah, a search didn't show it initially16:39
njohnstonlatest datapoint: neutron-tempest-iptables_hybrid 8% fails, neutron-tempest-iptables_hybrid-fedora 6% fails16:40
*** tssurya has quit IRC16:40
slaweqyes, there was one spike up to 100% about 2 weeks ago but it was the same for neutron-tempest-iptables_hybrid job and was related to os-vif issue IIRC16:40
slaweqother than that it's good and I think it can be voting IMO16:41
slaweqhaleyb: any thoughts?16:41
haleyb+1 for voting16:41
slaweqok, njohnston please propose patch for that16:41
njohnstonslaweq: https://review.openstack.org/64281816:42
slaweqthat was fast16:42
haleybas long as we remember we're at the end of stein :)16:42
slaweqyes, we can maybe wait with this patch couple of weeks to release stein and then merge this one16:42
slaweqbut IMO it will be good to have patch there and ask others what they think about it :)16:43
njohnston2I can -W until the branch16:43
bcafarelsounds good16:43
slaweqok, lets move on as I will need to go soon16:43
slaweqnext topic16:44
slaweq#topic functional/fullstack16:44
*** openstack changes topic to "functional/fullstack (Meeting topic: neutron_ci)"16:44
slaweqwe merged some fixes for recent failures and we are in better shape now16:44
slaweqI wanted to share with You one bug16:44
openstackLaunchpad bug 1818614 in neutron "Various L3HA functional tests fails often" [Critical,In progress] - Assigned to Slawek Kaplonski (slaweq)16:44
slaweqI was investigating it during last week16:44
slaweqand I found 2 different issues there16:44
slaweqone with neutron-keepalived-state-change for which patch is https://review.openstack.org/#/c/642295/16:45
slaweqbut I will need to check it once again16:45
slaweqso it's still -W16:45
slaweqbut I also opened second bug https://bugs.launchpad.net/neutron/+bug/181916016:45
openstackLaunchpad bug 1819160 in neutron "Functional tests for dvr ha routers are broken" [High,In progress] - Assigned to Slawek Kaplonski (slaweq)16:45
slaweqand I would like to ask haleyb and other L3 experts to take a look on it16:46
*** iyamahat has joined #openstack-meeting16:46
slaweqbasically it looks for me that in test_dvr_router module in functional tests we are not testing dvr ha routers16:46
slaweqor differently16:46
slaweqwe should test one dvr ha router spawned on 2 agents16:46
slaweqbut we are testing 2 independent dvr routers spawned on 2 different agents16:47
slaweqand both of them can be set to master in the same time16:47
slaweqIMO that is wrong and should be fixed but please check it if You will have time :)16:47
haleybwill look16:48
*** igordc has quit IRC16:48
slaweqok, anything else You want to discuss today, because I need to leave now :)16:48
njohnstonthat's it for me16:49
*** iyamahat_ has joined #openstack-meeting16:49
slaweqok, thx for attending and sorry for a nit shorter meeting today :)16:49
bcafarelnothing worth keeping you around!16:49
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:49
openstackMeeting ended Tue Mar 12 16:49:25 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:49
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-03-12-16.01.html16:49
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-03-12-16.01.txt16:49
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-03-12-16.01.log.html16:49
*** igordc has joined #openstack-meeting16:52
*** iyamahat has quit IRC16:52
*** igordc has quit IRC16:53
*** e0ne has quit IRC16:55
*** igordc has joined #openstack-meeting16:56
*** enriquetaso has joined #openstack-meeting16:58
*** ttsiouts has quit IRC17:01
*** ttsiouts has joined #openstack-meeting17:01
*** yamahata has joined #openstack-meeting17:05
*** enriquetaso has quit IRC17:05
*** enriquetaso has joined #openstack-meeting17:05
*** kopecmartin is now known as kopecmartin|off17:05
*** ttsiouts has quit IRC17:06
*** armax has quit IRC17:08
*** priteau has quit IRC17:08
*** igordc has quit IRC17:10
*** a-pugachev has quit IRC17:13
*** igordc has joined #openstack-meeting17:21
*** njohnston2 has quit IRC17:40
*** mattw4 has joined #openstack-meeting17:41
*** davidsha has quit IRC17:42
*** igordc has quit IRC17:43
*** igordc has joined #openstack-meeting17:48
*** Shrews has joined #openstack-meeting17:51
*** armax has joined #openstack-meeting17:55
*** zhubx has quit IRC17:55
*** zhubx has joined #openstack-meeting17:55
*** jbadiapa has joined #openstack-meeting17:58
*** iyamahat__ has joined #openstack-meeting17:58
*** iyamahat_ has quit IRC18:01
*** e0ne has joined #openstack-meeting18:02
*** e0ne has quit IRC18:06
*** erlon has quit IRC18:15
*** e0ne has joined #openstack-meeting18:18
*** sridharg has quit IRC18:18
*** electrofelix has quit IRC18:19
*** jamesmcarthur has quit IRC18:20
*** jamesmcarthur has joined #openstack-meeting18:21
*** e0ne_ has joined #openstack-meeting18:21
*** e0ne has quit IRC18:22
*** gmann is now known as gmann_afk18:26
*** jamesmcarthur has quit IRC18:27
*** electrofelix has joined #openstack-meeting18:32
*** jamesmcarthur has joined #openstack-meeting18:33
*** pcaruana has quit IRC18:39
*** e0ne_ has quit IRC18:41
*** jamesmcarthur has quit IRC18:51
clarkbAnyone else around for the infra meeting?19:00
clarkb#startmeeting infra19:01
openstackMeeting started Tue Mar 12 19:01:26 2019 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2019-March/006293.html19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
clarkbPTL Candidate nominations close today (March 12) at 23:45UTC.19:01
*** roman_g has joined #openstack-meeting19:01
clarkbI have submitted a self nomination to be infra PTL. Its not too late for someone else to run if they would like to :)19:02
clarkbThese nominations are for the Train cycle19:02
anteayais infra still governed by the tc?19:02
clarkbanteaya: it is19:02
anteayaand will it continue to be in future?19:02
anteayaand thanks19:02
funginot all of what the openstack infrastructure team does is moving into opendev19:03
*** jamesmcarthur has joined #openstack-meeting19:03
clarkbthat and I don't expect us to start really thinking about breaking pieces out until we've got a bit more opendev work done (probably during the next cycle)19:03
anteayaokay thanks19:03
fungiat some point it might be worthwhile to talk about converting the team to a sig, but i wouldn't begin to consider that until after opendev is nearing some steady state19:03
clarkbOn the agenda email I had noted you could submit forum sessions but I'm told that closed late yesterday?19:04
anteayaand thanks for standing as ptl again clarkb19:04
clarkbso that announcement item is no longer valid re forum ssessions19:04
mordredyes - forum session selection time has started19:04
clarkb#topic Actions from last meeting19:05
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:05
fungithough if there's something we really need to squeeze in, we might be able to talk the selection committee into it19:05
clarkbfungi: good to know thanks19:05
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-05-19.01.txt minutes from last meeting19:05
clarkbianw had an action to rotate our backup server volumes19:05
clarkbianw: has that happened?19:05
ianwno, on my todo list, sorry was out a couple of days last week19:05
clarkb#action ianw rotate backup server disks19:05
clarkb#topic Specs approval19:06
*** openstack changes topic to "Specs approval (Meeting topic: infra)"19:06
clarkbThere are no specs up for approval. However I wanted to quickly follow up on the letsencrypt spec19:07
clarkbianw: re ^ are you happy with the dns subdomain for acme validation? and do we think that needs a spec update?19:07
ianwclarkb: i think we can handle it in reviews at this point19:07
clarkbcool thanks19:07
clarkb#topic Priority Efforts19:08
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:08
clarkb#topic Update Config Management19:08
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:08
clarkbOn the puppet 4 side of things I've done some cleanup of the upgrade process so that we don't install puppet4 then downgrade to puppet 3 then back to puppet 4 again. This cleaned up puppet logs on review-dev19:09
clarkbOtherwise I've mostly used the dev servers as a set to clean up some of the things that puppet 4 points out. thank you for the reviews on that19:09
clarkbOn the docker side of things corvus is ready to document the speculative image building system and mark it as in production19:10
clarkbThis allows us to test changes to docker deployed systems before we merge their image build updates19:10
clarkbshould end up being super useful particularly as we end up deploying more complex systems with docker19:10
clarkbcorvus: ^ other than the password change happenign today and the docs updates that need to happen is there anything to keep in mind or look at for that now?19:11
corvusclarkb: nope -- just note that the jobs in system-config are the vanguard19:11
corvusso look to them for reference for how to do things19:11
corvusalso zuul-preview19:11
corvusi'll bring zuul and nodepool forward to use those jobs soon19:12
clarkbI'll be looking to merge the run puppet 4 in more places change(s) later this week. I'll probably ping people for reviews once I am happy that all the minor issues we've foudn on the dev servers are ignorable or not worth fixing19:13
clarkb#topic OpenDev19:13
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:13
*** yamamoto has quit IRC19:13
corvusi sent out the email announcement19:14
corvus#link opendev gerrit repo migration announcement http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html19:14
corvusand got some replies19:14
corvus#info jroll and dtroyer are POCs for openstack and starlingx respectively19:14
corvusi didn't see a reply from airship19:14
roman_gAirship is here as requested in an e-mail19:14
corvusroman_g: yay!19:14
roman_gHello all. =)19:15
corvus#info jroll, dtroyer, and roman_g are POCs for openstack, starlingx, and airship respectively19:15
fungium, i think you're forgetting one19:15
fungistarts with a z?19:15
corvusyeah... no one replied to that either.  i'm going to assume everyone assumes i'll handle it...19:15
corvusunless anyone wants to volunteer real quick? :)19:15
corvus#info jroll, dtroyer, roman_g, and corvus are POCs for openstack, starlingx, airship, and zuul respectively19:16
corvusokay, we have a full complement :)19:16
clarkbthank you to all our volunteers :)19:16
anteayaare edge or kata affected/19:16
fungii can volunteer to represent zuul in opendev discussions if you really want, but i have a feeling your bandwidth with yourself might be higher19:17
clarkbanteaya: kata doesn't really host with us currently :/ so not really19:17
anteayaah okay thanks19:17
clarkbbut I expect ttx would be happy to help there should we need a volunteer19:17
corvusthere's two main things we need to coordinate on -- any repository renames, and scheduling19:17
fungithe one repo kata does have in our gerrit is unlikely to be heavily impacted by this19:17
fungiand teh edge effort doesn't have any software19:18
fungithey are/were primarily a whitepaper writing effort19:18
clarkbcorvus: for scheduling I expect on our side of things we'll want to do things no sooner than the summit19:18
corvusregarding renames, i think it would be best if each of the POCs come back with a list of repos that should be renamed when we perform the migration19:18
clarkbcorvus: ++19:18
clarkbcorvus: along with the new names?19:19
corvusroman_g, dtroyer, corvus: for your groups, i'm kind of imagining you'd take the opportunity to just forklift all your repos to your own namespace.  eg openstack-infra/zuul will move to zuul/zuul.19:19
roman_g#action each of the POCs come back with a list of repos that should be renamed when we (#openstack-infra) perform the migration19:20
corvusi mean, other options exist, but that's sort of what i'm assuming things would look like19:20
dtroyercorvus: that is our plan, yes19:20
fungiopenstack will, predictably, probably me much more messy19:20
clarkbdtroyer: will you drop the stx- prefix?19:20
mordredthe larger/complicated list is potentially openstack - due to maybe moving stackforge thigns out19:20
mordredfungi: jinx19:20
dtroyerclarkb: still an open question, I imagine we will.19:20
clarkbdtroyer: those details can be sorted out with the listings, but that is stuff to think about probably19:20
corvusjroll: your job is harder :)  since openstack/ is hosting a bunch of unofficial projects, you (as in the TC) get to decide what you want to do about that.19:20
* mordred believes in jroll19:21
fungithere was also interest in openstack of not using just one namespace19:21
corvusjroll: you could leave them in openstack/ for now, we could move them all... or.. whatever :)19:21
fungie.g., neutron/neutron-lib and nova/python-novaclient and so on19:21
mordredyup. it's all good by us!19:22
clarkbThere were rumblings of bringing stackforge back19:22
corvusi don't think we need a dedicated org for unofficial projects19:22
fungiheh, yeah, as the catch-all namespace for opendev hosting of projects who don't request any particular namespace19:22
fungiwould we just duplicate their repository name as their namespace prefix too?19:23
mordredwe certainly _could_19:23
fungii gather gitea requires some namespace prefix19:23
corvusi did take the liberty of starting one action item for this, which is to set up a place where folks with unofficial projects can self-select a new name.  so regardless of whether the tc decides to let folks stay in the openstack namespace, projects can self-select to move along with the migration.19:23
corvus#link unofficial project move self-selection: https://ethercalc.openstack.org/opendev-transition19:24
corvuswe don't have many takers there yet.  :)19:24
corvusbut you can see my suggested answer to the namespace question :)19:24
fungihard to pass up that clickbait19:24
corvusanyway, we'll just need that list from each of the 4 osf projects before we execute the migration.  any questions on this point?19:26
clarkbOn the opendev/infra side of things do we think that automated rename tooling that doesn't require a gerrit outage is required before we make the transition (that will likely inform the timing of this whole process)19:26
corvusclarkb: i don't think so19:26
fungieven if there were automated rename tooling we wouldn't want to use it for this batch19:26
corvusi think our rename policy should stay the same for now -- we do it periodically as necessary and get really annoyed about it.  :(19:26
*** vishakha has quit IRC19:26
clarkbya I think I'm mostly fine with that as it is no longer cool to rename a project every other week19:27
mordredyeah. although I do think it would be great if the gerrit rename plugin became magically usable for online renames19:27
fungifor reasons such as task 29708 (which i haven't done more than ponder so far, sorry)19:27
fungiwell, i take that back, i did a little bit of prototyping19:27
anteayawhen was the last time a rename was done?19:27
corvusthat is something for jroll/openstack to consider in developing a policy for the openstack/ namespace.  if the idea is to revert to "official only" then keep in mind we're still not any better at renames as the last time.  however, we do have fewer projects incubating.19:28
clarkbanteaya: we did some renames a few months ago.19:28
clarkbI wnt to say november ish19:28
anteayahow long did the reindex take?19:28
clarkbanteaya: we are able to do it online now19:28
anteayaoh lovely19:28
roman_gШэь еруку фдкуфвн19:28
fungireindexes have been less of a concern now that they're no longer offline19:28
fungithat came with gerrit 2.11 i think19:28
mordredroman_g: I'm going to assume that translates to "mordred is super awesome" :)19:29
fungi(for us anyway)19:29
roman_gmordred: yes, sure =)19:29
anteayacool, thanks19:29
corvusthe other point we need to coordinate on is scheduling19:29
*** ttsiouts has joined #openstack-meeting19:29
corvusbroadly speaking, we have a couple weeks, then things get hairy for the openstack release19:30
corvuslet's assume we still need those couple weeks to finish up our pre-tasks, which means we're probably looking at after the stein release, week of apr 1219:30
fungiit sounded like openstack was wanting to see it happen after release activity for stein concludes19:30
anteayais there pressure to complete this prior to a certain time?19:30
clarkbcorvus: ya then we run into summit/ptg prep19:31
fungianteaya: the sooner we get it over with the sooner we can move on to other parts of the opendev transition19:31
anteayaah okay19:31
clarkbcorvus: I was thinking the ptg might be a good time for a sanity check of planning then we can do the transition after that?19:31
corvusas the zuul representative, i would like it to happen asap :)19:31
fungithis bit is a blocker for many other activities19:31
clarkbcorvus: also things tend to be quiet in the week or two after a big event like that which likely makes it a good time for our users19:31
dtroyerStarlingX is on a release schedule of OpenStack+N weeks, where N is currently between 4 and 619:31
corvusclarkb: hrm.  i wouldn't mind it happening before the summit19:31
dtroyerI think we would prefer either before summit or a month or so later19:32
corvusdtroyer: summit is ~3 weeks after release, so ... yeah that19:32
clarkbcorvus: my worry is that between summit prep, trusty upgrades, and gitea prep we don't have a lot of time between now and then19:32
*** igordc has quit IRC19:32
*** igordc has joined #openstack-meeting19:33
anteayawill you rename all projects at once?19:33
corvusi am performing zero summit prep, so april 15-26 are excellent for me19:33
fungianteaya: that's the current plan, yes19:33
*** iyamahat_ has joined #openstack-meeting19:33
anteayaif yes, then I think jroll's thoughts on how long it will take to sort out openstack names will factor19:34
corvuswe're changing a hostname, so there isn't really much of a choice19:34
mordredI think before the summit is potentially nice for being able to talk about things. maybe provisionally plan for april 15 week - but also plan for a "crap we're not ready" contingency?19:34
clarkbfwiw I have no objection to doing it pre summit if we get things lined up by then19:34
*** iyamahat__ has quit IRC19:34
corvusanteaya: if it takes longer than we have before the migration to sort that out, then i think it makes sense to perform further renames after the migration to opendev.19:35
anteayathat would make sense19:35
corvusi look at it this way -- we *must* rename all projects at least once.  we can *optionally* take the opportunity to avoid a second rename for anyone who is ready.19:35
clarkbmordred: ya I can see that. basically tell everyon we are targetting that week and then have an out if we need to19:35
fungiand basically results in at least two renames causing significant impact to each project involved19:35
clarkbThat gives us ~ 5 weeks19:35
corvusyes, from a technical point of view, i think we should be able to make that19:36
corvusgiven the remaining tasks on:19:36
corvus#link opendev gerrit story https://storyboard.openstack.org/#!/story/200462719:36
*** larainema has quit IRC19:36
clarkbis default branch setting still broken for unknown reasons?19:36
anteayawell two scheduled renames might be better than one big rename that can't be scheduled19:36
mordredclarkb: we have workaround in place19:36
corvusclarkb: i think it's fixed19:36
mordredwe don't know the root cause in upstream gitea and I donm't believe that's fixed - but I believe it's fixed in our automation19:37
fungianteaya: it's one big rename followed by maybe an additional rename for the same projects (first rename is the change of domani)19:37
fungii was typing latin there for a moment19:37
clarkbmordred: thanks19:37
anteayafungi /nod19:37
corvusdo we want to do it during the week of 15-19, or do we want to, erm, perform the move on easter weekend? :)19:37
anteayadon't bring god into it19:37
clarkbthen ya I agree I think the major known problems are addressed and the other technical work is coordination and planning19:37
mordredcorvus: I do not personally have any conflicts on the weekend known by some as easter19:38
corvusi'm happy to volunteer to work on the weekend if it helps reduce disruption.19:38
clarkbI've probably got egg hunting acitvities with the kids on sunday but otherwise am free19:38
corvusif it would be better to do it during the week because more folks would be available (either infra folks to move, or project folks to deal with unanticipated fallout), that wfm too19:38
mordredsame. I'm fine either way - happy to work on that weekend, or to not work on that weekend, whichever provides the most happiness for the largest number of humans19:39
clarkbFriday is a holiday for many, maybe do friday?19:39
clarkbthat then gives us the weekend to sort stuff out if necessary19:40
corvusyeah, 4/19 wfm19:40
corvusdtroyer: that still close enough to the release to make it okay-ish for starlingx?19:40
mordredyeah. also - it'll probably be slower then because of pre-summit anyway19:40
corvusand post release tension release :)19:41
dtroyercorvus: I think so, I'll ask tomorrow in our community meeting.19:41
dtroyerOur RC1 is scheduled for 4/29 right now19:41
corvusroman_g: any concerns?19:41
roman_gcorvus: no concerns. thanks for your work.19:41
corvus#info tentative date for opendev migration: april 1919:42
roman_gcorvus - are you POC for this change?19:42
corvuslet's wait to hear back from dtroyer and jroll, and confirm that next week, but start planning for that.19:42
corvusroman_g: yep19:42
roman_ggreat, thanks19:43
corvusroman_g, jroll, dtroyer: i don't think we need to ask you to attend every infra meeting, but you're welcome to.  this is a standing item on the agenda until its done, and we'll talk about it a little every week.19:43
corvusif we do need to get everyone back for another real-time discussion, i'll let you know19:44
corvusand that's EOT from me19:44
dtroyercorvus: ack19:44
clarkbfwiw I've grabbed the set launchpad when not using storyboard task. I don't know what fixup current errors means but we can followup on that outside of the meeting19:45
fungisorry, getting pulled in several conversational directions and then my workstation decided to spontaneously reboot, but ~easter week should work for me. there may be a day when i have company and possibly a wedding anniversary, but i can work around those19:45
corvusclarkb: means "everyone is currently set to use storyboard, so update the existing other projects to use lp"19:45
mordredclarkb: I believe that means "do a $something to change the links for existing projects in the gitea db"19:45
clarkbgot it19:45
corvusclarkb: and thanks!19:46
anteayafungi happy anniversary19:46
clarkbthank you to our volunteers19:46
clarkband now lets continue on as we only have ~14 minutes left19:46
clarkb#topic General Topics19:46
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:46
clarkbI wanted to do a quick update on the trusty server upgrade progress.19:47
clarkbI managed to get all three of the afs fileservers upgraded yesterday doing an inplace upgrade19:47
clarkbThere was a small hiccup where afs clients didn't seem to notice that they could use a different server to pull data from when the one they were talking to went down19:47
clarkbotherwise I think it went really well19:48
clarkbI've since starting sketching a plan for the afsdb server upgrades. https://etherpad.openstack.org/p/afs-dbserver-trusty-to-xenial19:48
fungiand the replacement wiki-dev is in the inventory now. i was going to check on it after the meeting, but instead i'll be figuring out why my workstation rebooted19:48
*** ralonsoh has quit IRC19:48
clarkbOne question I had for the group re ^ is do we feel comfortable using the upgrade of the fileservers as the template for upgrading the db servers? or do you think I should make a db server snapshot, boot that, then run through a do-release-upgrade?19:49
clarkbfrom what I can tell there is only a one package difference ebtween the two servers. openafs-dbserver is installed on the dbservers and not on the fileservers19:49
clarkbthen the differences are in openafs configs19:49
*** yamamoto has joined #openstack-meeting19:49
clarkbgiven that I don't really expect any trouble from the ubuntu upgrade process, but maybe those that know more about openafs think we should test the upgrade first?19:50
corvusi think the utility of a snapshot would be limited.  if things go very very badly, i'm not at all sure restoring from snapshot would be easier than starting from scratch.19:50
corvusand i don't think this is possible to test except by creating a new cell19:50
zbrre bindep question from yesterday, maybe you can cast some votes on https://ask.openstack.org/en/question/120501/what-atom-should-bindep-expose-about-official-distro-python-version/ ;)19:51
clarkbgiven ^ and the reliability of do-release-upgrade on 3 very similar hosts so far I think I'm comfortable just doing do-release-upgrade and answering the questions as I did for the fileservers19:51
corvusclarkb: ++19:51
clarkbI'll copy the process over from the fileserver plan and we can look at it in the new context19:51
clarkbotherwise thank you all for the help on getting that done as well as for the other server upgrades19:52
clarkbone thing to note is that when we do an in place upgrade from trusty to xenial we have to clean up apt things for our puppet install19:52
clarkbthis process is documentedin my etherpads above, but whoever does lists.o.o will likely need that too19:52
clarkb#topic Open Discussion19:53
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:53
ianwi guess i must have missed the 24 hour cut off for the agenda, but i did want to discuss git: -> https: as part of opendev19:53
clarkbianw: go for it19:53
ianwi have a bunch of links on the agenda wiki19:54
ianwin short, before i send out 400+ changes to move git:// -> https:// i'd like to discuss it :)19:54
corvuscan you link that?19:54
*** yamamoto has quit IRC19:54
ianwdo we have a service user to submit these from?19:54
ianwor, should we do that19:55
clarkbianw: we have the openstack gerrit bot it does the requirements update pushes iirc19:55
fungii think the bigger problem is not who submits the changes but getting them merged19:55
clarkbianw: dhellmann has recruited our contributors that like to push a lot of changes like this to do pushes like this in the past19:55
clarkbianw: dhellmann et al may have tooling that is useful19:56
corvusi think that's the "openstack release bot" ?19:56
ianwclarkb: well the scripts are ready and aimed ... it's just a matter of pushing enter :)19:56
clarkbianw: got it19:56
mordredyeah. I'd say dhellmann is the most recent "submit changes to every repo" champion19:56
fungifor the git-review and zuul configuration in repositories we're planning to just bypass ci entirely and (at least it's my current plan) merge them directly on disk while gerrit is offline19:56
clarkbianw: I think we should communicate it to the discuss list first and explain why we want this so we don't get blanket -1s19:56
corvusfungi: ++19:57
dhellmannoh, god, please do not submit changes to every repo19:57
mordredbut I don't think there's any specific reason to do it as not-ianw19:57
ianwdhellmann: well, i'm not sure what else to do?  some are +1/-1 diffs, others are not19:57
clarkbya I agree with mordred I think if ianw did the work to write the commits (even if automated) it is perfectly accurate for ianw to be author and committer19:57
roman_gianw: for how many says would git:// still work after a set of changes is pushed?19:57
fungiwe've also talked to the openstack tc about the possibility to say an infra representative is clear to just use gerrit admin privs to approve changes like that which block significant infrastructure activities19:57
clarkbroman_g: it will work until our transition date (that we just pencilled in for april 19)19:58
roman_ggood to know19:58
dhellmannfungi : yes, do that19:58
dhellmannsubmitting and managing hundreds of patches is just going to burn you out19:58
corvusi don't think that most *infrastructure* things will be broken if git/https changes don't merge, as devstack in the gate bypasses this.  but end-user use of things may break19:58
ianw#link https://review.openstack.org/#/q/topic:opendev-gerrit+owner:iwienand19:59
clarkbdhellmann: fungi is the concern use of gerrit and reviews or the waiting on people to review the cahnges?19:59
ianwis a sample19:59
fungiso at least we're covered where it comes to openstack official repositories if we want to give the tc a heads up that we're going to "just merge it"19:59
clarkbMy preferences is that we still go through gerrit reviews but then we can certainly approve via our infra backdoor19:59
fungiclarkb: my main concern is waiting for the cat herding which comes with expecting different review teams to approve the various changes19:59
corvusi believe that is why we are reluctant to invoke our "let us just merge this" authority on this (unlike .gitreview and .zuul.yaml, where we clearly know it will break infrastructure and we know exactly what we need to change)19:59
dhellmannclarkb : you will have to explain individually to hundreds of reviewers who don't read the mailing list why the patch is needed. You will have a bunch of folks show up out of nowhere and start submitting similar patches, frequently incorrectly. It's just a huge pain.19:59
anteayaI wouldn't have expected copycat patches20:00
clarkb(as a note we are offocially at time, but don't want to end this discussion until ianw has what he needs)20:00
corvusi'm open to the suggestion that this, while perhaps not technically an infrastructure blocker, is just too durned annoying to handle another way.20:00
anteayaguess its a thing20:00
mordredI'm in favor of infra-approving the patches in this case -- also, we should maybe do this over a weekend bease it's going to slam nodepool hard20:00
fungiianw: any feel for how many of those patches are actual functional changes vs how many are just readmes and other documentation?20:00
dhellmannanteaya : yep20:00
anteayadhellmann: wow20:01
corvuswe could consider rolling it up into the rename changes20:01
mordredcorvus: we could do that for sure20:01
dhellmanncorvus : I like that. Make it a "fix git things after migration" patch20:01
corvusmerge .gitreview, zuul.yaml, and git/https all at once during the outage.20:01
fungioh, also we're over time (though i don't know if anyone else is waiting to start a meeting in here)20:01
ianwfungi: this initial list is for anything with a devstack plugin.  i feel like they're all very minimally "functional" as in part of code that will execute, but very low risk20:01
clarkbfungi: there is no meeting that follows us20:01
mordredcorvus: although if we do the git:// removal first, we coudl actually roll out the git.openstack.org redirect before doing the larger cutover, since the redirects will keep stuff working20:02
dhellmannanteaya : look through some of the abandoned python3-first patches from this past cycle if you want examples20:02
mordredcorvus: I'm not sure we want to do that - but it would be open to us once git:// is gone20:02
cmurphyclarkb: do we care enough about the cgit servers to bother upgrading their puppet?20:02
anteayadhellmann: happy to take your word for it, and I thank you for the gerrit topic pointer20:02
ianwpersonally i feel like doing this first, then taking on a smaller mop-up of any remaining issues would be easier20:02
mordredcmurphy: I do not think so, no20:02
clarkbcmurphy: at this point given the 4/19 date probably not20:02
fungiyeah, the fact that the git://->https:// changes *should* be able to happen at any time between now and the maintenance makes it less critical that we roll it into the git-review and zuul configuration patch scripting, though it's certainly worth debating20:03
cmurphythose were the only centos ones then i think so that makes it easy20:03
corvusmordred: that's true... though it may make the migration more difficult... worth thinking about.20:03
mordredcorvus: ++20:03
clarkbcorvus: mordred ianw if ianw pushes teh change then self approves them that reduces the things we have to do20:03
clarkbya what fungi said20:03
mordredbiggest issue with self-approve will be tracking down/rechking phantom failures20:04
clarkbmordred: unless we also submit20:04
corvusif we did it along with the renames, we could do "git://git.openstack.org" -> "https://opendev.org" in one pass.20:05
mordredclarkb: if we also submit, will anything cancel the now-not-needed zuul jobs in flight?20:05
corvuslet me clarify that20:05
dhellmannas far as tooling goes, be careful linking that many patches to 1 story in storyboard (the UI can't cope and starts timing out), and the stuff I build for python3-first was all pretty custom for that work but it's in the goal-tools git repo if you want to look at it20:05
*** electrofelix has quit IRC20:05
corvusbecause we could technically do that now, but we could do "git://git.openstack.org/openstack/unofficial-project" -> "https://opendev.org/foobar/unofficial-project" during the migration20:06
mordredcorvus: that is true and a good point20:06
clarkbianw: is it relatively easy to generate the todo list?20:06
corvusdhellmann: oh good point, thanks.  if we do push patches, we could send a ML post and link the commit message to that instead <-- ianw20:06
clarkbwe'll need to generate that list just before the migration if we do it all in one go then20:07
corvusclarkb: i bet if we re-do the list a few days before the migration, that'll be enough20:07
ianwclarkb: it's all scripts, i can certainly beat them into a more generic shape, currently i hand run it20:07
clarkbianw: as long as it is repeatable in advance should be fine20:08
corvusclarkb: we can freeze new repo creation a couple days before migration too, so we don't have any surprise new devstack plugin repos show up20:08
clarkbgiven corvus' point I think there is value in doing it in the migration20:08
clarkbso if want to do it then wfm20:08
fungiconcern with doing more than protocol changes is we have some fairly complex rules for mapping urls from cgit and git http backend, which is happening on the old vhosts under the present plan20:08
fungiso if we do that in the patch we need to take care to perform the complete set of transformations because we can't rely on gitea for all of them once the requests no longer hit the redirect site for git.openstack.org20:09
clarkbfungi: I think most of that complexity is to capture the difference in cgit and gitea parameters20:09
mordredfungi: yeah. I don't thin we should try to auto-change any cgit links20:09
dhellmanncorvus : +1 for a mailing list post instead20:09
ianwi think the protocol changes are pretty straight forward.  i think if we URL renames, that should probably at least be in a separate step20:09
corvusfungi: correct -- but we can and should go directly to the end state.20:09
fungiclarkb: and also the cgit path prefix20:09
clarkbfungi: oh I thought we were handling that with project renames in gitea20:10
clarkb(corvus' got that patched merged upstream to make that work)20:10
fungioh, opendev.org/cgit/... works?20:10
fungii didn't even think to try that20:10
clarkbfungi: oh that prefix20:10
mordredyeah - cgit rewrite rules will make the old cgit urls work - and renames in gitea will make renames work20:10
clarkbsorry I thought you mean openstack/zuul -> zuul/zuul20:10
corvuswere there ever any "git://git.openstack.org/cgit" links?20:10
clarkbcorvus: no20:11
fungigreat point20:11
*** electrofelix has joined #openstack-meeting20:11
fungiwhich brings up the other question... is there a huge benefit to rewriting the domain part of git:// urls in repository content when we're not doing that for other protocols?20:11
fungifeels like it just increases the set of things we might possibly break in a scripted transition20:12
corvuswell, anything that uses git:// is going to break with the transition20:12
*** gibi has quit IRC20:12
funginot what i was asking, but i agree that's true20:13
mordrednot a technical one - the system should handle all the https:// urls already. BUT - if we're going to make a mass change like that - it might be nicer to do the mass change to a correct final state20:13
corvusthat's probably the best argument for doing something ahead of time.  it lets the change propagate to downstream users before we turn it off.20:13
mordredso the smallest possible working change is git:// to https:// and changing nothing else20:13
corvusfungi: oh, sorry, i grok your question20:13
*** igordc has quit IRC20:13
clarkbAs a time check we are almost 15 minutes over now and I can smell my lunch :) this may warrant a mailing list thread?20:14
clarkbsounds like there are enough moving parts with enough options that writing them down so that we can talk about them distinctly may help?20:14
ianwclarkb: sure, i think i've got enough to get that together, thanks20:14
*** jamesmcarthur has quit IRC20:14
mordredhttps://review.openstack.org/#/c/642652/ is an example change from ianw's script btw20:14
ianwtldr; i will *not* press enter and submit the changes right now :)20:15
clarkbgreat, I think it will help me to see them listed as individual items20:15
clarkbthank you everyone20:15
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"20:15
*** jamesmcarthur has joined #openstack-meeting20:15
openstackMeeting ended Tue Mar 12 20:15:17 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:15
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-12-19.01.html20:15
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-12-19.01.txt20:15
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-12-19.01.log.html20:15
*** electrofelix has quit IRC20:15
anteayathanks clarkb20:16
*** Shrews has left #openstack-meeting20:17
fungithanks clarkb!20:18
*** jamesmcarthur has quit IRC20:19
roman_gthank you all20:19
*** e0ne has joined #openstack-meeting20:20
*** senrique_ has joined #openstack-meeting20:32
*** enriquetaso has quit IRC20:34
*** hongbin has joined #openstack-meeting20:52
*** gmann_afk is now known as gmann20:54
*** mjturek has quit IRC20:57
*** Haunted330 has joined #openstack-meeting20:58
*** Haunted330 has quit IRC20:58
*** zhubx has quit IRC20:59
*** zhubx has joined #openstack-meeting20:59
*** eharney has quit IRC21:03
*** mriedem has quit IRC21:05
*** mriedem has joined #openstack-meeting21:08
*** raildo has quit IRC21:10
*** gibi has joined #openstack-meeting21:18
*** iyamahat__ has joined #openstack-meeting21:23
*** e0ne has quit IRC21:26
*** igordc has joined #openstack-meeting21:26
*** igordc has quit IRC21:26
*** iyamahat_ has quit IRC21:26
*** igordc has joined #openstack-meeting21:27
*** mattw4 has quit IRC21:30
*** mattw4 has joined #openstack-meeting21:30
*** senrique__ has joined #openstack-meeting21:32
*** senrique_ has quit IRC21:34
*** ab-a has quit IRC21:37
*** ab-a has joined #openstack-meeting21:38
*** senrique__ has quit IRC21:47
*** jamesmcarthur has joined #openstack-meeting21:50
*** awaugama has quit IRC21:51
*** mattw4 has quit IRC21:52
*** iyamahat_ has joined #openstack-meeting21:58
*** iyamahat__ has quit IRC22:00
*** rcernin has joined #openstack-meeting22:01
*** carlos_silva has quit IRC22:09
*** whoami-rajat has quit IRC22:12
*** munimeha1 has quit IRC22:22
*** jamesmcarthur has quit IRC22:22
*** jamesmcarthur has joined #openstack-meeting22:23
*** jamesmcarthur has quit IRC22:25
*** igordc has quit IRC22:25
*** jamesmcarthur has joined #openstack-meeting22:25
*** mattw4 has joined #openstack-meeting22:26
*** jamesmcarthur has quit IRC22:27
*** hongbin has quit IRC22:32
*** enriquetaso has joined #openstack-meeting22:38
*** jamesmcarthur has joined #openstack-meeting22:43
*** igordc has joined #openstack-meeting22:43
*** ttsiouts has quit IRC22:45
*** ttsiouts has joined #openstack-meeting22:46
*** ianychoi_ is now known as ianychoi22:47
*** ttsiouts has quit IRC22:50
*** mriedem has quit IRC22:51
*** jamesmcarthur has quit IRC22:55
*** mattw4 has quit IRC23:06
*** enriquetaso has quit IRC23:07
*** mattw4 has joined #openstack-meeting23:10
*** slaweq has quit IRC23:24
*** armax has quit IRC23:47
*** jamesmcarthur has joined #openstack-meeting23:53
*** yamamoto has joined #openstack-meeting23:54
*** Qiming has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!