Tuesday, 2019-07-16

*** artom has joined #openstack-meeting00:08
*** enriquetaso has quit IRC00:15
*** brinzhang_ has joined #openstack-meeting00:34
*** ijw has quit IRC00:36
*** brinzhang has quit IRC00:37
*** ekcs has quit IRC00:40
*** TxGirlGeek has joined #openstack-meeting00:48
*** Liang__ has joined #openstack-meeting00:58
*** igordc has quit IRC01:01
*** bbowen has quit IRC01:02
*** bbowen has joined #openstack-meeting01:02
*** yamamoto has joined #openstack-meeting01:03
*** yamamoto has quit IRC01:06
*** yamamoto has joined #openstack-meeting01:09
*** gyee has quit IRC01:14
*** yamamoto has quit IRC01:14
*** jaewook_oh_ has joined #openstack-meeting01:37
*** baojg has joined #openstack-meeting01:42
*** yamamoto has joined #openstack-meeting01:48
*** mhen has quit IRC01:48
*** yamamoto has quit IRC01:50
*** yamamoto has joined #openstack-meeting01:50
*** armax has quit IRC01:53
*** ykatabam has quit IRC01:54
*** apetrich has quit IRC01:57
*** yamamoto has quit IRC02:01
*** iyamahat has quit IRC02:09
*** yamahata has quit IRC02:09
*** yamamoto has joined #openstack-meeting02:10
*** yamamoto has quit IRC02:11
*** yamamoto has joined #openstack-meeting02:16
*** yamamoto has quit IRC02:20
*** hemna has quit IRC02:30
*** whoami-rajat has joined #openstack-meeting02:42
*** hyunsikyang has joined #openstack-meeting02:44
*** hyunsikyang__ has quit IRC02:48
*** ykatabam has joined #openstack-meeting02:51
*** hyunsikyang__ has joined #openstack-meeting02:52
*** hongbin has joined #openstack-meeting03:00
*** njohnston has quit IRC03:16
*** yamamoto has joined #openstack-meeting03:20
*** psachin has joined #openstack-meeting03:26
*** ykatabam has quit IRC03:30
*** armax has joined #openstack-meeting03:37
*** hongbin has quit IRC03:42
*** tpatil has joined #openstack-meeting03:47
*** TxGirlGeek has quit IRC03:59
*** imsurit has joined #openstack-meeting03:59
samP#startmeeting masakari04:00
openstackMeeting started Tue Jul 16 04:00:45 2019 UTC and is due to finish in 60 minutes.  The chair is samP. Information about MeetBot at http://wiki.debian.org/MeetBot.04:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.04:00
*** openstack changes topic to " (Meeting topic: masakari)"04:00
openstackThe meeting name has been set to 'masakari'04:00
samPHi all for masakari04:00
samP#topic Critical bugs and patches04:01
*** openstack changes topic to "Critical bugs and patches (Meeting topic: masakari)"04:01
samPAny critical bugs or patches?04:01
samPwe have one new spec,04:04
samP#link https://review.opendev.org/#/c/670476/04:04
samPI will review this. Please review this if you have spare time.04:04
*** nitinuikey has joined #openstack-meeting04:05
tpatilYes, I will review this spec in this week04:06
samPtpatil: thanks04:06
samP#topic Train Work Items04:06
*** openstack changes topic to "Train Work Items (Meeting topic: masakari)"04:06
samPPlease the updates of Train work items04:07
tpatil1) devstack support to install host-monitors04:07
tpatildone with the changes. It will install hostmonitor only where IPMI is enabled04:07
tpatilThe patch will be pushed sometime in this week04:08
*** yamahata has joined #openstack-meeting04:08
tpatilIn case of masakari-monitor, devstack script doesn't create a  failover segment against which notifications can be sent04:09
tpatilI think in masakari-monitor, there is a logic which will add host to the segment if notification fails04:09
tpatillet me check if that check is present04:10
*** armax has quit IRC04:10
tpatilI remember this kind of check discussions long back. I don't that check is present in the master code04:11
tpatilEven though devstack-monitor supports to install various monitors, it won't work automatically until a failover segment is created and hosts are added to it.04:12
samPsorry, question is not clear.04:12
tpatilit's not possible to automatically discover on which compute hosts masakari-monitors are installed04:12
samPQuestion is, are hosts added to failover segment automatically?04:12
samPtpatil: it is not.04:13
tpatilEven though we are adding support to install various masakari-monitors  in devstack, it won't work until a failover segment is created04:13
samPtpatil: yes04:13
tpatilthe creation of failover segment is outside the scope of devstack plugin04:13
tpatilbut I think it should be part of it04:14
samPtpatil: It should be one of the initialization setup for devstack04:14
tpatilbut there is no way to create add hosts to the failover segment automatically even if we decide to create a failover segment where masakari-api is installed04:14
samPLike in registering the glance images or create user04:15
tpatilcreation of failover segment is simple04:15
tpatilthe main question is how to add hosts to the failover segments04:15
samPah.. got it.04:17
samPif it is not in the CI, user should do it manually04:17
tpatilI will add a separate patch to create a failover segment in devstack script04:18
tpatiland after installation of masakari-monitors is complete on all nodes, user will need to add hosts to the segment manually04:19
samPtpatil: thanks.04:19
tpatil2) Masakari-monitors documentation04:20
tpatilwork is in progress04:20
samPthere was an proposal for automatically register nodes at the end of the masakari-monitor install process.04:20
samPlet me find it.04:20
tpatilFYI, there is one mailing thread where one of the user is facing problem to install masakari-monitors using kolla ansible plugin for masakari04:21
tpatilhe is facing this issue on CentOS04:21
samP#link https://review.opendev.org/#/q/topic:bp/auto-compute-node-registration+(status:open+OR+status:merged)04:21
tpatilYes, it requires failover segment uuid04:22
samPtpatil: I couldn't follow the discussion. I will check the thread04:22
tpatil# link : http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007565.html04:22
tpatilRight now, we are making changes to the patch : https://review.opendev.org/#/c/489095/, to add steps to install masakar-monitors on Ubuntu04:24
tpatilonce that is done, we will add the steps to install it on other OSes04:24
tpatilPlan is to push the patch sometime in the next week for Ubuntu04:25
tpatilThat's all update about Train work items from my side04:26
samPtpatil: Thanks04:26
samP#topic AOB04:27
*** openstack changes topic to "AOB (Meeting topic: masakari)"04:27
samPThere are some questions about how to install masakari, kolla support in ML04:27
samPI will try to support and answer them as much as I can.04:28
samPtpatil: Thanks for creating the docs for masakari04:29
samPNothing else from my side.04:29
samPIf no more items to discuss, we could finish today's meeting here.04:29
samPNext meeting will be 7/3004:30
tpatilGot it04:30
samPThank you all!04:30
tpatilsamP: Thanks04:30
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"04:30
openstackMeeting ended Tue Jul 16 04:30:37 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)04:30
openstackMinutes:        http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-07-16-04.00.html04:30
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-07-16-04.00.txt04:30
openstackLog:            http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-07-16-04.00.log.html04:30
*** hyunsikyang__ has quit IRC04:31
*** nitinuikey has quit IRC04:35
*** tpatil has quit IRC04:35
*** iyamahat has joined #openstack-meeting04:49
*** jamesmcarthur has quit IRC04:50
*** imsurit has quit IRC05:01
*** ykatabam has joined #openstack-meeting05:05
*** yamamoto has quit IRC05:05
*** igordc has joined #openstack-meeting05:07
*** pcaruana has joined #openstack-meeting05:08
*** ricolin_ has joined #openstack-meeting05:10
*** imsurit has joined #openstack-meeting05:10
*** ykatabam has quit IRC05:25
*** jamesmcarthur has joined #openstack-meeting05:29
*** ykatabam has joined #openstack-meeting05:29
*** yamamoto has joined #openstack-meeting05:34
*** jamesmcarthur has quit IRC05:37
*** vishalmanchanda has joined #openstack-meeting05:55
*** ykatabam has quit IRC05:56
*** SWDevAngel has quit IRC06:01
*** hyunsik_m has joined #openstack-meeting06:10
*** imsurit has quit IRC06:12
*** links has joined #openstack-meeting06:18
*** belmoreira has joined #openstack-meeting06:19
*** imsurit has joined #openstack-meeting06:36
*** iyamahat has quit IRC06:46
*** yamamoto has quit IRC06:51
*** yamamoto has joined #openstack-meeting06:52
*** yamamoto has quit IRC06:52
*** iyamahat has joined #openstack-meeting06:58
*** yamamoto has joined #openstack-meeting06:58
*** yamamoto has quit IRC06:58
*** rcernin has quit IRC07:00
*** dtrainor has quit IRC07:08
*** _pewp_ has quit IRC07:15
*** _pewp_ has joined #openstack-meeting07:15
*** igordc has quit IRC07:16
*** tssurya has joined #openstack-meeting07:20
*** belmoreira has quit IRC07:30
*** jamesmcarthur has joined #openstack-meeting07:33
*** ricolin_ is now known as ricolin07:33
*** jamesmcarthur has quit IRC07:38
*** slaweq has joined #openstack-meeting07:43
*** priteau has joined #openstack-meeting07:43
*** jraju__ has joined #openstack-meeting07:45
*** links has quit IRC07:45
*** jaewook_oh_ has quit IRC07:46
*** igordc has joined #openstack-meeting07:47
*** joxyuki has joined #openstack-meeting07:54
*** dkushwaha has joined #openstack-meeting08:02
dkushwaha#startmeeting tacker08:03
openstackMeeting started Tue Jul 16 08:03:03 2019 UTC and is due to finish in 60 minutes.  The chair is dkushwaha. Information about MeetBot at http://wiki.debian.org/MeetBot.08:03
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.08:03
*** openstack changes topic to " (Meeting topic: tacker)"08:03
openstackThe meeting name has been set to 'tacker'08:03
dkushwaha#topic Roll Call08:03
*** openstack changes topic to "Roll Call (Meeting topic: tacker)"08:03
dkushwahawho is here for Tacker weekly meeting?08:03
*** ttsiouts has joined #openstack-meeting08:03
dkushwahahello joxyuki hyunsik_m08:04
*** belmoreira has joined #openstack-meeting08:05
hyunsik_mI am hyunsikyang :) this is another account08:05
*** yamamoto has joined #openstack-meeting08:05
*** igordc has quit IRC08:06
dkushwahahyunsik_m, IMO you can use same nickname for different account too08:07
*** imsurit has quit IRC08:07
*** imsurit_ofc has joined #openstack-meeting08:07
*** kopecmartin|off is now known as kopecmartin08:07
hyunsik_mAh ok i will. Now i am in the outside. So.. please understand it this time!:)08:08
dkushwahanp :)08:08
dkushwahaseems no other members available today08:08
dkushwahaok lets start..08:09
dkushwaha#topic announcement08:09
*** openstack changes topic to "announcement (Meeting topic: tacker)"08:09
*** imsurit_ofc is now known as imsurit08:09
dkushwahaI propose Hiroyuki Jo  to join Tacker core team08:10
joxyukiwell, thanks dkushwaha08:10
hyunsik_mGood:) I will reply it soon!:)08:10
dkushwahajoxyuki, Welcome to Tacker core-Team. You did a great contribution.08:11
dkushwahaLooking for your more support now08:11
joxyukiyea, of course I will :)08:12
*** takahashi-tsc has joined #openstack-meeting08:12
*** ralonsoh has joined #openstack-meeting08:13
*** dtrainor has joined #openstack-meeting08:14
*** yamamoto has quit IRC08:15
dkushwahajoxyuki, Being core member, you also needs to keep tack on bugs part also. You can start getting those things slowly. And feel free to ask me for any confusion/doubts08:16
dkushwahamoving next..08:16
joxyukidkushwaha, all right. actually I am staring to prepare to fix bugs. I will update patches soon.08:17
*** jaewook_oh_ has joined #openstack-meeting08:17
dkushwahajoxyuki, thanks08:17
dkushwaha#topic BP  rolling upgrades08:18
*** openstack changes topic to "BP rolling upgrades (Meeting topic: tacker)"08:18
dkushwahahyunsik_m, any update on your BP ?08:18
dkushwahajaewook_oh_, ^08:19
hyunsik_mYes. I think we can find gap while we are developing this08:19
hyunsik_mI think basic  contents are almost done in this blueprint08:20
dkushwahahyunsik_m, if you have those gap list, feel free to discuss08:21
hyunsik_mOk!:) I will.08:23
dkushwahahyunsik_m, If you have initial patch sets available, feel free to push that WIP patch on gerrit where more reviewer give their opinion.08:25
dkushwaha#topic BP Multi-Interface for Containerized VNF08:26
*** openstack changes topic to "BP Multi-Interface for Containerized VNF (Meeting topic: tacker)"08:26
hyunsik_mActually I am developing this one first08:27
hyunsik_mSo. I will upload this patch soon as a WIP:)08:27
dkushwahahyunsik_m, nice08:27
dkushwahahyunsik_m, As these both BPs are already merged, so I request to please start drafting/pushing code part also. So that we will have sufficient time for code review08:28
dkushwahaeven WIP patch is also ok for now08:28
hyunsik_mOK I am supposed to go to business trip until next week. So I will upload it ASAP08:29
hyunsik_mThanks dkushwaha08:29
dkushwahahyunsik_m, thanks for the update :)08:30
dkushwahamoving next..08:30
*** yamamoto has joined #openstack-meeting08:30
dkushwaha#topic OpenDiscussion08:30
*** openstack changes topic to "OpenDiscussion (Meeting topic: tacker)"08:30
dkushwahaI have no other update for this week.08:31
dkushwahajoxyuki, anything from your side ?08:31
hyunsik_m I am done too08:31
jaewook_oh_Hello, for the open discussion, I updated the Prometheus plugin blueprint, could you check or comment for that one?08:32
dkushwahajaewook_oh_, yea, Its in my list.08:33
jaewook_oh_good thanks :)08:34
dkushwahajaewook_oh_, I have some questions on that, and will comment it on spec by today itself08:34
jaewook_oh_Great! OK I got it.08:35
dkushwahaDoes anyone have something more to discuss for now?08:36
dkushwahaotherwise we can close this meeting.08:36
dkushwahaOk, Thanks Folks for joining this meeting.08:37
dkushwahaclosing it now.08:37
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"08:37
openstackMeeting ended Tue Jul 16 08:37:32 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)08:37
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-07-16-08.03.html08:37
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-07-16-08.03.txt08:37
openstackLog:            http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-07-16-08.03.log.html08:37
*** hyunsikyang has quit IRC08:37
*** joxyuki has left #openstack-meeting08:37
*** belmoreira has quit IRC08:48
*** yamahata has quit IRC08:50
*** panda has quit IRC08:54
*** belmoreira has joined #openstack-meeting08:54
*** jamesmcarthur has joined #openstack-meeting08:56
*** panda has joined #openstack-meeting08:57
*** dkushwaha has quit IRC08:58
*** tetsuro has joined #openstack-meeting09:00
*** jamesmcarthur has quit IRC09:00
*** Liang__ has quit IRC09:20
*** ricolin has quit IRC09:20
*** jamesmcarthur has joined #openstack-meeting09:28
*** jamesmcarthur has quit IRC09:32
*** boxiang has joined #openstack-meeting09:41
*** belmoreira has quit IRC09:44
*** yamamoto has quit IRC10:00
*** yamamoto has joined #openstack-meeting10:05
*** yamamoto has quit IRC10:08
*** jaewook_oh_ has quit IRC10:09
*** priteau has quit IRC10:10
*** apetrich has joined #openstack-meeting10:16
*** tetsuro has quit IRC10:16
*** ttsiouts has quit IRC10:18
*** ttsiouts has joined #openstack-meeting10:19
*** ttsiouts has quit IRC10:23
*** yamamoto has joined #openstack-meeting10:38
*** yamamoto has quit IRC10:41
*** electrofelix has joined #openstack-meeting10:45
*** yamamoto has joined #openstack-meeting10:47
*** bbowen has quit IRC10:49
*** yamamoto has quit IRC10:54
*** yamamoto has joined #openstack-meeting10:56
*** yamamoto has quit IRC10:56
*** yamamoto has joined #openstack-meeting10:56
*** brinzhang has joined #openstack-meeting10:59
*** brinzhang_ has quit IRC11:03
*** njohnston_ has joined #openstack-meeting11:06
*** jraju__ has quit IRC11:12
*** links has joined #openstack-meeting11:12
*** ttsiouts has joined #openstack-meeting11:22
*** jamesmcarthur has joined #openstack-meeting11:29
*** tesseract has joined #openstack-meeting11:30
*** jamesmcarthur has quit IRC11:33
*** imsurit_ofc has joined #openstack-meeting11:37
*** imsurit has quit IRC11:37
*** imsurit_ofc is now known as imsurit11:37
*** jraju__ has joined #openstack-meeting11:53
*** links has quit IRC11:53
*** yamamoto has quit IRC11:54
*** takahashi-tsc has quit IRC11:54
*** raildo has joined #openstack-meeting11:55
*** lpetrut has joined #openstack-meeting11:57
*** dmacpher has quit IRC11:57
*** bbowen has joined #openstack-meeting12:00
*** dmacpher has joined #openstack-meeting12:01
*** jamesmcarthur has joined #openstack-meeting12:03
*** Lucas_Gray has joined #openstack-meeting12:06
*** jamesmcarthur has quit IRC12:08
*** jawad_axd has joined #openstack-meeting12:11
*** yamamoto has joined #openstack-meeting12:12
*** yamamoto has quit IRC12:12
*** jamesmcarthur has joined #openstack-meeting12:15
*** imsurit has quit IRC12:17
*** boxiang has quit IRC12:17
*** lpetrut has quit IRC12:28
*** lpetrut has joined #openstack-meeting12:32
*** Luzi has joined #openstack-meeting12:32
*** ricolin has joined #openstack-meeting12:35
*** ijw has joined #openstack-meeting12:38
*** jawad_axd has quit IRC12:43
*** ijw has quit IRC12:43
*** ykatabam has joined #openstack-meeting12:45
*** ykatabam has quit IRC12:50
*** tesseract has quit IRC12:53
*** yamamoto has joined #openstack-meeting12:54
*** tesseract has joined #openstack-meeting12:55
*** lpetrut has quit IRC12:57
*** yamamoto has quit IRC12:58
*** Lucas_Gray has quit IRC13:05
*** Lucas_Gray has joined #openstack-meeting13:11
*** zaneb has joined #openstack-meeting13:12
*** zaneb has quit IRC13:14
*** zaneb has joined #openstack-meeting13:14
*** ricolin has quit IRC13:18
*** jamesmcarthur has quit IRC13:19
*** eharney has quit IRC13:22
*** ykatabam has joined #openstack-meeting13:25
*** hemna has joined #openstack-meeting13:27
*** brinzhang has quit IRC13:28
*** shilpasd has quit IRC13:31
*** vishalmanchanda has quit IRC13:32
*** liuyulong has joined #openstack-meeting13:33
*** ttsiouts has quit IRC13:34
*** ttsiouts has joined #openstack-meeting13:35
*** trident has quit IRC13:38
*** ttsiouts has quit IRC13:38
*** ttsiouts has joined #openstack-meeting13:38
*** trident has joined #openstack-meeting13:39
*** enriquetaso has joined #openstack-meeting13:43
*** Luzi has quit IRC13:43
*** carloss has joined #openstack-meeting13:45
*** TxGirlGeek has joined #openstack-meeting13:48
*** psachin has quit IRC13:52
*** jraju__ has quit IRC13:53
*** mlavalle has joined #openstack-meeting13:55
*** tidwellr has joined #openstack-meeting13:59
slaweq#startmeeting networking14:00
openstackMeeting started Tue Jul 16 14:00:33 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: networking)"14:00
openstackThe meeting name has been set to 'networking'14:00
slaweqwelcome on neutron meeting :)14:00
slaweqso, lets start14:01
slaweq#topic Announcements14:01
*** openstack changes topic to "Announcements (Meeting topic: networking)"14:01
* mlavalle has to drive wife to follow up appointment after kidney transplant so will leave at 30 minutes after the meeting started14:01
slaweqmlavalle: good luck :)14:01
ralonsohmlavalle, good luck!!14:02
slaweqfirst announcement14:02
slaweqnext milestone is T-2 and it will be next week14:02
*** TxGirlGeek has quit IRC14:03
slaweqand a short reminder: train PTG summary is at http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006408.html if someone needs to look14:03
*** TxGirlGeek has joined #openstack-meeting14:03
slaweqthat's all from me, anyone else have got any announcement?14:03
mlavalleI have one14:04
amotokiThere is a discussion on repo retrirement process14:04
*** jrbalderrama has joined #openstack-meeting14:04
amotokiit affects neutron stadium process.14:04
amotokilet's keep our eyes on it.14:05
amotokiall from me.14:05
slaweqthx amotoki14:05
slaweqI will read it later today14:05
mlavallealso if you are planning to attend the PTG in Shanghai , please add your name to this etherpad14:06
mlavalleHopefully by the first week of August14:06
mlavalleI have to provide the Foundation with the number of people that will attend the PTG by August 11th14:07
mlavallefor planning purposes14:07
mlavalleand while you are at it, please start adding topics to be discussed there14:07
mlavallethat's it from me14:08
*** boden has joined #openstack-meeting14:08
slaweqthx mlavalle14:08
slaweqok, so lets move on14:08
*** TxGirlGeek has quit IRC14:08
slaweq#topic Blueprints14:08
*** openstack changes topic to "Blueprints (Meeting topic: networking)"14:08
slaweqTrain-2 blueprints: https://launchpad.net/neutron/+milestone/train-214:08
slaweqany updates on BPs?14:09
mlavalleI don't have any this week14:09
tidwellrI think subnets-different-pools-same-net is ready to go, I pushed one patch set this morning to clean up a nit14:09
*** TxGirlGeek has joined #openstack-meeting14:10
haleybtidwellr: it's on my list14:10
rubasovI managed to upload a few patches for the extraroute-atomic stuff: https://review.opendev.org/#/q/topic:extraroute+(p:openstack/neutron+OR+p:openstack/neutron-lib)14:10
*** TxGirlGeek has quit IRC14:10
slaweqtidwellr: I already +2'ed Your patch, I will check last PS today14:11
rubasovdespite being WIP I'd be happy if you can tell whether you like the basic direction in them or not14:11
slaweqrubasov: added to my list of reviews todo :)14:12
slaweqthx for updates14:12
rubasovslaweq: thanks a lot14:12
slaweqalso, for https://blueprints.launchpad.net/neutron/+spec/multiple-segment-per-network-per-host there is patch with spec proposed https://review.opendev.org/#/c/65717014:13
slaweqand we should review this spec :)14:13
mlavalleI'll look at it14:13
slaweqok, thx14:14
slaweqI think we can move on then14:14
slaweq#topic Community goals14:14
*** openstack changes topic to "Community goals (Meeting topic: networking)"14:14
slaweqI don't have any updates about IPv6-Only deployment goal, still waiting for gmann actions on it14:15
*** yamamoto has joined #openstack-meeting14:15
slaweqany updates about py3 or pdf generation?14:16
amotokifor pdf generation, a storyboard story was created. I assigned me to it.14:16
amotokiFor note, storyboard stories are prepared per project in this cycle14:17
slaweqamotoki: it says "review" for neutron, is there any patch related to this for neutron?14:18
amotokiI believe we can land neutron one first and then apply the same thing to other stadium projects, but we have more room to be improved.14:18
*** jrbalderrama has quit IRC14:18
amotokislaweq: you can find a corresponding patch in comment section below.14:18
slaweqamotoki: ok, thx14:19
slaweqone more question, what about api-ref? will it be also generated to pdf?14:19
amotokislaweq: this is the thing I would like to clarify to alex but I think it is not a target.14:19
slaweqamotoki: ok, please keep it in mind just :)14:20
amotokislaweq: sure14:20
slaweqthx amotoki for update on this14:20
slaweqabout py3 I can also mention that there is this etherpad: https://etherpad.openstack.org/p/neutron_stadium_python3_status14:20
slaweqwith list of what is still todo14:21
slaweqif anyone wants to help, You can check this etherpad and assign Yourself to some project/job and send patch :)14:21
slaweqok, lets move on14:22
slaweq#topic Bugs14:22
*** openstack changes topic to "Bugs (Meeting topic: networking)"14:22
slaweqI was on bug deputy last week14:22
*** yamamoto has quit IRC14:22
slaweqsummary is available at http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007763.html14:22
slaweqall high bugs are "in progress" so it's good14:23
slaweqthere is couple of medium/low which needs some volunteers14:23
rubasovI just assigned the metadata caching one to myself14:23
*** eharney has joined #openstack-meeting14:23
slaweqbasically there wasn't anything really scary during last week14:23
slaweqthx rubasov14:23
slaweqdo You want to talk about any other bug here?14:24
*** shintaro has joined #openstack-meeting14:24
slaweqok, I will take that as no :)14:25
slaweqour bug deputy this week is hongbin and he confirmed me that he will handle that14:25
slaweqand next week it's boden's turn14:26
bodenack, can barely wait :)14:26
slaweqthx boden :)14:26
*** links has joined #openstack-meeting14:26
mlavalleI had also sent hongbin an email last week14:26
slaweqok, next topic14:26
mlavalleso he got two reminders14:26
slaweqmlavalle: :)14:26
slaweqok, next topi then14:27
slaweq#topic neutron-lib14:27
*** openstack changes topic to "neutron-lib (Meeting topic: networking)"14:27
slaweqboden: any updates?14:27
bodennot much to report; business as usual... but could I ask for some help landing a bagpipe patch: https://review.opendev.org/#/c/664672/14:27
bodenthat's really all from the neutron-lib side unless others have comments14:28
* mlavalle looking at it14:28
slaweqboden: done :)14:29
haleybhah, triple whammy on that patch :)14:29
bodenthanks all!14:29
mlavalledang you all beat me to it14:29
slaweqok, lets move on then14:30
slaweq#topic On demand agenda14:30
*** openstack changes topic to "On demand agenda (Meeting topic: networking)"14:30
slaweqIn agenda there is nothing in this section14:30
*** ykatabam has quit IRC14:30
slaweqbut maybe someone wants to talk about something14:30
mlavalleso maybe I am not going to miss anything14:30
slaweqmlavalle: probably14:30
tidwellrslaweq: you had pinged me about the API tests for neutron-dynamic-routing moving to neutron-tempest-plugin14:30
tidwellrright now it's hung up on docker not being installed on the system when the jobs run14:31
*** shintaro has quit IRC14:32
slaweqtidwellr: do You want me to take a look at it?14:32
tidwellrI could use some help with that, if we can work that out https://review.opendev.org/#/c/652099/ will be good to go14:32
slaweqtidwellr: sure, I will check it tonight or tomorrow, and I will update Your patch if will find anything14:32
tidwellrsounds good, the scripts in neutron-dynamic-routing have a section installs these goodies before launching the tests, I'm not sure how to replicate that14:33
slaweqtidwellr: ok, I will try to figure it out14:34
* mlavalle departs the meeting14:35
slaweqok, if there is nothing else, I will give You back 25 minutes14:36
slaweqthx for attending14:36
slaweqand have a great week o/14:36
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:36
openstackMeeting ended Tue Jul 16 14:36:33 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:36
openstackMinutes:        http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-07-16-14.00.html14:36
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-07-16-14.00.txt14:36
rubasovthank you14:36
openstackLog:            http://eavesdrop.openstack.org/meetings/networking/2019/networking.2019-07-16-14.00.log.html14:36
*** mlavalle has left #openstack-meeting14:37
*** shintaro has joined #openstack-meeting14:38
*** boden has left #openstack-meeting14:38
*** armax has joined #openstack-meeting14:42
*** snierodz has joined #openstack-meeting14:43
*** davidsha has joined #openstack-meeting14:45
*** links has quit IRC14:52
ralonsoh#startmeeting neutron_qos15:00
openstackMeeting started Tue Jul 16 15:00:11 2019 UTC and is due to finish in 60 minutes.  The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: neutron_qos)"15:00
openstackThe meeting name has been set to 'neutron_qos'15:00
ralonsohYes, this is the command15:00
ralonsohapart from your feature and a bug almost resolved, there is no agenda15:01
ralonsohlet's wait 30 secs more15:01
ralonsohok, let's go15:01
ralonsoh#topic RFEs15:02
*** openstack changes topic to "RFEs (Meeting topic: neutron_qos)"15:02
ralonsohnothing in the agenda15:02
ralonsohlet's move to bug15:02
ralonsoh#topic Bugs15:02
*** openstack changes topic to "Bugs (Meeting topic: neutron_qos)"15:02
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/182669515:02
openstackLaunchpad bug 1826695 in neutron "[L3][QoS] cache does not removed when router is down or deleted" [Medium,In progress] - Assigned to LIU Yulong (dragon889)15:02
ralonsohpatch: #link https://review.opendev.org/#/c/656105/15:02
ralonsohIMO, the code is ok, I'll review it later today15:03
ralonsohnothing else in the list15:03
*** jamesmcarthur has joined #openstack-meeting15:03
ralonsohso, let's move to open discussion15:04
ralonsoh#topic Open Discussion15:04
*** openstack changes topic to "Open Discussion (Meeting topic: neutron_qos)"15:04
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/147652715:04
openstackLaunchpad bug 1476527 in neutron "[RFE] Add common classifier resource" [Wishlist,Triaged] - Assigned to Igor D.C. (igordcard)15:04
ralonsohdavidsha, any updates?15:04
ralonsohI've seen your POC updated today15:04
davidshaYup, I put up that patch that migrated the service plugin into Neutron: https://review.opendev.org/#/c/670050/15:04
*** shilpasd has joined #openstack-meeting15:05
ralonsohso 670050 is the first one, introducing the classification groups into the neutron repo15:05
ralonsoh(just to follow an order when reviewing)15:06
davidshaI need to migrated the RPC advertiser from the other patch over into the new patch and then rebase the QoS patch to remove the DB classification thing15:06
davidshaYup thats correct15:06
ralonsohok, so first the DB and then the RPC15:06
ralonsohanything peculiar in the DB patch?15:06
ralonsohBTW, https://review.opendev.org/#/c/670049/ should be reviewed first?15:07
ralonsoh670050 depends on the n-lib API change15:07
davidshaNo, it's basically a straight migration of the code in Neutron Classifier with a little refactoring to clarifying variable types in the functions.15:07
davidshaYup 49 is first but the guts of the change are in 5015:08
ralonsohshould you remove the "ccf" references?15:08
ralonsohjust asking15:08
davidshaWe can, you wanted to migrate the resources under qos, was it?15:09
ralonsohI think so15:09
davidshaor just leave the resources under / ?15:09
ralonsohplease, ask miguel and slaweq15:09
ralonsohI'm in doubt about this...15:09
ralonsohis the neutron classifier (in neutron core) going to be used outside qos?15:10
ralonsohthis is, IMO, a QoS extension15:10
ralonsohAm I correct?15:10
ralonsohDSCP -> CCF15:10
davidshaThere were rfe's to have security groups use it, but that was over a year ago.15:10
ralonsohhmmmm you are right15:11
ralonsohok, there is no need to make this to depend on QoS15:11
ralonsohmaybe you can leave the resources under /15:11
davidshaWhen it was proposed it was a QoS extension, but the original team saw values in expanding it out15:11
ralonsohyes, there is no hard dependency againts this15:11
ralonsohthat's the point,15:11
davidshawill I keep /ccf or just / ? ccf=common classification framework15:12
ralonsohI see no reason to keep CCF15:12
davidshakk, I'll remove it15:12
ralonsohbut, as I said, make this comment in the patch and I'll do the same15:13
ralonsohI'll review both patches today (or tomorrow morning)15:13
ralonsohsomething else??15:13
davidshaNope, thats it for me.15:13
ralonsohI think we can finish the meeting now (quite fast today)!15:14
ralonsohthanks David15:14
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:14
openstackMeeting ended Tue Jul 16 15:14:24 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:14
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-07-16-15.00.html15:14
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-07-16-15.00.txt15:14
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-07-16-15.00.log.html15:14
davidshaCool! talk to ya, enjoy the nice evening!15:14
*** e0ne has joined #openstack-meeting15:32
*** Lucas_Gray has quit IRC15:36
*** gyee has joined #openstack-meeting15:39
*** Lucas_Gray has joined #openstack-meeting15:41
*** diablo_rojo has joined #openstack-meeting15:41
*** eharney has quit IRC15:46
*** eharney has joined #openstack-meeting15:46
*** igordc has joined #openstack-meeting15:47
*** ttsiouts has quit IRC15:51
*** ttsiouts has joined #openstack-meeting15:52
*** tssurya has quit IRC15:55
*** ttsiouts has quit IRC15:57
*** manjeets has joined #openstack-meeting15:59
slaweq#startmeeting neutron_ci16:00
openstackMeeting started Tue Jul 16 16:00:10 2019 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: neutron_ci)"16:00
openstackThe meeting name has been set to 'neutron_ci'16:00
slaweqwelcome on another meeting :)16:00
njohnston_last meeting of the day for me! \o/16:00
slaweqnjohnston_: yes, for me too :)16:01
slaweqok, lets start as mlavalle will not be here today16:01
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate16:01
slaweqplease open the link and we can move on16:01
slaweq#topic Actions from previous meetings16:01
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"16:01
slaweqfirst one:16:02
slaweqmlavalle to continue debugging neutron-tempest-plugin-dvr-multinode-scenario issues16:02
slaweqhe send me some update about his findings:16:02
slaweq"In regards to the test connectivity with 2 routers scenario, when the L3 agent requests the router to the server, we call https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L1835-L185216:02
slaweqmy theory right now is that we are leaving one execption uncaught in that peace of code: duplicate entry. I will propose a WIP patch to prove the theory. That's the next step."16:02
slaweqso he is making progress on this one, hopefully this will be fixed soon :)16:03
slaweqI will add action for him for the next week to not forget about this16:03
slaweq#action mlavalle to continue debugging neutron-tempest-plugin-dvr-multinode-scenario issues16:03
slaweqnext one:16:04
slaweqslaweq to contact yamamoto about networking-midonet py3 status16:04
slaweqI emailed yamamoto and he respond me16:04
slaweqI pasted info from him to the related etherpad16:04
slaweqbasically there are 2 issues with midonet and py3 right now, they are aware of them but they don't know when this can be fixed16:05
slaweqnext one16:05
slaweqralonsoh to check how to increase, in FT, the privsep pool16:05
ralonsohslaweq, the number of threads is == num of CPUs16:06
ralonsohwe should not increase that number thus16:06
ralonsohhow to solve the timeouts? I still don't know16:06
*** vishalmanchanda has joined #openstack-meeting16:06
slaweqralonsoh: can we maybe limit number of test runners?16:06
ralonsohslaweq, but we'll reduce a lot the speed of execution16:07
ralonsohslaweq, I recommend to do this just as the last resource16:07
slaweqralonsoh: maybe it's worth to check how much longer it would take then16:07
*** bobh has joined #openstack-meeting16:07
ralonsohslaweq, we can do (num_cpu - 1)16:07
*** Lucas_Gray has quit IRC16:07
ralonsohjust for functional tests? or fullstack too?16:08
slaweqralonsoh: in fullstack it's differently I think I each test as got own "environment" so probably also own privsep-helpers pool16:08
*** ricolin has joined #openstack-meeting16:09
ralonsohI'll submit a patch to reduce the number of workers in FT16:09
slaweq#action ralonsoh to try a patch to resuce the number of workers in FT16:09
slaweqand the last one:16:10
slaweqslaweq to check neutron.tests.functional.agent.l3.test_ha_router.L3HATestCase issue16:10
slaweqBug reported https://bugs.launchpad.net/neutron/+bug/183656516:10
openstackLaunchpad bug 1836565 in neutron "Functional test test_keepalived_state_change_notification may fail do to race condition" [Medium,In progress] - Assigned to Slawek Kaplonski (slaweq)16:10
slaweqPatch proposed https://review.opendev.org/67081516:10
slaweqI found that it is race condition which may cause such failure in rare cases16:10
slaweqbut fortunatelly fix was quite easy :)16:10
slaweqany questions/comments?16:11
*** jawad_axd has joined #openstack-meeting16:11
ralonsohslaweq, I need to check the patch first, I'll comment on it16:11
slaweqralonsoh: sure, thx16:11
slaweqok, so lets move on16:12
slaweq#topic Stadium projects16:12
*** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)"16:12
slaweqPython 3 migration16:12
slaweqStadium projects etherpad: https://etherpad.openstack.org/p/neutron_stadium_python3_status16:12
slaweqI already talked about midonet plugin16:12
slaweqbut I have also short update anout neutron-fwaas and networking-ovn16:12
njohnston_I have a bagpipe update that looks ready to go: https://review.openstack.org/64105716:12
slaweqfor neutron-fwaas all voting jobs are switched to py3 now16:13
slaweqtoday I sent patch to switch multinode tempest job: https://review.opendev.org/67100516:13
slaweqbut I think I will have to wait with it and propose it to neutron-tempest-plugin repo when njohnston_ will finish his patches related to neutron-fwaas tempest plugin16:14
slaweqfor networking-ovn I today sent patch to switch rally job to py3: https://review.opendev.org/67100616:14
slaweqwith this patch I think we can consieder networking-ovn as done, because last job which isn't switched is based on Centos 7 so can't be run on py3 probably16:15
slaweqthat's all from me16:15
*** jawad_axd has quit IRC16:15
slaweqnjohnston_: thx for Your update about bagpipe16:15
slaweqI will review this patch tomorrow morning16:16
njohnston_slaweq: The fwaas change in the neutron-tempest-plugin repo is already merged,  so you could go ahead and propose it there immediately16:16
slaweqnjohnston_: ahh, yes, thx16:16
slaweqso I will do it that way16:16
slaweqso we are "almost" done with this transition :)16:17
*** mattw4 has joined #openstack-meeting16:17
slaweqany other questions/comments?16:17
igordcnot on this topic from me16:18
slaweqok, lets move on then16:18
slaweqtempest-plugins migration16:18
slaweqEtherpad: https://etherpad.openstack.org/p/neutron_stadium_move_to_tempest_plugin_repo16:18
slaweqany updates?16:18
njohnston_As I mentioned before, the first part of the fwaas move happened and I am tinkering with the second half now: https://review.opendev.org/64366816:18
*** mattw4 has quit IRC16:19
njohnston_If you propose your converted multinode tempest job in n-t-p then I will incorporate it into that change16:19
*** mattw4 has joined #openstack-meeting16:19
slaweqnjohnston_: ok, thx a lot :)16:19
slaweqfrom my side, I will tomorrow take a look at tidwellr's patch for neutron-dynamic-routing to help with it16:20
slaweqso that should be all related to the stadium projects16:21
slaweqI think we can move on to next topics16:21
slaweq#topic Grafana16:21
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"16:21
*** hemna has quit IRC16:22
njohnston_midonet cogate broken again16:22
slaweqI don't know why there is no any data for gate queue since last few days - didn't we merge anything recently?16:22
njohnston_the number of jobs that have been run in the gate queue seems pretty low, so I wonder if many patches are making it through16:23
njohnston_when lots of jobs have moderate fail numbers - even pep8 has an 11% failure right now, which is unusual - then it seems likely that something will fail in a given run.16:23
slaweqnjohnston_: about midonet job, You're right but I don't think we have anyone who wants to look into it16:24
slaweqI guess it will have to wait for yamamoto16:24
slaweqnjohnston_: about pep8 jobs I think I saw quite many WIP patches recently which may cause this failure rate16:25
*** mmethot has quit IRC16:25
slaweqfrom good things I see that functional/fullstack jobs are slightly better now (below 30%)16:26
njohnston_ok, could be.  I hope so!16:26
slaweqand also our "highest rates" jobs one with uwsgi and one with dvr are better now16:27
slaweqnot good but better at least :)16:27
slaweqanything else You want to add or can we move on?16:28
*** manjeets has quit IRC16:28
slaweqok, lets move on16:29
slaweq#topic fullstack/functional16:29
*** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)"16:29
njohnston_nothing else from me16:29
slaweqI saw at least twice this week issue with neutron.tests.functional.agent.linux.test_keepalived.KeepalivedManagerTestCase:16:29
*** jamesmcarthur has quit IRC16:29
slaweqbut IIRC it's related to privsep pool in tests, right ralonsoh?16:30
ralonsohlet me check16:30
ralonsohyes, and I think this is related to the sync decorator in ip_lib16:30
*** Shrews has joined #openstack-meeting16:30
ralonsohbut, of course, until we have a new pyroute2 release, we need to keep this sync decorator16:31
*** jamesmcarthur has joined #openstack-meeting16:31
ralonsohand in this order: first the sync and then the privsep16:31
ralonsohthis is killing sometimes the FTs16:31
ralonsohbecause you enter into the privsep and the wait16:31
ralonsohthat should be in the other way, but this is not suppoerted by py216:31
ralonsohthat' all16:32
slaweqok, thx ralonsoh for confirmation16:32
slaweqand do we have bug reported for that?16:32
ralonsohI'll find it16:32
ralonsohplease don;'t stop for me16:32
ralonsoh#link https://bugs.launchpad.net/neutron/+bug/183372116:33
openstackLaunchpad bug 1833721 in neutron "ip_lib synchronized decorator should wrap the privileged one" [Medium,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)16:33
*** tesseract has quit IRC16:33
slaweqralonsoh: thx a lot16:33
slaweqfor fullstack I found one new failure16:34
slaweqfailed test neutron.tests.fullstack.test_qos.TestMinBwQoSOvs.test_bw_limit_qos_port_removed16:34
slaweqdid You saw somethig like that before?16:34
ralonsohyes, but almost one year ago16:34
ralonsohand that was solved16:34
ralonsohI can take a look at this16:34
*** electrofelix has quit IRC16:35
slaweqI see such error in logs: http://logs.openstack.org/70/670570/1/check/neutron-fullstack/b563af4/controller/logs/dsvm-fullstack-logs/TestBwLimitQoSOvs.test_bw_limit_qos_port_removed_egress_.txt.gz#_2019-07-15_22_14_11_63816:35
slaweqbut I'm not sure if that is related really16:36
ralonsohthis should not happen16:36
ralonsohI'll download the logs and I'll grep to see if someone else is deleting this port before16:37
slaweqok, thx ralonsoh16:37
*** ayoung has joined #openstack-meeting16:37
slaweqcan You also report a bug for it, that we can track it there?16:38
ralonsohslaweq, perfect16:38
slaweq#action ralonsoh to report a bug and investigate failed test neutron.tests.fullstack.test_qos.TestMinBwQoSOvs.test_bw_limit_qos_port_removed16:38
slaweqthx ralonsoh :)16:38
slaweqanything else related to functional/fullstack tests?16:39
*** ijw has joined #openstack-meeting16:39
*** ijw has quit IRC16:39
*** ijw has joined #openstack-meeting16:39
slaweqok, lets move on then16:40
slaweq#topic Tempest/Scenario16:40
*** TxGirlGeek has joined #openstack-meeting16:40
*** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)"16:40
slaweqfirst few short informations16:40
slaweqTwo patches which should improve our gate a bit:16:40
slaweq    https://review.opendev.org/#/c/670717/16:40
slaweq    https://review.opendev.org/#/c/670718/16:40
slaweqplease take a look at them if You will have a minute :)16:40
njohnston_added them to the pile16:41
slaweqthx njohnston_16:41
slaweqand second thing16:41
slaweqI sent patches to propose 2 new jobs:16:41
slaweqhttps://review.opendev.org/#/c/670029/ - based on TripleO standalone16:41
igordcslaweq, do you think these will decrease the odds of sshtimeout?16:41
slaweqthis is non-voting job for now, please review it too16:41
slaweqand the second patch: https://review.opendev.org/#/c/670738/16:42
slaweqI realized recently that we don't have simple openvswitch job which runs neutron_tempest_plugin tests16:42
slaweqwe have only linuxbridge job like that16:42
slaweqso I'm proposing to have similar job for ovs too16:42
slaweqplease let me know what do You think about it16:43
slaweq(in patch review of course :))16:43
njohnston_thanks, these are great ideas16:43
slaweqthx njohnston_ :)16:43
slaweqigordc: According to SSH issue, I opened bug https://bugs.launchpad.net/neutron/+bug/183664216:43
openstackLaunchpad bug 1836642 in OpenStack Compute (nova) "Metadata responses are very slow sometimes" [Undecided,Incomplete]16:43
*** enriquetaso has quit IRC16:43
slaweqbasically there are 2 main issues with ssh:16:44
*** enriquetaso has joined #openstack-meeting16:44
slaweq1. covered by mlavalle's investigation which we talked on the beginning of the meeting16:44
*** hemna has joined #openstack-meeting16:44
slaweq2. this issue with slow metadata responses for /public-keys/ request which cause failure of confguring ssh key on instance and than failure with ssh16:45
slaweqaccording to 2) I reported it for both nova and neutron as I noticed that in most cases it is nova-metadata-api which is responding very slow16:45
slaweqbut sean-k-mooney found out today that (at least in one case) nova was waiting very long time on response from neutron to get security groups16:46
slaweq(this workflow is really complicated IMHO)16:46
igordcslaweq, is the error different than sshtimeout for 2. ?16:46
*** mmethot has joined #openstack-meeting16:47
slaweqso I will continue investigating this but if someone wants to help, please take a look at this bug, maybe You will find something interesting :)16:47
njohnston_wow, so nova has to ask for security group info in order to satisfy a metadata request?  that seems highly nonintuitive.16:47
slaweqigordc: basically in both cases final issue is authentication failure during sshing to the vm16:47
*** shintaro has quit IRC16:47
slaweqnjohnston_: yes, I was also surprised today when sean told me that16:47
igordcyep I will take some time and see if I can find out something.. some patches get sshtimeout almost every time16:48
*** armax has quit IRC16:48
slaweqigordc: difference between those 2 cases is that in 1) You will see about 20 attempts to get instance-id from metadata server16:48
*** hemna has quit IRC16:49
slaweqand in 2) You will see that instance-id was returned to vm properly and there was "failed to get .../public-keys" message in this console log16:49
slaweqand in this second case, in most cases which I checked it was like that becasuse response for this request took more than 10 seconds16:49
*** iyamahat has quit IRC16:50
slaweq10 seconds is timeout set in curl used in this ec2-metadata script in cirros16:50
slaweqthat's all from me about this issue16:51
slaweqany questions/comments?16:51
igordcwondering if certain patches with changes in certain places make this more likely16:52
slaweqigordc: I don't think so16:52
slaweqIMO, at least this second problem described by me is somehow related to performance of node at which tests are run16:53
slaweqand nothing else16:53
slaweqbut I don't have any proof for that16:53
slaweqlets move on16:54
slaweqfrom other things16:54
slaweqneutron-tempest-with-uwsgi job is now running better after my fix for tempest was merged16:55
slaweqbut it has a lot of timeouts16:55
slaweqmaybe it will be better when it will be switched to run only neutron and nova related tests16:55
slaweqbut I think it's worth to investigate why it's so slow so often16:56
slaweqso I will open bug for it16:56
slaweq#action slaweq to open bug about slow neutron-tempest-with-uwsgi job16:56
slaweqand that's all from me regarding tempest jobs16:57
slaweqanything else You want to add here?16:57
njohnston_not I16:57
slaweqok, thx for attending the meeting16:58
slaweqand have a nice week16:58
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"16:58
openstackMeeting ended Tue Jul 16 16:58:17 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:58
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-07-16-16.00.html16:58
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-07-16-16.00.txt16:58
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-07-16-16.00.log.html16:58
igordcty, bye16:58
*** dtrainor has quit IRC17:00
*** iyamahat has joined #openstack-meeting17:05
*** iyamahat_ has joined #openstack-meeting17:08
*** yamamoto has joined #openstack-meeting17:11
*** kopecmartin is now known as kopecmartin|off17:11
*** davidsha has quit IRC17:11
*** iyamahat has quit IRC17:11
*** hemna has joined #openstack-meeting17:14
*** TxGirlGeek has quit IRC17:17
*** igordc has quit IRC17:17
*** hemna has quit IRC17:18
*** e0ne has quit IRC17:20
*** yamahata has joined #openstack-meeting17:24
*** armax has joined #openstack-meeting17:34
*** hemna has joined #openstack-meeting17:34
*** ijw has quit IRC17:36
*** ijw has joined #openstack-meeting17:36
*** hemna has quit IRC17:38
*** ijw has quit IRC17:38
*** ijw has joined #openstack-meeting17:39
*** ricolin has quit IRC17:40
*** dtrainor has joined #openstack-meeting17:41
*** jamesmcarthur has quit IRC17:43
*** igordc has joined #openstack-meeting17:44
*** TxGirlGeek has joined #openstack-meeting17:53
*** hemna has joined #openstack-meeting17:54
*** hemna has quit IRC17:59
*** electrofelix has joined #openstack-meeting18:00
*** eharney has quit IRC18:02
*** electrofelix has quit IRC18:03
*** eharney has joined #openstack-meeting18:06
*** jamesmcarthur has joined #openstack-meeting18:07
*** efried has quit IRC18:12
*** efried has joined #openstack-meeting18:21
*** jamesmcarthur has quit IRC18:24
*** hemna has joined #openstack-meeting18:37
*** jamesmcarthur has joined #openstack-meeting18:42
*** jamesmcarthur has quit IRC18:51
*** jamesmcarthur has joined #openstack-meeting18:52
*** jamesmcarthur_ has joined #openstack-meeting18:53
*** jamesmcarthur_ has quit IRC18:54
*** mattw4 has quit IRC18:55
*** jamesmcarthur has quit IRC18:57
*** jamesmcarthur has joined #openstack-meeting18:58
*** jamesmcarthur_ has joined #openstack-meeting18:59
clarkbhello anyone else here for the infra meeting?18:59
clarkbmordred said he won't make it18:59
fungihe'll be missed19:00
fungiwho can throw wet animals in his place?19:00
clarkbhe will probably be underwater or something19:00
clarkbso all of the animals are already wet for him19:00
clarkb#startmeeting infra19:01
openstackMeeting started Tue Jul 16 19:01:04 2019 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:01
*** openstack changes topic to " (Meeting topic: infra)"19:01
openstackThe meeting name has been set to 'infra'19:01
clarkb#link http://lists.openstack.org/pipermail/openstack-infra/2019-July/006417.html19:01
clarkbFind our agenda at ^19:01
clarkb#topic Announcements19:01
*** openstack changes topic to "Announcements (Meeting topic: infra)"19:01
clarkbNothing major. As mentioned previously I'm largely afk today visiting people in town for oscon19:02
clarkb#topic Actions from last meeting19:02
*** openstack changes topic to "Actions from last meeting (Meeting topic: infra)"19:02
clarkb#link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-07-09-19.01.txt minutes from last meeting19:02
clarkb#action mordred create opendevadmin github account19:03
clarkb#action mordred clean up openstack-infra github org19:03
clarkbmordred has apparently started on this second item and is writing a currently non working script to do the work19:03
*** jamesmcarthur has quit IRC19:03
clarkb#topic Priority Efforts19:04
*** openstack changes topic to "Priority Efforts (Meeting topic: infra)"19:04
fungihopefully he's writing a working script, and its non-workingness is merely a transient state19:04
clarkbfungi: that was how I understood it :)19:04
clarkb#topic Update Config Management19:04
*** openstack changes topic to "Update Config Management (Meeting topic: infra)"19:04
*** jamesmcarthur_ has quit IRC19:04
*** bbowen has quit IRC19:05
clarkbhas anyone noticed yet the shiny job running optomization for system-config when you update it's .zuul.yaml?19:05
*** jamesmcarthur has joined #openstack-meeting19:05
clarkbOther than ^ I'm not sure we've made much progresson this item in the last week. Anyone have items to share related to this?19:05
fungiwhile i'm loathe to admit it, i haven't had occasion to update system-config's .zuul.yaml yet19:06
corvusi think the gitea repo creation is ready to merge19:06
fungi(since the changes took effect)19:06
*** jamesmcarthur_ has joined #openstack-meeting19:06
clarkbcorvus: do we need to squash any of the changes together because the chagnes only fully work near the end of the stack?19:06
corvusthat will unblock the next steps in the zuul-and-related-systems playbook19:07
clarkb(or an alternative would be to disable ansible for a bit, get everything merged, then turn ansible on19:07
corvusclarkb: i think we can merge them as-is, the worst that can happen if the broken version runs is that it immediately errors out19:07
clarkbgotcha so not failing and doing the wrong thing but failing safe. perfect19:07
corvusi'll start that going right now19:08
clarkbThe changes that I have reviewed so far look great (I need to rereview the parallelization change)19:08
corvus(since they have sufficient +2s)19:08
*** eharney has quit IRC19:08
clarkbas an added bonus I find the python script much more readable than the ansible yaml for this type of stuff (particularly in the manipulation of text/json)19:09
corvus:( i liked the yaml19:09
corvusclarkb: also, wait till you see the parallelized version before you say it's readable ;)19:09
clarkbI always find it hard to reason about the loop constructs with item and subelements...19:09
*** jamesmcarthur has quit IRC19:10
corvusbut yeah, we certainly wouldn't be doing *that* in ansible19:10
clarkbin any case it is much faster: about half an hour serialized instead of ~4 hours19:10
corvusand i'm aiming for ~10m with the updates19:10
clarkband down to ~9 minutes if done in parallel? great improvement19:10
fungii similarly find it hard to reason about threads and mutexes19:10
fungiso ymmv ;)19:10
corvusis how long it just took19:11
corvusso that node is a little slower than my workstation :)19:11
clarkbThat is probably a good transition into opendev topics19:11
clarkb#topic OpenDev19:11
fungiquite the speedup!19:11
corvusstill, all things considered, i think it probably means we can keep using the real data in the test jobs19:11
*** openstack changes topic to "OpenDev (Meeting topic: infra)"19:11
clarkbcorvus: nice19:12
clarkbcorvus: and at that speed can probably run it twice (to check the nooping too19:12
corvusyeah, that should be very fast19:12
clarkbThis related to opendev because this is sort of the very first step in improving project creation work that we discussed at the PTG19:12
*** diablo_rojo has quit IRC19:13
clarkbWe'd like to get to a place for orgs can largely self manage the proejcts under their umbrellas and making sure that doesn't take hours to reconcile is the very first step in that19:13
clarkbso yay19:13
clarkbNext we can have zuul trigger updates, then we can start distributing responsibility for managing those updates into the orgs ya?19:14
fungialso we can thoroughly test changes to those mechanisms in a timely manner19:15
fungi(in addition to speeding up gitea server replacements)19:15
fungiso much win19:16
clarkbfungi: those are actually done via database recovery currently to preserve the redirects19:16
clarkbfungi: so that doesn't take very long19:16
fungiahh, right, good point19:16
clarkb(we have docs on that)19:16
corvusthough we have plans for doing it from scratch19:16
fungithough we do have the redirects recorded in yaml19:16
corvusso we may get to that point in the future19:16
clarkbyup eventually we should eb able to make it more automated via direct restoration19:17
fungishould we need to rebuild them19:17
clarkbThe other opendev/gitea item I wanted to bring up was the one from today with the OOMing19:17
clarkbOur gitea servers don't have swap and under some circumstances (this needs further investigation) they OOM which can kill git which can prevent replication of refs19:18
clarkbI've got https://review.opendev.org/#/c/671102/ pushed up and if yall can take a quick look at that and decide it mostly does the right thing I can run that manually on gitea06. The reason for doing it on 06 is 06 has much more disk than the other gitea servers which unfortunately don't have much extra to spare19:18
fungii am mildly concerned that one client can basically cause an arbitrary gitea backend to cancel random git processes19:18
fungibut that ought to help19:18
clarkband ya we'll need to investigate further to see what is causing that and hopefully work to fix it in gitea19:19
clarkbAnything else opendev related or should we move on?19:19
fungicacti says outbound bandwidth spiked to at least 150mbps when this happened19:19
fungiwhere the baseline was around 10mbps19:20
fungiso maybe client rate limits could help, if haproxy has those19:20
clarkbjournald should have logs of what requests we made then right?19:20
clarkbcorvus: ^ does gitea log that information?19:20
fungiwell, apache will, presumable?19:20
fungier, presumably19:20
corvusi think gitea has it19:21
clarkbwe don't run apache with gitea19:21
corvusi don't think we have an apache (though we talked about it; we can add one if needed)19:21
corvusbut right now, our setup is simple enough we can do without19:21
fungioh, got it, so it's haproxy straight to the gitea sockets19:21
corvusyep, and gitea is terminating the tls19:21
fungiand haproxy won't know the nature of the requests because of not terminating ssl/tls19:21
clarkbfungi: it should know l3 information which may be sufficient for rate limiting (though that may make NAT users sad)19:22
fungiright, just won't tell us what they were requesting that might be so voluminous (maybe they were just recloning every repo though)19:22
corvuspretty sure gitea is logging the reqs19:23
clarkbcool so we can look into it further than19:23
fungiand docker-compose can spit those out?19:23
clarkbThe other thing I've done to help temporarily is triggered replication on against all gitea backends (currently on 06)19:24
clarkbfungi: ya or the docker command19:24
clarkb(and possibly journalctl too)19:24
fungicool, thanks19:24
clarkbThe extra replications are ensuring that all refs are in place which may have been OOMKilled19:24
fungiseems like a reasonable precaution until we get this under control19:25
*** jamesmcarthur_ has quit IRC19:25
clarkbSounds like we have rough ideas of how to address this further so lets move on19:26
clarkbstoryboard time19:26
clarkb#topic Storyboard19:26
*** openstack changes topic to "Storyboard (Meeting topic: infra)"19:26
clarkbdiablo_rojo_phon is at OSCON so unsure if paying attention today19:26
clarkbfungi: any new news19:26
fungiwe're going to try to do some story feature request subclassification on friday19:27
fungibut other than that, no major news i'm aware of19:27
clarkb#topic General Topics19:28
*** openstack changes topic to "General Topics (Meeting topic: infra)"19:28
clarkbfirst up is trusty server upgrades19:28
*** jamesmcarthur has joined #openstack-meeting19:28
clarkbfungi: any luck with the wiki git repo situation?19:28
clarkbI meant to help dig into that then got sucked into new cloud stuff19:28
clarkblink https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup19:29
*** jamesmcarthur_ has joined #openstack-meeting19:29
clarkbI think the new cloud stuff is largely under control/done so I can help with this tomorrow if still a problem19:29
fungii haven't had a chance to dig into what might be causing it. weird though... basically files like /srv/mediawiki/w/extensions/Renameuser/.git which just contain "gitdir: ../../.git/modules/extensions/Renameuser"19:29
fungiis that what submodules look like?19:30
fungiseems like the reverse of a submodule19:30
fungior maybe not19:30
clarkbmaybe it is subtree? which is like submodules in practice but implemeted differently19:30
clarkbbut ya my hunch would be somethign around submodules19:30
corvusi was just thinking git needed another way to be used wrong19:31
fungioh! we are cloning these into another git worktree, so... maybe!19:31
clarkbcorvus: I think they set out to make submodules work better :) and well ya19:31
fungii think the repo we're cloning to /srv/mediawiki/w/ may include files like extensions/Renameuser/.git19:31
fungii'll pursue that avenue19:32
clarkbcool let me know if I can help tomorrow19:32
*** jamesmcarthur_ has quit IRC19:32
clarkbNext up is New cloud updates19:32
clarkbWanted to mention that fortnebula is now in full use19:32
clarkbthank you donnyd for getting that set up19:32
*** jamesmcarthur_ has joined #openstack-meeting19:33
clarkbIn the process we fixed some issues with our cloud launcher config, problems in glean/dib with centos/fedora network manager compatibility and probably other things I'm forgetting now19:33
clarkbHopefully that makes it easier to set up the MOC and linaro resources that we expect will be coming up soon19:33
*** jamesmcarthur has quit IRC19:33
clarkbFor linaro Kevin Zhang has reached out to me and says they are starting to build an oepnstack arm cloud on packethost19:34
clarkband will get in touch when that is working and they ahve sorted out IP addressing (I gave them our IP addr requirements)19:34
*** jamesmcarthur has joined #openstack-meeting19:34
clarkbcorvus: mordred isn't here, do you know what is going on with MOC?19:34
fungior knikolla?19:34
clarkbjust curious if there was anything more to share today since it was mentioned last week19:36
fungiknikolla: mordred mentioned you were looking at providing moc resources to opendev's nodepool. have any details/updates?19:36
clarkbif not I'll wait patiently :)19:36
donnydclarkb: I still have 40% more to go, just waiting on more gear19:36
corvusknikolla: i think mordred made an account and maybe next thing is to add the second project to that?19:36
donnydI plan to have 100 builders19:36
knikollai remember seeing a request for an account on our cloud from mordred, i assume someone on our side approved that19:36
clarkbdonnyd: awesome19:37
fungiahh, cool19:37
fungithanks knikolla!19:37
knikollaif more than one project is needed, mordred can apply for another in the same way19:37
*** jamesmcarthur_ has quit IRC19:37
clarkbya typically we use two so that we can separate untrusted test nodes from trusted services like the mirror19:38
clarkbgood to know the process is the same just repeated19:38
corvuscool, sounds like progress and we should check with mordred19:38
knikollacool, does nodepool support application credentials?19:38
clarkbknikolla: it supports authentication via anything configurable through clouds.yaml19:39
clarkbso maybe?19:39
knikollathen most probably yes.19:39
cmurphyclouds.yaml supports app creds19:39
clarkbok sounds like next step is sync with mordred, thanks19:40
clarkbNext on the agenda is managing our PPA19:40
clarkbI've never personally had to manage a PPA so not sure I should lead this discussion but it was mentioned last week that it would be good for us to formalize the process a bit more19:41
clarkbmaybe that means docs or full on automation or some balance in between19:41
clarkbianw: ^ you've done much of the work there so probably have thoughts19:41
ianwyesterday I wrote up some docs at19:41
ianw#link https://review.opendev.org/#/c/670952/19:41
ianwif people want to make roles/jobs to build and sign debs and put them into the ppas with dput or whatever, that seems just fine19:42
clarkbThank you for the docs that seems like a good place to start19:42
clarkbcorvus maybe you can review that change and think about whether you'd like more (like automation)19:42
ianwi would say, for things like the afs work ... you're taking an upstream tarball and then shoe-horning it into the existing (older) debian package.  it's a lot of manual fiddling, and you've got to have a pretty good idea of how debian packaging works19:42
corvusif ianw is away, and someone says "we need a new openafs package" i'd like to know what to do...19:43
clarkbhrm ya in that case automating the shoehorning might be a good idea19:43
clarkbmaybe thats jsut a script we can run periodically?19:43
corvusi mean, i don't need a general guide, it's more, specifically, what commands do i run to make an updated openafs package?19:44
corvus(that could be a job, i'm ambivalent about that)19:44
corvuswe actually have precedent for this: https://docs.openstack.org/infra/system-config/nodepool.html#vhd-util19:46
clarkbgotcha so its that bit of documentation. FWIW I find the other bit ianw wrote useful too (particularly the stuff around perms)19:46
clarkbianw: so maybe we can add something like the vhd package build docs too and start from there?19:46
corvusthe equivalent of that for openafs i think would be sufficient19:47
*** ijw has quit IRC19:47
fungiif the openafs source package includes a debian/watch file it may be as simple as running uupdate in the source tree19:47
clarkbthat seems like a reasonable path forward. We only have 12 minutes left so I'll continue as the last item is fairly important too19:48
ianwfungi: it can be, but then we've also done quite a lot of backporting at times for things like arm64 support.  the complexity definitely varies based on what is being done.19:48
corvus(also, is that vhd-util stuff completely obsolete now?)19:48
clarkbcorvus: no we still use it to build rax images19:49
fungigiven rra used to handle the openafs uploads for debian, the package is probably fairly easy to update like that19:49
fungiand yeah, patching is always going to be the gotcha19:49
fungithough you can probably drop diffs into the debian/patches/ directory19:49
clarkbWe have been asked if we would like to attend the PTG as a team. I know I'll be going and expect fungi to be there as well and think mordred and corvus are going too. A rough headcount will be helpful for planning purposes but I think even with a small group we should have a chunk of time to work on opendev type things (this was useful in denver)19:50
clarkbAlso I know that we'd like to invite some of the gitea developers to have some space there too19:50
clarkbso I'll be requesting space for them as well19:50
fungii don't have a visa yet, and travel late in hurricane season is always a big question mark for me, but i will be there if at all possible19:50
clarkbI have until early august to fill out the survey but figure I'll probably send it in sometime this week19:51
clarkbso if you think you might be going just let me know so I can adjust the headcount numbers appropriately19:51
clarkb#topic Open Discussion19:52
*** openstack changes topic to "Open Discussion (Meeting topic: infra)"19:52
clarkbWe have a few minutes for any other business that you want to bring up19:52
*** jamesmcarthur has quit IRC19:53
*** jamesmcarthur has joined #openstack-meeting19:53
Shrewschina exceeds my travel tolerance by, like, a whole lot  :)19:53
fungimine too, but i've learned to repress my feelings19:54
ianwspeaking of docs19:54
*** jamesmcarthur_ has joined #openstack-meeting19:54
ianw#link https://review.opendev.org/66960219:54
ianw#link https://review.opendev.org/66883319:54
ianwthat's letsencrypt docs i wrote, and also updates to mirror-update docs to reflect the new server19:54
clarkbianw: thanks!19:55
ianwnote we're exporting the logs of the rsync runs now -> http://mirror.ord.rax.opendev.org/logs/rsync-mirrors/19:55
ianwthat should be helpful to people who want to contribute mirrors but previously had no way to see what was happening19:55
ianwand if you want to see them in your browser19:56
ianw#link https://review.opendev.org/67093419:56
ianwfixes that :)19:56
ianwoh, and for that to apply19:56
ianw#link https://review.opendev.org/67092719:56
fungiprobably better to link to http://files.openstack.org/mirror/logs/rsync-mirrors/19:56
clarkbShrews: thankfully there is a direct flight from seattle to shanghai19:57
ianwfungi: hrm yes, i guess that will need a mimetype update too19:57
*** ijw has joined #openstack-meeting19:57
fungithat way you don't have to pick a random mirror to look at the central logs from the mirror updater19:58
*** jamesmcarthur has quit IRC19:58
fungialternatively, we can just point them at the really trivial docs on how to set up your machine as an openafs client19:58
fungiand they can use whatever local tools they want to view those logs over afs19:58
ianwmany options :)19:59
clarkbAnd we are just about at time. Thank you everyone!19:59
fungithanks clarkb!19:59
clarkbFeel free to follow up on any and all topics here in #openstack-infra or at openstack-infra@lists.openstack.org19:59
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"19:59
openstackMeeting ended Tue Jul 16 19:59:39 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-07-16-19.01.html19:59
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-07-16-19.01.txt19:59
openstackLog:            http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-07-16-19.01.log.html19:59
*** Shrews has left #openstack-meeting20:00
*** mattw4 has joined #openstack-meeting20:04
*** eharney has joined #openstack-meeting20:06
*** bbowen has joined #openstack-meeting20:11
*** bobh has quit IRC20:33
*** ralonsoh has quit IRC20:35
*** diablo_rojo has joined #openstack-meeting20:37
*** rfolco is now known as rfolco_l8r20:39
*** ykatabam has joined #openstack-meeting20:41
*** dasp has quit IRC20:45
*** jamesmcarthur_ has quit IRC20:45
*** dasp has joined #openstack-meeting20:47
*** ykatabam has quit IRC20:47
*** ykatabam has joined #openstack-meeting20:52
*** pcaruana has quit IRC21:01
*** raildo has quit IRC21:04
*** armax has quit IRC21:43
*** diablo_rojo has quit IRC22:13
*** whoami-rajat has quit IRC22:31
*** rcernin has joined #openstack-meeting22:36
*** ijw has quit IRC22:39
*** ijw has joined #openstack-meeting22:43
*** slaweq has quit IRC22:55
*** armax has joined #openstack-meeting22:59
*** vishalmanchanda has quit IRC23:05
*** carloss has quit IRC23:17
*** diablo_rojo has joined #openstack-meeting23:23
*** hemna has quit IRC23:24
*** brinzhang has joined #openstack-meeting23:49
*** tobiash has quit IRC23:51
*** tobiash has joined #openstack-meeting23:52

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!