Wednesday, 2020-08-19

*** fyx has quit IRC00:03
*** fyx has joined #openstack-lbaas00:03
*** yamamoto has joined #openstack-lbaas00:17
openstackgerritStephanie Djajadi proposed openstack/octavia master: Create InfluxDB driver for statistics
*** yamamoto has quit IRC00:49
openstackgerritStephanie Djajadi proposed openstack/octavia master: Create InfluxDB driver for statistics
*** yamamoto has joined #openstack-lbaas01:05
*** devfaz has quit IRC02:03
*** devfaz has joined #openstack-lbaas02:04
*** zzzeek has quit IRC02:05
*** ataraday has quit IRC02:05
*** zzzeek has joined #openstack-lbaas02:11
*** rcernin has quit IRC02:21
*** rcernin has joined #openstack-lbaas02:59
*** shtepanie has quit IRC03:16
*** armax has quit IRC03:19
*** vishalmanchanda has joined #openstack-lbaas03:30
*** psachin has joined #openstack-lbaas03:37
*** yamamoto has quit IRC03:45
*** yamamoto has joined #openstack-lbaas04:02
*** rcernin has quit IRC04:14
*** rcernin has joined #openstack-lbaas04:21
*** rcernin has quit IRC04:37
*** rcernin has joined #openstack-lbaas04:46
*** ianychoi_ has joined #openstack-lbaas05:25
*** ianychoi has quit IRC05:28
openstackgerritSam Morrison proposed openstack/octavia-dashboard master: Fix up display of selected Availability zone in workflow.
*** mvorwerk has joined #openstack-lbaas05:34
*** mvorwerk_ has joined #openstack-lbaas05:39
*** mvorwerk has quit IRC05:40
*** gcheresh_ has joined #openstack-lbaas05:56
*** ramishra has joined #openstack-lbaas05:56
*** ataraday_ has joined #openstack-lbaas06:11
*** dmellado has quit IRC06:11
openstackgerritAnn Taraday proposed openstack/octavia master: [Amphorav2] Healthmonitor operation minor fixes
*** gthiemonge has quit IRC06:17
*** gthiemonge has joined #openstack-lbaas06:17
openstackgerritCarlos Goncalves proposed openstack/octavia stable/stein: Disable log offloading scenario tests
cgoncalves^ stable/stein gate fixer06:24
*** sapd1 has joined #openstack-lbaas06:25
openstackgerritCarlos Goncalves proposed openstack/octavia stable/stein: Remove scenario bionic job from check
*** dmellado has joined #openstack-lbaas06:41
*** happyhemant has joined #openstack-lbaas06:42
*** dmellado has quit IRC07:07
*** born2bake has joined #openstack-lbaas07:33
*** dmellado has joined #openstack-lbaas07:53
openstackgerritCarlos Goncalves proposed openstack/octavia stable/ussuri: Set amphora-agent build-only packages
openstackgerritCarlos Goncalves proposed openstack/octavia stable/ussuri: Switch back to using git for the agent ref
cgoncalves^ stable/ussuri gate fixer08:00
openstackgerritXiaoYu Zhu proposed openstack/octavia master: Alternative Distributor for L3 Active-Active, N+1 Amphora Setup
openstackgerritOmer Schwartz proposed openstack/octavia-tempest-plugin master: Fix MismatchError in tags API tests
*** sapd1 has quit IRC09:23
*** ramishra has quit IRC09:41
*** rcernin has quit IRC09:43
*** ramishra has joined #openstack-lbaas09:44
*** rcernin has joined #openstack-lbaas09:49
*** yamamoto has quit IRC10:02
*** ramishra has quit IRC10:07
*** ramishra has joined #openstack-lbaas10:08
*** yamamoto has joined #openstack-lbaas10:22
*** yamamoto has quit IRC10:26
*** yamamoto has joined #openstack-lbaas10:29
*** rcernin has quit IRC10:30
*** ataraday_ has quit IRC10:31
*** rcernin has joined #openstack-lbaas10:34
*** yamamoto has quit IRC10:41
*** yamamoto has joined #openstack-lbaas10:48
*** rcernin has quit IRC10:48
*** ramishra has quit IRC10:50
*** rcernin has joined #openstack-lbaas10:52
*** ramishra has joined #openstack-lbaas10:55
*** rcernin has quit IRC11:07
*** sapd1 has joined #openstack-lbaas11:09
openstackgerritXiaoYu Zhu proposed openstack/octavia master: Alternative Distributor for L3 Active-Active, N+1 Amphora Setup
*** yamamoto has quit IRC11:20
*** yamamoto has joined #openstack-lbaas11:20
*** yamamoto has quit IRC11:21
*** rcernin has joined #openstack-lbaas11:21
*** yamamoto has joined #openstack-lbaas11:31
*** yamamoto has quit IRC11:36
*** sapd1 has quit IRC11:43
*** yamamoto has joined #openstack-lbaas12:15
*** yamamoto has quit IRC12:30
*** mvorwerk_ has quit IRC12:37
*** mvorwerk has joined #openstack-lbaas12:38
*** rcernin has quit IRC12:46
*** rcernin has joined #openstack-lbaas12:48
openstackgerritDarragh O'Reilly proposed openstack/neutron-lbaas stable/rocky: Unplug tap from bridge on agent recovery after failover
openstackgerritBrian Haley proposed openstack/octavia-tempest-plugin master: Change pool create scenario test to wait for operating status
*** TrevorV has joined #openstack-lbaas13:46
*** ataraday_ has joined #openstack-lbaas13:47
*** KeithMnemonic has joined #openstack-lbaas14:09
*** happyhemant has quit IRC14:22
*** yamamoto has joined #openstack-lbaas14:28
*** tkajinam has quit IRC14:33
*** rcernin has quit IRC14:37
*** armax has joined #openstack-lbaas14:50
*** mvorwerk has quit IRC14:50
*** gcheresh_ has quit IRC14:50
*** mvorwerk has joined #openstack-lbaas14:55
*** mvorwerk has quit IRC15:05
*** kberger_ has joined #openstack-lbaas15:08
*** KeithMnemonic has quit IRC15:10
*** mvorwerk has joined #openstack-lbaas15:13
*** mvorwerk has quit IRC15:19
*** mvorwerk has joined #openstack-lbaas15:20
*** sapd1 has joined #openstack-lbaas15:26
*** yamamoto has quit IRC15:28
openstackgerritVishal Manchanda proposed openstack/octavia-dashboard master: [goal] Migrate testing to ubuntu focal
openstackgerritMerged openstack/octavia stable/stein: Disable log offloading scenario tests
*** mvorwerk has quit IRC15:33
*** mvorwerk has joined #openstack-lbaas15:34
johnsomBig agenda for the meeting today, hope everyone can make it15:36
*** mvorwerk has quit IRC15:39
*** mvorwerk has joined #openstack-lbaas15:41
cgoncalvesany recommendations for a HTTP/2 capable client library for Python? been playing with httpx and I like it so far15:43
johnsomAh, I did have one I would recommend. Let me see if I can find those notes15:43
*** shtepanie has joined #openstack-lbaas15:49
johnsomYeah, I think httpx was what I was looking at15:53
cgoncalvescool. I may propose it to global requirements so we can use in tempest15:54
johnsomI think they plan to do a 1.0 release in Sept15:55
cgoncalvesyes, they have that written in their home page15:55
cgoncalves"A 1.0 release is expected to be issued sometime around september 2020."15:55
*** yamamoto has joined #openstack-lbaas15:58
johnsomCores: This patch to fix neutron quotas for devstack will help with some tempest scenario failures we see: it is an easy review. Would be great to get merged soonish15:59
johnsom#startmeeting Octavia16:00
openstackMeeting started Wed Aug 19 16:00:11 2020 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: Octavia)"16:00
openstackThe meeting name has been set to 'octavia'16:00
johnsomHi everyone! Thanks for joining. We have a full agenda today.16:00
johnsom#topic Announcements16:01
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"16:01
johnsomJust a quick reminder, registration is open for both the Summit and PTG. Please register as it is free and virtual!16:01
johnsomThere are at least three Octavia presentations proposed for the Summit.16:02
johnsomOne is new features, one on A10 driver, one on the F5 driver16:02
johnsomI don't know if they got accepted yet, but good to see16:02
johnsomAlso I want to highlight some upcoming milestone dates. Feature freeze is coming up soon!16:03
johnsomFinal octavia-lib release is the week of August 31st (not feature freeze, the final release)16:03
johnsomFinal python-octaviaclient release is the week of September 7th (not feature freeze, the final release)16:03
johnsomAll other repositories go into feature freeze the week of September 7th16:04
johnsomSo, if you have feature work you want to get into Victoria, time is running out.16:04
johnsomI plan to setup a section the priority review this week to highlight feature work that has a upcoming deadline16:04
johnsomAny questions/comments about feature freeze?16:05
*** ZhuXiaoYu has joined #openstack-lbaas16:05
johnsomAny other announcements this week?16:05
johnsom#topic Brief progress reports / bugs needing review16:06
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"16:06
ataraday_Mostly do reviews and update some of amphorav2/default ciphers changes.16:06
ataraday_I would like to raise some of amphorav2 topics, but I will postpone it until Open Discussion.16:07
johnsomI have been working on an internal project a bit over the last week. Also continuing on the tempest updates, reviews, and reviving some of my older patches that are up for review.16:07
cgoncalvesreviews, ALPN and HTTP/2 work all around (octavia, client, SDK, tempest), fixing the gate16:08
aannuusshhkkaaWe are awaiting reviews on #link the refactor for amphora driver patch16:08
johnsomI need to wrap up creating this internal training stuff then I will focus on the priority reviews list.16:08
aannuusshhkkaaand on #link adding new metrics - response time patch16:08
johnsomCool, any other updates?16:09
aannuusshhkkaawe have the also written the driver to integrate InfluxDB #link and would be able to merge that on top of these other patches.16:10
johnsom#topic vPTG timeslots16:10
*** openstack changes topic to "vPTG timeslots (Meeting topic: Octavia)"16:10
johnsomSo virtual PTG planning has started. I have signed us up to have a session again.16:11
johnsomI have tentatively selected the same days and timeslot for our sessions. Tue Oct 27th and 28th 13:00-17:00 UTC16:11
johnsomI wanted to ask if that is good for everyone or if we should do another doodle poll?16:12
ataraday_works fine for me16:12
gthiemongesame here16:12
cgoncalvesworks for me but in the spirit of fairness we could change to another time to be nice to our US Western folks16:13
rm_workmy schedule adjusts to the things I need to do, so fine by me16:13
johnsomAh, I'm an odd person out, so I will be flexible and order extra coffee16:14
johnsomOk, anyone else that this time slot doesn't work for?16:14
johnsomThat seems like a pretty good quorum.16:14
johnsomYou know it is customary for the newest core members to run the PTG right?16:15
johnsomJust kidding. grin16:15
johnsomOk, unless I here from anyone in the next day or so, I will leave those timeslots on the vPTG calendar.16:16
johnsomWe could also do an additional session, but I think we did ok with the two time slots. Do we need more time or is this good?16:16
cgoncalvesthank you for organizing our PTG again16:17
johnsomYou are welcome16:17
johnsomOk. I think we are good on PTG topics for the week.16:18
johnsom#topic Community goals16:18
*** openstack changes topic to "Community goals (Meeting topic: Octavia)"16:18
johnsomI wanted to do a quick review of the Victoria goals as they are coming due.16:18
johnsomGoal 1 was to remove the legacy zuul jobs and migrate to zuulv3 only.16:18
johnsomI think we have accomplished this goal. Does anyone know of a case we still have running legacy zuul?16:19
johnsomIf I remember right it was only grenade that was left.16:19
cgoncalvesyeah, I think so16:20
johnsomOk, I will consider that one complete as I think Carlos took care of all of the grenade jobs. (Thank you!)16:20
johnsomGoal 2 is test on Ubuntu Focal16:21
johnsomThis one I think we have work to do.16:21
johnsomI have tested the control plane running on Focal. There was a problem with Barbican, but I think that is fixed now.16:21
johnsomSo we should be ok to switch the control plane (nodepool hosts) over to focal where we haven't already.16:22
johnsomWhat I know is broken is using Focal for the amphora image.16:22
johnsomOn boot, the lb-mgmt-net configured by nova/neutron fails to come up inside the image.16:22
johnsomI tested this again yesterday to see if Ubuntu had fixed the bug, but it appears to still be a problem.16:23
johnsomIs anyone interested in looking into this issue? Do we have anyone from Canonical here?16:24
johnsomMy guess is there is a bug in Focal that if the ifupdown compatibility package is installed, the cloud-init/netplan/systemd configuration doesn't work. This is purely a guess though16:25
johnsomShould we just fail this goal and leave focal broken? We would have to use a legacy OS image for the amphora, but it should work.16:26
cgoncalvesI am reading the goal. the way I read it, the goal is limited to the nodepool host running Focal16:27
johnsomI think we should consider moving our default gate hosts to centos, but I don't want to do that mid-cycle. I would prefer that to happen at the start of a cycle16:27
cgoncalvesso, we would not be failing the goal if we run Focal controller + CentOS/Bionic amphora16:27
johnsomYeah, it's the "spirit" of the goal though....16:29
rm_workyeah I think it was intended to mean we run a Focal amp...16:29
johnsomI wish we had someone from Canonical that could help with debugging this...16:29
rm_workat least, I would have assumed that16:29
cgoncalves(I don't recall a similar goal for when CentOS 8 was released...)16:29
johnsomTrue, but centos 8 is included in the PTI for this release:16:30
cgoncalvescorrect. it was also in the PTI for Ussuri16:31
johnsomOk, so I think a proposal on the floor is to move forward with focal control plane, bionic amphora, and have a non-voting failing focal amp check job?16:31
rm_workeugh, that seems gross16:32
rm_workbut i don't see a way around it16:33
johnsomI may try reaching out to the only person I know at Canonical and see if they know someone that can work on it.16:33
rm_worki don't have the cycles or the knowledge to be useful with the ubuntu (or any) networking stack internals16:33
johnsomYeah, I think this is going to take some time to figure out and I can't take that time away from my other priorities for it.16:34
johnsomI will take the action item to setup the gates as discussed in the proposal above.16:34
johnsomThank you for the discussion on this.16:35
johnsom#topic Kolla updates for Octavia need review16:35
*** openstack changes topic to "Kolla updates for Octavia need review (Meeting topic: Octavia)"16:35
johnsomI wanted to highlight that the Kolla team has been doing work to improve the Octavia support in Kolla. This is excellent!16:35
johnsomThey have some patches up for review and I would like us to support them in this effort.16:36
johnsomPlease if you have some time, help them review these patches.16:36
johnsomIt will help folks new to OpenStack have a better out-of-box experience when deploying with Kolla.16:37
*** yamamoto has quit IRC16:37
johnsomConsider it some "building the community" time. grin16:37
johnsom#topic Open Discussion16:38
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"16:38
ataraday_I’ve got a couple of things regarding amphorav2.16:38
johnsomOk, thank you for hanging in as we go through this full agenda. Other topics? I think ataraday_ had some.16:38
johnsomataraday_ All yours16:39
ataraday_I looked into the problem of and left some notes, but I think we can discuss it here.16:39
ataraday_This issue can happen if state of task cannot be written to db properly - so it will keep retrying to do it.16:39
ataraday_So, if we have DBConnection errors - nothing can be written so retries are acceptable.16:39
ataraday_The main problem is when task is completed successfully, but it cannot write its state to db16:40
ataraday_making the loop16:40
ataraday_Like some if the objects are not json serializable - in repro example.16:40
johnsomYeah, I think DB issues are ok to retry. The issue I had was an endless loop creating network ports until my cloud ran out of IP addresses.16:40
ataraday_But as I see this is something we should guarantee - that all our tasks are passing the correct type of data.16:40
johnsomIt "fork tested" my cloud and brought it down.16:41
ataraday_Or it is possible with some other scenario?16:42
johnsomGranted, it was a coding error that led to it, but it's a very scary thing to be a potential.16:42
ataraday_I propose to taskflow as a workaround16:42
johnsomI was wondering if there was a re-dispatch limit we could set.16:42
ataraday_Not really fixing - but at least we won't retry endlessly16:43
johnsomThank you for the patch. I will review it. (I am a bad taskflow core as I only look for patches there every once in a while).16:43
ataraday_johnsom, Thanks!16:43
ataraday_One more thing about amphorav2 failover refactor - I have some comments there -will you have time to resolve them?16:44
ataraday_Or I should step in?16:44
johnsomI saw them, I have not had time to resolve/investigate them. Thank you for the review.16:45
johnsomThere is a relationship between the notification types that I know exists, we haven't implemented, but I don't yet fully understand yet. I feel like I need to spend some quality time with the taskflow code to understand how we need to tie those two things together. Like a notification should also have a hook for the timer notification.16:46
johnsomIf you have ideas or time to look at that, I am open of course. You can post followup patch proposals any time.16:47
ataraday_OK, I just concerned with number of patches about amphorav2 for review got a bit stuck.16:48
johnsomYeah, I agree.16:48
ataraday_That's all from me for today :D16:49
johnsomataraday_ Ok, thank you. Please keep reminding us and updating the priority patch review list to highlight those v2 patches.16:49
johnsomAny other topics today?16:50
ZhuXiaoYuI have uploaded a POC of multi-active mode on: please have a look if you have time16:50
johnsomExcellent, thank you for working on that.16:51
johnsomThat is a reminder too, we still have an open specification.16:51
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Switch default SUT to CentOS 8
johnsomThank you Carlos for your review. Others, please consider taking some time to review it.16:52
*** psachin has quit IRC16:53
cgoncalvesleveraging Neutron ECMP support is a good idea. my main concern is with the amphora health check monitoring16:53
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Switch default SUT to CentOS 8
ZhuXiaoYuyes, I see those comments16:54
johnsomI think my last concern/comment was about the DR proposal that requires member servers to have a special configuration.16:54
ZhuXiaoYuIt is OK with DVR, neutron will pick ns to set configurations16:56
johnsomOk, we are coming up on time for the meeting. Any other comments on the specification or other topics?16:58
johnsomThank you everyone for hanging on for the long agenda! Have a great week.16:59
*** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list:"16:59
openstackMeeting ended Wed Aug 19 16:59:52 2020 UTC.  Information about MeetBot at . (v 0.1.4)16:59
openstackMinutes (text):
johnsomZhuXiaoYu Thank you for bringing this up!17:00
ZhuXiaoYuyou are welcome17:00
*** ZhuXiaoYu has quit IRC17:02
johnsomThis Ussuri gate fix need just one more core review:
johnsomThanks Carlos for the gate fixes today!17:03
cgoncalvesall good :)17:04
*** ataraday_ has quit IRC17:05
johnsomOn different patch: ERROR: Could not find a version that satisfies the requirement glance-store===2.2.0 Ugh17:05
johnsomThe mirrors (which aren't mirrors, but caches) seem to be having issues with new releases17:07
cgoncalvesyeah, also seen neutron-lib17:11
johnsomI mentioned it in opendev just to track these. It's been a problem on another package recently as well.17:15
johnsomOh yikes, yeah now it's neutron-lib too17:18
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Switch default system under test to CentOS 8
* cgoncalves braces for impact ^17:24
*** sapd1 has quit IRC17:28
*** sapd1 has joined #openstack-lbaas17:29
*** maciejjozefczyk_ has joined #openstack-lbaas17:47
*** maciejjozefczyk has quit IRC17:48
*** maciejjozefczyk has joined #openstack-lbaas17:49
*** sapd1 has quit IRC17:50
*** maciejjozefczyk_ has quit IRC17:52
cgoncalvesoh gosh, + libvirt-python. jobs are all failing18:09
johnsomlibvirt-python is a different issue. it's asking for a distro of 8.2 where pypi is 8 only18:13
johnsomBut yeah, the pypi caches are failing all over the place18:14
*** yamamoto has joined #openstack-lbaas18:36
*** maciejjozefczyk has quit IRC18:36
*** maciejjozefczyk has joined #openstack-lbaas18:37
*** vishalmanchanda has quit IRC18:49
*** yamamoto has quit IRC19:13
*** dayou has joined #openstack-lbaas20:33
*** dayou_ has quit IRC20:33
*** dayou has quit IRC20:40
*** dayou has joined #openstack-lbaas20:43
*** maciejjozefczyk has quit IRC21:08
*** yamamoto has joined #openstack-lbaas21:12
*** dmsimard7 has joined #openstack-lbaas21:14
*** dmsimard has quit IRC21:15
*** dmsimard7 is now known as dmsimard21:15
johnsomWell, the pypi cache issue has mysteriously resolved itself for now, gates seem to be functioning again.21:22
*** yamamoto has quit IRC21:39
*** yamamoto has joined #openstack-lbaas22:06
*** shtepanie has quit IRC22:19
*** yamamoto has quit IRC22:25
*** TrevorV has quit IRC22:31
openstackgerritMerged openstack/octavia stable/ussuri: Set amphora-agent build-only packages
*** born2bake has quit IRC22:49
*** tkajinam has joined #openstack-lbaas22:58
*** rcernin has joined #openstack-lbaas22:59
openstackgerritMerged openstack/octavia master: Increase the devstack secuirty group rules quota
openstackgerritMichael Johnson proposed openstack/octavia-tempest-plugin master: Adjust API tests for NotImplemented skip
*** spatel has joined #openstack-lbaas23:24
*** mvorwerk_ has joined #openstack-lbaas23:25
*** mvorwerk has quit IRC23:28
*** spatel has quit IRC23:30
*** spatel has joined #openstack-lbaas23:40

Generated by 2.17.2 by Marius Gedminas - find it at!