Wednesday, 2020-12-02

openstackgerritMerged openstack/octavia stable/train: Fix memory consumption issues with default connection_limit
openstackgerritMichael Johnson proposed openstack/octavia stable/train: Fix operational status for disabled UDP listeners
johnsomRebasing things so we can get the stable/train patches moving.00:07
openstackgerritMichael Johnson proposed openstack/octavia stable/ussuri: Followup Remove haproxy_check_script for UDP-only
openstackgerritMichael Johnson proposed openstack/octavia stable/train: Fix load balancers with failed amphora failover
*** vesper11 has joined #openstack-lbaas01:09
*** rcernin has quit IRC01:37
*** rcernin has joined #openstack-lbaas01:39
*** spatel has joined #openstack-lbaas01:48
*** sapd1_x has joined #openstack-lbaas02:30
*** rcernin has quit IRC02:36
*** sapd1_x has quit IRC02:42
*** rcernin has joined #openstack-lbaas03:06
*** xgerman has quit IRC04:02
openstackgerritMerged openstack/octavia stable/victoria: Fix load balancers with failed amphora failover
*** sapd1_x has joined #openstack-lbaas04:23
*** vishalmanchanda has joined #openstack-lbaas04:38
*** gcheresh has joined #openstack-lbaas04:47
*** sapd1_x has quit IRC05:01
*** gcheresh has quit IRC05:23
*** gcheresh has joined #openstack-lbaas05:29
*** rcernin has quit IRC05:43
*** rcernin has joined #openstack-lbaas05:55
*** gcheresh has quit IRC06:08
openstackgerritMerged openstack/octavia stable/ussuri: Add some details on enable_anti_affinity option
openstackgerritMerged openstack/octavia stable/ussuri: [Amphorav2] Healthmonitor operation minor fixes
*** spatel has quit IRC06:37
*** gcheresh has joined #openstack-lbaas06:38
*** zzzeek has quit IRC07:13
*** zzzeek has joined #openstack-lbaas07:14
*** rcernin has quit IRC07:38
*** ccamposr__ has joined #openstack-lbaas07:42
*** rcernin has joined #openstack-lbaas07:44
*** ccamposr has quit IRC07:45
*** rcernin has quit IRC07:48
*** rcernin has joined #openstack-lbaas08:01
*** damien_r has joined #openstack-lbaas08:12
*** rcernin has quit IRC08:16
*** damien_r has quit IRC08:17
*** rpittau|afk is now known as rpittau08:28
*** rcernin has joined #openstack-lbaas08:32
*** rcernin has quit IRC08:46
*** tkajinam has quit IRC08:49
*** sapd1_x has joined #openstack-lbaas09:00
*** luksky has joined #openstack-lbaas09:15
*** rcernin has joined #openstack-lbaas09:42
*** rcernin has quit IRC09:42
*** sapd1_x has quit IRC10:00
*** happyhemant has joined #openstack-lbaas10:05
noonedeadpunkhi! Does anybody have any idea why this might be happening?
noonedeadpunkI see it only on Debian buster (which has py3.7 on board)10:13
noonedeadpunkI'm not even sure where it's raised as not trace :(10:14
*** sapd1_x has joined #openstack-lbaas10:15
gthiemongenoonedeadpunk: could it be a client that timed out before receiving the reply? the api would write into a closed socket10:21
noonedeadpunkwell client is tempest in this case I guess. as eventually because of it's failure I went to look through service logs
noonedeadpunkhm, from other side it might be our LB issue...10:24
noonedeadpunkyeah it was marked as down
noonedeadpunkbut eventually everything was marked that way - interesting10:25
noonedeadpunkthanks for the point10:25
noonedeadpunkI'm wondering if octavia that run on host can influence work of the haproxy that is also launched on the same host....10:27
noonedeadpunk(or keepalived)10:27
gthiemongeno, it should not10:28
noonedeadpunkyeah, indeed as it manages things inside netns at least10:29
gthiemongethere are many health check timeouts in haproxy around the same time (~00:25)10:30
gthiemongefor many services10:30
noonedeadpunkyeah and interesting what could cause them... smth like oom maybe....10:35
noonedeadpunkyeah, `Dec  1 20:30:09 aio1 kernel: [ 4265.267970] uwsgi invoked oom-killer` - sorry for bothering :(10:37
noonedeadpunkwas just weird that fails only for debian and at the exactly same tests...10:37
*** sapd1_x has quit IRC10:43
*** ramishra_ has joined #openstack-lbaas10:54
*** ramishra has quit IRC10:57
*** sapd1_x has joined #openstack-lbaas11:15
openstackgerritArkady Shtempler proposed openstack/octavia-tempest-plugin master: New negative test case - test_delete_udp_scenario
*** sapd1_x has quit IRC11:40
*** vesper11 has quit IRC12:53
*** haleyb has joined #openstack-lbaas13:09
*** TrevorV has joined #openstack-lbaas13:16
*** sapd1_x has joined #openstack-lbaas13:24
openstackgerritMerged openstack/octavia stable/train: Fix load balancers with failed amphora failover
*** haleyb has quit IRC13:46
*** spatel has joined #openstack-lbaas13:52
*** wuchunyang has joined #openstack-lbaas13:53
*** spatel has quit IRC13:53
*** haleyb has joined #openstack-lbaas14:13
*** damien_r has joined #openstack-lbaas15:20
*** devfaz has quit IRC15:38
openstackgerritMerged openstack/octavia master: Fix nf_conntrack_buckets sysctl in Amphora
*** devfaz has joined #openstack-lbaas15:40
openstackgerritwu.chunyang proposed openstack/octavia master: Add notifications specification documens
*** sapd1_x has quit IRC15:44
*** wuchunyang has quit IRC15:53
*** ataraday_ has joined #openstack-lbaas15:59
johnsom#startmeeting Octavia16:00
openstackMeeting started Wed Dec  2 16:00:20 2020 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: Octavia)"16:00
openstackThe meeting name has been set to 'octavia'16:00
johnsomHi everyone16:00
johnsomI hope you all enjoyed the break from this meeting. grin16:01
johnsom#topic Announcements16:01
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"16:01
johnsomI don't have any announcements this week. Does anyone else?16:01
johnsom#topic Brief progress reports / bugs needing review16:02
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"16:02
johnsomI have been working on stable branch reviews and poking the tempest patch backlog16:02
johnsomAny other updates this week?16:03
ataraday_I'm a bit overwhelmed with internal tasks. Please review this fix
gthiemongeI've been working on octavia-dashboard, I added configuration for TLS options in pools and listeners16:03
johnsomatarady_ Same here.16:04
johnsomI will take a look at that after the meeting16:04
ataraday_I've got a question about tempest default provider - it set as 'octavia', should it be change to v2 path as well?16:05
ataraday_For testing I changed it to amphora
johnsomYes. We had "octavia" in there to test backward compatibility to when it was called "octavia". That deprecation cycle should be passed now I think, cgoncalves may know better.16:06
johnsomMaybe we should add an upgrade check for that. Or alias "octavia" over to the new driver as well.16:06
johnsomAliasing it over as well might just be simpler.16:08
*** luksky has quit IRC16:08
ataraday_OK, may be add it as part of alias change then16:08
*** damien_r has quit IRC16:08
ataraday_Thank you!16:09
johnsomThat would be my advice to not break people upgrading that haven't switched the config16:09
johnsomCool, any other comments or updates?16:09
johnsom#topic Stable branches16:09
*** openstack changes topic to "Stable branches (Meeting topic: Octavia)"16:09
johnsomPlease help us get the stable branches released. There are many backport patches open for review.16:10
johnsomI have been trying to work on reviews there, please help out.16:10
johnsomWe *really* need to do some stable branch releases.16:10
johnsomAny questions/comments on the stable branches?16:10
gthiemongeack, I'll review them16:10
johnsomThings seem to be merging there, it is just down to getting patches merged.16:11
*** redrobot has quit IRC16:11
johnsommerged/reviewed. lol16:11
johnsom#topic Open Discussion16:11
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"16:11
johnsomAny other topics this week?16:11
johnsomOk, thanks everyone. Have a good week.16:13
*** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list:"16:13
openstackMeeting ended Wed Dec  2 16:13:19 2020 UTC.  Information about MeetBot at . (v 0.1.4)16:13
openstackMinutes (text):
gthiemongethanks johnsom16:13
*** devfaz has quit IRC16:23
*** devfaz has joined #openstack-lbaas16:47
*** ataraday_ has quit IRC16:52
openstackgerritBrian Haley proposed openstack/octavia stable/ussuri: Handle listener not found during loadbalancer status update
*** Guest18921 has joined #openstack-lbaas16:58
*** Guest18921 is now known as redrobot17:01
*** rpittau is now known as rpittau|afk17:04
*** luksky has joined #openstack-lbaas17:18
*** xgerman has joined #openstack-lbaas17:33
*** vishalmanchanda has quit IRC18:58
*** ramishra_ has quit IRC19:00
*** gcheresh has quit IRC19:14
*** ccamposr__ has quit IRC19:20
*** ccamposr__ has joined #openstack-lbaas19:21
*** ccamposr has joined #openstack-lbaas19:25
*** ccamposr__ has quit IRC19:28
*** mchlumsky has quit IRC20:01
*** mchlumsky has joined #openstack-lbaas20:03
*** mchlumsky has quit IRC20:05
*** mchlumsky has joined #openstack-lbaas20:06
*** vesper11 has joined #openstack-lbaas20:15
*** vesper has joined #openstack-lbaas20:19
*** vesper11 has quit IRC20:19
*** vesper has quit IRC21:06
*** gregwork has joined #openstack-lbaas21:09
*** vesper11 has joined #openstack-lbaas21:30
*** TrevorV has quit IRC21:33
*** gcheresh has joined #openstack-lbaas21:53
haleybjohnsom: have a minute for an API question?  I seem to be finding the corner cases trying to get the OVN provider passing tests :(22:03
johnsomhaleyb Sure, go for it22:05
* haleyb reels in johnsom as he took the bait :)22:05
haleybI see a difference in the member_* tests22:06
haleybthe create and update (and delete) tests all create a member, but the update one checks for NO_MONITOR22:06
haleybbut only in the case of the noop driver22:07
haleybsearch above that and see the code is not the same at L91322:08
haleybthe issue is the OVN provider has set CONF.loadbalancer_feature_enabled.health_monitor_enabled=False which causes it to fail that update test because it assumes OFFLINE22:09
haleybthe code was updated in your refactor, it used to look at that config option22:09
johnsomYeah, that if conditional doesn't need to be there. This test suite doesn't create a health monitor on the pool, so it should always be "NO_MONITOR".22:09
johnsomThis junk: CONF.loadbalancer_feature_enabled.health_monitor_enabled just needs to be deleted.22:10
haleybi was just wondering if all those tests should just be the same with the simple22:10
haleyb        self.assertEqual(const.NO_MONITOR, member[const.OPERATING_STATUS])22:10
haleybjohnsom: well, that option is valid, right?  the health_monitor_enabled one22:11
johnsomNo, that "enabled" junk needs to be deleted. It was part of the test cleanup work I was doing. It's all garbage.22:12
haleybthat ^^ ?22:12
johnsomOk, so I see why update is different here:22:12
johnsomhaleyb Yeah, that is all of those junk options that were added for the OVN provider. They all need to go away. Well, all of them with the work "provider" in the description.22:13
johnsomSo, back to the update test.22:13
haleybso it's admin state down, so should transision to OFFLINE22:14
* haleyb didn't noticed that difference22:14
johnsomIt starts in admin state down. This why no-op will behave differently than a live driver. Yeah. Admin down should == OFFLINE per the API.22:15
johnsomDoes that help you out on the NO_MONITOR/OFFLINE/ONLINE scenario?22:16
haleybjohnsom: so it's probably a bug in the OVN driver not looking at that and setting OFFLINE correctly22:16
johnsomOk, yeah, could be. It may not be honoring admin down for the health monitor.22:17
haleybjohnsom: yes, many thanks, even the examples i found showed NO_MONITOR22:17
haleybwell, it doesn't support the health monitor, but i think the default on member create is NO_MONITOR, and each driver needs to change if necessary, like OFFLINE in this case22:18
*** gcheresh has quit IRC22:18
johnsomAs for the "enabled"s... This was the stuff I was working to pull out of the test code. There were still a few places that were using some of those. I had planned to fix those and then push a capstone patch to remove them from the config. But I got sidetracked as you know.22:19
johnsomhaleyb Here is what the initial state will be after a create call via the API:
haleybjohnsom: ack, the OVN driver actually sets a lot of them to False22:20
johnsomYeah, which does strange conditionals in the tests and becomes bad news.22:21
haleybjohnsom: so in the OVN case there is no health monitor, so that's why it's NO_MONITOR (as expected), looks like another patch to the driver to set OFFLINE22:23
johnsomYeah, probably.22:25
haleybwell, i confirmed it ignores admin_state_up in create :( only looks in update22:26
johnsomJob security22:27
*** rcernin has joined #openstack-lbaas22:34
*** tkajinam has joined #openstack-lbaas23:00
*** ccamposr__ has joined #openstack-lbaas23:18
*** ccamposr has quit IRC23:20
*** happyhemant has quit IRC23:54
*** luksky has quit IRC23:57

Generated by 2.17.2 by Marius Gedminas - find it at!