Wednesday, 2019-05-15

*** yamamoto has quit IRC00:19
*** goldyfruit has quit IRC00:26
*** fnaval has joined #openstack-lbaas00:37
*** goldyfruit has joined #openstack-lbaas00:38
openstackgerritMichael Johnson proposed openstack/octavia master: Update hacking version to latest
johnsomThose warnings were starting to bug me....00:39
*** ricolin has joined #openstack-lbaas00:48
openstackgerritMichael Johnson proposed openstack/octavia master: Fix pool API handling of None/null updates
rm_worki also hate warnings00:55
sapd1rm_work, thanks :)00:59
rm_workhaven't been able to test yet though00:59
rm_worki'm still figuring out my new devstack story, with new work laptop and no vmware :/01:00
openstackgerritMerged openstack/octavia master: Performance improvement for non-udp health checks
openstackgerritsapd proposed openstack/octavia master: Support create amphora instance from volume based.
*** rcernin has quit IRC01:16
*** rcernin has joined #openstack-lbaas01:17
*** rcernin has quit IRC01:18
*** rcernin has joined #openstack-lbaas01:19
sapd1rm_work, I don't know Do we need to check cinder's AZs and nova's AZs? If we have to check, so We will check in cinder-driver, it AZ does not match, We will raise Exception, right?01:25
rm_worki'm just struggling with it because it seems weird to require an AZ twice in config, especially if they have to match -- though if it's cinder without nova, then it'd make sense you'd need it in the cinder section... but i actually don't know if it's a separate AZ system or not even01:31
rm_workneed to poke at cinder myself anyway next week01:31
rm_workso i'll try to figure it out01:31
sapd1So we can use AZ config from nova section, It's easier way.01:32
openstackgerritsapd proposed openstack/octavia master: Support create amphora instance from volume based.
*** goldyfruit has quit IRC01:58
*** goldyfruit_ has joined #openstack-lbaas01:59
*** goldyfruit_ has quit IRC02:20
*** hongbin has joined #openstack-lbaas02:39
*** yamamoto has joined #openstack-lbaas03:08
*** yamamoto has quit IRC03:17
*** hongbin has quit IRC03:18
*** gcheresh has joined #openstack-lbaas03:21
openstackgerritErik Olof Gunnar Andersson proposed openstack/octavia stable/stein: Performance improvement for non-udp health checks
openstackgerritErik Olof Gunnar Andersson proposed openstack/octavia stable/rocky: Performance improvement for non-udp health checks
*** gcheresh has quit IRC03:44
*** yamamoto has joined #openstack-lbaas03:54
*** psachin has joined #openstack-lbaas03:56
*** gcheresh has joined #openstack-lbaas04:15
*** yamamoto has quit IRC04:18
*** gcheresh has quit IRC04:20
*** yamamoto has joined #openstack-lbaas04:35
*** yamamoto has quit IRC04:38
*** yamamoto has joined #openstack-lbaas04:38
johnsomeandersson You are on top of that...04:48
*** gthiemonge has quit IRC05:20
*** gthiemonge has joined #openstack-lbaas05:25
*** yamamoto has quit IRC05:39
*** vishalmanchanda has joined #openstack-lbaas05:43
*** yamamoto has joined #openstack-lbaas05:44
*** psachin has quit IRC05:54
*** yboaron_ has joined #openstack-lbaas05:55
*** psachin has joined #openstack-lbaas05:58
*** gthiemonge has quit IRC06:06
*** gthiemonge has joined #openstack-lbaas06:06
*** luksky has joined #openstack-lbaas06:43
*** gcheresh has joined #openstack-lbaas06:47
*** yamamoto has quit IRC06:50
*** gcheresh_ has joined #openstack-lbaas06:51
*** gcheresh has quit IRC06:52
*** yamamoto has joined #openstack-lbaas06:52
*** AlexStaf has joined #openstack-lbaas06:53
*** yamamoto has quit IRC07:00
*** rpittau|afk is now known as rpittau07:07
*** tesseract has joined #openstack-lbaas07:13
*** yamamoto has joined #openstack-lbaas07:16
*** rcernin has quit IRC07:25
*** zigo has quit IRC07:28
*** yamamoto has quit IRC07:38
*** yboaron_ has quit IRC07:39
*** ivve has joined #openstack-lbaas07:54
*** AlexStaf has quit IRC07:55
*** AlexStaf has joined #openstack-lbaas08:11
*** yamamoto has joined #openstack-lbaas08:13
rm_worksapd1 / sapd1_x : well, the issue with using the az config directly from nova is, what happens if it isn't with nova?08:15
rm_workWhich is why I just said I was struggling with it, not that it necessarily needed to go away. I'm not sure what the right answer is there08:15
*** yamamoto has quit IRC08:23
*** fnaval has quit IRC08:23
rm_workTo clarify, I mean, if the cinder driver is used with a new plugin in the future, like whatever we call the container driver08:23
rm_workThere's no guarantee that will have an az concept08:23
rm_workI think separate is probably better, I just am not sure what kind of validation we need to do, if any08:25
rm_workYou may be right that we can just assume the operator will configure it correctly and that's that08:25
rm_workSince it's not something dynamic the user has control over08:25
rm_worknmagnezi: were you working on fixing a bug with member subnet plugging in centos amps?08:30
rm_workI think we're running into that in our deployment08:30
rm_workHmm, meeting in 7.5 hours08:31
nmagnezirm_work, yup. working on it08:32
rm_workWas there a story for it? I forget08:32
rm_workOr have you posted anything yet? I thought I remembered a WIP patch but couldn't find anything today08:33
nmagnezirm_work, I did in the past. But now taking it to a different direction so that patch will look very different08:33
rm_workI THINK it's the same bug08:35
rm_workBut not positive actually...08:35
rm_workIt's like it's not plugging any subnet at all, but I need to ssh in and dig further tomorrow08:35
rm_workHopefully I wake up in time for the meeting ;)08:39
nmagnezirm_work, I know you never actually sleep so I'm not worried :)08:40
*** yamamoto has joined #openstack-lbaas08:43
*** pcaruana has joined #openstack-lbaas08:51
*** luksky has quit IRC09:01
*** yboaron_ has joined #openstack-lbaas09:02
sapd1rm_work, I agree, because normally that value has to right configured09:15
*** zigo has joined #openstack-lbaas09:35
cgoncalvesaaaand gates are broken again. this time it's wsme
*** yamamoto has quit IRC09:41
*** luksky has joined #openstack-lbaas09:43
*** yamamoto has joined #openstack-lbaas09:48
*** yamamoto has quit IRC09:53
openstackgerritGregory Thiemonge proposed openstack/octavia stable/rocky: Treat null admin_state_up as False
openstackgerritGregory Thiemonge proposed openstack/octavia stable/queens: Treat null admin_state_up as False
*** tesseract has quit IRC10:12
*** tesseract has joined #openstack-lbaas10:12
*** zigo has quit IRC10:30
*** zigo has joined #openstack-lbaas10:41
*** goldyfruit_ has joined #openstack-lbaas11:55
*** gcheresh has joined #openstack-lbaas12:04
*** gcheresh_ has quit IRC12:04
*** goldyfruit_ has quit IRC12:19
openstackgerritAdit Sarfaty proposed openstack/octavia master: Remove tests/tempest path from devstack settings and tox
*** goldyfruit_ has joined #openstack-lbaas12:36
*** gcheresh_ has joined #openstack-lbaas12:59
*** gcheresh has quit IRC12:59
h3rbzHi! I'm trying to manualy setup octavia in a dev env in preparation for building our production cloud. As you are probably aware, the openstack docs do not include concrete installation and configuration guides for Octavia. At least not in the way they do for e.g. Cinder or Glance. Can any of you recommend a write-up that covers this in more detail?13:01
h3rbzI was thinking about checking out some implementations behind ansible playbooks, but I'm not very familiar with ansible and actualy having a hard time finding the code behind the playbook stanzas13:04
goldyfruit_We are deploying Octavia with Kolla13:05
goldyfruit_If it could help13:05
*** sapd1_x has quit IRC13:07
h3rbzIt might.. If detailed docs are not available, I would like to see some code that touches config files, installs packages, generates certs etc13:09
h3rbzso I can go form there and figure out what to do in our env13:09
*** kobis1 has joined #openstack-lbaas13:09
johnsomYou can also reference our devstack plugin script13:10
kobis1Hey any clue why tempest still looks for octavia.tests.tempest on master branch, even when I have octavia/master and octavia tempest plugin?13:13
h3rbz@johnsom: devstack/ in openstack/octavia does reveal a lot of the details. Is that the one you are refering to? thx13:14
johnsomrm_work must have missed a tempest config setting in the removal of the v1 code.13:15
johnsomh3rbz: yes, that is the one. It is how we install for the test gates.13:15
cgoncalveskobis1, Adit proposed
h3rbzcool, thx a lot. I'll put on my lab coat and start disecting it ;)13:17
kobis1cgoncalves thanks :)13:18
cgoncalveshah! :D13:19
*** sapd1_x has joined #openstack-lbaas13:20
*** yamamoto has joined #openstack-lbaas13:27
*** vishalmanchanda has quit IRC13:48
*** kobis1 has quit IRC13:49
*** boden has joined #openstack-lbaas14:09
*** Vorrtex has joined #openstack-lbaas14:33
*** luksky has quit IRC14:41
*** AlexStaf has quit IRC14:41
openstackgerritAdit Sarfaty proposed openstack/octavia master: Remove tests/tempest path from devstack and tox
openstackgerritAdit Sarfaty proposed openstack/octavia master: Fix catching driver exceptions
*** vishalmanchanda has joined #openstack-lbaas15:01
*** sahid has joined #openstack-lbaas15:13
sahidhi, wondering whether some cores could have a look at this fix
*** ivve has quit IRC15:15
johnsomsahid It's on my list to look at. Sadly, the list is long....15:21
sahidjohnsom: thanks i appreciate that, let see if we have an other core to look at it so15:22
johnsomThere are six cores....15:23
*** gcheresh_ has quit IRC15:23
*** goldyfruit_ has quit IRC15:24
openstackgerritMichael Johnson proposed openstack/octavia master: Add provider feature support matrix
*** tesseract has quit IRC15:35
*** yboaron_ has quit IRC15:35
*** ataraday_ has joined #openstack-lbaas15:45
rm_work#startmeeting Octavia16:00
openstackMeeting started Wed May 15 16:00:17 2019 UTC and is due to finish in 60 minutes.  The chair is rm_work. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: Octavia)"16:00
openstackThe meeting name has been set to 'octavia'16:00
rm_work#topic Announcements16:01
openstackgerritAnn Taraday proposed openstack/octavia master: [WIP] Jobboard based controller
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"16:01
rm_workNeutron LBaaS is dead! Long live Octavia LBaaS!16:01
rm_workAs a result, Octavia's v1 API is also sleeping with the fishes.16:01
rm_workThere are still some remnants sitting around, so if you see them, please throw up a quick patch to remove them. I see there was some tempest stuff I missed, for example, so thanks to Adit for proposing a patch for that.16:02
johnsomThe big bit bucket in the sky...16:03
rm_workThat's really all I had, anyone else have any announcements?16:04
rm_work#item RIP Neutron-LbaaS16:04
rm_workI wonder if that works...16:05
johnsomSo I noticed you were keeping the stable branches. I think we should talk about that16:05
rm_workI thought we did talk about that?16:05
rm_workThat was the decision I remember from our previous discussions... But, we can totally discuss it again.16:05
rm_workAny other announcements? If not, we can move to that as an agenda item16:06
johnsomWe probably did. I'm just not 100% sure the reason we want to stay on the hook for those16:06
johnsomI don't have anything16:06
*** rpittau is now known as rpittau|afk16:06
rm_work#topic Neutron-LBaaS stable branches16:07
*** openstack changes topic to "Neutron-LBaaS stable branches (Meeting topic: Octavia)"16:07
rm_workSo, you feel like we should kill those too?16:07
cgoncalvesAndreas Jaeger was also asking if we could keep stable branches, right?16:07
cgoncalvesI think we should keep them. why not16:08
rm_workI think we'd end up with a LOT more pushback if we killed the stable branches as well... They were released, and would still be in support for two cycles or whatever, so I don't think we can just wipe those blindly (as much as it'd be nice to)16:08
johnsomWell, it means we have to babysit them.16:08
rm_workPlus, it has allowed me to somewhat assuage people's concerns by pointing to those branches to do builds if they need to16:09
johnsomThe point of the two cycle deprecation was to give warning, etc.16:09
rm_workYes, but they were official releases, and releases have a maintenance cycle....16:09
johnsomSo you are arguing to wait until they go to "Unmaintained" status in releases?16:10
nmagnezio/ (sorry to be late)16:10
cgoncalvescan't we just consider branches immutable/archived from now on?16:10
rm_work(to johnsom)16:10
colin-makes sense to me, fwiw16:10
rm_workcgoncalves: What I mentioned on the ML was that we would not be doing anything with them unless it was absolutely critical (security patches, etc)16:10
colin-(re: unmaintained status)16:11
johnsomRight immutable is what I am advocating for16:11
johnsomImmutable means we don't have to keep the gates running, etc.16:11
cgoncalvesrm_work, +1. queens to stein are deprecated releases. ideally we should even be up to bug fixing until series go EOL16:12
rm_workIt's all there... we may as well just leave it IMO16:12
cgoncalvesbut a compromise could be considering them immutable, i.e. do not delete code in stable branches but stop accepting patches (+ no CI)16:12
rm_workI just don't see a reason to kill it16:13
cgoncalvesremoving code from branches could break CI/CD or whatever other tools people use to pull code16:13
johnsomBTW, photos:16:14
johnsomDarn, should have read my e-mail.  Also, the Shanghai call for presentations is open:16:16
colin-i swear i was smiling16:17
rm_workthe good photo of me is when colin- has his eyes closed... >_>16:17
colin-haha, perfect16:18
rm_workI don't think there was anything else on the agenda officially?16:21
rm_workOh, we skipped this:16:22
rm_work#topic Brief progress reports / bugs needing review16:22
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"16:22
rm_worklots of good stuff showing up now that people are settling in after the PTG :)16:22
johnsomI don't see an agenda...
rm_workme either :D16:22
*** yamamoto has quit IRC16:22
rm_workis that something I'm supposed to be maintaining? ;)16:23
johnsomI have resumed work on fixing the None/null update APIs.  I'm through "pools" now. I will continue to work through those this week.16:23
*** tesseract has joined #openstack-lbaas16:24
*** goldyfruit has joined #openstack-lbaas16:24
johnsomI saw a patch about unsetting the admin_state_up. In general I have not been including boolean settings in the "unset" work.16:24
cgoncalvesI also resume work on VIP ACL (started in Oct-Nov last year and had been on hold ever since). My Python 3.7 patch was rebased to exclude API v1 stuff, and addressed comments in the spare pool tempest patch16:25
ataraday_I switched PoC to use redis.. and discover that it is not working as good as  with zookeper :( Spend some hours debuging - I will do some workarounds to make this work16:26
ataraday_about jobboard taskflow stuff ^^16:26
rm_workjohnsom: i kinda agree, it's up or down....16:26
johnsomataraday_ Ah, that is a bummer. Thanks for the work on that!16:26
cgoncalvesataraday_, what's not working as good as zookeeper? performance or some compatibility?16:27
rm_worki wonder if that warrants revisiting that discussion16:27
johnsomrm_work I guess 500 isn't good either. Maybe we still have some tests to add in addition to what I have been looking at.16:27
ataraday_I just don't resume jobs on restart :)16:27
rm_workwe assumed the choice was not going to affect the usage, I think16:27
ataraday_by default16:27
cgoncalvesok. yeah, thank you a lot for working on jobboard support!16:28
ataraday_it (redis) just don't resume jobs, yes, I was expecting it will work as zookeper, but there are some logic with claiming jobs there16:29
rm_workok, so do we want to follow up with that in...16:32
rm_work#topic Open Discussion16:32
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"16:32
rm_workor is that it?16:32
ataraday_not sure I have a lot to discuss, I hope to make it work this week16:33
ataraday_and than dig into refactor16:33
johnsomI think the intent was to not block the development work. If you got Redis working, great. If not, maybe we just use zookeeper for now and re-address when it comes time to consider etcd, etc.16:34
johnsomI would leave the decision to ataraday_ on the value trade off16:34
johnsomI know we talked about the plans to use Redis for health manager and that some folks have Redis in their clouds already, but I don't want to slow down progress either.16:35
colin-will this become the default implementation?16:37
ataraday_We wanted redis to be default - in ideal world it sould not matter what backend to use16:38
colin-sorry, i should have been more specific. i was referring to the jobboard body of work16:38
colin-my question is if it is merged will it become the default deployment mondel, requiring all users to deploy a zk/redis cluster in support of octavia or not16:38
rm_workah, not sure actually16:39
colin-np, early days still16:39
johnsomIt is being developed as an alternate controller driver, but I expect many will want to run it16:39
ataraday_not it will be experimental for the begining16:39
*** sahid has quit IRC16:39
rm_workit requires more setup, so probably not? since we want to stay as simple as possible (we're already pretty complex to deploy)16:39
johnsomWe also would like to see jobboard support etcd which is an OpenStack "base service", but that is also additional work.16:40
cgoncalvesright. we discussed that at the PTG. at the time the agreement was to make it experimental and thus an offer as alternative to existing model16:40
colin-thanks for the reminder ;)16:40
ataraday_johnsom, you said on last meeting you will do one of refactoring change as an example - will you have time for this?16:42
johnsomataraday_ Ah, yes, sorry, it dropped off my radar. I will work on that this week as to not block work for others.16:43
ataraday_johnsom, great, thanks!16:44
rm_workcool, ok16:48
rm_workSeems like maybe we're done?16:48
rm_workand I can go back to sleep \o/16:48
johnsomI don't have anything else16:49
colin-that's all for me16:49
rm_workAlright, thanks for coming!16:49
*** openstack changes topic to "Discussions for OpenStack Octavia | Train PTG etherpad:"16:49
openstackMeeting ended Wed May 15 16:49:36 2019 UTC.  Information about MeetBot at . (v 0.1.4)16:49
openstackMinutes (text):
* rm_work collapses16:49
*** ataraday_ has quit IRC17:04
*** ricolin has quit IRC17:07
*** luksky has joined #openstack-lbaas17:10
*** gthiemonge has quit IRC17:14
*** psachin has quit IRC17:15
*** AlexStaf has joined #openstack-lbaas17:25
*** yamamoto has joined #openstack-lbaas17:41
*** AlexStaf has quit IRC17:43
*** yamamoto has quit IRC17:50
*** goldyfruit has quit IRC17:59
openstackgerritMichael Johnson proposed openstack/octavia master: Rename to
*** tesseract has quit IRC18:01
*** goldyfruit has joined #openstack-lbaas18:23
*** luksky has quit IRC18:32
*** boden has quit IRC18:41
*** ianychoi has quit IRC18:42
rm_workjohnsom: did we not merge the thing with setting a custom amp user? or at least normalizing the username?18:56
johnsomI don't think we did. I remember the discussion, but not an outcome.18:57
*** vishalmanchanda has quit IRC19:01
*** boden has joined #openstack-lbaas19:01
rm_workjohnsom: it looks like member networks don't happen *at all* on centos19:23
rm_workdoesn't matter if the network is different O_o19:23
johnsomSay what?19:23
rm_workso, created a LB on networkA19:23
rm_workadded members on networkB (subnet B)19:24
johnsomI know the centos gate has been broken, but I thought it was just some timeout thing19:24
rm_workinside the netns there's still only networkA19:24
rm_worklooking at the agent log, i don't even see any request to plug that19:24
rm_workwhich lines up with what nir was saying to me last night19:24
johnsomWell, I can tell you it works on Ubuntu....19:25
johnsomI just stacked and created an LB that works fine.19:25
rm_workit's supposed to trigger something on the amp right?19:25
*** ivve has joined #openstack-lbaas19:25
rm_worklike, it hits the agent and says "plug this member subnet"19:25
johnsomIt's a pretty simple flow19:26
johnsom"AmphoraPostNetworkPlug is what sets up the IP inside the amp19:26
johnsomrm_work Are you on this or should I go poke at a centos amp?19:29
rm_workI'm looking at it19:29
rm_worki'll let you know if i need help19:29
openstackgerritMerged openstack/octavia master: Update hacking version to latest
rm_workah ok well19:41
rm_worki'm fixing it in a different way...19:41
* rm_work just merged a change to switch the amp image build to ubuntu19:42
* rm_work shrugs19:42
*** henriqueof has joined #openstack-lbaas19:45
openstackgerritMerged openstack/octavia master: Make sure amphora logging works the same on py2 and py3
johnsomrm_work Is that a "I'm no longer looking at fixing centos" kind of statement?19:47
rm_workyeah, pretty much19:47
rm_workwe were just using it because that's what people used elsewhere so someone thought it made sense to use it here19:48
rm_workbut there's no reason we have to (I asked)19:48
rm_workand it's never been anything but problematic for me either here or at GD :/19:48
johnsomOk, I should probably look at it then19:48
rm_workwhereas ubuntu typically just works19:48
rm_workbut it's not explicitly clear to me why this would be a centos issue19:49
rm_worki'm tracing code and i don't see where it'd do the config for the new network19:49
rm_workon create member, the only thing that happens is a listener update19:49
rm_worki mean, for the amp19:50
rm_workit calculates the network delta and does neutron plugs/unplugs19:50
rm_workand that does happen correctly in our deployment too (I see the correct new port is on the amp)19:50
rm_workbut i don't see how the agent would be triggered to bring it up in the netns19:50
johnsomWorks for me, so... What is different in you case20:02
rm_worknot sure20:05
rm_workwhere in the code does it write those configs out?20:05
rm_workis there an amp agent log entry for doing a plug?20:05
*** Vorrtex has quit IRC20:05
rm_worki don't see anything in member flows20:07
rm_workoh nm there it is, i'm blind20:08
rm_workwell, that never ran20:08
rm_workbut the networks are different20:08
rm_workbut we're on rocky?20:09
rm_workwe'll be on master shortly20:09
rm_workcan try again20:09
johnsomOk, I'm on master. Grabbed the test image.20:11
johnsomSomething is odd with your neutron network/subnet setup would be my guess.20:11
rm_workwe do use network-id only when creating LBs20:12
rm_work(seems like that follows me around)20:12
johnsomWould be interested in a network show and subnet show20:12
rm_workyeah I can prolly paste you stuff in PM20:12
johnsomIs the member subnet on the same network as the VIP?20:13
*** gthiemonge has joined #openstack-lbaas20:13
openstackgerritMerged openstack/octavia master: Remove tests/tempest path from devstack and tox
*** pcaruana has quit IRC20:14
*** gthiemonge has quit IRC20:31
*** gthiemonge has joined #openstack-lbaas20:58
*** henriqueof has quit IRC21:08
*** boden has quit IRC21:42
*** yamamoto has joined #openstack-lbaas21:49
*** yamamoto has quit IRC21:54
johnsommystery solved, the lb-mgmt-net subnet was the same as the member subnet, so yeah, it won't get plugged. There might be an open story about this already.22:46
*** kenaz has joined #openstack-lbaas22:49
*** kenaz has quit IRC22:55
*** kenaz has joined #openstack-lbaas23:00
*** kenaz has quit IRC23:13
*** rcernin has joined #openstack-lbaas23:14
*** goldyfruit has quit IRC23:22
*** yamamoto has joined #openstack-lbaas23:26
*** goldyfruit has joined #openstack-lbaas23:50
*** sapd1_x has quit IRC23:55

Generated by 2.15.3 by Marius Gedminas - find it at!