| *** spatel has quit IRC | 00:04 | |
| *** hongbin has quit IRC | 00:10 | |
| johnsom | Just a reminder, we are cutting the RC1 in a few days. There are still 30 patches on the priority review list. Some have had -1 for a few weeks. Please review your patches on the priority list and either move them to post-Victoria list or try to resolve the -1 comments. | 00:18 |
|---|---|---|
| *** sapd1 has quit IRC | 00:38 | |
| *** armax has quit IRC | 00:50 | |
| *** sapd1 has joined #openstack-lbaas | 02:09 | |
| openstackgerrit | Merged openstack/octavia-tempest-plugin master: Add ALPN protocol scenario tests https://review.opendev.org/746737 | 02:18 |
| openstackgerrit | Merged openstack/octavia master: Fix the tls_cipher_prohibit_list release note https://review.opendev.org/750518 | 02:18 |
| *** ramishra has quit IRC | 02:29 | |
| openstackgerrit | MaAoyu proposed openstack/octavia master: Remove install unnecessary packages https://review.opendev.org/753202 | 02:36 |
| openstackgerrit | MaAoyu proposed openstack/octavia master: Remove install unnecessary packages https://review.opendev.org/753202 | 02:38 |
| *** spatel has joined #openstack-lbaas | 02:50 | |
| *** zzzeek has quit IRC | 02:51 | |
| *** zzzeek has joined #openstack-lbaas | 02:54 | |
| *** spatel has quit IRC | 02:55 | |
| *** hongbin has joined #openstack-lbaas | 03:00 | |
| *** spatel has joined #openstack-lbaas | 03:01 | |
| *** spatel has quit IRC | 03:06 | |
| *** ramishra has joined #openstack-lbaas | 03:15 | |
| *** ramishra has quit IRC | 03:48 | |
| *** ramishra has joined #openstack-lbaas | 03:55 | |
| openstackgerrit | Merged openstack/octavia master: Fix AttributeError on TLS-enabled pool provisioning https://review.opendev.org/752239 | 04:11 |
| *** weiguo has quit IRC | 04:27 | |
| *** rcernin has quit IRC | 05:14 | |
| *** gcheresh has joined #openstack-lbaas | 05:19 | |
| *** hongbin has quit IRC | 05:26 | |
| *** servagem has quit IRC | 05:27 | |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia-tempest-plugin master: Add act-stdby scenario jobs to the gate https://review.opendev.org/742385 | 05:30 |
| *** rcernin has joined #openstack-lbaas | 05:34 | |
| *** servagem has joined #openstack-lbaas | 05:52 | |
| *** zzzeek has quit IRC | 05:54 | |
| *** zzzeek has joined #openstack-lbaas | 05:54 | |
| *** vishalmanchanda has joined #openstack-lbaas | 05:58 | |
| *** zzzeek has quit IRC | 06:26 | |
| *** zzzeek has joined #openstack-lbaas | 06:28 | |
| *** TMM has quit IRC | 06:30 | |
| *** TMM has joined #openstack-lbaas | 06:30 | |
| *** BlackFX has quit IRC | 06:53 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 06:57 | |
| *** rcernin has quit IRC | 07:13 | |
| *** spatel has joined #openstack-lbaas | 07:29 | |
| *** spatel has quit IRC | 07:34 | |
| *** AlexStaf has joined #openstack-lbaas | 07:34 | |
| *** rcernin has joined #openstack-lbaas | 07:40 | |
| *** ccamposr__ has joined #openstack-lbaas | 07:45 | |
| *** ccamposr has quit IRC | 07:47 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: Add SCTP support in Amphora https://review.opendev.org/753247 | 08:29 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-tempest-plugin master: Add SCTP protocol scenario tests https://review.opendev.org/738643 | 08:31 |
| ataraday | gthiemonge, Hi! Could you review https://review.opendev.org/#/c/738609/ ? | 08:51 |
| *** rcernin has quit IRC | 08:59 | |
| *** rcernin has joined #openstack-lbaas | 09:00 | |
| gthiemonge | ataraday: Hi, done! | 09:17 |
| *** rcernin has quit IRC | 09:19 | |
| openstackgerrit | Merged openstack/octavia master: Fixes API list handling of unscoped tokens https://review.opendev.org/753169 | 09:28 |
| dulek | cgoncalves: Hello! Quick question about allowed_cidrs on listeners - what's the "default" mentioned on the API reference? Is that None? Or empty list? | 09:51 |
| dulek | "A list of IPv4, IPv6 or mix of both CIDRs. The default is all allowed. When a list of CIDRs is provided, the default switches to deny all." | 09:51 |
| dulek | Or code just assumes the default is when that property is not sent at all with the request? | 09:51 |
| dulek | Ah, maybe some context… So this change broke our ovn-provider gate https://opendev.org/openstack/ovn-octavia-provider/src/commit/bfd98048cfabaa66ca8a61047b642e8bf446490c/ovn_octavia_provider/driver.py#L142. | 09:53 |
| dulek | Basically seems like openstacksdk always fills that option with an empty list even if we don't specify it. | 09:53 |
| dulek | I'm trying to figure out how it should really work. | 09:53 |
| cgoncalves | dulek, default is None | 09:54 |
| dulek | Alright, so I just need to make openstacksdk put None as a default, cool. Thanks! | 09:55 |
| openstackgerrit | Merged openstack/octavia-tempest-plugin master: Add HTTP/2 tempest scenario tests for listeners https://review.opendev.org/747959 | 10:17 |
| openstackgerrit | Merged openstack/octavia stable/ussuri: Fix amphora failover when VRRP port is missing https://review.opendev.org/753190 | 10:17 |
| *** spatel has joined #openstack-lbaas | 10:21 | |
| *** spatel has quit IRC | 10:25 | |
| dulek | haleyb: Hi there! A while ago you've introduced validating allowed_cidrs in ovn-octavia-provider. So apparently that's broken - it'll fail listener creation even if I won't set the option as seen on this log: http://paste.openstack.org/show/798197/ | 10:34 |
| *** ccamposr has joined #openstack-lbaas | 10:34 | |
| dulek | The listener request is '{"listener": {"loadbalancer_id": "faca9a1b-30dc-45cb-80ce-2ab1c26b5521", "protocol": "TCP", "protocol_port": 80, "admin_state_up": true}}', yet it fails with 501. | 10:34 |
| *** rcernin has joined #openstack-lbaas | 10:37 | |
| *** ccamposr__ has quit IRC | 10:37 | |
| *** rcernin has quit IRC | 10:37 | |
| *** rcernin has joined #openstack-lbaas | 10:37 | |
| dulek | Oh my, this happens all over ovn-octavia-provider tests as well, just all of them are skipped. :D | 10:38 |
| *** ccamposr__ has joined #openstack-lbaas | 11:00 | |
| *** ccamposr has quit IRC | 11:03 | |
| *** rcernin has quit IRC | 11:11 | |
| *** gcheresh has quit IRC | 11:14 | |
| *** gcheresh has joined #openstack-lbaas | 11:44 | |
| *** gcheresh has quit IRC | 12:13 | |
| haleyb | dulek: sigh, we can revert and just blacklist the test(s) | 12:14 |
| dulek | haleyb: Well, fixing the check works for me too. ;) It's probably comparing the value incorrectly, I printed it and apparently it's an empty list at that point. | 12:15 |
| haleyb | and i thought the gate being green was good :( | 12:16 |
| dulek | haleyb: Yeah, I'm not sure why so many tests are skipped there. | 12:17 |
| haleyb | we should only be skipping the ones with that option set, until we add support | 12:17 |
| *** sapd1 has quit IRC | 12:29 | |
| haleyb | dulek: https://review.opendev.org/#/c/753302/ - i just haven't verified on a running stack yet | 12:37 |
| dulek | haleyb: Yep, that should do the thing, thanks! | 12:39 |
| dulek | Let me try running tests based on that… | 12:39 |
| haleyb | dulek: thanks, that was my fault trying to get the gate green | 12:40 |
| dulek | Kuryr tests are running in https://review.opendev.org/#/c/753303/, let's see. | 12:41 |
| *** gcheresh has joined #openstack-lbaas | 12:41 | |
| *** rcernin has joined #openstack-lbaas | 12:55 | |
| *** rcernin has quit IRC | 13:10 | |
| *** sapd1 has joined #openstack-lbaas | 13:49 | |
| openstackgerrit | Merged openstack/octavia stable/stein: Fix accepting 'insert_headers' when unsupported https://review.opendev.org/746807 | 13:52 |
| *** TrevorV has joined #openstack-lbaas | 13:54 | |
| *** armax has joined #openstack-lbaas | 14:26 | |
| dulek | haleyb: The Kuryr tests I mentioned above failed, but it's because I made a typo in Depends-On. Checking again. | 15:00 |
| haleyb | dulek: there might be another issue as my patch is failing on another test, tracking that down now too | 15:01 |
| *** rcernin has joined #openstack-lbaas | 15:06 | |
| *** rcernin has quit IRC | 15:10 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia stable/ussuri: Fixes API list handling of unscoped tokens https://review.opendev.org/753382 | 15:33 |
| openstackgerrit | Michael Johnson proposed openstack/octavia stable/train: Fixes API list handling of unscoped tokens https://review.opendev.org/753383 | 15:33 |
| openstackgerrit | Michael Johnson proposed openstack/octavia stable/stein: Fixes API list handling of unscoped tokens https://review.opendev.org/753384 | 15:33 |
| *** gcheresh has quit IRC | 15:40 | |
| *** gcheresh has joined #openstack-lbaas | 16:07 | |
| *** irclogbot_0 has quit IRC | 16:08 | |
| *** irclogbot_2 has joined #openstack-lbaas | 16:09 | |
| *** dosaboy has quit IRC | 16:09 | |
| *** sapd1 has quit IRC | 16:11 | |
| *** maciejjozefczyk_ has joined #openstack-lbaas | 16:11 | |
| *** dosaboy has joined #openstack-lbaas | 16:13 | |
| *** maciejjozefczyk has quit IRC | 16:14 | |
| *** sapd1 has joined #openstack-lbaas | 16:15 | |
| *** redrobot has quit IRC | 16:25 | |
| *** johnsom has quit IRC | 16:26 | |
| *** gcheresh has quit IRC | 16:26 | |
| *** johnsom has joined #openstack-lbaas | 16:27 | |
| *** ccamposr has joined #openstack-lbaas | 17:08 | |
| *** ccamposr__ has quit IRC | 17:10 | |
| *** sapd1 has quit IRC | 17:34 | |
| *** ramishra has quit IRC | 17:42 | |
| *** maciejjozefczyk_ has quit IRC | 17:52 | |
| openstackgerrit | Merged openstack/octavia master: Fix backend certificate file paths https://review.opendev.org/752428 | 17:54 |
| *** gcheresh has joined #openstack-lbaas | 17:57 | |
| *** rouk has joined #openstack-lbaas | 18:09 | |
| rouk | @johnsom is there a hook somewhere to recreae the octavia-lb port? i had some tenant fire off 2 creates using a manually selected ip at the exact same time (lol kubernetes) and had 1 lb create stuck in early stages, didnt know this was the case, so i dumped the port to remove the conflict as i didnt notice 2 LBs were trying to create for a month | 18:12 |
| rouk | so now the stuck one created, and we cleanly deleted it, but i now have the correct LB with no octavia-lb port | 18:13 |
| johnsom | rouk if you have the new failover code, it will rebuild it on load balancer failover (not amphora failover) | 18:14 |
| rouk | new being... train? | 18:14 |
| johnsom | stable/train master has it. It hasn't been released packaged yet | 18:15 |
| rouk | aw | 18:15 |
| johnsom | https://docs.openstack.org/releasenotes/octavia/train.html | 18:15 |
| johnsom | I would guess we can release train in the next week or two. | 18:16 |
| rouk | so how can i create a port by hand to hold the ip reservation so octavia can manage it? | 18:16 |
| rouk | since, im in a good state now, just things could steal the ip | 18:16 |
| johnsom | Well, if you already deleted the port it is at risk of being used. | 18:16 |
| johnsom | Right, | 18:17 |
| johnsom | Which port exactly is it? VIP or VRRP? | 18:17 |
| rouk | octavia-lb, not octavia-lb-vrrp | 18:17 |
| johnsom | Ugh, ok, that is the harder one. | 18:17 |
| rouk | but i deleted it cause i guess octavia doesnt atomically check static ip reserveration? | 18:17 |
| johnsom | Any chance you can just delete the LB and re-create it with the ip address specified? | 18:18 |
| rouk | and allows 2 lbs to be requested on the same ip | 18:18 |
| johnsom | Or create a parallel LB? | 18:18 |
| rouk | id prefer not to make the tenant do that, but i guess i could... | 18:18 |
| johnsom | It doesn't allow it. When the second port is created the LB will go into error | 18:18 |
| rouk | was hoping i could just craft a port octavia would be happy about, since its just holding ip reservation | 18:18 |
| rouk | oh but it did allow it | 18:19 |
| rouk | i had 2 LBs come up on the same ip | 18:19 |
| rouk | thats why i deleted the first port, didnt know one was queued on the same ip | 18:19 |
| johnsom | On the same subnet? neutron should not allow that. | 18:19 |
| rouk | but neutron is spoken to later | 18:19 |
| rouk | 2 LBs were requested miliseconds apart | 18:20 |
| rouk | with the same ip | 18:20 |
| johnsom | Right, the second LB should go into error as neutron will reject the port create | 18:20 |
| rouk | first one came up, 2nd one got stuck till i freed up the port | 18:20 |
| rouk | never came up on the tenant's LB list. | 18:20 |
| rouk | suddenly appeared once i unstuck it by deleting the port | 18:20 |
| johnsom | Well, as soon as the command returns it's in their list. Something doesn't line up. | 18:21 |
| johnsom | Anyway, ummm. We can try to work through creating a port and getting things lined back up, but you have to deal with security groups, allowed address pairs ports, etc. | 18:22 |
| rouk | not sure, i dont have the script that made it, it was done by some abuse of kubernetes CNI | 18:22 |
| rouk | well... its a port that sits detached, it needs secgroups? | 18:22 |
| johnsom | Yeah, terraform can do dumb things | 18:23 |
| rouk | we have terraform abuse too, but this was just pure kubernetes octavia integration | 18:23 |
| rouk | im just saying what i saw, and that there might be a race condition, i dont advocate that my users use things correctly or sanely. | 18:24 |
| rouk | i dont have these problems, they find every bug under the sun somehow by flailing around. | 18:24 |
| johnsom | Yeah, I just know this was a tested scenario and neutron did the right thing. Because we also had the high rate of change k8s stuff | 18:25 |
| rouk | maybe k8s had it queued. | 18:25 |
| *** gcheresh has quit IRC | 18:25 | |
| rouk | and it was just being replayed constantly without the user knowing. | 18:25 |
| johnsom | Give me a minute to switch context here and look at what all we would need to handle. | 18:25 |
| rouk | yeah, if remaking the port is horrible ill ask them to remake the lb, just rather avoid it if i can just remake it in 1 command where octavia will be happy. | 18:26 |
| johnsom | Ha, well, happy enough to limp along until you get the new failover that repairs broken vip ports | 18:27 |
| johnsom | Ok, SINGLE or ACT/STDBY? | 18:27 |
| rouk | active standby | 18:27 |
| johnsom | Ok, can you "openstack port show" both of the octavia-lb-vrrp ports and validate that they have the right allowed address pairs configured? | 18:28 |
| rouk | yep they do. | 18:29 |
| rouk | i also have another port in the same project for another lb as a reference. | 18:29 |
| johnsom | Ok, take note of the security group on those two VRRP ports. We will need it on the new port | 18:30 |
| rouk | why? if the port sits detached? or is octavia using it as a reference for something | 18:30 |
| johnsom | There is a strange relationship with the allowed address pairs and that port. We always keep them in sync to make sure the rules stay applied | 18:30 |
| rouk | ah | 18:31 |
| johnsom | Ok, next step is to get the octavia service account credentials so you can run the openstack command under your octavia project | 18:32 |
| johnsom | It is the service_auth section in your octavia.conf. | 18:32 |
| johnsom | Do you know how to do that? | 18:32 |
| rouk | i cant just admin myself into the project? | 18:32 |
| rouk | i can get the creds, yeah. | 18:33 |
| johnsom | Well, however you do it, the port should be owned by the octavia account | 18:33 |
| rouk | octavia project? or does the userid have to be octavia as well on the port? | 18:34 |
| johnsom | Technically users can bring a port, but it's best when we are doing this to have it under the right account | 18:34 |
| johnsom | project_id needs to be in this case. We will aslo set the device owner to a Octavia | 18:35 |
| rouk | ah ports dont even have a userid field | 18:35 |
| rouk | so yeah im good, got secgroup id, and can set --project. | 18:35 |
| johnsom | Ok. In your octavia database: "select octavia_owned where load_balanacer_id = "<LB UUID>";" | 18:36 |
| johnsom | Is that 1 or 0? | 18:36 |
| johnsom | Actually we should just select the whole data to make sure there is no qos_policy | 18:37 |
| rouk | user is agreeing to remake it | 18:39 |
| johnsom | lol | 18:39 |
| johnsom | So close | 18:39 |
| johnsom | lol | 18:39 |
| johnsom | So, in the future, if you see something like this, please collect the worker logs for the conflicting elements. Maybe neutron API is no longer concurrently safe or something. | 18:41 |
| rouk | well, considering that this is kubernetes, and kubernetes likes to infinitely retry things | 18:41 |
| johnsom | We expect neutron to raise a "conflict" error on the second port create call. | 18:42 |
| rouk | i bet they had one sitting in the pipe stuck on the kube side | 18:42 |
| rouk | it was. | 18:42 |
| rouk | i didnt notice that the error was coming from a create command for a different lb | 18:42 |
| rouk | cause the user claimed they didnt have another create happening | 18:44 |
| rouk | cause they have no idea what theyre doing | 18:44 |
| johnsom | So k8s should go well for them then... lol Well, good luck to you all! | 18:45 |
| rouk | yep... im always the one with problems | 18:45 |
| johnsom | Keep an eye out for a new trail bug fix release in the next few weeks. | 18:45 |
| rouk | yeah i need to do a ussuri upgrade in the next couple weeks | 18:46 |
| rouk | so ill get the update then | 18:46 |
| johnsom | Ok, good plan. We will also release a bug fix version for that soon. | 18:46 |
| johnsom | We are focused on getting Victoria out the door this week. | 18:47 |
| rouk | sweet | 18:47 |
| rouk | im stuck with punting users off fwaas before i can get to V | 18:49 |
| rouk | cause that ship sunk | 18:49 |
| rouk | so one day ill join you | 18:49 |
| johnsom | Yeah, sad. I think it has a place | 18:49 |
| rouk | we used it as a central authority per project for knowing holes | 18:50 |
| rouk | so we didnt have to make come big red twine conspiracy theory pegboard to map out our network holes | 18:50 |
| rouk | but, i wrote some policy verification system for securty groups... fills our auditing needs i guess. | 18:51 |
| *** vishalmanchanda has quit IRC | 18:58 | |
| *** zzzeek has quit IRC | 19:08 | |
| *** zzzeek has joined #openstack-lbaas | 19:09 | |
| *** ccamposr__ has joined #openstack-lbaas | 20:20 | |
| *** ccamposr has quit IRC | 20:22 | |
| *** ianychoi_ has joined #openstack-lbaas | 20:23 | |
| *** ianychoi has quit IRC | 20:27 | |
| *** maciejjozefczyk_ has joined #openstack-lbaas | 20:31 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 20:33 | |
| *** maciejjozefczyk_ has quit IRC | 20:35 | |
| *** gcheresh has joined #openstack-lbaas | 20:44 | |
| *** gcheresh has quit IRC | 20:56 | |
| *** rouk has quit IRC | 21:27 | |
| *** ccamposr has joined #openstack-lbaas | 21:28 | |
| *** ccamposr__ has quit IRC | 21:32 | |
| *** maciejjozefczyk has quit IRC | 21:44 | |
| *** TrevorV has quit IRC | 21:48 | |
| *** spatel has joined #openstack-lbaas | 21:59 | |
| *** spatel has quit IRC | 21:59 | |
| *** gmann has quit IRC | 22:04 | |
| *** gregwork has quit IRC | 22:04 | |
| *** andrein has quit IRC | 22:04 | |
| *** aannuusshhkkaa has quit IRC | 22:04 | |
| *** nicolasbock has quit IRC | 22:04 | |
| *** andrein has joined #openstack-lbaas | 22:05 | |
| *** gmann has joined #openstack-lbaas | 22:05 | |
| *** nicolasbock has joined #openstack-lbaas | 22:05 | |
| *** aannuusshhkkaa has joined #openstack-lbaas | 22:06 | |
| *** mnaser has quit IRC | 22:06 | |
| *** gregwork has joined #openstack-lbaas | 22:08 | |
| *** mnaser has joined #openstack-lbaas | 22:10 | |
| *** xgerman has joined #openstack-lbaas | 22:21 | |
| *** servagem has quit IRC | 22:47 | |
| *** servagem has joined #openstack-lbaas | 22:48 | |
| *** rcernin has joined #openstack-lbaas | 22:55 | |
| *** tkajinam has joined #openstack-lbaas | 22:57 | |
| *** servagem has quit IRC | 23:00 | |
| *** AlexStaf has quit IRC | 23:16 | |
| *** AlexStaf has joined #openstack-lbaas | 23:16 | |
| *** servagem has joined #openstack-lbaas | 23:22 | |
| *** zzzeek has quit IRC | 23:30 | |
| *** zzzeek has joined #openstack-lbaas | 23:31 | |
| *** spatel has joined #openstack-lbaas | 23:44 | |
| *** spatel has quit IRC | 23:49 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!