| *** luksky has quit IRC | 00:08 | |
| *** servagem has quit IRC | 00:16 | |
| *** servagem has joined #openstack-lbaas | 00:17 | |
| *** sapd1 has quit IRC | 00:37 | |
| *** zzzeek has quit IRC | 00:38 | |
| *** zzzeek has joined #openstack-lbaas | 00:39 | |
| *** ramishra has quit IRC | 01:14 | |
| *** sapd1 has joined #openstack-lbaas | 01:16 | |
| *** ramishra has joined #openstack-lbaas | 02:32 | |
| *** rcernin has quit IRC | 02:39 | |
| *** rcernin has joined #openstack-lbaas | 02:40 | |
| openstackgerrit | YuehuiLei proposed openstack/octavia master: setup.cfg: Replace dashes with underscores https://review.opendev.org/c/openstack/octavia/+/788124 | 03:00 |
|---|---|---|
| *** xgerman has quit IRC | 03:01 | |
| *** sapd1_x has joined #openstack-lbaas | 04:07 | |
| *** ramishra has quit IRC | 04:25 | |
| *** ramishra has joined #openstack-lbaas | 04:25 | |
| *** vishalmanchanda has joined #openstack-lbaas | 04:50 | |
| *** gcheresh has joined #openstack-lbaas | 05:01 | |
| *** sapd1_x has quit IRC | 05:20 | |
| *** wolsen has joined #openstack-lbaas | 05:23 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Use valid_vip_subnet_tag to select subnets https://review.opendev.org/c/openstack/octavia/+/788133 | 05:50 |
| *** luksky has joined #openstack-lbaas | 05:59 | |
| *** ramishra has quit IRC | 06:58 | |
| *** ramishra has joined #openstack-lbaas | 06:58 | |
| *** damien_r has joined #openstack-lbaas | 07:11 | |
| *** rpittau|afk is now known as rpittau | 07:45 | |
| *** rcernin has quit IRC | 08:08 | |
| *** rcernin has joined #openstack-lbaas | 08:45 | |
| *** sapd1_x has joined #openstack-lbaas | 09:00 | |
| *** QG has joined #openstack-lbaas | 09:02 | |
| *** rcernin has quit IRC | 09:46 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Use valid_vip_subnet_tag to select subnets https://review.opendev.org/c/openstack/octavia/+/788133 | 09:55 |
| rm_work | johnsom: modified the idea we talked about slightly -- allowing it to be overridden by the flavor is a lot better even... I think this should be acceptable? needs testing, but I will wait to do that until we have some basic consensus on whether this approach is good | 10:01 |
| rm_work | cgoncalves / gthiemonge ^^ would like early feedback on whether you think this feature will be acceptable to merge, added a use-case example in the commit message, but I can also talk to it further | 10:01 |
| *** rcernin has joined #openstack-lbaas | 10:04 | |
| *** rcernin has quit IRC | 10:18 | |
| *** rcernin has joined #openstack-lbaas | 10:50 | |
| *** sapd1_x has quit IRC | 11:46 | |
| *** damien_r has quit IRC | 12:12 | |
| *** damien_r has joined #openstack-lbaas | 12:16 | |
| *** tkajinam has quit IRC | 12:18 | |
| *** tkajinam has joined #openstack-lbaas | 12:18 | |
| *** damien_r has quit IRC | 12:22 | |
| *** tkajinam has quit IRC | 12:24 | |
| *** tkajinam has joined #openstack-lbaas | 12:25 | |
| *** damien_r has joined #openstack-lbaas | 12:28 | |
| *** rcernin has quit IRC | 12:41 | |
| *** damien_r has quit IRC | 12:52 | |
| *** damien_r has joined #openstack-lbaas | 13:03 | |
| *** tkajinam has quit IRC | 13:27 | |
| *** tkajinam has joined #openstack-lbaas | 13:27 | |
| *** sapd1_x has joined #openstack-lbaas | 13:30 | |
| openstackgerrit | Vishal Manchanda proposed openstack/octavia-dashboard master: Add horizon-nodejs14-job template. https://review.opendev.org/c/openstack/octavia-dashboard/+/788295 | 13:36 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: Fix task_flow.max_workers with persistence in amphorav2 https://review.opendev.org/c/openstack/octavia/+/788307 | 14:10 |
| *** njohnston has joined #openstack-lbaas | 14:11 | |
| *** ccamposr has quit IRC | 14:32 | |
| *** ccamposr has joined #openstack-lbaas | 14:33 | |
| openstackgerrit | Mohammed Naser proposed openstack/octavia stable/train: Fix empty Batch Member Update to unlock objects https://review.opendev.org/c/openstack/octavia/+/788015 | 14:38 |
| zigo | rm_work: johnsom: I'm getting this in /var/log/syslog of my amphoaes, any idea why ? http://paste.openstack.org/show/804800/ | 15:02 |
| *** sorrison has quit IRC | 15:06 | |
| johnsom | zigo Yes, since you are not using the standard image build tool, I expect you are missing this: https://github.com/openstack/octavia/blob/master/elements/amphora-agent/static/etc/rsyslog.d/55-octavia-socket.conf | 15:26 |
| zigo | johnsom: Thanks. | 15:39 |
| zigo | johnsom: Looks like we don't have a working Victoria amphora yet, but having correct logs will help a lot. | 15:40 |
| zigo | johnsom: I've been asked why I put dkms and acpid in my Octavia agent, and that was the reason why my patch to the debian official images was rejected. Could you explain why it's in amphora-agent/package-installs.yaml ? It doesn't make sense to me either ... | 15:48 |
| zigo | irqbalance is also something to avoid. | 15:48 |
| johnsom | Yeah, all of those are probably not needed in your case. They were copy/pasted over when nodepool made some changes that broke us. | 15:49 |
| johnsom | I agree, I would pull those | 15:49 |
| zigo | Oh ok. | 15:49 |
| johnsom | Just a smudge of tech debt..... lol | 15:50 |
| zigo | ifenslave, haveged, irqbalance, vlan too, IMO... | 15:50 |
| zigo | ethtool ... | 15:50 |
| johnsom | Hmm, ifenslave *might* have a strange dependency from the old ifup/down stuff. | 15:51 |
| johnsom | Please leave ethtool. We have a roadmap to use that for nic IRQ cpu pinning | 15:51 |
| johnsom | It's part of the "improve multi-core performance" plan | 15:52 |
| zigo | Ok. | 15:52 |
| johnsom | haveged, not sure on. If your nova can pass through rng, you don't need it. | 15:52 |
| johnsom | I think we had a user/use case that needed it so it was added. | 15:53 |
| zigo | johnsom: Nobody needs it with fairly recent kernels, that's the point. | 15:53 |
| johnsom | Yeah, that rings a bell too | 15:53 |
| zigo | Recent kernels do have ways to gather (lower quality) entropy (if nothing better is available). | 15:54 |
| zigo | johnsom: Do you guys really need gawk ? Any specifics that's not in mawk ? | 15:54 |
| johnsom | Yeah, I know there have been a lot of changes there since we added haveged. | 15:54 |
| zigo | johnsom: I'm asking all of this because I need the image to be lower than 256 GB, which is the limit of the artifacts in the Debian gitlab. | 15:55 |
| johnsom | I am not sure on that one | 15:55 |
| zigo | So the more I can take away... | 15:55 |
| zigo | ... the better. | 15:55 |
| johnsom | Agreed. I have some notes that some locales and shared-mime-info junk is ending up in some images, that may also be low hanging fruit | 15:55 |
| zigo | I wonder what ipvsadm is for as well ... | 15:55 |
| johnsom | We need that for debugging and some stats stuff for UDP/SCTP protocol support | 15:56 |
| zigo | Ok, thanks. | 15:56 |
| zigo | I'll try the way it is now, at least... :) | 15:56 |
| zigo | Removed: acpid, dkms, haveged, irqbalance, vlan. | 15:57 |
| * zigo crosses fingers so that's enough removal. | 15:57 | |
| johnsom | I don't know if you have options on the number of kernel modules installed, but that was a huge difference on one of the other distros. We don't need the kitchen sink driver if you know what I mean | 15:58 |
| zigo | johnsom: Yeah, there's a cloud kernel in Debian that helps saving space ! :) | 15:58 |
| zigo | johnsom: Is rm /var/lib/octavia/vrrp/check_scripts/haproxy_check_script.sh somehow generated? | 16:02 |
| zigo | What is configuring it? | 16:03 |
| zigo | It "exits 1" and then nothing works. | 16:03 |
| johnsom | The agent manages those as needed | 16:03 |
| zigo | If we remove it, reboot the amphora, then everything goes back to working ... | 16:03 |
| johnsom | There have been some bugs fixes around that over the last release or two. I wonder if you are missing those somehow | 16:04 |
| johnsom | https://review.opendev.org/c/openstack/octavia/+/753189 for example | 16:05 |
| zigo | johnsom: I'm rebuilding an image with the latest Victoria point release (so 17.1.0 instead of 17.0.1, if I'm not mistaking). | 16:05 |
| *** QG has quit IRC | 16:05 | |
| johnsom | https://review.opendev.org/c/openstack/octavia/+/766381 | 16:06 |
| johnsom | https://review.opendev.org/c/openstack/octavia/+/678525 | 16:07 |
| johnsom | That last one is pre-victoria, so I guess doesn't apply here. | 16:07 |
| zigo | I do have the s/sh/bash/ thingy in 17.1.0 :) | 16:10 |
| *** xgerman has joined #openstack-lbaas | 16:17 | |
| *** jamesden_ has joined #openstack-lbaas | 16:21 | |
| *** jamesdenton has quit IRC | 16:22 | |
| zigo | johnsom: When I do, from the amphora: "ip netns exec amphora-haproxy elinks http://<ip-of-my-backend>" | 16:27 |
| zigo | though when I try to do "ip netns exec amphora-haproxy elinks http://<ip-of-the-vip>" it's not working, I wonder why, looks like haproxy is broken or something ... | 16:27 |
| zigo | haproxy runs on the amphora-haproxy netns, right? | 16:28 |
| johnsom | zigo We don't bring up localhost inside the netns, so you can't get to the VIP by default. | 16:28 |
| zigo | Oh... | 16:28 |
| johnsom | Just up the lo interface | 16:29 |
| zigo | Well, my load balancer has an issue anyways... | 16:29 |
| zigo | Not enough, still doesn't work. | 16:29 |
| zigo | I don't get what's going on, because: | 16:30 |
| zigo | ip netns exec amphora-haproxy netstat -anple | grep 80 | 16:30 |
| zigo | shows that haproxy is listening on port 80 in the amphora ... | 16:30 |
| johnsom | Hmm, that should work. You did the ip link up inside the netns right? | 16:30 |
| zigo | Right. | 16:31 |
| zigo | did: ip netns exec amphora-haproxy ifup lo | 16:31 |
| johnsom | Yeah, that usually works for me. | 16:31 |
| zigo | # ip netns exec amphora-haproxy netstat -anple | grep 80 | head -n 1 | 16:32 |
| zigo | tcp 0 0 1.2.3.4:80 0.0.0.0:* LISTEN 0 18080 2096/haproxy | 16:32 |
| zigo | So this looks good. | 16:32 |
| zigo | But then: | 16:32 |
| zigo | ip netns exec amphora-haproxy elinks http://1.2.3.4 <--- No page showing up ... :/ | 16:32 |
| *** rpittau is now known as rpittau|afk | 16:33 | |
| zigo | # ip netns exec amphora-haproxy nmap -p 80 192.168.56.10 | 16:33 |
| zigo | [...] | 16:33 |
| zigo | PORT STATE SERVICE | 16:33 |
| zigo | 80/tcp open http | 16:33 |
| johnsom | Are you running active/standby? Are you on the amp that owns the IP? | 16:33 |
| zigo | Yep, I am. | 16:33 |
| zigo | Oh, maybe the wrong amphora ? | 16:33 |
| zigo | That's possible. | 16:34 |
| zigo | How can I check it owns the IP ? | 16:34 |
| johnsom | ip a | 16:34 |
| zigo | I mean from the amphora ... | 16:34 |
| zigo | Yeah, saw the IP. | 16:34 |
| johnsom | Right, inside the netns, ip a, the VIP address will be a secondary IP on the interface | 16:34 |
| johnsom | If it is there, you are on the right amp | 16:35 |
| zigo | I am in the right amp, ip is showing, and lo is there by default for me. | 16:36 |
| johnsom | https://www.irccloud.com/pastebin/13Sd58W0/ | 16:36 |
| johnsom | Is "lo" up though. | 16:37 |
| johnsom | https://www.irccloud.com/pastebin/9pqo0BcQ/ | 16:37 |
| johnsom | lo will be there, just not "UP" by default | 16:37 |
| zigo | Ah no, it's down ... :) | 16:38 |
| johnsom | Grin, the details in my comments... lol | 16:38 |
| zigo | Oh, indeed, now I got a working http://<ip-of-the-vip> when I try in the amphora. | 16:39 |
| zigo | Reloading the page changes backend, as expected ... | 16:39 |
| zigo | So the issue looks like network related, and NOT Octavia related ... | 16:39 |
| zigo | ie: we're not able to reach the port. | 16:39 |
| zigo | So not your concern, I believe. | 16:40 |
| johnsom | Not a surprise.... | 16:40 |
| zigo | We'll investigate further, thanks a lot. | 16:40 |
| johnsom | o/ | 16:40 |
| rm_work | johnsom: want to discuss that patch we were talking about last night? :D | 17:58 |
| johnsom | I had it on my agenda for a bit later in the day | 17:58 |
| johnsom | Something you want to cover now? | 17:58 |
| * rm_work shrugs | 17:59 | |
| rm_work | i'm here now :D but if you're busy no worries | 17:59 |
| johnsom | Brian gave you some comments already | 18:00 |
| rm_work | had to wake up early for a staff meeting | 18:00 |
| rm_work | ah nice | 18:00 |
| rm_work | haleyb: responded! thanks for your comments | 18:13 |
| *** vishalmanchanda has quit IRC | 18:14 | |
| * haleyb wonders what constitutes early :) | 18:22 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Use valid_vip_subnet_tag to select subnets https://review.opendev.org/c/openstack/octavia/+/788133 | 18:27 |
| rm_work | heh | 18:27 |
| rm_work | fixes applied | 18:27 |
| rm_work | early == before noon PST :P | 18:27 |
| rm_work | or alternatively "before I would otherwise have to wake up" lol | 18:27 |
| *** openstackstatus has quit IRC | 18:31 | |
| *** openstackstatus has joined #openstack-lbaas | 18:33 | |
| *** ChanServ sets mode: +v openstackstatus | 18:33 | |
| *** njohnston has quit IRC | 19:03 | |
| *** rcernin has joined #openstack-lbaas | 20:30 | |
| johnsom | rm_work posted you some comments | 20:47 |
| johnsom | In general I think the path is ok, just needs some polish, tests, docs, check in the mail, etc. | 20:47 |
| johnsom | Just kidding about the check part, but not the tests and docs. | 20:48 |
| johnsom | grin | 20:48 |
| johnsom | Also, it looks like you really want this to merge: https://review.opendev.org/c/openstack/octavia/+/788047 | 20:50 |
| johnsom | hint, hint | 20:50 |
| *** damien_r has quit IRC | 21:01 | |
| *** gcheresh has quit IRC | 21:03 | |
| *** rcernin has quit IRC | 21:22 | |
| *** rcernin has joined #openstack-lbaas | 21:40 | |
| *** rcernin has quit IRC | 21:46 | |
| *** rcernin has joined #openstack-lbaas | 22:00 | |
| *** rcernin has quit IRC | 22:07 | |
| *** rcernin has joined #openstack-lbaas | 22:16 | |
| *** luksky has quit IRC | 22:34 | |
| *** rcernin has quit IRC | 22:47 | |
| *** rcernin has joined #openstack-lbaas | 22:50 | |
| *** rcernin has quit IRC | 22:50 | |
| *** rcernin has joined #openstack-lbaas | 22:51 | |
| rm_work | haleyb: how much are you stuck on the service-type vs tag thing? :D | 23:49 |
| rm_work | johnsom: what should the varname be? `vip_subnet_selection_tag=`? | 23:49 |
| rm_work | haleyb: is it because you think this is *exactly* the intended purpose of the service-type field? :D | 23:51 |
| rm_work | haleyb: `Must be a valid device owner value for a network port` concerns me? otherwise, sure, whatever :D | 23:52 |
| *** rcernin has quit IRC | 23:54 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!