Wednesday, 2023-05-31

opendevreviewNeil Hanlon proposed openstack/openstack-ansible-rabbitmq_server master: Use el/9 repo urls instead of el/8 since a repo is available for it  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/88471900:53
opendevreviewNeil Hanlon proposed openstack/openstack-ansible-rabbitmq_server master: bump rabbitmq from 3.11.10-1 to 3.11.13-1  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/88478200:55
opendevreviewNeil Hanlon proposed openstack/openstack-ansible-rabbitmq_server master: Use el/9 repo urls instead of el/8 since a repo is available for it  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/88471901:10
jrossergood morning08:05
admin1good morning 08:27
noonedeadpunko/10:09
noonedeadpunkI kinda wonder if it makes sense to switch repo to el9 at all, given it contains still el8 packages, but not all of them10:11
derekokeeffe85jrosser noonedeadpunk, thanks a million for the advice I now have three vlan internet accessible networks and my tenant networks are back working!!! The last part I need to solve now is the neutron routers, I can do that in my setup as well?10:19
noonedeadpunkI think yes, why not?10:20
noonedeadpunkgiven you have all networks required on net nodes - they should "just work"10:21
derekokeeffe85They did work before but now when I go to the Routers tab I get "Unable to retrieve router list" similar error when trying to create one. I did have to comment out this line service_plugins = ovn-router in my neutron.conf file on the neutron container as the horizon dashboard won't load with that in place so that's probably the issue I'm having I guess noonedeadpunk10:25
noonedeadpunkOh, well, if you've disabled functionality, no wonder they're not available :D10:28
derekokeeffe85That is true :) Changing it from ovn-router to just router allowed me create one. Is there anything wrong with doing that?10:30
noonedeadpunkwhat driver are you using? ml2.ovn or ml2.ovs?10:31
derekokeeffe85ovs10:31
noonedeadpunkfor ovs usually we have `neutron_plugin_base: [router, metering]`10:32
noonedeadpunkas a default10:32
derekokeeffe85Ok perfect, thanks for that. Think I'm in a good place now to do a bit of testing and make sure to backup all the configs, thanks again to you both10:33
opendevreviewMerged openstack/openstack-ansible master: Bump SHAs for Antelope RC2  https://review.opendev.org/c/openstack/openstack-ansible/+/88476010:33
noonedeadpunkno worries! 10:49
noonedeadpunkfeel free to reach out :)10:49
noonedeadpunkoh damn....10:50
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Bump SHAs for Antelope RC2  https://review.opendev.org/c/openstack/openstack-ansible/+/88481610:51
jrosser /o\ doh10:51
noonedeadpunkI should have merged bump for 2023.1 branch already10:51
noonedeadpunk*pushed10:51
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Implement support for haproxy_accept_both_protocols  https://review.opendev.org/c/openstack/openstack-ansible/+/88481710:53
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Fix repo url in healthcheck-infrastructure.yml  https://review.opendev.org/c/openstack/openstack-ansible/+/88481810:53
noonedeadpunkI wonder what else could slip... damiandabrowski can you check that your recent patches ended up in stable/2023.1 ?11:00
damiandabrowskilooks like you cherry picked everything, thanks!11:05
noonedeadpunkdamn, I wonder if releases repo will complain about cherry-picking 884760.. As commit SHA would be the same...11:16
noonedeadpunkI guess we'll see11:16
NeilHanlonnoonedeadpunk: yeah i am kinda wondering the same re: if it's worth changing to el9 at all considering11:32
mgariepygood morning everyone i'm back.11:53
noonedeadpunkmgariepy: you wasn't back for too long lol12:59
mgariepyhaha13:01
mgariepywell i was off since may 11th.13:01
mgariepydoing construction work.13:01
noonedeadpunkoh, don't say you've finished it13:11
noonedeadpunkAs construction can only be started and never finished13:11
noonedeadpunkPaused at best13:12
mgariepythe exterior is done ;) 13:20
mgariepyhaha13:20
mgariepyneed to fix 2 windows and 2 doors interior tho.13:21
mgariepy* at some point in the future.. haha13:21
derekokeeffe85Neutron routers working, thanks noonedeadpunk13:42
damiandabrowskigood to have you back mgariepy !13:48
derekokeeffe85Well I'm a carpenter by trade originally so any questions around that I should be the man haha :)14:15
mgariepydid try carpentry for a couple of weeks lately .. i perfer sysadmin stuff full time than carpentry :D14:17
derekokeeffe85I do like the sysadmin stuff but I know mostly everything in carpentry which can't be said for this game lol14:19
mgariepyquite the opposite for me. but i did get comparared to a 2-3 year expericenced carpenter. last week :D14:23
derekokeeffe85Nice compliment to get, nice to see you back anyway14:26
mgariepyi really don't like being over more than 2 scaffold. haha14:28
derekokeeffe85haha14:40
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Update upgrade guide to mention SLURP  https://review.opendev.org/c/openstack/openstack-ansible/+/88348814:52
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Refactor log collection from lxc containers  https://review.opendev.org/c/openstack/openstack-ansible/+/87738214:54
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Reduce output by leveraging loop labels  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/87630214:59
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Update upgrade guide to mention SLURP  https://review.opendev.org/c/openstack/openstack-ansible/+/88348815:00
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not try to install packages on each log_instance_info run  https://review.opendev.org/c/openstack/openstack-ansible/+/86637215:04
NeilHanlonmgariepy: welcome back :) just got back this morning from a home inspection myself... so I probably have construction in my future, too :P 15:46
opendevreviewMerged openstack/openstack-ansible stable/2023.1: Implement support for haproxy_accept_both_protocols  https://review.opendev.org/c/openstack/openstack-ansible/+/88481715:50
mgariepynice NeilHanlon 15:51
mgariepynew home ?15:51
NeilHanlonmgariepy: yep, first home we own, hopefully! 15:53
mgariepynice :D15:55
noonedeadpunklucky you!16:00
opendevreviewMerged openstack/openstack-ansible stable/2023.1: Fix repo url in healthcheck-infrastructure.yml  https://review.opendev.org/c/openstack/openstack-ansible/+/88481816:15
opendevreviewMerged openstack/openstack-ansible master: Update master for stable/2023.1  https://review.opendev.org/c/openstack/openstack-ansible/+/88471716:15
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_service master: Reduce output by leveraging loop labels  https://review.opendev.org/c/openstack/ansible-role-systemd_service/+/87630216:24
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Allow to manage more the one vhost with mq_setup  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/87539916:56
noonedeadpunkanother vote is needed here https://review.opendev.org/c/openstack/openstack-ansible/+/88481617:05
mgariepydone 17:06
jrosser`infra1-glance-container-16db71dd uwsgi[27920]: Wed May 31 16:54:56 2023 - *** uWSGI listen queue of socket "10.96.241.58:9292" (fd: 3) full !!! (101/100) ***`17:09
jrosseranyone seen that sort of thing before? ^^17:09
jrosserservice is not sufficiently broken to fail the LB healthcheck, but totally broken for API17:09
noonedeadpunkoh yes, we did17:14
jrosserany idea what causes it17:16
jrosseralso i was looking at how we could have seen that coming and saw there is a nice looking uwsgi exporter that exposes queue depth17:16
noonedeadpunkI think in our case that was wrong MTU on storage network inside glance containers17:17
jrossernot sure if it is in scope of our uwsgi role to add this or not https://github.com/timonwong/uwsgi_exporter17:17
noonedeadpunkwell, we usually jsut enable if smth is provided natively by service rather then install exporters17:19
jrosseryes, we do17:20
noonedeadpunkmaybe we could get some role that will install exporters and include it from uwsgi conditionally... (that's not best idea frankly)17:20
noonedeadpunkI think my biggest concern here, is when we will come to libvirt exporter...17:21
noonedeadpunkAs this is smth... I don't wanna promote, let's put it this way17:21
jrosseractually for uwsgi it is probably a case of making sure the stats can be on a tcp port from some variables17:23
jrosserthen if you want an exporter it can be somewhere else17:23
noonedeadpunkyeah, we can do that, sure17:24
noonedeadpunkbut in our case we had some underlying issue that led to this listen queue full17:25
jrosseryes - i'm sure we will do too but not clear yet what it is17:26
jrosseror if it's been creeping up on queue length for a long time, or happened suddenly17:26
noonedeadpunkah, I read ticket wrong, MTU was wrong on lxc host, not in container, but they were different. Anyway - mtu issue...17:27
noonedeadpunkI'm pretty sure we saw more, but can't find now :(17:27
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/zed: Bump OpenStack-Ansible SHAs for Zed  https://review.opendev.org/c/openstack/openstack-ansible/+/88491317:38
noonedeadpunkjrosser: btw I've answered comments for https://review.opendev.org/c/openstack/openstack-ansible/+/88348817:55
jrosseroh yes17:57
jrosseri was wondering for SLURP if we should link in the our docs to what that means17:58
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Bump OpenStack-Ansible SHAs for Yoga  https://review.opendev.org/c/openstack/openstack-ansible/+/88491817:58
noonedeadpunkyeah, I linked to https://releases.openstack.org/ after all17:59
noonedeadpunkthe page has a refference to the TC decision with more detailed explanation17:59
jrosseroh of course i'm looking at the wrong revision17:59
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Update upgrade guide to mention SLURP  https://review.opendev.org/c/openstack/openstack-ansible/+/88348818:01
noonedeadpunksry, just decided to try out zero-width space instead....18:01
noonedeadpunklikely this will get rid of extra space once we remove `previous_slurp_name` definition18:02
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Mark Xena as EM in docs  https://review.opendev.org/c/openstack/openstack-ansible/+/88492018:04
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/2023.1: Remove other releases from 2023.1 index page  https://review.opendev.org/c/openstack/openstack-ansible/+/88492118:07
opendevreviewMerged openstack/ansible-role-uwsgi stable/xena: Replace imports with dynamic includes  https://review.opendev.org/c/openstack/ansible-role-uwsgi/+/88437418:31
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Start 2023.2 (Bobcat) development  https://review.opendev.org/c/openstack/openstack-ansible/+/88492418:33
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Add 2023.1 to the compatability matrix  https://review.opendev.org/c/openstack/openstack-ansible/+/88492518:35
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/xena: Switch roles and services to track stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/88492618:40
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Disable upgrade jobs after switching Xena to EM  https://review.opendev.org/c/openstack/openstack-ansible/+/88492818:42
noonedeadpunkI've pushed couple of patches related to releases...18:45
noonedeadpunkthough I bet that 884926 will fail dramatically, due to latest ossa fix...18:46
noonedeadpunkas we don't have code to allow multiple roles...18:47
noonedeadpunkmaybe we can backport that just for cinder/nova/glance....18:47
noonedeadpunkwill see tomorrow18:57
opendevreviewMerged openstack/openstack-ansible stable/2023.1: Bump SHAs for Antelope RC2  https://review.opendev.org/c/openstack/openstack-ansible/+/88481619:22
hamburglerCurious if this is going to be looked at again: https://review.opendev.org/c/openstack/openstack-ansible/+/85944619:27
hamburgler^skyline19:27
NeilHanlonhamburgler: in last PTG we discussed it, but no one volunteered to take it on iirc. I believe one of the hurdles is the reference deployment for skyline uses nginx, but we are using haproxy, so we cannot just copy/paste config and would need to make it fit into our haproxy roles, etc. 19:34
hamburglerNeilHanlon: gotcha, makes sense, bit of a segway - is there confidence behind the skyline project that even if added, there will be ongoing contributions to it? Right now most of the commits are only by a handful of people it seems.19:35
NeilHanlonit's a bit of a chicken/egg situation, where it needs support in deployment frameworks to have broader adoption. i think kolla-ansible added it just a few months ago. As for the project itself, I'm not sure entirely, but that's also probably something that will be aided by more adoption and internationalisation efforts19:38
hamburglerdefinitely :), hoping that happens19:42
hamburglerthank you 19:42
NeilHanlonour next PTG is in just a couple weeks - you're welcome to join :) https://etherpad.opendev.org/p/vancouver-june2023-os-ansible19:44
jrosserhamburgler: there are things missing from skyline that make it not attractive for me, for example19:44
NeilHanlonand if you're in the area of Vancouver, BC, join us in person! (I might be able to arrange a ticket, even ;) )19:44
hamburglerhaha I live in Van :D19:44
jrossernot feature parity with horizon unfortunately19:45
jrosserso I don’t have much incentive to maintain the role for OSA currently19:45
jrosserwe need someone who can work on it actively19:45
hamburglerjrosser: that makes sense, there were only a couple of things I was more or less interested in with skyline over horizon, one is obviously it looks a little nicer for end user but more importantly having the option to select volume_type during instance creation, rather than if we want to do that it has to be created under volumes first, then attached to the instance during creation19:47
hamburglerUsing the API/microversion (I think it was 2.6.3 >) I can specify the volume_type while the instance is being created all in one shot19:47
jrosserthere is a strange openid implementation I think in skyline19:48
jrosserbut I need something more modern like openid-connect and I think that’s a great example of features existing that match the needs of the very small number of contributors19:49
jrossersame really holds true for magnum, imho19:49
NeilHanlonyeah i have not checked it out myself, honestly, outside of looking at some screenshots and demos. It looks really nice and I am excited for it, but don't have time myself to dedicate for it20:00
hamburglerNor here, work from a smaller company and maybe will end up pushing the dev team above us to build something of their own since it seems like neither Horizon or Skyline fit our needs at the moment20:02
hamburglerfor*20:02
hamburglerNeilHanlon: This event is the OpenInfra Summit? 20:03
hamburglerThe PTG: https://openinfra.dev/ptg/ ?20:04
NeilHanlonhamburgler: yep! the PTG is coinciding with OpenInfra Summit20:04
NeilHanlonhttps://openinfra.dev/summit/vancouver-202320:04
hamburglerYeah definitely would love to attend :). Had tried to get the boss to foot the bill :') Now the prices are even higher since I didn't jump on it sooner20:16

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!