| opendevreview | Merged openstack/openstack-ansible master: Make ceph use storage network https://review.opendev.org/c/openstack/openstack-ansible/+/864144 | 00:13 |
|---|---|---|
| opendevreview | Merged openstack/openstack-ansible master: [doc] Better document integrated usage of ceph-ansible https://review.opendev.org/c/openstack/openstack-ansible/+/862676 | 00:13 |
| *** ysandeep|out is now known as ysandeep | 05:57 | |
| *** kleini_ is now known as kleini | 07:20 | |
| dokeeffe85 | Morning, quick one, should I be able to ping 8.8.8.8 from inside the containers? I was asked the question and actually never tested that. I can't as it happens | 08:45 |
| *** ysandeep is now known as ysandeep|lunch | 08:45 | |
| jrosser | dokeeffe85: that "depends" | 08:52 |
| jrosser | dokeeffe85: eth0 inside the containers should be the default route, which is hooked up through to the host and NAT with dnsmasq | 08:52 |
| jrosser | you will be able to see this with something like `brctl show` on the host | 08:53 |
| jrosser | so if your host has a route to 8.8.8.8 and everything is hooked up correctly via dnsmasq to your container eth0, you should be able to ping it from inside | 08:53 |
| dokeeffe85 | Ok perfect, thank you both | 09:10 |
| dokeeffe85 | sorry thank you jrosser :) | 09:11 |
| *** akahat|ruck is now known as akahat|ruck|afk | 09:53 | |
| *** ysandeep|lunch is now known as ysandeep | 10:20 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Allow extra plugin installation ironic/inspector venvs https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/864511 | 10:31 |
| opendevreview | Merged openstack/openstack-ansible-os_barbican master: Update barbican api paste https://review.opendev.org/c/openstack/openstack-ansible-os_barbican/+/864420 | 11:06 |
| noonedeadpunk | mmm, octavia also seems broken due to sdk/collections... | 11:20 |
| noonedeadpunk | and sahara as well... | 11:20 |
| *** akahat|ruck|afk is now known as akahat|ruck | 11:32 | |
| dokeeffe85 | Hi all, another quick one if anyone has the time to answer. You guys pointed me towards the overrides the last day I was on here to have an EMC SAN as my backend storage. I've only gotten around to reading it now and think I understand but just so I'm sure I thought I might run it buy you first. So I create the manual config I need to connect to the SAN, then I override the cinder role to inject/use that manual config under the correct | 11:46 |
| dokeeffe85 | headings [default] etc... do I understand this correctly? | 11:46 |
| jrosser | dokeeffe85: first thing to do is put the required things into cinder.conf with https://github.com/openstack/openstack-ansible-os_cinder/blob/master/defaults/main.yml#L386 | 12:08 |
| jrosser | you could put that in /etc/openstack_deploy/user_variables.yml | 12:08 |
| dokeeffe85 | Yep I saw that example on the link you gave me the last day (it was nova but I got the idea) then I can use my manual config and that will be pulled into my cinder.conf? something similar to this https://paste.openstack.org/show/bKAMJmGWL5FtsnHo8XgY/ | 12:13 |
| jrosser | dokeeffe85: yes something like this https://paste.opendev.org/show/bspN0fL3YtyAeGfTS8zI/ | 12:15 |
| jrosser | that config override pattern is used all over the place in openstack-ansible, any of these files you need to adjust you can do with that | 12:16 |
| jrosser | you should also be able to run the cinder playbook with `--tags cinder-config` to skip over the installation tasks and just update the config files | 12:17 |
| jrosser | that will be quicker | 12:17 |
| dokeeffe85 | Brilliant, thanks. If I get it working I'll let you all know in case you want to add it as an option to the docs for others for EMC SAN | 12:18 |
| jrosser | you're welcome to submit a patch :) contributions are always welcome | 12:19 |
| dokeeffe85 | Thank you jrosser | 12:23 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_heat master: Install git into placement containers Просмотр исходного кода If venv_wheel_build_enable is defined to False, placement will fail to clone and install repositories due to missing git binary. https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/864526 | 12:57 |
| opendevreview | Merged openstack/openstack-ansible stable/yoga: Bump OpenStack-Ansible Yoga https://review.opendev.org/c/openstack/openstack-ansible/+/864428 | 12:58 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_heat master: Install git into placement containers https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/864526 | 13:00 |
| kleini | but the config override pattern is not used for https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/tasks/neutron_post_install.yml#L156 and I need to add --no-negcache to dnsmasq to get around https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/1974230 | 13:38 |
| kleini | I think, I will introduce something like neutron_dnsmasq_noresolv | 13:39 |
| mgariepy | https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/templates/dnsmasq-neutron.conf.j2 | 13:40 |
| mgariepy | you can set any config you want for it. | 13:40 |
| mgariepy | kleini, https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/defaults/main.yml#L118 | 13:41 |
| kleini | I can only set key=value pairs, not https://github.com/imp/dnsmasq/blob/master/dnsmasq.conf.example#L578 | 13:41 |
| mgariepy | ho. yea i guess we need to patch it a bit to accomodate that. | 13:46 |
| kleini | I am going to introduce neutron_dnsmasq_nonegcache. And then later can be decided whether this is default on in openstack-ansible | 13:47 |
| mgariepy | is there a lot of options like that one that can be set ? | 13:48 |
| mgariepy | would it be ok to have a list of options we can set ? | 13:48 |
| kleini | I see a lot of these options in https://github.com/imp/dnsmasq/blob/master/dnsmasq.conf.example#L578 | 13:51 |
| kleini | For me this would be totally okay to have a list of options, that can be set. I am not sure, how I can achieve that. For neutron_dnsmasq_nonegcache I am able to create a change request. | 13:53 |
| kleini | Maybe I have some time tomorrow to look through os_neutron role and see, how I can create such a list and create the change request for it. | 13:54 |
| jrosser | it's not really an ini format file though either | 13:54 |
| mgariepy | we can set a neutron_dhcp_config_list = [], the iterate over it in the template. | 13:55 |
| mgariepy | this way next time we need an option to fix we can only adjust the var. | 13:57 |
| mgariepy | can some of you check this one an comment ? :D https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/862403 | 13:59 |
| kleini | I get the idea and will try to create the change request. Thanks jrosser, mgariepy | 14:00 |
| *** ysandeep is now known as ysandeep|dinner | 14:41 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Update ansible-collections-openstack version https://review.opendev.org/c/openstack/openstack-ansible/+/864553 | 14:52 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Enable /healthcheck for Octavia API https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/864419 | 14:53 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:00 |
| opendevmeet | Meeting started Tue Nov 15 15:00:22 2022 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:00 |
| noonedeadpunk | #topic rollcall | 15:00 |
| noonedeadpunk | o/ | 15:00 |
| *** ysandeep|dinner is now known as ysandeep | 15:00 | |
| damiandabrowski | hi! | 15:00 |
| opendevreview | Merged openstack/openstack-ansible stable/xena: Bump OpenStack-Ansible Xena https://review.opendev.org/c/openstack/openstack-ansible/+/864431 | 15:01 |
| mgariepy | hey ! | 15:03 |
| noonedeadpunk | #topic office hours | 15:03 |
| noonedeadpunk | so, repo for zookeeper has been create. There's one BUT though | 15:03 |
| noonedeadpunk | there're redirect from openstack namespace to windmill on gitea | 15:04 |
| noonedeadpunk | so whenever you clone from opendev - you will clone wrong one | 15:04 |
| noonedeadpunk | Infra is aware and will discuss better solution for this situation | 15:04 |
| noonedeadpunk | skyline repo is not created yet | 15:04 |
| noonedeadpunk | I still going to push zookeeper role to gerrit to start reviewing/preapre tests | 15:05 |
| noonedeadpunk | I also trying to figure out failure reasons for sahara. Octiavia is broken due to collections version we use - patch for bumping version is already proposed | 15:09 |
| noonedeadpunk | Another failure is regarding swift/ironic | 15:10 |
| jrosser | o/ in another meeting at the same time :( | 15:10 |
| noonedeadpunk | Will try to take a look unless someone is already working with that | 15:10 |
| jrosser | i am unlikely to have time this week | 15:10 |
| noonedeadpunk | ok, then I will try to check on that | 15:11 |
| noonedeadpunk | damiandabrowski: any progress on internal ssl? | 15:11 |
| damiandabrowski | yeah, I've been extensively testing James Gibson's changes. Generally they look fine but I found some bugs, I will upload patches soon. My target is to finish internal SSL project before Zed release | 15:13 |
| damiandabrowski | I have 2 things to discuss | 15:13 |
| damiandabrowski | 1. Even this whole thing is called "internal" SSL, we probably want to provide a way to switch from http to https for external haproxy vip along with internal one, right? (from technical perspective I don't see any blockers) | 15:14 |
| noonedeadpunk | Well, yeah, that would be sweet. But mention that before release deadline we also need to branch repos and do RC | 15:14 |
| noonedeadpunk | well, I think switching ssl on/off for VIP should not cause any extra complexity from what we have now | 15:15 |
| damiandabrowski | complexity in terms of our code or user actions? | 15:16 |
| damiandabrowski | I'm talking about this change which for now covers only internal VIP: https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/829899 | 15:16 |
| cloudnull | has anyone had issues with the setting "send-proxy-v2" w/ galera? -- I just updated to the latest head of master and that setting was resulting in "ERROR 1130 (HY000): Proxy header is not accepted from ..." | 15:16 |
| noonedeadpunk | well, both. It's smth we already have now and there shouldn't be changes to that? | 15:17 |
| damiandabrowski | it's super easy to make it cover also external one | 15:17 |
| cloudnull | it looks like galera is the only service that uses that backend option ? | 15:17 |
| damiandabrowski | transition from unencrypted to encrypted traffic is not covered for now | 15:17 |
| damiandabrowski | https://review.opendev.org/c/openstack/openstack-ansible-specs/+/822850/1..3/specs/yoga/internal-tls.rst#b43 | 15:18 |
| damiandabrowski | Jonathan and James explain it here ^ | 15:18 |
| noonedeadpunk | cloudnull: um, no, never sam that | 15:19 |
| jrosser | damiandabrowski: doesnt james spec allow haproxy to have either http or https backends so it is possible to have a transition, or switch back and forth? | 15:20 |
| damiandabrowski | yes, but only for internal VIP | 15:21 |
| damiandabrowski | my proposal is to allow that transition also for external VIP | 15:21 |
| damiandabrowski | it's just a minor change in a code | 15:21 |
| damiandabrowski | (and i'm not talking about backends now) | 15:21 |
| noonedeadpunk | but PR you're mentioning only about backends? | 15:22 |
| noonedeadpunk | (kind of) | 15:22 |
| damiandabrowski | nope, that's why it also implements `haproxy_tcp_upgrade_frontend` var | 15:23 |
| noonedeadpunk | ah,ok,yes | 15:23 |
| noonedeadpunk | so we "force" tls for internal VIP when we want to secure backends | 15:23 |
| noonedeadpunk | but we might need also to force external vip in case it's not tls? | 15:24 |
| noonedeadpunk | Sounds like valid thing to do | 15:24 |
| damiandabrowski | ok | 15:25 |
| noonedeadpunk | it was so long time ago I already forgot what were caveats | 15:26 |
| damiandabrowski | i need to leave for a moment, sorry | 15:27 |
| noonedeadpunk | you also had second thing to discuss ;) | 15:28 |
| mgariepy | ovn ssl patch is ready for reviews, i probably need to add a note. and update the commit message. https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/862403 | 15:28 |
| mgariepy | do we want to support upgrading existing deployment to ssl or we skip that? | 15:28 |
| noonedeadpunk | oh, wow, it's huge | 15:31 |
| noonedeadpunk | and what upgrade would involve? As I'm not sure it's feasible at all? | 15:31 |
| mgariepy | i have no idea how we could proceed. | 15:31 |
| *** akahat|ruck is now known as akahat|ruck|dinner | 15:31 | |
| noonedeadpunk | Though I dunno if ovn is alike mysql, when client can talk both tls and non-tls | 15:32 |
| mgariepy | i'm not sure it would cause big issue (the state won't change on connection loss i guess) | 15:33 |
| noonedeadpunk | maybe we should set `neutron_ovn_ssl: false` in defaults, but enable that for CI? | 15:33 |
| noonedeadpunk | But indeed I have no idea on impact during such upgrade | 15:33 |
| mgariepy | i'd like to have it on by default | 15:33 |
| mgariepy | we can add release note for upgrade? | 15:34 |
| noonedeadpunk | yes, we can do that | 15:34 |
| mgariepy | jamesdenton, what do you think ? | 15:34 |
| mgariepy | we only have a couple deployers we know that uses it. | 15:34 |
| mgariepy | the deployment order is kinda weird also. | 15:36 |
| jrosser | i would be surprised if it can talk tls and non-tls at the same time | 15:36 |
| jrosser | the point of the tls is to secure the comms so doing also non-tls would be odd | 15:36 |
| jrosser | feels like an upgrade would be hard becasue as soon as the control plane parts get configured for tls then the compute/gateway nodes will be unable to connect to them | 15:38 |
| noonedeadpunk | well, mysql does that as well as libvirt and rabbit for example | 15:38 |
| mgariepy | the order is kinda weird there https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-neutron-install.yml | 15:39 |
| noonedeadpunk | while mysql does support that on the same port, and it's up to client either establish tls or not (unless explicitly restricted on server side), rabbit/libvirt just listen on different ports | 15:39 |
| mgariepy | northd is not deployed before the computes. | 15:39 |
| mgariepy | but if the connection is cut the ports in and security rules won't be updated until it's back online ? | 15:40 |
| mgariepy | i guess.. | 15:40 |
| noonedeadpunk | well when connection in ovs is cut to rabbit - huge shitshtorm with everything blicking comes when it's re-established | 15:41 |
| noonedeadpunk | I've heard that ovsdb is not a strongest part either... | 15:41 |
| mgariepy | it's kinda hard to test that in the CI. | 15:42 |
| noonedeadpunk | yeah, totally | 15:42 |
| NeilHanlon | o/ sorry I'm (very) late... timezones and Dst. I'm in Texas for SuperComputing22 | 15:43 |
| noonedeadpunk | but I think if it's only management that will be cut and flows will remain - it might be good enough | 15:43 |
| noonedeadpunk | Oh, lucky you NeilHanlon! | 15:43 |
| mgariepy | just like anything else when you don't have load on the server it works just like on the dev computer :D | 15:43 |
| jamesdenton | mgariepy until now we have OVN in experimental state, so i'm not sure upgrades are in scope if we want to default only to SSL | 15:44 |
| jamesdenton | seems like solving for the... 1 deployment out there | 15:44 |
| mgariepy | all that work for you jamesdenton ;p | 15:44 |
| noonedeadpunk | so indeed if it's just mgmt - let's cover with release note and set ssl enabled by default | 15:44 |
| jamesdenton | and maybe spatel lol | 15:44 |
| noonedeadpunk | also anskiy I guess? | 15:45 |
| mgariepy | yay ok will do that. | 15:45 |
| jamesdenton | perhaps. early adopters get to work out the kinks | 15:45 |
| jamesdenton | cobra kai. no mercy. | 15:45 |
| mgariepy | when you upgrade jamesdenton tell us about your experience :D | 15:46 |
| mgariepy | haha | 15:46 |
| jamesdenton | i most certainly will | 15:48 |
| noonedeadpunk | I will try to check PR then today/tomorrow morning | 15:48 |
| mgariepy | cool jrosser i'd like you opinion as well on the pki role usage :D | 15:50 |
| jrosser | ok :) | 15:50 |
| jrosser | unrelated to all this i have tried and failed to use the ironic openstack ansible modules | 15:55 |
| jrosser | certainly for Y the ironic API appears to want a system scoped token when the modules talk to it | 15:55 |
| jrosser | but in my utility container the CLI is fine | 15:55 |
| jrosser | i've not had time to investigate further but ironic api seems to have different behaviour to the others | 15:56 |
| noonedeadpunk | system scoped tokens should not be enforced in Y at least... | 15:57 |
| jrosser | i did note that our openrc in the utility container is making a token scoped to the admin project and not a system scoped one | 15:57 |
| noonedeadpunk | I can imagine bugs in collection itself - it's really quite huge mess as of today | 15:57 |
| jrosser | though i was sure we had a patch related to that some time ago | 15:57 |
| noonedeadpunk | I think, it;s matter of variable now | 15:58 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-openstack_openrc/commit/fdc640ddcbc13de17609005f4ca34cc4067cd5f8 | 15:58 |
| spatel | spatel i am up for any kind of testing :) | 15:58 |
| noonedeadpunk | openrc_system_scope | 15:58 |
| noonedeadpunk | So we basically have multiple openrc files when it;s enabled | 15:59 |
| noonedeadpunk | some logic for clouds.yaml present though | 15:59 |
| jrosser | ahha ok | 16:00 |
| noonedeadpunk | but tbh I'm really not sure if that's ironic or sdk/collection | 16:00 |
| noonedeadpunk | As with X sdk and Y collections even keystone failed the same way | 16:00 |
| noonedeadpunk | So might be API expects some param to be provided, but collection does not provide it | 16:00 |
| noonedeadpunk | as it was missed/not updated | 16:01 |
| noonedeadpunk | Evenrually sad thing about collections, that 2.0 is going to be release in January | 16:02 |
| damiandabrowski | i'm back sorry. So regarding internal TLS there is one more problem with uWSGI. | 16:02 |
| damiandabrowski | In order to use https/https-socket options, the OpenSSL development headers are required(e.g. libssl-dev on Debian). | 16:02 |
| damiandabrowski | It's not installed by default and after dev headers are installed, uWSGI python package needs to be reinstalled. | 16:02 |
| damiandabrowski | But the old package without https support can be still stored in pip cache(/root/.cache/) which will prevent the correct uWSGI package to be installed. | 16:02 |
| damiandabrowski | It can be fixed by disabling pip cache when rebuilding venv: `openstack-ansible /opt/openstack-ansible/playbooks/os-glance-install.yml -e venv_rebuild=true -e uwsgi_pip_install_args='--no-cache-dir'` | 16:02 |
| noonedeadpunk | So we will have to release with SHA in ansible-collection-requirements | 16:02 |
| damiandabrowski | I guess we want to add an extra logic for that, right? So when libssl-dev is not installed but TLS is enabled, uwsgi playbook will re-install uwsgi. | 16:02 |
| damiandabrowski | Or maybe you have some better ideas? | 16:02 |
| *** ysandeep is now known as ysandeep|out | 16:02 | |
| noonedeadpunk | I don't think it's an issue? | 16:03 |
| noonedeadpunk | As we can always build uwsgi with ssl support? | 16:04 |
| damiandabrowski | it's the issue on existing environments which doesn't have SSL enabled | 16:04 |
| damiandabrowski | i guess we need to provide a smooth upgrade path | 16:04 |
| noonedeadpunk | and since we introduce this with new release - uwsgi version will be different, thus cache won't have any effect? | 16:04 |
| jrosser | i think there is a recent new release of uwsgi | 16:05 |
| noonedeadpunk | because cache might be an issue only when we re-build same uwsgi version but with different flags, while we don't? | 16:05 |
| jrosser | so we can probably catch this with enable ssl for it and bump version all at the same time | 16:05 |
| noonedeadpunk | oh... where we bump uwsgi :D | 16:06 |
| jrosser | global pins? | 16:06 |
| noonedeadpunk | ah https://opendev.org/openstack/openstack-ansible/src/branch/master/global-requirement-pins.txt#L11 | 16:06 |
| noonedeadpunk | yeah, was surprised not to find it in u-c | 16:06 |
| jrosser | yeah so i think we have a neat way to do that hopefully | 16:06 |
| noonedeadpunk | yup | 16:06 |
| jrosser | it just needs libssl-dev in the repo server? | 16:07 |
| damiandabrowski | ah yes i think you are right, if we assume that uwsgi will be upgraded it's just a matter of adding libssl-dev to the `uwsgi_package_list` | 16:07 |
| noonedeadpunk | 2.0.21 was released ~month ago | 16:07 |
| noonedeadpunk | it's weird kind of... does pip just - oh, I have libssl - I will build SSL support for uwsgi? | 16:08 |
| noonedeadpunk | and if not - well, ignore that? | 16:08 |
| damiandabrowski | yeah | 16:08 |
| damiandabrowski | it's mentioned here: https://uwsgi-docs.readthedocs.io/en/latest/HTTPS.html | 16:09 |
| noonedeadpunk | well, actually I think it works same for journald I guess.... | 16:09 |
| damiandabrowski | "In order to use https option be sure that you have OpenSSL development headers installed (e.g. libssl-dev on Debian). Install them and rebuild uWSGI so the build system will automatically detect it." | 16:09 |
| jrosser | uwsgi has a totally custom build system | 16:09 |
| jrosser | it's pretty wild | 16:09 |
| noonedeadpunk | ok then.... | 16:09 |
| damiandabrowski | okok, thanks for the input guys | 16:09 |
| noonedeadpunk | So yes - let's bump uwsgi version and call it a day | 16:10 |
| noonedeadpunk | oh. we're overtime:) | 16:10 |
| jrosser | iirc there was some discussion in the infra channel this last week about conflucting versions of pyOpenssl (?) and that breaking uwsgi buids | 16:10 |
| noonedeadpunk | #endmeeting | 16:10 |
| opendevmeet | Meeting ended Tue Nov 15 16:10:18 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:10 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-11-15-15.00.html | 16:10 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-11-15-15.00.txt | 16:10 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-11-15-15.00.log.html | 16:10 |
| jrosser | would be worth looking back through the irc logs about that as i feel it might be relevant here | 16:10 |
| noonedeadpunk | sounds like it is... as new uwsgi was released on 24.10.2022 and before that it was in 2021... | 16:11 |
| noonedeadpunk | (and I also can recall it broke smth) | 16:11 |
| spatel | sorry for not participating because my current workload is so high :( | 16:22 |
| gokhani | Hi folks, which free backup solutions are you using or recommending? I tried bacula but ceph broke it and we can not copy disk images directly. | 16:24 |
| damiandabrowski | you may want to check it out: https://backy2.com/ | 16:27 |
| damiandabrowski | I've been backing up small ceph cluster to backblaze 2 years ago | 16:27 |
| cloudnull | there's another weird one -- Nov 15 16:23:36 compute-0 neutron-api[1528269]: 2022-11-15 16:23:36.102 1528269 ERROR stevedore.extension [req-c50cafb6-05b6-4331-a7f1-72a0c70b7ff2 req-ba496483 | 16:29 |
| cloudnull | -f72e-4cb6-a426-c9e61a9f06a1 875f6580fc714f98bd0c35b574f28ff1 192f9fa5cae445348bcc9030ad6b9720 - - default default] Could not load 'oslo_cache.memcache_pool': N | 16:29 |
| cloudnull | o module named 'bmemcached': ModuleNotFoundError: No module named 'bmemcached' | 16:29 |
| cloudnull | it looks like dogpile is trying to use bmemcached ? | 16:29 |
| jrosser | hmm that sounds familiar | 16:29 |
| jrosser | i wonder if thats similar to the error messages there were about etd3gw | 16:30 |
| jrosser | where some plugins import every possible backend even if they're not used | 16:30 |
| cloudnull | potentially | 16:30 |
| cloudnull | I only see it in the logs when the service starts, and it doesn't seem to repeat or be blocking | 16:31 |
| cloudnull | but I guess there's a new-ish dep that needs to be added or blocked? | 16:32 |
| gokhani | thanks damiandabrowski, ı will check it | 16:34 |
| cloudnull | installing python-binary-memcached seems to make the error go away | 16:34 |
| opendevreview | Kevin Carter proposed openstack/openstack-ansible-os_neutron master: Add dep python-binary-memcached https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/864572 | 16:42 |
| noonedeadpunk | cloudnull: that I know about.... | 16:43 |
| cloudnull | oh? | 16:44 |
| noonedeadpunk | cloudnull: I've pushed this to cover the issue to some other repos https://opendev.org/openstack/openstack-ansible-os_keystone/commit/0a24c61e3e65e126691577c9b2843a7d9f448301 | 16:44 |
| noonedeadpunk | Though I guess it's master? | 16:44 |
| cloudnull | yeah Im running the head of master | 16:45 |
| cloudnull | rolled forward last night and was reviewing some of the new errors :D | 16:45 |
| cloudnull | "features" I mean :P | 16:45 |
| noonedeadpunk | hehe | 16:46 |
| noonedeadpunk | For some reason I thought that neutron should not be affected though.... | 16:49 |
| noonedeadpunk | ah, maybe that's due to `memcache_use_advanced_pool` | 16:49 |
| cloudnull | it seems to be only neutron impacted ? | 16:51 |
| cloudnull | https://paste.openstack.org/show/bptBUp0NEmmJao1hSoHd/ | 16:51 |
| noonedeadpunk | but then still installing `oslo.cache[dogpile]` as pip should fix that | 16:51 |
| noonedeadpunk | Looking at https://opendev.org/openstack/keystonemiddleware/src/branch/master/releasenotes/notes/deprecate-eventlet-unsafe-memcacheclientpool-f8b4a6733513d73e.yaml I think we're missing a thing in keystone_authtoken | 16:52 |
| noonedeadpunk | for rest of the services | 16:52 |
| cloudnull | the requirement is in global https://github.com/openstack/requirements/blob/master/global-requirements.txt#L290 | 16:52 |
| noonedeadpunk | hm | 16:53 |
| cloudnull | https://github.com/openstack/oslo.cache/blob/master/setup.cfg#L40 | 16:53 |
| noonedeadpunk | but then shouldn't it be installed.... | 16:54 |
| noonedeadpunk | oh | 16:55 |
| cloudnull | I would have expected that looking at the deps. | 16:55 |
| noonedeadpunk | you use uwsgi for neutron? | 16:55 |
| noonedeadpunk | I don't see it being missing in aio I spawned today.... | 16:57 |
| cloudnull | yeah its installed in my nova venv, just not neutron | 16:57 |
| noonedeadpunk | yeah, also don't see it in neutron, but neither see error in the log..... | 16:58 |
| cloudnull | weird ☁️ | 16:59 |
| noonedeadpunk | anyway I left a comment :) | 16:59 |
| cloudnull | ++ | 16:59 |
| opendevreview | Kevin Carter proposed openstack/openstack-ansible-os_neutron master: Add dep python-binary-memcached https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/864572 | 17:09 |
| *** akahat|ruck|dinner is now known as akahat|ruck | 18:31 | |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible master: Bump uWSGI version https://review.opendev.org/c/openstack/openstack-ansible/+/864579 | 18:33 |
| spatel | Anyone using Trove here? | 21:26 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!