| noonedeadpunk | doh, wtf happened to all ubuntu jobs... | 08:04 |
|---|---|---|
| noonedeadpunk | "Error loading unit file 'sysctl-container.service': org.freedesktop.systemd1.BadUnitSetting".... | 08:05 |
| noonedeadpunk | wtf | 08:05 |
| noonedeadpunk | btw that looks way more global, as also rocky and centos are failing same way | 08:10 |
| noonedeadpunk | mhm, it's related to this and likely config_template changes... https://opendev.org/openstack/openstack-ansible-lxc_container_create/src/branch/master/vars/main.yml#L85-L102 | 08:12 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Install kata containers from source https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/880288 | 09:01 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Allow to customize location of kv storage https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/883645 | 09:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Add my_ip to the zun config https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/883554 | 09:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Ensure service is restarted on unit file changes https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/880043 | 09:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_zun master: Add TLS support to zun backends https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/881140 | 09:06 |
| noonedeadpunk | yeah, it's related to config_template change.... | 09:08 |
| jrosser | there must be something we miss in the config template tests | 09:23 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_container_create master: Properly render sysctl-container https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/883649 | 09:38 |
| noonedeadpunk | jrosser: I think it's not config_template fault to be frank.... | 09:38 |
| noonedeadpunk | But at the same time I wonder how much things like that we have | 09:39 |
| noonedeadpunk | and how to find them... | 09:40 |
| jrosser | oooh right that would be the second pass of templating…. | 09:41 |
| noonedeadpunk | yup. which kinda doesn't make much sense to me personally | 09:41 |
| noonedeadpunk | But now I see why it was added | 09:41 |
| noonedeadpunk | As if I just debug of _lxc_container_systemd_services - it will be `"execstarts": "{{ sysctl_path }}/sysctl -p"` which is wrong at the first place imo... | 09:43 |
| noonedeadpunk | But yeah, we can have quite some bugs in all kind of places now... | 09:43 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-config_template master: Revert "Publish openstack.config_template on galaxy" https://review.opendev.org/c/openstack/ansible-config_template/+/881668 | 10:10 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/883649 jsut passed - it will unblock other jobs. | 11:28 |
| noonedeadpunk | btw, for jammy upgrade job - looks like for ovn we need to trigger neutron-server restart whenever OVN is upgraded majorly | 11:29 |
| noonedeadpunk | jamesdenton_: do you have any thoughts/experience with that? ^ | 11:30 |
| noonedeadpunk | as we get ovn agents "down" when we upgrade ubuntu 22.04 from Z to 2023.1 | 11:30 |
| noonedeadpunk | where ovs and ovn are upgraded | 11:31 |
| noonedeadpunk | btw thinking about that... I guess we might need there to leverage local facts we got rid of for nova/cinder... | 11:32 |
| noonedeadpunk | As we might need to restart neutron-server when ovn on other hosts are changed, but not sure.... | 11:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Define service_user for nova services https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/883520 | 11:48 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump upstream SHAs for Antelope https://review.opendev.org/c/openstack/openstack-ansible/+/883550 | 11:50 |
| noonedeadpunk | jrosser: was you ever checking on keystone password_hash_algorithm? As seems that current default password legth is limited to 54 and then it's just stripped: https://opendev.org/openstack/keystone/src/branch/master/keystone/common/password_hashing.py#L71-L89 | 13:05 |
| noonedeadpunk | So I'm thinking if it's worth to change default to scrypt | 13:06 |
| noonedeadpunk | as well as enable `security_compliance` by default.... | 13:07 |
| jrosser | I’ve not looked at these | 13:14 |
| jrosser | the only password users we have are the service users and admin | 13:15 |
| noonedeadpunk | ah, ok, fair enough then... | 13:54 |
| noonedeadpunk | I was just a bit o_O at trimming password to 54symbols | 13:54 |
| NeilHanlon | thanks noonedeadpunk for the lxc_container_create fix. was scratching my head last night like "why are all these jobs failing..." | 14:01 |
| noonedeadpunk | Yeah, I was quite o_O as well | 14:02 |
| noonedeadpunk | and then checked /etc in logs and spotted that variable was not renderred | 14:02 |
| noonedeadpunk | in unit file of the service mentioned in logs | 14:02 |
| noonedeadpunk | thankfully, we collect this... | 14:03 |
| NeilHanlon | so i guess we'll need to update some change sets to apply on top of that | 14:03 |
| noonedeadpunk | Well, we can just merge 883649 and re-check rest once it merged | 14:08 |
| NeilHanlon | oh, even better | 14:09 |
| noonedeadpunk | btw, distro path feels better now with https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/883520 | 14:11 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/66d31ce1de614919a6c355461b475f6d | 14:12 |
| noonedeadpunk | also SHA bump _almost_ passes, except some intermittent things on ceph, for instance | 14:12 |
| noonedeadpunk | hm, dstat seems to be dying early after launch... https://zuul.opendev.org/t/openstack/build/17fc200b49564adebbd3324f53cc3dcf/log/logs/openstack/instance-info/dstat.log.txt#982-999 | 14:17 |
| noonedeadpunk | whatever though.... | 14:18 |
| NeilHanlon | that's a new one to me lol | 14:19 |
| NeilHanlon | OverflowError: cannot convert float infinity to integer | 14:19 |
| NeilHanlon | division by zero, maybe? | 14:19 |
| noonedeadpunk | float infinity sounds slightly different, though still crazy thing... | 14:21 |
| noonedeadpunk | meh, probably I should have dropped dstat running in CI at all rather then fixing it... it sounded useful though... | 14:21 |
| noonedeadpunk | btw, there was a story about this tool NeilHanlon related to RedHat :) https://github.com/dstat-real/dstat/issues/170 | 14:23 |
| NeilHanlon | oof. | 14:23 |
| NeilHanlon | while i have comments, they're better shared over beers, i think... | 14:25 |
| noonedeadpunk | ++, there should be some in Vancouver is assume :D | 14:26 |
| noonedeadpunk | damiandabrowski: mgariepy if you're somewhere around - having another vote on https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/883649 would be great | 14:28 |
| noonedeadpunk | actually, there's an etherpad of patches we need to land https://etherpad.opendev.org/p/osa-antelope-leftover-patches | 14:28 |
| damiandabrowski | done, i'll go though the above patches during the evening | 14:30 |
| NeilHanlon | i'm going through the etherpad and adding some notes | 14:32 |
| NeilHanlon | 881824 and the ceph patches can go after 883649 merges | 14:33 |
| noonedeadpunk | oh, yes, ordering is quite messy | 15:02 |
| spatel | Hey! Do i need rgw to configure object storage on ceph? | 15:40 |
| spatel | Or rgw is something to just interact with cluster and administer? | 15:41 |
| noonedeadpunk | rgw is what implements object storage api for ceph | 16:16 |
| noonedeadpunk | like mds does for shared file systems | 16:16 |
| opendevreview | Merged openstack/openstack-ansible-lxc_container_create master: Properly render sysctl-container https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/883649 | 16:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Reduce stepca scenario to minimum setup https://review.opendev.org/c/openstack/openstack-ansible/+/883497 | 16:55 |
| *** melwitt_ is now known as melwitt | 17:11 | |
| noonedeadpunk | hm... seems we deploy spice frontend/backend regardless now | 17:24 |
| noonedeadpunk | https://paste.openstack.org/show/bfqofjyCAFIRIDoaG0Eo/ | 17:24 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Configure spice console on haproxy when it is used https://review.opendev.org/c/openstack/openstack-ansible/+/883688 | 17:29 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Configure spice console on haproxy only when it is used https://review.opendev.org/c/openstack/openstack-ansible/+/883688 | 17:29 |
| jrosser | noonedeadpunk: we need to revisit the console enablement | 17:34 |
| noonedeadpunk | Yeah, I can recall it was related to ironic | 17:34 |
| noonedeadpunk | and quite complex | 17:34 |
| jrosser | because if you add either ironic or arm computes then it's [spice|novnc] + serial | 17:34 |
| noonedeadpunk | but having backend that's always DOWN is also... meh... | 17:34 |
| jrosser | yes thats not great either | 17:35 |
| opendevreview | Merged openstack/ansible-role-python_venv_build master: setup.cfg: Replace dashes with underscores https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/882253 | 17:35 |
| noonedeadpunk | jrosser: well, serial condition is there | 17:35 |
| jrosser | i think for antelope there is now at least a dedicted haproxy setup per console type | 17:35 |
| noonedeadpunk | but it was like spice + vnc + serial when ironic is present | 17:35 |
| jrosser | yes | 17:36 |
| jrosser | though we don;t cover the case of x86 + arm without ironic | 17:36 |
| jrosser | i realised ours was only working by accident | 17:36 |
| noonedeadpunk | ah, and for arm it should be spice? | 17:36 |
| jrosser | serial | 17:36 |
| noonedeadpunk | ok, so spice should be only when it's explicitly selected? | 17:37 |
| jrosser | somehow, yes | 17:37 |
| jrosser | tbh i was thinking remove all the logic and make the console type a list | 17:37 |
| noonedeadpunk | at least patch is not absolutley wrong then :) | 17:37 |
| noonedeadpunk | * https://review.opendev.org/c/openstack/openstack-ansible/+/883688/2/inventory/group_vars/nova_all.yml | 17:38 |
| jrosser | yeah thats fine for now | 17:38 |
| noonedeadpunk | but yes, we should simplify/rework current logic for sure | 17:39 |
| noonedeadpunk | btw, looks like smth similar to https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/883520 for cinder as ceph jobs fail for $reason | 17:48 |
| noonedeadpunk | but according to what I read it should not... | 17:51 |
| opendevreview | Merged openstack/ansible-config_template master: Revert "Publish openstack.config_template on galaxy" https://review.opendev.org/c/openstack/ansible-config_template/+/881668 | 19:08 |
| opendevreview | Merged openstack/openstack-ansible-os_ceilometer stable/zed: Fix logic of defining polling_namespaces https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/883548 | 20:20 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server master: Fix use of haproxy_backend_ssl when haproxy_backend_ca is not defined https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/879452 | 21:01 |
| opendevreview | Merged openstack/ansible-role-python_venv_build master: Fix venv_build_targets when used with multi-arch https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/881848 | 21:45 |
| opendevreview | Merged openstack/openstack-ansible master: Tune settings in galera server for reduced ram in all-in-one build https://review.opendev.org/c/openstack/openstack-ansible/+/877278 | 21:46 |
| opendevreview | Merged openstack/openstack-ansible master: Bump down etcd version for zun https://review.opendev.org/c/openstack/openstack-ansible/+/883194 | 21:46 |
| opendevreview | Merged openstack/openstack-ansible master: Move repo_packages to group_vars https://review.opendev.org/c/openstack/openstack-ansible/+/881824 | 21:52 |
| opendevreview | Merged openstack/openstack-ansible-ceph_client master: Allow ceph clients to be enabled or disabled https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/882828 | 21:55 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!