14:00:16 <rafaelweingartner> #startmeeting cloudkitty 14:00:16 <opendevmeet> Meeting started Mon Nov 25 14:00:16 2024 UTC and is due to finish in 60 minutes. The chair is rafaelweingartner. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:16 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:16 <opendevmeet> The meeting name has been set to 'cloudkitty' 14:00:19 <rafaelweingartner> Hello guys! 14:00:22 <rafaelweingartner> Roll count 14:00:23 <rafaelweingartner> \o 14:01:20 <priteau> o/ 14:01:21 <MattCrees[m]> Hello! o/ 14:03:47 <rafaelweingartner> Awesome that you all are here! 14:03:56 <rafaelweingartner> Let's start 14:04:00 <rafaelweingartner> #topic Drop python 3.8 support 14:04:21 <rafaelweingartner> #link https://review.opendev.org/c/openstack/cloudkitty/+/933144, it is jut waiting for your reviews 14:05:44 <priteau> Can you add a release note like this one? https://review.opendev.org/c/openstack/hacking/+/931106/1/releasenotes/notes/drop-python38-cca191b19e091b6c.yaml 14:06:05 <rafaelweingartner> sure 14:08:09 <MattCrees[m]> Will we follow up with patches to the other cloudkitty repos? dashboard, pythonclient, etc. 14:08:13 <opendevreview> Rafael Weingartner proposed openstack/cloudkitty master: Drop Python 3.8 support https://review.opendev.org/c/openstack/cloudkitty/+/933144 14:08:28 <rafaelweingartner> done 14:08:32 <rafaelweingartner> I guess so, right? 14:09:05 <rafaelweingartner> I will do that 14:09:06 <priteau> Yes, we should 14:09:28 <priteau> Actually we could have removed support in Dalmatian: https://governance.openstack.org/tc/reference/runtimes/2024.2.html 14:09:50 <priteau> But it's fine to do now 14:11:30 <rafaelweingartner> Moving on, we have the following topic 14:11:34 <rafaelweingartner> #topic Replace futurist with a native python solution? 14:11:45 <rafaelweingartner> however, we decided not to do anything for now, right? 14:12:04 <rafaelweingartner> The current library is already using a native implementation 14:12:56 <priteau> IIRC I raised it because we may be able to use the Python standard library 14:13:03 <priteau> But it's not critical to fix 14:14:20 <rafaelweingartner> ok 14:14:39 <priteau> Let's revisit another week 14:14:47 <rafaelweingartner> sure 14:15:26 <rafaelweingartner> Would you like to discuss the CI issues? 14:16:25 <priteau> Yes, so I noticed recently some failing Zuul jobs 14:17:06 <priteau> It started with some jobs in python-cloudkittyclient, e.g. https://review.opendev.org/c/openstack/python-cloudkittyclient/+/935061 14:18:12 <priteau> I am not finding the old logs right now, but the symptoms are that ck-proc doesn't start properly and this makes apache2 fail 14:18:51 <priteau> Usually in the ck-proc logs we can see: keystoneauth1.exceptions.catalog.EndpointNotFound: internalURL endpoint for metric service in RegionOne region not found 14:19:05 <priteau> I saw it today again in https://review.opendev.org/c/openstack/cloudkitty-tempest-plugin/+/934271 14:19:13 <priteau> Error in https://cf20bec4345c5c53d470-0a60532523bd9fb5f045ad0db727d57f.ssl.cf5.rackcdn.com/934271/2/gate/cloudkitty-tempest-full-v2-storage-elasticsearch/5aa5df8/controller/logs/screen-ck-proc.txt 14:19:17 <rafaelweingartner> I see 14:19:18 <rafaelweingartner> weird 14:19:21 <rafaelweingartner> did something change? 14:19:30 <rafaelweingartner> codewise nothing seem to have changed 14:19:36 <priteau> I thought it was some upstream change in devstack configuration, that would remove gnocchi/ceilometer 14:19:50 <priteau> But I tried adding them and it doesn't fix it 14:19:57 <rafaelweingartner> I see 14:20:06 <rafaelweingartner> if gnocchi and ceilometer is being remove 14:20:15 <rafaelweingartner> cloudkitty depends on it for gnocchi backend 14:20:22 <priteau> Actually we don't seem to require gnocchi/ceilometer in our jobs, somehow we run tests against cloudkitty without data? 14:21:08 <priteau> But I see 2024.2 jobs are passing so something must have changed recently 14:21:25 <rafaelweingartner> we inject data 14:21:28 <rafaelweingartner> I mean, fake data 14:21:35 <rafaelweingartner> in the functional tests 14:21:50 <priteau> Hum, actually the error about "internalURL endpoint for metric service in RegionOne region not found" is also in the job that is successful: https://aa8be72778fc9eca3151-0fe3e45fb7547d226c8ec0206078bbe6.ssl.cf2.rackcdn.com/934271/2/gate/cloudkitty-tempest-full-v2-storage-influxdb-2024-2/a0e2ff9/controller/logs/screen-ck-proc.txt 14:22:01 <priteau> So potentially a red herring 14:23:10 <priteau> Working job: 14:23:17 <priteau> 2024-11-25 12:44:07.813637 | controller | Created symlink /etc/systemd/system/multi-user.target.wants/devstack@ck-proc.service → /etc/systemd/system/devstack@ck-proc.service. 14:23:17 <priteau> 2024-11-25 12:44:08.311316 | controller | Enabling site cloudkitty. 14:23:17 <priteau> 2024-11-25 12:44:08.316451 | controller | To activate the new configuration, you need to run: 14:23:19 <priteau> 2024-11-25 12:44:08.316503 | controller | systemctl reload apache2 14:23:21 <priteau> 2024-11-25 12:44:23.473972 | controller | Waiting for ck-api (10.209.36.65:8889) to start... 14:23:33 <priteau> And not working job: 14:23:36 <priteau> 2024-11-25 12:46:21.613658 | controller | Created symlink /etc/systemd/system/multi-user.target.wants/devstack@ck-proc.service → /etc/systemd/system/devstack@ck-proc.service. 14:23:36 <priteau> 2024-11-25 12:46:22.156742 | controller | Enabling site cloudkitty. 14:23:36 <priteau> 2024-11-25 12:46:22.163568 | controller | To activate the new configuration, you need to run: 14:23:38 <priteau> 2024-11-25 12:46:22.163617 | controller | systemctl reload apache2 14:23:40 <priteau> 2024-11-25 12:46:37.346993 | controller | Job for apache2.service failed because the control process exited with error code. 14:23:42 <priteau> 2024-11-25 12:46:37.347123 | controller | See "systemctl status apache2.service" and "journalctl -xeu apache2.service" for details. 14:23:44 <priteau> 2024-11-25 12:46:37.351714 | controller | exit_trap: cleaning up child processes 14:23:46 <priteau> 2024-11-25 12:46:37.354770 | controller | Error on exit 14:23:48 <priteau> 2024-11-25 12:46:42.919708 | controller | *** FINISHED *** 14:24:38 <rafaelweingartner> maybe, related to this #link https://review.opendev.org/c/openstack/cloudkitty/+/936057? 14:25:06 <priteau> Ah, found it! 14:25:11 <priteau> Nov 25 12:46:37 np0039158722 apachectl[90967]: AH00526: Syntax error on line 4 of /etc/apache2/sites-enabled/cloudkitty.conf: 14:25:11 <priteau> Nov 25 12:46:37 np0039158722 apachectl[90967]: Invalid command 'WSGIDaemonProcess', perhaps misspelled or defined by a module not included in the server configuration 14:25:11 <priteau> Nov 25 12:46:37 np0039158722 apachectl[90964]: Action 'start' failed. 14:25:21 <priteau> Yes, must be related 14:25:29 <rafaelweingartner> it seems that some dependency was removed 14:26:06 <priteau> I'll re-run for the ipv6 job failure 14:26:26 <rafaelweingartner> ok 14:26:40 <priteau> So we need to focus on fixing this issue. Maybe stable branches are broken in some different way, not sure. 14:27:03 <rafaelweingartner> ok 14:29:49 <rafaelweingartner> besides those, we have the following #link https://review.opendev.org/c/openstack/cloudkitty/+/921339 14:29:55 <rafaelweingartner> which might need rebasing 14:30:25 <priteau> This one would be useful too 14:31:11 <opendevreview> Corentin ARNOULD proposed openstack/cloudkitty master: Allow Multiple Ratings for the same Metric on Prometheus https://review.opendev.org/c/openstack/cloudkitty/+/921339 14:31:23 <rafaelweingartner> I just rebased 14:33:47 <rafaelweingartner> besides those, do you guys have some other topic? 14:34:57 <priteau> Nothing special at the moment 14:35:51 <MattCrees[m]> Nothing from me either 14:37:08 <rafaelweingartner> cool 14:37:17 <rafaelweingartner> if you guys need some help 14:37:19 <rafaelweingartner> just ping me 14:38:27 <rafaelweingartner> if you do not have anything else, we can close 14:39:12 <MattCrees[m]> Cool, thanks rafaelweingartner 14:39:29 <priteau> Thanks Rafael 14:40:30 <rafaelweingartner> Thank you guys for participating. Have a nice week. 14:40:35 <rafaelweingartner> #endmeeting