Thursday, 2024-03-07

fricklerwe are seeing this failure in kolla which is likely related to tooz 6.0.0, has anyone seen this? https://a8c5389f08768dc97af2-76b70ec5cd2ca2755a9230dd387d9dfa.ssl.cf5.rackcdn.com/911591/1/check/kolla-ansible-rocky9-cephadm/3d6ec78/primary/logs/kolla/cinder/cinder-api-error.txt07:46
tkajinamfrickler, ugh. it's likely a regression caused by recent fixes in redis driver (to handle ssl connections for sentinel). let me dig into it08:13
tkajinamok I found the problem. we may need to fix it and get a new release for stable/2024.1 ...08:16
opendevreviewTakashi Kajinami proposed openstack/tooz master: Fix broken redis sentinel support  https://review.opendev.org/c/openstack/tooz/+/91164808:29
tkajinamfrickler, fyi ^^^08:29
tkajinamonce the 2024.1 branch is created I'll backport it to 2024.1 and then request a new release...08:29
fricklerI don't think we need to wait for that if we consider this a release critical bug (which I guess it is if it breaks kolla)08:43
fricklertkajinam: also commented on the patch08:46
opendevreviewTakashi Kajinami proposed openstack/tooz master: Fix broken redis sentinel support  https://review.opendev.org/c/openstack/tooz/+/91164808:52
tkajinamoops sorry it picked up additional (but incomplete) changes I attempted (to make the logic consistent with taskflow)08:53
tkajinamwe have had functional tests in tooz but it tuned out these tests do not really test functionalities. /me sighs09:17
opendevreviewTakashi Kajinami proposed openstack/tooz master: dnm: Fix missing TOOZ_TEST_URL env  https://review.opendev.org/c/openstack/tooz/+/91168109:40
opendevreviewTakashi Kajinami proposed openstack/tooz master: dnm: Fix missing TOOZ_TEST_URL env  https://review.opendev.org/c/openstack/tooz/+/91168109:41
opendevreviewTakashi Kajinami proposed openstack/tooz master: Fix sentinel tests not running  https://review.opendev.org/c/openstack/tooz/+/91168210:01
opendevreviewTakashi Kajinami proposed openstack/tooz master: Fix sentinel tests not running  https://review.opendev.org/c/openstack/tooz/+/91168210:02
tkajinamfrickler, should fix the functional tests. I could reproduce the error with this patch but without the fix10:02
tkajinamI mean the above 911682 should fix the functional tests10:02
tkajinamwe used redis backend in the redis sentinel scneario. /facepalm10:03
*** fnordahl_ is now known as fnordahl10:13
tkajinamdamani hberaud_ tobias-urdin , could you review https://review.opendev.org/q/topic:%22bug/2056429%22+status:open when you have time (CI failure in the 2nd patch is irrelevant so I'm rerunning CI). these are needed to fix regression in tooz 6.0.013:27
tkajinamI've prepared a release patch to get 6.0.1 with that bug fix. RFE mail has been sent in email and that contains a bit more details.13:27
hberaud_done13:52
hberaud_would be good to release that ASAP before req freeze13:52
*** hberaud_ is now known as hberaud13:52
tkajinamthanks !13:53
hberaudYW13:53
tkajinamiirc we had req freeze last week so we need an exception. I've sent out an email to follow the formal process but I expect it may be accepted relatively easily.13:54
hberaudyep13:54
tkajinamwill try to settle this down asap13:55
hberaudthx13:56
hberaudtkajinam, damani: the branching deadline is today so please take a look to => https://review.opendev.org/c/openstack/releases/+/910921 and do not hesitate to -1 this patch if you feel we need to wait for a patch, else, without response from you both, the branches will be cuts tomorrow13:58
hberaudI just fixed the conflict related to castellan13:59
tkajinamok13:59
hberaudthx in advnce14:00
hberaudadvance14:00
hberaud(tooz is an independent deliverable, so it is not impacted by this branch cut)14:02
opendevreviewOpenStack Release Bot proposed openstack/automaton master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/automaton/+/91172514:13
opendevreviewOpenStack Release Bot proposed openstack/castellan master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/castellan/+/91172714:14
opendevreviewOpenStack Release Bot proposed openstack/debtcollector master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/debtcollector/+/91172914:14
opendevreviewOpenStack Release Bot proposed openstack/futurist master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/futurist/+/91173114:14
opendevreviewOpenStack Release Bot proposed openstack/oslo.cache master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.cache/+/91173314:14
opendevreviewOpenStack Release Bot proposed openstack/oslo.concurrency master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.concurrency/+/91173514:15
opendevreviewOpenStack Release Bot proposed openstack/oslo.config master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.config/+/91173714:15
opendevreviewOpenStack Release Bot proposed openstack/oslo.context master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.context/+/91173914:15
opendevreviewOpenStack Release Bot proposed openstack/oslo.db master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.db/+/91174114:15
opendevreviewOpenStack Release Bot proposed openstack/oslo.i18n master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.i18n/+/91174314:15
opendevreviewOpenStack Release Bot proposed openstack/oslo.limit master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.limit/+/91174514:16
opendevreviewOpenStack Release Bot proposed openstack/oslo.log master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.log/+/91174714:16
opendevreviewOpenStack Release Bot proposed openstack/oslo.messaging master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.messaging/+/91174914:16
opendevreviewOpenStack Release Bot proposed openstack/oslo.middleware master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.middleware/+/91175114:16
opendevreviewOpenStack Release Bot proposed openstack/oslo.policy master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.policy/+/91175314:16
opendevreviewOpenStack Release Bot proposed openstack/oslo.privsep master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.privsep/+/91175514:17
opendevreviewOpenStack Release Bot proposed openstack/oslo.reports master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.reports/+/91175714:17
opendevreviewOpenStack Release Bot proposed openstack/oslo.rootwrap master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.rootwrap/+/91175914:17
opendevreviewOpenStack Release Bot proposed openstack/oslo.serialization master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.serialization/+/91176114:17
opendevreviewOpenStack Release Bot proposed openstack/oslo.service master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.service/+/91176314:17
opendevreviewOpenStack Release Bot proposed openstack/oslo.upgradecheck master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.upgradecheck/+/91176514:18
opendevreviewOpenStack Release Bot proposed openstack/oslo.utils master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.utils/+/91176714:18
opendevreviewOpenStack Release Bot proposed openstack/oslo.versionedobjects master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.versionedobjects/+/91176914:18
opendevreviewOpenStack Release Bot proposed openstack/oslo.vmware master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslo.vmware/+/91177114:18
opendevreviewOpenStack Release Bot proposed openstack/oslotest master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/oslotest/+/91177314:19
opendevreviewOpenStack Release Bot proposed openstack/osprofiler master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/osprofiler/+/91177514:19
opendevreviewOpenStack Release Bot proposed openstack/stevedore master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/stevedore/+/91177714:19
opendevreviewOpenStack Release Bot proposed openstack/taskflow master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/taskflow/+/91177914:19
opendevreviewOpenStack Release Bot proposed openstack/tooz master: reno: Update master for unmaintained/victoria  https://review.opendev.org/c/openstack/tooz/+/91178114:19
opendevreviewOpenStack Release Bot proposed openstack/automaton master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/automaton/+/91178314:20
opendevreviewOpenStack Release Bot proposed openstack/castellan master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/castellan/+/91178514:20
opendevreviewOpenStack Release Bot proposed openstack/oslo.cache master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.cache/+/91178714:20
opendevreviewOpenStack Release Bot proposed openstack/oslo.config master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.config/+/91178914:20
opendevreviewOpenStack Release Bot proposed openstack/oslo.context master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.context/+/91179114:21
opendevreviewOpenStack Release Bot proposed openstack/oslo.db master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.db/+/91179314:21
opendevreviewOpenStack Release Bot proposed openstack/oslo.limit master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.limit/+/91179514:21
opendevreviewOpenStack Release Bot proposed openstack/oslo.messaging master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.messaging/+/91179714:21
opendevreviewOpenStack Release Bot proposed openstack/oslo.metrics master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.metrics/+/91179914:21
opendevreviewOpenStack Release Bot proposed openstack/oslo.middleware master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.middleware/+/91180114:22
opendevreviewOpenStack Release Bot proposed openstack/oslo.policy master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.policy/+/91180314:22
opendevreviewOpenStack Release Bot proposed openstack/oslo.privsep master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.privsep/+/91180514:22
opendevreviewOpenStack Release Bot proposed openstack/oslo.serialization master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.serialization/+/91180714:22
opendevreviewOpenStack Release Bot proposed openstack/oslo.service master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.service/+/91180914:22
opendevreviewOpenStack Release Bot proposed openstack/oslo.upgradecheck master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.upgradecheck/+/91181114:23
opendevreviewOpenStack Release Bot proposed openstack/oslo.utils master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.utils/+/91181314:23
opendevreviewOpenStack Release Bot proposed openstack/oslo.versionedobjects master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.versionedobjects/+/91181514:23
opendevreviewOpenStack Release Bot proposed openstack/oslo.vmware master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/oslo.vmware/+/91181714:23
opendevreviewOpenStack Release Bot proposed openstack/stevedore master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/stevedore/+/91181914:23
opendevreviewOpenStack Release Bot proposed openstack/tooz master: reno: Update master for unmaintained/wallaby  https://review.opendev.org/c/openstack/tooz/+/91182114:24
opendevreviewOpenStack Release Bot proposed openstack/automaton master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/automaton/+/91182314:24
opendevreviewOpenStack Release Bot proposed openstack/castellan master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/castellan/+/91182514:24
opendevreviewOpenStack Release Bot proposed openstack/oslo.cache master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.cache/+/91182714:24
opendevreviewOpenStack Release Bot proposed openstack/oslo.config master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.config/+/91182914:25
opendevreviewOpenStack Release Bot proposed openstack/oslo.context master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.context/+/91183114:25
opendevreviewOpenStack Release Bot proposed openstack/oslo.db master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.db/+/91183314:25
opendevreviewOpenStack Release Bot proposed openstack/oslo.limit master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.limit/+/91183514:25
opendevreviewOpenStack Release Bot proposed openstack/oslo.messaging master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.messaging/+/91183714:25
opendevreviewOpenStack Release Bot proposed openstack/oslo.metrics master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.metrics/+/91183914:26
opendevreviewOpenStack Release Bot proposed openstack/oslo.middleware master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.middleware/+/91184114:26
opendevreviewOpenStack Release Bot proposed openstack/oslo.policy master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.policy/+/91184314:26
opendevreviewOpenStack Release Bot proposed openstack/oslo.privsep master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.privsep/+/91184514:26
opendevreviewOpenStack Release Bot proposed openstack/oslo.serialization master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.serialization/+/91184714:26
opendevreviewOpenStack Release Bot proposed openstack/oslo.service master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.service/+/91184914:27
opendevreviewOpenStack Release Bot proposed openstack/oslo.upgradecheck master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.upgradecheck/+/91185114:27
opendevreviewOpenStack Release Bot proposed openstack/oslo.utils master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.utils/+/91185314:27
opendevreviewOpenStack Release Bot proposed openstack/oslo.versionedobjects master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.versionedobjects/+/91185514:27
opendevreviewOpenStack Release Bot proposed openstack/oslo.vmware master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/oslo.vmware/+/91185714:28
opendevreviewOpenStack Release Bot proposed openstack/stevedore master: reno: Update master for unmaintained/xena  https://review.opendev.org/c/openstack/stevedore/+/91185914:28
opendevreviewDaniel Bengtsson proposed openstack/oslo.log stable/2023.1: Fix logging in eventlet native threads  https://review.opendev.org/c/openstack/oslo.log/+/91194015:54
opendevreviewMerged openstack/tooz master: Fix broken redis sentinel support  https://review.opendev.org/c/openstack/tooz/+/91164816:32
-opendevstatus- NOTICE: Jobs that fail due to being unable to resolve mirror.dfw.rackspace.opendev.org can be rechecked. This error was an unexpected side effect of some nodepool configuration changes which have been reverted.16:55
opendevreviewMerged openstack/tooz master: Fix sentinel tests not running  https://review.opendev.org/c/openstack/tooz/+/91168219:56

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!