| dalees | hi jakeyip , are you around this week? | 08:44 |
|---|---|---|
| jakeyip | I am here | 09:00 |
| jakeyip | dalees: anything you want to discuss? | 09:00 |
| dalees | yeah, I wanted to discuss https://bugs.launchpad.net/magnum/+bug/2067345 and proposed fix to see if it's on the right track. | 09:01 |
| dalees | anyway, I added to meeting notes :) | 09:02 |
| jakeyip | ok | 09:02 |
| jakeyip | #startmeeting magnum | 09:02 |
| opendevmeet | Meeting started Wed Sep 11 09:02:53 2024 UTC and is due to finish in 60 minutes. The chair is jakeyip. Information about MeetBot at http://wiki.debian.org/MeetBot. | 09:02 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 09:02 |
| opendevmeet | The meeting name has been set to 'magnum' | 09:02 |
| jakeyip | #link https://etherpad.opendev.org/p/magnum-weekly-meeting | 09:03 |
| jakeyip | #link https://etherpad.opendev.org/p/magnum-weekly-meeting | 09:03 |
| jakeyip | #link https://etherpad.opendev.org/p/magnum-weekly-meeting | 09:03 |
| jakeyip | o/ | 09:03 |
| jakeyip | #topic Roll Call | 09:03 |
| jakeyip | o/ | 09:03 |
| dalees | o/ | 09:03 |
| jakeyip | #topic Cluster create returns before uuid is valid | 09:05 |
| jakeyip | #link https://bugs.launchpad.net/magnum/+bug/2078390 | 09:05 |
| jakeyip | dalees: I just read the bug, haven't reviewed it yet, apologies | 09:06 |
| dalees | no worries, I understand you've been away! | 09:06 |
| jakeyip | yeah just got back Mon. :) | 09:07 |
| jakeyip | seems valid. I wonder if you referred to other services for the pattern? (synch. call to init db entry) | 09:08 |
| dalees | No, I didn't. I also considered adding a new state 'INITIALIZED', but figured that was more change than it was worth. It would help identify a lost or slow async RPC create message, if that ever occurred. | 09:10 |
| jakeyip | ok, I don't think that's necessary | 09:14 |
| jakeyip | I find the last two lines of the change log confusing - what can be upgraded first? | 09:14 |
| jakeyip | oh sorry I misread, ignore me | 09:15 |
| dalees | conductor first - it will handle both the old and new rpc create messages | 09:16 |
| dalees | (and by 'old' I just mean that initialize hasn't been called yet) | 09:18 |
| jakeyip | first read it looks ok, I will look more after this meeting | 09:19 |
| dalees | well, I will deploy this change and ensure it sorts out our Tempest problems. Maybe there is more to do, but it feels like a reasonable change even though it adds an RPC round-trip. | 09:19 |
| dalees | cheers, all good. | 09:19 |
| jakeyip | just curious, which tempest tests are you running? | 09:21 |
| dalees | we run ClusterTest.test_create_list_sign_delete_clusters regularly. | 09:22 |
| jakeyip | yeah same. do you use template id or let the tempest test create a template? | 09:23 |
| dalees | template id | 09:24 |
| jakeyip | ok same | 09:26 |
| dalees | the symptoms we end up seeing in this race condition is that tempest will end up deleting a cluster as Magnum does the helm install. So some secrets are wiped from CAPI and the cluster in Magnum is gone, but the helm resources are all created and CAPI tries to keep reconciling the cluster. | 09:28 |
| dalees | so keep a eye out in your CAPI management cluster for extra tempest clusters that Magnum doesn't know about. | 09:28 |
| dalees | I don't think it would be limited to the CAPI helm driver, as the conductor is doing all of this async. But maybe it's easier to reproduce with helm. | 09:30 |
| jakeyip | so there will be a k8s cluster but no corresponding magnum cluster? | 09:30 |
| dalees | yes, but it usually fails to create as Magnum deletes the OpenStack secrets, or deletes the app cred. | 09:31 |
| jakeyip | yeap ok | 09:31 |
| jakeyip | for the RPC incompatibility, I am looking if there's some recommendations for this kind of change. it's not typical like adding a new object field. have you looked into that? | 09:38 |
| dalees | Not at other prior art, no. I understand some services have RPC versioning they can use, which prevents upgrading until all agents are on the new version. I'm not sure if Magnum registers agents in the same way as Nova/Neutron though. | 09:41 |
| jakeyip | yeah there's a `openstack coe service list`, but no version from what I can tell | 09:46 |
| jakeyip | getting late, let's do this offline | 09:46 |
| jakeyip | anything else you want to discuss? | 09:47 |
| dalees | nope, that's all from me | 09:47 |
| jakeyip | ok let me take some time to review | 09:50 |
| jakeyip | seeya next week? (no pressure :P ) | 09:50 |
| dalees | Yes, see you next week | 09:54 |
| jakeyip | #endmeeting | 09:56 |
| opendevmeet | Meeting ended Wed Sep 11 09:56:05 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 09:56 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/magnum/2024/magnum.2024-09-11-09.02.html | 09:56 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/magnum/2024/magnum.2024-09-11-09.02.txt | 09:56 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/magnum/2024/magnum.2024-09-11-09.02.log.html | 09:56 |
| jakeyip | dalees: https://github.com/openstack/magnum/blob/master/magnum/common/rpc_service.py#L35 :P | 10:09 |
| jakeyip | oh well | 10:09 |
| dalees | haha 2014 | 10:23 |
| mnasiadka | 10 years ago, nice | 11:17 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!