| opendevreview | melanie witt proposed openstack/nova-specs master: Re-propose specs for ephemeral encryption https://review.opendev.org/c/openstack/nova-specs/+/907654 | 05:23 |
|---|---|---|
| opendevreview | melanie witt proposed openstack/nova-specs master: Re-propose specs for ephemeral encryption https://review.opendev.org/c/openstack/nova-specs/+/907654 | 05:45 |
| gibi | bauzas: I'm not sure you saw it in the last weeks meeting log that we discussed https://blueprints.launchpad.net/nova/+spec/shared-security-groups as a specless bp. The cores present had no objection to approve it but we agreed to wait for you to have a final word on it and approve it if it is OK to you too | 07:19 |
| gibi | relevant meeting log https://blueprints.launchpad.net/nova/+spec/shared-security-groups | 07:20 |
| bauzas | gibi: I haven't yet looked at last week's meeting | 07:20 |
| gibi | https://meetings.opendev.org/meetings/nova/2024/nova.2024-05-07-16.00.log.html#l-34 | 07:21 |
| bauzas | btw. today is a specs review day | 07:21 |
| bauzas | gibi: ack, I'll accept it then | 07:21 |
| gibi | thanks | 07:21 |
| opendevreview | Merged openstack/nova stable/2024.1: [ironic] Fix rebooting instance https://review.opendev.org/c/openstack/nova/+/918180 | 09:52 |
| opendevreview | Merged openstack/nova-specs master: Repropose per-process-healthchecks for Dalmatian https://review.opendev.org/c/openstack/nova-specs/+/919050 | 09:57 |
| opendevreview | Merged openstack/nova-specs master: Update and Re-propose "Allow Manila shares to be directly attached to an instance when using libvirt" for Dalmatian https://review.opendev.org/c/openstack/nova-specs/+/913997 | 09:57 |
| sahid | o/ | 10:02 |
| opendevreview | sean mooney proposed openstack/nova master: use futerist for RBD calls https://review.opendev.org/c/openstack/nova/+/917962 | 11:13 |
| gibi | dansmith: when you are around I'm wondering if you were part of the userdata discussion during the PTG as I see discrepancies here https://review.opendev.org/c/openstack/nova-specs/+/863884/7#message-e1f200505bbf81d0b9a254c4634f6fdc477871dd | 11:34 |
| sean-k-mooney | im kind of having a hard time reading and parsing sepcs today so i think im oging to stop unless some one asks me specific for input on somethign | 11:56 |
| opendevreview | Konrad Gube proposed openstack/nova master: Use Cinder's os-extend_volume_completion volume action. https://review.opendev.org/c/openstack/nova/+/873560 | 12:03 |
| opendevreview | Takashi Kajinami proposed openstack/nova-specs master: libvirt: Stateless firmware support https://review.opendev.org/c/openstack/nova-specs/+/908297 | 12:13 |
| opendevreview | Merged openstack/nova-specs master: libvirt: Stateless firmware support https://review.opendev.org/c/openstack/nova-specs/+/908297 | 13:01 |
| opendevreview | Stephen Finucane proposed openstack/placement master: Add placement.wsgi.api module https://review.opendev.org/c/openstack/placement/+/919569 | 13:24 |
| opendevreview | Takashi Kajinami proposed openstack/nova-specs master: libvirt: AMD SEV-ES support https://review.opendev.org/c/openstack/nova-specs/+/907702 | 13:35 |
| ykarel | Hi reported https://bugs.launchpad.net/nova/+bug/2065689 and https://bugs.launchpad.net/nova/+bug/2065693, please check if in case those are something known already | 13:40 |
| dansmith | bauzas: gibi: can you guys land this quick? it's blocking a devstack performance improvement: https://review.opendev.org/c/openstack/nova/+/918689 | 13:41 |
| dansmith | it's trivial | 13:41 |
| dansmith | gibi: we had some discussion about it yeah, I can have a look | 13:42 |
| bauzas | dansmith: ack | 13:42 |
| dansmith | gibi: hmm, yeah I feel like some context is missing from the PTG summary.. I thought there was a different conclusion reached about what to do, but now I'm not sure what that is.. maybe bauzas remembers? | 13:48 |
| * bauzas looks | 13:51 | |
| bauzas | gibi: dansmith: I think we said the owner provided the right version eventually about our concerns | 13:58 |
| gibi | dansmith: if you are OK to use the RPC cast to trigger the regeneration of the config drive instead of having a flag in sysmeta and do the regeneration at spawn then I'm OK with it. But reading back on the comments I felt that you regen at spawn idea was not discussed at the PTG | 14:08 |
| dansmith | gibi: no, it can't be a cast, right? | 14:12 |
| opendevreview | Stephen Finucane proposed openstack/placement master: tox: Remove envdir overrides https://review.opendev.org/c/openstack/placement/+/919578 | 14:13 |
| opendevreview | Stephen Finucane proposed openstack/placement master: Integrate pre-commit https://review.opendev.org/c/openstack/placement/+/919579 | 14:13 |
| opendevreview | Stephen Finucane proposed openstack/placement master: pre-commit: Add sphinx-lint https://review.opendev.org/c/openstack/placement/+/919580 | 14:13 |
| gibi | I think it is propsed as a cast now, and the compute sets the result of the instance action based on the success / failure of the regen | 14:13 |
| gibi | the nova-api can check the compute version but should not wait for the regen to happen via an RPC call | 14:14 |
| gibi | as regen takes time | 14:14 |
| opendevreview | sean mooney proposed openstack/nova master: add initial healthcheck support https://review.opendev.org/c/openstack/nova/+/825015 | 14:14 |
| opendevreview | sean mooney proposed openstack/nova master: add healthcheck manager to manager base https://review.opendev.org/c/openstack/nova/+/827844 | 14:14 |
| opendevreview | sean mooney proposed openstack/nova master: add healthcheck tracker to nova context https://review.opendev.org/c/openstack/nova/+/829468 | 14:14 |
| opendevreview | sean mooney proposed openstack/nova master: add healthcheck endpoint to proxy commands https://review.opendev.org/c/openstack/nova/+/830703 | 14:14 |
| opendevreview | sean mooney proposed openstack/nova master: [WIP] add iniital health check to compute agent https://review.opendev.org/c/openstack/nova/+/906911 | 14:14 |
| opendevreview | sean mooney proposed openstack/nova master: [WIP] add libvirt connection healtcheck https://review.opendev.org/c/openstack/nova/+/907424 | 14:14 |
| dansmith | okay but then we've told the http client "yes, we will do that thing" even though it might show up at the compute that has no idea what to do with it | 14:15 |
| gibi | if the service version of the compute is old the the API can fail the instance action, but if the service version is new enough then the api can cast down to the compute to ask for the regen | 14:16 |
| gibi | (except for shelved_offloaded instances) | 14:16 |
| gibi | at least this is how I understand the PTG notes and the spec now | 14:16 |
| dansmith | okay I haven't re-read it this morning yet | 14:19 |
| bauzas | I'll try to review it too today | 14:20 |
| bauzas | but that's an old spec | 14:20 |
| gibi | it was revised around the PTG so the content is fairly fresh | 14:22 |
| opendevreview | Stephen Finucane proposed openstack/placement master: Add placement.wsgi.api module https://review.opendev.org/c/openstack/placement/+/919569 | 14:22 |
| opendevreview | Stephen Finucane proposed openstack/placement master: setup: Remove pbr's wsgi_scripts https://review.opendev.org/c/openstack/placement/+/919582 | 14:22 |
| gibi | Does anybody remember why we name our db contraints like uniq_key_pairs0user_id0name It feels like there is a reason for that specific structure but I never come accross the explanation | 14:25 |
| gibi | https://github.com/openstack/nova/blob/7096423b343ffce9622fd078fc2b3a87fd3386f7/nova/db/api/models.py#L308-L310 | 14:25 |
| dansmith | yeah I think there was a reason, but it might have just been "consistency" | 14:31 |
| bauzas | indeed | 14:45 |
| gibi | OK, then I ask Uggla_ to follow the pattern just for consistency | 14:56 |
| gibi | here https://review.opendev.org/c/openstack/nova/+/912518/2/nova/db/main/models.py#767 | 14:56 |
| Uggla_ | gibi, ok | 15:03 |
| stephenfin | fyi the sqlalchemy 2.x bump *finally* merged, so if you want to free up some resources we can merge these now https://review.opendev.org/c/openstack/nova/+/915562 https://review.opendev.org/c/openstack/placement/+/915568 | 15:04 |
| ykarel | #topic Stable branches | 15:10 |
| ykarel | all good | 15:10 |
| ykarel | #topic Stadium projects | 15:10 |
| ykarel | all green | 15:10 |
| ykarel | #topic Rechecks | 15:10 |
| ykarel | 6/14 bare rechecks | 15:10 |
| frickler | ykarel: wrong channel? | 15:10 |
| ykarel | ahh right | 15:11 |
| ykarel | thx frickler | 15:11 |
| opendevreview | Gorka Eguileor proposed openstack/nova master: Support os-brick specific lock_path https://review.opendev.org/c/openstack/nova/+/849328 | 15:16 |
| bauzas | nova meeting in 30 mins | 15:32 |
| opendevreview | Takashi Kajinami proposed openstack/nova-specs master: Remove template files from non-empty directories https://review.opendev.org/c/openstack/nova-specs/+/919597 | 15:33 |
| bauzas | gibi: shit, I forgot I'll need to travel to the French OpenInfra Day in Paris on next Tuesday | 15:56 |
| bauzas | so I couldn't be able to run the meeting | 15:56 |
| gibi | bauzas: I will be off on next Tuesday | 15:59 |
| bauzas | no worries, I'll ask in our meeting today | 15:59 |
| bauzas | #startmeeting nova | 16:01 |
| opendevmeet | Meeting started Tue May 14 16:01:17 2024 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| opendevmeet | The meeting name has been set to 'nova' | 16:01 |
| bauzas | hey folks | 16:01 |
| tkajinam | o/ | 16:01 |
| elodilles | o/ | 16:01 |
| fwiesel | o/ | 16:02 |
| gibi | o/ | 16:02 |
| Uggla_ | o/ | 16:02 |
| bauzas | okay, let's softly start | 16:02 |
| bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | 16:02 |
| bauzas | #topic Bugs (stuck/critical) | 16:03 |
| bauzas | #info No Critical bug | 16:03 |
| bauzas | #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | 16:03 |
| bauzas | anything important to tell about bugs ? | 16:03 |
| bauzas | looks not | 16:03 |
| bauzas | #topic Gate status | 16:04 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:04 |
| bauzas | #link https://etherpad.opendev.org/p/nova-ci-failures-minimal | 16:04 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status | 16:04 |
| bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:04 |
| bauzas | #info Please try to provide meaningful comment when you recheck | 16:04 |
| tkajinam | just fyi. global u-c was updated and sqlalchemy was bumped to 2.0. I don't expect any problems with core services since sqlahcmely 2.0 has been tested for some time, but we may need to pay some attention to gate. | 16:05 |
| * bauzas still awaits Zull to provide me the peridic jobs | 16:05 | |
| bauzas | okay, all of the perodics are green, even the nova-emulation one | 16:05 |
| dansmith | o/ | 16:05 |
| sean-k-mooney | the emultaion one was just being OMM killed | 16:06 |
| sean-k-mooney | so it can pass currently i have a patch to correct that | 16:06 |
| bauzas | oh, for the moment, the latest master job for nova-emulation was still failing | 16:06 |
| bauzas | we'll see it next week then | 16:06 |
| bauzas | do people want to discuss about the guest kernel issues now or not ? | 16:07 |
| sean-k-mooney | just an fyi https://review.opendev.org/c/openstack/nova/+/918464 is the fix for the emulator job | 16:08 |
| sean-k-mooney | regarding panics | 16:08 |
| sean-k-mooney | im not sure i have seen them get any worse but i know dansmith wants to try reverting the way we define the glance images | 16:08 |
| sean-k-mooney | dansmith: did you have time to push a patch for that | 16:08 |
| bauzas | okay | 16:08 |
| dansmith | no, I need to do that still | 16:09 |
| sean-k-mooney | ack if you dont get around to it by the next meeting ill push a patch to do it | 16:09 |
| bauzas | kk | 16:10 |
| bauzas | moving on then | 16:11 |
| bauzas | #topic Release Planning | 16:12 |
| bauzas | #link https://releases.openstack.org/dalmatian/schedule.html | 16:12 |
| bauzas | #link https://review.opendev.org/c/openstack/releases/+/918422 nova specific schedule is proposed | 16:12 |
| bauzas | I saw sean-k-mooney's comment, I'll provide a new revision | 16:12 |
| bauzas | (but fwiw, we discussed it on the PTG) | 16:12 |
| bauzas | #info Dalmatian-1 this week | 16:12 |
| bauzas | #info Spec review day is today | 16:12 |
| bauzas | I started to review some specs and I saw also some other comments | 16:13 |
| bauzas | thanks to the folks that already started to review <3 | 16:13 |
| bauzas | #topic Review priorities | 16:13 |
| bauzas | #link https://etherpad.opendev.org/p/nova-dalmatian-status | 16:13 |
| bauzas | tomorrow, I'll modify the etherpad by adding the new series related to the specs that will be merged today | 16:14 |
| bauzas | #topic Stable Branches | 16:15 |
| bauzas | elodilles: hehe | 16:15 |
| bauzas | \o | 16:16 |
| elodilles | ~o~ | 16:16 |
| elodilles | #info all stable gates are OK (stable/2024.1, stable/2023.2, stable/2023.1) as grenade jobs are fixed and passing | 16:16 |
| elodilles | #info stable nova versions released: nova 28.1.0 (2023.2 Bobcat), nova 27.3.0 (2023.1 Antelope) | 16:16 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:16 |
| elodilles | that's all for today :) | 16:16 |
| bauzas | cool | 16:19 |
| bauzas | #topic vmwareapi 3rd-party CI efforts Highlights | 16:19 |
| elodilles | cool indeed | 16:19 |
| bauzas | fwiesel: anything to tell ? | 16:19 |
| fwiesel | Not much... | 16:19 |
| fwiesel | #info Sync script in place. CI working through the backlog. | 16:20 |
| sean-k-mooney | is https://review.opendev.org/c/openstack/nova/+/910627 the next patch on your review list | 16:20 |
| fwiesel | So, if you do not see any messages, it is because of that. | 16:20 |
| fwiesel | sean-k-mooney: Yes, that would be the next one. That should cut down the errors to one, IRC. | 16:20 |
| bauzas | ok | 16:21 |
| fwiesel | That would be from my side, unless there are more questions. | 16:21 |
| bauzas | not from me | 16:21 |
| bauzas | cool | 16:21 |
| bauzas | and thanks | 16:21 |
| bauzas | #topic Open discussion | 16:22 |
| bauzas | I have one item | 16:22 |
| sean-k-mooney | i also have one but i can wait | 16:22 |
| bauzas | next Tuesday, I'll be on a train when the meeting starts | 16:22 |
| bauzas | (going to the OpenInfra Day in Paris) | 16:22 |
| bauzas | so anyone happy to lead the meeting ? | 16:22 |
| bauzas | if not, we'll just punt it to the next week | 16:23 |
| JayF | Hey, can | 16:23 |
| JayF | whoops, one thing | 16:23 |
| JayF | Can https://blueprints.launchpad.net/nova/+spec/ironic-guest-metadata be re-approved for Dalmation? | 16:23 |
| bauzas | JayF: you're next in queue after sean-k-mooney's point :) | 16:23 |
| bauzas | we're on meeting now :) | 16:24 |
| bauzas | so, anyone volunteering for running the meeting next week ? | 16:24 |
| sean-k-mooney | bauzas: if we dont have anything urgent i think we can skip next week | 16:24 |
| bauzas | no worries, then we'll skip it | 16:24 |
| sean-k-mooney | lets do JayF point first | 16:24 |
| sean-k-mooney | im fine with reappoving that | 16:24 |
| bauzas | #agreed next nova meeting on May 21 is CANCELLED | 16:24 |
| bauzas | #action bauzas to communicate about the cancellation | 16:25 |
| bauzas | okay, done for my item | 16:25 |
| bauzas | next one then, Jay | 16:25 |
| andrewbonney | One query on bugs if I could too after others | 16:25 |
| sean-k-mooney | andrewbonney: sure. JayF do you want to summerise | 16:25 |
| bauzas | JayF: I don't see anything controversial with reapproving https://blueprints.launchpad.net/nova/+spec/ironic-guest-metadata as specless | 16:25 |
| JayF | thank! | 16:26 |
| JayF | *thanks | 16:26 |
| bauzas | most of my concerns come with testing this, but this is not related to the blueprint itself | 16:26 |
| bauzas | anyone having concerns with reapproving ^ as specless ? | 16:26 |
| bauzas | speak now or forever hold your peace | 16:27 |
| bauzas | okay, I'll do the paperwork stamp | 16:27 |
| bauzas | #agreed https://blueprints.launchpad.net/nova/+spec/ironic-guest-metadata reapproved as specless for Dalmatian | 16:27 |
| bauzas | next one, who's this ? sean-k-mooney I think ? | 16:28 |
| sean-k-mooney | so hopefully my item will be jsut as quick | 16:28 |
| sean-k-mooney | https://blueprints.launchpad.net/nova/+spec/eventlet-removal-part-1 | 16:28 |
| sean-k-mooney | can we appove that as specless | 16:28 |
| sean-k-mooney | i mostly have it implmeneted | 16:28 |
| dansmith | the critical bits are the implementation details, | 16:29 |
| sean-k-mooney | there is some feedback form stephen on the implmentaion but i think it will be ready for review next week | 16:29 |
| dansmith | so I think it's reasonable to call this cleanup/maintenance and focus on the actual code, IMHO | 16:29 |
| dansmith | i.e. no API or RPC or DB changes, just internals | 16:29 |
| bauzas | yeah I think we said it's just for tracking | 16:29 |
| bauzas | tracking progress and adding it to our review eyes | 16:29 |
| sean-k-mooney | cool well the patches are avaiable for review here https://review.opendev.org/q/topic:%22eventlet-removal-part-1%22 | 16:29 |
| bauzas | I'm fine with approving it then as specless and I think we even agreed on that at the PTG | 16:30 |
| sean-k-mooney | yep | 16:30 |
| bauzas | okay, | 16:30 |
| bauzas | #agreed https://blueprints.launchpad.net/nova/+spec/eventlet-removal-part-1 accepted as specless blueprint as agreed during the PTG | 16:30 |
| sean-k-mooney | i belive andrewbonney had a question around a bug? | 16:30 |
| sean-k-mooney | assuming there was nothign else on the adgenda | 16:31 |
| andrewbonney | Thanks. Just a procedure thing really. I have a few reports since March and just wanted to check if I should be following up here generally or in meetings for feedback? | 16:31 |
| andrewbonney | I appreciate you get a lot so just trying to do the right thing | 16:31 |
| sean-k-mooney | as in you filed a number of bug reporst since march? | 16:32 |
| andrewbonney | Yeah that's right, https://bugs.launchpad.net/nova/+bugs?field.bug_reporter=andrewbonney | 16:32 |
| sean-k-mooney | and you want to knwo if they have been looked at or how to progress them | 16:32 |
| andrewbonney | Sure. For maybe one I could propose a patch, but generally they need review from those more familiar with the codebase | 16:33 |
| bauzas | andrewbonney: we agreed on previous PTG that we can't really triage the new bugs every week as we lack volunteers and time for that | 16:33 |
| andrewbonney | Sure | 16:33 |
| sean-k-mooney | looking at the first one https://bugs.launchpad.net/nova/+bug/2058725 sound familar and im trying to recall if it was intentional | 16:34 |
| bauzas | unfortunately, we then let the reporters to communicate to us any bug in particular they could consider as critical | 16:34 |
| bauzas | andrewbonney: for the other bugs that are less important, our channel is always open for discussion and you shouldn't be afraid of pinging ourselves | 16:34 |
| gibi | andrewbonney: if you could try to push a test case that reproduces the issue in unit or functional test env would greatly increas the chance that we get to it | 16:35 |
| bauzas | some people usually bug ourselves in this chat, asking either for reviews, help or categorisation | 16:35 |
| opendevreview | Stephen Finucane proposed openstack/nova-specs master: Follow-up for openapi spec https://review.opendev.org/c/openstack/nova-specs/+/919603 | 16:35 |
| andrewbonney | Thanks, just don't want to annoy | 16:36 |
| sean-k-mooney | its fine. https://bugs.launchpad.net/nova/+bug/2065403 sound like you are not running it in a loop until completion | 16:37 |
| sean-k-mooney | although the interaction with -max-count problmeatic | 16:37 |
| andrewbonney | I think the default behaviour does run in a loop, that's where the problem lies. With max-count it doesn't run in a loop as far as I can see | 16:37 |
| bauzas | if people agree, we can close the meeting now and let people to discuss about andrewbonney's bug reports just after | 16:38 |
| sean-k-mooney | sure | 16:38 |
| bauzas | cool | 16:39 |
| bauzas | thanks all then | 16:39 |
| bauzas | #endmeeting | 16:39 |
| opendevmeet | Meeting ended Tue May 14 16:39:38 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:39 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2024/nova.2024-05-14-16.01.html | 16:39 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2024/nova.2024-05-14-16.01.txt | 16:39 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2024/nova.2024-05-14-16.01.log.html | 16:39 |
| opendevreview | Merged openstack/nova master: Upload glance image with --file in ceph job https://review.opendev.org/c/openstack/nova/+/918689 | 16:40 |
| fwiesel | Thanks | 16:41 |
| sean-k-mooney | hum 29147 instnace in batches of 50. i wonder if we are missing a marker/stable order | 16:41 |
| andrewbonney | I was wondering why populate_instance_compute_id relied on exception handling rather than checking for 'node' presence in the initial DB query, but there may be good reasons | 16:45 |
| andrewbonney | The fact there's no DB cursor means it hits the same un-migratable records on each iteration | 16:46 |
| sean-k-mooney | well node here https://opendev.org/openstack/nova/blame/commit/7096423b343ffce9622fd078fc2b3a87fd3386f7/nova/objects/instance.py#L1366 | 16:47 |
| sean-k-mooney | is not coming form the instnace | 16:47 |
| sean-k-mooney | that is looking up the compute node record | 16:48 |
| sean-k-mooney | instnace.node i think is the compute node name | 16:49 |
| dansmith | yeah I don't understand the question.. we're not "relying" on exception handling for anything in that code | 16:49 |
| dansmith | ... and we do loop ... | 16:50 |
| sean-k-mooney | my incliantion is to say that of the 30k instances a large proporation fo those recored are for deleted instnaces? | 16:51 |
| sean-k-mooney | thats a guess | 16:51 |
| andrewbonney | Yes that's right | 16:51 |
| andrewbonney | I think this is only an issue where lots of records are not running instances | 16:51 |
| andrewbonney | Do you end up in a state where the method exits because the first 50 records returned by the DB don't qualify for migration, which triggers the end of the loop | 16:52 |
| andrewbonney | *So | 16:52 |
| sean-k-mooney | so a workaround in that case would be to run nova-manage archive-deleted-rows | 16:52 |
| dansmith | sorry I missed something, what is the issue ? | 16:52 |
| sean-k-mooney | when they run the commadn its printing "50 rows matched query populate_instance_compute_id, 6 migrated" | 16:53 |
| dansmith | oh I see above, you're expecting to be able to run just --max-count $number_of_active_instances ? | 16:53 |
| sean-k-mooney | https://bugs.launchpad.net/nova/+bug/2065403 | 16:53 |
| dansmith | yeah I mean you have to run those things untll they're done | 16:53 |
| dansmith | sometimes we can skip records that are deleted and other times we can't really | 16:54 |
| andrewbonney | We ran without max-count initially as part of a standard upgrade. It completed, but hasn't migrated everything because once the first 50 records are unsuitable it exits | 16:54 |
| andrewbonney | I only realised our first deployment was in this state because I noticed the migration taking ~15 mins plus on a second deployment, because it keeps trying to migrate unsuitable records | 16:54 |
| dansmith | ah, I see because you have a bunch of (hopefully deleted) instances that point to missing compute nodes? | 16:55 |
| andrewbonney | Yeah that's it | 16:55 |
| dansmith | gotcha | 16:55 |
| andrewbonney | Happy to work around it for us, but I doubt we'll be the only ones to hit it | 16:55 |
| sean-k-mooney | so just looking at our docs https://docs.openstack.org/nova/latest/cli/nova-manage.html#db-online-data-migrations | 16:55 |
| sean-k-mooney | we state that if you dont pass --max-count | 16:56 |
| dansmith | yeah, I'm not sure the best way to handle that in the context of how that manage command works | 16:56 |
| sean-k-mooney | we will loop internanlly in batchs of 50 | 16:56 |
| dansmith | yeah, but we don't have marker stuff there to prevent us from looping over the same 50 over and over | 16:56 |
| sean-k-mooney | but if you do pass --max-count you have to loop externally and look at the repsoce code | 16:56 |
| sean-k-mooney | well we dont actully need that | 16:56 |
| dansmith | so I mean, we could just ignore deleted instances, but that means we may have a problem adding a constraint later where this will come up again | 16:57 |
| sean-k-mooney | since we only get instnace that dont have the compute_id set | 16:57 |
| sean-k-mooney | but the issue is we are not settign a sentinel for those that dont match | 16:57 |
| dansmith | right but the problem is that if the 50 are not migratable, we'll never mark them as identified and keey finding them | 16:57 |
| dansmith | yeah | 16:57 |
| sean-k-mooney | so we need to set it to all 0 or something like that | 16:57 |
| dansmith | I mean, that's one way to go, but it's not really reasonable I think | 16:57 |
| dansmith | because that makes it not an FK | 16:57 |
| sean-k-mooney | oh right ya it would fail the FK check | 16:58 |
| sean-k-mooney | can we return the instance uuid as a marker if we add an orderby | 16:58 |
| dansmith | I mean, it's not marked as an FK right now, because we suck at that, but still, it's "wrong" I think | 16:58 |
| dansmith | right, so we could add in marker stuff there, but that's kindof a bigger deal really | 16:59 |
| dansmith | changes the interface of all of these migration routines | 16:59 |
| sean-k-mooney | well it woudl need a new cli arg for a start and it woudl chnage the resonce so not backporable | 16:59 |
| dansmith | one thing we could do is if we get to the bottom of this loop and couldn't migrate any of the things we found, just raise an exception and say "yo dawg, you need to archive your db" | 17:00 |
| dansmith | it's not a CLI thing so much as the interface for these migration functions | 17:00 |
| dansmith | they all return all,hit right now | 17:00 |
| dansmith | I need to be reviewing the monster EE spec, so let's just table this and I | 17:01 |
| dansmith | will do some thinking on it later | 17:01 |
| andrewbonney | Thanks for the time | 17:03 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!