| gibi | melwitt: thanks for noticing that https://review.opendev.org/c/openstack/nova/+/820381 and https://review.opendev.org/c/openstack/nova/+/820549 are related | 08:24 |
|---|---|---|
| gibi | I left comments in the former | 08:24 |
| gibi | We need to figure out which direction is better | 08:25 |
| gibi | bauzas: you could also be interested in ^^ | 08:25 |
| bauzas | morning | 08:30 |
| bauzas | gibi:sure, will look a bit later | 08:30 |
| gibi | bauzas: o/ | 08:30 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/820682 | 10:00 |
| *** redrobot8 is now known as redrobot | 10:20 | |
| bauzas | gibi: OK, so I briefly looked at both https://bugs.launchpad.net/nova/+bug/1952915 and https://bugs.launchpad.net/nova/+bug/1953359 | 10:27 |
| bauzas | I see you made a duplicate | 10:27 |
| gibi | I'm pretty sure the root cause of those bugs are the same | 10:27 |
| gibi | hence the duplication | 10:27 |
| gibi | and we can choose from which fix we want | 10:28 |
| bauzas | so I guess for fixing https://bugs.launchpad.net/nova/+bug/1952915 we need to merge https://review.opendev.org/c/openstack/nova/+/820549 ? | 10:28 |
| bauzas | ahah, I see the difference for both patches | 10:28 |
| gibi | I think both proposed fix fixes the same root cause | 10:28 |
| gibi | but in a different way | 10:29 |
| gibi | I tried to summ it up in https://review.opendev.org/c/openstack/nova/+/820381/4#message-122f30f371867edb0992d6260871f4cb65c047aa | 10:29 |
| bauzas | gibi: yeah I'd prefer your own change | 10:32 |
| gibi | it is a trade off really. Mine is less impact, but the other might fixes other currently unreported erros too, but maybe break something else in the process | 10:32 |
| bauzas | gibi: I provided my thoughts | 10:35 |
| gibi | thanks | 10:36 |
| bauzas | gibi: I'll review your own change later today | 10:41 |
| bauzas | I just merged for the moment the functional test | 10:41 |
| bauzas | (with some comments) | 10:41 |
| gibi | bauzas: OK, I try to extend the test a bit later | 10:42 |
| bauzas | gibi: nah, just open thoughts | 10:42 |
| gibi | not due to your thoughts (havent checked them yet) but do to the fact that the other bug reports two visible issue 1) periodic fails 2) revert resize fails. My func test only covers 1) not 2) | 10:43 |
| gibi | as I was only able to reliable reproduce 1) in my local env | 10:43 |
| gibi | but I saw the 2) too | 10:44 |
| opendevreview | Ilya Popov proposed openstack/nova master: Fix to implement 'pack' or 'spread' VM's NUMA cells https://review.opendev.org/c/openstack/nova/+/805649 | 10:47 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: [doc] propose Review-Priority label for contribs https://review.opendev.org/c/openstack/nova/+/816861 | 10:50 |
| opendevreview | Merged openstack/nova master: Reproduce bug 1953359 https://review.opendev.org/c/openstack/nova/+/820540 | 12:10 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: Extend the reproducer for 1953359 and 1952915 https://review.opendev.org/c/openstack/nova/+/820856 | 12:24 |
| gibi | bauzas, melwitt: ^^ extended my reproducer to cover both bugs | 12:24 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: [rt] Apply migration context for incoming migrations https://review.opendev.org/c/openstack/nova/+/820559 | 12:25 |
| gibi | bajj | 12:26 |
| gibi | bahh | 12:26 |
| gibi | the rebase went sideways | 12:26 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: Extend the reproducer for 1953359 and 1952915 https://review.opendev.org/c/openstack/nova/+/820856 | 12:29 |
| gibi | what am I doing /o\ | 12:29 |
| * gibi needs to slow down | 12:29 | |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: [rt] Apply migration context for incoming migrations https://review.opendev.org/c/openstack/nova/+/820559 | 12:30 |
| *** bhagyashris_ is now known as bhagyashris | 12:31 | |
| opendevreview | Balazs Gibizer proposed openstack/nova master: [rt] Apply migration context for incoming migrations https://review.opendev.org/c/openstack/nova/+/820549 | 12:32 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Extend the reproducer for 1953359 and 1952915 https://review.opendev.org/c/openstack/nova/+/820859 | 12:32 |
| gibi | now it is clean ^^ | 12:34 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: Reproduce bug 1953359 https://review.opendev.org/c/openstack/nova/+/820558 | 12:43 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: [rt] Apply migration context for incoming migrations https://review.opendev.org/c/openstack/nova/+/820559 | 12:43 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/820682 | 13:20 |
| sean-k-mooney | lyarwood: are we still seing issue with the qemu tb_size in the debian jobs? | 13:21 |
| sean-k-mooney | lyarwood: i have not had much time to work on https://review.opendev.org/c/openstack/devstack/+/817075 lately | 13:21 |
| sean-k-mooney | just wondering if that need to be work on again | 13:22 |
| lyarwood | I think both Debian and CentOS stream are working around it by using additional swap | 13:22 |
| sean-k-mooney | i basically need to just uninstall apparmor and i think it will work at that point | 13:22 |
| sean-k-mooney | ok i might see if i can make time to rework it this week but i wont priorties it if it snot currently urgent | 13:23 |
| sean-k-mooney | lyarwood: do we require the cherry-picked form lines for rdo backports? https://review.rdoproject.org/r/c/openstack/nova-distgit/+/37069 | 13:32 |
| lyarwood | sean-k-mooney: there's no hard requirement no | 13:32 |
| sean-k-mooney | or is that just something we require in nova/openstack | 13:32 |
| sean-k-mooney | ok | 13:32 |
| bauzas | gibi: ack thanks for the FUP | 13:34 |
| sean-k-mooney | a simple stable backport if people are about https://review.opendev.org/c/openstack/nova/+/820682 this has already been merged all the way to train and this is just taking it to stein | 13:39 |
| ricolin | stephenfin, FYI, Pain point discussion meeting have scheduled on 1500UTC this Wednesday(Dec. 8th) in https://meet.google.com/xsx-inbw-mos | 13:45 |
| opendevreview | Pierre Libeau proposed openstack/nova master: Nova resize don't extend disk in one specific case https://review.opendev.org/c/openstack/nova/+/820531 | 14:03 |
| bauzas | gibi: I'll have to go off at 4:20pm and should be back hopefully at 5pm our time, in time for the meeting | 15:00 |
| bauzas | gibi: but in case I'm a bit late, could you start ? | 15:00 |
| gibi | bauzas: sure | 15:02 |
| bauzas | thanks | 15:02 |
| bauzas | shouldn't arrive after 5:05pm | 15:03 |
| stephenfin | ricolin: ack, thanks | 15:09 |
| opendevreview | Belmiro Moreira proposed openstack/nova master: Over quota instance resize can give confusing error message https://review.opendev.org/c/openstack/nova/+/820898 | 15:14 |
| bauzas | reminder : nova meeting in 9 mins here at #openstack-nova | 15:51 |
| bauzas | gibi: I'm finally back ;) | 15:51 |
| gibi | bauzas: you were fast :) | 15:51 |
| bauzas | my daughter's teacher forgot about the appointment :p | 15:52 |
| gibi | ups | 15:53 |
| bauzas | well, she said that 41 was a prime number when I told her my age at my birthday :) | 15:58 |
| bauzas | so, she's fine. :p | 15:59 |
| bauzas | anyway, nova meeting starts in secs | 15:59 |
| bauzas | #startmeeting nova | 16:00 |
| opendevmeet | Meeting started Tue Dec 7 16:00:01 2021 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'nova' | 16:00 |
| gibi | o/ | 16:00 |
| bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | 16:00 |
| elodilles | o/ | 16:00 |
| bauzas | hola everybody | 16:00 |
| gmann | o/ | 16:01 |
| pslestang | hey | 16:01 |
| bauzas | ok, looks like we can start | 16:02 |
| bauzas | #topic Bugs (stuck/critical) | 16:02 |
| bauzas | #info No Critical bug | 16:02 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 24 new untriaged bugs (+1 since the last meeting) | 16:02 |
| bauzas | #help Nova bug triage help is appreciated https://wiki.openstack.org/wiki/Nova/BugTriage | 16:02 |
| kashyap | Do we have a lot of bugs coming in for triage? | 16:02 |
| bauzas | the +1 is a new gate bug, but let's discuss it after | 16:03 |
| kashyap | Okay, 24... | 16:03 |
| bauzas | kashyap: we had 3 new bugs this week AFAIK | 16:03 |
| bauzas | sometimes we have 10 | 16:03 |
| bauzas | #link https://storyboard.openstack.org/#!/project/openstack/placement 25 open stories (+0 since the last meeting) in Storyboard for Placement | 16:03 |
| bauzas | anything to discuss about bugs but not the gate ones ? | 16:04 |
| bauzas | looks not | 16:05 |
| bauzas | #topic Gate status | 16:05 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:05 |
| bauzas | so we now have https://bugs.launchpad.net/nova/+bug/1953478 | 16:05 |
| bauzas | I didn't had time to look at the CI | 16:06 |
| bauzas | have someone looked at it ? ^ | 16:06 |
| bauzas | looks not | 16:07 |
| bauzas | okay, I'll try to see whether logstash continues to work | 16:07 |
| bauzas | and I'll tell it's a High bug | 16:07 |
| bauzas | but ok, moving on | 16:08 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status | 16:08 |
| bauzas | this is fun, I got 503 ^ | 16:09 |
| kashyap | bauzas: Sorry, just reading back - the CI issue seems intermittent | 16:09 |
| kashyap | (And I doubt there's much bandwidth for folks to debug intermittent issues :-() | 16:09 |
| bauzas | yeah no worries | 16:10 |
| bauzas | anyway, about placement jobs... | 16:10 |
| bauzas | looks like we have a problem with the Zuul API | 16:10 |
| gibi | yeah zuul dashboard seems to be down | 16:10 |
| bauzas | will ping the infra folks after the meeting | 16:10 |
| bauzas | #action bauzas to ask infra folks why Zuul dashboard returns HTTP503 | 16:11 |
| bauzas | moving on then | 16:11 |
| bauzas | #info Please look at the gate failures, file a bug, and add an elastic-recheck signature in the opendev/elastic-recheck repo (example: https://review.opendev.org/#/c/759967) | 16:12 |
| bauzas | #topic Release Planning | 16:12 |
| bauzas | #info Yoga-2 is due Jan 6th | 16:12 |
| bauzas | #link https://releases.openstack.org/yoga/schedule.html#y-2 | 16:12 |
| bauzas | #info Next spec review day to happen next week on Dec 14th | 16:12 |
| bauzas | sooooo | 16:12 |
| bauzas | spec, spec, spec, folks | 16:12 |
| bauzas | remember that we will have less reviews after Dec 20 | 16:13 |
| bauzas | as some folks are on holidays or off | 16:13 |
| bauzas | so, prepare your specs in advance | 16:13 |
| bauzas | #info Reminder that Spec Freeze deadline is at Yoga-2 | 16:14 |
| bauzas | anything to discuss about it ? | 16:14 |
| bauzas | heh, no | 16:15 |
| kashyap | bauzas: A quick one | 16:15 |
| bauzas | #topic Review priorities | 16:15 |
| bauzas | #undo | 16:15 |
| opendevmeet | Removing item from minutes: #topic Review priorities | 16:15 |
| bauzas | kashyap: sure ? | 16:15 |
| kashyap | bauzas: Can the spec review days be moved somewhere in the 2nd week of Jan? | 16:15 |
| kashyap | Many will be on PTO from late next week... | 16:15 |
| bauzas | kashyap: that's why we have the spec review day on Dec 14th | 16:16 |
| sean-k-mooney | kashyap: yep most should be around on the 14th | 16:16 |
| bauzas | so people could provide new revisions after this day and cores could merge them after | 16:16 |
| kashyap | bauzas: Nod. Okay, you can move on to reviw prio | 16:16 |
| kashyap | sean-k-mooney: I'm off from 14Dec myself | 16:16 |
| bauzas | I mean, before Dec 20 | 16:16 |
| bauzas | but, that said, I'm not against moving the spec freeze deadline after Jan 6th if people really want but, | 16:17 |
| bauzas | keep in mind we already have a good number of accepted bps that we need to review | 16:17 |
| sean-k-mooney | lets see if there is a need after m2 | 16:17 |
| gibi | yeah, we can get back to this on 11th of Jan meeting | 16:18 |
| sean-k-mooney | ya i think we likely will have enough on our plate but we can assess on the 11th | 16:18 |
| bauzas | for the moment, those are the BPs we have for Yoga https://blueprints.launchpad.net/nova/yoga | 16:18 |
| bauzas | but I need to add new BPs that specs were merged during this week | 16:18 |
| sean-k-mooney | yep i also likly need to file 1-2 blueprints for heathchecks for one | 16:19 |
| sean-k-mooney | ill try and do that soon | 16:19 |
| bauzas | actually https://review.opendev.org/q/project:openstack/nova-specs+is:merged tells me that's it for the moment | 16:19 |
| bauzas | so Launchpad is OK | 16:19 |
| sean-k-mooney | ack | 16:19 |
| bauzas | but ok, let's revisite this on Jan 4th and Jan 11th to see whether folks want some exceptions | 16:20 |
| bauzas | this said, I have a point to discuss in this meeting about holidays, btw. | 16:20 |
| bauzas | (in the open discussion section) | 16:21 |
| bauzas | do people agree with it ? we will at least see what's going on with specs after the review day | 16:21 |
| gmann | +1 | 16:22 |
| sean-k-mooney | yep +1 | 16:22 |
| *** artom__ is now known as artom | 16:22 | |
| bauzas | ok, moving on | 16:22 |
| bauzas | #topic Review priorities | 16:23 |
| bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement)+label:Review-Priority%252B1 | 16:23 |
| bauzas | #link https://review.opendev.org/c/openstack/nova/+/816861 bauzas proposing a documentation change for helping contributors to ask for reviews | 16:23 |
| bauzas | I'd appreciate a second core for https://review.opendev.org/c/openstack/nova/+/815373 | 16:24 |
| bauzas | also, | 16:24 |
| gibi | bauzas: I will check that | 16:25 |
| bauzas | no update coming from https://review.opendev.org/c/openstack/nova/+/790519 since more than one month, can we consider to remove the prio ? | 16:25 |
| * bauzas looks at some people he knows ;) | 16:25 | |
| gibi | yepp drop it from prio | 16:25 |
| gibi | we can add it back once the dependecies are sorted out | 16:25 |
| bauzas | yeah this is related to the centos9 devstack effort, right? | 16:26 |
| gibi | im not sure | 16:26 |
| bauzas | from adalee's last comment, looks like it | 16:27 |
| bauzas | lyarwood: around ? | 16:27 |
| gibi | ahh I see, yes | 16:27 |
| lyarwood | yup /me reads | 16:27 |
| bauzas | lyarwood: you could have more context than me on this FIPS job | 16:27 |
| lyarwood | I've not touched the review in a while | 16:28 |
| bauzas | lyarwood: well, looks like there are some deps issues | 16:28 |
| lyarwood | kk well it's on Ade to push it forward tbh | 16:28 |
| bauzas | agreed | 16:30 |
| bauzas | so, I'll drop the review-prio flag | 16:30 |
| bauzas | but I want to make sure there is progress on it | 16:30 |
| bauzas | hopefully my gerrit update will remind Ade there is work to do | 16:30 |
| gmann | there are few tempest one (in the series of those deps) also needs to merged before | 16:31 |
| gmann | which is under review | 16:31 |
| bauzas | yeah looks like there are deps | 16:32 |
| bauzas | quite a massive work | 16:32 |
| gmann | and this is goal proposal in case to know broader pic #link https://review.opendev.org/c/openstack/governance/+/816587 | 16:32 |
| bauzas | haha | 16:32 |
| bauzas | gtk | 16:32 |
| bauzas | thanks gmann | 16:33 |
| bauzas | anyway, I think we discuss about all the subject, next one | 16:34 |
| bauzas | discussed* (dang) | 16:34 |
| bauzas | oh, my change | 16:34 |
| bauzas | https://review.opendev.org/c/openstack/nova/+/816861 is ready for reviews | 16:34 |
| bauzas | I dropped the whole "I'm the owner, help me" thing | 16:34 |
| bauzas | so, this is now only about "I'm non-core and I like this change, so I'd appreciate if cores could review it given I commit myself to review it too" | 16:35 |
| bauzas | good opportunity for review visibility, just sayin' | 16:35 |
| sean-k-mooney | ya i think that is a good chagne | 16:35 |
| gibi | bauzas: I will get back to that too | 16:36 |
| bauzas | thanks | 16:37 |
| sean-k-mooney | if we are more or less happy with it i can start drafting the project config change for it but ill waith until after the spec review day next week to work on it | 16:37 |
| bauzas | the idea is to give visibility on review time for non-cores and giving them trust | 16:37 |
| gmann | I think this is good. and if every non core adding RP as +1 on their patches then we can re-iterate it | 16:37 |
| bauzas | so, if you're non-core and you'd appreciate some visibility on review time, this is for you | 16:38 |
| sean-k-mooney | i think this is more or less like +1 ing your own patch and we can treat it as such | 16:38 |
| gmann | yeah, 'yes i reviewed it and it looks important to me so can core have a look' | 16:38 |
| sean-k-mooney | e.g. you should in general avoid it but you may still want to do it in some cases | 16:39 |
| bauzas | sean-k-mooney: I punted this use case | 16:39 |
| gmann | may be good to mention in that doc for more clarity | 16:39 |
| sean-k-mooney | bauzas: ya that is ok | 16:39 |
| bauzas | comments are then appreciated on the change itself :) | 16:39 |
| bauzas | next one | 16:39 |
| bauzas | #topic Stable Branches | 16:39 |
| bauzas | elodilles: your dog. | 16:40 |
| elodilles | well, this will be short as not so much happening around stable branches | 16:40 |
| elodilles | * stable gates should be OK (though no patches have been merged in the last couple of days) | 16:40 |
| elodilles | that's it ^^^ | 16:40 |
| bauzas | huzzah | 16:41 |
| bauzas | I remember I promised some reviews | 16:41 |
| bauzas | but lacked my homework | 16:41 |
| bauzas | ok, moving on then | 16:42 |
| bauzas | #topic Sub/related team Highlights | 16:42 |
| bauzas | Libvirt (lyarwood) | 16:42 |
| bauzas | nothing special to mention ? | 16:42 |
| lyarwood | Nothing from me this week, FWIW we will need a new owner for this point in the future. | 16:42 |
| lyarwood | I'll talk about that more later | 16:42 |
| bauzas | you're right | 16:43 |
| bauzas | or, | 16:43 |
| bauzas | not sure we still need to have a subteam | 16:43 |
| bauzas | given afaik only lee and me work on this :) | 16:43 |
| lyarwood | Either or, it's up to the remaining team | 16:43 |
| bauzas | who's teamed up ? | 16:43 |
| bauzas | anyway, we could make a call at next meeting | 16:44 |
| lyarwood | I mean the wider Nova team ;) | 16:44 |
| bauzas | ahah | 16:45 |
| bauzas | anyway, last topic | 16:46 |
| bauzas | #topic Open discussion | 16:46 |
| bauzas | (bauzas) Holiday period and nova audience for both reviews and meetings | 16:46 |
| bauzas | so, we'll have less people around in the last weeks of Dec | 16:47 |
| sean-k-mooney | i expect other then perhaps alex there will likely be no? cores around for the last 2 weeks | 16:48 |
| bauzas | I dunnoi | 16:48 |
| gibi | I will be off from 20th | 16:48 |
| bauzas | we can discuss if people want | 16:48 |
| gibi | and back at 10th | 16:48 |
| sean-k-mooney | 17th -> 4th in my case. | 16:49 |
| bauzas | ok, Dec 21 -> Jan 3rd for me | 16:49 |
| sean-k-mooney | i dont think we really need to spend much time on it | 16:49 |
| gmann | I will be on and off during last week dec | 16:49 |
| lyarwood | 17th -> 4th here also | 16:49 |
| sean-k-mooney | i assume we are going to cancel meetings for the last 2 weeks | 16:49 |
| sean-k-mooney | the 21th and 28th | 16:50 |
| bauzas | sean-k-mooney: yeah I was about to propose to cancel the meetings | 16:50 |
| gibi | agree with cancelling those | 16:50 |
| elodilles | (20th -> 4th) | 16:50 |
| gmann | +1 on canceling | 16:50 |
| elodilles | ++ | 16:51 |
| sean-k-mooney | unless there is an object i would suggest just sending a mail to the list to inform those that are not here | 16:51 |
| gmann | 21 and 28 one right? | 16:51 |
| bauzas | correct | 16:51 |
| gmann | k | 16:51 |
| bauzas | ok, looks like we have an agreement | 16:52 |
| bauzas | #agreed Cancel Dec 21th and Dec 28th nova meetings given there are less audience | 16:53 |
| bauzas | I'll write an email about it | 16:53 |
| bauzas | and thanks for providing your time-offs, appreciated. | 16:53 |
| bauzas | we can look at this meeting' notes to remember when people are around | 16:54 |
| bauzas | last item on our agenda, | 16:54 |
| bauzas | (pslestang) Specless BP approval request for soft deleting instance action | 16:54 |
| bauzas | pslestang: your turn | 16:54 |
| pslestang | sure | 16:55 |
| pslestang | so at OVH we do not use the restore instance feature and we would like the instance action do be soft deleted when an instance is soft deleted, this is why we propose to add a configuration option to soft delete instance action on instance soft delete | 16:56 |
| bauzas | this one, I'm OK with a specless BP | 16:56 |
| bauzas | but, | 16:56 |
| bauzas | "We should also implement the possibility to retrieve deleted instance actions as we do for instances." seems like an API change, right? | 16:56 |
| sean-k-mooney | pslestang: so just to be clear the meaning of soft delete upstream is the row is kept in the db but marked as delete | 16:57 |
| sean-k-mooney | so that it can be mared as not deleted again later | 16:57 |
| pslestang | bauzas: I did not dive into the API so need to be checked | 16:57 |
| pslestang | sean-k-mooney: yes | 16:57 |
| bauzas | are we sure that instance-actions table is soft-deletable already ? | 16:57 |
| sean-k-mooney | i tought it was | 16:58 |
| bauzas | I dunno, hence my question | 16:58 |
| pslestang | bauzas: yes it is | 16:58 |
| bauzas | if yes, looks like a config option setting | 16:58 |
| sean-k-mooney | so that if we resotred a soft deleted instance its action woudl be preserved | 16:58 |
| bauzas | if no, this is a db change hence a spec required | 16:58 |
| sean-k-mooney | pslestang: so today we shoudl be soft deleting them already | 16:59 |
| pslestang | the instance_actions table is soft-deletable | 16:59 |
| sean-k-mooney | and the new config option you would instead hard delete them | 16:59 |
| bauzas | sean-k-mooney: yeah I'm assuming we would mark the records soft-deleted in the DB and we would also unmark them when undeleting | 16:59 |
| bauzas | actually, we're at time | 16:59 |
| bauzas | but we can continue off the meeting | 16:59 |
| bauzas | #endmeeting | 17:00 |
| opendevmeet | Meeting ended Tue Dec 7 17:00:09 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:00 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2021/nova.2021-12-07-16.00.html | 17:00 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2021/nova.2021-12-07-16.00.txt | 17:00 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2021/nova.2021-12-07-16.00.log.html | 17:00 |
| gibi | I think there is a reason to keep the actions visible after the instance is deleted | 17:00 |
| lyarwood | fun, I did have another topic | 17:00 |
| lyarwood | A quick heads for upstream folks who might not be aware, I'll be stepping away from OpenStack in the new year from ~Jan 28th, moving to a different project within Red Hat. My focus until then is to close out the various open specs, bugfixes etc that I currently have. | 17:00 |
| pslestang | sean-k-mooney: today when we soft-delete an instance instance actions are not soft-deleted (I suppose to let the operator have an history on a deleted instane) | 17:00 |
| gibi | lyarwood: I will miss you! | 17:00 |
| gibi | lyarwood: but good luck! | 17:01 |
| elodilles | lyarwood: :-o | 17:01 |
| lyarwood | gibi: thanks | 17:01 |
| gibi | and let me know if help needed to close up things | 17:01 |
| sean-k-mooney | pslestang: ah ok then i think that might be a bug then | 17:01 |
| bauzas | lyarwood: sorry, I stopped the meeting on time given i didn't see any other item | 17:01 |
| sean-k-mooney | pslestang: we should keep the records in the db for auit use until they are archived | 17:01 |
| lyarwood | oh I'll be chasing for reviews don't worry about that gibi :) | 17:01 |
| * gibi stopped worrying :D | 17:01 | |
| lyarwood | bauzas: yeah np I didn't post it in the agenda | 17:01 |
| elodilles | lyarwood: same as gibi says: will miss you (especially at stable team!) :( | 17:01 |
| gmann | lyarwood: ohh. it is great help from you in many areas not just nova where we all will miss your contribution. | 17:02 |
| pslestang | sean-k-mooney: agree with that | 17:02 |
| elodilles | lyarwood: good luck with the new position! | 17:02 |
| bauzas | pslestang: sean-k-mooney: I think instance actions be kept when soft-delete isn't a bug but rather made on purpose | 17:02 |
| lyarwood | gmann / elodilles ; thanks both | 17:02 |
| sean-k-mooney | bauzas: pslestang i belive is reproting that currently they are not kept | 17:02 |
| gibi | pslestang, sean-k-mooney: I need the instance actions to remind me that I deleted my instance yesterday hence not finding it | 17:02 |
| bauzas | but I'm not opposed to let operators define this behaviour as opt-out | 17:02 |
| bauzas | pslestang: can you then clarify ? | 17:03 |
| sean-k-mooney | gibi: right so my expection is that the instacne action woudl remain until we do archive-delete-rows or whatever that is called | 17:03 |
| gmann | what is purpose of disabling it via config instead just ignore that action record if not needed ? | 17:03 |
| sean-k-mooney | so that we could retrive them for deleted instance to see who deleted them | 17:03 |
| bauzas | gmann: good question, what's the use case ? | 17:04 |
| gibi | I need to drop, but I will read back tomorrow | 17:04 |
| bauzas | not letting the instance action appearing at the API level ? | 17:04 |
| bauzas | but given you need an instance UUID, I don't get the problem | 17:04 |
| sean-k-mooney | bauzas: you can list delete instanes | 17:04 |
| sean-k-mooney | to get the uuid | 17:04 |
| bauzas | sure, and then you can get the instance actions of that deleted UUID | 17:05 |
| bauzas | which is expected behaviour | 17:05 |
| sean-k-mooney | but pslestang can you confrim when you delete an instance today are the isntance actions deleted from the db | 17:05 |
| bauzas | in order to know why this disappeared | 17:05 |
| gmann | yeah | 17:05 |
| pslestang | the use case is to have the table uniformly soft-deleted because we use a custom tool to archive the data | 17:05 |
| sean-k-mooney | bauzas: i think pslestang is suggeting that is not what happens today | 17:06 |
| sean-k-mooney | pslestang: yes i think i have seen your tool | 17:06 |
| sean-k-mooney | its now in the ops tools repo right | 17:06 |
| bauzas | again, what's the current behaviour ? | 17:06 |
| pslestang | sean-k-mooney: it should be I guess | 17:06 |
| sean-k-mooney | pslestang: i tought ye upstreamed it recently into one of the opendev repos | 17:06 |
| sean-k-mooney | i rembere a mail thread about it | 17:07 |
| pslestang | the current behaviour is: on instance soft delete, instance actions are not soft deleted | 17:07 |
| sean-k-mooney | pslestang: the db rows in the instance action table are deleted? or just not marked as deleted | 17:07 |
| pslestang | when running nova db-archive, all the data ar copied into shadow table | 17:07 |
| bauzas | that | 17:08 |
| pslestang | sean-k-mooney: the db rows in the instance action table are not marked as deleted | 17:08 |
| bauzas | OK, that's then expected behavioour and not a bug | 17:08 |
| sean-k-mooney | ok and you just want them to be marked as deleted when the new config opiton is set | 17:08 |
| bauzas | given the records aren't marked as soft-deleted, you can retrieve them thru the API | 17:08 |
| bauzas | sean-k-mooney: yeah he wants the records to not be showable | 17:09 |
| sean-k-mooney | yes you should be able too | 17:09 |
| pslestang | and on nova db purge the data are remove from sahdow table and there is a if in the code for instance action to rely on updated_at instead of deleted_at | 17:09 |
| pslestang | sean-k-mooney: yes that's it | 17:09 |
| bauzas | he prefers data consistency over API | 17:09 |
| pslestang | bauzas: exact | 17:10 |
| bauzas | well, 'he' being OVH, not pslestang I guess :) | 17:10 |
| sean-k-mooney | well its not really a consitency issue | 17:10 |
| sean-k-mooney | they are two differnece reseoucces | 17:10 |
| sean-k-mooney | the instnace and the instnace actions | 17:10 |
| bauzas | pslestang: do you understand that you won't be able to investigate an instance deletion thru the API if you mark the action records as soft-deleted ? | 17:10 |
| sean-k-mooney | the sate of the instnace does not modify the sate of the actions | 17:10 |
| bauzas | sean-k-mooney: agreed | 17:11 |
| bauzas | sean-k-mooney: this is two different models but, | 17:11 |
| bauzas | OVH wants their script to be simplier | 17:11 |
| sean-k-mooney | bauzas: the way around that would be to extend the api to allow a --delete | 17:11 |
| sean-k-mooney | like we do for instnace list | 17:11 |
| sean-k-mooney | in a new microverion | 17:11 |
| pslestang | bauzas: yes we know but this is also the point you mention earlier to be able to retrieve instance action for a deleted instance | 17:11 |
| bauzas | sean-k-mooney: that's exactly why I said in the meeting that I'm opposed to this be a specless BP if we touch the API | 17:11 |
| bauzas | the scope of this BP needs to be clarified | 17:12 |
| sean-k-mooney | bauzas: right so if we want to change the api it would need to be a spec | 17:12 |
| gmann | yeah, I think it is good to add spec and then we can discuss all API or DB change needed | 17:12 |
| pslestang | bauzas: ok, can we proceed in 2 steps? First one would be to add a config option to soft delete instance action if I understand well we could do it as a specless BP | 17:13 |
| bauzas | actually I said I was OK with a specless BP for just the config flag, but,n | 17:13 |
| sean-k-mooney | ok so i dont think we are against making this change at a high level but just want a spec to explain exactly what the behavior shoudl be | 17:13 |
| bauzas | there are interop concerns | 17:13 |
| bauzas | if one cloud stops reporting instance actions for a deleted instance and one reporting them | 17:14 |
| sean-k-mooney | there are yes it woudl be config dirven api behaivor | 17:14 |
| pslestang | the second will require a spec to add a --delete flag | 17:14 |
| bauzas | we somehow need to make the API public that is a behavioural change, right? | 17:14 |
| sean-k-mooney | i think we would have to either do this always when a server is delete with the new microverion or never | 17:14 |
| bauzas | I'm not an API interop expert | 17:14 |
| sean-k-mooney | so new microverion to make instnace action soft deleted and then allow you to list with --deleted | 17:15 |
| bauzas | and I don't know how we treat config-driven API behaviours | 17:15 |
| gmann | yeah, I am not sure if doing it with config is good idea. means API config based behavior is not good | 17:15 |
| sean-k-mooney | and if you use an old microverion for instnace action it should ignore the deleted field | 17:15 |
| sean-k-mooney | gmann: i think we need a microverion too | 17:16 |
| sean-k-mooney | not a config option | 17:16 |
| bauzas | gmann: this is more than a microversion problem | 17:16 |
| gmann | yeah | 17:16 |
| gmann | we should avoid config driven API behaviors | 17:16 |
| sean-k-mooney | bauzas: well no for old microverion we would jsut ignore the deleted colum in the instance action table | 17:16 |
| bauzas | OVH wants their clouds to stop reporting instance actions by default | 17:16 |
| sean-k-mooney | so you would get consitent behavior | 17:16 |
| bauzas | sean-k-mooney: the problem is not the API query | 17:16 |
| bauzas | here, I see OVH wanting to change the DB | 17:17 |
| sean-k-mooney | and with the new microversion for server delete and instance action show we will take it into account and intoduce the --deleted option to the isntance_action api | 17:17 |
| bauzas | sean-k-mooney: the other way around | 17:17 |
| bauzas | sean-k-mooney: by default, we need to report soft-deleted records even if config changes | 17:18 |
| pslestang | bauzas: we do not want to change the DB just allow the operator do soft delete instance action on instance soft delete | 17:18 |
| sean-k-mooney | bauzas: no you are missundestanding me | 17:18 |
| sean-k-mooney | bauzas: im suggeting not having a config opiton at all | 17:18 |
| gmann | yeah agree with sean-k-mooney on not to have config option | 17:18 |
| bauzas | sean-k-mooney: I understand you, I'm playing devil's advocate with OVH wishes | 17:18 |
| sean-k-mooney | with old micro verions we woudl report soft-deleted instanstnce actions | 17:18 |
| gmann | and do it with new microversion only | 17:19 |
| sean-k-mooney | with new microversion we would filter | 17:19 |
| pslestang | I'm sorry I need to go, hope to be back in 20minutes if you are stille there | 17:19 |
| bauzas | sean-k-mooney: I think pslestang's request is not an API change | 17:19 |
| bauzas | sean-k-mooney: he wants the DB records be marked as "deleted" | 17:19 |
| sean-k-mooney | bauzas: i know but i dont think we can make there change they way they want | 17:19 |
| bauzas | period. | 17:19 |
| sean-k-mooney | bauzas: yep and for the interop reason i dont think we can do that | 17:19 |
| sean-k-mooney | but we can do it with a microverion for server delete | 17:20 |
| gmann | bauzas: yeah but doping it via config option is not good | 17:20 |
| sean-k-mooney | so all new isntance will always be soft deleted | 17:20 |
| bauzas | sean-k-mooney: that's why I'm saying the only way to achieve this is to return the soft-deleted instance actions *either way* | 17:20 |
| gmann | *doing | 17:20 |
| bauzas | so we wouldn't change the API behaviour | 17:20 |
| sean-k-mooney | bauzas: i think that is not semanticly correct behavior in the api | 17:20 |
| bauzas | the DB would change and mark the actions be deleted (if operator wants) but the API would continue to return those results | 17:21 |
| bauzas | sean-k-mooney: that's a good point, I dunno | 17:21 |
| sean-k-mooney | bauzas: i dont think we shoudl make this an operator choice | 17:21 |
| bauzas | sean-k-mooney: in this case, the answer to pslestang is "no, we can't accept your BP" | 17:21 |
| sean-k-mooney | bauzas: we can accpet somehting simiilar | 17:22 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/db/main/models.py#L885-L909 | 17:22 |
| sean-k-mooney | also there is no db change required | 17:22 |
| gmann | specless BP right but we are ok to discuss to do with API change | 17:22 |
| sean-k-mooney | its already marked as soft deleable | 17:22 |
| bauzas | sean-k-mooney: yup, I know | 17:22 |
| bauzas | sean-k-mooney: again, the problem is interop | 17:22 |
| sean-k-mooney | not whith what i porpose | 17:22 |
| sean-k-mooney | if we do this as a spec with an api change there is no interop issue | 17:23 |
| bauzas | sean-k-mooneyso, we would soft-delete the actions in a new microversion, OK | 17:23 |
| gmann | with --delete option right and default it shows deleted instance action ? | 17:23 |
| bauzas | sean-k-mooney: but what with 2.1 ? | 17:24 |
| bauzas | gmann: right, that's the problem we need to solve here | 17:24 |
| gmann | so no change for existing users also | 17:24 |
| sean-k-mooney | in 2.1 you would see soft-deleted instance actions | 17:24 |
| sean-k-mooney | in 2.100 or whatever you would not | 17:24 |
| sean-k-mooney | unless you pass --deleted | 17:24 |
| gmann | yes | 17:24 |
| bauzas | sean-k-mooney: what's the difference with what I proposed ? | 17:24 |
| bauzas | you would also have semantically incorrect values for 2.1 | 17:25 |
| sean-k-mooney | in 2.100 you would only see non soft deleted ones by default | 17:25 |
| sean-k-mooney | no you would not | 17:25 |
| sean-k-mooney | in 2.1 we are reflecting the fact that instnace action have a sperate lifetime form the instnace they refer too | 17:25 |
| sean-k-mooney | which were the sematics in 2.1 | 17:25 |
| gmann | with 2.100 with deleted it shows all otherwise only non-soft deleted one | 17:25 |
| sean-k-mooney | in 2.100 we are saying they share the same lifetime | 17:26 |
| sean-k-mooney | in 2.100 soft deleting the instance will soft delete the actions | 17:26 |
| sean-k-mooney | that is a semantic change in the behavior of the api hence need for new microverion | 17:27 |
| bauzas | I can understand this | 17:27 |
| sean-k-mooney | does that make sense | 17:27 |
| bauzas | no config but a change for all | 17:27 |
| sean-k-mooney | yes | 17:27 |
| bauzas | just to be sure, default of 2.100 is 'delete' be always True, right? | 17:27 |
| sean-k-mooney | no | 17:27 |
| bauzas | then, we need to discuss this with ops | 17:28 |
| sean-k-mooney | default fo server delete will be to soft delete the instance actions | 17:28 |
| sean-k-mooney | default for instance_action show would be deleted=false | 17:28 |
| gmann | and we cannot fix it as bug as it is changing API behaviour and so does effect existing users | 17:28 |
| bauzas | I'm pretty sure I can find a handful of operators who wouldn't want to opt-in for v2.100 | 17:28 |
| bauzas | the problem is that if operators want v2.101, they're stuck with shipping 2.100 too | 17:29 |
| bauzas | which they dislike | 17:29 |
| sean-k-mooney | for which endpoint | 17:29 |
| bauzas | the os-instance-actions one, of course | 17:29 |
| sean-k-mooney | if we have a 2.101 for that endpoint | 17:30 |
| sean-k-mooney | the client already needs to be updated to use it | 17:30 |
| sean-k-mooney | so they can just pass deleted=false | 17:30 |
| bauzas | honestly, the more I think, the less I consider it as something good for Nova if we make this general for ops | 17:30 |
| sean-k-mooney | *deleted=true | 17:30 |
| gmann | well that is drawback but applicable for all the changes via microversion | 17:30 |
| bauzas | while on the other hand, the archive problem is solvable with a smarter script | 17:30 |
| sean-k-mooney | the fact we are adding a query arg in 2.100 means they can choose what behvior they want in 2.101 | 17:31 |
| bauzas | that's one thing to let OVH do what they want with their own APIs | 17:31 |
| gmann | sean-k-mooney: I think we can return all deleted one like it is today so that other existing users are not effected and allow --deleted=false to hide deleted actions | 17:31 |
| bauzas | that's another thing to consider that ops don't care about getting information why their instance is deleted | 17:31 |
| sean-k-mooney | gmann: we could do that yes | 17:31 |
| sean-k-mooney | but i think it makes sense to change the default we coudl defer that to a later microverion | 17:32 |
| bauzas | sean-k-mooney: this was my proposal... default be "show-deleted" | 17:32 |
| sean-k-mooney | bauzas: right | 17:32 |
| bauzas | either way, this is a spec | 17:32 |
| sean-k-mooney | but if we are making this change i was suggesting changing the bhavior now instead of having 2 microverions for it | 17:32 |
| sean-k-mooney | if its less contoversion to not change the defualt behavior im not agaisnt that | 17:33 |
| bauzas | honestly, as I said, I'm not super happy with changing the default behaviour | 17:33 |
| sean-k-mooney | provided we never change the default behavior or atleast not in the near future | 17:33 |
| bauzas | (from an ops perspective) | 17:33 |
| sean-k-mooney | bauzas: ok then lets not | 17:33 |
| gmann | yeah. let's do spec. keeping existing behavior as default will be useful | 17:33 |
| sean-k-mooney | ok | 17:33 |
| bauzas | gmann: sean-k-mooney: the fun fact is that this whole conversation starts from a case from OVH coming because they don't wanna change their archive script to be model-specific | 17:34 |
| sean-k-mooney | the thing i want to avoid is two microverions to 1 allow filtering and 2 change the default close toghter | 17:34 |
| sean-k-mooney | if we do the first part and oepraters give feedback we can reconsider the second | 17:34 |
| bauzas | gmann: sean-k-mooney: if we were changing the default, this would mean all ops but OVH would have to modify their own clients to use the new param | 17:35 |
| bauzas | looks unfair, no ? | 17:35 |
| sean-k-mooney | bauzas: actully they do not use our archive feature at all | 17:35 |
| sean-k-mooney | bauzas: no only if they use the latest microverion | 17:35 |
| sean-k-mooney | if they are using microverions correctly that is not going to affect them | 17:35 |
| gmann | yeah and nobody complained on current default behavior so good to keep it | 17:35 |
| sean-k-mooney | its why osc orginally didn not defualt to latest microverion | 17:35 |
| sean-k-mooney | so that the cli would be stable | 17:36 |
| bauzas | gmann: even OVH hasn't complained about the current behaviour, they just complain about the fact their tooling doesn't work with our DB | 17:36 |
| sean-k-mooney | anyway i think we are agred. needs a spec and keep current behviaor by default | 17:36 |
| gmann | bauzas: yeah. | 17:36 |
| gmann | sean-k-mooney: +1 | 17:36 |
| bauzas | sean-k-mooney: even with a spec, I think this is taking a hammer for chasing a firefly | 17:37 |
| sean-k-mooney | bauzas: well as i said ovh do not use our showdown tables anda archiving mechanium | 17:37 |
| bauzas | sean-k-mooney: that's the whole point, they don't use what we provide | 17:37 |
| bauzas | why should we make modifications for them so ? | 17:37 |
| sean-k-mooney | right because not all service implement it | 17:37 |
| bauzas | if it's all about DB persistency | 17:37 |
| sean-k-mooney | there tool work for neutorn glance cinder consitently | 17:37 |
| sean-k-mooney | bauzas: i think ovh and i woudl both be happy if we fully remove the shadow tables | 17:38 |
| bauzas | I still think we're ending into a weird state | 17:39 |
| bauzas | we said in the past "this is expected behaviour, as actions table is here for recording user actions" | 17:39 |
| bauzas | now we're about to publicly express that the actions API is just usable by default for non-deleted instances | 17:40 |
| bauzas | (if we change the default) | 17:40 |
| sean-k-mooney | bauzas: this is what they use https://github.com/ovh/osarchiver/ | 17:40 |
| bauzas | if we don't change the default, we just provide a new microversion that won't change the behaviour by default, which is also weird | 17:41 |
| sean-k-mooney | we often add microverion that dont change default behavior | 17:41 |
| bauzas | and why ? because we consider the current behaviour is enough good to not touch it | 17:41 |
| bauzas | sean-k-mooney: we used microversions for "signaling", I know | 17:41 |
| sean-k-mooney | no | 17:41 |
| sean-k-mooney | the feature this would be interocudeing is the ablity to filter on the soft delete status | 17:42 |
| sean-k-mooney | that does not require use to change the defualt behavior | 17:42 |
| sean-k-mooney | so its not jsut signaling | 17:42 |
| gmann | yeah, adding new filtering which is asked in current specless BP | 17:44 |
| gmann | currently there is no way to filter soft deleted instance actions | 17:44 |
| gmann | with new microversion, we solve both use case 1. existing keep working 2. way to filter the soft deleted one | 17:45 |
| sean-k-mooney | so i think we agree that the current behaiovr is not broken or incorrect, that we could start marking the records as soft deleted when the instance is soft delete but only if we maintain the currnt behavior of returning both soft and not soft deleted records and we woudl also want to allow filtering if we made this change | 17:45 |
| sean-k-mooney | the filtering is an api change as we are adding a new query arg even if the default behvior does not change. | 17:47 |
| pslestang | sean-k-mooney: +1 for your proposition | 17:48 |
| sean-k-mooney | pslestang: by the way what happend with your upstreaming efforts http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020383.html | 17:49 |
| sean-k-mooney | that was the last mail i could find on that trhead was there a tc desicion made | 17:49 |
| sean-k-mooney | i dont see it in https://opendev.org/openstack/osops/src/branch/master so i guess it was not added to osops | 17:50 |
| sean-k-mooney | and i dont see a new repo for https://github.com/ovh/osarchiver/ | 17:51 |
| sean-k-mooney | did ye decide to just keep it in github in the end | 17:51 |
| sean-k-mooney | pslestang: i think the prefernce was option 3 "Move it under its own repository under opendev and propose it as a new official OpenStack project" but i dont think that happened | 17:53 |
| sean-k-mooney | if that was to happen i coudl see a day where nova could discontinue our current shadown tables eventually and rely on OSArchiver after a deprecation period | 17:54 |
| pslestang | sean-k-mooney:this is what I was looking for, exact option 3 prefered but nothing done | 17:54 |
| pslestang | We prefer to put it under opendev than keeping it in github, we really thing that other operators could take benefit of this tool | 17:57 |
| sean-k-mooney | yep and honestly i think its a better approch then we have today in nova | 17:57 |
| sean-k-mooney | so if it was mature enouch and operators started to use it i think we coudl eventually depercante and remove nova archiveal supprot | 17:58 |
| sean-k-mooney | and replace it with your external soltuion once we had time to update the installer tools and work with the differnt sake holders to adopt it | 17:58 |
| sean-k-mooney | the fact it can archive the data to an entirly differnt db or a csv file can signifcalty help with db load on the main db | 17:59 |
| sean-k-mooney | and help keep the main db small | 17:59 |
| pslestang | We use it on a weekly basis (crontab) to clean our DB, and we do not have any issue (except with instance_actions_* ;-)) | 17:59 |
| sean-k-mooney | oh i know it works well for you but for use to ever remove the native support in nova we woudl need all the installaiton tools or at least most of them to supprot your alternivie and we would need an upgrade stragy for exsing clouds | 18:00 |
| sean-k-mooney | so its more maturing that process that would need work | 18:01 |
| sean-k-mooney | cern were intersted in this too if i rememebr the mail thread | 18:01 |
| pslestang | sure I understand | 18:01 |
| pslestang | yes and I know also that ubisoft is using it | 18:03 |
| bauzas | sean-k-mooney: distros would have to agree on it | 18:06 |
| bauzas | this is shipping yet another dep | 18:06 |
| sean-k-mooney | bauzas: it would be yes although we have talked about removing shadow tables in the past and not providing a replacement too | 18:11 |
| sean-k-mooney | this would just be a way to provide a replacement | 18:11 |
| opendevreview | mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935 | 19:08 |
| opendevreview | Merged openstack/nova stable/wallaby: Add functional regression test for bug 1853009 https://review.opendev.org/c/openstack/nova/+/811805 | 22:31 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!