pmatulis | dansmith, yeah, i'm going to change a nova policy to make that available to non-admins | 00:01 |
---|---|---|
*** zhanglong has joined #openstack-nova | 00:01 | |
*** slaweq has joined #openstack-nova | 00:03 | |
*** slaweq has quit IRC | 00:08 | |
*** slaweq has joined #openstack-nova | 00:11 | |
*** mriedem has quit IRC | 00:11 | |
*** jangutter has quit IRC | 00:13 | |
*** jangutter has joined #openstack-nova | 00:13 | |
efried | dansmith: here's a weird one: If I shelve my instance, and then my compute goes down, would a subsequent offload "succeed"? | 00:16 |
efried | The virt driver's destroy() would have to happen when the compute came back up later, just like we reap instances that are deleted when their compute is down. | 00:16 |
efried | But do we do that, or do we just fail the offload and ERROR the instance? | 00:16 |
*** slaweq has quit IRC | 00:16 | |
*** tbachman has quit IRC | 00:16 | |
*** TxGirlGeek has joined #openstack-nova | 00:18 | |
dansmith | efried: offload should succeed when it comes back up, but would fail while the compute is down with just an rpc failure type thing | 00:28 |
efried | okay. thanks. | 00:29 |
dansmith | O dpm | 00:29 |
dansmith | wow. | 00:29 |
dansmith | I don't think there's a local delete sort of thing for offload | 00:29 |
*** brault has quit IRC | 00:29 | |
dansmith | yeah, there's not | 00:29 |
*** brault has joined #openstack-nova | 00:30 | |
efried | okay. | 00:30 |
efried | dansmith: typos courtesy of "shift right"? | 00:30 |
dansmith | yep | 00:30 |
dansmith | efried: also, shelve_offload() is a cast, so no error result or moving instance to ERROR if it fails due to a down compute | 00:30 |
efried | I use dvorak | 00:30 |
efried | so if I shift right, I just look like I'm having a stroke. | 00:31 |
dansmith | heh | 00:31 |
efried | dansmith: that was my next question: offload kinda happens "later" sometimes, right? | 00:31 |
efried | so... does it still have the context (auth etc) of the user who did the original shelve? | 00:31 |
dansmith | it can be requested via the API, or via periodic after a timeout yeah | 00:32 |
dansmith | efried: no it could be the operator via the API, or no context via periodic | 00:32 |
efried | ugh | 00:32 |
dansmith | in fact, I can't think of any reason a regular user would ever manually offload | 00:32 |
efried | make space? | 00:33 |
efried | but anyway, it's another monkey wrench for me. | 00:33 |
dansmith | shelve is what causes them to start saving money, offload is the cache purge for the operator so something else can go there, but shouldn't really impact the user, other than to make unshelve take longer | 00:33 |
*** brinzhang has quit IRC | 00:33 | |
dansmith | right, the user doesn't care to make space, they're already saving the money | 00:33 |
efried | oh, yeah, I guess the space is reclaimed at shelve time. | 00:33 |
*** brinzhang has joined #openstack-nova | 00:33 | |
dansmith | the operator might offload a shelved instance to get the space back before the timer | 00:33 |
dansmith | well, not really, | 00:34 |
efried | shitballs, I need to go back and rewrite this rewrite. | 00:34 |
dansmith | the user doesn't get any space (quota) back, but they do get charged less (theoretically) | 00:34 |
dansmith | the operator doesn't get space (i.e. room for another instance on that host) until the offload | 00:34 |
efried | oh | 00:35 |
efried | allocations etc only go away at offload? | 00:35 |
dansmith | I would think.. the virt driver doesn't know a shelve has happened until the offload | 00:35 |
efried | so yeah, that's what I meant by "make space". But again yeah, that's something the operator cares about, not the user. | 00:35 |
dansmith | right | 00:35 |
efried | the virt driver just gets told to snapshot(). I guess they could figure it out from a task state? | 00:35 |
efried | but yeah, they don't. | 00:35 |
dansmith | well, it would be super problematic if they did, | 00:36 |
*** brinzhang_ has joined #openstack-nova | 00:36 | |
efried | Well, on that note... | 00:36 |
efried | what's the task state during the virt driver's destroy() during the offload? | 00:36 |
dansmith | because the point of the shelved state is that it is still ready to be powered back on, which means that like POWERED_OFF state, it still takes the same amount of resources as ACTIVE | 00:36 |
efried | (I'm going to have to find out anyway, but if you know off the top...) | 00:36 |
dansmith | I dunno, but why? the virt driver shouldn't be implying anything from task_state during an op like that | 00:37 |
dansmith | the only time virt should ever really look at that stuff is when it's recovering and trying to figure out if something should be deleted or ignored or something like that | 00:37 |
efried | I was going to save the vTPM backing file during the offload. But that has to be done by the virt driver, so the virt driver was going to have to have some way of knowing it needed to do that. | 00:37 |
efried | But the fact that the offload is ever (usually) done with no (or admin?) context blows that out of the water. | 00:38 |
efried | I'm going to have to do it during shelve. | 00:38 |
dansmith | task_state is really a lock for the api and downstream services to avoid taking two simultaneous actions on an instance | 00:38 |
dansmith | then you need to tell the virt driver to do it, | 00:38 |
*** tbachman has joined #openstack-nova | 00:38 | |
efried | Which means similarly the virt driver needs to be able to know it should do that during snapshot | 00:38 |
dansmith | not have the virt driver imply that it needs to do it from the state | 00:38 |
dansmith | because, for example, shelve is used under the covers for cross-cell-migration | 00:38 |
efried | so add a kwarg to snapshot()? | 00:38 |
efried | or a new virt driver method, prep_shelve()? | 00:39 |
dansmith | yeah, but also, shouldn't that be the job of the compute manager anyway? | 00:39 |
dansmith | like "give me the tpm so I can save it for you" | 00:39 |
efried | no, can't be. | 00:39 |
efried | oh, that could work, yeah. | 00:39 |
efried | but obv we can't have the compute manager knowing libvirt internals | 00:39 |
*** brinzhang has quit IRC | 00:39 | |
efried | i.e. where that file lives. | 00:40 |
dansmith | no, of course not, but it can ask for the tpm in a portable form | 00:40 |
efried | like... an IMAGE!!! (not) | 00:40 |
dansmith | or even "give me things I should stash for you during a shelve-like snapshot" | 00:40 |
* efried checks the contract for snapshot()'s :return:... | 00:41 | |
efried | oo, nothing today. | 00:41 |
dansmith | then you let compute manager decide whether it is going to save or "lose" those things, because compute manager knows what high-level operation is happening | 00:41 |
dansmith | virt does not really | 00:41 |
efried | So could that be a file-like that compute manager is responsible for streaming from? | 00:42 |
dansmith | and, you let compute manager decide *how* to save those things, instead of teaching every virt driver about all the ways things can be saved (swift, glance, etc) | 00:42 |
dansmith | efried: I would call it a BytesIO yeah | 00:43 |
efried | okay, that can work. | 00:43 |
dansmith | or IOBase or whatever the right bytes-based primitive is | 00:43 |
efried | That's what I meant: "the appropriate streamable thing that I'm going to have to go learn about cause I clearly don't know it yet" | 00:44 |
dansmith | BytesIO for your unit test | 00:44 |
dansmith | file(name, 'rb') for your libvirt from-file implementation | 00:45 |
dansmith | er, open(name, 'rb') | 00:45 |
efried | That's gonna be a tarball that I need to clean up afterward. How do I do that? | 00:46 |
dansmith | which is an io.BufferedReader | 00:46 |
efried | (Musing to self) | 00:46 |
dansmith | then just spawn tar and read from the tar pipe, don't create it on disk first | 00:46 |
efried | I imagine there must be a way to just create the ... yeah, pipe. | 00:46 |
dansmith | p = subprocess.Popen('tar c - file1 file2', stdout=subprocess.PIPE) | 00:47 |
dansmith | return p.stdout | 00:47 |
openstackgerrit | Eric Fried proposed openstack/nova-specs master: Spec: Ussuri: Encrypted Emulated Virtual TPM https://review.opendev.org/686804 | 00:48 |
efried | dansmith: thanks for the help, this is gonna make some (more) things easier, again. | 00:49 |
efried | or at least simpler | 00:49 |
efried | And with that, I'm outta here. | 00:50 |
efried | o/ | 00:50 |
*** zhanglong has quit IRC | 00:55 | |
melwitt | TheJulia: around? | 00:55 |
*** zhanglong has joined #openstack-nova | 00:57 | |
*** Liang__ has joined #openstack-nova | 01:01 | |
*** mriedem has joined #openstack-nova | 01:01 | |
*** mriedem has quit IRC | 01:01 | |
*** mriedem has joined #openstack-nova | 01:01 | |
*** abaindur has quit IRC | 01:05 | |
mriedem | efried: fwiw reading scrollback, the libvirt driver does check task_state to flatten rbd images while unshelving i think | 01:05 |
*** abaindur has joined #openstack-nova | 01:06 | |
mriedem | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L9246 | 01:06 |
*** TxGirlGeek has quit IRC | 01:06 | |
mriedem | efried: also btw, immediate shelve offload is the default behavior and there is a context | 01:06 |
mriedem | the periodic or shelveOffload api only kick in if you've configured the computes to not immediately offload | 01:07 |
*** artom has quit IRC | 01:07 | |
mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.shelved_poll_interval | 01:07 |
openstackgerrit | Eric Xie proposed openstack/nova master: Report trait 'COMPUTE_IMAGE_TYPE_PLOOP' https://review.opendev.org/698132 | 01:07 |
mriedem | dansmith: btw, "because, for example, shelve is used under the covers for cross-cell-migration" is not true - it was for the original poc, but not what i have now | 01:08 |
mriedem | parts are similar though | 01:08 |
dansmith | mriedem: well, yeah, sorry I'm collapsing shelve-like snapshot behavior into it | 01:09 |
dansmith | mriedem: and the task_state check down in the rbd stuff is just a total hack, I hope you'd agree | 01:09 |
dansmith | because of the layers of plumbing that would be required | 01:10 |
mriedem | it is | 01:10 |
mriedem | we're the ones that approved it too :) | 01:11 |
dansmith | mriedem: given that a snapshot for migration would be very shelve-like, but a task-state check for shelving to save the tpm would be the exact kind of guessing the virt driver shouldn't be doing because composite operations on top of the primitives can't rely on predictable behavior | 01:11 |
mriedem | yup, case in point https://review.opendev.org/#/c/691991/ | 01:12 |
dansmith | very shelve-like, meaning it would *need* to store the tpm, but wouldn't if the task_state wasn't shelving, so migrations that happened to cross cells would end up with an empty tpm | 01:12 |
dansmith | which is exactly my point | 01:12 |
dansmith | ha yeah | 01:12 |
dansmith | this marks my 12th hour behind the keyboard today | 01:13 |
* dansmith & | 01:14 | |
* mriedem thinks of pretzels | 01:14 | |
*** zhanglong has quit IRC | 01:16 | |
*** zhanglong has joined #openstack-nova | 01:21 | |
mriedem | dansmith: for tomorrow - a compute rpc api method that didn't return anything but now does - version bump? i would think so at least for the behavior change. | 01:24 |
*** zhanglong has quit IRC | 01:33 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug cross-cell resize https://review.opendev.org/698304 | 01:33 |
*** mriedem has quit IRC | 01:40 | |
*** zhanglong has joined #openstack-nova | 01:40 | |
*** gyee has quit IRC | 01:45 | |
openstackgerrit | sean mooney proposed openstack/nova master: Sync ImageMetaPropsPayload fields https://review.opendev.org/698613 | 01:48 |
sean-k-mooney | gibi: efried ^ wanted to get that done before i went to sleep there are only 3 fields that were missing | 01:50 |
sean-k-mooney | anyway night o/ | 01:50 |
*** larainema has joined #openstack-nova | 01:57 | |
*** mdbooth has quit IRC | 02:04 | |
*** Sundar has joined #openstack-nova | 02:04 | |
*** mdbooth has joined #openstack-nova | 02:05 | |
*** boxiang has joined #openstack-nova | 02:06 | |
*** abaindur has quit IRC | 02:30 | |
pmatulis | when implementing a policy override does one need to place /etc/nova/policy.json on just a single compute node? | 02:30 |
*** mdbooth has quit IRC | 02:32 | |
*** tbachman has quit IRC | 02:35 | |
*** zhanglong has quit IRC | 02:37 | |
melwitt | pmatulis: not the compute node, the controller (where nova-api process is running) | 02:45 |
melwitt | policies are an api thing | 02:46 |
pmatulis | melwitt, ah ha right | 02:48 |
*** tbachman has joined #openstack-nova | 02:48 | |
*** Sundar49 has joined #openstack-nova | 02:50 | |
pmatulis | melwitt, this works but do you see anything wrong with it? | 02:54 |
pmatulis | https://paste.ubuntu.com/p/jcq7znKczd/ | 02:54 |
pmatulis | i left the default as a comment | 02:54 |
melwitt | pmatulis: I think you can't have a comment in a json file (but you said it works?). the new entry looks fine to me | 02:56 |
pmatulis | melwitt, yeah works :) | 02:57 |
*** Dinesh_Bhor has joined #openstack-nova | 02:58 | |
melwitt | kewl | 02:58 |
*** Sundar49 has left #openstack-nova | 02:59 | |
*** mkrai has joined #openstack-nova | 03:00 | |
pmatulis | melwitt, so this is odd. i removed /etc/nova/policy.json (it did not exist prior) and now i get error (log: "...stacktrace...cannot find policy.json). i put it back and error goes away | 03:15 |
pmatulis | confirm i can just drop a file in w/o stopping the service? | 03:16 |
melwitt | pmatulis: yeah, it shouldn't need it (all defaults are in the code). I would have thought you could drop it without restarting the service but I haven't tried it before. did you have to restart for it to notice it when you added it or no? | 03:16 |
melwitt | when you added it the first time I mean | 03:17 |
pmatulis | melwitt, no, i did not | 03:19 |
pmatulis | i read that it wasn't necessary | 03:19 |
melwitt | yeah, I do know it tries to read the file each policy check (if it is there) and you are meant to be able to change it on the fly | 03:20 |
melwitt | I don't know why it would error when you remove it. if you pastebin the stacktrace we can at least see where in the code it fails | 03:21 |
openstackgerrit | Guo Jingyu proposed openstack/nova master: Define and catch InstanceAgentNotRunning for set-password failure https://review.opendev.org/661466 | 03:22 |
*** psachin has joined #openstack-nova | 03:35 | |
pmatulis | melwitt, http://paste.ubuntu.com/p/YBVThgRWDh/ | 03:38 |
*** TxGirlGeek has joined #openstack-nova | 03:43 | |
*** udesale has joined #openstack-nova | 03:45 | |
melwitt | ah, built in caching in oslo.policy https://github.com/openstack/oslo.policy/blob/master/oslo_policy/policy.py#L548. guess we have a bug, need to handle that somehow in nova | 03:46 |
melwitt | I didn't find a bug already open for this | 03:46 |
*** Sundar has left #openstack-nova | 03:49 | |
melwitt | opened this https://bugs.launchpad.net/nova/+bug/1856119 | 03:53 |
openstack | Launchpad bug 1856119 in OpenStack Compute (nova) "FileNotFoundError: [Errno 2] No such file or directory: '/etc/nova/policy.json' when policy file removed" [Undecided,New] | 03:53 |
pmatulis | sweet | 03:54 |
*** nweinber has joined #openstack-nova | 04:00 | |
*** bbowen has quit IRC | 04:09 | |
*** bbowen has joined #openstack-nova | 04:10 | |
*** bhagyashris has joined #openstack-nova | 04:22 | |
*** nweinber has quit IRC | 04:46 | |
*** bhagyashris has quit IRC | 05:06 | |
*** brault has quit IRC | 05:13 | |
*** bhagyashris has joined #openstack-nova | 05:13 | |
*** brault has joined #openstack-nova | 05:13 | |
*** bbowen has quit IRC | 05:14 | |
*** bbowen has joined #openstack-nova | 05:14 | |
*** links has joined #openstack-nova | 05:41 | |
*** TxGirlGeek has quit IRC | 05:41 | |
*** tetsuro has quit IRC | 05:42 | |
*** awalende has joined #openstack-nova | 05:48 | |
*** awalende has quit IRC | 05:52 | |
*** tetsuro has joined #openstack-nova | 05:57 | |
*** damien_r has quit IRC | 06:16 | |
*** damien_r has joined #openstack-nova | 06:17 | |
*** pcaruana has joined #openstack-nova | 06:18 | |
*** mkrai has quit IRC | 06:23 | |
*** mkrai_ has joined #openstack-nova | 06:24 | |
*** brinzhang has joined #openstack-nova | 07:03 | |
*** damien_r has quit IRC | 07:06 | |
*** brinzhang_ has quit IRC | 07:06 | |
*** awalende has joined #openstack-nova | 07:15 | |
*** awalende has quit IRC | 07:21 | |
*** lpetrut has joined #openstack-nova | 07:21 | |
*** boxiang has quit IRC | 07:35 | |
*** bhagyashris has quit IRC | 07:43 | |
*** maciejjozefczyk has joined #openstack-nova | 07:47 | |
*** damien_r has joined #openstack-nova | 07:55 | |
*** tesseract has joined #openstack-nova | 07:59 | |
*** dpawlik has joined #openstack-nova | 08:02 | |
*** tkajinam has quit IRC | 08:04 | |
*** brinzhang_ has joined #openstack-nova | 08:05 | |
*** brinzhang has quit IRC | 08:07 | |
gibi | sean-k-mooney: attaching ports with resource request is still not supported. https://docs.openstack.org/api-guide/compute/port_with_resource_request.html | 08:10 |
gibi | sean-k-mooney: for ussuri on my todolist in priority order: live migration, unshelve, cross cell resize and saying something about heal port allocation CLI usage on Stein with train nova-manage in a venv | 08:12 |
*** bhagyashris has joined #openstack-nova | 08:12 | |
gibi | sean-k-mooney: so no, I don't have port attach on my ussuri TODO list | 08:12 |
*** slaweq has joined #openstack-nova | 08:15 | |
*** awalende has joined #openstack-nova | 08:23 | |
*** tosky has joined #openstack-nova | 08:27 | |
*** mkrai_ has quit IRC | 08:32 | |
brinzhang_ | gibi: hello, do you know this spec http://specs.openstack.org/openstack/nova-specs/specs/train/approved/numa-aware-live-migration.html | 08:32 |
brinzhang_ | gibi: I have a question, if the destination server's NUMA topology (e.g. nume_node=2) < source server's NUMA topology (e.g. numa_noed=4) in a instance. If I am living *this* instance, what will be happend? | 08:35 |
*** bhagyashris has quit IRC | 08:36 | |
*** slaweq has quit IRC | 08:36 | |
brinzhang_ | gibi: Context in "Proposed change" the "Third" aspect. "Third, information about the instance’s new NUMA characteristics needs to be generated on the destination (an InstanceNUMATopolgy object is not enough, more on that later)" | 08:36 |
*** corvus has quit IRC | 08:40 | |
brinzhang_ | And efried completed it's status in Train release. notartom works on it. | 08:41 |
*** mkrai_ has joined #openstack-nova | 08:41 | |
*** ralonsoh has joined #openstack-nova | 08:46 | |
*** ircuser-1 has quit IRC | 08:56 | |
*** tonyb has quit IRC | 08:56 | |
*** tonyb has joined #openstack-nova | 08:56 | |
*** ircuser-1 has joined #openstack-nova | 08:57 | |
*** slaweq has joined #openstack-nova | 08:59 | |
*** corvus has joined #openstack-nova | 08:59 | |
*** rpittau|afk is now known as rpittau | 09:00 | |
*** martinkennelly has joined #openstack-nova | 09:08 | |
*** jcosmao has joined #openstack-nova | 09:08 | |
*** jcosmao has left #openstack-nova | 09:08 | |
*** dpawlik has quit IRC | 09:11 | |
*** bhagyashris has joined #openstack-nova | 09:12 | |
*** dpawlik has joined #openstack-nova | 09:13 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: Remove 'nova-xvpvncproxy' https://review.opendev.org/687909 | 09:13 |
*** derekh has joined #openstack-nova | 09:19 | |
*** iurygregory has joined #openstack-nova | 09:31 | |
*** ccamacho has joined #openstack-nova | 09:35 | |
stephenfin | gibi: Morning o/ I've a couple of functional test rework patches here you might be interested in, if you have some free time today https://review.opendev.org/#/c/689181/ | 09:40 |
*** ccamacho is now known as ccamacho|pto | 09:48 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Add unified '_(build|create)_flavor' helper functions https://review.opendev.org/697538 | 09:48 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Add '_delete_server' to 'InstanceHelperMixin' https://review.opendev.org/697539 | 09:48 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: functional: Add '_create_server' helper https://review.opendev.org/697540 | 09:48 |
*** dtantsur|afk is now known as dtantsur | 09:52 | |
*** martinkennelly has quit IRC | 10:01 | |
*** Liang__ has quit IRC | 10:02 | |
*** mmethot has quit IRC | 10:03 | |
*** mmethot has joined #openstack-nova | 10:04 | |
openstackgerrit | Merged openstack/nova master: Add ConfirmResizeTask https://review.opendev.org/637070 | 10:13 |
*** mkrai_ has quit IRC | 10:22 | |
*** martinkennelly has joined #openstack-nova | 10:26 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove unused 'stub_out_db_network_api' https://review.opendev.org/697151 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove db methods for ProviderMethod https://review.opendev.org/697150 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Convert remaining unit tests to neutron https://review.opendev.org/696510 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove 'is_neutron_security_groups' function https://review.opendev.org/696511 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove nova-network security group driver https://review.opendev.org/696512 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Correct some broken VIF tests https://review.opendev.org/696513 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove firewall support https://review.opendev.org/696514 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Copy shared utils from nova-net module https://review.opendev.org/696515 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove final references to nova-network https://review.opendev.org/696516 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove 'MetadataManager' https://review.opendev.org/696748 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Add TODOs to remove security group-related objects https://review.opendev.org/696517 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove dependency on nova-net from fake cache https://review.opendev.org/696749 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Kill it https://review.opendev.org/696518 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove unused exceptions https://review.opendev.org/697149 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove unnecessary 'neutronv2' prefixes https://review.opendev.org/696776 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Rename 'nova.network.neutronv2' -> 'nova.network' https://review.opendev.org/696745 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Make the security group API a module https://review.opendev.org/696746 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove remaining nova-network quotas https://review.opendev.org/697152 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove use of legacy 'FloatingIP' object https://review.opendev.org/697153 | 10:29 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove use of legacy 'Network' object https://review.opendev.org/697154 | 10:29 |
*** avolkov has joined #openstack-nova | 10:33 | |
*** mmethot has quit IRC | 10:34 | |
*** mmethot has joined #openstack-nova | 10:42 | |
*** larainema has quit IRC | 10:50 | |
*** salmankhan has joined #openstack-nova | 10:53 | |
*** salmankhan has joined #openstack-nova | 10:54 | |
*** rcernin has quit IRC | 10:56 | |
*** pcaruana has quit IRC | 11:03 | |
*** pcaruana has joined #openstack-nova | 11:06 | |
*** dtantsur is now known as dtantsur|brb | 11:09 | |
*** shilpasd has joined #openstack-nova | 11:10 | |
*** udesale has quit IRC | 11:14 | |
gibi | stephenfin: ack, added to my queue | 11:16 |
*** mkrai has joined #openstack-nova | 11:35 | |
*** dviroel has joined #openstack-nova | 11:47 | |
*** tbachman has quit IRC | 11:47 | |
*** mmethot has quit IRC | 11:52 | |
*** mmethot has joined #openstack-nova | 11:52 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove dead code https://review.opendev.org/698703 | 11:56 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: db: Drop cells v1 tables https://review.opendev.org/698704 | 11:56 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: db: Drop snapshots table https://review.opendev.org/698705 | 11:56 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: db: Drop tables that were moved to API DB https://review.opendev.org/698706 | 11:56 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: db: Remove placement tables https://review.opendev.org/698707 | 11:56 |
*** mmethot has quit IRC | 11:58 | |
*** belmoreira has joined #openstack-nova | 12:02 | |
*** mkrai has quit IRC | 12:04 | |
*** zhanglong has joined #openstack-nova | 12:04 | |
*** mkrai_ has joined #openstack-nova | 12:04 | |
*** nicolasbock has joined #openstack-nova | 12:06 | |
*** zhanglong has quit IRC | 12:09 | |
*** mmethot has joined #openstack-nova | 12:09 | |
*** mgariepy has joined #openstack-nova | 12:14 | |
*** dpawlik has quit IRC | 12:22 | |
sean-k-mooney | gibi: ok good to know. i need to work on a bug with attch port for numa aware vswitchs | 12:28 |
sean-k-mooney | gibi: being able to attach an sriov port has been a long standing desire, when i fix the numa awareness checking i might look into sriov and port request since they both would have to check and claim resouces although that will have to wait till febuary for me to figure out. | 12:30 |
sean-k-mooney | im planning on spending january finishing the image metatdata prefilter and working on ci so i wont be starting anything new in january | 12:30 |
gibi | sean-k-mooney: ack. if you happen to start port request part of the interface attach, then let me know and I will try to keep a close eye on that commit | 12:36 |
sean-k-mooney | well i dont want anything i do to conflcit with your work | 12:36 |
sean-k-mooney | but if im working on the attachment code anyway ill take a look and see how invovled it would be to do | 12:37 |
gibi | sean-k-mooney: sthanks | 12:43 |
*** dpawlik has joined #openstack-nova | 12:46 | |
*** mkrai_ has quit IRC | 12:57 | |
*** dtantsur|brb is now known as dtantsur | 12:58 | |
*** nweinber has joined #openstack-nova | 13:07 | |
*** bhagyashris has quit IRC | 13:08 | |
openstackgerrit | Merged openstack/nova master: functional: Unify '_wait_until_deleted' implementations https://review.opendev.org/689181 | 13:11 |
openstackgerrit | Merged openstack/nova master: functional: Unify '_build_minimal_create_server_request' implementations https://review.opendev.org/695024 | 13:11 |
*** belmorei_ has joined #openstack-nova | 13:12 | |
*** belmorei_ has quit IRC | 13:13 | |
*** belmoreira has quit IRC | 13:13 | |
*** belmoreira has joined #openstack-nova | 13:13 | |
*** tbachman has joined #openstack-nova | 13:14 | |
openstackgerrit | Merged openstack/nova master: functional: Remove 'get_invalid_image' https://review.opendev.org/697454 | 13:17 |
*** FlorianFa has joined #openstack-nova | 13:22 | |
*** mriedem has joined #openstack-nova | 13:23 | |
*** mmethot has quit IRC | 13:24 | |
efried | brinzhang_: That one might be a better question for stephenfin | 13:29 |
efried | since artom doesn't appear to be here? | 13:29 |
*** mmethot has joined #openstack-nova | 13:30 | |
efried | sean-k-mooney: thanks for that sync-up patch, will review today (barring unforseen) | 13:30 |
efried | mriedem: ah, gtk about the task_state, I'm going to make use of that. | 13:31 |
sean-k-mooney | efried: it has one bug which ill fix shortly. dealing with a customer issue at the momemnt but there were not many filed missing | 13:32 |
mriedem | efried: in the case of the libvirt driver flattening the image on unshelve it's actually checking the vm_state but similar idea | 13:32 |
mriedem | and as discussed with dan in here last night it's also a hack | 13:32 |
mriedem | efried: while you're about, and i know you've got a full plate, but this is a 7 LOC FUP that if you will approve will help unblock several other approved changes in the series https://review.opendev.org/#/c/698028/ | 13:34 |
efried | mriedem: done. | 13:36 |
mriedem | thanks | 13:36 |
openstackgerrit | Merged openstack/nova master: functional: Remove 'api' parameter https://review.opendev.org/697694 | 13:37 |
*** mkrai has joined #openstack-nova | 13:42 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: debug cross-cell resize https://review.opendev.org/698304 | 13:42 |
*** jangutter has quit IRC | 13:51 | |
efried | Nova meeting in 7 minutes in -meeting | 13:53 |
shilpasd | efried: (y) | 13:54 |
*** pcaruana has quit IRC | 13:57 | |
*** CeeMac has joined #openstack-nova | 13:58 | |
*** dpawlik has quit IRC | 13:58 | |
*** nweinber has quit IRC | 14:03 | |
*** jangutter has joined #openstack-nova | 14:05 | |
mriedem | stephenfin: do you have a py38 system? https://bugs.launchpad.net/nova/+bug/1855493 | 14:05 |
openstack | Launchpad bug 1855493 in OpenStack Compute (nova) "py38 unit test failure - test_validate_patternProperties_fails" [Undecided,New] | 14:05 |
mriedem | i know you love bleeding edge fedora | 14:06 |
sean-k-mooney | kasyap like bleeding edge fedora | 14:07 |
sean-k-mooney | mriedem: stephenfin and i tend to use an alternitive operating system for dev although i think stephenfin still runs fedora on his laptop | 14:08 |
mriedem | alternative, like early 90s R.E.M.? | 14:09 |
sean-k-mooney | that is one way do describe ubuntu 18.04 | 14:10 |
sean-k-mooney | which happens to have python 3.8 in its packages becasue i just installed it | 14:10 |
sean-k-mooney | ill run the test and see what happens | 14:11 |
sean-k-mooney | from the bug report it like like this a byte string vs text string issue if im reading that right | 14:13 |
shilpasd | stephenfin: where i can get team photo during shanghai PTG? | 14:15 |
sean-k-mooney | mriedem: so yes it fails as reported | 14:15 |
sean-k-mooney | there are some other warning too | 14:16 |
mriedem | shilpasd: there was a link in the ML | 14:16 |
shilpasd | mriedem: ok, sure will check, thanks | 14:17 |
mriedem | shilpasd: https://www.dropbox.com/sh/1my6wdtuc1hf58o/AACU49pjWxzFNzcZJgjLG8n1a?dl=0 | 14:18 |
shilpasd | mriedem: thanks | 14:19 |
luyao | stephenfin: Are you around? | 14:22 |
gmann | mriedem: stephenfin FYI. there is plan to add py38 unit test job in ussuri testing runtime. - https://review.opendev.org/#/c/693401/ | 14:26 |
gmann | this is hold till V cycle name is decided | 14:26 |
luyao | sean-k-mooney: could you review vpmem live migration spec if you get time, you are familiar with the vpmem feature, so do stephenfin, but he seems not here :) https://review.opendev.org/#/c/695863/ | 14:27 |
*** pmatulis has left #openstack-nova | 14:30 | |
luyao | stephenfin, sean-k-mooney : FYI. the spec is short, an initial patch based on spec is also submitted, I has verified its function on hardware env, still need further polishing | 14:32 |
stephenfin | mriedem: I do. Will try reproduce now | 14:34 |
stephenfin | luyao: I'll try hit that before EOD | 14:34 |
stephenfin | shilpasd: You should ask Kendall Nelson but I've forgotten her IRC nick /o\ | 14:35 |
*** pcaruana has joined #openstack-nova | 14:36 | |
luyao | stephenfin: thanks :D | 14:36 |
*** links has quit IRC | 14:36 | |
efried | stephenfin: it's diablo_rojo. But mriedem found them for shilpasd. | 14:36 |
stephenfin | efried: whoops, missed that | 14:37 |
stephenfin | cool cool | 14:37 |
efried | stephenfin: diablo rojo is spanish for diabhal dearg | 14:37 |
efried | (how's your gaelic :P) | 14:37 |
shilpasd | efried: thanks | 14:37 |
shilpasd | stephenfin: i got it | 14:38 |
shilpasd | thanks | 14:38 |
efried | lyarwood: this is currently the oldest patch on the review dashboard (http://status.openstack.org/reviews/#nova) | 14:39 |
efried | https://review.opendev.org/#/c/270289 | 14:39 |
efried | Is it still a thing? | 14:39 |
efried | oh, I see you restored it recently, so I guess it must be. | 14:39 |
lyarwood | efried: yeah the spec has landed, I was going to ask if I should use a fresh change-id given this is so old and crusty | 14:39 |
lyarwood | efried: hopefully pushing things for review later today once I've worked on some issues with the tempest tests | 14:40 |
openstackgerrit | Guo Jingyu proposed openstack/nova master: Make scheduling more debuggable https://review.opendev.org/698421 | 14:40 |
efried | lyarwood: is there anything useful in the history? | 14:40 |
efried | I assume the rebase will be a train wreck | 14:40 |
lyarwood | efried: not really tbh | 14:40 |
lyarwood | efried: yeah it was | 14:40 |
efried | Then meh, use a new change-id but ref the old one in the commit message | 14:41 |
lyarwood | ack will do | 14:41 |
dansmith | mriedem: for sure | 14:43 |
dansmith | mriedem: theoretically you could handle it in rpcapi but definitely should have a bump imho | 14:44 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move _get_request_group_mapping() to RequestSpec https://review.opendev.org/696541 | 14:44 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move _update_pci_request_spec_with_allocated_interface_name https://review.opendev.org/696574 | 14:44 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Support live migration with qos ports https://review.opendev.org/695905 | 14:46 |
huaqiang | stephenfin: morning | 14:47 |
huaqiang | stephenfin: for https://review.opendev.org/#/c/668656/, can you make a review when you have time | 14:48 |
gibi | efried, mriedem: fixed up your comments in the qos live migration series | 14:48 |
stephenfin | mriedem: Yeah. I can reproduce that too. Will try root out a fix, assuming sean-k-mooney hasn't fixed it yet | 14:48 |
sean-k-mooney | luyao: left a few comments but more or less i think the direction of the spec is good. just grabing something to eat so i wont be around for a the next 30 mins or so | 14:49 |
mriedem | gibi: thanks, much better | 14:49 |
sean-k-mooney | stephenfin: nope havent started was going to take a look later today but have a few other thing to get done first | 14:49 |
*** tbachman has quit IRC | 14:50 | |
*** davee_ has joined #openstack-nova | 14:50 | |
luyao | sean-k-mooney: thanks for your comments:) | 14:54 |
*** ccamacho|pto has quit IRC | 14:54 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Keep pre-commit inline with hacking and fix whitespace https://review.opendev.org/674057 | 14:56 |
stephenfin | sean-k-mooney: Cool | 14:57 |
stephenfin | also, "foo is 'string'" raises a syntax deprecation warning in Python 3, so we should probably merge https://review.opendev.org/#/c/695732/ sooner rather than later | 14:57 |
*** nweinber has joined #openstack-nova | 14:58 | |
huaqiang | :xa | 14:59 |
*** huaqiang has quit IRC | 14:59 | |
*** huaqiang has joined #openstack-nova | 15:00 | |
efried | stephenfin: +2 | 15:01 |
stephenfin | ta | 15:02 |
stephenfin | huaqiang: I've a few other things ahead of it, but I'll try | 15:02 |
*** lbragstad has joined #openstack-nova | 15:04 | |
huaqiang | stephenfin: I see. I especially need comments for if we need to create 'mixed' instance through 'resources:PCPU' and 'resources:VCPU' and specifying the count of dedicated vCPU number in flavor/image or using a CPU mask for them | 15:05 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Don't hardcore Python versions in test https://review.opendev.org/698732 | 15:05 |
stephenfin | efried, mriedem, sean-k-mooney: ^ | 15:05 |
stephenfin | hardcore. FML | 15:06 |
efried | love it | 15:06 |
mriedem | stephenfin: efried: what's wrong with the dict usage for urlencode here? https://review.opendev.org/#/c/695732/2/nova/tests/unit/api/openstack/compute/test_volumes.py | 15:06 |
huaqiang | stephenfin: I know we have had some discussion on these two questions, but there are some other opionions on them | 15:06 |
efried | mriedem: URLs allow repetition of keys; dicts don't. | 15:06 |
efried | mriedem: querystrings are also ordered. | 15:06 |
mriedem | oh guh | 15:06 |
mriedem | didn't even think about the duplicate keys | 15:06 |
efried | So old encoders just kinda went with it | 15:06 |
efried | but good code should be explicit | 15:07 |
*** KeithMnemonic has joined #openstack-nova | 15:07 | |
huaqiang | stephenfin: so I want to reach consitency on these two questions. | 15:07 |
*** henriqueof has quit IRC | 15:08 | |
*** henriqueof1 has joined #openstack-nova | 15:08 | |
KeithMnemonic | Hello mridem. Do you think there will be any movement on the queens/pike cherry picks of your "marker" fix? https://review.opendev.org/#/c/690725/2. I know those are EOL but since the patches were created was not sure if you thought they could still get in? | 15:08 |
KeithMnemonic | argh mriedem: | 15:09 |
awalende | Anyone knows what nova-compute wants to tell me when trying to attach a volume to a new instance on a freshly deployed ubuntu 18.04 compute node?: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' | 15:09 |
*** dtantsur is now known as dtantsur|brb | 15:10 | |
mriedem | KeithMnemonic: you have to ask stable cores | 15:11 |
KeithMnemonic | ok thanks | 15:11 |
mriedem | those branches aren't EOL either, | 15:11 |
mriedem | they are EM = extended maintenance | 15:11 |
mriedem | https://review.opendev.org/#/admin/groups/540,members | 15:11 |
*** jangutter has quit IRC | 15:15 | |
*** artom has joined #openstack-nova | 15:15 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Don't hardcore Python versions in test https://review.opendev.org/698732 | 15:15 |
KeithMnemonic | mriedem is there a preferred way to ping the "stable cores" | 15:15 |
efried | stephenfin: I thought you were going to fix that commit title? | 15:16 |
KeithMnemonic | should i attend the next IRC meeting? | 15:16 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Use reasonable name for provider mapping https://review.opendev.org/698735 | 15:16 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Don't hardcode Python versions in test https://review.opendev.org/698732 | 15:16 |
efried | KeithMnemonic: you might be waiting a while. bauzas dansmith melwitt lyarwood | 15:16 |
stephenfin | efried: yeah, done | 15:16 |
efried | stephenfin: though I rather liked it the way it was before. | 15:16 |
*** huaqiang is now known as huaqiang_ | 15:17 | |
* stephenfin had an unrelated patch on master and wanted to rebase to avoid the dependency on said patch | 15:17 | |
efried | ah | 15:17 |
dansmith | KeithMnemonic: I've reviewed the other backports of this | 15:17 |
dansmith | mriedem: any concern about the queens one to keep me from slamming this? | 15:18 |
mriedem | stephenfin: "hardcore" | 15:18 |
mriedem | filthy | 15:18 |
KeithMnemonic | thanks efried: | 15:18 |
efried | yahyoubetcha | 15:18 |
*** tbachman has joined #openstack-nova | 15:18 | |
mriedem | Intel OpenStack CI is failing on merge conflicts - it must be patching something into the CI | 15:20 |
mriedem | isn't that the one asking for vpmem live migration support? | 15:21 |
stephenfin | mriedem: Can you restore this for me? https://review.opendev.org/#/c/685756/ | 15:21 |
efried | luyao: ---^ | 15:21 |
mriedem | stephenfin: done | 15:21 |
stephenfin | (Gerrit won't let me backport the fixes otherwise) | 15:21 |
stephenfin | ta | 15:21 |
openstackgerrit | Stephen Finucane proposed openstack/nova stable/train: Also enable iommu for virtio controllers and video in libvirt https://review.opendev.org/685756 | 15:21 |
openstackgerrit | Stephen Finucane proposed openstack/nova stable/train: Switch to uses_virtio to enable iommu driver for AMD SEV https://review.opendev.org/698738 | 15:21 |
openstackgerrit | Stephen Finucane proposed openstack/nova stable/train: Create a controller for qga when SEV is used https://review.opendev.org/698739 | 15:21 |
mriedem | dansmith: when is the last time we actually dropped old tables? https://review.opendev.org/#/q/topic:db-cleanup+(status:open+OR+status:merged) | 15:23 |
bbobrov | oh nice | 15:23 |
dansmith | tables? | 15:23 |
bbobrov | so we backport the fixes to train | 15:23 |
dansmith | never? | 15:23 |
mriedem | that's what i thought | 15:23 |
stephenfin | bbobrov: You mean you (SUSE) have done it already | 15:23 |
stephenfin | dansmith: You sit on a throne of LIES | 15:23 |
stephenfin | Search for 'assertTableNotExists' | 15:25 |
mriedem | i guess there are some old migrations that dropped really old tables | 15:25 |
stephenfin | in 'nova/tests/unit/db/test_migrations.py' | 15:25 |
mriedem | 232, 233 | 15:25 |
dansmith | mriedem: what were the tables? | 15:25 |
stephenfin | 346 | 15:25 |
stephenfin | https://review.opendev.org/#/c/698705/1/nova/tests/unit/db/test_migrations.py@234 | 15:25 |
mriedem | dump_*, compute_node_stats, instance_group_metadata | 15:25 |
mriedem | iscsi_targets, volumes | 15:26 |
mriedem | 292 was the most recent table drop migration | 15:26 |
dansmith | oh, gosh of course, from glance and cinder splits | 15:26 |
mriedem | https://review.opendev.org/#/c/166214/ being the most recent | 15:26 |
dansmith | stephenfin: 346 was a column | 15:26 |
*** tbachman has quit IRC | 15:26 | |
mriedem | liberty, 2015 | 15:27 |
stephenfin | ah, that explains why it told me I wasn't allowed drop a column when I was dropping a table. Same heuristic, I guess | 15:27 |
mriedem | the wild west of yore | 15:27 |
stephenfin | I assume there's a reason not to drop tables? | 15:28 |
dansmith | mriedem: liberty was the 292? | 15:28 |
mriedem | dansmith: yeah | 15:28 |
dansmith | mriedem: if so, those tables must've been unused for longer than that | 15:28 |
stephenfin | The big scary comments say don't do it but not why | 15:28 |
mriedem | dansmith: likely since grizzly/folsom after the cinder split | 15:28 |
dansmith | stephenfin: for the same reasons as not dropping columns, but if the code is gone it's different | 15:29 |
dansmith | stephenfin: so the only thing that worries me is doing the models and the migrations at the same time | 15:29 |
dansmith | i.e. I worry about applying the migrations from U to T at runtime (which we should be able to do) and then introspection being mad | 15:30 |
mriedem | https://review.opendev.org/#/c/698705/ (snapshots table) should have likely gone with the 292 migration since it's related to volume snapshots | 15:30 |
dansmith | stephenfin: so if you can do the migrations first and the model drops after and not have anything complain, that'd be a good test | 15:30 |
stephenfin | I'll give that a lash | 15:31 |
stephenfin | Need some TLC anyway, clearly | 15:31 |
stephenfin | *needs | 15:31 |
mriedem | stephenfin: in case you're taking notes, quota tables are in the api db too https://review.opendev.org/#/c/698706/1 | 15:31 |
dansmith | stephenfin: also, I know you're just going for stats, but there's no reason to add a bunch of new migrations each to drop a thing | 15:31 |
dansmith | stephenfin: so, it'd be better if you put the drops all in one patch ahead of these, and then you can remove the models character by character in the following patches if you want | 15:31 |
dansmith | each migration slows down the process of building the schema on a new machine | 15:32 |
*** dave-mccowan has joined #openstack-nova | 15:32 | |
stephenfin | Kewl. I was trying to separate them logically | 15:32 |
stephenfin | didn't think about perf | 15:32 |
stephenfin | do we need to still care about placement-related tables? | 15:33 |
*** bhagyashris has joined #openstack-nova | 15:33 | |
stephenfin | i.e. will the separate placement service ever use nova's DB? | 15:33 |
dansmith | well, I was going to say no, | 15:33 |
dansmith | but... people could easily have just pointed placement at nova and kept going, until some point | 15:34 |
dansmith | has placement had a db migration since they left? | 15:34 |
mriedem | the cutover was stein, you'd use the nova api db during the upgrade, but yeah..that | 15:34 |
mriedem | i think so | 15:34 |
dansmith | either way, probably worth a healthy warning | 15:34 |
mriedem | placement added a couple of blocker migrations | 15:35 |
dansmith | and tbh, if we're being very responsible, we might want a nova-manage delete-old-placement-stuff, which empties the tables and then drop the tables in the next release, contingent on them being empty | 15:35 |
stephenfin | your call | 15:35 |
TheJulia | melwitt: to your cpu arch question, I suspect you can rip out shipping the cpu arch from nova to ironic. Most of the code seems to rely upon stated properties of the machine's cpuarch and not the instance_info's cpu arch which makes sense from a deployment standpoint. | 15:36 |
TheJulia | melwitt: worth trying and making sure ironic doesn't explode with a test change? | 15:36 |
mriedem | dansmith: and what happens if someone doesn't run delete-old-placement-stuff? does the drop migration in V become a blocker migration until delete-old-placement-stuff is run? | 15:36 |
dansmith | mriedem: yeah, the drop just fails because we check for emptiness and raise | 15:37 |
mriedem | and is delete-old-placement-stuff checking for anything or just deleting records? | 15:37 |
dansmith | mriedem: no, just deleting.. figure if you're running a command like that you're opting in | 15:37 |
mriedem | so it's purely a signal to say yes i've migrated the placement tables from nova_api to placement back in stein (or whenever i upgraded) and i'm good to go, | 15:38 |
mriedem | and if not, i can't get to V | 15:38 |
dansmith | right | 15:40 |
dansmith | we could just nuke and drop and say "well, you should have known" | 15:40 |
mriedem | and in V+3 releases someone will be like, "wtf bro!" | 15:40 |
dansmith | but given the original prescription was to use this db as the way to migrate, it'd be super easy for people to have not moved | 15:40 |
mriedem | though thankfully you can rebuild the basic placement db stuff pretty easily as long as you're not using qos ports of cyborg thingies | 15:41 |
mriedem | *or cyborg thingies | 15:41 |
mriedem | ...or blazar... | 15:41 |
mriedem | or whatever VIO hacks into it for out of tree unicorns | 15:42 |
mriedem | it's all very stable | 15:42 |
openstackgerrit | Stephen Finucane proposed openstack/nova stable/train: Revert "docs: Highlight the current broken state of SEV" (partially) https://review.opendev.org/698746 | 15:43 |
*** TxGirlGeek has joined #openstack-nova | 15:45 | |
mriedem | TheJulia: while you're about, i've marked this as incomplete but does it make any sense to you? https://bugs.launchpad.net/nova/+bug/1853389 | 15:49 |
openstack | Launchpad bug 1853389 in OpenStack Compute (nova) "The ipv6 address in network_data.json within configdrive is wrong when using IronicDriver to deploy BM" [Undecided,Incomplete] - Assigned to Eric Lei (leiyashuai) | 15:49 |
mriedem | ok i guess they call https://docs.openstack.org/api-ref/baremetal/#attach-a-vif-to-a-node while creating a server | 15:51 |
mriedem | i'd think ironic would disallow attaching a vif while a node is being provisioned | 15:51 |
mriedem | oh i can't add ironic to this....storyboard | 15:51 |
*** dtantsur|brb is now known as dtantsur | 15:52 | |
*** bhagyashris_ has joined #openstack-nova | 15:54 | |
*** bhagyashris has quit IRC | 15:55 | |
*** Sundar has joined #openstack-nova | 15:55 | |
mriedem | eandersson: i've left some comments in https://bugs.launchpad.net/nova/+bug/1854992 but marked it incomplete | 15:58 |
openstack | Launchpad bug 1854992 in OpenStack Compute (nova) "Frequent instances stuck in BUILD with no apparent failure" [Undecided,Incomplete] | 15:58 |
mriedem | it's not clear to me what you think nova could do besides changing rpc casts to calls and i'm not sure what that would help if rabbit is dead | 15:58 |
*** davee__ has joined #openstack-nova | 15:58 | |
mriedem | e.g. let's say rabbit is f'ed and the compute gets an error back, it needs to use rabbit to update the instance status to ERROR, so you have a chicken and egg problem | 15:58 |
*** bhagyashris_ has quit IRC | 15:59 | |
mriedem | if it's conductor that notices it has direct access to the db so then we could update, but i don't think we're going to be changing the build_and_run_instances cast to a call... | 15:59 |
mriedem | dansmith: maybe you have ideas ^ | 15:59 |
mriedem | conductor could grow a threaded poll per instance that's building or something with a hard timeout? idk | 16:00 |
dansmith | conductor should definitely not imply some instance state like that | 16:00 |
dansmith | if rabbit is effed then all bets are off | 16:01 |
*** mlavalle has joined #openstack-nova | 16:01 | |
sean-k-mooney | mriedem: woudl a reset-state command not just update the statis in the db. i did not think that need to do an rpc to set the instance state | 16:01 |
mriedem | right any attempts here seem like hacks | 16:01 |
mriedem | reset state from the api is direct to the db | 16:01 |
mriedem | so yeah you can reset the state to error or just delete the instance | 16:01 |
mriedem | but they want the instances to automatically go to error when rabbit is down | 16:01 |
sean-k-mooney | ya so woudl that not bypass your chicken and egg problem | 16:01 |
mriedem | or at least have nova log something | 16:02 |
sean-k-mooney | oh right | 16:02 |
mriedem | sean-k-mooney: the admin would still have to manually do the needful | 16:02 |
sean-k-mooney | did i look at this last week. i this for blizzard? | 16:02 |
mriedem | they could have an external background service/script/whatever that polls the servers that are building and if they pass some timeout they just reset to error or nuke them and try again | 16:02 |
mriedem | yes | 16:03 |
*** maciejjozefczyk has quit IRC | 16:03 | |
sean-k-mooney | ok so we were talking about the mandatory flag for rabbitmq | 16:03 |
mriedem | if rabbit is dead b/c it can't keep up though, constantly hammering it is probably not the solution | 16:03 |
sean-k-mooney | right so the mandatory flag woudl raise an error on our side if we could not enqueue the packet which woudl allows us to detect the failure and then either retry with a different host if rabbit was up but the compute node queue was not found or error | 16:04 |
dansmith | trying to work around dead rabbit is like saying nova should be able to do something smart if the db is down | 16:04 |
sean-k-mooney | dansmith: right but there are cases where rabbit is fine but the compute agent topic queue is not present | 16:05 |
*** gyee has joined #openstack-nova | 16:05 | |
dansmith | sean-k-mooney: how is that not a rabbit failure? | 16:05 |
mriedem | dansmith: hear me out, if the db is down, auto switch to etcd and then when the db is back up, just like merge all the etcd stuff back into mysql | 16:05 |
dansmith | I don't mean a failure where we can point to rabbit as the sole responsible party, but for something that causes us to not be able to use rabbit for whatever reason | 16:06 |
dansmith | mriedem: obviously that's the proper solution | 16:06 |
mriedem | \o/ | 16:06 |
mriedem | now we're cooking with robusto! | 16:06 |
sean-k-mooney | dansmith: it has to do with how oslo.messaging works. if rabbit dide or was restarted the way we curretly do a retry of sending the message to rabbit will recreate teh connection to the exchange but not the topic queue | 16:07 |
sean-k-mooney | so when the compute agent reconnect it does not always recreate it queue | 16:07 |
sean-k-mooney | you can fix it by restarting the compute agent | 16:07 |
dansmith | sean-k-mooney: I'm all for fixing bugs in oslo.messaging, but that has nothing to do with us trying to hack around a stuck instance state | 16:07 |
*** dave-mccowan has quit IRC | 16:07 | |
sean-k-mooney | the mandaotry flag woudl allow the conductor to detect that the queue was not present and we could make the compute node as down | 16:07 |
dansmith | jesus lord, pelase no | 16:08 |
sean-k-mooney | dansmith: https://bugs.launchpad.net/oslo.messaging/+bug/1661510 | 16:08 |
openstack | Launchpad bug 1661510 in oslo.messaging "topic_send may loss messages if the queue not exists" [Medium,In progress] - Assigned to Gabriele Santomaggio (gsantomaggio) | 16:08 |
sean-k-mooney | dansmith: well we dont have to mark it as down but it would be nice to at least log the fact | 16:08 |
dansmith | rabbit has split brain, one conductor thinks the compute queue is gone, and starts marking computes down? | 16:08 |
sean-k-mooney | ya marking as down is proably an over reaction | 16:09 |
dansmith | log what we know, not what we guess.. so sure, "hmm, seems like there should be a compute queue here..." | 16:09 |
sean-k-mooney | anyway the oslo change have merged but we are not using them in nova yet | 16:11 |
*** irclogbot_1 has quit IRC | 16:14 | |
*** irclogbot_2 has joined #openstack-nova | 16:15 | |
*** lbragstad has quit IRC | 16:16 | |
*** tbachman has joined #openstack-nova | 16:17 | |
*** tbachman_ has joined #openstack-nova | 16:19 | |
*** tbachman has quit IRC | 16:21 | |
*** tbachman_ is now known as tbachman | 16:21 | |
mriedem | oof https://review.opendev.org/#/c/697694/ ruins my life | 16:27 |
gibi | there is no progress without pain ;) | 16:29 |
mriedem | yeah yeah | 16:30 |
gibi | I will feel the pain when I have to backport functional tests | 16:30 |
mriedem | this is the cross cell thing, so no pain for you there | 16:30 |
mriedem | but i've got a lot of patches to sift through and update | 16:30 |
gibi | yeah I figured out that you now have a nice big merge conflict | 16:31 |
efried | dansmith: we finally got a grenade fail here https://review.opendev.org/692402 | 16:31 |
efried | It was my magic touch | 16:31 |
mriedem | gibi: the merge conflict is trivial, it's the behavior/interface change | 16:31 |
efried | (hopefully it was for the issue we care about) | 16:31 |
melwitt | sean-k-mooney: I read the backscroll about the rabbit thing and I'm wondering, was there not a way that oslo.messaging could have recreated the missing queue? I wonder why restarting our service is the only way? | 16:33 |
mriedem | gibi: after https://review.opendev.org/#/c/695905/ isn't there supposed to be a patch to enable live migration with qos ports in the api? | 16:34 |
mriedem | with a release note and docs update and such? | 16:35 |
*** lbragstad has joined #openstack-nova | 16:35 | |
mriedem | oh nvm i see, "subsequent patches will add support for migration with target host and other edge case like reschedule." | 16:35 |
gibi | mriedem: that patch only supporst the happy path. I'm working on the re-schedule cases | 16:35 |
mriedem | ack | 16:35 |
gibi | mriedem: yeah | 16:35 |
gibi | stay tuned :) | 16:35 |
sean-k-mooney | melwitt: it proably could although we coudl also proably try to do it if we got the error that it was not deliverable | 16:36 |
sean-k-mooney | melwitt: i think they would prefer if we as the client of oslo.messaging did that since they dont want to assume it is correct | 16:36 |
dansmith | efried: okay, did you look for any juiciness? | 16:37 |
dansmith | efried: I can try to look through there later, but I barely even remember what the deal was | 16:38 |
*** damien_r has quit IRC | 16:38 | |
*** Sundar has quit IRC | 16:38 | |
efried | dansmith: I didn't look at all, just continue to be very interested in making that f'in bug go away. | 16:38 |
efried | but I never really understood the original issue, so not sure it would do anybody much good for me to dig. | 16:38 |
*** TxGirlGeek has quit IRC | 16:45 | |
*** tesseract has quit IRC | 16:48 | |
openstackgerrit | Eric Fried proposed openstack/nova-specs master: Spec: Ussuri: Encrypted Emulated Virtual TPM https://review.opendev.org/686804 | 16:52 |
*** TxGirlGeek has joined #openstack-nova | 16:52 | |
efried | stephenfin, gibi: That's ready for your (hopefully final) nod ^ | 16:53 |
gibi | efried: ack | 16:53 |
stephenfin | cool | 16:53 |
efried | This iteration is simpler in terms of weirdnesses | 16:53 |
efried | and fleshes out some of the interplay necessary for object store | 16:53 |
efried | jroll: ^ | 16:53 |
efried | TxGirlGeek: Spec: Ussuri: Encrypted Emulated Virtual TPM https://review.opendev.org/686804 | 16:53 |
*** bhagyashris has joined #openstack-nova | 16:55 | |
TxGirlGeek | @efried Check | 17:00 |
*** Sundar has joined #openstack-nova | 17:01 | |
*** lpetrut has quit IRC | 17:07 | |
*** lpetrut has joined #openstack-nova | 17:08 | |
* efried back in ~3h | 17:08 | |
*** efried is now known as efried_afk | 17:08 | |
*** ociuhandu has joined #openstack-nova | 17:10 | |
*** ociuhandu has quit IRC | 17:10 | |
*** ociuhandu has joined #openstack-nova | 17:10 | |
*** iurygregory has quit IRC | 17:11 | |
*** peeejayz has joined #openstack-nova | 17:12 | |
mriedem | stephenfin: so your change to _build_minimal_create_server_request | 17:13 |
mriedem | to remove the API that's passed in, | 17:13 |
mriedem | that's a problem for any testing doing multi-tenant testing | 17:13 |
mriedem | like https://review.opendev.org/#/c/697331/4/nova/tests/functional/test_scheduler.py | 17:13 |
mriedem | *any test | 17:13 |
mriedem | so to avoid temporarily mutating the self.api in these types of tests we're going to have to add an api kwarg or something | 17:14 |
stephenfin | I thought I did, no? | 17:14 |
stephenfin | looking | 17:14 |
mriedem | nope https://review.opendev.org/#/c/697694/1/nova/tests/functional/integrated_helpers.py@a98 | 17:15 |
stephenfin | okay, that's a good point. Can you -1 that and I'll respin with the api parameter | 17:16 |
stephenfin | I had it for some of the helpers to deal with that but clearly missed this one | 17:16 |
stephenfin | *missed some | 17:16 |
mriedem | same https://review.opendev.org/#/c/694179/2/nova/tests/functional/test_external_networks.py | 17:17 |
mriedem | your change is merged, | 17:17 |
mriedem | which is why i'm in rebase hell | 17:17 |
mriedem | i'll add it when i move onto reworking one of these | 17:18 |
mriedem | https://review.opendev.org/#/q/path:%255Enova/tests/functional/.*+status:open+owner:self+branch:master :( | 17:18 |
mriedem | i guess that won't work for others... | 17:18 |
mriedem | there we go https://review.opendev.org/#/q/path:%255Enova/tests/functional/.*+status:open+owner:mriedem+branch:master | 17:18 |
stephenfin | /o\ sorry | 17:19 |
stephenfin | you're gonna hate https://review.opendev.org/#/c/697537/ so | 17:19 |
stephenfin | I can respin those for you, if you want | 17:19 |
stephenfin | seeing as I broke em | 17:19 |
mriedem | i'll do it | 17:19 |
mriedem | i realize to make an omelette you have to break a few eggs | 17:19 |
mriedem | but these are my faberge eggs | 17:20 |
*** lpetrut has quit IRC | 17:20 | |
*** pmatulis has joined #openstack-nova | 17:25 | |
pmatulis | how do i know what command will provide a known attribute, such as 'OS-EXT-SRV-ATTR:kernel_id'? the 'server show' command does not show this | 17:26 |
mriedem | you're likely not using a high enough microversion | 17:27 |
mriedem | you need >= 2.3 for that | 17:28 |
mriedem | openstack --os-compute-api-version 2.3 server show <server> | 17:28 |
mriedem | https://docs.openstack.org/api-ref/compute/?expanded=show-server-details-detail#show-server-details | 17:28 |
mriedem | tells you the response params and required min versions | 17:28 |
*** peeejayz has quit IRC | 17:29 | |
pmatulis | mriedem, wow ok, i've never encountered that option before. thank you | 17:30 |
*** sapd1 has joined #openstack-nova | 17:30 | |
melwitt | TheJulia: sorry, what did you mean about the cpu_arch? it | 17:30 |
*** iurygregory has joined #openstack-nova | 17:32 | |
melwitt | TheJulia: *it's the opposite, we take the cpu_arch from the ironic node properties to use as an advertised capability of a compute, for scheduling. what is the correct thing for us to do if an ironic node has _no_ cpu_arch in it's node properties? today we are advertising no cpu_arch in that case, which means any glance image with arch specified will not match an arch-less host | 17:32 |
sean-k-mooney | melwitt: do you need a filter to get that behavior? | 17:34 |
melwitt | sean-k-mooney: yeah it's the ImagePropertiesFilter | 17:34 |
sean-k-mooney | oh i assumeed the computecapablitys filter | 17:34 |
melwitt | which is enabled by default | 17:34 |
sean-k-mooney | the reason i was asking is that shoudl move to placemnt at some point | 17:35 |
sean-k-mooney | cpu architure is the perfect thing to model as a trait | 17:35 |
melwitt | I'm having a really hard time finding out what we should do in the case when an ironic node has no cpu_arch specified in it | 17:35 |
sean-k-mooney | well if the value is not set in the hoststate object we could just allow the host | 17:36 |
melwitt | we've been doing the "no match" thing forever, since the ironic driver code was first added. but recently it seems to be a problem | 17:36 |
sean-k-mooney | it may fail but if we dont have the info we cant really do the right thing either way | 17:36 |
melwitt | yeah, that's what I was wondering if we should use arch.ALL if there's nothing in the ironic node | 17:36 |
*** KeithMnemonic has quit IRC | 17:36 | |
sean-k-mooney | well maybe a .ANY would be better | 17:36 |
sean-k-mooney | its a arch is a feild object so .ALL would be the list of all arches | 17:37 |
*** dtantsur is now known as dtantsur|afk | 17:37 | |
melwitt | I thought that's how we would get an "any" behavior is by saying "this node supports all archs" | 17:37 |
*** ociuhandu has quit IRC | 17:37 | |
sean-k-mooney | yes but .ALL is a list not a singel value | 17:38 |
sean-k-mooney | so the types would not match | 17:38 |
sean-k-mooney | well maybe not https://github.com/openstack/nova/blob/master/nova/objects/fields.py#L172 | 17:39 |
*** ociuhandu has joined #openstack-nova | 17:39 | |
melwitt | I was thinking that based on this https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L48 but maybe I'm missing something | 17:40 |
sean-k-mooney | look like . all is https://github.com/openstack/nova/blob/master/nova/virt/arch.py#L57 | 17:40 |
*** salmankhan has quit IRC | 17:42 | |
sean-k-mooney | yes so that function just returns a single value | 17:42 |
sean-k-mooney | i think | 17:43 |
*** bhagyashris has quit IRC | 17:43 | |
sean-k-mooney | oh no you are right | 17:44 |
sean-k-mooney | its arch.ALL https://github.com/openstack/nova/blob/master/nova/conf/scheduler.py#L552-L566 | 17:44 |
melwitt | no, I think you're right. it's choices=arch.ALL so you have to pick one | 17:44 |
sean-k-mooney | oh yes | 17:44 |
sean-k-mooney | its choice not default | 17:44 |
melwitt | you have to set the default, there's no default. which confused me xD | 17:45 |
sean-k-mooney | ya me too, so looking at how its used https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L52-L54 | 17:45 |
sean-k-mooney | if you dont set the config option and its not in the image image_arch will be teh empty stirng or None so it will be Falsy | 17:46 |
melwitt | the thing that's bugging me is, the glance images must have arch set in this NoValidHost case. and I was thinking, if you want anything to match, shouldn't you just not put arch on the glance image props? but I don't know that much about glance images and whether or not that is thing that could make sense | 17:46 |
sean-k-mooney | and we would bailout on line 64 | 17:46 |
*** ociuhandu has quit IRC | 17:46 | |
*** ociuhandu has joined #openstack-nova | 17:46 | |
sean-k-mooney | melwitt: although perhaps that is being done automatically by inspection of the image | 17:47 |
sean-k-mooney | i think with the new image import workflow in glance you can have hooks to add metadta automatically | 17:47 |
sean-k-mooney | so maybe that is what is happening | 17:47 |
melwitt | ohhh.. hm | 17:47 |
sean-k-mooney | i dont know if that is one of the things that can be set by the way | 17:48 |
sean-k-mooney | its just a guess | 17:48 |
melwitt | downstream they were saying that in queens this same scenario worked fine and starting in rocky it fails | 17:48 |
*** rpittau is now known as rpittau|afk | 17:49 | |
melwitt | and I don't find anything changed in the code in like 5 years. so it feels like it has to be a difference in the image props | 17:49 |
stephenfin | lyarwood: It's late now, but think you could hit [1] tomorrow? [1] https://review.opendev.org/#/q/topic:bp/amd-sev-libvirt-support+(status:open+OR+status:merged)+branch:stable/train | 17:49 |
lyarwood | stephenfin: yeah, I might even hit it tonight while watching the results come in. | 17:49 |
sean-k-mooney | melwitt: ya do you know what code path they are hitting is it the return False on line 75 or 104 | 17:50 |
sean-k-mooney | melwitt: i assume 75? | 17:50 |
melwitt | sean-k-mooney: no I don't. I'd have to see if the sosreport is still around to look | 17:51 |
*** lbragstad has quit IRC | 17:52 | |
melwitt | (if debug logging is enabled) | 17:52 |
sean-k-mooney | well from inspection if the image had an arch request and the ironci node did not have detail i think it would fail as written | 17:52 |
sean-k-mooney | removing the metadata form the image should work if they do not set the config option | 17:53 |
sean-k-mooney | but im kind of wondering if https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py#L75 shoudl be configurable | 17:53 |
melwitt | that was my conclusion as well. but I don't have an answer yet about what is going on with the images they're using | 17:53 |
melwitt | configurable ... I don't like the sound of that. | 17:55 |
sean-k-mooney | melwitt: i was thinking a workaround config option for backports only | 17:56 |
*** ircuser-1 has quit IRC | 17:56 | |
sean-k-mooney | incase they need the arch for some other reason to be set in the image | 17:56 |
sean-k-mooney | but i think the correct thing to do would be for them to update the ironic nodes with the arch in that case | 17:57 |
sean-k-mooney | an arm image wont work on an x86 host after all so really we should always know the arch of a compute node | 17:58 |
dansmith | stephenfin: I might have gotten out of sync, but .. is the qga thing just "also broken" in addition to the scsi device one? | 17:58 |
*** ircuser-1 has joined #openstack-nova | 17:58 | |
melwitt | sean-k-mooney: yeah ... I don't really get the use case. it's a scsi deploy and they say it's single arch so for simplicity they don't want to specify the arch | 17:59 |
*** TxGirlGeek has quit IRC | 17:59 | |
melwitt | and apparently they used to be able to prior to rocky, which I don't know how that's possible unless the images had no arch metadata before | 18:00 |
sean-k-mooney | ya i think that is the most likely reason | 18:00 |
*** derekh is now known as derekh_afk | 18:00 | |
dansmith | okay I see the qga was in the original note about being broken too, I had just always thought it was the virtio-scsi thing that was the big deal | 18:00 |
sean-k-mooney | im just going to check my local openstack install | 18:01 |
sean-k-mooney | virtio scisi virtio gpu and qga | 18:01 |
stephenfin | dansmith: Yeah, it was multiple things | 18:02 |
sean-k-mooney | but yes all virtio devices just need the iommu driver opt and they orginally missed a few | 18:02 |
dansmith | stephenfin: yeah I see now | 18:02 |
sean-k-mooney | those patches all meged this moring right | 18:02 |
stephenfin | cool | 18:02 |
sean-k-mooney | melwitt: i just checked my home deployment and by default it looks like the arch does not get added | 18:03 |
sean-k-mooney | melwitt: glance is added os_hash_value and os_hash_algo by default now | 18:03 |
melwitt | sean-k-mooney: arch does not get added to images automatically you're saying? | 18:04 |
sean-k-mooney | correct, so its doing some things on import by default but adding the arch is not one of them | 18:04 |
sean-k-mooney | you might be able to confgiure it but that does not seem to be the defualt upstream behavior | 18:05 |
melwitt | gah, I wonder what happened then | 18:05 |
melwitt | the bug was opened several months ago but was already marked Triaged so we didn't see it until recently. gonna be hard to get answers this far later | 18:06 |
sean-k-mooney | so i was assuming that it might be somethin like https://docs.openstack.org/glance/latest/admin/interoperable-image-import.html#the-image-property-injection-plugin | 18:07 |
*** pmatulis has left #openstack-nova | 18:08 | |
sean-k-mooney | but they would have had to enable it and specyif the arch | 18:08 |
*** psachin has quit IRC | 18:09 | |
sean-k-mooney | i dont see one that use libguestfs or soemthing else to inspec the image and add things like teh arch but it might exists | 18:09 |
*** awalende_ has joined #openstack-nova | 18:09 | |
*** martinkennelly has quit IRC | 18:09 | |
sean-k-mooney | the plugin interfce was added so operators could do things like that or scan for vulnerablity on import ectra | 18:09 |
sean-k-mooney | but its much simpler to assume a human did it | 18:10 |
*** mkrai has quit IRC | 18:12 | |
*** awalende has quit IRC | 18:13 | |
*** awalende_ has quit IRC | 18:13 | |
*** awalende has joined #openstack-nova | 18:14 | |
*** awalende has quit IRC | 18:15 | |
*** awalende has joined #openstack-nova | 18:15 | |
*** aloga has quit IRC | 18:15 | |
*** awalende has quit IRC | 18:20 | |
*** igordc has joined #openstack-nova | 18:21 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Pass the target in os-services APIs policy https://review.opendev.org/676688 | 18:22 |
*** ralonsoh has quit IRC | 18:24 | |
*** ociuhandu has quit IRC | 18:25 | |
*** ociuhandu has joined #openstack-nova | 18:26 | |
dansmith | mriedem: yesterday I asked if there was anything worrying about this queens version such that I shouldn't just fast approve: https://review.opendev.org/#/c/690725/2 | 18:28 |
*** TxGirlGeek has joined #openstack-nova | 18:30 | |
*** ociuhandu has quit IRC | 18:31 | |
mriedem | there is a test below it which needed a simple mod, but the change linked there doesn't have merge conflicts so i don't think so | 18:33 |
mriedem | would have been nice for KeithMnemonic, who has been asking for reviews on these backports for weeks, to have actually reviewed them himself | 18:33 |
mriedem | dansmith: unrelated, but i'd like your 2nd opinion on https://review.opendev.org/#/c/696574/10/nova/objects/instance.py@1239 | 18:34 |
mriedem | it's a refactor but i'm not sure i'm comfortable with where the method is moving *to* | 18:34 |
mriedem | note that "NeutronFixture sugar" is what i add to my xmas cookies to give them a little something extra | 18:38 |
*** awalende has joined #openstack-nova | 18:38 | |
dansmith | mriedem: commented | 18:39 |
mriedem | thanks | 18:41 |
*** awalende has quit IRC | 18:43 | |
sean-k-mooney | efried_afk: gibi stephenfin will ye be able to review https://review.opendev.org/#/c/674072/17 today or tomorrow. artom will likely take this over form me after that as ill be gone. | 18:46 |
sean-k-mooney | or ill pick it back up in the new year but it would be cool to wrap that up before i go on PTO | 18:47 |
*** ociuhandu has joined #openstack-nova | 18:49 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add new default roles in Admin Action API policies https://review.opendev.org/676682 | 18:49 |
*** ociuhandu has quit IRC | 18:51 | |
gmann | johnthetubaguy: ^^ trying without deprecating the rule which should fail the project admin token tests which is why i thought of deprecating the rule. I might not be understanding your comment clearly but I will read it again with fresh mind. | 18:52 |
*** gmann is now known as gmann_afk | 18:53 | |
mriedem | haleyb: do you think there is anything for nova to do with this? https://bugs.launchpad.net/nova/+bug/1827489 | 18:55 |
openstack | Launchpad bug 1827489 in neutron "Wrong IPV6 address provided by openstack server create" [Low,In progress] - Assigned to Brian Haley (brian-haley) | 18:55 |
*** ociuhandu has joined #openstack-nova | 18:59 | |
openstackgerrit | sean mooney proposed openstack/nova master: Sync ImageMetaPropsPayload fields https://review.opendev.org/698613 | 18:59 |
haleyb | mriedem: no, it's really a config issue in the guest | 19:00 |
sean-k-mooney | mriedem: this seam ver config specific. when you enable ipv6 in neutron you have several option includign stateful dhcp6 | 19:00 |
sean-k-mooney | i use dhcp6 for my home cluster and the ip that my vms get are not related to the back at all as far as i can see | 19:01 |
mriedem | ok, invalidated, thanks | 19:02 |
sean-k-mooney | *mac | 19:03 |
*** ociuhandu has quit IRC | 19:03 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: db: Drop tables for removed services, features https://review.opendev.org/698706 | 19:07 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: db: Remove placement tables https://review.opendev.org/698707 | 19:07 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Follow up to I5b9d41ef34385689d8da9b3962a1eac759eddf6a https://review.opendev.org/698028 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize conductor RPC method https://review.opendev.org/637075 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize from the API https://review.opendev.org/637316 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize_at_dest compute method https://review.opendev.org/637630 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Deal with cross-cell resize in _remove_deleted_instances_allocations https://review.opendev.org/639453 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add finish_revert_snapshot_based_resize_at_source compute method https://review.opendev.org/637647 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add RevertResizeTask https://review.opendev.org/638046 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Flesh out RevertResizeTask.rollback https://review.opendev.org/695334 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.opendev.org/638047 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.opendev.org/638048 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional cross-cell revert test with detached volume https://review.opendev.org/695335 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.opendev.org/638268 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add archive_deleted_rows wrinkle to cross-cell functional test https://review.opendev.org/651650 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.opendev.org/614353 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add test_resize_cross_cell_weigher_filtered_to_target_cell_by_spec https://review.opendev.org/695336 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional test for anti-affinity cross-cell migration https://review.opendev.org/661859 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Support cross-cell moves in external_instance_event https://review.opendev.org/658478 | 19:08 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: flatten rbd image during cross-cell move spawn at dest https://review.opendev.org/691991 | 19:08 |
*** mgariepy has quit IRC | 19:08 | |
stephenfin | Thanks for the reviews, dansmith | 19:09 |
stephenfin | sean-k-mooney: I can take a look tomorrow, sure (y) | 19:09 |
*** ociuhandu has joined #openstack-nova | 19:09 | |
dansmith | stephenfin: -1ing the fsck out of that right now | 19:09 |
stephenfin | I was talking about the backports :D | 19:10 |
stephenfin | but do your worst | 19:10 |
stephenfin | I like the pain | 19:10 |
* stephenfin -> 🏠 | 19:10 | |
dansmith | ah, well, then, sure :) | 19:10 |
mriedem | that doesn't render for me but i hope it's a studded paddle | 19:10 |
sean-k-mooney | it is not :( its a boaring house | 19:12 |
mriedem | stephenfin: when you're back about, i had to rebase https://review.opendev.org/#/c/637316/ because of the great api-removeninig of 2019 in the functional tests so can you re-approve that when you get a chance | 19:13 |
dansmith | what is a boaring house? | 19:14 |
mriedem | something that elon musk created | 19:14 |
dansmith | some sort of pig farming facility? | 19:14 |
*** ociuhandu has quit IRC | 19:14 | |
sean-k-mooney | :) you know what i ment | 19:14 |
dansmith | I don't, but I didn't see whatever he was referring to | 19:15 |
sean-k-mooney | stephens unicode house 🏠 | 19:15 |
* dansmith is totes confused | 19:16 | |
sean-k-mooney | which you proably didnt also see there because i just copy pasted it | 19:16 |
dansmith | I see the house, I just don't know what a boaring house is..or what a studded paddle has to do with it | 19:16 |
dansmith | but whatevs | 19:16 |
mriedem | stephen said he liked the pain, | 19:17 |
mriedem | i made an S&M joke | 19:17 |
mriedem | and now we're here | 19:17 |
dansmith | and a boaring house is like a S&M dungeon? | 19:17 |
sean-k-mooney | no its just me misspelling boring | 19:17 |
dansmith | oh | 19:18 |
mriedem | alternative to me waiting for stephen or the gib in the morning to re-approve, if someone else wants to eyeball PS56 to PS57 here and confirm that the only things i changed are what i called out https://review.opendev.org/#/c/637316/57 i'll re-approve and unblock the other 4 or so approved changes after that in the gate | 19:18 |
mriedem | since it takes a couple of days to merge code i'd like to start the recheck bus now | 19:19 |
*** ganso has quit IRC | 19:19 | |
dansmith | 57 was just changes due to rebase? | 19:20 |
mriedem | yes | 19:21 |
sean-k-mooney | im looking at it now so far it looks identical | 19:21 |
mriedem | caused by this https://review.opendev.org/#/c/697694/1 | 19:21 |
*** tosky has quit IRC | 19:21 | |
*** iurygregory has quit IRC | 19:22 | |
openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Add --instance option to heal_allocations https://review.opendev.org/698529 | 19:22 |
openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Add BFV wrinkle to TestNovaManagePlacementHealAllocations https://review.opendev.org/698531 | 19:22 |
*** iurygregory has joined #openstack-nova | 19:23 | |
sean-k-mooney | mriedem: ya so i dont see any change other then the ones you noted | 19:24 |
mriedem | ack thanks sean-k-mooney and dansmith | 19:24 |
mriedem | bottom 6 cross-cell changes are on their way which is nice | 19:25 |
sean-k-mooney | yep it is | 19:25 |
sean-k-mooney | ok im goign to get dinner and likely call it a day. o/ | 19:27 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Do not reschedule on ExternalNetworkAttachForbidden https://review.opendev.org/694179 | 19:28 |
*** ociuhandu has joined #openstack-nova | 19:30 | |
*** artom has quit IRC | 19:30 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Create instance action when burying in cell0 https://review.opendev.org/694165 | 19:30 |
*** ociuhandu has quit IRC | 19:34 | |
*** efried_afk is now known as efried | 19:38 | |
efried | sean-k-mooney: I'll try to get to it in the next couple hours. | 19:38 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add NodeOwnerFilter https://review.opendev.org/697331 | 19:38 |
*** mmethot has quit IRC | 19:40 | |
mriedem | dansmith: in order to avoid a compute service version check workaround thing, what do you think about just checking for vcenter here in U and remove in V? https://review.opendev.org/#/c/695220/3/nova/compute/api.py@3881 | 19:44 |
dansmith | mriedem: seriously? | 19:44 |
mriedem | won't work for any out of tree cluster driver that supports same-host cold migration but....so what? | 19:44 |
mriedem | well, what's the alternative? get the compute service version and if old, fallback to existing behavior? | 19:45 |
mriedem | that was my original thought | 19:45 |
dansmith | I definitely don't want to start a trend of putting virt-specific hacks into upper layers | 19:45 |
mriedem | ok so do what i have documented there as the todo | 19:45 |
mriedem | check service versoin, if old fallback to use the config | 19:46 |
mriedem | remove that in V | 19:46 |
dansmith | yes | 19:46 |
mriedem | alright then | 19:46 |
* mriedem slinks away in shame | 19:46 | |
dansmith | as you should | 19:46 |
dansmith | let it be remembered for all time, the last thing mriedem tried to do was an api-level hack for vmware | 19:47 |
* mriedem borrows the paddle | 19:47 | |
dansmith | can you self paddle? | 19:47 |
mriedem | can a bear shit in the woods? | 19:47 |
dansmith | nevermind, don't answer | 19:47 |
dansmith | heh | 19:47 |
sean-k-mooney | hehe if only this was archive publicly for al time | 20:00 |
sean-k-mooney | *all | 20:00 |
*** ganso has joined #openstack-nova | 20:08 | |
*** iurygregory has quit IRC | 20:08 | |
*** KeithMnemonic has joined #openstack-nova | 20:09 | |
*** tbachman has quit IRC | 20:11 | |
KeithMnemonic | mriedem do you have a few to help me with a different marker issue. this one is in master as well and easily reproducible | 20:11 |
KeithMnemonic | not sure if it is just how horizon handles it or is it something in nova. under instances, there is not a prev/next button that came from this patch https://review.opendev.org/#/c/498018/34 | 20:12 |
KeithMnemonic | s/not/now | 20:12 |
KeithMnemonic | so that patch added a prev button, prior there was only next. but if you jump forward and back a few times you see it does not work as expected | 20:13 |
KeithMnemonic | seems to be due to the original list is not sorted or the sort is not maintained after doing 'next' than 'prev' | 20:14 |
KeithMnemonic | for example i have 5 instances test-1 through test-5. i set the "items per page" under admin settings to 2 | 20:15 |
KeithMnemonic | if i jump fresh to instances i see test-5, test-4, if i click next i get test-2, test-1. if i click prev, i get test test-3 and test-1 | 20:17 |
KeithMnemonic | so horizon is using marker= and prev_marker= and that seems to not work well | 20:17 |
KeithMnemonic | has anyone else seen this? | 20:18 |
mriedem | prev_marker isn't a thing in nova | 20:18 |
KeithMnemonic | so something horizon bonked? | 20:18 |
mriedem | maybe they aren't providing the full query params to maintain a consistent sort order, i.e. not passing the sort_key/sort_dir params when paging? | 20:18 |
*** pcaruana has quit IRC | 20:19 | |
*** abaindur has joined #openstack-nova | 20:21 | |
KeithMnemonic | could be maybe somewhere in here https://opendev.org/openstack/horizon/src/branch/master/horizon/tables/base.py#L699 | 20:23 |
*** abaindur has quit IRC | 20:24 | |
*** abaindur has joined #openstack-nova | 20:24 | |
mriedem | idk | 20:25 |
*** abaindur has quit IRC | 20:25 | |
mriedem | don't really have time to dig into that | 20:25 |
*** abaindur has joined #openstack-nova | 20:25 | |
KeithMnemonic | no problem let me go bother horizon, thanks for confirming | 20:26 |
mriedem | but it does look like they are losing the sort key/dir | 20:26 |
*** belmoreira has quit IRC | 20:27 | |
efried | sean-k-mooney: https://review.opendev.org/#/c/674072/ +A, nice work on those tests. (See inline though.) | 20:29 |
mriedem | a couple of low-hanging bugs if someone wants something to play with | 20:31 |
mriedem | https://bugs.launchpad.net/nova/+bug/1856241 | 20:31 |
openstack | Launchpad bug 1856241 in OpenStack Compute (nova) "Compute API in nova - servers_links should link to paging doc" [Undecided,New] | 20:31 |
mriedem | https://bugs.launchpad.net/nova/+bug/1856240 | 20:31 |
openstack | Launchpad bug 1856240 in OpenStack Compute (nova) "wait_for_versioned_notifications gives unhelpful error message "ValueError: Not a text type application/octet-stream" on timeout" [Undecided,Confirmed] | 20:31 |
*** abaindur has quit IRC | 20:41 | |
*** abaindur has joined #openstack-nova | 20:42 | |
*** abaindur has quit IRC | 20:44 | |
*** lvbin01 has quit IRC | 20:47 | |
*** lvbin01 has joined #openstack-nova | 20:47 | |
*** derekh_afk has quit IRC | 20:48 | |
*** damien_r has joined #openstack-nova | 20:53 | |
jroll | melwitt: re https://bugzilla.redhat.com/show_bug.cgi?id=1688838 - I feel like cpu_arch used to be required, but not 100%. looks like we intentionally made it not required here, at least: https://review.opendev.org/#/c/620634/ | 20:57 |
openstack | bugzilla.redhat.com bug 1688838 in openstack-nova "Ironic should not treat cpu_arch as mandatory" [Medium,New] - Assigned to mwitt | 20:57 |
*** damien_r has quit IRC | 20:59 | |
*** damien_r has joined #openstack-nova | 20:59 | |
openstackgerrit | Eric Fried proposed openstack/nova master: DRY: Build ImageMetaPropsPayload from ImageMetaProps https://review.opendev.org/698803 | 20:59 |
efried | dansmith: question in here ^ for you | 20:59 |
efried | sean-k-mooney: that ^ passes tests :) | 21:00 |
dansmith | efried: if the question is whether they need to be instances or classes, they need to be the former | 21:01 |
dansmith | the instance is what contains things like nullable=, default=, etc | 21:02 |
dansmith | and just because those don't have any of those set (currently) doesn't affect what the rest of the code is expecting | 21:02 |
efried | The question is whether they need to be discrete instances for every field of every object, or whether I could just copy in the instances from the other class. | 21:02 |
efried | like, could I theoretically do this: | 21:03 |
efried | f = IntegerField() | 21:03 |
efried | class MyObj(...): | 21:03 |
efried | fields = { | 21:03 |
efried | int1: f, | 21:03 |
efried | int2: f, | 21:03 |
efried | } | 21:03 |
dansmith | oh, well, I'll have to go look, | 21:03 |
dansmith | but the convention is obviously for each to be new | 21:03 |
dansmith | I can't remember if we stash any object-specific state in the field | 21:04 |
efried | Okay. Then prolly leave the code as it is now | 21:04 |
efried | thanks | 21:05 |
openstackgerrit | Eric Fried proposed openstack/nova master: DRY: Build ImageMetaPropsPayload from ImageMetaProps https://review.opendev.org/698803 | 21:06 |
efried | done, and changed the TODO to a NOTE | 21:06 |
*** ganso has quit IRC | 21:09 | |
*** ircuser-1 is now known as netlensbot | 21:24 | |
sean-k-mooney | efried: oh i forgot the has. i just ran the test i added | 21:27 |
sean-k-mooney | *hash | 21:27 |
*** netlensbot is now known as ircuser-1 | 21:28 | |
*** tbachman has joined #openstack-nova | 21:28 | |
efried | sean-k-mooney: yahbut, see my fup | 21:28 |
efried | was going to say maybe you want to squash; but really they are quite different things, so let's keep them separate. | 21:28 |
efried | plus I don't want to lose the ability to +2 yours :P | 21:28 |
sean-k-mooney | yep i did i was wondering about that | 21:29 |
sean-k-mooney | if we do the follow up then we might as well nuke the object | 21:29 |
sean-k-mooney | well | 21:29 |
sean-k-mooney | we could replace the ImageMetaPayload 'properties': fields.ObjectField('ImageMetaPropsPayload') definition | 21:31 |
sean-k-mooney | with another filed that just reference the other object | 21:32 |
melwitt | jroll: thanks. I'm trying to figure out what's the correct thing to do in the nova ironic driver if ironic is not filling in a cpu_arch in a node. should we choose a default arch to report for ironic scheduling? if so, how should we choose it? a new config option? those are the kind of questions I'm trying to find answers about | 21:32 |
*** lvbin02 has joined #openstack-nova | 21:33 | |
sean-k-mooney | efried: ok just read your comments on the patch below. ill create a fup to add the last combination and move the comments to doc strings afat i fix the Sync patch | 21:35 |
sean-k-mooney | and ya the ddt module is really nice to use. ill try to use it more often when it makes sense | 21:35 |
*** lvbin01 has quit IRC | 21:36 | |
*** lvbin02 is now known as lvbin01 | 21:36 | |
efried | sean-k-mooney: ++ | 21:37 |
efried | sean-k-mooney: Reasons not to nuke the object: it would make version boundaries weird; and I'm not sure what happens with SCHEMA. | 21:38 |
efried | It may be possible. But I'd leave it up to gibi | 21:38 |
sean-k-mooney | well with your follow up patch every time we add a filed to the normal nova object the notificaiton object fileds and schema will auto update chaning the hash | 21:39 |
sean-k-mooney | so if we apply that patch we need to rememebr to bump the notificion object version and document the new filed even though we are not modifying the class | 21:40 |
efried | ohhh, good point, the version. That needs a -1. | 21:40 |
sean-k-mooney | that is still less work then today however. ya i can put that in a review comment | 21:40 |
efried | Yeah, we at least need to make a note in the class with that instruction. | 21:41 |
efried | the test will fail, but you could get around it by just updating the hash without bumping the version. | 21:41 |
sean-k-mooney | yep | 21:41 |
efried | an astute reviewer ought to notice, but... | 21:41 |
jroll | melwitt: it feels like it should be a required field for an ironic node | 21:42 |
jroll | there's no way to guess at it | 21:42 |
*** eharney has quit IRC | 21:43 | |
*** ircuser-1 is now known as chasmo_ | 21:52 | |
sean-k-mooney | jroll: the cpu architecture | 21:56 |
sean-k-mooney | ya i mean some hyperisors can emulate other archs but that is kind of true for all virt drivers | 21:57 |
*** dviroel has quit IRC | 21:58 | |
mriedem | ironic != hypervisor | 22:00 |
sean-k-mooney | also true | 22:00 |
*** dosaboy has quit IRC | 22:01 | |
sean-k-mooney | i was makeing the point that most of the virt driver only support one archtecture at a time so you amost always want to know the cpu architecture if you care about check the image for compatablity | 22:01 |
*** TxGirlGeek has quit IRC | 22:03 | |
sean-k-mooney | there was that project denvor chip that nviad wanted to make but then never closed the license agreement with intel for x86 | 22:03 |
*** TxGirlGeek has joined #openstack-nova | 22:04 | |
sean-k-mooney | iwas going to be an arm architecture chip with native traslation of x86 to arm opcode in hardware allowing them to run code comiled for either achtecture | 22:04 |
sean-k-mooney | im sure that would have been a pain to debug but it would have been cool if it ever worked | 22:05 |
*** henriqueof1 has quit IRC | 22:08 | |
openstackgerrit | sean mooney proposed openstack/nova master: Sync ImageMetaPropsPayload fields https://review.opendev.org/698613 | 22:11 |
*** awalende has joined #openstack-nova | 22:15 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Use COMPUTE_SAME_HOST_COLD_MIGRATE trait during migrate https://review.opendev.org/695220 | 22:15 |
*** awalende has quit IRC | 22:20 | |
sean-k-mooney | efried: actully my brain has already switched off today so ill do the FUP tomorrow the other FUP i promised is https://review.opendev.org/#/c/698260/ but there is no really rush with reviewing that | 22:23 |
efried | ack | 22:24 |
openstackgerrit | Merged openstack/nova master: trivial: Resolve (most) flake8 3.x issues https://review.opendev.org/695732 | 22:24 |
*** gmann_afk is now known as gmann | 22:29 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add NodeOwnerFilter https://review.opendev.org/697331 | 22:41 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Allow overriding self.api in _build_minimal_create_server_request https://review.opendev.org/698813 | 22:41 |
*** nweinber has quit IRC | 22:51 | |
openstackgerrit | Merged openstack/nova master: Don't hardcode Python versions in test https://review.opendev.org/698732 | 22:51 |
*** mriedem is now known as mriedem_away | 23:01 | |
*** tkajinam has joined #openstack-nova | 23:06 | |
*** mriedem_away has quit IRC | 23:10 | |
*** chasmo_ has quit IRC | 23:10 | |
*** mriedem has joined #openstack-nova | 23:11 | |
*** irclogbot_2 has quit IRC | 23:11 | |
*** irclogbot_0 has joined #openstack-nova | 23:12 | |
*** TxGirlGeek has quit IRC | 23:14 | |
*** ircuser-1 has joined #openstack-nova | 23:15 | |
*** dosaboy has joined #openstack-nova | 23:23 | |
*** dosaboy has quit IRC | 23:29 | |
*** slaweq has quit IRC | 23:30 | |
*** slaweq has joined #openstack-nova | 23:35 | |
openstackgerrit | Merged openstack/nova master: Keep pre-commit inline with hacking and fix whitespace https://review.opendev.org/674057 | 23:39 |
openstackgerrit | Merged openstack/nova stable/queens: Add functional regression test for bug 1849409 https://review.opendev.org/690724 | 23:42 |
openstack | bug 1849409 in OpenStack Compute (nova) queens "openstack server list --deleted --limit -1 hangs" [High,In progress] https://launchpad.net/bugs/1849409 - Assigned to Matt Riedemann (mriedem) | 23:42 |
*** nicolasbock has quit IRC | 23:42 | |
*** tbachman has quit IRC | 23:50 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!