| gmann | gibi: ack | 00:16 |
|---|---|---|
| gmann | gibi: sean-k-mooney[m] melwitt just tested that nova API query param also allow duplicate query param and take latest one. I remember somewhere we convert that to python dict that is why the last one is considered. | 02:30 |
| melwitt | well that's fun | 02:31 |
| sean-k-mooney[m] | there are some query parmaters that we all to be repeated like field that can be used to generate an array | 02:31 |
| sean-k-mooney[m] | but unless we expressly document that as a supported usecase in teh api ref using it should be considerd a bug | 02:32 |
| sean-k-mooney[m] | if its not in the api-ref or in the orginal spec it is not part of the public api contract | 02:32 |
| sean-k-mooney[m] | gmann which nova api query parmater were you testing? | 02:33 |
| gmann | list server | 02:33 |
| gmann | limit filter | 02:33 |
| sean-k-mooney[m] | no user should ever depend on the last instance of a parmater being used in the nova api we are free to change that at any point unless there is api documenation that expressless states that for any given parmater. | 02:37 |
| sean-k-mooney[m] | put another way i do not think we should ever accpeat a bug if we change that behaivor without a microversion as it is an out of contract use fo the api and governed by the micorversion contract as a result | 02:39 |
| *** cozyurt is now known as Guest1343 | 02:45 | |
| sean-k-mooney[m] | ack i would consider depending on that behavior to be unsupported, https://docs.openstack.org/api-ref/compute/?expanded=list-servers-detail#list-servers does not declare you are allowed to repate the args as part of the filter support | 02:48 |
| sean-k-mooney[m] | we should be retrunnign a 400 but we are not validateing that. if we change the behavior in the future it should not require a microversion as this is not part of the api specification today | 02:49 |
| gmann | gibi: melwitt sean-k-mooney[m] we validate it by converting it to set which will add multiple passed value in query param - https://github.com/openstack/nova/blob/master/nova/api/validation/__init__.py#L176 | 02:52 |
| gmann | which is good. | 02:52 |
| gmann | but here this guy just pick latest one https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L179 | 02:52 |
| gmann | sean-k-mooney[m]: additional param in query param was also not written behavior in api-ref but we change that with microversion so that we do not break users | 02:53 |
| sean-k-mooney[m] | i really think this raised to the level of a v3 api change if we dont reject it without a microversion change | 02:54 |
| sean-k-mooney[m] | i really dont think this is something we shoudl require a microversion change for | 02:54 |
| gmann | I consider API usage as what code allow than what is there in api-ref. api-ref are not complete set of 'how not to use' its just 'how to use' | 02:54 |
| sean-k-mooney[m] | to me the is a unaccpable burden of mantaince to contineu to supprot. the api is not defiend by the implemantion its defiend by the specification and this is not part of the specificaiont | 02:55 |
| gmann | it is like we allowed our API to be used that wrong way and for many users it was working that as successful case. | 02:55 |
| gmann | sean-k-mooney[m]: not our API :) api -ref was written based on what implementation was. | 02:55 |
| gmann | we try to match that but if api-ref mention something and code does not behave that way then yes we consider as bug | 02:56 |
| gmann | but if api-ref which is not complete does not mention anything does not mean code is wrong and we can change that without versioning | 02:57 |
| gmann | in past we used to document that as known bug/behavior and fix in new microversion | 02:57 |
| sean-k-mooney[m] | i disagree if the api and spec for the feature do not refernce the behaivor it is not part of the public contract | 02:57 |
| sean-k-mooney[m] | i dont think we should leak the impleation into the public contract | 02:58 |
| gmann | it should if we have designed our API that way but it is not. it was always what implementation was and that is being used. | 02:58 |
| sean-k-mooney[m] | we did design our apis that way | 02:58 |
| sean-k-mooney[m] | its why we require a spec for all api changes | 02:59 |
| sean-k-mooney[m] | the really issue i guess is some fo the apis are form pre microversion and did not have a spec | 02:59 |
| gmann | that is fine. I am saying for old and existing behavior. that is why microversion was introduced. for new APis/feature I agree as per spec driven. | 02:59 |
| sean-k-mooney[m] | for example the regex support today in list servers is directly tied to the db you use | 03:00 |
| sean-k-mooney[m] | i really think it would reduce the quality of our software to not fix what in my mind is broken validation code | 03:00 |
| gmann | even rejecting unknown and invalid query param (which we ignored silently in code) was also changed with microversion so that we do not break users | 03:01 |
| gmann | I am not saying that is good and should not be fixed. it should be but with microversion | 03:01 |
| sean-k-mooney[m] | i guess with my downstream hat on i would be much less willing to supprot this. upstream we might be able to allow invalid input but form a downstream perspecitive i dont think i would accpete this form a customer bug. the api contract exits for a reason and not enforcing that contract exposes us to suppporting thing we never intended that is harmful to the healt of the product in my view | 03:03 |
| gmann | my 1 month old saying to stop arguing on APIs :) its time to feed him. :P | 03:04 |
| sean-k-mooney[m] | :) | 03:04 |
| sean-k-mooney[m] | go do that | 03:04 |
| sean-k-mooney[m] | but if we cant fix this without a microverion or validation middleware i think we have to reconsider raising the min microversion to fix this | 03:05 |
| *** oklhost_ is now known as oklhost | 07:16 | |
| *** hemna4 is now known as hemna | 07:38 | |
| gibi | the more I think about API contract the more I feel that we are on the wrong path. Saying that we should not change the behavior of our APIs without a microversion could be abused to the extent that forbid any kind of change of the implementation behind the API. | 08:04 |
| gibi | say we have a placement sql bug that breaks resource acconting today in a certain situation. As this is how the service works today one can say that a client might depend on the buggy behavior of the resource accounting hence we cannot fix the accounting outside of a microversion so placement servers out on the field will have the buggy behavior until everybody upgrades to future mastyer | 08:06 |
| gibi | so imagine two customers | 08:07 |
| gibi | i) customer depending on the buggy behavior and getting a stable bugfix would break their usage of placement | 08:07 |
| gibi | ii) customer is facing the buggy behavior and their system is broken until the buggy behavior is removed | 08:07 |
| gibi | clearly we cannot support both customer | 08:08 |
| gibi | so it boils down to which customer we would like to support more? i) who depends on a bug ii) who are hit by the same bug | 08:08 |
| bauzas | gibi: agreed with you | 08:36 |
| bauzas | I wonder why we should support the i) customer if they already have an issue | 08:37 |
| gibi | i) has no issue it is happy who the api behaves today | 08:38 |
| gibi | s/who/how/ | 08:38 |
| bauzas | gibi: well, I'm not sure this customer knows about the behaviour actually | 08:42 |
| bauzas | maybe he's just happy because it works | 08:42 |
| gibi | yeah, it can be accidental use | 08:44 |
| *** akekane_ is now known as abhishekk | 08:51 | |
| bauzas | just a thought, we always said to avoid as much as possible to directly pass a placement request by flavors | 08:51 |
| bauzas | gibi: so that would mean that customers doing this (and passing two same request params) looks to me not a bug | 08:52 |
| gibi | I don't see why repeating required param today is a good idea as it does not give a consistent result. E.g. you say required=A&required=B but A is ignored. Why would you do that intentionally? | 08:55 |
| gibi | so next time you add required=C to the list, B also gets ignored | 08:56 |
| gibi | so whathever client does this that client is wrong and a real bug is about to happen with that client | 08:56 |
| gibi | (btw openstack client is good, --required A --required B creates a REST request with required=A,B) | 08:57 |
| gibi | also imagine that the client today does required=A,B&required=A,B and that produce a good result today | 08:59 |
| gibi | then a new requirement came to add C to the request in that client | 09:00 |
| gibi | the dev looking at the current client code can assume that required parameters can be repeated | 09:00 |
| gibi | as it is repeated today and works | 09:00 |
| gibi | but he might decide not to add C to both instance of required | 09:01 |
| gibi | then he just implemented a bug | 09:01 |
| bauzas | gibi: when I say "not a bug" I mean that's something we can close without a microversion honestly | 09:06 |
| opendevreview | Rajat Dhasmana proposed openstack/nova master: WIP: Add support for volume backed server rebuild https://review.opendev.org/c/openstack/nova/+/820368 | 10:32 |
| sean-k-mooney[m] | so i really think we should fix this however if we cant agree to just do it i would like to add a new default middleway to reject repated args outside a allowed set | 11:24 |
| sean-k-mooney[m] | by the way changes to policy i confider much more likely to break peole then this. we dont reqiure micoro versions for default policy changes for some reason which to me feels like a feautre yet the assertion is we cant fix broken behavior without one feels wrong | 11:28 |
| gibi | sean-k-mooney[m]: I don't see how we would be able to agree on the middleware being a fix without a microversion. Or do you propose that as a microversion bump? | 11:37 |
| sean-k-mooney[m] | no | 12:01 |
| sean-k-mooney[m] | middleware is not part of the versioned api | 12:01 |
| sean-k-mooney[m] | its operator configurable via paste.ini | 12:02 |
| sean-k-mooney[m] | so my proposal in order of preference would 1.) fix without a microversion, 2.) fix with a micorversion and provide middleware for older microverion that returns 400 if there is an unsupported repeated argument | 12:03 |
| sean-k-mooney[m] | 3.) fix with a microverion and serioiusly consider raising min micorversion over the next few cycle until we finally get to this one | 12:04 |
| sean-k-mooney[m] | gibi at some point i stongly feel this needs to be fixed by default for everyone that uses nova and placment since the same broken behavior affects both | 12:05 |
| sean-k-mooney[m] | so if we force a microverion for this we need to look at starting the process of raising or min micorversion | 12:06 |
| gibi | sean-k-mooney[m]: ahh I see, so the middleware would be optional therefore does not need a microversion. Yeah that could be tried. Probably on can say that becomes a config driven API but paste.ini is created exactly for that kind of things | 12:09 |
| sean-k-mooney1 | ya i was lookign are our exising middelware it does not look like it would be that hard to create one that did this | 12:10 |
| sean-k-mooney1 | basically using https://github.com/openstack/nova/blob/master/nova/api/openstack/identity.py as a template | 12:11 |
| *** sean-k-mooney1 is now known as sean-k-mooney | 12:11 | |
| sean-k-mooney | we would then add a new filter lin and add that filter to the pipeline https://github.com/openstack/nova/blob/master/etc/nova/api-paste.ini#L68-L81 | 12:12 |
| sean-k-mooney | i belive that is how that work but i have never had a need to do that before but that seam to be the patteren | 12:12 |
| sean-k-mooney | i read over most of our docs last night to try and find if they provdied any guidence on if repated arges were ever intended to be support and i cant fine any to that efffect for what its worth | 12:14 |
| sean-k-mooney | with that said the https://github.com/openstack/nova/blob/master/doc/source/reference/stable-api.rst#v2-api-compatibility-mode-based-on-v21-api doc does imply that addtion request parmaters are not intended to be allowed | 12:16 |
| sean-k-mooney | """v2.1 API is exactly same as v2 API except strong input validation with no additional request parameter allowed and Microversion feature.""" | 12:16 |
| gibi | "additional" probably means "unknown to nova" not "repeated know" params | 12:21 |
| gibi | as we allowed (and ignored) unknown params in the past | 12:21 |
| gibi | but I agree, I think we need to put a rule to repeated params | 12:22 |
| pmonteir | Good morning everybody! Does anybody know how the "live_migration_downtime" parameter was tested? Been having some trouble trying to understand how this works | 12:27 |
| sean-k-mooney | pmonteir: its not really. we just pass that to libvirt nova is not in contol of the downtime | 12:29 |
| sean-k-mooney | once nova start the migration libvirt is basically in charge until its done | 12:29 |
| sean-k-mooney | we can call libvirt with addtional commands like force complete or abort | 12:30 |
| sean-k-mooney | but nova does not activly monitor the downtime and progress we passivly recive the events form libvirt and we can query it in resonce to an api request but we just get out of libvirts way and let it do its thing | 12:31 |
| pmonteir | Oh... I see, so if the max downtime is hit, that's just something we let libvirt handle? | 12:34 |
| sean-k-mooney | i can check, in the case of max im not sure but the normal pauses ectra are not done by nova | 12:35 |
| sean-k-mooney | we configure the max down time on the guest https://github.com/openstack/nova/blob/master/nova/virt/libvirt/migration.py#L475-L527 | 12:36 |
| sean-k-mooney | im just checkign to see if we handel when its exceeded in nova or preconfigure the action to take | 12:37 |
| pmonteir | Yh, that's exactly the thing I'm having some trouble trying to find out. If something is actually done when its exceeded | 12:38 |
| sean-k-mooney | i know we have action if the over all migration timeout is exceeded but not sure about max_downtime | 12:39 |
| sean-k-mooney | kashyap: do you happen to know ^ | 12:39 |
| * kashyap reads back | 12:39 | |
| kashyap | I don't remember off-hand, have to dig in too | 12:41 |
| sean-k-mooney | pmonteir: what are you trying to achive by modifyign this by the way | 12:42 |
| sean-k-mooney | pmonteir: i assume you have seen https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/doc/source/admin/configuring-migrations.rst#advanced-configuration-for-kvm-and-qemu | 12:42 |
| kashyap | Yeah, let's step back to understand the bigger picture | 12:42 |
| sean-k-mooney | the auto convergence and post copy options might be of interest to you if you are seeign excessive downtime | 12:43 |
| sean-k-mooney | if you are seeing network downtime/ping loss that is likely unrealted to this | 12:43 |
| pmonteir | Yh, I have. So basically I was trying to understand this parameter and test it out. But I wasn't able to see it in action (the max downtime value being exceeded and a timeout being triggered) | 12:44 |
| sean-k-mooney | unless the vm is hevially loaded and dirtying memory you wont hit this in a normal migration | 12:45 |
| pmonteir | I was trying to migrate whilst dirtying memory, the migration kept going for some time and (once) I used "virsh list" and the vm being migrated got paused and stayed like that for a while, which I found it weird | 12:47 |
| sean-k-mooney | pmonteir: the migration timeout proably got exceeded and use force-compelte | 12:47 |
| sean-k-mooney | https://github.com/openstack/nova/blob/0d84833e9688e0df97f3d24e06025e512bca3ce3/nova/conf/libvirt.py#L372-L388 | 12:48 |
| sean-k-mooney | its our default | 12:48 |
| sean-k-mooney | that is based on live_migration_completion_timeout not based on downtime and downtime has no effect on how long the vm will be paused | 12:49 |
| sean-k-mooney | pmonteir: if that is what happened you should see https://github.com/openstack/nova/blob/2f644a82fec13bad8fcdfa195c9316a6f09ee15a/nova/virt/libvirt/migration.py#L468-L469 in the log | 12:50 |
| kashyap | pmonteir: So, one more way to make sure libvirt is actually *setting* the thing by passing it to QEMU is to observe the libvirt/QEMU interaction logs | 12:51 |
| kashyap | pmonteir: You can do it this way: | 12:51 |
| kashyap | (1) Config the libvirt/QEMU log filters using the 'virt-admin' tool (it gets installed as part of "libvirt-daemon" package on Fedora; check for your distro): | 12:53 |
| kashyap | $> virt-admin daemon-log-outputs "1:file:/var/log/libvirt/libvirtd.log" | 12:53 |
| kashyap | $> virt-admin daemon-log-filters "1:qemu_monitor" | 12:53 |
| kashyap | (2) Migrate your guest from source to destination (and also make sure you've set the "_downtime" config parameter) | 12:53 |
| kashyap | (3) That's it. Now you can `grep` for "downtime-limit" on source and destination libvirtd.log | 12:54 |
| kashyap | Or ... you can post the output of this: grep -Ei '(Send Command|QEMU_MONITOR_RECV_)' /var/log/libvirt/libvirtd.log | 12:54 |
| kashyap | In a paste-bin, I can analyze it. (Get the above output from both source and destination compute nodes) | 12:55 |
| sean-k-mooney | gibi: im more or less +2+w on https://review.opendev.org/c/openstack/nova/+/792356/8//COMMIT_MSG but do you want me to hold off so that teh implemented ... can be added? | 12:55 |
| gibi | sean-k-mooney: no, lets get it landed | 12:57 |
| pmonteir | I will try config those logs and see what I can get. Thank you guys! | 12:57 |
| sean-k-mooney | gibi: ok ill hit it when i get back just going to grab a drink | 12:58 |
| gibi | ack | 12:58 |
| dmitriis | sean-k-mooney: do you know if test_tagged_attachment is known to be flaky? Seem to be hitting it intermittently in both https://review.opendev.org/c/openstack/nova/+/819494 and https://review.opendev.org/c/openstack/nova/+/826675 | 12:58 |
| dmitriis | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_366/826675/5/check/nova-next/3661dcb/testr_results.html | 12:58 |
| *** dasm|off is now known as dasm | 13:02 | |
| *** dasm is now known as dasm|rover | 13:02 | |
| kashyap | pmonteir: Also, a broader tip: whenever you're debugging something that involves libvirt and QEMU: the above logs with the said filters are useful to investigate. (Upstream DevStack actually logs these by default) | 13:04 |
| sean-k-mooney | dmitriis: not that im aware of but volume attachment can be | 13:06 |
| pmonteir | kashyap: Oh, thanks! I'm kinda noob to openstack/libvirt so this is really helpful! :D | 13:06 |
| kashyap | pmonteir: No prob. The above logs are specifically good to debug all live migration issues | 13:07 |
| pmonteir | got it! | 13:07 |
| kashyap | pmonteir: Oh, wait. I had an error in the second step. It needs a bit more: | 13:08 |
| dmitriis | sean-k-mooney: ack, ty | 13:10 |
| kashyap | Not the second step, but the second command in the first step, for filters. The correct set of filters are: | 13:10 |
| kashyap | $> virt-admin daemon-log-filters "1:libvirt 1:qemu 1:conf 1:security 3:event 3:json 3:file 3:object 1:util" | 13:10 |
| kashyap | pmonteir: --^ Make the above edit, if you took a note of this. | 13:10 |
| pmonteir | oh, nice, thanks! | 13:12 |
| *** bhagyashris_ is now known as bhagyashris | 14:42 | |
| gibi | bauzas: fyi I've filed a gate-failure bug https://bugs.launchpad.net/nova/+bug/1959677 it is happening daily | 14:47 |
| bauzas | thanks, I was about to update the agenda | 14:47 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [yoga] Add PCI VPD Capability Handling https://review.opendev.org/c/openstack/nova/+/808199 | 14:58 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [yoga] Include pf mac and vf num in port updates https://review.opendev.org/c/openstack/nova/+/824833 | 14:58 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [yoga] Introduce remote_managed tag for PCI devs https://review.opendev.org/c/openstack/nova/+/824834 | 14:58 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: Bump os-traits to 2.7.0 https://review.opendev.org/c/openstack/nova/+/826675 | 14:58 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [yoga] Add support for VNIC_TYPE_SMARTNIC https://review.opendev.org/c/openstack/nova/+/824835 | 14:58 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: Filter computes without remote-managed ports early https://review.opendev.org/c/openstack/nova/+/812111 | 14:58 |
| dmitriis | sean-k-mooney, gibi: ta for the reviews. Fixed the first two, looking at gibi's comments to the third one. | 15:04 |
| gibi | dmitriis: ack, I will check back at some point | 15:05 |
| gibi | sean-k-mooney: when you read further into the smartnic series, I would like to see your opinion about the dynamicy capability handling at https://review.opendev.org/c/openstack/nova/+/812111/16/nova/compute/resource_tracker.py#1176 I have not problem with it but it is someting new | 15:15 |
| *** artom__ is now known as artom | 15:16 | |
| sean-k-mooney | dynamicy capability handeling? | 15:16 |
| opendevreview | Jonathan Race proposed openstack/nova master: Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/822053 | 15:17 |
| sean-k-mooney | oh the traits reporting | 15:17 |
| artom | Is "'name': old_flavor['name'] + 'extra-' + data_utils.rand_int_id(), | 15:17 |
| artom | TypeError: can only concatenate str (not "int") to str" a known thing | 15:17 |
| artom | Happening in nova-next | 15:17 |
| sean-k-mooney | gibi: we have something simiarl i belive else where but slightly less clever in implemenation | 15:17 |
| artom | Looks like a Tempest code error, perhaps data_utils.rand_int_id() changed to return an int or something, but I feel like if it was tempest it would be all over the place | 15:17 |
| gibi | artom: that logic is a recent addition in tempest | 15:18 |
| sean-k-mooney | artom: so ya that looks like it shoudl have a str() call | 15:18 |
| artom | gibi, ah, right, cae966812a4a5070c3e7f82d16ebe697da57e5c5 | 15:18 |
| sean-k-mooney | althoguh personally i woudl be tempted to use an fstring | 15:19 |
| artom | Well, it's breaking nova-next | 15:19 |
| gibi | yepp you were faster finding it | 15:19 |
| * artom fix | 15:19 | |
| gibi | artom: thanks | 15:19 |
| sean-k-mooney | the fix is firly simple just use sting interpulation of fstring instead fo concationation or cast | 15:20 |
| gibi | sean-k-mooney: cool | 15:20 |
| sean-k-mooney | normally we do simple things like this https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L395-L420 | 15:20 |
| gibi | artom: the orginal tempest fix did not trigger nova-next (or an ovs full tempest from neutron) and therefore the test case that is changed was not run for the tempest patch | 15:20 |
| artom | Yeah, all good. Quick enough fix | 15:21 |
| artom | It'll take me longer to file the LP bug than changing the code :P | 15:21 |
| sean-k-mooney | but we also build trait dymically like this https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L8412-L8434 | 15:21 |
| gibi | artom: please add a DNM patch in nova that depends on the tempest fix to see nova-next is green after it :) | 15:21 |
| sean-k-mooney | gibi: is your concern that htis is a mor dynimic capblity | 15:21 |
| sean-k-mooney | not just a dynmic trait | 15:22 |
| artom | gibi, good idea | 15:22 |
| gibi | sean-k-mooney: this is a new thing and it will create precedent | 15:22 |
| gibi | sean-k-mooney: I mean that we dinamically apply capability based on resource inventory | 15:22 |
| gibi | sean-k-mooney: so I'd like to make a conscious introducing it | 15:23 |
| sean-k-mooney | oh as in it only reports the trait when we have a pool that support it | 15:23 |
| gibi | yepp | 15:23 |
| sean-k-mooney | right | 15:23 |
| gibi | but personally I have no problem with it | 15:23 |
| sean-k-mooney | well we technially dont need to do that actully | 15:24 |
| sean-k-mooney | i dont really either | 15:24 |
| sean-k-mooney | but we coudl jsut always report it | 15:24 |
| sean-k-mooney | if the host support it | 15:24 |
| gibi | it is basically indirectly config driver (by the whitelist config) and we already have config driven capabilities | 15:24 |
| sean-k-mooney | the inventory check is going to be enforce by the pci filter after the fact | 15:24 |
| gibi | s/driver/driven/ | 15:24 |
| sean-k-mooney | yes we do | 15:24 |
| sean-k-mooney | the main advantage to having it is the filtering will partly be done by placment | 15:25 |
| gibi | yepp | 15:25 |
| gibi | that is a good thing | 15:25 |
| sean-k-mooney | the smart nics port dont have inventories in placement currntly but this will at least find host with them in the resouce tracker | 15:25 |
| sean-k-mooney | so ya in the long run i agree i think this better and also that yes this is a new thing and setting precident | 15:26 |
| gibi | awesome, then we are on the same page | 15:26 |
| sean-k-mooney | am since we are talking about pci i agree with you regfarding the pack/spread change. i raised the behaivor delta in a prevous version | 15:27 |
| gibi | ohh, cool then | 15:28 |
| sean-k-mooney | i kind fo feel like we shoudl ignore the pack config option for pci devices for backward compatiablity with the excption that if you ask for one we shoudl actully prefer the nodes with pci devices | 15:28 |
| sean-k-mooney | today we dont prefer if you ask for a pci device we jsut avoid if we dont | 15:28 |
| opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test fix for bug 1959682 https://review.opendev.org/c/openstack/nova/+/827306 | 15:29 |
| sean-k-mooney | i had said in the previous version i was ok with debating that in a followup if others agreed to mvoe it forward | 15:29 |
| sean-k-mooney | i have remvoed my +2 for now so we can disucss it in the patch | 15:29 |
| gibi | ok | 15:31 |
| opendevreview | Jonathan Race proposed openstack/nova master: Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/822053 | 15:46 |
| bauzas | reminder : nova meeting here in 10 mins | 15:50 |
| bauzas | elodilles: no agenda for stable branches ? | 15:57 |
| elodilles | bauzas: well, i forgot to update it well before you so i waited a bit... maybe for too long o:) sorry :) | 15:59 |
| bauzas | elodilles: just do it now | 15:59 |
| bauzas | I'll refresh | 15:59 |
| elodilles | let me update it quickly :) | 15:59 |
| elodilles | (not so much to update) | 15:59 |
| bauzas | #startmeeting nova | 15:59 |
| opendevmeet | Meeting started Tue Feb 1 15:59:59 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:59 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:59 |
| opendevmeet | The meeting name has been set to 'nova' | 15:59 |
| bauzas | hello everybody | 16:00 |
| bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | 16:00 |
| elodilles | o/ | 16:00 |
| chateaulav | \o | 16:00 |
| gibi | o/ | 16:00 |
| bauzas | ok let's start, should be quick hopefully | 16:01 |
| * kashyap waves, but will have to come and read back. Have to be AFK | 16:01 | |
| bauzas | #topic Bugs (stuck/critical) | 16:01 |
| bauzas | #info No Critical bug | 16:01 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 39 new untriaged bugs (+3 since the last meeting) | 16:01 |
| bauzas | #help Nova bug triage help is appreciated https://wiki.openstack.org/wiki/Nova/BugTriage | 16:01 |
| bauzas | #link https://storyboard.openstack.org/#!/project/openstack/placement 27 open stories (+2 since the last meeting) in Storyboard for Placement | 16:02 |
| bauzas | I need to do a bit of triage | 16:02 |
| bauzas | but working on Tempest trampled me | 16:02 |
| bauzas | anyway, we'll discuss about the new gate failure bug after this topic | 16:02 |
| bauzas | any other bug to want to discuss ? | 16:03 |
| opendevreview | Artom Lifshitz proposed openstack/nova master: WIP: live-migration: only wait for external events for active VIFs https://review.opendev.org/c/openstack/nova/+/827314 | 16:03 |
| bauzas | looks not | 16:04 |
| bauzas | #topic Gate status | 16:04 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:04 |
| bauzas | gibi: do you want to discuss about https://bugs.launchpad.net/nova/+bug/1959677 ? | 16:04 |
| ganso | o/ | 16:04 |
| gibi | bauzas: nope | 16:04 |
| gibi | bauzas: I had not time to investigate | 16:04 |
| gibi | bauzas: just so it happening and reported it | 16:04 |
| bauzas | okay | 16:04 |
| bauzas | I'll triage it btw. | 16:05 |
| artom | Someone needs to tag https://bugs.launchpad.net/tempest/+bug/1959682 as a Nova gate failure, I don't have Tempest bug tagging powers | 16:05 |
| gibi | artom: I added the tag but that does not mean much as the bug is against tempest project | 16:05 |
| gibi | artom: btw, somebody was faster than you proposing the fix | 16:06 |
| bauzas | is it a problem for nova ? | 16:06 |
| artom | Yeah, and anyways it should go away quickly enough | 16:06 |
| gibi | artom: I noted that in your fix | 16:06 |
| gibi | bauzas: yes | 16:06 |
| gibi | bauzas: nova-next is down | 16:06 |
| bauzas | if so, should be a nova bug | 16:06 |
| * bauzas adds the nova project to the bug report | 16:06 | |
| gibi | bauzas: the actual fault is in the tempest code | 16:06 |
| gibi | (we probalby overcomplicate :D) | 16:06 |
| bauzas | okay, added nova to the bug | 16:07 |
| bauzas | we'll need to triage it | 16:07 |
| artom | gibi, well yeah, I spend most of my time writing up the commit message and bug report :P | 16:07 |
| artom | Of course someone else was faster | 16:07 |
| bauzas | if people want to triage it btw... :) | 16:07 |
| gibi | artom: :) | 16:07 |
| * bauzas doesn't try to look at someone he knows | 16:07 | |
| bauzas | anyway | 16:08 |
| bauzas | moving on ? | 16:08 |
| sean-k-mooney | bauzas: its not a nova bug in this case | 16:08 |
| sean-k-mooney | since the error is in tempest | 16:09 |
| bauzas | sean-k-mooney: then please triage it | 16:09 |
| sean-k-mooney | we can skip it tempoarlly as a workaround | 16:09 |
| sean-k-mooney | we can move on | 16:09 |
| bauzas | the question is, what project needs to be modified for this report ? | 16:09 |
| sean-k-mooney | tempest | 16:09 |
| bauzas | if it's not nova, then we can triage it simply | 16:10 |
| bauzas | sean-k-mooney: then please do https://bugs.launchpad.net/nova/+bug/1959682 | 16:10 |
| sean-k-mooney | yep i have it open | 16:10 |
| sean-k-mooney | ill mark it invalid and explian why | 16:10 |
| bauzas | cool thanks | 16:10 |
| bauzas | I just added nova to ask for it | 16:10 |
| bauzas | anyway, moving | 16:10 |
| sean-k-mooney | the only question is do we want to make it non-voting until tis fixed but it sound like it wont take long so let leave it for now and move on | 16:11 |
| gibi | sean-k-mooney: it wont take long | 16:11 |
| gibi | there is already two fixes up ;) | 16:11 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status | 16:12 |
| bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:12 |
| bauzas | nothing to tell about the periodic jobs, they work | 16:12 |
| bauzas | #topic Release Planning | 16:12 |
| bauzas | #link https://releases.openstack.org/yoga/schedule.html#y-ff FeatureFreeze in 3.5 weeks | 16:12 |
| bauzas | #link https://etherpad.opendev.org/p/nova-yoga-blueprint-status Etherpad for blueprints tracking | 16:12 |
| bauzas | as you can see, I tried to look at all the accepted blueprints | 16:13 |
| bauzas | I saw some changes, but not for all the blueprints | 16:13 |
| opendevreview | Artom Lifshitz proposed openstack/nova master: DNM: Test fix for bug 1959682 https://review.opendev.org/c/openstack/nova/+/827306 | 16:14 |
| gibi | bauzas: nice list! | 16:14 |
| bauzas | #action if you are one of the owners of accepted blueprints for yoga, please modify https://etherpad.opendev.org/p/nova-yoga-blueprint-status to provide your changes | 16:14 |
| bauzas | #info if we don't know your changes, we can't review them | 16:14 |
| gibi | bauzas: does the order in the pad means some kind of priority order? | 16:14 |
| bauzas | gibi: not for the moment | 16:14 |
| gibi | ack | 16:14 |
| bauzas | anyone can review what they want | 16:15 |
| bauzas | next week, I could ask whether we would prefer to have some priorities | 16:15 |
| bauzas | my only concern is that half of the blueprints seem not open yet | 16:15 |
| bauzas | gibi: about https://blueprints.launchpad.net/nova/+spec/qos-minimum-guaranteed-packet-rate I'll mark it as complete, thanks | 16:16 |
| gibi | bauzas: thanks | 16:16 |
| gibi | bauzas: that is complete | 16:16 |
| bauzas | saw it indeed | 16:16 |
| bauzas | so again, I'm saying loud : please look at this etherpad and provide your changes if you're one owner and you don't see your changes in the etherpad | 16:17 |
| bauzas | tbc, if Feb 15th (2 meetings from now), we're not able to see changes for a blueprint, I'll punt the blueprint as something not prioritary | 16:18 |
| bauzas | as I don't want reviewers to be rushed by new changes coming down at the last minute | 16:19 |
| gibi | agree | 16:19 |
| bauzas | not saying we'll hold merging things for those last-minute changes | 16:19 |
| bauzas | just telling this won't be prioritary | 16:20 |
| bauzas | anyway, for the moment, we have enough to look at :) | 16:20 |
| bauzas | anything to say else about blueprints ? | 16:21 |
| bauzas | guess not | 16:21 |
| bauzas | #topic Review priorities | 16:21 |
| bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+label:Review-Priority%252B1 | 16:21 |
| bauzas | some of the blueprints above are priorities | 16:22 |
| sean-k-mooney | i guess we shoudl use ^ after the 15 to | 16:22 |
| bauzas | sean-k-mooney: yeah, I was about to say it | 16:22 |
| bauzas | and also, we should remove priorities for changes that are not blueprints | 16:23 |
| bauzas | (I've 3 patches in mind) | 16:23 |
| sean-k-mooney | we might want to make exception for gate issue but sure | 16:23 |
| bauzas | but for the moment, I'm OK with leaving this list as it is | 16:23 |
| bauzas | sean-k-mooney: oh, of course | 16:23 |
| bauzas | point is, owners would appreciate if we say out loud that we prioritize blueprints before FeatureFreeze | 16:24 |
| bauzas | I'll think about it during this week and I'll prepare something to discuss next week's meeting | 16:24 |
| sean-k-mooney | by the way i have not had time to do the project-config change yet | 16:25 |
| sean-k-mooney | but its still on my radar | 16:25 |
| bauzas | sean-k-mooney: I don't think it's important for this cycle | 16:25 |
| bauzas | we can hold until RC1 | 16:25 |
| bauzas | this is a bit late to be added this cycle, tbh | 16:25 |
| sean-k-mooney | well its not really realted to the milestones we just need to land it in the proejct config repo | 16:25 |
| sean-k-mooney | so ill work on it when i have time | 16:26 |
| sean-k-mooney | just wanted to say its still on my todo list | 16:26 |
| bauzas | sean-k-mooney: agreed but we need to document and explain what to do | 16:26 |
| bauzas | people could be confused while we're close to featurefreeze and I want things crystal clear for the last two weeks | 16:27 |
| bauzas | anyway, next topic | 16:28 |
| bauzas | #topic Stable Branches | 16:28 |
| bauzas | #topic Stable Branches | 16:28 |
| elodilles | #info stable gates are not blocked completely, but it is hard to merge patches due to intermittent failures | 16:28 |
| elodilles | :) | 16:28 |
| bauzas | yeah... | 16:28 |
| elodilles | that was my quick update | 16:28 |
| gibi | yeah https://review.opendev.org/c/openstack/nova/+/820555 has 19 rechecks :/ | 16:29 |
| bauzas | appreciated | 16:29 |
| bauzas | gibi: :-/ | 16:29 |
| elodilles | of course there were gate blocker issues meantime as well.. :X | 16:30 |
| gibi | I will try to get some time spen on the gate status later this week | 16:30 |
| bauzas | I can try to look at this but I promised code reviews as well | 16:30 |
| bauzas | anyway | 16:31 |
| bauzas | help is appreciated from anybody | 16:31 |
| bauzas | #topic Sub/related team Highlights | 16:31 |
| bauzas | #info No subteam left | 16:31 |
| bauzas | #topic Open discussion | 16:31 |
| bauzas | nothing on the agenda | 16:31 |
| bauzas | wrapping up or someone yelling about something ? | 16:32 |
| sean-k-mooney | elodilles: have the pip issue been adressed in ci yet | 16:32 |
| sean-k-mooney | e.g. that pip does not supprot pyton 3.6 anymore | 16:32 |
| elodilles | sean-k-mooney: if i'm not mistaken we don't have that in stable | 16:33 |
| sean-k-mooney | we likely will in the future | 16:33 |
| sean-k-mooney | but ok | 16:33 |
| elodilles | e.g. gibi's linked patch (above^^^) just had a successful zuul run | 16:33 |
| chateaulav | sean-k-mooney: I added it to the etherpad, but are we able to get a push from redhat for libvirt bug: https://bugzilla.redhat.com/show_bug.cgi?id=1432101 | 16:33 |
| * sean-k-mooney is still woried more things will drop 3.6 support before end fo cycle | 16:34 | |
| sean-k-mooney | ack thats all i had for the meeting chateaulav is your item for the meeting | 16:34 |
| chateaulav | related to MIPs support for bp | 16:34 |
| chateaulav | i huess for outside | 16:34 |
| sean-k-mooney | we can continue discussing this on the channel i think | 16:35 |
| sean-k-mooney | bauzas: if you you want to finsih up | 16:35 |
| chateaulav | k | 16:35 |
| bauzas | cool | 16:35 |
| bauzas | I have to go to the post office | 16:35 |
| bauzas | (absolutely needed to be known) | 16:35 |
| bauzas | thanks all | 16:35 |
| bauzas | #endmeeting | 16:35 |
| opendevmeet | Meeting ended Tue Feb 1 16:35:54 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:35 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2022/nova.2022-02-01-15.59.html | 16:35 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-02-01-15.59.txt | 16:35 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2022/nova.2022-02-01-15.59.log.html | 16:35 |
| elodilles | thanks o/ | 16:36 |
| sean-k-mooney | chateaulav: so looking at that im not sure how much tractkign it will get | 16:36 |
| * bauzas is actually now rushing to the post office before it closes | 16:36 | |
| sean-k-mooney | chateaulav: in our product we do not supprot qemu emulation | 16:36 |
| sean-k-mooney | so im not sure they will want to fix a mips64el issue | 16:37 |
| elodilles | sean-k-mooney: if you meant the pip bootstrap what we talked about last week then i think this patch added another fix: https://review.opendev.org/c/openstack/devstack/+/825920 | 16:37 |
| elodilles | sean-k-mooney: so hopefully this will be enough | 16:37 |
| sean-k-mooney | ya so there are 2 failures that one and i saw a tox failure somewhere | 16:38 |
| sean-k-mooney | but that might have been downstream | 16:38 |
| chateaulav | we are going to do further testing, just want to make sure from the feature context it wont hold anything up if support isnt initially there for that one architecture. (if i cant find a work around) | 16:38 |
| sean-k-mooney | chateaulav: the issue might be related to the machine type you selected | 16:38 |
| sean-k-mooney | for example on arm i know virt support pci but not all do | 16:38 |
| chateaulav | sean-k-mooney: fair, thats why ill do further testing just to make sure. appreciate it | 16:39 |
| sean-k-mooney | this is the qemu docs for mips https://www.qemu.org/docs/master/system/target-mips.html | 16:43 |
| sean-k-mooney | you need to use either the R4000 ot Malta machine type | 16:45 |
| chateaulav | Ok | 16:46 |
| sean-k-mooney | actully R4000 might not work | 16:49 |
| sean-k-mooney | malter is the only one that mentions pci | 16:49 |
| sean-k-mooney | you likely need to try a few an see if any work | 16:50 |
| sean-k-mooney | chateaulav: is mips one of you main usecases by the way | 16:58 |
| sean-k-mooney | or are you just testing the different options | 16:58 |
| chateaulav | sean-k-mooney: It's one of the main ones we were planning on. I will be finalizing the testing for that and riscv this afternoon. | 17:05 |
| chateaulav | sean-k-mooney: so I'll know more then as far as if mips won't work | 17:07 |
| sean-k-mooney | ack | 17:08 |
| opendevreview | Ilya Popov proposed openstack/nova master: Fix to implement 'pack' or 'spread' VM's NUMA cells https://review.opendev.org/c/openstack/nova/+/805649 | 17:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: Move 'hw:pmu', 'hw_pmu' parsing to nova.virt.hardware https://review.opendev.org/c/openstack/nova/+/792364 | 17:58 |
| opendevreview | Stephen Finucane proposed openstack/nova master: docs: Follow-ups for cells v2 doc https://review.opendev.org/c/openstack/nova/+/827336 | 18:32 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: Introduce remote_managed tag for PCI devs https://review.opendev.org/c/openstack/nova/+/824834 | 19:18 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: Bump os-traits to 2.7.0 https://review.opendev.org/c/openstack/nova/+/826675 | 19:18 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [yoga] Add support for VNIC_TYPE_SMARTNIC https://review.opendev.org/c/openstack/nova/+/824835 | 19:18 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: Filter computes without remote-managed ports early https://review.opendev.org/c/openstack/nova/+/812111 | 19:18 |
| dmitriis | gibi: resent https://review.opendev.org/c/openstack/nova/+/824834 | 19:23 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: [yoga] Add support for VNIC_TYPE_SMARTNIC https://review.opendev.org/c/openstack/nova/+/824835 | 19:59 |
| opendevreview | Dmitrii Shcherbakov proposed openstack/nova master: Filter computes without remote-managed ports early https://review.opendev.org/c/openstack/nova/+/812111 | 19:59 |
| opendevreview | Merged openstack/placement master: update placement for os-traits 2.7.0 release https://review.opendev.org/c/openstack/placement/+/826478 | 20:24 |
| opendevreview | Jonathan Race proposed openstack/nova master: Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/822053 | 20:26 |
| *** dasm|rover is now known as dasm|off | 22:23 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!