Tuesday, 2022-06-28

*** abhishekk is now known as akekane|home04:48
*** akekane|home is now known as abhishekk04:48
manuvakery1Hi .. if i take an instance snapshot after resizing to a higher flavor the min_disk  property it set to the older flavor disk size. Is this a known issue in train?04:58
sean-k-mooneyade_lee: so first bit of feedback is the jobs is not captureing logs form comptue-0 only compute-105:31
sean-k-mooneyade_lee: second bit of feedback is this is a know issue05:31
sean-k-mooney2022-06-27 10:32:42.103 2 ERROR nova.virt.libvirt.driver [-] [instance: 1fb29abc-c443-4404-81df-312b233d05ca] Live Migration failure: End of file while reading data: 05:32
sean-k-mooneyWe trust you have received the usual lecture from the local System05:32
sean-k-mooneyAdministrator. It usually boils down to these three things:05:32
sean-k-mooney    #1) Respect the privacy of others.05:32
sean-k-mooney    #2) Think before you type.05:32
sean-k-mooney    #3) With great power comes great responsibility.05:32
sean-k-mooneysudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper05:32
sean-k-mooneyhttps://bugzilla.redhat.com/show_bug.cgi?id=208952005:33
sean-k-mooneyand its the same root cause05:33
sean-k-mooneylive_migration_uri=qemu+ssh://nova_migration@%s:2022/system?keyfile=/etc/nova/migration/identity&proxy=netcat05:33
sean-k-mooneyyou are definign ^05:34
sean-k-mooneythe live_migration_uri is depercated and we should not be using it downstream05:34
sean-k-mooneybut the actual issus is that that netcat at the ned if forceing netcat to be used in the live migration wrapper05:35
sean-k-mooneyso we are taking this if branch instead of the previous one https://github.com/rdo-packages/nova-distgit/blob/rpm-master/nova-migration-wrapper#L73-L75=05:35
sean-k-mooneythere are two issues with that first nc is not install and second the sudoers file only allow it to be used if you use the fully qualifed path05:36
sean-k-mooneyhttps://github.com/rdo-packages/nova-distgit/blob/rpm-master/nova_migration-sudoers#L3=05:37
sean-k-mooneyif nc was actully installed it might work as the command not found helper might eb trigering the sudo prompt so the patch might not be the problem05:37
sean-k-mooneybut the issue is its takign the netcat path05:37
sean-k-mooneyhttps://gitlab.com/libvirt/libvirt/-/blob/65312001bd972df8b7d4f11ea4662aff4889bee5/src/rpc/virnetclient.c#L446-44805:43
sean-k-mooneythis is the relevent libvit code05:43
sean-k-mooneyproxy=netcat will not work on 17 we shoudl etierh use auto or force the virt-ssh-helper via the native proxy05:49
sean-k-mooneychange the migration uri to proxy=native05:49
sean-k-mooneyade_lee: based on jame's comments and looking at the code this is a puppet-nova bug https://bugzilla.redhat.com/show_bug.cgi?id=2089520#c3 they obviouly never got the memo that using the uri directly has been deprecated since like osp 13 and that netcat was going away in 17 and they needed to stop forcing netcat and use either auto or native or stop generating the uri to let05:56
sean-k-mooneylibvirt use the virt-ssh-helper.05:56
sean-k-mooneyactully looking at puppet-nova this look liek its a ooo issue06:00
kashyapsean-k-mooney: At least there seems to be upstream bug that says "fix released", but no patch there: https://bugs.launchpad.net/tripleo/+bug/191825006:01
sean-k-mooneynice find06:02
kashyapYep, here it is (it's in Mar 2021): https://review.opendev.org/c/openstack/puppet-tripleo/+/779784/06:02
kashyapsean-k-mooney: Unrelated: You seem to be quite early up today06:02
sean-k-mooneywell06:02
sean-k-mooneythat might have been regressed06:02
sean-k-mooneyya i woke up at 5:30 and didnt feel like going back to sleep06:03
sean-k-mooneyso i got up06:03
kashyapIs there a new issue?06:03
kashyap(I see)06:03
sean-k-mooneylive migration does not work on 17 period06:03
sean-k-mooneywell at least not with tls-e config06:03
kashyapIs there a bz link?06:03
sean-k-mooneynot sure about with it disabled06:03
sean-k-mooneykashyap: yes ill get it in a sec but the issue is that proxy=netcat is gettign generated in the migration uri06:04
fricklerare you sure you are in the right channel? this sounds like rdo talk06:04
sean-k-mooneyand netcat is not installed in the container since we should be using the virt-ssh-helper in 1706:04
kashyapfrickler: Heh, I briefly wondered: if it's an upstream or a downstream-specific discussion06:04
sean-k-mooneyfrickler: actully this was ment to be downstream but meh its a ooo bug06:05
sean-k-mooneyit was ment to be rhos-dev but i clicked the wrong tab06:05
sean-k-mooneyi have upstream on the top pane and downstream on the bottom 06:05
gibi o/07:00
sean-k-mooneygibi: o/07:17
sean-k-mooneyfrickler: so kashyap found https://review.opendev.org/c/openstack/puppet-tripleo/+/779313/ and i have filed the revert https://review.opendev.org/c/openstack/puppet-tripleo/+/847818 just to close the loop on the downstream topic i rasied upstream :) in case you were wondering07:21
sean-k-mooneytl;dr is we forced netcat to workaround the lack of support in the rdo package for virt ssh-helper and since then netcat has been removed form teh ooo contianer07:22
sean-k-mooneyso we should just go back to auto now that we have support in the package07:22
whoami-rajathi #openstack-nova , would like to request reviews on my volume backend instance rebuild feature (3 patches in chain). the spec has merged and it was targeted for yoga cycle (but couldn't make it to the Feature freeze) so would like to get some early feedback07:23
whoami-rajathttps://review.opendev.org/c/openstack/nova/+/82036807:23
sean-k-mooneywhoami-rajat: specs are approved per release so the imporant thing is that it was reappoved for zed07:33
whoami-rajatsean-k-mooney, yep it was re-approved07:35
whoami-rajathttps://review.opendev.org/c/openstack/nova-specs/+/84015507:36
sean-k-mooneywhoami-rajat: yep i remember07:36
sean-k-mooneyi have your review open. i need to do some downstream jira work quickly but ill review them when im done07:37
whoami-rajatgreat, what i meant was it was approved last cycle as well but just wanted early reviews on it so i can make the deadline this time :)07:37
whoami-rajatsean-k-mooney, thanks!07:37
fricklersean-k-mooney: thx for the update. I really didn't want to drive you away, it just looked a bit out of context08:11
jkulikIs there something in Nova that lets me express anti-affinity towards a group of hosts in the same rack? Use-case: we structure our HVs into racks and there might be customers not wanting their VM to run on the same HV (current anti-affinity I know about) and customers who don't want to run in the same rack as another VM, while staying in the same AZ.09:03
gibijkulik: I think there is no automatic way to express rack level (or host aggregate level) anti-affinity. What you can do is to create separate falvors for separate racks by referring the specific rack in the flavor09:06
gibibut that does not scale well for many racks09:06
gibiand many flavors09:06
gibiI do remember discussing this on the last summit in berlin09:07
gibiwas it in the blazar session?09:07
gibihttps://etherpad.opendev.org/p/blazar-preemptible-and-gpus L6409:08
gibior more like L5709:08
sean-k-mooneyjkulik: we only supprot this at the az level09:11
sean-k-mooneywell no09:11
sean-k-mooneyat the host level09:11
sean-k-mooneyyou can use AZs per rack09:11
sean-k-mooneybut we have no aggreate or az anti affintiy concept in nova09:11
sean-k-mooneyand its not simple to add09:12
gibisean-k-mooney: I had a suggestion in the above etherpad how to add it to nova09:12
sean-k-mooneyjkulik: if you want to isolate custoemr we have tenat isolation filters09:12
sean-k-mooneygibi: we coudl do it via placment aggreate if we wanted too09:13
gibisean-k-mooney: nope09:13
sean-k-mooneybut i was just sayign we dont currently support it09:13
gibiyes, we not currently support it09:13
gibithe placement way would be harder than a nova way as placmenet aggregates has no metadata09:13
sean-k-mooneywe have also rejected it in the past as we did not want to add more orchestration to nova09:13
gibiwhile nova aggregates has metadata09:13
sean-k-mooneywell nova aggreate are mapped to palcemnt aggreates09:14
gibiwithout the metadata piece09:14
sean-k-mooneybut sure for the rack affinty that is doable09:14
gibiwe need the metadata to mark an aggregate as target for affinity/anti-affinity09:14
sean-k-mooneyaz anti affintiy which was the otehr request cant be done that way but we can do rack/row/room antiaffinity with aggreate metadta and a filter09:15
sean-k-mooneygibi: ya so if wee were to do this i would make the filter generic09:15
gibisure09:15
gibithat was an afterthought for me as well09:15
sean-k-mooneyso that you can define a set of lables and then express the anti affintiy requirement in the flavor09:15
sean-k-mooneylike you do with ceph09:16
sean-k-mooneyand the palcment maps09:16
gibiyep generic label based affinity/anti-affinity either via the flavor or via the server group API09:16
sean-k-mooneyyep09:16
gibiso we only need some devs to propose a spec and then the implementation :D09:17
gibieasy peasy :D09:17
sean-k-mooneythat i would be ok with but you could map the info to placment too perhaps as a step 209:17
gibisean-k-mooney: you mean extend the aggregate concept in placement with metadta?09:17
sean-k-mooneyno we could likely modle this with aggreates and custom_traits09:17
sean-k-mooneyi have not fully tought that out09:18
gibiI don't like it, as then the trait needs to be on the all the PRs in the aggregate09:18
sean-k-mooneybut it feels like we shoudl be able to do that09:18
sean-k-mooneywell i was thinking more like how misc_share_via_aggrate works09:18
gibitechnically doable but I would extend the aggregate concept instead in placement09:18
sean-k-mooneyi would be fine with extendign aggreates too if we had a clean way to extend it09:19
sean-k-mooneyperhaps traits on aggretes or some other metadta but doing it in nova first is alot simpler as you said09:19
sean-k-mooneyso get it working (nova) then make it fast (placment)09:19
gibiwe are in agreement :)09:20
sean-k-mooneyjkulik: interested in working on ^09:20
sean-k-mooneyjkulik: its not an uncommen request so we can also try an pitch it to our pm dowstream09:20
sean-k-mooneybut no promices they will go for it and we wont look at it until at least the A cycle in any case09:21
sean-k-mooneyjkulik: but if you had time to work on it we have 2 week to spec freeze and we could proably reivew it if you had somethign ready before code freeze09:21
sean-k-mooneyjkulik:realisticly though its proably an A or lather cycle thing09:22
sean-k-mooneybut it woudl be doable in an out of tree schduler filter today09:22
sean-k-mooneyat least if you took the flavor approch initaly to expres the affinity requiremnt09:22
sean-k-mooneyextending teh server group api woudl require an api change which need a spec and is not backportable due to how api versioning works09:23
sean-k-mooneygibi: what was the blazar usecase/imporance of this09:24
sean-k-mooneyi assume some sort of ha/fault tollernace usecase09:24
sean-k-mooneyoh no performance09:25
sean-k-mooneyi.e. reserve three GPU VMs in the same rack, where possible (hard/soft affinity).09:25
sean-k-mooneySpecifically this is to reduce network latency (MPI/horovod), and unnessacery TOR switch network contention.09:25
gibiyepp09:25
sean-k-mooneyi mean i gues it could be used for either09:25
gibibut there was limited time so we did not dig deep09:25
sean-k-mooneyack09:25
sean-k-mooneyso there soft vs hard "requirement" would be filter vs weigher09:26
sean-k-mooneybut most of the logic would be the same so i woudl proably just do both09:26
jkuliksean-k-mooney: I don't think I can make it in 2 weeks, but would be able to do it in the longer run09:26
sean-k-mooneyand try and shre the code09:26
sean-k-mooneyjkulik: ack09:26
jkulikcustom scheduler filter would be my idea, too. but we'd need it in the server-group API anyways as having one flavor per rack really doesn't scale09:28
sean-k-mooneynot per rack09:29
sean-k-mooneyso in the flavor you would have a policy09:29
sean-k-mooneyaggreate_anti_affinity:rack09:29
sean-k-mooneyaggreate_anti_affinity:room09:30
sean-k-mooneyaggreate_anti_affinity:row09:30
jkulikah, but still. customers are already overwhelmed by the number of flavors as is. I don't think we can add more for this :)09:30
sean-k-mooneyya that is why the server-group api woudl be beeter09:30
sean-k-mooneyther is a hack that you could use if you promise not to mention my name as the source :P09:31
jkulik:D09:31
sean-k-mooneyso you could use server tags or instance metadta for now09:31
bauzasI wonder how feasible a weigher could be09:31
bauzaswe could weigh per aggregatesd09:32
sean-k-mooneyso you could add aggreate_anti_affinity_* set of server tags09:32
sean-k-mooneyand then look at those in the filter/weigher09:32
bauzaseach host getting the same weigh09:32
jkuliksean-k-mooney: that sounds wrong :D but thanks for the idea09:32
sean-k-mooneybauzas: that is doable yes09:32
sean-k-mooneyjkulik: tags are somethimes used for this in other out of tree implemenation like tripleos instance ha feature09:33
jkulikspeaking of the server-group API: we've extended it to allow adding servers to server-groups after they're spawned. this can be helpful if you need to spawn a new instance anti-affin to a previously spawned one, where you didn't know that requirement, yet.09:33
bauzasI'm still confused by the weights09:33
sean-k-mooneyits not how the api is inteded to be used but still09:33
bauzasbut09:33
jkulikis that something that has a chance upstream, if we write a spec for it?09:33
bauzasyou could pass a hint09:33
bauzasand then have a weigher looking up at the hint09:34
sean-k-mooneyschduler hint ya but that also versioned09:34
sean-k-mooneybauzas: oh for the pack vs spread policy09:34
sean-k-mooneyfor weighers09:34
bauzasyup09:34
sean-k-mooneyya so i was thining the weigher woudl jsut look at the metadta in the server-group policy09:35
sean-k-mooneyand ignore the multiplpere sign09:35
sean-k-mooneyor we woudl just set min =0 for the config option09:35
bauzasI'm not in a favor of adding more in the existing server group API but...09:35
sean-k-mooneyso the multiplper for the weigher woudl afffect only magnitude09:35
bauzasI guess we need to agree on the use case09:36
sean-k-mooneywell server group api is preferable to schduler hit09:36
bauzassean-k-mooney: from an UX, I think so09:36
bauzasbut, you know how much I like our server group implementations09:36
* bauzas wonders whether we should just do this for multicreate09:37
bauzaslike --min 2 --hint spread_my_stuff09:37
gibibauzas: you can look at the soft-anti-affinity weigher for reference how to do it09:45
gibiie how to do weigher for soft things09:46
gibijkulik: adding existing instance to server group needs a decision from a) allow adding an instance only if the policy is valid b)allow adding instance even if the policy is not valid yet but will be at the next move09:47
gibijkulik: I think in the past we was not able to agree on which way we should go09:47
gibibut you can try again09:47
jkulikwe've opted for a) ... 09:53
jkulikwhere would be the right point to discuss this?09:53
sean-k-mooneyhttps://github.com/openstack/nova-specs/tree/master/specs09:54
sean-k-mooneysorry my wifi dropped so missed the last few minutes09:54
sean-k-mooneyso not sure waht a) is09:54
sean-k-mooneybut if you want to creat a spec to discuss the desgin upstream copy  https://github.com/openstack/nova-specs/blob/master/specs/zed-template.rst09:55
sean-k-mooneyand fill it in and we can discuss on the spec and on irc09:55
sean-k-mooney[m]ah so looking at my matix client i see i did not miss much09:56
sean-k-mooney[m]i dont like multi create09:56
sean-k-mooney[m]so i dont think we should add this tere09:57
* sean-k-mooney switch back to irc09:57
sean-k-mooneygibi: correct we did not come to an agreement on how to handel the case where the instance did not comply with the policy09:58
sean-k-mooneyi did not like the idea of the add operation implictly live migratinging the instnace09:58
sean-k-mooneywe have basicaly 3 options, 1 reject the request if the policy would be violated, 2 accpet but automticaly trigger a move operation to reconsile the state, 3 allow it to be invalide and retturn some kind or warnign and leave it to the end user to reconsile the state with a move operation10:00
sean-k-mooneyat a later date10:00
jkulikis moving an instance allowed for normal users?10:01
sean-k-mooneytechnially yes but only via a resize or shelve10:01
sean-k-mooneylive migrate and cold migrate are admin only 10:01
gibisean-k-mooney: yeah, from those options I would go with either reject the add if violates the policy, or accept it but only warn (or extend the server group api to show if the policy is invalid), but I definitely don't want to trigger a move automatically10:11
sean-k-mooneyjkulik: gibi  this was the previous spec on the topic https://review.opendev.org/c/openstack/nova-specs/+/78235310:26
jkuliksean-k-mooney: thanks. that will be helpful10:28
jkulikoh, that's proposed as a server-action10:29
jkulikfyi, this is how we built it downstream https://github.com/sapcc/nova/commit/7220be3968ee1dd257c9add88228cc5bb9857795 (+ some commits afterwards to fix certain stuff)10:33
sean-k-mooneyi see that still has the same problem10:41
fricklerdoes that ring a bell for someone? "nova.exception.InternalError: Unexpected vif_type=unbound" unstable failure in OSC jobs, shows as JSONDecodeError https://zuul.opendev.org/t/openstack/build/181d8177eab5428a82facc4d95ce00e210:41
sean-k-mooneyvif_type unbound is what the netron itnerface has before you set the host-id10:42
sean-k-mooneyfrickler: so } openstackclient.tests.functional.compute.v2.test_server.ServerTests.test_server_attach_detach_floating_ip  might be racing10:43
sean-k-mooneywith the server boot10:43
sean-k-mooneyif it has not finished booting when you try to attach the floating ip then you woudl get that issue i guess10:43
frickleroh, so yet another set of tests needing wait-for-ssh things10:43
sean-k-mooneymaybe have not looked at the test yet10:44
sean-k-mooneyits not nessiarly sshable 10:44
sean-k-mooneyit woudl need to be active10:44
sean-k-mooneyso this is the test https://github.com/openstack/python-openstackclient/blob/master/openstackclient/tests/functional/compute/v2/test_server.py#L339=10:45
sean-k-mooneyit shoud be waiting for active 10:45
sean-k-mooneyit looks like the error is coming form self.server_create10:45
sean-k-mooneyah from here https://github.com/openstack/python-openstackclient/blob/20e7b01af8f0fb4cf0f4af253270ad470926ba4e/openstackclient/tests/functional/compute/v2/common.py#L89=10:46
sean-k-mooneyfrickler: so its assuming that a value will be populated in the output i guess10:46
sean-k-mooneyfrom the trace it looks like it got an empty responce of something like that10:47
sean-k-mooney    json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)10:47
sean-k-mooneyfrickler: without a request id it might not be easy to see what the api returned10:48
sean-k-mooney136.246758s that seamed to take  a very long time10:49
sean-k-mooneyis there a timeout for wait?10:49
fricklerneutron says "Concurrent port binding operations failed on port 5b1348e9-4048-4b39-a1df-1161a798052e" before nova fails, so more likely an issue on their side11:15
sean-k-mooneyconcurrent?11:19
sean-k-mooneyhow11:19
sean-k-mooneythat could only happne if we failed reschulded and tried to bind again11:19
sean-k-mooneybut they were sitll binding the port to the first host11:19
sean-k-mooneyso i guess yes that woudl be a neutorn issue11:20
sean-k-mooneythat is not as far as i am aware one of the exctpiotn they are allowed to raise at teh api level11:20
fricklerthat's just a warning in the q-svc log and iiuc they retry and succeed after that. but it may be that they send a notification to nova about the first attempt anyway11:42
sean-k-mooneymaybe i know ralonsoh has a wip patch for somethign else that i dont think will actully help but a race was mentioned in cotext to that11:43
sean-k-mooneyfrickler: https://review.opendev.org/c/openstack/neutron/+/846422/311:44
sean-k-mooneyhttps://bugs.launchpad.net/neutron/+bug/197907211:45
sean-k-mooneythat shoudl actully be fixed in nova11:45
ralonsohsean-k-mooney, right, this is just a WIP patch11:45
ralonsohtrying to address an issue that is on the Neutron side11:45
sean-k-mooneyright please dont11:46
ralonsohhowever that doesn't address the problem of "concurrent port binding"11:46
sean-k-mooneyat least not with a periodic11:46
ralonsohdon't what?11:46
sean-k-mooneytry an fix it form neutron11:46
ralonsohok but the problem is anytime we have this issue, the bug is bug Neutron11:46
sean-k-mooneywe need to fix this form the nova side to avoid posible races between nova and neturon11:46
sean-k-mooneyright so there are two ways to fix this in nova. 1 make sure we delete the inactive port bidning when we revert11:47
sean-k-mooneyi think we try that already today but it can fial11:47
sean-k-mooneysecond when we live migrate or try to create a portbidning 11:47
sean-k-mooneyand it already exits delete and recreate11:47
sean-k-mooneythat will prevent this form breaking in the future11:48
fricklerthe failure in osc isn't related to migration. it happens on initial server create. so I don't understand what could cause the duplicate there11:48
sean-k-mooneyif the initall port create faild and we resuchlde then we will try binding it to a second host11:48
opendevreviewSergii Golovatiuk proposed openstack/nova master: Replace "db archive" with "db archive_deleted_raws"  https://review.opendev.org/c/openstack/nova/+/84796311:48
sean-k-mooneywe wont create a second binding 11:48
sean-k-mooneywe will just update the host-id11:48
sean-k-mooneybut if neutron is still binding it form the first failed attempet we woudl get a concurrent error11:49
sean-k-mooneyfrickler: did you check the logs to see if the vm was retired on a second host?11:49
fricklersean-k-mooney: it is a single-node job, I would be surprised if that happened11:51
sean-k-mooneyfrickler: ok well the only other thing i can think of is the client retry11:53
sean-k-mooneyralonsoh: could this happen if nova retryed creating the port binding because thet inital call timed out11:54
sean-k-mooneyi think we try 3 times11:54
sean-k-mooneythis being "Concurrent port binding operations failed on port ..."11:54
opendevreviewBalazs Gibizer proposed openstack/nova master: Add regression test for bug 1838392  https://review.opendev.org/c/openstack/nova/+/84796512:19
*** dasm|off is now known as dasm13:04
bauzasgibi: sean-k-mooney: jkulik: fwiw, this is not a new meat https://review.opendev.org/c/openstack/nova-specs/+/13000513:08
sean-k-mooneybauzas: yep i know13:08
bauzassure, but I provided the existing context13:08
sean-k-mooneythere are other specs more recnet then that13:09
bauzasjust look at the gerrit comments13:09
sean-k-mooneybut ya we have rejected dynmic server groups and adding other affinity policies in the past13:09
bauzasyes and no, this is just the fact that the spec was split https://review.opendev.org/c/openstack/nova-specs/+/139272 and https://review.opendev.org/c/openstack/nova-specs/+/13648713:10
bauzaslook again about the comments13:10
bauzassean-k-mooney: not for you, but rather jkulik13:10
jkulikbauzas: thanks, will take a look13:11
sean-k-mooneybauzas: ack13:11
bauzasjkulik: there was by then a try to have a new API instead of server groups https://review.opendev.org/c/openstack/nova-specs/+/183837/4/specs/liberty/approved/generic-scheduling-policies.rst13:13
sean-k-mooneybauzas: im not sure how that would help13:13
sean-k-mooneythe issue with server groups is adding an instance that violates the policy13:14
bauzasjust saying this is a can of worms13:14
sean-k-mooneyit is13:14
bauzassean-k-mooney: the problem with server groups is that if we touch it, it creates more races than the ones it fixes13:14
sean-k-mooneybut that is a sperate topic form the orginal thing that jkulik  raised13:14
bauzassean-k-mooney: well, this is all about colocality13:15
bauzaswe express this in Nova with server groups13:15
sean-k-mooneywell affinity and anti affity13:15
sean-k-mooneywith differnt granularity13:15
sean-k-mooneybut yes13:15
bauzasbut the question remains about the best UX we may have13:15
bauzasanyway, me goes back at bug scrub13:18
Ugglaquestion, from the api, I try to check that a scheduling is impossible. To do that I look at 'No valid host found for unshelve instance' in the log. My test seems to work, but I have to wait before checking the logs. Is there a proper way to do that ?14:29
Ugglabauzas, gibi ^14:30
sean-k-mooneyusing notifications14:33
sean-k-mooneybut in general we dont tend to use logs in tests14:33
sean-k-mooneywe do sometimes but there are often better way to do that14:33
Ugglais  _wait_for_action_fail_completion a possible option ?14:33
gibiUggla: you wait for the server to go to ERROR state then you can check the fault in the server to see if it is a no valid host14:43
gibigive me a sec and I will find an example14:43
*** diablo_rojo is now known as Guest352514:43
gibiUggla: for example https://github.com/openstack/nova/blob/c53ec4e48884235566962bc934cbf292ad5b67b8/nova/tests/functional/test_servers.py#L4100-L410814:45
Ugglagibi, ok probably better that what I have just done using wait_for_assert().14:45
bauzasUggla: not sure I understand your question15:07
bauzasyou're asking how nova-api could know there is a scheduling error ?15:07
Ugglabauzas, no worries Gibi is helping me right now.15:07
bauzasoh, for testing15:08
sean-k-mooneyyep testing15:08
sean-k-mooneypresumable functional testing15:08
sean-k-mooneyrather then unit15:08
gibiyepp func testing15:11
gibiUggla just found the limitation of the our nova.tests.functional.integrated_helpers.InstanceHelperMixin._wait_for_instance_action_event assert15:11
gibiit always checks the first action of a given type from the list of instance actions15:12
gibiand he had two unshelve actions in the test case15:12
gibiand the assert only checked the first 15:12
gibieven though the second was in error state15:12
bauzasreminder: nova meeting in 30 mins15:29
sean-k-mooneyoh fun15:31
sean-k-mooneyi was not aware we had that limitation15:31
gibime neither, but now Uggla can improve on that :)15:40
Ugglagibi, sean-k-mooney , currently not completely sure but no the code looks ok. It is more the event I'm looking for which is not the good one.15:42
gibiit can be that the listing of the instance actions are not stable so sometimes the code founds the proper unshelve action sometimes not15:43
gibiI mean the sorting is not stable15:48
*** diablo_rojo__ is now known as diablo_rojo15:54
bauzas#startmeeting nova16:00
opendevmeetMeeting started Tue Jun 28 16:00:11 2022 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
opendevmeetThe meeting name has been set to 'nova'16:00
bauzashello everyone16:00
bauzasmmmm16:01
bauzasheyhoooooooo16:01
* bauzas awaits the echoing16:01
gibio/16:02
bauzashah, I hear some sound16:02
gibicrickets ?16:02
bauzasI'm maybe in a cave16:02
bauzasgibi: not sure we have quorum for today's meeting :(16:03
elodilleso/16:03
gibisean-k-mooney already gone for today16:03
gibiI guess melwitt is still on PTO16:03
bauzashah16:03
bauzasyes indeed + artom16:04
bauzaswhat would you want ? 16:04
gibido we have somebody here for the todays nova meeting to talk about specific things? (if not then we can close this)16:05
bauzaswe still have one critical bug16:05
elodillesnothing special news from stable point of view, so -16:05
bauzasI can be the bug baton owner for the next week16:06
bauzasthat said, next week we will have a spec review day16:06
bauzasI'll email it16:06
bauzasthat's it for me16:06
gibinothing from me I spent most of the week downstream 16:07
bauzas#info Next bug baton is still for bauzas16:07
bauzas#info One Critical bug16:07
bauzas#link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 8 new untriaged bugs (-4 since the last meeting)16:07
bauzas#link https://storyboard.openstack.org/#!/project/openstack/placement 27 open stories (+1 since the last meeting) in Storyboard for Placement 16:07
bauzas#link https://storyboard.openstack.org/#!/story/2010108 new Placement bug16:07
bauzasbut I'll tell it again next week16:08
bauzas#info Spec review day on July 5th16:08
bauzasthat's basically it16:08
gibiI guess in the critical bug we keep the job non-votig while waiting for the fix to be released in centos stream 916:08
bauzasgibi: yeah, we'll discuss about it next week16:08
gibiok16:08
bauzasI have a concern about the centos 9 stream job16:09
gibithen I think we can close early today16:09
bauzasok, then16:09
bauzas#agreed given we don't have quorum for this meeting, let's punt it for this week until next week16:09
bauzas#info remember we'll have a spec review day next week16:09
bauzasthat's it, thanks16:10
bauzas#endmeetingh16:10
bauzasmeh16:10
bauzas#endmeeting16:10
opendevmeetMeeting ended Tue Jun 28 16:10:14 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:10
opendevmeetMinutes:        https://meetings.opendev.org/meetings/nova/2022/nova.2022-06-28-16.00.html16:10
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-06-28-16.00.txt16:10
opendevmeetLog:            https://meetings.opendev.org/meetings/nova/2022/nova.2022-06-28-16.00.log.html16:10
bauzaseven16:10
gibihave a nice evening folks16:11
elodillesyou too o/16:11
Ugglabauzas, already done ! 16:15
bauzasUggla: yeah we didn't had a quorum16:15
Ugglaoh ok.16:16
opendevreviewMerged openstack/osc-placement master: Replace deprecated assertRaisesRegexp  https://review.opendev.org/c/openstack/osc-placement/+/81736516:27
bauzasgibi: sean-k-mooney: Uggla: others: I forgot to tell I'll be on PTO tomorrow16:55
sean-k-mooney[m]ok16:57
*** diablo_rojo is now known as Guest354417:40
*** dasm is now known as dasm|afk19:34
opendevreviewDan Smith proposed openstack/nova master: WIP: Remove system scope from all APIs  https://review.opendev.org/c/openstack/nova/+/84802119:47
dansmithgmann: ^ passes unit tests locally for me, we'll see what happens in functional19:47
dansmithalso, side note19:47
dansmithI'm f**king tired of policy stuff19:48
gmanndansmith: thanks for that. I was also fixing some unit tests but functional test will be good i think.19:49
gmanndansmith: agree, same here on policy stuff.19:49
dansmithack, it would be GREAT if I don't have to mess with functional failures19:49
*** mfo is now known as Guest357222:51
*** mfo_ is now known as mfo22:51

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!