Thursday, 2024-07-18

opendevreviewzhou zhong proposed openstack/nova master: nova-manage: modify image properties in request_spec  https://review.opendev.org/c/openstack/nova/+/92431902:29
opendevreviewzhou zhong proposed openstack/nova master: nova-manage: modify image properties in request_spec  https://review.opendev.org/c/openstack/nova/+/92431902:42
*** bauzas_ is now known as bauzas04:41
*** chuanm0 is now known as chuanm07:25
*** bauzas_ is now known as bauzas07:29
bauzasgibi: other cores : we need a second core approval against https://review.opendev.org/c/openstack/nova-specs/+/90770209:49
bauzasas a reminder, today EOB is our spec approval freeze day09:50
gibibauzas: done, I agree we can document the remaining agreements in a follow up to the spec.10:22
gibiso I approved it.10:23
mikalbauzas: ugh, I'm not going to be able to update the kerbside spec before the end of your day. I thought the deadline was tomorrow, and I have most of a prototype but then need to update the spec to match.10:29
mikalbauzas: I wanted to get a working prototype beforehand so I was confident I was describing it correctly.10:30
mikalbauzas: what are my chances of a one or two day extension?10:30
opendevreviewMerged openstack/nova-specs master: libvirt: AMD SEV-ES support  https://review.opendev.org/c/openstack/nova-specs/+/90770210:37
mikalSpeaking of which, has anyone ever gotten a GET /os-console-auth-tokens/{console_token} to work? I get a 404 which I can't really explain.11:42
opendevreviewAndrey Kurilin proposed openstack/nova master: Propagate neutron's error during removeSecurityGroup action  https://review.opendev.org/c/openstack/nova/+/92439911:51
opendevreviewMichael Still proposed openstack/nova-specs master: Propose a new API microversion for a spice-direct console.  https://review.opendev.org/c/openstack/nova-specs/+/91519012:29
mikalsean-k-mooney: I've updated the spec. I just got the console access URL stuff you asked for working in a prototype.12:30
mikalsean-k-mooney: sorry it took so long, I had some personal life stuff I've had to deal with in the last week or so.12:30
sean-k-mooneymikal: you dont need to appoliges, life happens12:32
mikalsean-k-mooney: yeah I dunno, I feel like the last month or so has been pretty weird over here.12:33
mikalsean-k-mooney: anyways, I have the flow you wanted working: request console access URL through existing API with new console type -> get URL to kerbside with console access token -> request URL -> kerbside uses token to validate and lookup connection info -> kerbside returns a .vv file for your SPICE client.12:34
mikalsean-k-mooney: the code is not ready to push to gerrit yet, but I am now confident I can make that flow work.12:34
mikalsean-k-mooney: which means arguably there is no API security impact, as the console auth lookup API is pre-existing and already did what I needed12:34
sean-k-mooneydo we need a sepreate config option ? CONF.spice.kerbside_base_url or would https://docs.openstack.org/nova/latest/configuration/config.html#serial_console.base_url work12:35
sean-k-mooneysorry not serial12:35
sean-k-mooneyhttps://docs.openstack.org/nova/latest/configuration/config.html#spice.html5proxy_base_url12:35
sean-k-mooneyi guess hte name of that is slightly confusing12:36
sean-k-mooneymikal: im basically wonderign if we could rename html5proxy_base_url to proxy_base_url12:36
sean-k-mooneyand use it for both12:36
mikalsean-k-mooney: yeah, I wanted to differentiate the two services.12:36
mikalsean-k-mooney: I would be ok with that, but that would come with upgrade work for deployers.12:37
sean-k-mooneydo you want to suport haveing both the existing html5 proxy and kerbside at the same time12:37
sean-k-mooneymikal: evenrually when we do a rename we keep the old value for a couple of cycles12:37
mikalsean-k-mooney: actually, yes I suspect people would want that. HTML5 is fine for quick and dirty things, but not for desktop VDI replacement.12:38
sean-k-mooneyok that works for me12:38
mikalThat is, if I was building Citrix on OpenStack, I'd probably want both.12:38
sean-k-mooneymikal: ya so to get the port/host infor you will need to call the /os-console-auth-tokens api as admin i belive12:40
mikalsean-k-mooney: yeah, I am assuming the security for that is already correct given that's pre-existing.12:40
sean-k-mooneyit would be nice to allow that to be called as the serivce user so that is the only policy change requried12:40
mikalDoes that need to be in the spec?12:40
sean-k-mooneybut i agree that most of the api change is removed now12:40
sean-k-mooneyit woudl be nice to cacue the tin the spec but i would not stirlcly hold a -1 for that alone12:41
sean-k-mooneyif you want to respine that is fine but i woudl be ok with a follow up patch post spec freeze too12:41
sean-k-mooneymikal: stricly speaking spec freeze is eob today  so im going to finish reviewing this and ill leave a comemnt to that effect regarding policy12:42
mikalsean-k-mooney: yeah, I feel like I am skating on the edges of missing the deadline. I asked bauzas if I could have a day or two extension, but didn't get a reply yet.12:42
mikalI thought the freeze was EOD tomorrow, which is my bad.12:43
sean-k-mooneymikal: so on one hand we premetivly moved it 2 weeks past our normal deadline. on the other given you have a partly wroking poc and you have already adapted the appoch to the direction i asked for i would be ok with extending until the next team meeing via the spec freez process if bauzas is12:44
mikalOh bugger. I _do_ need a schema change. There is only one port column in the ConsoleAuthToken table and I need two (one for TLS).12:44
mikalI'll tweak the spec now.12:44
sean-k-mooneyok well that is easy to add to the spec i dont think that woudl be contoverial12:45
mikalLook, if I've missed the deadline so be it. Its frustrating but its not the end of the world.12:45
opendevreviewMichael Still proposed openstack/nova-specs master: Propose a new API microversion for a spice-direct console.  https://review.opendev.org/c/openstack/nova-specs/+/91519012:47
sean-k-mooneymikal: i think you have some other issues12:48
sean-k-mooneymikal: im not seeing the propsoe chage header12:48
mikalI'm not sure what a "propose change header" is?12:49
mikalOh you mean in Gerrit?12:49
sean-k-mooneyyep it will fail our ci job12:49
sean-k-mooneyhttps://github.com/openstack/nova-specs/blob/master/specs/2024.2-template.rst?plain=1#L9212:49
sean-k-mooneyoh no12:50
sean-k-mooneyyou have it but you started descibibg the propsoed change in the problem description section12:50
sean-k-mooneylets ignore that for now but basiclly the "Problem description" shoudl descibe the "what" and "why" not the "how"12:51
sean-k-mooneyso the discussion of config option or api shoudl be in the propsoe changes btu that not a big deal12:51
mikalSo basically everything except the first para of "problem description" should move down?12:51
sean-k-mooneytechnially yes to the sart of the propsoed changes12:52
sean-k-mooneythe usecases section is correct12:52
mikalDone.12:52
opendevreviewMichael Still proposed openstack/nova-specs master: Propose a new API microversion for a spice-direct console.  https://review.opendev.org/c/openstack/nova-specs/+/91519012:52
sean-k-mooneycool ill contineu reviewing the new version12:53
sean-k-mooneysofar the content looks fine to me12:53
sean-k-mooneymikal: by the way if you have not seen this https://review.opendev.org/c/openstack/nova-specs/+/920687 i think that woudl also be very useful to you12:56
sean-k-mooneywe may not have time to fully review and approve that todya as well 12:56
sean-k-mooneybut i think both your fature and that could work nicely togetter going forward12:57
mikalOh that's interesting. SPICE tunnels USBredir over its transport, and that already works in my prototype13:03
mikalThat spec presumably is also proposing to add the same USB devices that I need to add for that to work in SPICE? I shall read the spec.13:04
sean-k-mooneyyes and adding image/flavor properies to say how many13:04
sean-k-mooneythere is a lot of overlap in some respect although they were happy with the html5 proxy for there usecase13:05
sean-k-mooneyform the naming im pretty sure this is for https://shadow.tech/13:06
mikalThe HTML5 proxy supports USB redirection? I find that surprising.13:06
sean-k-mooneymikal: why? it a javascript applcation hosted on a webserver that tunnes the raw spice protocal over a webseocket 13:06
sean-k-mooneywe are litrally just wrapping the spice tcp connetcion into a websocket and providign that to the browser13:07
mikalSo I haven't looked at the websocket proxy much, but the javascript app is definitely missing stuff -- it doesn't support all the compression options that the protocol does for example. Its missing GLZ IIRC.13:07
sean-k-mooneymikal: right but that not a limitaiton fo openstack 13:08
mikalThat's fair.13:08
sean-k-mooneyshowdow likely have there own client that they are using with the websocket directly13:08
sean-k-mooneythey are defeitly not using horizon's or skylines13:09
mikalTheir site definitely mentions apps, but also says they have browser support.13:10
mikalBut yeah, they seem to be proposed a relatively overlapping subset of what I am proposing.13:10
sean-k-mooneyyep but you can do a lot in in a moddern browser :)13:11
sean-k-mooneyanyway if we can enabel both yoru use cases i would be happy too13:14
sean-k-mooneymikal: +1 https://review.opendev.org/c/openstack/nova-specs/+/915190 minor line lenght issue on line 79 and you still need to menation addding the tls_port in the respocne13:18
sean-k-mooneymikal: but if you fix those i think im +2 on the spec13:19
sean-k-mooneyi have also resolved teh irrelevent comments form older version so it shoudl be simpler to review now13:20
sean-k-mooneybauzas: gibi  can ye review ^ if you have time13:20
bauzasI'll do13:20
opendevreviewMichael Still proposed openstack/nova-specs master: Propose a new API microversion for a spice-direct console.  https://review.opendev.org/c/openstack/nova-specs/+/91519013:31
mikalsean-k-mooney: I've done both of those.13:31
mikalsean-k-mooney: all of your comments about docs and so forth seem reasonable to me, but I am not passionate about updating the spec to include them right now given I worry that will derail the review process.13:33
mikalsean-k-mooney: thanks for taking the time to work through the spec with me by the way, I appreciate it.13:35
sean-k-mooneymikal: +2, no worries, happy to collaberate on thigns like this13:44
sean-k-mooneyi may or may not be able to comit to reviewing it before feature freeze but ill try as time allows13:44
mikalpep8 hates me.13:45
sean-k-mooneyyou know you can run that locally via docs right :P13:45
mikalIt only reported one error last time, so I thought if I fixed that it wouldn't be angry any more...13:45
sean-k-mooneyah13:46
mikalI have learnt my lesson13:46
sean-k-mooneyya we normaly config it to keep going but that does not alwasy work depending on the tool we use13:46
opendevreviewMichael Still proposed openstack/nova-specs master: Propose a new API microversion for a spice-direct console.  https://review.opendev.org/c/openstack/nova-specs/+/91519013:47
mikalWell, it passes locally now so let's see what Zuul thinks.13:47
mikalI really should go to bed, its nearly midnight here. I guess I'll see if this thing is approved in the morning. Thanks again for your help.13:48
mikalsean-k-mooney: it passes CI now, so that's nice. On that note I'm off to bed.13:57
bauzassean-k-mooney: mikal : I have two comments in https://review.opendev.org/c/openstack/nova-specs/+/91519014:49
bauzasone about testing at least14:49
bauzasI'm okay with accepting this spec, but only provided we're clear on the testing side14:50
sean-k-mooneybauzas: well i suggested tempest testign at the ptg and we said we shoudl not make that a requirement14:51
sean-k-mooneyso the expection is unit and fucntional tests only14:51
sean-k-mooneywiht no devstack based tempest job14:51
dansmithdid we?14:52
sean-k-mooneyi origally asksed for a devstack plugin and to enabel it in one of our jobs but dansmith and other feel that was two high a bar to require in the spec14:52
dansmithI really thought I brought up that we at least sniff test the RFB banner for our existing vnc stuff14:52
bauzassean-k-mooney: see my comment14:52
sean-k-mooneydansmith: i susggested a temept test like that but i tought you said no to that14:53
bauzasI proposed a fake object in the functional tests that would reproduce some external call 14:53
sean-k-mooneydansmith: im not agaisn doing that just liek we do for vnc14:53
dansmithsean-k-mooney: okay I thought the opposite, but it's been a while14:53
bauzasbut I'm afraid unittests and API functional tests aren't enough14:53
sean-k-mooneybauzas: sure so i think we can approve the spec as is if you have no other obejctsion and work on a trivial devstack plugin with mikal as part fo the implemnation and add this into one of our exsitign jobs14:56
bauzassean-k-mooney: I'm not really asking for a devstack plugin14:56
sean-k-mooneythe plugin would resided in the kerbside repo or a new dedeicated repo14:56
sean-k-mooneybauzas: well something need to be able to deploy kerbside14:56
bauzasI'm just asking for a fake object that would simulate the HTTP calls14:56
bauzasso we just call we correctly call it14:57
sean-k-mooneywe dont call kerbside 14:57
sean-k-mooneyit calls us14:57
sean-k-mooneywe shoudl have tempest coverage for the new api changes 14:57
sean-k-mooneyor could14:57
sean-k-mooneybut im not sure that give use anything over the api sample tests14:58
sean-k-mooneyunless we actully deploy with kerbside as well in a tempest job14:58
bauzashmmmmmmmmm14:58
bauzasthe whole spec is about passing a configurable URL to our API 14:59
bauzasthat's it14:59
sean-k-mooneynot entirly 14:59
sean-k-mooneyit about that + adding new devices to the xml14:59
sean-k-mooneyand  2 addtion tional api chagnes14:59
sean-k-mooney1 to define the console type15:00
sean-k-mooneyand another to expsoe the tls_port via the existing admin only console token api15:00
bauzaslemme be clear15:00
bauzasI was confusing15:00
bauzas"15:00
bauzas"The user then fetches that URL, and Kerbside delivers a .vv file with the15:00
bauzasconnection information for a SPICE client. Kerbside uses a call to15:00
bauzas`/os-console-auth-tokens/bf2e6883-...` to determine the validity of the15:00
bauzasconsole authentication token, and the connection information for the console."15:00
bauzashere, I think we can have a functional test that would reproduce the kerbside server by calling the os-console-auth-tokens15:01
bauzasas a mock object15:01
sean-k-mooneysure 15:01
sean-k-mooneybut thats just a standar fucntional test15:01
bauzasthis way, we wouldn't need to depend on kerbside15:01
sean-k-mooneysure but i was edxpecting that already15:02
bauzasright, that's what I'm asking in https://review.opendev.org/c/openstack/nova-specs/+/915190/13/specs/2024.2/approved/libvirt-spice-direct-consoles.rst15:02
bauzasfrom what I just read, mikal only proposed manual testing15:02
sean-k-mooneywe can make that more explit but i was starting form a baseline of "all new feature should have unit and functional tests as our minium baseline"15:03
sean-k-mooneyno15:03
bauzasfor functional tests, quoted " However, a test for the15:03
bauzasAPI microversion will be added"15:03
bauzaswhich is different from having a mock object that would simulate kerbside external call to Nova15:03
sean-k-mooneyso i expect the testing section of the spec to call out any special testing behiound our baselien15:04
sean-k-mooneyi dont in genrall expect to say "we will add unit and fucntional tests" but if you want that to be updated to included that im fine with that15:04
sean-k-mooneythe sepcific thing you asked for however i dont think makes sense for us to test15:05
sean-k-mooneynova has 2 existign apis both fo which have test coverage15:05
sean-k-mooneythe console url show rpvoded a url that is templated based on  `spice.kerbside_base_url`15:06
bauzas_okay, then I'll accept the spec but asking for a FUP15:06
sean-k-mooneythe url provided to the use rwhen they do show has a console token which kebside will use to call /os-console-auth-tokens15:06
sean-k-mooneyso im expeicting 2 seperte tests15:06
sean-k-mooneyone to cover the new console type for console url show15:07
sean-k-mooneyand one to test the addtion of a tls_port for spice-direct in /os-console-auth-tokens15:07
*** bauzas_ is now known as bauzas15:12
opendevreviewMerged openstack/nova-specs master: Re-propose using extend volume completion action for 2024.2  https://review.opendev.org/c/openstack/nova-specs/+/91713315:53
bauzassean-k-mooney: dansmith: gibi: I know this is late, but could you please look at https://review.opendev.org/c/openstack/nova-specs/+/922201 if you have time today ?16:05
bauzasI can also try to look at it before leaving16:05
dansmiththere's a ton of unresolved comments on that?16:09
bauzas_I think fwiesel replied on sean's comments 16:12
bauzas_but sure, I can look at those 16:12
dansmithbut there's a bunch that reference lines not even in the file anymore and so they're stacked at the top16:13
bauzas_due to an older revision16:13
bauzas_but I can clean them up or stick them 16:13
bauzas_lemme look before you go in16:13
sean-k-mooneyif the lazy loadign was optional and off by defualt. i could maybe see triling this but im still not entirly sure this the best approch16:16
bauzassean-k-mooney: can you then just reply to Fabian's comments ? 16:27
bauzasI agree with the fact that I wouldn't rush to merge that spec given your concerns16:27
bauzasI'd rather propose that spec to be discussed at the next PTG16:27
bauzasfwiesel ^ 16:27
dansmithhe does specify it being optional, but not sure about default16:31
dansmithbut I agree, I'm really not sure about the claims made in there, as this seems like it would be a gateway to increasing load16:31
sean-k-mooneyi woudl actully prefer to reduce the load on neutron by reviving https://review.opendev.org/c/openstack/nova/+/78634816:32
dansmithit may be better in the one case where you want to re-fetch one thing, but it seems like it opens the ability (and perhaps even required behavior) of generating many more small requests at the behest of a client that may want to DoS the metadata server16:32
dansmithI thought the plan was just to cache that stuff in with metadata instead of always fetching it from neutron, tbh16:33
sean-k-mooneywell thats what the older patch is doing16:33
dansmithidk that we need to store it in info_cache directly and I can see reasons why that would be bad and confusing16:33
dansmiththe older patch is storing it in info_cache, rihgt?16:33
sean-k-mooneyits adding the security groups to the network info cache16:33
sean-k-mooneywhich speed up both the metadta api and server show16:33
dansmithright, but what I'm saying I thought the plans was, is to just cache it in the built metadata16:33
dansmithyeah I know, but if you immediately add a floating and then server show and it's not there, that's going to suck16:34
sean-k-mooneyoh am i was not in the ptg session so i dont know16:34
dansmithwhich I think is why we're not doing that today, IIRC16:34
sean-k-mooneywe do already cache all the metadata once built today btu its time based with no way to triger the invalidation16:34
dansmithnetwork-changed doesn't dispatch for adding a floating right?16:34
sean-k-mooneyit might but i dont think applciationcation should be relaying on the metadata contnet to be up to date 16:35
sean-k-mooneywe do get network-chaged for a number of things but i dont knwo of a list that is written down anywhere to check16:35
dansmithI was talking about server show16:35
dansmithI don't think applications can rely on anything other than metadata for that info right?16:36
sean-k-mooneythey shoudl really call neutorn if the need the athoritive info16:36
sean-k-mooneybut if they dont know they are on openstack then no16:36
sean-k-mooneythis is all they have16:36
dansmithapplications inside guests can't do that unless they have creds, which we also don't provide them with :)16:36
dansmithlike some thing that needs to provision on boot and know it's external IP so it can offer it up to clients needs to ask metadata for that info16:37
sean-k-mooneyyep ops have asked for that JWS tokens speficically16:37
sean-k-mooneydansmith: so the boot case shoudl be up to date today beacue its the first request but yes if it change during the boot16:38
sean-k-mooneyi.e. a floting ip was added while it was booting it could be racy16:39
sean-k-mooneywhat would be nice at somepoint is to be able to povide a short lived bootsrap took like kubernbetes does but thats a much bigger feature16:40
sean-k-mooneydansmith: may main concern with the lazy loading spec is it does not explain how its going to do this16:41
sean-k-mooneyliekk is the plan ot create a url router within the metadata code to fetch only the conent for that file via decompoing things into lots of funcitons16:42
sean-k-mooneyor do some kind of properlty based laze loading on a class16:42
sean-k-mooneythere is really just https://review.opendev.org/c/openstack/nova-specs/+/922201/9/specs/2024.2/approved/lazy-metadata-loading.rst#7916:44
dansmithsean-k-mooney: yeah, if there's a race in the boot, add floating, cloud-init sequence it could be wrong16:53
sean-k-mooneyso to me the metadata api comes under the banner of "no more proxy apis" in nova17:08
sean-k-mooneyi.e. its not a data socue that shoudl be proxiying up to date info form other service its just a cached view of some useful info17:08
sean-k-mooneyso the idea that applciations are writen to poll it at a rate 17:09
dansmithidk, I don't think that's reasonable if there is literally no other way to get the info and we're holding the guest in our hand17:09
sean-k-mooneythat is enough to cause load issues is kind of bizar to me17:09
dansmithlike, if we start provisioning them with credentials and a directory, then I'm all for it17:09
dansmith(provisioning with credentials that can be renewed I should say)17:09
sean-k-mooneyya so at some point i think that should be done like how pods in k8s get service credital tokens to be able to talk to k8s17:10
*** elodilles is now known as elodilles_ooo17:10
sean-k-mooneybut short term if we can optimise it then ok im just concerned by the lack of detial on how17:10
dansmithright17:10
dansmithpeople have been asking for that for a long time17:10
opendevreviewSylvain Bauza proposed openstack/nova master: cpu: Only check governor type on online cores  https://review.opendev.org/c/openstack/nova/+/92442717:11
sean-k-mooneyya i think its a reason able thing to od at least if the user opts into it17:11
sean-k-mooneybut its not what the spec is adressing17:11
bauzassean-k-mooney: I had to drop due to a bugfix, but please just capture your concerns on that lazy-loading metadata spec so you can move on 17:12
bauzasand we'll revisit that spec at the PTG17:12
sean-k-mooneybauzas: i rased most of them in the spec already, mainily i dont know how they are planning to implemnte the lazy loading and im concured it will break cloud-init or cirros-init due to the fact they dont retry by default and some recuest take longer then other to formulate17:16
bauzas++17:17
sean-k-mooneybauzas: bascily cloud-init and cirros-init today only retry the first request and assuem the rest will be fast and work first try by default17:17
sean-k-mooneycloud init cna be configured to retry its just not the default17:17
sean-k-mooneyso if this is on by defualt im concerned it woudl regression previous bugs we had where vms would not get an ssh key inejcted ectra17:18
dansmithsean-k-mooney: yeah, that's exactly my concern about making this less obvious by having some requests work and some not because they're all independent on the backend17:20
dansmithI missed your comment about that but +10017:20
dansmithI think if we want to defeat caching in certain circumstances we *should* make cloud-init declare when it wants uncached data or not,17:20
dansmithand otherwise we're just guessing17:20
dansmithbut splitting this into N backend requests doesn't seem to improve things at all to me17:21
sean-k-mooneyi was wondering if there was soem way to use etag or just support triggerting the cache invalidation form the clinet side i.e. cluod init17:26
sean-k-mooneyor curl17:26
sean-k-mooneyi dont know what the standard convetion is for that at a http level but im sure there is one17:26
sean-k-mooneysound like there is a semi standardisaed HTTP PURGE method that cloudflare, varnish and  squid  all support to do this17:30
dansmithcache-control, I mentioned it in my review17:31
sean-k-mooneycache-contol is slightly different17:31
sean-k-mooneythat says if caching layers are allowed to cache it or not and for how long17:32
dansmithit's usable on request as well17:32
sean-k-mooneythat works with etag to allow the caching server to know if the content changed17:32
sean-k-mooneyoh really17:32
dansmithhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control17:32
sean-k-mooneyso you can send a cache-contole no-store or similr17:32
dansmithmax-age, no-cache, etc would be perfect here I think17:32
sean-k-mooneyya i  tought that was responce only 17:33
sean-k-mooneybut if it can be in the request17:33
sean-k-mooneythen yes17:33
sean-k-mooneyya so that seams quite reasonabel to look into17:34
opendevreviewMerged openstack/nova-specs master: Propose a new API microversion for a spice-direct console.  https://review.opendev.org/c/openstack/nova-specs/+/91519017:56
*** bauzas_ is now known as bauzas19:26
opendevreviewmelanie witt proposed openstack/nova master: nova-manage: Add flavor scanning to migrate_to_unified_limits  https://review.opendev.org/c/openstack/nova/+/92411020:40
opendevreviewmelanie witt proposed openstack/nova master: nova-manage: Add flavor scanning to migrate_to_unified_limits  https://review.opendev.org/c/openstack/nova/+/92411023:40

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!