Tuesday, 2023-10-31

*** ministry is now known as __ministry08:11
opendevreviewSylvain Bauza proposed openstack/nova master: WIP: Remove no longer used RPs  https://review.opendev.org/c/openstack/nova/+/89962511:22
sahidbauzas, dansmith o/11:31
sahidhow do you feel about making progress on that one? https://review.opendev.org/c/openstack/nova/+/89651211:31
opendevreviewSylvain Bauza proposed openstack/nova master: WIP: Remove no longer used RPs  https://review.opendev.org/c/openstack/nova/+/89962513:53
*** han-guangyu_ is now known as han-guagnyu13:58
*** han-guagnyu is now known as han-guangyu13:59
bauzasokay, this time I guess this reminder is needed : nova meeting in 1 hour here 15:00
bauzaseuropean folks, you now know15:00
elodillesohh, thanks, i completely forgot about the meeting will start one hour earlier... :S damn phones & stuffs that adjust their time automatically /o\15:14
elodilles(* 1 hr earlier for european folks who just moved to winter time)15:16
clarkbelodilles: if you use iceland time (on android at least) this is basically UTC since they don't do DST15:21
elodillesclarkb: ah, good to know :)15:28
bauzas#startmeeting nova16:00
opendevmeetMeeting started Tue Oct 31 16:00:26 2023 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
opendevmeetThe meeting name has been set to 'nova'16:00
bauzashey folks16:00
bauzas#link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting16:00
bauzaswho's around ?16:00
bauzasdo we have quorum ?16:00
bauzasnothing urgent to say this week given we had the PTG last week16:00
elodilleso/16:01
Ugglao/16:01
elodilleseveryone is travelling back from PTG i guess :D16:01
bauzasyeah16:02
sean-k-mooneyo/16:02
bauzassome rh folks also have some meetings16:02
sean-k-mooneythose have finished already fyi16:02
sean-k-mooneywell ignorign the one that overruning 16:02
bauzasokay we can wait a bit16:02
sean-k-mooneywe proably can get started16:03
bauzaswell, okay16:03
* bauzas was working on a customer bug meanwhile :)16:03
bauzas#topic Bugs (stuck/critical) 16:03
bauzas#info No Critical bug16:03
bauzas#link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 36 new untriaged bugs (-11 since the last meeting)16:03
bauzaskudos to melwitt and gibi for triaging !16:03
bauzas#info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster16:04
bauzasUggla: would you be happy with taking the baton this week ?16:04
Ugglabauzas, yep it's ok.16:04
bauzascool thanks16:05
bauzas#info bug baton is Uggla16:05
bauzas#topic Gate status 16:05
bauzas#link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:05
gmanno/16:05
bauzas#link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status16:05
bauzasall greens16:05
elodilles\o/16:05
bauzas#info Please look at the gate failures and file a bug report with the gate-failure tag.16:05
bauzasI think I've seen some gate failures on my rpc series16:06
bauzasbut I need to verify which jobs were having problems16:06
bauzasI hadn't time yet to recheck16:06
bauzasmoving on ?16:07
bauzas#topic Release Planning 16:08
bauzas#link https://releases.openstack.org/caracal/schedule.html16:08
bauzas#info Caracal-1 milestone in 2 weeks16:08
bauzaswe agreed last week on the fact today should be a spec review day but unfortunately I hadn't provided an email16:09
bauzasso if you want, next week will be the first spec review day16:09
bauzasfine ?16:09
bauzasnext tuesday*16:09
sean-k-mooneythe spec review day was not ment to be today16:09
bauzaswell, I will be on bank holiday tomorrow (+ Uggla + kashyap IIRC) and on Friday I'll be on PTO16:10
kashyap*Yep)16:10
bauzashttps://etherpad.opendev.org/p/nova-caracal-ptg#L175 said R-22 would be our spec review day16:10
sean-k-mooneybauzas: i gues syou did say r-2216:12
sean-k-mooneyi feel like that was a mistak16:12
sean-k-mooneyi tought i suggested m1 for that bvut no worries16:12
bauzasno, I remember we said 'the week after PTG'16:12
sean-k-mooneywe can do it next week16:12
bauzasm-1 was for reviewing implementations16:13
sean-k-mooneyoh we are doing the implemation review day on m116:13
sean-k-mooneyyep16:13
sean-k-mooneyso ya we missed the first sepc review day16:13
bauzasyeah, my fault16:13
sean-k-mooneyhonestly im still burnt out form ptg so next week woudl be better16:13
bauzas-ETOOMANYTHINGS16:13
bauzasokay, agreed16:13
bauzas#info Next Tuesday (Nov 7) we will have a spec review day16:14
bauzas#action bauzas to community on that review day16:14
bauzas#unfo16:14
bauzas#undo16:14
opendevmeetRemoving item from minutes: #action bauzas to community on that review day16:14
bauzas#action bauzas to communicate on that review day16:14
bauzasI'll also communicate the implementation review day in 2 weeks from now16:15
bauzasmoving on16:15
bauzas#topic Caracal vPTG planning 16:15
bauzas#undo16:15
opendevmeetRemoving item from minutes: #topic Caracal vPTG planning 16:15
bauzas#topic Caracal vPTG wrapup16:15
bauzas#link https://etherpad.opendev.org/p/nova-caracal-ptg PTG etherpad16:15
bauzasI have some draft in my emailbox that I need to eventually write16:16
bauzasI just don't know when yet16:16
bauzasfor the moment, please look at the etherpad16:16
elodilles++16:16
bauzaswe had one single topic we were unable to talk16:17
sean-k-mooneybauzas: wehn you send the eamil16:17
sean-k-mooneyplease use the readonly link for the etherpad16:17
bauzaslike every cycle I do ? sure16:17
bauzasI already did16:17
sean-k-mooneyack not everyone does16:17
bauzas+ I downloaded a text file 16:17
gmann++16:18
bauzasand I tagged the latest rev16:18
bauzassean-k-mooney: look at summary emails I write for a long time now16:18
bauzasyou'll see what I provide16:18
bauzasanyway16:19
bauzas#topic Review priorities 16:19
bauzaswe agreed on something last week, so I'll punt this topic for this week16:19
bauzas#topic Stable Branches 16:19
bauzaselodilles: please16:19
elodilleso716:19
elodillesstable/2023.2 branch was broken due to openstacksdk-functional-devstack job, but it is fixed -- https://review.opendev.org/c/openstack/openstacksdk/+/89915416:19
elodillesthanks gmann for pinging me about this ^^^16:20
elodillesother stable gates state should be also OK16:20
elodilles(or at least i'm not aware of any issue)16:20
bauzascool16:20
elodillesstable release patches proposed: https://review.opendev.org/q/project:openstack/releases+is:open+intopic:nova16:20
elodillesso release liaisons please review ^^^16:21
bauzasyeah I need to review them16:21
elodilleswe might not need all of them16:21
bauzasthat said, I have some bugfix I'd like to backport :)16:21
bauzasbut let's not wait for it :)16:21
bauzasor16:21
bauzasas you want16:21
gmannalso, these hyperV and vmware driver deprecation backport to stable/2023.1 are ready for 2nd review https://review.opendev.org/q/topic:deprecate-virt-drivers+status:open16:21
elodillesif you want to wait for any bugfix, then just -1 it16:21
gmannthanks bauzas for reviews. 16:21
bauzasactually I have two series I want to backport16:21
gmannmaybe it will be good to include those in releases elodilles mentioned16:21
bauzasthe compute rpc fix and the sriov gpu fix16:22
bauzaselodilles: could we then wait a little bit before releasing ?16:22
elodillesbauzas: for sure16:22
bauzaselodilles: if you don't mind, I'll -1 the release patches16:22
dansmiththe compute rpc one for sure!16:22
bauzasI need to create the backports16:23
dansmithsorry someone was at the door when we talked about gate stability,16:23
dansmithbut I've had to recheck those a few times now, all for different reasons :(16:23
bauzasnp16:23
dansmithfeels like we're slipping here16:23
bauzasI've seen multi-cell 16:23
bauzasbut I haven't yet looked at *what* failed for this job16:23
dansmitheach one I've looked at has been a different thing16:24
bauzaslovely16:24
bauzassomething due to some stuff with 'v' name that ends by 'olume' ?16:25
dansmithsome, but not all16:25
bauzasif that's something starting with 'n' and ending by 'etwork' I can understand16:25
bauzasanyway, I need to do homework, I just don't know yet when16:26
bauzasokay, I guess we're done with stable 16:27
elodillesah, maybe the general info:16:28
bauzasnothing we need to discuss for now, we just need to look16:28
elodilles#info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci16:28
bauzasah yeah16:28
elodillesand that's all :)16:28
bauzas#topic Open discussion 16:28
bauzasgmann: wants to talk the topic you had ?16:28
bauzas(gmann) RBAC: service role usage for Nova to talk to other services16:29
gmannoh, which one?16:29
gmannk16:29
bauzas#link https://etherpad.opendev.org/p/nova-caracal-ptg#L87116:29
gmannthere are some notes there on the thing I wanted to discuss. we discussed it long back on how to proceed on service role16:30
gmannwhich was 1. make policy default to service role only 2. accept service token in API 3. use nova generated admin context whereever it is required (for example getting servers in server external event API)16:30
gmannbut when I was doing it for swap volume API (which is lot o call from cinder to nova and nova to cinder), it end up that we need to use admin context for almost all the operations in this API. so it looks like we are going to make policy default to service role only and use service token to just check it is called from service not form user16:32
gmannBUT use admin context for all actual operations16:32
gmannthis is something we discussed on Monday PTG but left nova specific discussion for later16:33
gmannsean-k-mooney: has idea of passing user as well service token in service-only-APIs whihc is also listed in etherpad16:33
sean-k-mooneyright so when swap volume is called by cinder on behalf of user sean16:34
gmannsean-k-mooney: if I understand your idea correct you want to do it same way we do currently right? passing service token as header in user token - https://github.com/openstack/nova/blob/b37d50fecc92c17c3b30e41f6ad8869e1f9bd2ef/nova/service_auth.py#L3316:34
sean-k-mooneywe shoudl be recordign that as coming form sean not cinder16:35
sean-k-mooneygmann: yes it would be the presence fo a service token(with service role) in the header and  any other token that would make it a service request16:35
bauzasI'm a bit confused here16:36
bauzasprobably because I'm not an expert16:36
gmannand that service token can be used to know if request is from service or not. if not then 403. 16:36
sean-k-mooneythe user token is so we knwo on behlaf of which (user/project) the request was made16:36
dansmithauthorized by the service token but limited in scope (and recorded in audit) as the user16:36
bauzasbut I thought we were already providing both tokens ?16:36
sean-k-mooneydansmith: exactly16:36
gmannbauzas: we do but we do not check service token is must provided or not16:37
sean-k-mooneybauzas: we are but some of these api are admin only today16:37
bauzasokay, so that's a question server-side ?16:37
gmanndansmith: sean-k-mooney so if no service token presence in those servcei-ony API then 403 right?16:37
sean-k-mooneyso the user token in that case is an admin token created form the config for the cinder user16:37
sean-k-mooneygmann: yes16:37
gmanncool. 16:37
sean-k-mooney403 if service token is not passed16:37
sean-k-mooneywhen callling swap volume or external events api for example16:38
gmannyeah, we do not check that currently so it will achieve the same thing we wanted via  policy default16:38
sean-k-mooneyfor compatiblity we are currently supproting "service or admin" right16:38
gmannit is admin only no service but that is going to be "service or admin"16:39
sean-k-mooneymeaning we woudl allow it if the user token was an admin token or if there was an addtional service token16:39
gmannI mean "<whatever default currently admin or member + service role>"16:39
sean-k-mooneyso on the devstack side oen thing we shoudl do eventually is have a way to make sure the service_user does nto have admin in its role assignemtns16:40
gmannI am thinking 1. fetch the service token form header 2. use that as target in oslo policy so that if no service role then it fail16:40
sean-k-mooneyfor the service_only policy rihgt16:41
gmannyes16:41
sean-k-mooneyfor the service_or_admin we would need to fall back16:41
gmannand policy default will be "admin or service"16:41
sean-k-mooneyright so that woudl check the service token if present or user token if not in that case16:41
gmannor we can use "role:service" only in policy default also if we are going to use service token only for policy enforcement16:42
sean-k-mooneyis that somethign we can do today?16:42
gmanntricky part is if anyone want to override the policy to non-service role also then it will be little complicated to allow those16:42
dansmithyeah it kinda needs policy to check the service token, but the rest of cinder should use user token for scoping and audit logging16:42
gmannmaybe we need to pass both user and service token in oslo.policy16:43
gmanndansmith: yeah16:43
sean-k-mooneygmann: i think ^ is likely needed yes16:43
sean-k-mooneyboth tokens and likely an extention to policy.json16:43
dansmithgmann: can we have a way to say "apply this rule to service token"? then that can be the default and you could reset it to user if you prefer16:43
gmannwe need to make sure overriden policy keep working and it is configurable 16:43
sean-k-mooneyso that we can express the requireemnt for for roles on each one16:43
sean-k-mooneydansmith: ya i think wee need to be able to say somethign like "service_token_required_role=service or default_token_required_role=admin"16:44
gmanndansmith: need to check if such fallback there in oslo.policy otherwise we can combine the both token in the targets to oslo.policy16:44
gmannif we keep rule as 'role:service' only and passing both token (service and user) as targets to oslo.policy then it will work for our default (service only) as well as for any overridden rule too16:46
sean-k-mooneygmann: that would allow a user that was given the service role to make a request with a single token16:47
sean-k-mooneyim not sure we want to allow that16:47
gmannif we will use rule default as 'admin or service' then it will be comlicated to disallow only admin to pass the checks16:47
sean-k-mooneyi wanted to disallow the user_token to pass if it had the service role without also having a service token16:47
gmannsean-k-mooney: but admin-or-service also allow such user having service role16:47
sean-k-mooneyperhaps that shoudl be allowed but im not convinece it shoudl16:47
sean-k-mooneygmann: it woudl allow it because of the admin role16:48
sean-k-mooneynot because of the service role16:48
dansmithwell, I was thinking more like "this rule applies to the user token" or "this rule applies to the service token" and default to user, which is current behavior16:48
gmanndansmith: ohk. and change default behavior to check rule for service token right?16:49
dansmithso if you set the swap_volume rule to service token, then it will use the service token to decide if a request can or cannot run that operation instead of the user token (and require it be present of course)16:49
gmannso service token check first if it pass then go for user token check otherwise fail early aonly16:49
dansmithgmann: change default just for swap_volume you mean right?16:49
sean-k-mooneyswap_volume and i think external events are the two api this would apply too for now16:49
dansmithand detach16:49
gmanndansmith: I mean first we check service token rule and then user token rule16:50
dansmithgmann: no, not what I was thinking16:50
gmannhumm16:50
sean-k-mooneydansmith: well volume detach instnace action will not need this change16:50
dansmithfor swap_volume, we only care about the service_token for *authorization* so policy only needs to check that16:50
dansmithcinder still uses the user_token for querying the DB and logging, but the ability to do the operation is authorized by service token16:51
dansmithcreate_volume would still be authorized by user token only16:51
gmannand default policy will be "role:service" right?16:51
sean-k-mooney"roleLservice token:service"16:51
dansmithsean-k-mooney: well, detach is currently checking the service token explicitly right? so if we extend the syntax to allow saying "this operation is authorized by the service token" we'd want that rule to work the same way Itink16:51
dansmithokay I think we're getting confused here16:52
sean-k-mooneywhere "role:service" woudl imply "role:service token:user" or current behavior16:52
dansmithperhaps we should take this outside the nova meeting16:52
* bauzas is already confused fwiw16:52
sean-k-mooneydansmith: detach on the cinder api is16:52
bauzasyeah, I think we need some gmeet16:52
gmannyeah, maybe let's have a call16:52
bauzasI can organize it, but I won't be able to attend it I guess16:53
sean-k-mooneydansmith: ya so i think you and i are on the same page16:53
sean-k-mooneywe need a syntax in the role to say this is authorised by the service token16:53
gmannlet me scheudle one after TC meeting today (which is after 3 hr from now). sean-k-mooney is that late for you?16:53
dansmithyeah16:53
dansmithgmann: can we make it a different day//16:54
gmannsure16:54
dansmithtoday is already busy and full of meetings for me16:54
dansmithI'm about meetinged out16:54
gmanntomorrow same time as nova meeting today?16:54
bauzasI'm off this wed and this friday, but please do this when I'm not here :)16:54
dansmithgmann: +1h from this meeting I think16:54
dansmithgmann: we have a downstream meeting this time tomorrow16:54
gmannsean-k-mooney: yeah, i got point for service_token vs service role. but need to check syntext and how oslo.policy going to enforce it16:55
dansmithor 1h before this one would work16:55
* bauzas will stamp whatever needs to be stamped but I feel ignorant here16:55
gmann+1 hr is good for me. sean-k-mooney how about you ?  tomorrow 17 UTC16:55
dansmithbauzas: you mean you're delegating and we'll brief you when we get a solution :)16:55
bauzasdansmith: exactly, sold16:55
sean-k-mooneyill ateend whenever16:55
gmanncool. i will send invite16:56
sean-k-mooneyi think dansmith  more or less is on the same page as me so if there there its likely fine and ill attend if im around16:56
bauzasI just pass my PTL service token to dansmith and sean-k-mooney and tell to use my auth token for signing-off16:56
sean-k-mooneyi shoudl be free at that time for what tis worth16:56
dansmithbauzas: badum tish16:56
bauzaslove it when a plan comes together !16:58
* bauzas goes burning a cigar16:58
bauzasand ends the meeting 16:58
bauzasthanks all16:59
bauzas#endmeeting16:59
opendevmeetMeeting ended Tue Oct 31 16:59:11 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:59
opendevmeetMinutes:        https://meetings.opendev.org/meetings/nova/2023/nova.2023-10-31-16.00.html16:59
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/nova/2023/nova.2023-10-31-16.00.txt16:59
opendevmeetLog:            https://meetings.opendev.org/meetings/nova/2023/nova.2023-10-31-16.00.log.html16:59
elodillesthanks o/16:59
opendevreviewJohn Garbutt proposed openstack/nova-specs master: WIP: Add spec for PCI Groups  https://review.opendev.org/c/openstack/nova-specs/+/89971917:00
sean-k-mooney:) i should add ^ to my reading list17:02
bauzassean-k-mooney: dansmith: when you're around, fwiw, I wrote a bugfix for the sriov gpu problem https://review.opendev.org/c/openstack/nova/+/89962517:03
bauzaswe basically agreed on the solution but I needed to add another meat17:04
bauzassince we create a new mediated device when spawning, there is some race condition until the periodic update_rp() runs 17:04
bauzasso I tried to find a solution, which is to call the update_rp() method after the contextmanager run17:06
dansmithbauzas: okay I need to catch up on context17:12
bauzasas a reminder, a GPU has multiple VFs17:13
bauzaseach of the VFs can have only one mdev17:13
bauzasbut,17:13
bauzasthis depends on the vgpu type17:13
bauzassay a vgpu type only supports 2 vGPUs, then if a GPU has 16 VFs, only two of them can have a mdev17:14
bauzasfor the first mdev, this doesn't change the sysfs : we see that one VF has available_instances be 0 while the 15 else have available_instances = 117:15
dansmithI think this was discussed in the nova room after I had to leave for tc stuff right?17:15
bauzasbut once we use another VF for creating a mdev, then all of the 14 else now have available_instances = 017:15
bauzasdansmith: well, we discussed this before I think but I haven't verified whether you were around17:16
bauzashttps://etherpad.opendev.org/p/nova-caracal-ptg#L67817:16
dansmithyeah I wasn't, AFAIK17:17
bauzasso, the problem was due to https://github.com/openstack/nova/blob/9c9cd3d9b6d1d1e6f62012cd8a86fd588fb74dc2/nova/virt/libvirt/driver.py#L9110-L911117:17
dansmithI'm in another convo right now leading up to the TC meeting soon, but I'll circle back17:17
bauzasbasically, the libvirt driver correctly numbers the values and then it passes a total=017:18
bauzasso we need now to delete the RP if so17:18
bauzasdansmith: sure, no worries, I understand this needs a lot of context17:18
bauzasand probably some hardware environment :)17:19
*** elodilles is now known as elodilles_pto20:29
opendevreviewsean mooney proposed openstack/nova master: add pyproject.toml to support pip 23.1  https://review.opendev.org/c/openstack/nova/+/89975321:25
-opendevstatus- NOTICE: Gerrit on review.opendev.org will be restarted to pick up a configuration change required as part of Gerrit 3.8 upgrade preparations.22:02
JayFsean-k-mooney: gave you co-authored-by on an ironic change that looks like 899753; your toml file LGTM and the commit message was way better than what I would've written so I cribbed it all :D 22:45
JayFthank you :D22:45

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!