16:00:26 <bauzas> #startmeeting nova
16:00:27 <opendevmeet> Meeting started Tue Oct 31 16:00:26 2023 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:27 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:27 <opendevmeet> The meeting name has been set to 'nova'
16:00:31 <bauzas> hey folks
16:00:34 <bauzas> #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
16:00:36 <bauzas> who's around ?
16:00:40 <bauzas> do we have quorum ?
16:00:55 <bauzas> nothing urgent to say this week given we had the PTG last week
16:01:02 <elodilles> o/
16:01:10 <Uggla> o/
16:01:46 <elodilles> everyone is travelling back from PTG i guess :D
16:02:10 <bauzas> yeah
16:02:12 <sean-k-mooney> o/
16:02:21 <bauzas> some rh folks also have some meetings
16:02:35 <sean-k-mooney> those have finished already fyi
16:02:50 <sean-k-mooney> well ignorign the one that overruning
16:02:53 <bauzas> okay we can wait a bit
16:03:03 <sean-k-mooney> we proably can get started
16:03:14 <bauzas> well, okay
16:03:32 * bauzas was working on a customer bug meanwhile :)
16:03:38 <bauzas> #topic Bugs (stuck/critical)
16:03:42 <bauzas> #info No Critical bug
16:03:46 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 36 new untriaged bugs (-11 since the last meeting)
16:03:58 <bauzas> kudos to melwitt and gibi for triaging !
16:04:04 <bauzas> #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster
16:04:19 <bauzas> Uggla: would you be happy with taking the baton this week ?
16:04:35 <Uggla> bauzas, yep it's ok.
16:05:16 <bauzas> cool thanks
16:05:21 <bauzas> #info bug baton is Uggla
16:05:25 <bauzas> #topic Gate status
16:05:29 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:05:31 <gmann> o/
16:05:32 <bauzas> #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status
16:05:42 <bauzas> all greens
16:05:52 <elodilles> \o/
16:05:53 <bauzas> #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:06:06 <bauzas> I think I've seen some gate failures on my rpc series
16:06:29 <bauzas> but I need to verify which jobs were having problems
16:06:41 <bauzas> I hadn't time yet to recheck
16:07:50 <bauzas> moving on ?
16:08:16 <bauzas> #topic Release Planning
16:08:53 <bauzas> #link https://releases.openstack.org/caracal/schedule.html
16:08:57 <bauzas> #info Caracal-1 milestone in 2 weeks
16:09:23 <bauzas> we agreed last week on the fact today should be a spec review day but unfortunately I hadn't provided an email
16:09:35 <bauzas> so if you want, next week will be the first spec review day
16:09:37 <bauzas> fine ?
16:09:49 <bauzas> next tuesday*
16:09:53 <sean-k-mooney> the spec review day was not ment to be today
16:10:35 <bauzas> well, I will be on bank holiday tomorrow (+ Uggla + kashyap IIRC) and on Friday I'll be on PTO
16:10:51 <kashyap> *Yep)
16:10:59 <bauzas> https://etherpad.opendev.org/p/nova-caracal-ptg#L175 said R-22 would be our spec review day
16:12:01 <sean-k-mooney> bauzas: i gues syou did say r-22
16:12:07 <sean-k-mooney> i feel like that was a mistak
16:12:37 <sean-k-mooney> i tought i suggested m1 for that bvut no worries
16:12:38 <bauzas> no, I remember we said 'the week after PTG'
16:12:41 <sean-k-mooney> we can do it next week
16:13:02 <bauzas> m-1 was for reviewing implementations
16:13:03 <sean-k-mooney> oh we are doing the implemation review day on m1
16:13:12 <sean-k-mooney> yep
16:13:25 <sean-k-mooney> so ya we missed the first sepc review day
16:13:34 <bauzas> yeah, my fault
16:13:37 <sean-k-mooney> honestly im still burnt out form ptg so next week woudl be better
16:13:39 <bauzas> -ETOOMANYTHINGS
16:13:49 <bauzas> okay, agreed
16:14:15 <bauzas> #info Next Tuesday (Nov 7) we will have a spec review day
16:14:24 <bauzas> #action bauzas to community on that review day
16:14:28 <bauzas> #unfo
16:14:31 <bauzas> #undo
16:14:31 <opendevmeet> Removing item from minutes: #action bauzas to community on that review day
16:14:39 <bauzas> #action bauzas to communicate on that review day
16:15:13 <bauzas> I'll also communicate the implementation review day in 2 weeks from now
16:15:35 <bauzas> moving on
16:15:38 <bauzas> #topic Caracal vPTG planning
16:15:42 <bauzas> #undo
16:15:42 <opendevmeet> Removing item from minutes: #topic Caracal vPTG planning
16:15:52 <bauzas> #topic Caracal vPTG wrapup
16:15:59 <bauzas> #link https://etherpad.opendev.org/p/nova-caracal-ptg PTG etherpad
16:16:18 <bauzas> I have some draft in my emailbox that I need to eventually write
16:16:31 <bauzas> I just don't know when yet
16:16:42 <bauzas> for the moment, please look at the etherpad
16:16:53 <elodilles> ++
16:17:00 <bauzas> we had one single topic we were unable to talk
16:17:01 <sean-k-mooney> bauzas: wehn you send the eamil
16:17:11 <sean-k-mooney> please use the readonly link for the etherpad
16:17:27 <bauzas> like every cycle I do ? sure
16:17:46 <bauzas> I already did
16:17:46 <sean-k-mooney> ack not everyone does
16:17:53 <bauzas> + I downloaded a text file
16:18:01 <gmann> ++
16:18:15 <bauzas> and I tagged the latest rev
16:18:39 <bauzas> sean-k-mooney: look at summary emails I write for a long time now
16:18:58 <bauzas> you'll see what I provide
16:19:02 <bauzas> anyway
16:19:06 <bauzas> #topic Review priorities
16:19:27 <bauzas> we agreed on something last week, so I'll punt this topic for this week
16:19:40 <bauzas> #topic Stable Branches
16:19:43 <bauzas> elodilles: please
16:19:48 <elodilles> o7
16:19:55 <elodilles> stable/2023.2 branch was broken due to openstacksdk-functional-devstack job, but it is fixed -- https://review.opendev.org/c/openstack/openstacksdk/+/899154
16:20:08 <elodilles> thanks gmann for pinging me about this ^^^
16:20:18 <elodilles> other stable gates state should be also OK
16:20:44 <elodilles> (or at least i'm not aware of any issue)
16:20:45 <bauzas> cool
16:20:49 <elodilles> stable release patches proposed: https://review.opendev.org/q/project:openstack/releases+is:open+intopic:nova
16:21:06 <elodilles> so release liaisons please review ^^^
16:21:10 <bauzas> yeah I need to review them
16:21:12 <elodilles> we might not need all of them
16:21:20 <bauzas> that said, I have some bugfix I'd like to backport :)
16:21:26 <bauzas> but let's not wait for it :)
16:21:30 <bauzas> or
16:21:33 <bauzas> as you want
16:21:34 <gmann> also, these hyperV and vmware driver deprecation backport to stable/2023.1 are ready for 2nd review https://review.opendev.org/q/topic:deprecate-virt-drivers+status:open
16:21:37 <elodilles> if you want to wait for any bugfix, then just -1 it
16:21:43 <gmann> thanks bauzas for reviews.
16:21:47 <bauzas> actually I have two series I want to backport
16:21:53 <gmann> maybe it will be good to include those in releases elodilles mentioned
16:22:03 <bauzas> the compute rpc fix and the sriov gpu fix
16:22:16 <bauzas> elodilles: could we then wait a little bit before releasing ?
16:22:26 <elodilles> bauzas: for sure
16:22:52 <bauzas> elodilles: if you don't mind, I'll -1 the release patches
16:22:53 <dansmith> the compute rpc one for sure!
16:23:10 <bauzas> I need to create the backports
16:23:11 <dansmith> sorry someone was at the door when we talked about gate stability,
16:23:21 <dansmith> but I've had to recheck those a few times now, all for different reasons :(
16:23:22 <bauzas> np
16:23:28 <dansmith> feels like we're slipping here
16:23:40 <bauzas> I've seen multi-cell
16:23:56 <bauzas> but I haven't yet looked at *what* failed for this job
16:24:08 <dansmith> each one I've looked at has been a different thing
16:24:17 <bauzas> lovely
16:25:08 <bauzas> something due to some stuff with 'v' name that ends by 'olume' ?
16:25:22 <dansmith> some, but not all
16:25:46 <bauzas> if that's something starting with 'n' and ending by 'etwork' I can understand
16:26:08 <bauzas> anyway, I need to do homework, I just don't know yet when
16:27:59 <bauzas> okay, I guess we're done with stable
16:28:13 <elodilles> ah, maybe the general info:
16:28:15 <bauzas> nothing we need to discuss for now, we just need to look
16:28:20 <elodilles> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci
16:28:28 <bauzas> ah yeah
16:28:35 <elodilles> and that's all :)
16:28:36 <bauzas> #topic Open discussion
16:28:45 <bauzas> gmann: wants to talk the topic you had ?
16:29:04 <bauzas> (gmann) RBAC: service role usage for Nova to talk to other services
16:29:05 <gmann> oh, which one?
16:29:12 <gmann> k
16:29:21 <bauzas> #link https://etherpad.opendev.org/p/nova-caracal-ptg#L871
16:30:08 <gmann> there are some notes there on the thing I wanted to discuss. we discussed it long back on how to proceed on service role
16:30:59 <gmann> which was 1. make policy default to service role only 2. accept service token in API 3. use nova generated admin context whereever it is required (for example getting servers in server external event API)
16:32:21 <gmann> but when I was doing it for swap volume API (which is lot o call from cinder to nova and nova to cinder), it end up that we need to use admin context for almost all the operations in this API. so it looks like we are going to make policy default to service role only and use service token to just check it is called from service not form user
16:32:40 <gmann> BUT use admin context for all actual operations
16:33:25 <gmann> this is something we discussed on Monday PTG but left nova specific discussion for later
16:33:47 <gmann> sean-k-mooney: has idea of passing user as well service token in service-only-APIs whihc is also listed in etherpad
16:34:50 <sean-k-mooney> right so when swap volume is called by cinder on behalf of user sean
16:34:58 <gmann> sean-k-mooney: if I understand your idea correct you want to do it same way we do currently right? passing service token as header in user token - https://github.com/openstack/nova/blob/b37d50fecc92c17c3b30e41f6ad8869e1f9bd2ef/nova/service_auth.py#L33
16:35:02 <sean-k-mooney> we shoudl be recordign that as coming form sean not cinder
16:35:45 <sean-k-mooney> gmann: yes it would be the presence fo a service token(with service role) in the header and  any other token that would make it a service request
16:36:20 <bauzas> I'm a bit confused here
16:36:29 <bauzas> probably because I'm not an expert
16:36:33 <gmann> and that service token can be used to know if request is from service or not. if not then 403.
16:36:34 <sean-k-mooney> the user token is so we knwo on behlaf of which (user/project) the request was made
16:36:38 <dansmith> authorized by the service token but limited in scope (and recorded in audit) as the user
16:36:43 <bauzas> but I thought we were already providing both tokens ?
16:36:49 <sean-k-mooney> dansmith: exactly
16:37:13 <gmann> bauzas: we do but we do not check service token is must provided or not
16:37:23 <sean-k-mooney> bauzas: we are but some of these api are admin only today
16:37:26 <bauzas> okay, so that's a question server-side ?
16:37:27 <gmann> dansmith: sean-k-mooney so if no service token presence in those servcei-ony API then 403 right?
16:37:37 <sean-k-mooney> so the user token in that case is an admin token created form the config for the cinder user
16:37:46 <sean-k-mooney> gmann: yes
16:37:55 <gmann> cool.
16:37:58 <sean-k-mooney> 403 if service token is not passed
16:38:13 <sean-k-mooney> when callling swap volume or external events api for example
16:38:33 <gmann> yeah, we do not check that currently so it will achieve the same thing we wanted via  policy default
16:38:36 <sean-k-mooney> for compatiblity we are currently supproting "service or admin" right
16:39:00 <gmann> it is admin only no service but that is going to be "service or admin"
16:39:01 <sean-k-mooney> meaning we woudl allow it if the user token was an admin token or if there was an addtional service token
16:39:16 <gmann> I mean "<whatever default currently admin or member + service role>"
16:40:14 <sean-k-mooney> so on the devstack side oen thing we shoudl do eventually is have a way to make sure the service_user does nto have admin in its role assignemtns
16:40:43 <gmann> I am thinking 1. fetch the service token form header 2. use that as target in oslo policy so that if no service role then it fail
16:41:10 <sean-k-mooney> for the service_only policy rihgt
16:41:15 <gmann> yes
16:41:21 <sean-k-mooney> for the service_or_admin we would need to fall back
16:41:26 <gmann> and policy default will be "admin or service"
16:41:59 <sean-k-mooney> right so that woudl check the service token if present or user token if not in that case
16:42:09 <gmann> or we can use "role:service" only in policy default also if we are going to use service token only for policy enforcement
16:42:11 <sean-k-mooney> is that somethign we can do today?
16:42:49 <gmann> tricky part is if anyone want to override the policy to non-service role also then it will be little complicated to allow those
16:42:54 <dansmith> yeah it kinda needs policy to check the service token, but the rest of cinder should use user token for scoping and audit logging
16:43:05 <gmann> maybe we need to pass both user and service token in oslo.policy
16:43:13 <gmann> dansmith: yeah
16:43:17 <sean-k-mooney> gmann: i think ^ is likely needed yes
16:43:29 <sean-k-mooney> both tokens and likely an extention to policy.json
16:43:37 <dansmith> gmann: can we have a way to say "apply this rule to service token"? then that can be the default and you could reset it to user if you prefer
16:43:37 <gmann> we need to make sure overriden policy keep working and it is configurable
16:43:41 <sean-k-mooney> so that we can express the requireemnt for for roles on each one
16:44:45 <sean-k-mooney> dansmith: ya i think wee need to be able to say somethign like "service_token_required_role=service or default_token_required_role=admin"
16:44:46 <gmann> dansmith: need to check if such fallback there in oslo.policy otherwise we can combine the both token in the targets to oslo.policy
16:46:29 <gmann> if we keep rule as 'role:service' only and passing both token (service and user) as targets to oslo.policy then it will work for our default (service only) as well as for any overridden rule too
16:47:04 <sean-k-mooney> gmann: that would allow a user that was given the service role to make a request with a single token
16:47:10 <sean-k-mooney> im not sure we want to allow that
16:47:12 <gmann> if we will use rule default as 'admin or service' then it will be comlicated to disallow only admin to pass the checks
16:47:40 <sean-k-mooney> i wanted to disallow the user_token to pass if it had the service role without also having a service token
16:47:47 <gmann> sean-k-mooney: but admin-or-service also allow such user having service role
16:47:50 <sean-k-mooney> perhaps that shoudl be allowed but im not convinece it shoudl
16:48:06 <sean-k-mooney> gmann: it woudl allow it because of the admin role
16:48:11 <sean-k-mooney> not because of the service role
16:48:13 <dansmith> well, I was thinking more like "this rule applies to the user token" or "this rule applies to the service token" and default to user, which is current behavior
16:49:04 <gmann> dansmith: ohk. and change default behavior to check rule for service token right?
16:49:11 <dansmith> so if you set the swap_volume rule to service token, then it will use the service token to decide if a request can or cannot run that operation instead of the user token (and require it be present of course)
16:49:23 <gmann> so service token check first if it pass then go for user token check otherwise fail early aonly
16:49:30 <dansmith> gmann: change default just for swap_volume you mean right?
16:49:52 <sean-k-mooney> swap_volume and i think external events are the two api this would apply too for now
16:49:59 <dansmith> and detach
16:50:03 <gmann> dansmith: I mean first we check service token rule and then user token rule
16:50:18 <dansmith> gmann: no, not what I was thinking
16:50:26 <gmann> humm
16:50:33 <sean-k-mooney> dansmith: well volume detach instnace action will not need this change
16:50:36 <dansmith> for swap_volume, we only care about the service_token for *authorization* so policy only needs to check that
16:51:03 <dansmith> cinder still uses the user_token for querying the DB and logging, but the ability to do the operation is authorized by service token
16:51:16 <dansmith> create_volume would still be authorized by user token only
16:51:23 <gmann> and default policy will be "role:service" right?
16:51:43 <sean-k-mooney> "roleLservice token:service"
16:51:54 <dansmith> sean-k-mooney: well, detach is currently checking the service token explicitly right? so if we extend the syntax to allow saying "this operation is authorized by the service token" we'd want that rule to work the same way Itink
16:52:12 <dansmith> okay I think we're getting confused here
16:52:17 <sean-k-mooney> where "role:service" woudl imply "role:service token:user" or current behavior
16:52:19 <dansmith> perhaps we should take this outside the nova meeting
16:52:22 * bauzas is already confused fwiw
16:52:27 <sean-k-mooney> dansmith: detach on the cinder api is
16:52:34 <bauzas> yeah, I think we need some gmeet
16:52:46 <gmann> yeah, maybe let's have a call
16:53:09 <bauzas> I can organize it, but I won't be able to attend it I guess
16:53:14 <sean-k-mooney> dansmith: ya so i think you and i are on the same page
16:53:46 <sean-k-mooney> we need a syntax in the role to say this is authorised by the service token
16:53:52 <gmann> let me scheudle one after TC meeting today (which is after 3 hr from now). sean-k-mooney is that late for you?
16:53:52 <dansmith> yeah
16:54:03 <dansmith> gmann: can we make it a different day//
16:54:09 <gmann> sure
16:54:09 <dansmith> today is already busy and full of meetings for me
16:54:12 <dansmith> I'm about meetinged out
16:54:34 <gmann> tomorrow same time as nova meeting today?
16:54:38 <bauzas> I'm off this wed and this friday, but please do this when I'm not here :)
16:54:43 <dansmith> gmann: +1h from this meeting I think
16:54:58 <dansmith> gmann: we have a downstream meeting this time tomorrow
16:55:00 <gmann> sean-k-mooney: yeah, i got point for service_token vs service role. but need to check syntext and how oslo.policy going to enforce it
16:55:01 <dansmith> or 1h before this one would work
16:55:08 * bauzas will stamp whatever needs to be stamped but I feel ignorant here
16:55:40 <gmann> +1 hr is good for me. sean-k-mooney how about you ?  tomorrow 17 UTC
16:55:45 <dansmith> bauzas: you mean you're delegating and we'll brief you when we get a solution :)
16:55:52 <bauzas> dansmith: exactly, sold
16:55:59 <sean-k-mooney> ill ateend whenever
16:56:11 <gmann> cool. i will send invite
16:56:29 <sean-k-mooney> i think dansmith  more or less is on the same page as me so if there there its likely fine and ill attend if im around
16:56:40 <bauzas> I just pass my PTL service token to dansmith and sean-k-mooney and tell to use my auth token for signing-off
16:56:52 <sean-k-mooney> i shoudl be free at that time for what tis worth
16:56:58 <dansmith> bauzas: badum tish
16:58:33 <bauzas> love it when a plan comes together !
16:58:46 * bauzas goes burning a cigar
16:58:50 <bauzas> and ends the meeting
16:59:07 <bauzas> thanks all
16:59:11 <bauzas> #endmeeting