14:00:16 <efried> #startmeeting nova
14:00:17 <openstack> Meeting started Thu Jan  9 14:00:16 2020 UTC and is due to finish in 60 minutes.  The chair is efried. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:20 <openstack> The meeting name has been set to 'nova'
14:00:43 <efried> #link agenda https://wiki.openstack.org/wiki/Meetings/Nova#Weekly_Nova_team_meeting
14:00:44 <gibi> o/
14:00:58 * gibi is on a parallel meeting so a bit on and off
14:01:11 <alistarle> o/
14:01:13 <alex_xu> o/
14:03:10 <sean-k-mooney> o/
14:03:11 <kaisers> o/
14:03:11 <huaqiang> o/
14:03:24 <lyarwood> \o
14:03:29 <efried> Okay, let's get started.
14:03:44 <efried> Welcome back
14:03:47 <efried> #topic Last meeting
14:03:48 <efried> #link Minutes from last meeting: http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-12-12-14.00.html
14:05:30 <efried> #topic Bugs (stuck/critical)
14:05:30 <efried> No Critical bugs
14:05:45 <efried> stop me if you have things to say on a topic.
14:05:55 <efried> #topic Reminders
14:05:55 <efried> 5 weeks to spec freeze.
14:06:05 <efried> #link ussuri open specs https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+path:%255Especs/ussuri/approved/.*
14:06:15 <efried> couple dozen there.
14:06:23 <sean-k-mooney> efried: has your vtpm spec merged
14:06:56 <sean-k-mooney> you adressed stephenfins nits right
14:06:57 <efried> sean-k-mooney: not yet. stephenfin wanted that little update yesterday and said he would fast approve, but it looks like he's been pulled away
14:07:02 <efried> yes
14:07:13 <sean-k-mooney> ok cool
14:07:30 <sean-k-mooney> ill ping him after the meeting
14:07:34 <efried> thanks sean-k-mooney
14:07:46 <efried> gibi: minor update to that if you want to re+2 for form's sake.
14:07:58 <efried> #link vTPM spec https://review.opendev.org/686804
14:08:18 <efried> also
14:08:28 <efried> #link ussuri blueprints https://blueprints.launchpad.net/nova/ussuri
14:08:54 <gibi> efried: ack
14:09:00 <efried> which may not be the same list, because not every blueprint needs a spec; and it's possible some of those specs don't have blueprints (they should - that would be a miss by the author) though I haven't checked.
14:09:21 <efried> #action efried to paw through blueprints and specs and reconcile
14:09:34 <efried> unless someone else wants to volunteer for that drudgery ^
14:09:40 <efried> I know, way to sell it.
14:09:50 <sean-k-mooney> efried: do we intend to still do a desinge /direction assement of approved blueprint when we reach sepc freeze
14:10:18 <sean-k-mooney> it feel like we have less the last cycle
14:10:40 <efried> Agree, but I still think it would be prudent to prune and prioritize
14:10:57 <efried> especially since we've lost half of the core team since December.
14:11:06 <efried> by which I mean mriedem
14:11:09 <sean-k-mooney> ack just confirming we will still do that when we have hit spec freeze
14:11:49 <efried> there are 17
14:11:49 <efried> #link ussuri merged specs https://review.opendev.org/#/q/project:openstack/nova-specs+status:merged+path:%255Especs/ussuri/approved/.*
14:12:09 <efried> and IIRC we were talking about a cutoff in the neighborhood of 30
14:12:26 <brinzhang_> https://review.opendev.org/#/q/owner:%22Brin+Zhang+%253Czhangbailin%2540inspur.com%253E%22++project:openstack/nova-specs++status:open
14:12:34 <efried> so if we follow that, we would expect approx a dozen to be cut.
14:12:49 <brinzhang_> hi I have some specs, could you review this? or talking about it now?
14:13:25 <efried> Sure, now is as good a time as any :)
14:13:33 <efried> anything specific to discuss here brinzhang_, or just a general request for reviews?
14:14:26 <sean-k-mooney> it looks like each is a small independ feature
14:14:27 <brinzhang_> https://review.opendev.org/#/c/580336/ this I want to know shold I continue it, and I was pushed it's poc code already
14:14:40 <sean-k-mooney> am i correct in saying there are no depencies between these sepcs
14:14:51 <brinzhang_> https://review.opendev.org/#/c/693828/
14:15:05 <alex_xu> the concern for 580336 is swap_volume is admin API
14:15:25 * lyarwood reads
14:15:30 <brinzhang_> sean-k-mooney: yeah, it's samll, but I think need to continue
14:15:42 <alex_xu> one way is adding a separate policy rule for the delete_on_terminate update
14:16:48 <sean-k-mooney> alex_xu: could you not that in the spec
14:16:57 <sean-k-mooney> *note
14:17:03 <brinzhang_> yeah, this is a way to avoid the policy role
14:17:07 <alex_xu> sean-k-mooney: I note that in previous PS
14:17:27 <sean-k-mooney> ah ok
14:18:23 <efried> Anything else on specs/blueprints for now?
14:18:46 <efried> huaqiang: this would probably be a better time to bring up yours
14:19:00 <huaqiang> thanks efried
14:19:01 <efried> [Huaqiang] Spec: Use PCPU and VCPU in one instance: https://review.opendev.org/#/c/668656/. We have agreements in PTG meeting on the following aspects but during recent review process some different opinions raised:
14:19:01 <efried> Specifying instance dedicated CPU mask vs specifying CPU count. Sean-k-mooney raised a flavor example for using CPU mask proposal in this place: https://review.opendev.org/#/c/668656/14/specs/ussuri/approved/use-pcpu-vcpu-in-one-instance.rst Line#146
14:19:01 <efried> Keep the interface of creating 'mixed' instance through 'resources:PCPU' and 'resources:VCPU' vs removing this interface. In L253, sean-k-mooney thinks that keeping the interface will cause issues with NUMA mechanism.
14:19:34 <huaqiang> as stated, we have agreements in the PDT meeting
14:19:43 <efried> it would probably be best to have stephenfin in the room if we're going to make progress on this
14:19:59 <huaqiang> but I think sean-k-moony's comments are prtty reasonable
14:20:08 <huaqiang> agree
14:20:25 <sean-k-mooney> my concuer with the resouce: synatx is that it couples the flavor to the current modelingin placment
14:20:47 <sean-k-mooney> so fi that evovles we dont have a laryer of indrection to hide that form our users so it fragile
14:21:16 <efried> I haven't been paying attention to this
14:21:17 <efried> but
14:21:33 <efried> if the question is whether we should be using placement-ese vs. flavor-ese syntax,
14:21:37 <alex_xu> yea, that we don't have consistent agreement at here. we support pcpu with resource syntax, but we don't support vpmem with reosurce syntax
14:21:38 <efried> I want the latter
14:22:06 <efried> I recognize we enabled support for the former for a few things, but I think we should move away from that in general.
14:22:25 <sean-k-mooney> i have a stong prefernce of the flavor-ese approch too.
14:22:39 <efried> Two main reasons
14:22:40 <alex_xu> I'm ok with flavor-ese also
14:23:49 <huaqiang> Another for discussion, is specify a CPU mask for dedicated CPUS
14:23:52 <efried> 1) Placement-ese syntax is hard. It's powerful for the API, but tough for humans to generate sensibly. Specifically in the case of CPU type stuff, they're already familiar with the flavor-ese syntax
14:24:22 <efried> 2) We will sometimes/often want non-placement side effects to be triggered by these options.
14:24:42 <efried> In some cases we would be able to guess what those should be, but in some cases we'll need separate knobs
14:24:53 <efried> and in those cases we would end up with a mix of placement- and flavor-type syntax, which is icky.
14:24:54 <alex_xu> yea, special after have numa in placement
14:25:02 <efried> for sure ^
14:25:13 <sean-k-mooney> huaqiang: so to the second point on mask vs count
14:25:50 <sean-k-mooney> i prefer a mask of list of pinned cpus so that with in the guest i have a way of know which cpus are pinned so i can confure my gues accordingly
14:25:51 <huaqiang> yes. need input for these two
14:26:14 <efried> sean-k-mooney: are we talking about enumerating the *host* CPU IDs or the *guest* CPU IDs?
14:26:16 <sean-k-mooney> if we have a count we need to either discover the info dymicaly via medatadata api or hav a convention
14:26:24 <sean-k-mooney> guest
14:27:01 <alex_xu> efried: guest CPU IDs
14:27:04 <sean-k-mooney> so i was sugestin a mask so you could say core 0 floats and the rest re pined so you could in the guest use core 0 for the os and 1-n for the application
14:27:05 <alex_xu> or just bitmask
14:27:06 <efried> I remember talking about this. We need some way for the guest to discover which CPUs are dedicated. I'm not sure anything in the flavor helps us achieve that.
14:27:27 <dansmith> efried: by placement-ese you mean things like resources:THING=1 ?
14:27:34 <efried> dansmith: yes
14:28:03 <dansmith> and flavor-ese being what?
14:28:15 <efried> hw:numa_widgets=4
14:28:15 <sean-k-mooney> hw:whatever=
14:28:50 <dansmith> sorry I'm just jumping in here, but what's the exact example here, I was thinking pinning of some sort?
14:29:20 <sean-k-mooney> yes so we are discussing mix cpus so a vm with some shared/floating cpus and some pinned
14:29:23 <efried> In this specific case we're talking about mixing PCPUs and VCPUs in one instance
14:29:37 <efried> and how the flavor should specify those.
14:29:40 <dansmith> vcpu and pcpu together, but what is the knob we'd be turning in that case
14:30:21 <efried> well, we're discussing whether the syntax should allow you to specify CPU IDs or just a count. If the former, obv placement-ese doesn't work.
14:30:34 <dansmith> right, so that was going to be my point:
14:30:36 <efried> but even if the latter, I don't like the idea of using placement-ese for reasons
14:30:57 * bauzas waves after the meeting he had
14:31:17 <dansmith> for cases where we're specifying traits and simple resource counts, I see the placement-ese as a major improvement over how things have been in the past, simply because it's simple and standard
14:32:20 <dansmith> are you saying you want to reverse course on resource and trait specifications in that syntax entirely?
14:33:13 <sean-k-mooney> righti  if we dind resouces=VCPU:2,PCPU6 that is speficfly an umbered group which means if we model pcpus per num node and have a two numa node guest you would have to change the syntax to use the number form
14:33:19 <efried> I'm not saying we should remove support for placement-ese syntax. I'm saying that for new work, I think we should get in the habit of not using it when it's more than just a scheduling filter.
14:33:41 <dansmith> efried: why is that? meaning, what are the "reasons" ?
14:33:52 <efried> iow I don't like it when placement-ese syntax results in side effects, e.g. guest config
14:34:13 <sean-k-mooney> where as if we did hw:pinned_cpus=2-4,5-7 hw:numa_node=2
14:34:27 <efried> like trait:CAN_DO_THING=required resulting in <thing>on</thing>
14:34:36 <sean-k-mooney> then we can keep the flavor the same and change the placmenet query depending on if we have numa in placment or not
14:35:12 <efried> dansmith: see 1) and 2) at time stamp :23:52
14:35:45 <dansmith> efried: okay, well, that's something I can follow I guess. I'm not sure why it's bad though.. making the users and ops know the distinction of "well, I asked for some resource, why don't I have it?" or "why do I have to specify things in this format if it's devicey and not if not?"
14:35:47 <efried> I've gone into this reasoning more eloquently/completely in various reviews as well
14:36:00 <efried> dansmith: precisely.
14:36:20 <dansmith> what, you think making them realize the difference is a good thing?
14:36:53 <efried> Supporting pass-through placement-ese which is sometimes side-effecty and sometimes not is bad. So I'm saying we shouldn't be promoting/documenting that pass-through-ness.
14:37:04 <efried> If you want to do a specific thing, follow the documentation for the syntax for that thing
14:37:29 <efried> and if we decide a *specific* placement-ese trait or resource k=v is appropriate for that, it'll be in the docs
14:37:45 <efried> though in general I would prefer to avoid even that
14:38:03 <sean-k-mooney> efried: you dont want to oterwise identical vms that land on the same host to have different side ieefec if one requested a trait and the other did not right
14:38:22 <efried> I guess that's one way to think of it.
14:38:41 <sean-k-mooney> /side ieefect/sideffects/
14:39:19 <efried> I want trait:FOO=required to make you land on a host with FOO. I want hw:make_foo_happen to a) add required=FOO to the placement request *and* b) turn <foo>on</foo> in the guest XML.
14:39:30 <dansmith> okay, I guess I have a major reaction to what seems to be reverting to the "old" non-standard, confusing has-changed-and-broken-people way of doing that, where individual patch authors make up their own syntax for each feature, and placement syntax being strict and simple was a blessing to me
14:40:49 <alex_xu> will placement syntax be simple after we have numa?
14:41:00 <efried> strict and simple, but not always powerful enough.
14:41:05 <sean-k-mooney> ok so in the interest of time lets loop back to the mask vs cound question
14:41:06 <dansmith> no, and I get that it's not powerful enough as it is
14:41:10 <sean-k-mooney> *count
14:41:26 <efried> So whether we decide to use
14:41:26 <efried> - $pinned_count, a convention saying e.g. the first $pinned_count guest CPU IDs are pinned and the remaining ($total - $pinned_count) are shared; or
14:41:26 <efried> - a $pinned_mask
14:41:26 <efried> ...there needs to be some way for the guest to find out either $pinned_count or $pinned_mask. What did we decide that mechanism would be? Config drive?
14:41:39 <dansmith> yep, sorry, I just literally flipped on the monitor when I woke up and saw the placement-ese thing, didn't mean to derail
14:42:13 <sean-k-mooney> shoudl we allow the flaovor to say wich cores are pinned and which float (with a mask or list) or how many cores are pinned(possible per numa node) and have the driver decide which ones are
14:42:20 <efried> dansmith: it is relevant, specifically one of the topics on the agenda. So we need to continue the discussion. But maybe we can take it back to -nova after the meeting.
14:43:59 <sean-k-mooney> i prefer hw:pinned_cpus=2-4 where core 0 is floating but the alterinve would be hw:pinned_cores_count=3
14:44:26 <sean-k-mooney> the extrapec propsed in the spec are different but the name dont matter right now
14:44:34 <brinzhang_> sean-k-mooney: I think it's a good idea, we should konw which vcpu pinned which numa node
14:44:57 <efried> yes
14:45:09 <efried> maybe not immediately
14:45:14 <sean-k-mooney> dansmith: efried effectivly im suggestion we just use the syntax form the vcpu_pin_set
14:45:31 <efried> I'm in favor of that
14:45:32 <sean-k-mooney> but the cores you list would be the logical guest cores not the host cores
14:46:16 <efried> might be nice if the name indicated that somehow, e.g. by including '_guest_'. But we can bikeshed that.
14:46:19 <efried> But my question remains:
14:46:28 <efried> how does the guest *find out*?
14:46:47 <alex_xu> there is propose for metadata API
14:46:48 <sean-k-mooney> in the spec i belive the metadata api was going to be extended to include it
14:47:01 <sean-k-mooney> also i assuem the toplogy api we just added could also be extended
14:47:12 <efried> there wasn't some kind of magic where it shows up automatically in sysfs?
14:47:16 <efried> or procfs
14:47:20 <sean-k-mooney> no
14:47:22 <alex_xu> no
14:47:23 <efried> okay.
14:47:27 <dansmith> wait, what?
14:47:43 <dansmith> the guest will know about its topology in the usual fashion, right? isn't that what efried is asking?
14:47:46 <efried> dansmith: the guest needs to be able to figure out which of its CPUs are pinned and which shared.
14:47:54 <dansmith> oh, I see
14:48:01 <dansmith> yeah, that has to be metadata
14:48:03 <sean-k-mooney> dansmith: yes but there is no way to tell the kernel what is pinned or not
14:48:05 <dansmith> right
14:48:13 <efried> okay cool.
14:48:17 <efried> did we land?
14:49:11 <huaqiang> so, do we have conclusion? count or mask?
14:49:11 <sean-k-mooney> i think we laned on use extra spec/flavor syntack and use a list like the vpcu_pin_set brinzhang_ ? that work for you alex_xu ?
14:49:39 <alex_xu> works for me
14:49:49 <brinzhang_> me too
14:50:17 <huaqiang> thanks
14:50:40 <efried> I guess the question remains whether we should (or can) block the placement-ese syntax
14:51:01 <dansmith> so, based on what you said above (I think):
14:51:04 <sean-k-mooney> well we can but im not sure if we should
14:51:05 <efried> IIRC stephenfin's PCPU work in train made a big deal of supporting that specifically
14:51:12 <efried> so this would be a step backwards
14:51:13 <dansmith> you're okay with hw: implying resources, but not resources implying hw: stuff?
14:51:18 <efried> I must not have complained loudly enough.
14:51:25 <efried> dansmith: yes, exactly that.
14:52:07 <dansmith> what would be the blocking then? if they specify any pcpu in resources they're toast?
14:52:08 <efried> because resources syntax isn't always powerful enough to express hw: stuff so we would need to mix in hw: syntax anyway in those cases, which IMO is more confusing
14:52:56 <efried> Yeah, that's what I'm trying to noodle. Because I think stephenfin's work in train may have made it so resources:PCPU does in fact imply some (simple) hw:-y stuff.
14:53:25 <sean-k-mooney> it did which i did not like either
14:53:25 <dansmith> right, just asking for one PCU:1 gets you one pinned cpu with minimal opinion on how right?
14:54:07 <efried> So I think we're okay actually if we just leave support for that (the train syntax) but explode if you try to *mix* with placement-ese.
14:54:17 <efried> IOW if you specify both resources:PCPU and resources:VCPU, boom
14:54:24 <sean-k-mooney> ya it will but  PCPU:2 hw:numa_nodes=2 would break if we had numa in placement
14:54:49 <efried> that too
14:54:51 <dansmith> efried: I don't think that works, because vcpu is always specified by the flavor's own property, and we've said they can override that with VCPUs and PCPUs right?
14:55:24 <efried> dansmith: I don't have the details swapped in, but stephenfin wrote some intricate and very specific checks for those permutations.
14:55:29 <sean-k-mooney> well we translate flavor.vcpu into either PCPUs or VCPUs
14:55:43 <sean-k-mooney> and you can override them but yes
14:55:54 <sean-k-mooney> stephenfin: also added a lot of checks for edgecases
14:55:56 <efried> ...but I'm pretty sure they included checks so that if you wound up with both PCPUs and VCPUs it would fail.
14:56:25 <efried> Oh, right, because there's a hard restriction on flavor.vcpus=0
14:56:26 <sean-k-mooney> yes i think that is blocked currently
14:56:45 <efried> or am I confusing that with something else?
14:57:14 <dansmith> yeah, I'm talking about the more general rule we've had (specifically for ironic) where overriding the resources specified in the flavor itself are done with the resources syntax,
14:57:34 <dansmith> so it would be weird in this case to have that not be a thing because we're trying to ban specifying both or something
14:57:38 <sean-k-mooney> dansmith: yes in that case you set resouces=0 rather then the flavor values
14:58:20 <efried> ...so rather than lifting that restriction, we decided you had to specify vcpus=$count and then also specify resources:PCPU=$count (the same $count) if you wanted them to be pinned.
14:58:20 <efried> So we would either need to lift that restriction, or make some new rule about e.g. resources:VCPU=$s,resources:PCPU=$p, flavor.vcpus=$(s+p) (ugh)
14:58:22 <sean-k-mooney> i think we can proceed with the feature without specificly blocking the overdies and just document dont do this
14:59:10 <dansmith> efried: the easy thing to allow is the case where you don't specify VCPUs or you do and the math is right, right?
14:59:11 <efried> dansmith: we may be overthinking it. Today I'm 95% sure if you specify both resources:PCPU and resources:VCPU you'll explode. Keep that.
14:59:22 <efried> yeah
14:59:29 <efried> oh
14:59:32 <efried> um
14:59:54 <efried> you mean "specify VCPUs" in placement-ese?
14:59:55 <efried> No
15:00:02 <dansmith> well, whatever, it just seems like we're doing the thing I'm concerned about, which is that every feature has its own set of complex rules for this, and things go up in smoke if you ever need to combine two complex features :/
15:00:11 <dansmith> yes, that's what I meant
15:00:14 <efried> Time.
15:00:14 <efried> Let's continue in -nova.
15:00:14 <efried> #endmeeting