Wednesday, 2019-06-12

*** zzzeek has quit IRC00:47
*** zzzeek has joined #openstack-placement00:47
*** wwriverrat has quit IRC00:55
*** Sundar has quit IRC01:39
*** yikun has joined #openstack-placement01:56
*** zzzeek has quit IRC01:57
*** zzzeek has joined #openstack-placement01:58
*** zzzeek has quit IRC02:13
*** minmin has quit IRC02:16
*** zzzeek has joined #openstack-placement02:16
*** Sundar has joined #openstack-placement02:45
*** takashin has joined #openstack-placement03:32
*** yikun has quit IRC03:51
*** yikun has joined #openstack-placement04:03
*** e0ne has joined #openstack-placement05:51
*** e0ne has quit IRC06:22
*** Sundar has quit IRC06:40
*** gibi has joined #openstack-placement06:50
*** helenfm has joined #openstack-placement07:29
*** tetsuro has joined #openstack-placement07:30
*** ttsiouts has joined #openstack-placement07:57
*** e0ne has joined #openstack-placement07:57
*** takashin has left #openstack-placement08:01
*** guilhermesp has quit IRC08:01
*** guilhermesp has joined #openstack-placement08:03
*** tetsuro has quit IRC08:13
*** ttsiouts has quit IRC08:16
*** ttsiouts has joined #openstack-placement08:17
*** tetsuro has joined #openstack-placement08:19
*** ttsiouts has quit IRC08:21
*** ttsiouts has joined #openstack-placement08:24
*** tetsuro has quit IRC08:29
*** tetsuro has joined #openstack-placement08:30
*** tssurya has joined #openstack-placement08:35
*** tetsuro has quit IRC08:36
*** cdent has joined #openstack-placement08:58
cdentgibi: if you're back and have the time there's quite a bit of interesting nested-related discussion/work going on that you'll probably want to catch up on when you get a chance. Also if you could kick this through if you have a chance that would be helpful: https://review.opendev.org/65746309:03
openstackgerritMerged openstack/placement master: Correctly limit provider summaries when nested  https://review.opendev.org/66459409:45
*** ttsiouts has quit IRC10:05
gibicdent: ack.10:59
cdentthanks11:00
gibicdent: I'm back so I will catch up11:00
cdentthe discussion on the nested magic 1 spec is pretty interesting11:00
*** ttsiouts has joined #openstack-placement11:01
openstackgerritChris Dent proposed openstack/placement master: Add a blocker migration for missing consumer records  https://review.opendev.org/66462711:12
openstackgerritChris Dent proposed openstack/placement master: Remove incomplete consumer inline migrations  https://review.opendev.org/66485811:12
* cdent airport runs11:52
openstackgerritBalazs Gibizer proposed openstack/placement master: DNM: Visualize RP trees in gabbit fixtures  https://review.opendev.org/66487812:41
cdentgibi: https://github.com/gibizer/osc-placement-tree is awesome13:14
cdentmaybe we should consider putting that in osc-placement proper? Or at least release it to pypi13:23
*** artom has quit IRC13:29
*** dklyle has quit IRC13:34
gibicdent: I can definitly try to release it to pypi. do you have a good description how to do that? (this will be my first package on pypi)13:37
cdentgibi: one sec, looking for something13:37
cdentgibi: https://packaging.python.org/ is the canonical source of info13:39
cdentbut mostly what you're after is 'python setup.py sdist bdist_wheel' and then 'twine upload'13:39
gibicdent: pulling it to osc-placement can also be done I guess. But it is already an oppinionated repo. For example it is using black from the start13:40
gibicdent: thank. I will read up13:40
cdent(black from the start)++13:40
*** mriedem has joined #openstack-placement13:49
cdentefried: were you wanting to office hours chat this afternoon (in an hour). I can if required, but if not I have some other stuff I need to do (my sister's luggage didn't arrive, she needs to shop before moving on to another destination)14:06
cdentmriedem: I unwipped the ensure consumer blocker14:08
mriedemok14:09
mriedemfeels like we should have a reno in here somewhere...14:10
mriedemalthough, 'make sure you run your online data migrations' is already a thing14:11
mriedemand this one really should have been done before using extracted placement from nova anyway14:11
mriedemi think it was added to nova in...rocky?14:11
mriedemyeah https://github.com/openstack/nova/blob/stable/rocky/nova/cmd/manage.py#L41614:12
cdentmriedem: yeah, that was the same logic I followed when not reno-ing14:13
mriedem+2 on both14:19
cdentthanks14:19
* cdent is cruising through TODO/NOTE/FIXME looking for opportunities14:20
mriedemi've got a question for you before proceeding, on https://review.opendev.org/#/c/657016/14:21
mriedemi think the change itself is sound, the only hangup is the assertion in the commit message that if we fail to delete any providers that we'll stop and not delete the service,14:22
mriedemso i just need to fix that wording - since fixing the other thing is a different change IMOM14:22
mriedem*IMO14:22
mriedemcdent: do you agree?14:22
mriedemas noted inline, i've got a functional test recreating the "provider delete failed, compute service was still deleted" mess https://review.opendev.org/#/c/663737/14:22
cdentmriedem: yeah, I think that's fine.14:26
cdentyou're making things better, even though not perfect14:26
cdentso, win14:27
*** ttsiouts has quit IRC14:27
mriedemi believe the fix is we check if any provider for the service has allocations and if so we bail14:28
efriedcdent: Yes, I was assuming we would try to make progress as a group on "flow down" etc.14:28
*** ttsiouts has joined #openstack-placement14:28
mriedemthat would include currently running instances, unconfirmed migrations and evacuated instances that aren't cleaned up14:28
cdentefried: okay, I'll to situate myself appropriately, but may be slightly late depending on traffic14:30
efriedcdent: That's assuming it'll be more than just you & me. If others don't show up to office hours, then we can have that chat any time.14:31
cdentas usual, I've left yet more questions in response on the magic 1 spec14:31
efriedI'm responding to your response14:32
* cdent plans to respond to your response to my response to your response ...14:32
cdentbbs14:32
*** cdent has quit IRC14:32
*** ttsiouts has quit IRC14:32
* gibi will be here for the office hour14:33
*** dklyle has joined #openstack-placement14:45
*** tetsuro has joined #openstack-placement14:46
*** ttsiouts has joined #openstack-placement14:47
efriedtetsuro: o/ You sticking around for office hours?14:56
tetsuroYes I am14:56
efried++14:56
openstackgerritMerged openstack/placement master: Add NUMANetworkFixture for gabbits  https://review.opendev.org/65746314:58
gibio/14:59
efriedHi gibi. edleafe, are you going to hang with us?14:59
efriedcdent is on the road, but I think is trying to get back pretty shortly.15:00
gibiyes, I'm here15:00
efriedIs everyone more or less caught up on the major issue of the day?15:00
edleafeI is here15:01
edleafeAnd that issue would be... ?15:01
gibiI got back from a long weekend today so I'm not up to speed on the nested magic thing yet15:01
efriedOkay, the issue is that we have several use cases coming from nova that we're trying to address in the scope of the nested magic work, and the edges are really messy.15:02
*** artom has joined #openstack-placement15:02
efriedespecially if we try to address them in a way that's not completely nova-centric15:03
*** cdent has joined #openstack-placement15:04
efriedFor example, how do we add a component like "make sure I don't land on a WINDOWS_LICENSED host" to a request for other resources/traits/aggregates in such a way that it comes out the way we want/expect, whether our host is modeled flat or nested?15:04
efriedo/ cdent15:04
cdentahoy15:05
efriedfor now let's assume the above needs to be handled with a forbidden trait15:05
efrieddue to potential nesting of host, we can't guarantee that any resources will actually come from the root RP15:05
efriedbut the root RP is where the WINDOWS_LICENSED trait would be if it was present.15:05
efried"intuitively" I could add required_$UNIQUE=!WINDOWS_LICENSED (by _$UNIQUE I'm saying this request group isn't used for anything else)15:07
* edleafe loses interest15:08
edleafeIt's a simple query in Neo4j:15:08
edleafeMATCH (root:RESOURCE_PROVIDER)-[*]->(rp:RESOURCE_PROVIDER)-[:PROVDES]->(:MEMORY_MB)15:08
edleafeWHERE not exists(root.WINDOWS_LICENSED)15:08
edleafeBut I'm sure you're all sick of hearing that15:09
efriedyes edleafe, it's just as simple a query in the existing established database structure15:09
efriedThat's not the point15:09
efriedThe point is that it confers a particular meaning to a "resourceless request group with a forbidden trait"15:09
cdentcan we back up slightly15:09
efriedwhich might be worded as "must not exist anywhere in the tree surrounding the providers that actually contribute resource to the request"15:09
cdentsorry for arriving late, was in a car15:09
edleafeMy point is that you don't need to jump through the "resourceless request group" hoops15:10
edleafeBUt I'll shut up15:10
gibican we assume that the client knows that WINDOWS_LICENSED traits is always on the root RP?15:10
cdentwhy "required_$UNIQUE=!WINDOWS_LICENSED" instead of "required=!WINDOWS_LICENSED"?15:10
cdentI ask this because I continue to think that efried's interpretation of "the solution is incorrect"15:11
cdentwe are not askign for providers15:11
cdentwe are asking for allocatation candidates15:11
cdentan allocation candidate is made up of a set of providers which satisfy the request15:11
efriedright, I'm guessing you haven't had time to read the response to your response that I responded to mere minutes ago15:11
cdentno, I just did15:11
cdentand is why I'm bring it up. I still disagree with you.15:11
cdentyour state about today's definition is wrong15:12
efriedthe part I have trouble with isn't "the set of providers which satisfy the request". The part I have trouble with is how that definition would *not* incorporate "bad cousins".15:12
cdentbut we dont' want bad cousins, so what's wrong?15:12
efriedthe definition "set of providers which satisfy the request" would include them.15:13
cdentno, it does not15:13
cdentbecause they don't satisfy the request15:13
cdenta bad cousin is external to the set of providers15:13
cdentby its very nature of being a cousin15:13
efriedso is a root node that doesn't provide any resource15:13
cdentthere is no path to it in the graph15:13
cdenta root node is include because it is an ancestor of something that does provide15:14
efried^ this15:14
efriedis where my problem lies15:14
*** amodi has joined #openstack-placement15:14
efriedThat has always struck me as a completely arbitrary definition/distinction.15:14
efriedbut15:14
cdentwhich "that"15:14
cdentplease explain deeper the why15:14
efried"a (root) node is included because it is an ancestor of something that does provide [resource]"15:15
edleafeIMO, the root *is* the resource provider15:16
*** Sundar has joined #openstack-placement15:16
cdentwhen you ask for allocation candidate the thing that  you are asking for is a target15:16
cdentthe root is the target15:16
cdentso it _has_ to matter15:16
edleafejinxish15:16
cdentyes15:16
efriedwhat about intervening ancestors that aren't the root?15:16
cdentthe root is where the workload lands15:16
cdentthey matter too15:16
efriedwhy???15:17
cdentbecause they are on the path15:17
cdentwhether we use a graph db or not, a nested provider is _nested_15:17
cdentit's a graph15:17
cdentfrom the root to the leaves, all those providers are what we are "getting"15:17
efriedum15:17
gibiif a nested RP cannot exists without it ancestorts then I think it is fair to include them in the result15:17
efriedlook15:18
efriedI accept that this will solve our (nova-centric) use cases in a tidy way.15:18
efriedSo rather than arguing until the definition makes sense in my head, which it may never do, let's just accept that I am able to comprehend it and move on.15:18
efriedas long as we effect this change (and let's be clear: it is a change) in a new microversion15:18
cdentI _really_ want to understand what your issue is here, because as far as I can tell this is making sense to everyone but you, and since you're a critical piece of the pie, I'd like to reach a more complete understanding15:18
cdentbecause I still don't understand your objection, other than arbitrariness, and I'm pretty sure your position is arbitrary++15:19
cdentI think if we can clarify it, we'll achieve something closer to a tao or zen of nested15:19
cdentit's right there in the name15:19
cdent_nested_15:20
efriedoh, careful there, lest we start conferring meaning to things based on the name we chose for them (cf. "traits")15:20
cdentif you're willing to just accept it, I guess that can be fine, but I don't like it. I nearly failed calculus out of stubborness against that kind of thing15:20
efriedIt may just be that I have a looser notion of the binding that tree-ness imbues.15:20
cdentwe chose the name on purpose to mean hierarchical containership. if you're opposed to that hierarchy and that containment, that's the problem that needs to be fixed15:21
gibiso far we only have examples where a nested RP means a hard containment15:21
cdentsounds like it, and you need to disabuse yourself of that15:21
efriedor the fact that I don't accept it as meaningful to say "give me an SRIOV VF that supports MULTI_ATTACH"15:21
cdentyou're not askign for that15:21
efried"on a host that supports..." is a different query, to me.15:21
cdentyou're asking for a target that support MUTLI_ATTACH that also have an SRIOV_VF15:22
cdentthat is the _only_ query15:22
cdentthat's what placement does15:22
cdentagain it's in the name15:22
cdentand while it could be more than that15:22
efriedYes. And we don't have a syntax that indicates "a target"15:22
cdentwe don't want it to be15:22
cdentwe have a fast path to the target with  root_provider_uuid15:22
efriedcool, I can get behind making the root implicit. But that doesn't account for the intervening non-root ancestors (in my head)15:23
cdentthat we have a weird syntax is actually outside the original model. Originally it was just "show some resource providers", allocation candidates evolved organically and has warts as a result, but it would be good not to have those warts change the meaning of the system15:24
gibiefried: the SRIOV_VF resides inside the target (the root) to find it you need a path of RPs15:24
cdentwhat gibi said15:24
cdentit's a topology15:24
cdentand it's members provide meaning in various ways15:25
* edleafe is biting my tongue about graphs15:25
* cdent sews up edleafe's mouth for the time being15:25
efriedI'm not going to fail calculus, cdent. I understand the concept here. I don't agree with it, but I don't need to. That's what consensus is.15:25
cdentI wish you did agree, or we could get you there, because the more we talk about it the more I think you've got an entirely different model for how things _should_ work from many other people, and that's going to cause hiccups15:26
efriedMaybe some day I will. But for now, let's make progress.15:26
gibiin my head we cannot consume an SRIOV_VF without knowing, and accepting the properties of the full topology leading to the SRIOV_VF15:26
cdentnot only what gibi said, but also, at least to me thinking of it and using it as a toplogy makes it easier to grasp how the system works15:27
cdentokay, so we have an emerging consensus, can we state it?15:27
efried"A resource provider implicitly accumulates the traits and aggregates of its ancestors"15:29
efried?15:29
cdents/resource/nested resource/15:30
tetsuroI'm not sure we can achieve it with microversion without performance degradation15:30
cdents/ancestors/ancestors because nested implies a hierarchy of containership/ (or something more clear than that)15:30
efried"An 'allocation candidate' (a) explicitly includes providers providing resources to the request; and (b) implicitly includes all linear ancestors of those in (a)."15:31
cdents/An/Finding an/15:31
edleafeefried: that sounds upside down to me15:31
efriededleafe: how so?15:32
tetsuroIf nova wants trait flow down feature, alternatively why not have inherit option in create rps API?15:32
tetsuroopenstack resource provider create <rp_name> --inherit_traits15:32
edleafeA request is for a cloud resource - for Nova that would be a compute node15:32
tetsuroand recursive option in set trait API?15:32
cdenttetsuro: I think that's an option, but not a very elegant one, given what we've said about about hierarchy and containership15:32
edleafeHow the various resources are contained within it is a detail of the allocation15:32
cdentedleafe: that's why I made the "Finding an" edit15:33
efriedtetsuro: I think we should defer talking about how to implement until we've defined and shown how it addresses use cases15:33
edleafecdent: yeah, another jinxish message15:33
efriedI'm still unclear how we will define the behavior of suffixed resourceless with required or forbidden traits15:34
cdenttetsuro: another thing to look into is seeing if there is any real extant use case we break by making these adjustments. I don't think there's much _real_ "nested" out there in the world yet. And it _might_ be fair to make some adjustments outside the usual process15:34
efriedI'm way opposed to ^15:34
cdentbut yeah, agree we should postpone inmplementation discussion15:34
cdentefried: yet nothing is broken, ye4t15:34
cdentwe _think_ we have some legacy to support but that's simply not true15:35
cdentand we _think_ we are breaking a bunch of stuff but there's no evidence to support that assertion15:35
cdentso let's keep it in the box of options15:35
efriedcdent: This is broken: https://review.opendev.org/#/c/657510/7/placement/tests/functional/gabbits/granular-resourceless.yaml@2015:35
efriedbut yeah, let's defer that discussion too.15:36
cdentthat test was _written_ broken15:36
cdentwe're trying to fix it with this discussion15:36
efriedthe test succeeds with "flow down" but fails without it.15:36
efriedlet's defer please15:36
cdent[t dK28]15:37
purplerbot<efried> I'm still unclear how we will define the behavior of suffixed resourceless with required or forbidden traits [2019-06-12 15:34:08.065833] [n dK28]15:37
efriedYes:15:37
efriedCan someone explain to me what the meaning of15:37
efriedrequired_$UNIQUE=FOO15:37
efriedwould be, and how the definitions above incorporate that meaning (or are extended to do so)15:37
efried?15:37
efried(spoiler, next will be !FOO)15:37
efried[Aside: I'm suffering from several injuries of varying degrees at the moment. Apologies if pain+meds are making me dumber and/or grumpier than usual. (Yeah, yeah, you hadn't noticed, blah blah)]15:38
tetsuroIf traits flow down, that means there exists a provider that has FOO in the tree. If traits don't flow down, same.15:39
efried"in the tree"?15:39
efriedSo like, anywhere in the tree?15:39
tetsuroanywhere in the tree15:39
efriedNot limited to the providers satisfying the rest of the request?15:39
cdentyou're likely using "tree" differently15:39
efried...or their ancestors?15:40
cdentwithin the code there is a concept of a "tree" which is not the entire tree from root15:40
efriedthere is?15:40
cdentin the implementation, yes15:40
efriedwow15:40
efriedI was totally unaware of this15:40
cdentit's why implement resource hacks was so "easy"15:40
gibieach returned candidate needs to have a provider in it that has the trait FOO (it is implicit that each candidate needs to consist of a single tree + sharing PRs)15:41
cdentthe internal concept of tree is how we are able to get different sets of candidates from the same target15:42
cdenta tree is a description of the path through from root that provides a solution15:42
efriedwhen we're talking about "within the code" and "in the implementation" we're talking about stuff in the actual master branch, not including things proposed-but-not-merged15:43
cdentyes15:43
efriedokay, because my perception of how that works has nothing to do with tracing paths up trees15:43
efriedit has to do with collecting all the providers in the (whole) tree and then filtering down to the ones that satisfy the request.15:43
cdentmathwise, that's really the same thing15:44
cdentwhich is perhaps where I'm adding confusion to this discussion15:44
cdentin any case, we're veering15:45
efriedit may be possible to do a formal mathematical proof that demonstrates that the result is the same. But the process/formula is different.15:45
edleafeMathwise, maybe, but certainly not concept-wise15:45
efriededleafe++15:45
edleafeAnother jinxish!15:45
efriedbut yes, we are veering.15:45
cdentthe point I was trying to make was that it seemed like tetsuro and efried were potentially meaning different things with "anywhere in the tree", a critical difference15:45
efriedyes, we definitely need to nail that down.15:46
cdentfor efried "anywhere in the tree" includes bad cousins15:46
efriedyes15:46
cdentfor me (and maybe tetsuro, not sure?) it does not15:46
efriedfor the code, it would also include bad cousins15:46
cdentmaybe for master15:46
efriedthe code as it stands today15:46
efriedyes15:46
cdentbut not for the hack I did yesterday15:46
tetsuro for me "anywhere in the tree" includes bad cousins :)15:46
cdentah okay, good to know15:47
edleafeI think that the "tree" notion is misleading. Only some branches are involved in an allocation candidate, not the entire tree15:47
edleafe(which is why the graph approach works out more cleanly)15:47
efriedyes, can we start using another term when we're talking about the subset of the tree (ascending to the root) that provides the solution?15:47
SundarSorry for jumping in the middle. I just caught up (I think). "traits flow down" - this makes no sense to me. If the compute node has a trait like AVX512, that doesn't mean the nested RPs (e.g. devices) have that instruction support.15:48
cdentcan we call it "solution" or "solution tree" versus "provider tree"?15:48
edleafeI'd prefer "path"15:48
cdentpath is fine too15:48
efriedas long as we accept that it allows for branching15:48
*** helenfm has quit IRC15:48
efriedbut only in one direction15:48
edleafeof course15:48
cdentSundar: when we talk about this stuff we're not talking about whether the provider has the thing, but whether it is in a solution that has it available15:49
edleafeefried: heh, now you're sounding like a graph proponent :)15:49
cdentso if I ask for an SRIOV_VF and AVX512 the SRIOV doesn't have AVX512 but the compute node does15:49
cdent"the compute node" is "the target"15:50
edleafeSundar: another way to look at it is that every RP doesn't have to have that trait, but it does need to be present15:50
cdentusing the new term it's whether AVX512 is in the "path"15:50
cdentso: what is the relationship between a "path" and bad cousins?15:52
efriedright, so to me required_$UNIQUE=FOO "intuitively" means "include a provider in the solution path that has FOO"15:52
efriedwhich tetsuro and I think means "regardless of anything else that's happening in the solution path"15:52
efriedi.e. required_$UNIQUE=FOO is *augmenting* the solution path15:52
edleafeI believe you've defined "bad cousins" as having a common root, but not on the path15:52
*** ttsiouts has quit IRC15:53
cdentedleafe: yes, that's what I think, I'm not sure it is mutally agreed15:53
*** ttsiouts has joined #openstack-placement15:53
efriedWhat we're trying to find a way to say is:15:53
efriedrequired_$UNIQUE=FOO needs to apply as a filter affecting the solution path as determined by the *other* pieces of the request15:53
edleafeI think "bad" is not a good description. At most, they're irrelevant15:54
gibiin the containmentship view a forbidden trait means that your solution should never enter a container that has the forbidden property15:54
gibilike you never touch a NUMA node with bad smell15:55
edleafeI never touch NUMA nodes regardless of their odor15:55
cdentefried: I think that's kind of the nature of request groups already, yes: we do them in what amounts to a kind of order15:55
efriedtoday we don't15:56
efriedthe only thing that's "ordered" is we do "isolate" as a late filter.15:56
efriedother than "isolate", request groups don't interact with each other15:56
efried(except for having to land in the same tree)15:56
efriedwhat we're trying to find a way to express now is a change to that concept15:57
efried"use the information from this request group to affect what's allowed to be returned from other request groups"15:57
cdentbrb15:58
*** ttsiouts has quit IRC15:58
edleafeOne last thing before we end...15:59
edleafeI just want to let everyone know that I've been "resource actioned" (IBM speak for "laid off"), with my last day in early September. So I most likely won't be around after that.15:59
cdentmeh to that...16:00
gibiedleafe: that is sad :(16:00
Sundaredleafe: Tough16:00
edleafeThanks everyone. Now if you hear of any openings... :)16:00
efriededleafe: Sorry to hear that16:01
efriedDo we need to "end"? Seems like we have made progress, but there's a lot more to close on. Can folks stick around?16:02
cdentyeah, I can still be here. this seems to be where we are [t dxjE]16:02
purplerbot<efried> "use the information from this request group to affect what's allowed to be returned from other request groups" [2019-06-12 15:57:49.494855] [n dxjE]16:02
edleafeI was thinking of those in timezones where the clock is later16:03
cdentthat's what [t 13xo] does16:03
purplerbot<efried> "A resource provider implicitly accumulates the traits and aggregates of its ancestors" [2019-06-12 15:29:22.501625] [n 13xo]16:03
cdentbecause it is not request groups influencing each other16:03
cdentit's each request group having more to look at/think about16:03
efriedYou're going to have to help me understand how that works (again).16:04
cdent(for traits and aggregates, but not inventory)16:04
cdentthe set of resource providers is that satisfies (independent of other things)  required_$UNIQUE=FOO is all those providers which have trait foo, or an ancestor with trait FOO (which will sometimes be a huge set)16:06
cdentthat set is winnowed during the equivalent of _merge_candidates16:06
efriedRight - ^ doesn't include in its definition "that satisfy other parts of the request"16:06
efriedroot R has two children, A(rc:X, trait:FOO) and B(rc:X, trait:BAR)16:07
efriedIf I ask for ?resources=X&required_1=FOO16:07
efriedthe required_1=FOO part is saying "all those providers which have trait FOO, or an ancestor with trait FOO"16:07
cdentthere are a variety of ways to optimize that. one of them is to do a "late" trait check16:07
cdentso you get back R A as the result path16:08
efried...which will get me solution paths16:08
efried- A(X, FOO)16:08
efried- A(FOO), B(X)16:08
efriedthe B(X) satisfies required_1=FOO16:08
cdenthow? it ha not ancestor with FOO16:08
efriedagh, sorry, disregard16:08
efriedThe A(FOO) satisfies required_1=FOO16:08
efriedlet me be more complete16:09
* cdent waits for an EOF16:09
cdent(while that's percolating, it would be great if gibi and tetsuro could write down somewhere their own take on this stuff, because it feels like this discussion and has been the chris and eric show and I think it's likely the case that tetsuro and gibi understand some of these things if not better, at least differently and we need to incorporate that)16:11
gibi(I'm slower than you guys, I will try to sum my view in the review)16:13
tetsuroWhether we go with and without trait flow down feature we will get A(FOO), B(X), in that example, I think?16:14
cdenttetsuro: can you explain why, because to me that makes no sense16:15
efriedtetsuro: ++16:15
efriedwithout additional definitionizing, using "the set of resource providers is that satisfies (independent of other things)  required_$UNIQUE=FOO is all those providers which have trait foo, or an ancestor with trait FOO"16:15
efriedwith provider tree  root R() with two children, A(rc:X, trait:FOO) and B(rc:X, trait:BAR)16:15
efriedwith request        ?resources=X&required_1=FOO16:15
efriedwill result in two solution paths:16:15
efried:1: R(), A(X, FOO)16:15
efried:2: R(), A(FOO), B(X)16:15
efriedIn :1:, A(X, FOO) satisfies both resources=X and               required_1=FOO; and R() is included by virtue of being an ancestor of      A16:15
efriedIn :2:, B(X) satisfies           resources=X; A(FOO) satisfies required_1=FOO; and R() is included by virtue of being an ancestor of both A and B16:15
efriedWhat (I think) we are trying to do is contrive a definition such that the request will result in *only* solution path :1:. I don't know how we're going to do that.16:15
efriedEOM16:15
Sundarcdent: I am interested in the device colocation use case -- wherein a device has multiple components, each component is a RP with its own RCs and traits, and there could be multiple such devices on the same compute node. The use case is that all resources for a VM should come from the same device (co-located), not different devices in the same node16:15
cdentthanks efried16:15
cdentSundar: can we come back to that a bit later?16:16
* cdent digests efried 16:16
* efried finds stronger pain meds while cdent digests him16:16
* Sundar wonders how efried will exit cdent after the digestion16:17
cdenthmm. I still don't understand 2. There is a path R -> B. Nowhere in that path is FOO so that is not a solution path. If FOO were on R and traits flowed down, then both R->B and R->A would be solution paths.16:19
cdentIn 2 B isn't even a "bad cousin", it's simply irrelevant16:19
efriedI made it relevant when I asked for required_1=FOO16:20
efriediow16:20
efriedin today's definition, required_1=FOO has nothing to do with resources=X16:20
efriedwe're trying to contrive a definition where we're saying that the required_1 and the resources=X have to affect each other16:21
cdentyou keep saying that, and I still think that's backwards16:21
edleafeTraits are applied to RPs that have the necessary resources16:21
cdentthe set of providers for which required_1=FOO is true is A16:21
cdentthe set of provider for which resources=X  is A, B16:22
cdentthe intersection is A16:22
efriedIn this request16:22
efried?resources=X&required_1=FOO16:22
efriedI'm saying:16:22
efriedGive me a solution path which includes two things:16:22
efried- A resource provider that can provide X16:22
efried- A resource provider that has FOO16:22
cdentA's ancestor is R, so R A16:22
efriedwhich means there are two possibilities16:22
efried:1: a permutation where both bullets are satisfied by the same provider16:22
efried:2: a permutation where a separate provider satisfies each bullet16:22
efriedthis is, as tetsuro said, regardless of traits flowing anywhere.16:23
* cdent chews again16:23
efriedlooklook, maybe this helps, let's talk about master, where there are no resourceless request groups and there is no flowing16:25
efried?resources1=X16:25
efried&resources2=A&required2=B16:25
efriedThe required2=B does not influence which solution paths involve X resources16:25
cdentin a nested tree R with children A and B, for any request the potential solution trees are R->A, R->B and R->A&B, yes?16:25
efriedyes16:25
* cdent keeps chewing16:26
tetsuro R(resource:X) with children A(resource:Y) and B(resource:Y); If we ask ?resource1=Y&resource2=Y&group_policy=none It has 3 alloc_reqs [(A), (B), (A, B)]16:30
tetsuroFWIW16:31
gibiI feel one level of confusion in me is because traits are attached to RPs but we consume resources so I my brain shortcut it to traits-defining-properties-of-resources-we-consume16:31
efriedgood summary of why traits-flow-down doesn't make sense to me without additional definitions of "and we're implicitly including this host" ^16:32
efriedbut let's not get off on that tangent again16:32
efriedtetsuro: Yes, I think that example holds now (master) and in any conceivable future. I don't think any decisions we're talking about would change it.16:33
cdentI'm going to be called off to dinner very soon.16:33
cdentI'm trying to discern if we've confused everything, or reached some clarity on some points16:34
SundarIn efried's example, for the request ?resources1=X&required_2=FOO, if there is  one RP that satisfies both, should we return that alone? Or should we return 2 different RPs? -- That is the question, right?16:34
efriedSundar: Yeah16:35
efried2 different *candidates* (or solution paths)16:35
SundarThat would depend on the group_policy, one would think16:35
efriedwell, since you brought it up,16:36
cdentoh dear16:36
efriedresources1=X&required2=FOO16:36
efriedwith group_policy=none would return the two candidates I listed above :1: and :2:16:36
efriedwith group_policy=isolate would return *only* :2: !!16:36
efried(which, reminder, is the one we actually want to get rid of for this use case)16:37
cdentif you asked for resources1=X&required2=FOO&group_policy=isolate, getting (only) two candidates would be the correct result, albeit not likely what you wanted16:39
cdentsorry I mean only the second candidate option16:39
* cdent is brain hungry16:39
efriedTo summarize this issue:16:40
efriedToday (master) request groups don't influence each other.16:40
efriedWe're looking for a syntax that says "late-filter candidates derived by existing master-ish code paths such that they include only solution paths containing trait FOO"16:40
efriedAnd thus far we've been trying to make resourceless+required satisfy that16:40
efriedBut I haven't figured out a unifying definition that makes that ^ make sense16:40
efriedAnd perhaps we should consider a late_check_solution_path_required=FOO instead16:40
cdentwhich suggests: can we just say the original proposition is a bad request? That is, what is the request form that gets the results we want?16:41
efriedyes, let's start looking from that angle16:41
cdentwhich is X and FOO in the same group, yes?16:41
efried"in the same solution path"16:42
efrieduh16:42
efriedlet me try to state it differently:16:42
* gibi needs to drop soon16:42
cdenti mean in the same query: resources=X&required=FOO -> A16:42
efriedFIRST: find me solution paths that provide resources=X16:43
efriedTHEN:  filter those such that I'm only left with ones that have FOO16:43
efriedresources=X&required=FOO => A is what we have today (master) fwiw16:43
cdentyes16:43
efriedyes which?16:43
cdentyes to fwiw16:43
efriedcan we agree that FIRST/THEN is what we're trying to enable?16:44
*** e0ne has quit IRC16:44
cdentisn't that what we already do when in the same request group?16:44
efriedYeah, I can buy that.16:44
cdentso back a couple questions:16:45
cdentIs '?resources=X&required_1=FOO' a "rational" query?16:45
efriedIMO as long as it means "the solution path includes a provider *anywhere in the tree* that has FOO" then I think it's meaningful. Whether it satisfies any "rational" use case is another question.16:46
cdent(I have 5 minutes or less)16:46
efriedBut IMO it could be used to "rationally" formulate requests where we know the trait is on the root provider.16:47
cdentI think that's likely fair16:47
efriedbecause the root will always be part of the solution path anyway16:47
efried(as long as group_policy=none btw)16:47
* cdent nods16:47
efried(otherwise we run the risk of intersecting ourselves to the empty set)16:47
cdentokay, i gotta go16:49
gibi(I have to drop, I will definitly sleep on the problem and try to say somthing about it)16:49
cdentit feels like we made some smidge of progress but not certain16:49
* gibi brain hurts16:50
cdentme too16:50
cdentbig headache16:50
cdentneed deins16:50
cdentdins16:50
cdentby16:50
cdentbye16:50
gibibye16:50
efriedyeah, we still haven't solved "make sure FOO exists in the solution path otherwise established"16:50
efriedbye16:50
*** cdent has quit IRC16:50
*** tetsuro has quit IRC16:55
openstackgerritBalazs Gibizer proposed openstack/placement master: Add support for osprofiler in wsgi  https://review.opendev.org/66394517:39
openstackgerritBalazs Gibizer proposed openstack/placement master: Add support for osprofiler in wsgi  https://review.opendev.org/66394517:42
*** Sundar has quit IRC17:46
*** tssurya has quit IRC17:46
*** e0ne has joined #openstack-placement18:10
*** e0ne has quit IRC18:12
*** e0ne has joined #openstack-placement18:12
*** e0ne has quit IRC18:17
*** e0ne has joined #openstack-placement18:19
*** e0ne has quit IRC18:35
*** e0ne has joined #openstack-placement19:23
*** e0ne has quit IRC19:26
*** cdent has joined #openstack-placement20:32
cdent[t e5RQ]20:33
purplerbot<efried> yeah, we still haven't solved "make sure FOO exists in the solution path otherwise established" [2019-06-12 16:50:17.567519] [n e5RQ]20:33
cdenttwo thoughts on that: a) remind me what's the (real world) use case for that, b) if we can rely on modeling; that can be required=FOO and FOO on the root, c) ?20:36
* cdent leaves again20:36
*** cdent has quit IRC20:37
*** artom has quit IRC20:59
openstackgerritChris Dent proposed openstack/placement master: Prepare objects for allocation request mappings  https://review.opendev.org/66278521:19
openstackgerritChris Dent proposed openstack/placement master: Implement allocation candidate mappings  https://review.opendev.org/66224521:19
efried"a) remind me what's the (real world) use case for that" [where "that" == "make sure FOO exists in the solution path otherwise established"]21:20
efriedThe only one that can't be solved with resourceless+same_subtree is "make sure my host can FOO" (or "not FOO").21:20
efriedThe problem with "b) if we can rely on modeling; that can be required=FOO and FOO on the root" (plus "flow down") is that we21:20
efried(i) need to tie the trait to some potentially arbitrary/unrelated resource; and21:20
efried(ii) shouldn't need to construct such a complicated query for what should be a simple & common case.21:20
efriedUnless you're suggesting that specifically resourceless+unsuffixed required=FOO has special meaning of "on the root". Which would work, but seems arbitrary and will certainly be a special case in the code.21:20
efriedAnd finally, "c) ?" <== I was getting geared up to suggest some kind of new request-wide syntax.21:20
efriedToday we have "request groups" which include {resources, required, member_of, in_tree} (in_tree is weird to me as being part of a request group, not sure why we landed there, but going with it for now). As previously discussed, one request group doesn't interact with another except to land in the same tree/aggregate.21:20
efriedAnd then we have request-wide options: {limit, group_policy}. These effectively come into play to do late filters after we've already processed request groups ^21:20
efriedSo it makes sense (to me) that "...in the solution path otherwise established" would indicate a new thing on the "request-wide" side, not in the "request group".21:20
efriedTo expand on the problem statement: "For each allocation candidate we've come up with by processing request groups, filter down to only those whose 'solution path' contains a provider with FOO" ("... contains *no* provider with FOO" for the forbidden case. Similar parallels for member_of etc).21:20
efriedI don't have a great suggestion for what the query param key(s - need to support required/forbidden x trait/aggregate) would be. But it (they) would *not* be subject to granular-ness.21:20
efriedUsing the unsuffixed group would be semantically great - except that we already have meaning attached to that guy.21:20
efriedBut if we want to special-case as discussed above by saying that this ^ is the behavior when the unsuffixed group is resourceless... I could possibly be persuaded of that.21:20
efriedcdent ^21:21
*** mriedem has quit IRC21:24
*** takashin has joined #openstack-placement21:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!