Tuesday, 2019-04-23

*** gerrykopec has joined #openstack-nova00:03
*** tetsuro has joined #openstack-nova00:12
*** tetsuro_ has joined #openstack-nova00:14
*** Sundar has quit IRC00:14
*** tetsuro has quit IRC00:16
*** ttsiouts has joined #openstack-nova00:19
*** ttsiouts_ has joined #openstack-nova00:26
*** ttsiouts has quit IRC00:28
*** gyee has quit IRC00:29
*** mriedem has quit IRC00:30
*** ttsiouts has joined #openstack-nova00:34
*** dave-mccowan has joined #openstack-nova00:35
*** ttsiouts_ has quit IRC00:36
*** hemna has quit IRC00:37
*** hemna has joined #openstack-nova00:38
*** ttsiouts has quit IRC00:41
*** ttsiouts has joined #openstack-nova00:43
*** tetsuro_ has quit IRC00:50
*** ttsiouts has quit IRC00:51
*** markvoelker has joined #openstack-nova00:51
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Fix a description for config_drive parameter  https://review.opendev.org/65368300:59
*** whoami-rajat has joined #openstack-nova01:01
openstackgerritya.wang proposed openstack/nova-specs master: Expose auto converge and post copy  https://review.opendev.org/65168101:14
*** ricolin has joined #openstack-nova01:15
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Updates for OpenDev transition  https://review.opendev.org/65442901:20
*** yedongcan has joined #openstack-nova01:31
*** takashin has joined #openstack-nova01:43
*** HuaChangWang has joined #openstack-nova01:45
mnasershopping an idea01:49
mnaserhow do you feel about killing devstack.01:49
mnaserand replacing it by a deployment tool01:49
*** brinzhang has joined #openstack-nova01:50
*** nicolasbock has quit IRC02:04
*** dakshina-ilangov has quit IRC02:05
*** _erlon_ has quit IRC02:05
openstackgerritzhongshengping proposed openstack/nova master: Replace git.openstack.org URLs with opendev.org URLs  https://review.opendev.org/65439802:08
*** cfriesen has quit IRC02:11
*** HuaChangWang has quit IRC02:30
*** lbragstad has joined #openstack-nova02:39
lbragstadefried get it figured out?02:40
lbragstadefried as far as i know, the duplicated sessions were completely removed from the schedule02:41
lbragstadhttps://www.openstack.org/summit/denver-2019/summit-schedule/events/23642/increasing-api-accessibility-with-granular-policy-and-default-roles is the one i have on my schedule02:43
*** dklyle has quit IRC02:45
*** dklyle has joined #openstack-nova02:46
*** ttsiouts has joined #openstack-nova02:48
*** dave-mccowan has quit IRC02:48
*** takashin has left #openstack-nova03:03
*** hongbin has joined #openstack-nova03:10
*** cfriesen has joined #openstack-nova03:17
*** ttsiouts has quit IRC03:21
*** cfriesen has quit IRC03:32
*** lpetrut has joined #openstack-nova03:50
*** lpetrut has quit IRC04:13
openstackgerritErik Olof Gunnar Andersson proposed openstack/nova master: Pass on region when we don't have a valid ironic endpoint  https://review.opendev.org/65469204:15
openstackgerritMerged openstack/nova stable/rocky: libvirt: set device address tag only if setting disk unit  https://review.opendev.org/65351104:16
eanderssonefried, ^ we could honestly just pass on the region, since region_name is only actually consumed if endpoint or session is missing.04:16
eanderssonBut I am fine just passing it on when we don't have an endpoint.04:17
*** udesale has joined #openstack-nova04:22
*** hongbin has quit IRC04:33
*** lbragstad has quit IRC04:47
*** ratailor has joined #openstack-nova04:56
*** markvoelker has quit IRC04:57
*** sridharg has joined #openstack-nova05:11
*** jaosorior has joined #openstack-nova05:17
*** ttsiouts has joined #openstack-nova05:18
*** Luzi has joined #openstack-nova05:36
*** ivve has joined #openstack-nova05:38
*** ttsiouts_ has joined #openstack-nova05:43
*** ttsiouts has quit IRC05:45
*** ccamacho has quit IRC05:46
*** ttsiouts has joined #openstack-nova05:47
*** ttsiouts_ has quit IRC05:49
*** ttsiouts has quit IRC05:52
*** lpetrut has joined #openstack-nova06:00
*** pcaruana has joined #openstack-nova06:06
*** dpawlik has joined #openstack-nova06:18
*** slaweq has joined #openstack-nova06:21
*** udesale has quit IRC06:35
*** udesale has joined #openstack-nova06:37
*** udesale has quit IRC06:44
*** udesale has joined #openstack-nova06:44
*** markvoelker has joined #openstack-nova06:52
*** ircuser-1 has quit IRC06:53
*** ircuser-1 has joined #openstack-nova06:56
*** ccamacho has joined #openstack-nova07:13
*** lee1 has joined #openstack-nova07:17
*** tosky has joined #openstack-nova07:17
*** udesale has quit IRC07:18
*** lee1 is now known as lyarwood07:18
*** udesale has joined #openstack-nova07:19
*** ttsiouts has joined #openstack-nova07:22
*** ccamacho has quit IRC07:27
*** ccamacho has joined #openstack-nova07:27
openstackgerritBoxiang Zhu proposed openstack/python-novaclient master: Add host and hypervisor_hostname to create servers  https://review.opendev.org/64767107:37
*** kashyap has joined #openstack-nova07:46
kashyapefried: Was on PTO yesterday.  Just saw the scroll.  I will address yours and mriedem's questions on that "CPU selection..." spec.07:52
kashyapefried: I tried to be conscious to "not assume anything", _still_ my phrasing seems to be not sufficiently clear enough.  Will rework.07:54
*** dtantsur|afk is now known as dtantsur07:55
sean-k-mooneykashyap: i think the issue is you assumed people understand and follow libvirt and did not present the usecase in terms of nova but instead present it as libvirt has this new api we shoudl use it with the assumtion people knew what libvirt api we currently use and what features it is used for07:55
kashyapsean-k-mooney: I partly disagree.  I didn't "assume" it that way.07:56
kashyapHow much more clearer can you phrase this:07:56
kashyap[quote]07:56
kashyap[/quote]07:57
kashyapsean-k-mooney: My comment on PS-4, on: Apr 9 6:38 PM07:57
sean-k-mooneythat did not rendeer from me07:57
sean-k-mooneyi just see open and close quotes07:58
kashyapI can add additional details and answer questions -- sure.  But what is already there is sufficiently clear enough, no?07:58
kashyapsean-k-mooney: Yeah, that was intentional; sorry :-) I didn't paste anything07:58
kashyap(Because I didn't wanted to paste the large comment from the change)07:58
kashyapsean-k-mooney: See my comment on PS-4: https://review.opendev.org/#/c/645814/407:58
*** luksky has joined #openstack-nova07:59
sean-k-mooneyya but the poblem with that comment is you dont say how nova will handel it08:00
kashyapsean-k-mooney: I told Nova's guest CPU selection is lacking.  And even provided a rough example like:08:00
kashyap     "Is this combination of IvyBridge + 'pcid' & 'pdpe1gb' supported by KVM, QEMU and libvirt on the host?"08:00
kashyapsean-k-mooney: Yes, that part, I will addres.08:00
sean-k-mooneye.g. will the agent refuse to start08:00
kashyapRight, will address them, once I finish addressing comments on the Secure Boot spec.08:00
sean-k-mooneyby the way nova has no cpu selection logic outside of host-model so its not really lacking as non existent08:01
sean-k-mooneyi just found out that weechat has mouse support and gesture ...08:02
sean-k-mooneyi dont think i like them08:03
kashyapsean-k-mooney: I'm of course aware that we default to 'host-model'.  That's not the point.  When I say "CPU selection", it also means CPU model + features (obviously).08:04
sean-k-mooneykashyap: for what its worth i only barely followed what you were proposing in that spec.08:04
kashyaphttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3948,#L395708:04
sean-k-mooneykashyap: yes but that is hardcoded effectivly via the config08:04
sean-k-mooneyits not nova making the selection in this case its the operator08:04
kashyap(I will rephrase the spec with a couple more examples.)08:04
kashyap(Not entirely, some bits.  While adding additional details.)08:05
openstackgerritBoxiang Zhu proposed openstack/nova master: Add host and hypervisor_hostname flag to create server  https://review.opendev.org/64552008:05
sean-k-mooney one thing i was wondering is do you want to merge this spec with https://review.opendev.org/#/c/642030/08:06
kashyapNo, I want to keep it separate.08:06
sean-k-mooneythe current one has no end user visable impact08:06
sean-k-mooneyok08:06
kashyapAs I'm not even sure if we should do: https://review.opendev.org/#/c/642030/08:06
sean-k-mooneyya i guessed that is why08:06
sean-k-mooneyif we do implment the second spec https://review.opendev.org/#/c/642030/ it will depend on this new api08:07
sean-k-mooneyor it should use the new api08:07
kashyapAlso, not every spec needs to be "user visible".  (That said, in this case, it _is_ "operator visible" in a way.  Will save the write-up for the spec)08:07
*** helenafm has joined #openstack-nova08:07
sean-k-mooneywell its not other then the agent will stop if its not compatible08:08
*** rpittau|afk is now known as rpittau08:08
sean-k-mooneythere is not rest api changes08:08
sean-k-mooneyor traits changes08:08
kashyapsean-k-mooney: It is not just about 'compat check'.  The resulting CPU + flags will be more _useful_ and _useable_08:09
sean-k-mooneyhow08:09
kashyapI explained it in the spec.  I wonder if you even read that.08:10
sean-k-mooneythe set of flag we get back for what the host support wont change08:10
sean-k-mooneyyes i did08:10
sean-k-mooneyyou did not explain it well08:10
sean-k-mooneyin terms of how its more useful to nova08:10
sean-k-mooneybeyond we can as if the value in the config are valid08:10
kashyapsean-k-mooney: Did you read this at all?08:12
kashyap[quote]08:12
kashyap- While determining guest CPU models, Nova can take into account several other aspects, e.g. the type of virtualization (pure emulation vs. hardware-accelerated), QEMU binary's capabilities, guest machine type, and CPU architcture to construct a better-informed guest CPU.08:12
kashyap[/quote]08:12
kashyapWhat is unclear about that?08:12
kashyapYour "how" is answered by the above.08:12
sean-k-mooneywhat is uncleare is what does "better-informned guest cpu" mean08:12
*** tssurya has joined #openstack-nova08:12
sean-k-mooneywill the cpu flag in the guest (via lscpu) change08:13
kashyapPlease wait.08:13
kashyapOne thing at a time.08:13
kashyapYou get a "better-informed guest CPU" because: the newer API, when calculating baseline, baselineHypervisorCPU() will *filter out* CPU features that are not supported by QEMU on the host.08:14
* kashyap --> goes heads-down; be back later08:15
sean-k-mooneyo/08:16
*** jangutter has joined #openstack-nova08:18
*** ttsiouts_ has joined #openstack-nova08:21
*** ttsiouts has quit IRC08:23
*** dikonoor has joined #openstack-nova08:27
*** derekh has joined #openstack-nova08:28
*** yedongcan has quit IRC08:32
*** yedongcan has joined #openstack-nova08:32
*** tkajinam has quit IRC08:42
*** gibi_off is now known as gibi08:47
openstackgerritSurya Seetharaman proposed openstack/nova-specs master: Support server power state update through external event  https://review.opendev.org/63613208:52
*** ttsiouts has joined #openstack-nova08:53
*** jiaopengju has joined #openstack-nova08:53
*** Dinesh_Bhor has quit IRC08:55
*** yankcrime has joined #openstack-nova08:55
*** ttsiouts_ has quit IRC08:56
*** dikonoor has quit IRC08:56
*** dikonoor has joined #openstack-nova09:09
*** phasespace has joined #openstack-nova09:24
*** ratailor_ has joined #openstack-nova09:25
*** ratailor has quit IRC09:27
*** lpetrut has quit IRC09:30
*** jaosorior has quit IRC09:31
*** jaosorior has joined #openstack-nova10:14
*** threestrands has quit IRC10:15
*** NewBruce0 has quit IRC10:39
*** NewBruce has joined #openstack-nova10:39
*** tbachman has quit IRC10:40
*** tetsuro has joined #openstack-nova10:56
*** nicolasbock has joined #openstack-nova11:00
*** jaosorior has quit IRC11:00
*** jaosorior has joined #openstack-nova11:02
*** dikonoor has quit IRC11:03
*** tetsuro has quit IRC11:03
*** dikonoor has joined #openstack-nova11:06
alex_xuemm...US evus enrollment not compatible with google chrome :(11:21
*** yedongcan has left #openstack-nova11:27
*** belmoreira has joined #openstack-nova11:45
jangutteralex_xu: Could be worse, according to https://help.cbp.gov/app/answers/detail/a_id/1090/~/technical-difficulties-submitting-my-esta-application the ESTA system supports IE 5 SP 2, or Netscape 6.211:46
sean-k-mooneyjangutter: firefox worked for me but ya government sites then to be updated infrequently11:53
*** tbachman has joined #openstack-nova11:55
*** panda is now known as panda|lunch11:57
*** markvoelker has quit IRC12:01
*** jiaopengju has quit IRC12:01
jangutter(the versions of Netscape and IE probably correspond to the date that the original software was delivered by the contractors.)12:01
*** jiaopengju has joined #openstack-nova12:02
sean-k-mooneyya the current maintainer (if there is one) proably has never seen that page12:02
*** jiaopengju has quit IRC12:02
*** jiaopengju has joined #openstack-nova12:03
*** jiaopengju has quit IRC12:04
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove unused param from _fill_provider_mapping  https://review.opendev.org/65510712:04
openstackgerritBalazs Gibizer proposed openstack/nova master: Move _fill_provider_mapping to the scheduler_utils  https://review.opendev.org/65510812:04
openstackgerritBalazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth  https://review.opendev.org/65510912:04
openstackgerritBalazs Gibizer proposed openstack/nova master: allow getting resource request of every bound ports of an instance  https://review.opendev.org/65511012:04
openstackgerritBalazs Gibizer proposed openstack/nova master: Pass network API to the conducor's MigrationTask  https://review.opendev.org/65511112:04
openstackgerritBalazs Gibizer proposed openstack/nova master: handle port allocation during migration  https://review.opendev.org/65511212:04
openstackgerritBalazs Gibizer proposed openstack/nova master: func test for migrate server with ports having resource request  https://review.opendev.org/65511312:04
openstackgerritBalazs Gibizer proposed openstack/nova master: Extend NeutronFixture to handle migrations  https://review.opendev.org/65511412:04
*** jiaopengju has joined #openstack-nova12:04
*** mchlumsky has joined #openstack-nova12:06
*** mriedem has joined #openstack-nova12:11
mriedemi need a stable core to hit this queens backport https://review.opendev.org/#/c/640198/12:13
*** jiaopengju_1 has joined #openstack-nova12:16
*** brinzhang has quit IRC12:18
*** jiaopengju has quit IRC12:19
openstackgerritsean mooney proposed openstack/nova master: [WIP] Libvirt: add nfv job  https://review.opendev.org/65219712:22
openstackgerritMatt Riedemann proposed openstack/nova master: DNM: debug config opts infinite recursion  https://review.opendev.org/65446812:26
openstackgerritMatt Riedemann proposed openstack/nova master: Don't run tempest/devstack jobs on nova/test.py only changes  https://review.opendev.org/65512112:26
aspiersvcpu:112:32
aspiersoops12:32
*** mchlumsky has quit IRC12:38
*** mchlumsky has joined #openstack-nova12:38
*** tobias-urdin has joined #openstack-nova12:39
kashyapaspiers: When you're about -- remind me again, are you planning to cache the result of getDomainCapabilities(), as part of SEV work?12:40
aspierskashyap: not just planning, but https://review.opendev.org/#/c/633855/12/nova/virt/libvirt/host.py already does that12:41
*** arshad777 has joined #openstack-nova12:41
kashyapaspiers: Ah:12:41
kashyap                domain_caps[arch][machine_type] = \12:41
kashyap                    self._get_domain_capabilities(emulator_bin, arch,12:41
kashyap                                                  machine_type, virt_type)12:41
aspiersright12:42
kashyapaspiers: So something to consider:12:42
kashyapaspiers: This morning, while talking to a libvirt developer (Michal Privoznik), he was saying: caching domain capabilities probably doesn't make sense -- because it will be outdated everytime libvirt and QEMU are updated.12:43
arshad777Hi everyone: I am trying to create an instance on multi-node setup. But getting error "no valid host found". For detail refer url http://paste.openstack.org/show/749631/12:43
arshad777compute services are running. Please refer url http://paste.openstack.org/show/749630/ for more detail12:43
aspierskashyap: it would only be cached in the running service, not persisted to disk12:43
kashyapaspiers: As we know, there are 4 input values (path to qemu binary, guest architecture, machine type and virt type) -- so any change if one of them and we get a different set of capabilities12:43
kashyapaspiers: Noted12:44
aspierskashyap: if a compute node's libvirt/QEMU gets updated, it seems reasonable to expect that nova-compute will get restarted too12:44
aspiersor at least for major updates12:44
kashyapaspiers: Yes, not "reasonable", it should, to avoid any surprises :-)12:44
mriedemjaypipes: can you take a look at the stein regression bug fix series starting with the functional recreate test here? https://review.opendev.org/#/c/653268/12:45
aspierskashyap: qemu binary path will have a single value and is programmatically obtained12:45
aspiersand virt type will presumably be fixed within the context of the libvirt driver anyway12:46
aspiersso in reality it's just two values which can change12:46
mriedemgibi: can you take a look at the stein regression bug fix series starting here? https://review.opendev.org/#/c/653098/12:46
kashyapaspiers: True, good point.  (So you're aware of it; just thought I'd notify.)12:46
gibimriedem: looking12:46
aspiersunless it's possible to have the libvirt driver allowing both Xen and KVM on one host12:46
aspierswhich I assumed not12:46
aspiersor at least that noone would be crazy enough to try that ;-)12:46
kashyapaspiers: No, that's too FrankenStack to mix it like that :D12:47
aspiersright12:47
kashyapOkay, /me moves "office", home --> library; bbiab12:47
aspierskashyap: getting the emulator was the whole point of https://review.opendev.org/#/c/640483/12:48
sean-k-mooneyaspiers: what were you going to use the emulator for?12:51
aspiersfor calling the API as per above12:52
aspiersgotta run, biab12:52
sean-k-mooneyaspiers: you can only have kvm and zen on the same host if you run two agents. and are happy to support it entirly yourself because noone else will touch it  :)12:54
*** lbragstad has joined #openstack-nova12:55
sean-k-mooneykashyap: if you did not restat the agent i think we would pick you the capablity changes via the periodic task but ya it would be an ill defined state untill that happened12:56
kashyapRight12:57
mriedemmelwitt: about 5am i was thinking about quota and rebuild from cell0 - thoughts are in the spec https://review.opendev.org/#/c/648686/2/specs/train/approved/enable-rebuild-for-instances-in-cell0.rst@96 if you can check my thinking13:03
mriedemtssurya: is your locked_reason stuff ready for review? if so, can you queue it up for runways?13:03
*** eharney has quit IRC13:04
gibimriedem: what will happen if we run the migration, run archive, run migration, run archive again. Does that second archive fail to move the marker instance due to a unique constraint on the uuid field in the shadow table?13:04
openstackgerritNikolay Fedotov proposed openstack/nova master: Fix disappearing ironic hypervisors after rebuilding hashring  https://review.opendev.org/65458413:05
*** belmoreira has quit IRC13:07
gibimriedem: never mind, we dont add the unique constraint to the shadow tables13:07
gibimriedem: series starting at https://review.opendev.org/#/c/653098/ looks good to me13:07
mriedemright13:07
*** jaosorior has quit IRC13:08
mriedemthinking about your migration + archive thing, i don't think we archive the deleted marker instance because of the fkey relationship to the vif13:08
*** tbachman has quit IRC13:09
mriedemi also don't think we ever delete the marker vif even if we don't need to migrate any more records,13:09
mriedemso eventually when we remove that online data migration, we'll likely also need to delete the marker vif so archiving can remove it and we can remove the hack in the api13:10
*** belmoreira has joined #openstack-nova13:11
gibimriedem: hm, then I don't get how your functional test passes at https://review.opendev.org/#/c/653098/2/nova/tests/functional/regressions/test_bug_1825034.py@7713:13
gibimriedem: or we delete the marker instance but no the marker vif?13:13
mriedemcorrect13:13
mriedemthe instance record is soft deleted but the marker vif is not13:14
mriedemefried: i reckon anything going into a runway slot today will get an extra week because of the summit yeah?13:14
gibimriedem: I see13:14
mriedemmelwitt: i removed the counting quotas from placement series out of the runway slot since the time expired but feel free to queue it back up13:15
*** francoisp has joined #openstack-nova13:15
bauzasmriedem: sorry was on bank holiday yesterday, how can I help with some review ?13:17
openstackgerritLucian Petrut proposed openstack/nova master: Expose Hyper-V supported image types  https://review.opendev.org/65513713:17
mriedembauzas: stein regression series starting here would be good https://review.opendev.org/#/c/653268/13:18
mriedembauzas: and i need review on this queens backport https://review.opendev.org/#/c/640198/13:18
mriedemwe're very close to closing out pike13:18
mriedemextra credit for https://review.opendev.org/#/c/639382/ which is #2 in my series of 45+ changes in the cross-cell series13:19
mriedemgibi: could i also ask, when you get a min, to review this functional recreate test for a latent bug https://review.opendev.org/#/c/641521/13:20
gibimriedem: sure, looking13:20
*** lpetrut has joined #openstack-nova13:21
mriedemthanks13:22
*** panda|lunch is now known as panda13:23
*** ratailor_ has quit IRC13:25
gibimriedem: done, looks good13:28
mriedemthanks again, and i see you took the extra credit :)13:29
mdboothlyarwood: Just sent a thing on swap-volume to the ML which you might be interested in.13:30
gibimriedem: :)13:30
lyarwoodmdbooth: swap volume and interested should never appear in the same sentence13:31
* lyarwood looks13:32
*** phasespace has quit IRC13:33
mdboothlyarwood: I proposed a green field implemention :) You never know, it might even make you happy.13:33
mdboothlyarwood: Anyway, I wanted *something* written down before the PTG session just to ensure it has some structure. If we throw it away that's fine.13:34
tssuryamriedem: not yet I am still writing test cases and notification changes. Will add it to the queue once that is done13:34
lyarwoodmdbooth: ack thanks13:36
efriedmriedem: Extra runway time: sure13:38
mriedemmdbooth: without reading, if i wrote swap volume today i'd probably use the os-server-external-events API as the callback from cinder like we use for neutron and cinder's extend volume api13:38
efriedkashyap: ack, thx13:38
efriedlbragstad: Yup, got it figured, thanks.13:38
efriedeandersson: ack, will look. Either way would be fine. I was going to say this is temporary until we rip out ironicclient; but the region_name and min/max fixes should be backported, so the stable branches will carry that code for a while.13:40
efriedmriedem: ack, saw the test.py thing, +2, good call.13:40
*** mlavalle has joined #openstack-nova13:43
mdboothmriedem: My proposal wasn't explicit about what rpc method was used, but it is explicit about synchronous guarantees.13:43
*** liuyulong has joined #openstack-nova13:44
mdboothmriedem: If we can use something more appropriate to the task than the rest api whilst having explicit calls and waiting between cinder and nova that would be awesome.13:44
mdboothI don't think it's possible to do with 1-way asynchronous calls, though, and 2-way asynchronous calls would be more complex.13:45
*** tbachman has joined #openstack-nova13:46
*** psachin has joined #openstack-nova13:58
*** bryan_stephenson has joined #openstack-nova13:58
*** boxiang has quit IRC13:59
*** boxiang has joined #openstack-nova14:00
*** jaosorior has joined #openstack-nova14:00
mdboothmriedem: Thanks, btw. I'd completely forgotten to discuss rpc mechanisms in that mail. I just tacked it on.14:01
mriedemmdbooth: i wasn't talking about rpc, i was talking about rest api14:03
mdboothmriedem: I was talking about 'rpc', where the rest api is a type of rpc14:04
mriedemmdbooth: do you want a new dedicated swapVolume server action that is synchronous to cinder?14:04
mdboothmriedem: It's described in the ML post, but basically there would be 2 separate calls and cinder would do the bulk of the copy itself.14:05
*** yan0s has joined #openstack-nova14:06
*** Luzi has quit IRC14:06
mdboothNote that the currently implementation is essentially synchronous as it has a callback, btw. Except when it doesn't :/14:09
openstackgerritChristian Berendt proposed openstack/nova stable/rocky: Fix regression in glance client call  https://review.opendev.org/65516714:09
*** sridharg has quit IRC14:17
kashyapefried: sean-k-mooney: A "traits" question: If I were to add a 'COMPUTE_UEFI_SECURE_BOOT' trait, to indicate the host QEMU+libvirt+OVMF is capable of Secure Boot support, what existing similar example would you suggest to look at, in the Nova code?14:19
mriedemkashyap: look at the capabilities stuff the drivers report as traits14:20
mriedemand what dansmith is doing with supported image types14:20
efriedmriedem has... yeah14:20
*** eharney has joined #openstack-nova14:20
mriedemhttps://review.opendev.org/#/c/652710/14:20
kashyapmriedem: Ah, yes.  I took a cursory look at Dan's image_types.  /me looks14:20
kashyapThanks, both14:21
efriedkashyap: also https://review.opendev.org/#/c/645316/14:22
kashyapefried: Yep, bookmarked & reading.  Thanks for the (non-null) pointers14:22
kashyapefried: mriedem: Oh, speaking of traits, it reminds of something I noticed the other day: e.g. I see the CPU_TRAITS_MAPPING work that alex_xu and co. did in the past14:25
kashyap...I see that we're missing some useful and important CPU traits like PCID feature, etc.14:26
sean-k-mooneykashyap: i propsoed that in the past. https://review.opendev.org/#/c/514713/4/os_traits/fw/uefi.py14:26
kashyapIs there a "recommended" way to systematically add new / missing traits?14:26
sean-k-mooneyalthough it was a slightly different usecase14:26
kashyapsean-k-mooney: Right, I just saw the remark on your (abandoned) change14:27
mriedemkashyap: propose them to the os-traits library14:27
kashyapefried: Or is it just a matter of: "go, simply add the missing trait"?14:27
mriedemlpetrut: was asking the same yesterday14:27
sean-k-mooneyif COMPUTE_UEFI_SECURE_BOOT means this host is capable of virtualising a vm with secure boot i think its fine14:27
kashyapYeah, so far I was "ignoring" traits thingie.  Now I can no longer :D14:27
efriedkashyap: Yeah, I don't think we want a strategy of "add all these traits that might ever be relevant".14:28
kashyapmriedem: Thanks!  This one: https://github.com/openstack/os-traits14:28
sean-k-mooneywhat i was orginally trying to model was this host host secure boot ebabled14:28
sean-k-mooneythat is what the pushback was reated too14:28
efriedRather, we want to add as needed.14:28
aspiersefried: thanks for the SEV spec review, just replying now14:28
efriedaspiers: ack14:28
*** obre has quit IRC14:28
kashyapefried: Not "might".  E.g. a practical example: if you applied all the Meltdown/Spectre fixes, then your guests are suddenly awfully w.r.t performance (while you may get security)14:29
*** obre has joined #openstack-nova14:29
mriedemlyarwood: per your email about extracted placement + tripleo, does that mean train tripleo ci is using extracted placement in base deploys?14:29
sean-k-mooneykashyap: os-traits uses story board so you will want to creat a story for the new triat14:29
kashyapefried: ... unless you specify PCID.  (https://kashyapc.fedorapeople.org/Reducing-OpenStack-Guest-Perf-Impact-from-Meltdown.txt)14:29
efriedkashyap: If you have a use case that encompasses some traits, that counts.14:29
kashyapsean-k-mooney: Sigh, it seems more red tape, why a "storyboard" ticket for such a simple thing?  A commit message should be able to handle it14:30
*** yan0s has quit IRC14:30
efriedkashyap, sean-k-mooney: I agree.14:30
efriedsorry14:30
efriedI agree that we shouldn't need a story for new traits.14:30
kashyapefried: I'm only looking at CPU traits that are practically important / relevant.14:30
efriedunless there's some massive architectural thing going on.14:30
kashyapsean-k-mooney: Let's not sleep-walk into mind-numbing process.  It shoos away more contributors.14:30
efriedYou can expect some bikeshedding on namespaces and whatnot, but all of that can happen in review.14:31
sean-k-mooneykashyap: im not14:31
sean-k-mooneyonce added a trait can never be remvoed14:31
kashyapefried: Yeah, that's fine.14:31
sean-k-mooneyso i just wanted it to not slip between the cracks14:31
efriedbut kashyap, note that storyboard in this sense would be more akin to bugs.launchpad, not blueprints.launchpad.14:31
kashyapsean-k-mooney: The merits / demerits of a trait can be discussed in the review.  If there is consensus that it should not be added, any reasonable person will concur.14:32
kashyapefried: Yep, I've filed a few 'stories' myself :-)14:32
kashyap(For the infra project, though)14:32
efriedI'm one for less process anywhere possible14:32
efriedkashyap: btw, in case your reference to the github repo was meaningful, os-traits is developed in gerrit14:33
efriedso like don't go submitting a pull request.14:33
kashyapYep, figured as much.14:33
efriedcool14:33
kashyapHehe, I by default assume all these are Gerrit-based.14:34
jaypipesmriedem: yessir14:34
openstackgerritsean mooney proposed openstack/nova master: [DNM] AUTOPEP8 ignore  https://review.opendev.org/65517114:37
*** artom has joined #openstack-nova14:39
jaypipesefried: can you link me the review I said I'd re-look at that I responded "yessir" to you last evening? my scrollback doesn't have it (and no, I don't log irc)14:39
jaypipesefried: tia14:40
lyarwoodmriedem: sorry missed your ping, yes AFAIK it's the default in CI now14:40
efriedjaypipes: It was aspiers' SEV reproposal: https://review.opendev.org/64199414:40
jaypipesdanke14:40
efriedjaypipes: he said he's responding to my last couple of comments now.14:40
jaypipesk14:40
*** lpetrut has quit IRC14:40
kashyapjaypipes: Will answer your question shortly on the review.  (And interesting pointer on "Shielded VMs", I guess that's the nail on the "use case coffin" :D)14:41
kashyapjaypipes: Also on "stupid questions"...recall what the inimitable Carl Sagan said:14:42
kashyap"There are naive questions, tedious questions, ill-phrased questions, questions put after inadequate self-criticism. But every question is a cry to understand the world. There is no such thing as a dumb question."—Carl Sagan14:42
*** udesale has quit IRC14:45
*** udesale has joined #openstack-nova14:46
*** ccamacho has quit IRC14:46
jaypipeskashyap: the complete quote ends with "unless you're speaking with Jay Pipes. If that's the case, yes, there is definitely such a thing as a stupid question."14:48
kashyapHehe, go easy on yourself14:48
jaypipesefried: k. to be clear, have all of danpb's concerns been addressed?14:49
efriedjaypipes: I definitely want to get a re-ack from danpb before we merge the spec. My review was only in terms of the placement- and nova-isms.14:49
efriedaspiers: Any progress on contacting danpb to sign off?14:50
efriedjaypipes: Of course you should dig into the SEV-specific gorp as you are able/willing, but I mainly wanted to make sure you were on board with the switchover to quantitative inventorying of SEV context.14:52
jaypipesis there a danpb bat-signal?14:52
efried"we're going to rewrite nova"14:52
efried^ that should do it14:52
jaypipesefried: ack, lemme focus on those changes.14:52
jaypipesefried: lol14:52
jaypipesefried: danpb long ago gave up the nova ghost.14:53
*** luksky has quit IRC14:54
efriedeandersson: I want to backport your region_name fix, so just need a bug associated with it and I think we're good to go.14:54
kashyapjaypipes: Hehe, from what I see Dan is largely back to working on libvirt/QEMU14:55
kashyap(But he's still on #virt, and #qemu OFTC.  That's where I ask some gnarly questions.)14:55
*** lpetrut has joined #openstack-nova14:56
* efried bbiab14:59
*** tetsuro has joined #openstack-nova14:59
*** tetsuro has quit IRC15:01
kashyapefried: jaypipes: On the SEV thing, DanPB just said: if you stopped using 'hard limit' (for memory) then the important bit is addressed.15:04
jaypipesack15:07
kashyapartom: BTW, I spent 2-ish hours this morning responding to your (and cfriesen's) comments on the Secure Boot spec.  Thanks for your time.  When you can, let me know (on the review) if I answered your questions15:09
*** gyee has joined #openstack-nova15:09
artomkashyap, yeah, saw your responses in an email, will need to circle back...15:10
* artom needs to "methodicize" his upstream work15:10
kashyapWe should fast-track artom into management, he's using the right jargon: "circle back" :D15:10
artomFor instance, "Tuesday and Thursday afternoons (or whatever) are for upstream"15:10
artomkashyap, I'm a freaking buzzword bingo generator15:11
mnaseryou haven't gone full management till you start "redlining" specs instead of reviewing them15:11
mnaser:P15:11
artomI take synergies and value-add process efficiencies.15:11
kashyapartom: On the off chance, wonder if you've seen Urban Dictionary's definition of "circle back".15:11
artomkashyap, oh, I'm sure it's unspeakable in this channel15:12
kashyapartom: Not quite.  But it makes the point well15:12
aspiershttps://www.urbandictionary.com/define.php?term=Circle%20Back15:12
aspierspretty funny, and surprisingly SFW15:12
artomYeah, I was fully expecting "back" to be, err, backdoor stuff15:13
kashyapartom, Back to work, on your main N->N-1 migration and SMM XML bit: good news!  It is a non-problem for us.  See my answer on the change15:13
kashyapaspiers: Heh, yeah15:13
mriedemlyarwood: fyi apparently https://review.opendev.org/#/q/I3ed3303309fe2a25c0043fd206f36bada4b3b8f9 broke ceph-backed snapshots15:14
artomkashyap, because we're guaranteed to have a libvirt/qemu that support it, correct?15:14
mriedemhttps://review.opendev.org/#/c/655167/15:14
lyarwoodmriedem: looking15:14
kashyapartom: Yes15:14
mriedemsince the ceph job is non-voting i didn't notice the failure in your backports15:14
kashyapartom: The min libvirt/QEMU versions in Nova are way beyond what's required.15:14
mriedembut indeed the ceph job failed with snapshot tests15:14
artomkashyap, OK, might be worth writing down in the spec. Unless I15:14
artomUnless I'm really the only confused lost soul15:14
kashyapartom: To put things in perspective, first commit to QEMU on SMM support was in 2006 sometime (I was still in college) :D15:15
kashyapartom: Yeah, I've added the versions required already in the spec.15:15
artomkashyap, ack.15:15
lyarwoodmriedem: crap, ack.15:15
mriedemjust fyi since you might hear about it internally :)15:16
mriedemmaybe we should put some effort behind making the ceph job non-voting15:16
mriedemi'm not really sure why it's non-voting now anyway, besides maybe a history of intermittent failures a few releases ago15:16
kashyapartom: Lastly: min QEMU version that has the relevant SB-related features libvirt looks for is 2.4.  (Current Nova min versoina take satisfy those too.)15:17
kashyaps/take//15:17
*** ccamacho has joined #openstack-nova15:17
lyarwoodmriedem: yeah, happy to help keep it GREEN during Train and on stable if it means we avoid issues like this tbh.15:18
kashyapmriedem: When you get a sec, so if I add a CPU flag 'trait' to `os-traits` library, then I should also add it to CPU_TRAITS_MAPPING dict (https://github.com/openstack/nova/blob/master/nova/virt/libvirt/utils.py#L49,#L81)15:19
* kashyap looks about15:20
mriedemthe cpu flag name is literally 'trait'?15:21
*** lpetrut has quit IRC15:22
mriedemif you want the cpu feature flag to show up as a trait on the compute node resource provider automatically then yeah i guess it goes in that mapping15:23
mriedemalex_xu is probably the authority on that code15:23
alex_xuo/15:23
sean-k-mooneykashyap: is this realted to secure boot15:23
kashyapsean-k-mooney: No-no, completely unrelated15:23
sean-k-mooneyok i was going to say that is not a cpu flag15:24
sean-k-mooneynever mind15:24
kashyapsean-k-mooney: I'm adding PCID, SSBD, etc.15:24
sean-k-mooneyah ok15:24
kashyapOf course, it's not.  Huh :-)15:24
alex_xukashyap: if hope the nova to discover the PCID and report PCID trait to the placement, then you should add it to CPU_TRAITS_MAPPING15:25
kashyapalex_xu: Yes, exactly.  Thought so15:25
alex_xuI guess you also need to raise the required version of os-traits15:25
kashyapalex_xu: So, along with adding to /os-traits/os_traits/hw/cpu/x86.py, I will add it to the CPU_TRAITS_MAPPING dict15:25
kashyap(Yes?)15:26
alex_xuyes15:26
kashyapalex_xu: Where do I raise the 'os-traits' version?15:26
*** ttsiouts has quit IRC15:27
alex_xurequirement15:27
*** ttsiouts has joined #openstack-nova15:27
alex_xukashyap: requirement.txt15:28
openstackgerritChristian Berendt proposed openstack/nova stable/queens: Fix regression in glance client call  https://review.opendev.org/65518615:28
kashyapalex_xu: Thanks!15:30
aspiersefried: after much deliberating I've finally figured out how to articulate my thoughts https://review.opendev.org/#/c/641994/1115:30
kashyapalex_xu: I see currently 0.9.0 traits is released.  And in Nova, we have: os-traits>=0.8.015:31
kashyapalex_xu: I am going to make that to:15:31
kashyap- os-traits>=0.8.015:31
kashyap+ os-traits>=0.10.015:31
alex_xuyea, to the version has PCID trait15:31
aspierskashyap: thanks for relaying the message from danpb - indeed hard_limit is no longer proposed in the latest version of the SEV spec http://logs.openstack.org/94/641994/11/check/openstack-tox-docs/a439f50/html/specs/train/approved/amd-sev-libvirt-support.html#memory-locking-and-accounting15:31
*** ttsiouts has quit IRC15:32
kashyapaspiers: Noted.  Thanks for the pointer15:33
kashyapaspiers: BTW, I must say, this is one of the most well-written specs I've evern seen.  Nice work.15:33
aspierskashyap: thanks! :)15:33
aspiersI received a lot of great advice on it over 2 cycles, which has helped hugely15:35
*** mvkr has quit IRC15:35
kashyapYeah, it shows the careful, unhurried effort to think-through the problem and incorporating the feedback.15:35
aspiersWell I'm too stubborn to give up, and not quite stupid enough to believe that I could shove a spec down your throats before it was ready and expect to succeed ;-)15:37
*** dustinc_away is now known as dustinc15:37
openstackgerritBalazs Gibizer proposed openstack/nova master: nova-manage: heal port allocations  https://review.opendev.org/63795515:37
*** helenafm has quit IRC15:37
*** jamesdenton has joined #openstack-nova15:38
kashyapalex_xu: By the way, the CPU_TRAITS_MAPPING takes both AMD _and_ Intel flags, yes?15:40
alex_xuyes15:41
kashyapaspiers: Hehe, noted.  (Sustained concentration is a "feature" in this distracted world with 280-char attention spans.)15:42
kashyapalex_xu: Thx.15:42
aspiersIndeed15:42
* artom hopes he'll receive kashyap's feature patch soon.15:43
artomAlthough realistically, that'll probably mean drugs, so maybe not?15:43
kashyapWhat a charitable interpretation.15:43
aspiersLOL15:43
artomIs it? Charitable, I mean?15:44
aspiersIt's not a coincidence that I'm currently doing a meditation course on productivity ;-)15:44
kashyapalex_xu: As I add these missing traits to the CPU_TRAITS_MAPPING, I have an uneasy feeling that we're duplicating work15:44
kashyapalex_xu: I think we're OK with its growth over the years...15:44
kashyapaspiers: Nice.15:45
alex_xukashyap: yes, welcome any better idea15:45
kashyapalex_xu: Nothing currently :-)  Will keep thinking about this problem, will let you know if I come up with anything better15:46
alex_xukashyap: cool, thanks15:46
*** ccamacho has quit IRC15:46
*** ccamacho has joined #openstack-nova15:47
mriedemadrianc: some comments in https://review.opendev.org/#/c/649345/15:50
mriedemadrianc: if you agree, i'm willing to make that change and then still +215:51
mriedemit's just docs and variable naming15:52
*** dave-mccowan has joined #openstack-nova15:57
efrieddansmith, mriedem, gibi: Did we ever settle on the accepted way to add resource/trait/aggregate things via a request_filter so they end up in the GET /a_c request?15:58
*** bryan_stephenson has quit IRC15:59
mriedemnot that i know of, i know gibi posted a patch,16:00
mriedemand my eyes glazed over at the discussion within my multiattach capability trait patch16:00
mriedemfor now i'm just shoving a trait in the flavor extra spec on the request spec before it hits the scheduler and making sure the change isn't persisted16:00
mriedemi.e. hitching a ride to scheduler town16:00
mriedemcash, grass, or ...16:01
mriedemno traits ride for free16:01
efriedmaking sure the change isn't persisted?16:02
mriedemRequestSpec.flavor gets persisted16:03
efriedso you're adding it to the flavor extra spec until the GET /a_c and then removing it?16:03
mriedemi want to make sure that twiddling RequestSpec.flavor.extra_specs for a one-time scheduler run doesn't get persisted16:03
mriedemnothing is explicitly removing it,16:03
efried...you just aren't saving it16:03
mriedemyou reset changes on the object so that when it's saved those changes aren't persisted16:03
efriedokay16:03
*** belmoreira has quit IRC16:03
*** dikonoor has quit IRC16:05
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Delete approved template in move_implemented_specs  https://review.opendev.org/59275516:05
*** dave-mccowan has quit IRC16:07
jaypipesmdbooth: what is the primary use case for the volume-update operation? (as opposed to just doing a detach-volume, start-new-instance-elsewhere, attach-volume flow?16:08
mriedemvolume live migration and retypes16:10
mriedemfor a retype it's a new volume, else it's the same volume on a different backend with the same type (i think)16:11
mriedemhttps://docs.openstack.org/nova/latest/user/support-matrix.html#operation_swap_volume16:11
jaypipesmriedem: what's the use case though?16:13
aspiersjaypipes wins the Review Of The Week award for a perfect use of a Big Lebowski quote16:13
jaypipesaspiers: yw16:13
mriedemjaypipes: this feels like a trap16:14
mriedemwhat's the use case for live migration?16:14
jaypipesmriedem: no, I'm honestly curious.16:14
jaypipesmriedem: is volume live migration different from live migration?16:14
mriedemyes16:14
mriedemit's initiated through cinder16:14
mriedemsmcginnis: maybe you want to elaborate here ^16:14
jaypipesmriedem: so this is not an operation a user would ever issue, right?16:15
mriedemhttps://docs.openstack.org/cinder/latest/admin/blockstorage-volume-migration.html16:15
jaypipes(like instance live migration is not an operation any normal user even knows about)16:15
mriedemfor volume migration i think that is correct, it's admin-only by default,16:15
mriedembut retype is admin or owner of the volume aka the user16:15
mriedemhttps://developer.openstack.org/api-ref/block-storage/v3/?expanded=migrate-a-volume-detail,retype-a-volume-detail#retype-a-volume16:15
mriedem"Change the volume type of existing volume, Cinder may migrate the volume to proper volume host according to the new volume type."16:16
mriedemi think of volume retype like server resize16:16
mriedemby default retyping should not migrate the volume, but the user can override that16:16
mriedem"If the volume is attached to a server instance and will be migrated, then by default policy only users with the administrative role should attempt the retype operation."16:17
mriedem^ is because the nova swap volume api is admin-only by default16:17
mriedemit's f'ing great16:17
jaypipeshrmph16:17
mriedemso i as a non-admin user could try to retype my attached volume and migrate it which could then blow up b/c nova rejects my non-admin user token passed by proxy from cinder16:18
aspiersefried: when you have a moment, if you can give me some newb pointers about how encrypted_memory=true would be implemented, I can submit a new patch set proposing that16:18
mriedemnot sure if cinder has added anything to escalate that token to admin/service user in the last 3 years16:18
mriedemthe more sane thing to do would probably be stop the instance, detach the volume, retype it, re-attach and then restart the server16:19
jaypipesor design your application to write to multiple volumes in a replica set instead of one giant panda volume.16:20
jaypipesbut I digress.16:20
mriedemmy enterprise panda volume is very important and can never go down16:20
mriedemit runs my website visit counter16:20
jaypipes:)16:20
jaypipesnew band name: Enterprise Panda Volume.16:20
jaypipesor EPV for the cool kids.16:21
mriedemEPV if you're talking to execs16:21
mriedemyeah :)16:21
pandauh ?16:21
jaypipes:)16:21
mnaserLOL16:21
jaypipeshahahaha16:21
aspiersthere is actually a panda on this channel who is gonna be wondering why they suddenly got a bunch of notifications :-D16:21
jaypipespanda: :)16:21
aspiersoh, I was too late16:21
pandacan I get back to sleep ?16:21
panda:)16:21
aspiers:)16:21
jaypipespanda: sorry, a "Panda" is a pet virtual machine that can not be killed otherwise it's an international incident :)16:21
jaypipessorry for waking you!16:22
pandaoh my.16:22
mriedemoh it also looks like swap volume will work on root volumes, whereas detach / attach does not (yet)16:23
jaypipesmriedem: yuck.16:23
*** mrhillsman is now known as openlab16:23
jaypipes:P16:23
pandanice to know more nice uses of my nick :)16:23
*** openlab is now known as mrhillsman16:23
aspiersjaypipes: I guess I need to update my slides https://youtu.be/uMCMDF9VkYk?t=48316:23
jaypipesaspiers: yes. yes you do.16:24
*** mrhillsman is now known as openlab16:24
*** wwriverrat has joined #openstack-nova16:24
jaypipesaspiers: and add a Big Lebowski reference while you're at it.16:24
aspiersgood plan16:24
*** openlab is now known as mrhillsman16:25
*** dtantsur is now known as dtantsur|afk16:28
*** itlinux has joined #openstack-nova16:30
*** whoami-rajat has quit IRC16:30
*** rcernin has quit IRC16:36
*** whoami-rajat has joined #openstack-nova16:39
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Add HW_CPU_X86_* traits for Meltdown/Spectre mitigation  https://review.opendev.org/65519116:40
openstackgerritMerged openstack/nova-specs master: Delete approved template in move_implemented_specs  https://review.opendev.org/59275516:42
openstackgerritKashyap Chamarthy proposed openstack/os-traits master: Add CPU traits for Meltdown/Spectre mitigation  https://review.opendev.org/65519316:45
*** tosky has quit IRC16:46
kashyapAny "Traits People", please let me know if I got the above even partly right...16:46
* kashyap bbiab16:46
*** udesale has quit IRC16:47
efriedaspiers: I was just doing that in the review when I received an interrupt, getting back to it now.16:47
aspiersefried: if it's easier you can just educate me here16:47
sean-k-mooneykashyap: do we need seperate VIRT-SSDB and AMD-SSDB  traits16:48
aspiersefried: since I'll probably have questions which the extra round trip latency from Gerrit won't help16:48
efriedaspiers: okay, sure. One minute...16:48
kashyapsean-k-mooney: Yes, we do.16:48
sean-k-mooneywhy?16:48
*** eharney has quit IRC16:48
kashyapsean-k-mooney: Let me quote the upstream QEMU documentation:16:49
kashyap[quote]16:49
kashyapvirt-ssbd16:49
kashyapRequired to enable the CVE-2018-3639 fix. Not included by default in any AMD CPU model. Must be explicitly turned on for all AMD CPU models. This should be provided to guests, even if amd-ssbd is also provided, for maximum guest compatibility. Note for some QEMU / libvirt versions, this must be force enabled when when using “Host model”, because this is a virtual feature that doesn’t exist16:49
kashyapin the physical host CPUs.16:49
kashyap[/quote]16:49
kashyapNote the: *even if amd-ssbd- is also provided* part.16:49
kashyapsean-k-mooney: They are separate CPU flags, so of course we need separate traits.16:49
sean-k-mooneywell we dont that is basically my point16:49
kashyapOh?16:50
kashyapsean-k-mooney: What do you mean?16:50
sean-k-mooneythere does not need to be a 1:1 mapping between tratis adn cpus flags16:50
*** altlogbot_2 has quit IRC16:50
sean-k-mooneythe traits dont impact the xml generation16:50
sean-k-mooneyso for the tratis we need to think about whgat we schdule on16:50
kashyapPlease comment in the review; I need to leave the library.  It's closing16:50
sean-k-mooneysure will do16:51
kashyapMuch appreciated16:51
artomWhat's the ? at the beginning of https://github.com/openstack/nova/blob/95e782dfd86caa4201d28ee86ba2bb475e0a409f/devstack/tempest-dsvm-lvm-rc#L33 ?16:51
artom'r="^(?!.*"'16:51
artomI thought I groked regex, but this confuses me.16:51
artom? would mean 0 or 1, right? So... "maybe a (" ?16:52
* efried puts on regex cape16:52
artomBut... why?16:52
aspiersthat's a negative look-ahead assertion16:52
aspiers(?!foo.*bar) means "foo.*bar can't come next"16:53
efried..."but don't capture it"16:53
aspiersright16:53
aspiersit's one of the more obscure features of pcre, been around since Perl 5 days though16:53
artomAh, I was reading it as a quantifier16:53
artomThanks aspiers16:53
aspiershttps://xkcd.com/208/ is the obvious reference here16:53
artom*snerk*16:54
* aspiers shows his age16:54
artomMore like https://xkcd.com/1171/16:54
aspiershahahaha16:54
aspiersthat's good16:54
efriedYe gods, my goats are freaking out. aspiers, back atcha in 5...16:55
* aspiers . o O ( goats?! )16:55
efriedmeanwhile, digest this:16:55
efriedOkay, so what I had been thinking thus far was that you would be implementing a request filter method here:16:55
efriedhttps://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py16:55
efriedThose methods get the RequestSpec, which has the image and flavor in it, so you can grab encrypt_mem=true from either/both.16:55
*** altlogbot_1 has joined #openstack-nova16:55
artomOr the well known "Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems. "16:56
aspiersefried: ahah! yep, makes sense16:56
artomefried, you confuse me. A Francophone American BJJ practitioner (right? I seem to remember that from somewhere) with goats.16:56
*** derekh has quit IRC16:57
aspiersartom: my Perl regexp knowledge comes from the same era as my personal website https://twitter.com/adamspiers/status/111592935073661337616:59
aspiersI should try not to think too much about how long ago that is16:59
artomYour Twitter picture doesn't fit your website's copyright notice :)17:02
artomUnless you were, like, 10 when you made it17:02
artom(The website, not the picture)17:02
* artom wonders17:03
artomOh god it's still out there: http://web.archive.org/web/20040611101606/http://coth.no-ip.org/17:03
artomWhatever pride I have left just went out the window17:03
stephenfinefried: Spent the day working on summit stuff but I'll create the nova-console removal (specless) BP tomorrow, plus more for nova-consoleauth removal (I've that series drafted already). Hit me if I don't17:03
artomI wasn't even that your when I made that17:03
artom*that young17:04
stephenfin(though not really. I imagine that would hurt)17:04
artomLike, 1917:04
aspiershaha17:06
aspiersartom: the internet is a cache of dirty secrets17:06
aspiersartom: I was 20 in 199517:07
artomSo we're literally the same age, what are you feeling old for?17:07
artomWait, no, I fail at math17:07
artomI was 1017:07
aspiersLOL17:07
artomOh god, more of my "work": http://web.archive.org/web/20021010075531/http://geocities.com/the_lambda_place/17:08
* stephenfin was three17:08
* stephenfin shuffles out17:08
artomYou're grounded, go to your room17:08
aspiershahahaha17:09
aspiersartom: looks disturbingly similar to https://www.adamspiers.org/computing/quake/17:09
aspiersI think that deserves to be labelled as "Web 0.2"17:09
artomaspiers, well now adze has to be your Friday nick17:10
efriedcatching up (in reverse order)17:10
*** ivve has quit IRC17:10
aspiersnooooo17:10
*** nicolasbock has quit IRC17:10
aspiersefried: best if you skip the last 10 minutes ;-)17:10
efriedI'm the same age as artom, give or take a year.17:10
efriedstephenfin: ack, thanks17:10
efriedI used to be a perl wiz, actually taught a class at one time17:10
efriedand artom yes, all of those things. And http://www.thunderpony.com and I was also a child actor :P17:11
efriedand regular expressions are awesome.17:11
efriednow, back to request filters...17:11
artomefried, damn, you sing as well17:12
artomOh, right, the thing I'm actually paid to do17:12
* artom goes back to test regexes17:12
efriedaspiers: So the RequestSpec is how you find out if your magic key=val is in the flavor and/or image.17:13
efriedTthe jury is still out as to what happens next.17:13
aspiersartom: here's some fun with regexes for you https://adamspiers.org/computing/perl_signatures.html17:13
efriedmriedem and I were discussing that about 1h15m ago ^^17:14
aspiersefried: and that would automatically work for both extra specs and image properties?17:14
aspiersefried: thanks, scrolling up 75m17:14
efriedaspiers: well, it would work for both if you implemented both.17:14
efriedyou have access to both because they're in the RequestSpec.17:14
artomaspiers, yeah, no :P17:14
efriedaspiers: I knew a guy who copy/pasted one of those obfuscated perl regex commands into his interpreter. It was one of those //ee things, and translated to something like rm -rf /17:15
aspiersefried: OK. I guess this will start making a lot more sense as I start reading that code and hacking on it17:15
aspiersefried: Ouch, that's a more sinister case of the "curl ... | sudo bash" syndrome17:16
*** luksky has joined #openstack-nova17:16
aspiersalthough I promise that all of those .sigs are totally innocent and safe to run17:16
aspiersWell the top one still works, at least17:17
efriedaspiers: Detecting the property from the request spec is going to be the easy part. The thing mriedem and I were talking about stemmed from his patch here https://review.opendev.org/#/c/645316/ which does a very similar thing to what you're wanting to do in terms of adding stuff that'll make its way into the placement request.17:17
* aspiers looks17:17
efriedtl;dr: one way to do it would be to stuff the placement-ese translation of your request - so like a resource request for 1 MEM_ENC_CONTEXT - into the actual RequestSpec.flavor.extra_specs, and make sure that that change to the flavor does *not* get persisted.17:18
efriedHowever, longer term, we're going to want to do such things by stuffing them into RequestSpec.requested_resources: https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L93-L10017:20
*** rpittau is now known as rpittau|afk17:20
efriedbut as hinted by that TODO, we're only using that for bandwidth resources at the moment.17:20
aspiersok17:21
aspiersI guess this can all be done as one of the later work items in the spec, right? Since it's more about improving the UX than actually enabling SEV17:22
efriedaspiers: Well, you don't need to talk about anywhere near this level of detail in the spec17:22
efriedI'm just showing you that it's definitely doable to support the same generic property (encrypt_my_memory=true) in the flavor and image props17:23
efriedand get it translated to placement-ese by "nova".17:23
efriedSo the UX is encrypt_my_memory=true, done.17:23
aspiersefried: but without that detail how will I pip generic-resource-pools.rst to the post for the largest spec ever? ;)17:23
efriedAdd a seqdiag17:24
aspiersbut seriously, yup - it was always a goal of the spec to allow booting SEV via image properties17:24
aspiersohhhhh nice idea17:24
efriedhttp://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/reshape-provider-tree.html17:24
efriedthat seqdiag is only ~38L source :(17:25
aspiers:D17:26
efriedoh, I've got a better idea17:26
efriedascii diagrams of nested provider trees17:26
efriedthose take up some good vertical space.17:26
jaypipesI'm sure those make cdent shiver.17:27
aspiersI was thinking more in terms of the file size17:27
aspiersI'm about 400 bytes behind the leader ;-)17:28
efriedoh, you're going byte size?17:28
efriedI thought you were going #lines17:28
*** psachin has quit IRC17:28
aspierswell the ranking is the same regardless17:29
efriedone of my earlier specs, don't remember which one, dansmith literally -1'd it because it was too long. I distinctly remember trimming it by 10% to earn his +2.17:29
aspiershaha17:29
mriedemgmann: if i'm defining a new ci job which extends tempest-multinode-full-py3 and modify the devstack_localrc vars from that job, does it merge the vars or do i need to do a full replace with what i want?17:29
dansmithpretty sure that's exactly how that went down, no embellishment at all17:29
aspiersI never intended SEV to be so long, just kept getting feedback on new angles17:30
efriedaspiers: I think it was this one: https://review.opendev.org/#/c/510244/17:31
efriedhah, yup http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2017-10-17.log.html#t2017-10-17T15:21:53  <== dansmith17:33
aspiersefried: just looked at https://review.opendev.org/#/c/645316/2/nova/compute/api.py - IIUC I could just add an invocation of a new _modify_request_spec_for_encrypted_memory() to _provision_instances() which would apply the translation to the flavor extra specs or image properties?17:41
mriedemheh n-obj is still enabled by default in devstack17:41
mriedemderp17:41
openstackgerritArtom Lifshitz proposed openstack/nova master: Run revert resize tests in nova-live-migration  https://review.opendev.org/65349817:43
openstackgerritArtom Lifshitz proposed openstack/nova master: Revert "Wait for network-vif-plugged on resize revert"  https://review.opendev.org/63939617:43
openstackgerritArtom Lifshitz proposed openstack/nova master: Revert resize: wait for external events in compute manager  https://review.opendev.org/64488117:43
efriedaspiers: more or less. I think this would be better positioned as a method in request_filter.py personally, but that's details. The thing that you'll be needing that isn't applicable in --^ is how you determine whether to add the thing.17:43
aspiersefried: so the new method would frob either reqspec.flavor.extra_specs or reqspec.image?17:43
efriedyes17:43
aspiersahah OK, request_filter.py looks pretty trivial to extend17:44
efriedaspiers: But that code path has access to the reqspec too, so whatevs.17:44
aspiersright17:44
efriedyes, in fact I should -1 mriedem's patch for that.17:44
aspiersI was going to ask, why didn't https://review.opendev.org/#/c/645316/2/nova/compute/api.py extend request_filter.py :)17:44
efriedwell, I was going to say it's because the logic of how he's determining whether to add the filter comes from someplace that's not easy to get at from request_filter.py17:45
mriedemefried: that is the rason17:45
mriedem*reason17:45
aspiersahah, right17:45
mriedemthe multiattach info is on the bdm, and even then only after it's connected17:45
aspiersbut in my case all I need is the request_spec17:46
efriedyup17:46
mriedemso during server create, the request filter would have to loop the bdms looking for the volume_id, then query cinder to get the volume multiattach flag17:46
mriedemwhich is pretty shitty for performance during scheduling17:46
efriedand since this is all about performance during scheduling...17:46
aspiersOk awesome, this is starting to make a lot of sense17:46
mriedemright17:46
aspiersefried: tweaking the spec now17:46
mriedemwell, it's about performance and not picking a compute that doesn't support multiattach17:46
efriedaspiers: Hold on a tick, I'm answering your other concern too.17:47
aspiersefried: ok17:47
mriedemif we stored a multiattach attribute on the bdm object and stored something about those in the request spec it'd be a different story17:47
mriedembut that's a lot of ifs17:47
tssuryagibi: if I bump the version on the instance_payload do I need to manually also bump the versions on all payloads inheriting from the instance_payload ?17:51
efriedaspiers: responded. And hanging out if you have questions.17:53
aspiersefried: thanks!17:54
*** ccamacho has quit IRC17:54
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add nova-multi-cell job  https://review.opendev.org/65522217:54
mriedemdansmith: let's cross our fingers and toes ^17:54
dansmithhoo boy17:54
efriedaspiers: Also note that jaypipes responded.17:55
aspiersyep, already saw that17:55
efriedjaypipes: Are you okay with "unlimited" being expressed as MAXINT inventory?17:55
efriedor, heh, an allocation_ratio of 1E9999 :P17:58
jaypipesefried: there is no such thing as an unlimited inventory. it there was, it would be a trait, because it isn't quantitative/consumable.17:59
aspiersefried: "It'll be up to the driver to determine what that inventory should look like and what it's based on, knowing that the request will always be for 1 unit." - my point was that maybe we *don't* know that17:59
aspiersefried: e.g. picking a stupid example, what if MKTME counted it per vcpu rather than per host?18:00
efriedjaypipes: rightright, this is a case where there's always a limit, but sometimes we don't know exactly what it is, so we need to let it be imposed by the host (not by placement). But we still need an inventory, so the driver can in that case specify an inventory that's "arbitrarily large".18:00
jaypipesefried: obviously, there's nothing *stopping* anyone from putting MAX_INT in there :)18:00
jaypipesefried: just saying it doesn't really make sense to me :)18:01
aspiersefried: but in the future I'm guessing AMD machines *may* remove that limit18:01
*** lbragstad has quit IRC18:01
aspiersat least it's not something we can immediately rule out18:02
efriedaspiers: Then the driver would use an inventory of actual_limit/num_vcpus or something clever18:02
efriedaspiers: But I also meant to mention this:18:02
efriedif something like that does come up, we can always add a new, specialized resource class for that.18:02
aspierswell sure, but that's why I was suggesting starting with an implementation-specific resource class18:03
efriedaspiers: But do you understand my counter-usecase for that?18:03
efriedIt means you'll *never* be able to schedule generically.18:03
*** lbragstad has joined #openstack-nova18:03
efriedwhereas if you start generic, you can always get more specific.18:04
aspiersI don't see why, since our plan is for the user to always use encrypt_memory=true18:04
aspiersthe request filter can then decide into which resource classes to translate that request18:04
efriedyes, exactly.18:04
efriedIn an env where there's AMD_SEV and INTEL_MKTME, which would it choose?18:05
aspiersso a RequestSpec can't do "give me either foo or bar"?18:05
efriednope18:05
aspiersahah18:05
sean-k-mooney you can do that with traits18:05
efriedmore to the point, placement can't18:05
sean-k-mooneybut not resouce classes18:06
sean-k-mooneyactully that has not merged yet18:06
aspiersOK18:06
sean-k-mooneyor been coded18:06
aspiersefried: so it sounds like basically this isn't even a debate, the only way to make it work is to have a vendor-agnostic resource class :)18:07
sean-k-mooneyim think of https://review.opendev.org/#/c/649992/18:07
efriedsean-k-mooney: We have ?traits=in:X,Y,Z, don't we??18:07
*** tssurya has quit IRC18:07
aspiersI'm fine with that - it's certainly going to make it easier to justify in the spec18:07
sean-k-mooneyefried: there is a spec for that18:07
sean-k-mooneywhich is the one i linked https://review.opendev.org/#/c/649992/18:07
* aspiers takes a short break, back in ~1518:08
*** ivve has joined #openstack-nova18:10
*** tssurya has joined #openstack-nova18:10
efriedaspiers: One can envisage starting off with a specific resource class, and then moving to a generic one when the next one is introduced. Resulting in an interesting upgrade problem where you may have to "reshape" existing inventories/allocations from the old resource class to the new one. And we still support mismatched conductor/compute environments, IIRC, though I can never remember which one has to go first, so that could also18:10
efriedaspiers: So it's still a debate, yes; it could be made to work either way, yes; but IMO for reasons stated, generic is a bit more future-proof without actually losing you anything.18:11
efriedsean-k-mooney: Hmph, I coulda sworn we already had required=in:..., but I don't see it in the api-ref.18:12
efriedyup, that spec.18:13
sean-k-mooneyya so they were previously approved for stein but no implemented18:14
sean-k-mooneyanyway with a generic resocue class and specific trats we could support that usecase if we hav requrie=in18:15
sean-k-mooneyor we could reshape18:15
sean-k-mooneywe do however already have the trait18:16
*** ricolin has quit IRC18:17
efriedmriedem: Would you be able to run the nova meeting this Thursday? I got suckered^wroped into school runs. I should be back around :15-:20.18:22
openstackgerritEric Fried proposed openstack/nova master: Hacking N363: Don't use spec[_set]='string'  https://review.opendev.org/65037018:31
*** nicolasbock has joined #openstack-nova18:31
openstackgerritErik Olof Gunnar Andersson proposed openstack/nova master: Pass on region when we don't have a valid ironic endpoint  https://review.opendev.org/65469218:36
*** eharney has joined #openstack-nova18:39
mriedemefried: yeah i think i can do that18:41
efriedthanks mriedem18:41
openstackgerritSurya Seetharaman proposed openstack/nova master: [WIP] Support adding the reason behind a server lock  https://review.opendev.org/64866218:42
efriedeandersson: Hmph, I see why you did that; perhaps I need a separate bug for min/max18:42
efriedas currently stacked, min/max would have to be partial and yours would have to be closes18:42
eanderssonNah I think your bug is the real fix18:42
eanderssonMine is just a fallback bug fix18:42
*** amodi has joined #openstack-nova18:43
eanderssonbut I am fine with separate bug reports as well18:43
efriedeandersson: Mine should really have a bug that says, "ksa endpoint lookup never worked because of min_version"18:43
efriedYour bug is about region_name not being respected, which is true regardless of my fix.18:44
efriedeandersson: You want to open the other or should I?18:44
efriedI mean, credit's to you for finding the problem (and pretty much for tracking down the cause too) so...18:45
* efried attempts to use flattery to get eandersson to do the paperwork18:45
eanderssonHaha18:47
eanderssonActually yours fixes my bug 99.9%18:47
eanderssonof the time18:47
eanderssonThe only time your fix does not fix my bug is if ServiceNotFound is raised18:47
efriedwhich probably would mean that ironicclient would raise the same18:48
eanderssonVery likely18:48
efriedI can live with that. I can change mine to Partial-Bug and yours can be Closes-Bug. Or we can swap the order of the fixes. But I don't think we're allowed to have Partial-Bug *after* Closes-Bug. mriedem, a ruling?18:49
eanderssonThe only difference is that the ironicclient is less opiniated (in it's current implementation)18:49
eanderssonSure18:49
efried...I guess we can break them apart too. There's actually no reason for them to be in a series, I don't think.18:49
mriedemefried: don't think that matters too much, but partial-bug will change the assignee, whereas related-bug wouldn't18:50
mriedemso i'd use related-bug or separate bugs18:50
efriedthey're truly both fixing part of the bug.18:51
eanderssonI assumed that partial-bug wouldn't change the assignee18:51
efriedbut okay, if the assignee is the only bit of process that matters18:51
eanderssonThe more you know18:51
efriedand we can always change the assignee18:52
efriedso effit, let's go as is.18:52
eanderssonSounds good18:52
efriedmriedem: series at https://review.opendev.org/#/c/654457/ when you get a chance -- ironic ksa stuff never worked (/me sheepish); but ironicclient covered for us; but wasn't respecting region_name when it did, we were just getting lucky most of the time.18:53
efriedwhat other core can tackle ksa stuff?18:53
tssuryamriedem: silly question - does the addition of an existing key (locked) to filters/sorting have anything to do with microversions ? as in we just allow them without any checks from the moment we implement them right ?18:53
eanderssonIt was also partially broken on the ironic client side https://github.com/openstack/python-ironicclient/commit/c038f1db67e7809513d5535648d15c3590b191d518:54
mriedemtssurya: there is a whitelist on filters and sorting params,18:54
tssuryaright,18:54
mriedemspecifying filters that aren't in the whitelist are just ignored for now (gmann has a spec to make that an error),18:54
mriedemspecifying sort params that aren't in the whitelist is a 40018:55
mriedemso you can't add locked as a filter/sort param without a microversion bump18:55
mriedemefried: ack18:55
*** amodi has quit IRC18:55
mriedemtssurya: that's how i verified you couldn't just filter/sort on the instance.hidden field in my change18:55
tssuryamriedem: ah because of change in behaviour for sort params?18:56
mriedemyes18:56
mriedemi thought we said in your spec we weren't going to start filtering/sorting on locked?18:56
tssuryaoh we did ? :D18:56
mriedemoh nvm18:56
mriedem"Filtering/Sorting: The locked key will be added to the existing list of valid sorting/filtering keys so that instances can be filtered/sorted based on this field."18:56
tssuryathe last iteration was that we would18:56
mriedemi might have been thinking of locked_by18:56
tssuryayea18:56
tssuryaokay since I have the bump anyways might as well add this too18:57
mriedemyeah so you'll need a microversion18:57
tssuryathanks mriedem18:57
mriedemnp18:57
mriedemgmann: you might want to comment on this https://review.opendev.org/#/c/648919/4/specs/train/approved/int-replace-default-value-of-the-flavor-swap.rst@7719:08
mriedemit's a small inconsistency cleanup in the api spec so people have said it should be merged into your general cleanup spec, but that's just adding more stuff to your already somewhat large api change19:09
-openstackstatus- NOTICE: the zuul scheduler is being restarted now in order to address a memory utilization problem; changes under test will be reenqueued automatically19:09
mriedemefried: questions in https://review.opendev.org/#/c/654457/19:23
efried...19:23
efriedeandersson: ^19:24
mriedemwhy not squash those 2 changes together?19:25
*** mvkr has joined #openstack-nova19:25
efriedThey're pretty unrelated.19:27
*** nicolasbock has quit IRC19:27
efriedone fixes how we do ksa stuff, the other fixes how we construct the ironicclient19:27
*** nicolasbock has joined #openstack-nova19:27
mriedemit seems the answer to my question here https://review.opendev.org/#/c/654457/2/nova/virt/ironic/client_wrapper.py@131 though is 'ironicclient will try to sort it out' but needs the region_name passed along as well (if you've got multiple regions and are configuring nova-compute to talk to different ones)19:29
*** igordc has joined #openstack-nova19:29
openstackgerritSurya Seetharaman proposed openstack/nova master: [WIP] Support adding the reason behind a server lock  https://review.opendev.org/64866219:31
efriedmriedem: Yes.19:31
efriedThough I think that was actually not the case until recently.19:32
efriedI think the logic to "figure it out" was broken until https://github.com/openstack/python-ironicclient/commit/c038f1db67e7809513d5535648d15c3590b191d5 ?19:32
efriedmriedem: so, do you want these squashed? Am I being silly asserting that they're doing different things?19:33
*** boxiang has quit IRC19:36
mriedemwell, it looks like it might be better to keep them separate because of how ironicclient handles the kwargs in older releases, it looks like region_name was only since Ifc7b45d047c8882a41021e1604b74d17eac2e6e8 in rocky and before that it was os_region_name19:37
mriedemso backports to queens for eandersson's change could be a bit trickier19:37
mriedemespecially since we don't have integration testing for this stuff19:37
*** boxiang has joined #openstack-nova19:37
*** tssurya has quit IRC19:39
mriedemok +2 on the bottom and +W on the top19:41
efriedthanks mriedem. Any idea who else should look at the bottom one?19:41
mriedemhttps://review.opendev.org/#/q/topic:bug/1825583+(status:open+OR+status:merged)19:41
mriedemcleans that up19:41
mriedemyou guys and your silly tp's19:41
mriedem*tb's19:42
mriedemefried: ummm, melwitt?19:42
* mriedem spins the nova core wheel of fortune19:42
artomWheel of cortune19:42
efriednice19:42
artomThat's basically the extent of my contribution here. Puns.19:43
eanderssonNot backporting mine beyond Rocky is fine, as the main patch will fix most cases.19:49
eanderssonAnd endpoint_override is a valid work around.19:50
*** idlemind has quit IRC19:52
mriedemand you're on rocky so you don't have to care about queens :)20:00
*** igordc has quit IRC20:10
mriedemit's sad that something as simple as "pass az to unshelve" https://review.opendev.org/#/c/624689/ is this complicated20:13
*** igordc has joined #openstack-nova20:14
*** igordc has quit IRC20:15
mriedemsorrison: i think this old bug https://bugs.launchpad.net/nova/+bug/1453629 has been fixed since pike due to https://review.opendev.org/#/c/446053/ - are you on pike or later and can confirm?20:16
openstackLaunchpad bug 1453629 in OpenStack Compute (nova) "Creating neutron ports uses incorrect instance availability zone" [Low,Confirmed]20:16
*** colby_ has joined #openstack-nova20:18
colby_Hey All. Is there any way to have shelved_offloaded instances not count toward cpu hours/ram hour usage?20:18
aspiersefried: if resources:foo=1 doesn't currently work with image properties, I'm unsure how it would work when translating another image property gimme_foo=true into it within request_filter.py20:19
aspiersefried: wouldn't we have to add support to ResourceRequest.from_image_props() at least?20:21
sean-k-mooneyaspiers: you are not allowed to request resoruce from an image20:22
sean-k-mooneyand i dont think that should be part of teh SEV spec if we intoduce it20:23
aspierssean-k-mooney: is the term "resource" overloaded here?20:24
sean-k-mooneyi assume resouce:foo was a placment resouce class request20:24
aspierssean-k-mooney: by the way it's named, ResourceRequest.from_image_props() considers an image property requesting a trait as a resource request20:24
sean-k-mooneythat is not allowed in an image20:25
aspiersso in that naming, a "resource" can also refer to something wider which can include traits, not just something from a resource class20:25
sean-k-mooneyis that in the nova code?20:25
aspiersyes20:25
aspiersnova/scheduler/utils.py20:26
aspiersResourceRequest.from_image_props()20:26
aspierscurrently it only supports required traits20:26
sean-k-mooneythat jus tgets the traits20:26
aspiersand you seem to be saying that it shouldn't support required resource classes20:26
sean-k-mooneyit does not extrage resouces20:26
sean-k-mooney*extract20:26
sean-k-mooneycorrect it should not20:27
aspierswhy is that not allowed?20:27
sean-k-mooneybecause i can very eassilay do a denical of serivce attack20:27
aspiersI got the impression from other conversations with efried that we could do this20:27
sean-k-mooneyor get access to hardware im not billed for20:27
aspiershow?20:27
sean-k-mooneyuser can upload images20:28
aspiersoh I see20:28
sean-k-mooneyif they can request any resouce then can ask for a vgpu for example but select the smalest flavor20:28
aspiershow is that different with traits?20:28
sean-k-mooneytraits do not change teh abount of resocue you are requesting20:28
sean-k-mooneythen just filter the posible resouce providers that the resouce can come form20:29
artomI think aspiers's point is that a public cloud might want to bill for a qualitative thing20:29
aspiersright20:29
artomLike SEV ;)20:29
sean-k-mooneyartom: they might20:29
sean-k-mooneybut its not a security risk20:29
efriedaspiers: Until we polish up that RequestSpec.requested_resources thing:20:30
efriedYou're translating encrypt_my_memory=true, which you glean from either the image or the flavor extra specs, gleaned from the RequestSpec20:30
efried==> to a resources:MEM_ENC_CTX=1 in the same RequestSpec's flavor extra specs20:30
efriedbut that's okay because that request spec's flavor is *not* persisted to the db.20:30
efriedSo the placement-ese resource request exists as a flavor extra spec when it's used to create the placement GET /a_c request, but only then.20:30
artomWell, no, but it's still a financial risk if users can give themselves qualitative things through image props20:30
sean-k-mooneyartom: if you care you can use glances upload filetre to prevent it or not allow user to set metadta on an image20:31
artomTrue20:31
aspiersefried: ohhh I see20:31
efriedtechnically you're requesting resource from an image property. But not explicitly with placement-y resources keys.20:31
sean-k-mooneybut the point is we deliberly choose not to allow resocue request wehn we enabeld tratis20:31
*** pcaruana has quit IRC20:32
mriedemcolby_: there is no support for that but it's come up several times20:34
mriedemcolby_: tbc, do you mean not count in the os-simple-tenant-usage API?20:34
mriedemor not count toward quota?20:35
aspiersefried, sean-k-mooney: can we quickly bikeshed the name for the resource class? e.g. MEM_ENC_CTX vs. MEMORY_ENCRYPTED_CONTEXT etc. Do we even need the _CONTEXT suffix? I can't see any precedent for it20:35
sean-k-mooneyi think we need the sufics otherwise its not a noun20:35
aspiersit could be ENCRYPTED_MEMORY20:36
efriedaspiers: The suffix is traditionally units. A sev context thingy is kinda unitless, so not sure that applies.20:36
sean-k-mooneythats still technically an ajitive20:36
efriedENCRYPTED_MEMORY makes it sound like a trait.20:36
sean-k-mooneyyep20:36
aspiersencrypted is an adjective, memory is a noun20:37
sean-k-mooneynot really20:37
aspiersreally ;-)20:37
aspierswell, for sure memory is a noun20:38
aspiersI think encrypted is a gerund or maybe a gerundive20:38
aspiersand they function as an adjective at least20:38
sean-k-mooneymy other issue with encryped_memory is the units it impiles20:38
aspiersagreed20:38
sean-k-mooneywe have memory_mb20:39
sean-k-mooneyi woudl expect encrpted_memory to also be in mb20:39
aspiersyup, so that doesn't work20:39
sean-k-mooneyencrypted_memory_context20:39
sean-k-mooneyi woudl expect ot have idfferent units20:39
aspiersmemory_encrypted_guests?20:40
aspiersthe unit is one guest20:40
sean-k-mooneyno the units are stil wrong20:40
aspierswrong how?20:41
sean-k-mooneyif i have a multi numa node guest how many sev contextes do i need 1 or more then 120:41
aspiers120:41
aspiersAFAIK20:41
sean-k-mooneythe bit that buggs me with memory_encrypted_guests is if we port the num_instance_filter to placement in the future20:42
sean-k-mooneywe could have an inventory of guests/isntance20:42
sean-k-mooneyont he hypervior20:42
aspierswhy is that an issue/20:42
sean-k-mooneyso it coudl be a bit strange to consome 1 allocation of memory_encrypted_guest and 1 allocation of insntace20:43
aspiersdoesn't seem particularly strange to me :)20:43
aspiersthey are different counts20:44
sean-k-mooneywhy not just SEV_CONTEXT20:44
aspiersbecause efried doesn't want a vendor-specific name20:44
sean-k-mooneyi had tought efried last stamet was to go with a speficig vendero one first20:44
aspiersas discussed earlier20:44
aspiersno, the opposite20:44
sean-k-mooneyefried: ?20:44
aspiersI proposed SEV_CONTEXT and he argued for the opposite20:45
aspiersyou can see in the latest spec comments20:45
aspiershttps://review.opendev.org/#/c/641994/11/specs/train/approved/amd-sev-libvirt-support.rst@54120:45
sean-k-mooneywe went back and fort on that a few times so i was nto sure what the latest was20:45
aspiersthere are pros and cons both ways20:46
aspiershttp://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-04-23.log.html#t2019-04-23T18:03:3220:46
*** artom has quit IRC20:48
sean-k-mooneywell unless its partiaclarly egregious im not going to -1 over the resocue class name20:48
aspiersOK thanks but would still be nice to pick a good name ;-)20:48
aspiersI was just wondering where the _CONTEXT idea came from20:49
sean-k-mooneyi take it you dislike the presence of context20:49
sean-k-mooneyproably me20:49
aspiersonly slightly, just feels a bit vague20:49
sean-k-mooneywell for me i thing of security context in many forms20:49
aspiersbut I haven't really come up with anything more precise except _GUEST20:49
aspiersit would be good to find something which conveys the "1 per guest" notion20:50
sean-k-mooneylike an ipsec context20:50
aspierslike _SLOTS20:50
aspiersnot sure if slots has other meanings in nova already?20:51
sean-k-mooneyslot have a more generic meaning to me20:51
sean-k-mooneybut kindof20:51
aspierswell at least slots are clearly discrete integer values20:51
aspierscontext doesn't imply discrete or continuous or int or float etc.20:51
sean-k-mooneyyou havent worked with enough hardware people20:51
aspiershahah20:51
aspiersthankfully20:51
sean-k-mooneyyou can make a nice logical assumtionlike slot can be subdevided an they will alwyas find a reason / way to make it happen20:52
aspierstrue20:53
sean-k-mooneyyou can proable kick the precise name out of the sepc and leave it to the implemetnaiton20:53
aspiersOK maybe let's wait for $0.02 from efried20:53
aspiersyes good point20:53
aspiersI'll stick with MEM_ENCRYPTED_CONTEXT for now20:53
aspiersand I'm gonna check with AMD that it really is 1 per guest20:54
aspiershttps://www.redhat.com/archives/libvir-list/2019-January/msg00652.html is all I have to go on currently20:54
sean-k-mooneyaspiers: looking at the livirt schema21:01
sean-k-mooneyhttps://github.com/libvirt/libvirt/blob/545b0574fd27b56f243e21711935f99160cec214/docs/schemas/domaincommon.rng#L458-L49021:01
sean-k-mooneyit only allows 1 launchSecurity element21:01
sean-k-mooneyso regarless of what amd requires libvirt can only enabel 121:02
aspierswell, launchSecurity has a bunch of parameters21:03
sean-k-mooneyyes21:03
aspiersand whatever hardware resource is being consumed by each guest could feasibly depend on other things21:03
aspierslike it could be one per vcpu for example21:03
sean-k-mooneybut it can only exist once and non fo the sub element can be repeteded21:03
aspiersthen it wouldn't be dependent on this XML21:03
aspiersor 1 per numa node21:03
aspiersbut I'm just sending an email to the AMD guys now to double-check21:04
sean-k-mooneylet me double check21:04
aspiersso we should know for sure soon21:04
sean-k-mooneycool21:04
aspierssent21:05
colby_mriedem: yea currently we use nova usage for billing purposes. I have gnocchi running and have not looked into if it excludes shelved instances or not from usage data21:05
sean-k-mooneyaspiers: i can only exist once in the domain https://github.com/libvirt/libvirt/blob/545b0574fd27b56f243e21711935f99160cec214/docs/schemas/domaincommon.rng#L81-L8321:06
aspierssean-k-mooney: yeah, we already have code parsing that https://review.opendev.org/#/c/636318/3/nova/virt/libvirt/config.py21:07
aspiers(which is unfinished BTW)21:08
efriedaspiers: I like MEM_ENCRYPTED_CONTEXT21:10
aspiersefried: that's lucky, since I just did a search and replace to change everything to that ;-)21:10
efriedI agree with the gripe about _GUEST or _INSTANCE or similar. _SLOT seems too loaded with meaning, like I/O slot.21:11
aspiersyeah fair21:11
efriedMEM_ENCRYPTION_CONTEXT would be better actually.21:11
efriedION vs ED21:11
aspiersoh yeah21:12
efriedotherwise it's the context that's mem-encrypted.21:12
efriedwhen really you want a mem encryption context21:12
efriedfor your instance :)21:12
aspiersright21:12
aspiersefried: and the user-friendly extra spec / property name is gonna be what exactly? since we're bike-shedding :)21:14
aspiersI'm adding that to the spec now21:14
sean-k-mooneyaspiers: hw:mem_encrypt=True21:15
sean-k-mooneyhw:mem_encrypt=sev21:15
aspiersLOL21:15
efriedsean-k-mooney: Why the hw: prefix, out of curiosity?21:15
aspierswell the whole point of this one is to be vendor-agnostic21:15
efriedotherwise, mem_encrypt=True is fine by me. (Is it True or true?)21:15
sean-k-mooneybecause all standard flavor extraspecs are namespaced21:15
aspiersespecially since it will get mapped to resources:MEM_ENCRYPTION_CONTEXT=121:16
sean-k-mooneyefried:whatever the user provides we should lower case21:16
sean-k-mooneyand document "tRue" to mess with stephenfin21:17
efriedheh21:17
sean-k-mooneyim jsut catich up with the last few comment on the spec by they way21:18
efriedokay, if there's some good reason to have a prefix, and there's some good reason for that prefix to be hw:, then some variant of hw:mem_encrypt=$bool is fine with me. hw:mem_encryption would have a nice symmetry with the resource class name.21:18
*** igordc has joined #openstack-nova21:19
sean-k-mooneyya that would be good with me too21:19
aspiersI like the consistency of the hw: prefix21:20
efriedbtw, aspiers, can I strike this topic from the PTG agenda?21:20
efriedsince it seems pretty much resolved?21:20
aspiersefried: the spec topic? only if we get it merged before the PTG ;-)21:20
sean-k-mooney:)21:20
aspiersI hope I can still find an excuse to spend time in the nova room though21:20
aspiersinvent some more SEV problems for us to solve together ;-)21:21
aspierswell I guess there's still the implementation haha21:21
aspiersI'll just hack away in a corner and occasionally ask stupid questions21:21
sean-k-mooneyadd numa to the spec somewhere. it will delay it by a decade21:21
efriedaspiers: You can own this one:21:21
efriedCompute capabilities traits placement request filter (mriedem)21:21
aspierssean-k-mooney: LOL21:21
aspiersefried: uhoh :)21:21
efriedit'll actually be very closely related to what you're doing here.21:22
aspierswell sure that sounds interesting21:23
efriedhmph, except for the wrinkle in the existing patch about not being able to get at the multiattach bool from within request_filter.py, the generic solution would be pretty easy there.21:23
sean-k-mooneyits also related in a way to the image metadata prefilteing im hoping to work on although not quite the same21:24
openstackgerritMerged openstack/nova stable/queens: Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree  https://review.opendev.org/64019821:24
sean-k-mooneyanyway looking at the sev spec which i just finsished again.21:24
sean-k-mooneywe proably could merge it as is and fixup the last few nits in a follow up patch21:24
sean-k-mooneybut that up to ye21:25
sean-k-mooneyi didnt see any really blocker anymore21:25
aspierssean-k-mooney: thanks, just got a few minor tweaks based on last few hours of discussion21:25
aspiersI'll submit one more patch set and hopefully that will be enough21:25
sean-k-mooneyon the + side your feature already has excelent documentation21:26
aspierscool21:26
aspiersoh I already got a response from Brijesh at AMD21:27
aspiershe confirms it's 1 unit per guest21:27
sean-k-mooneyexcelent21:27
aspiersand there will always be a limit, which can be queried through the CPUID21:27
sean-k-mooneyya that is not surprising21:27
efriedalways always?21:27
efriedlike, don't need the conf var always?21:28
aspiersalways always a limit21:28
*** tosky has joined #openstack-nova21:28
aspiershttps://marc.info/?l=qemu-devel&m=155502702424182&w=221:28
sean-k-mooneywell i think they ment there will alway be a limit in hardware21:28
aspiersyes21:28
sean-k-mooneybut not sure that mean we can discover it via cpuid yet21:28
aspiersthat patch exposes it in QEMU21:28
aspiersnext step would be to expose in libvirt getDomainCapabilities()21:29
aspiersat which point we can get it in nova21:29
efriedbut that would require a bump in the min qemu version and stuff...21:29
efriedwhich I thought was kind of already what was discussed with danpb21:29
sean-k-mooneyok so we need to cary the config option for at least 2 years21:29
aspiersefried: not just with danpb but already covered in the spec21:31
aspierssean-k-mooney: yeah most likely21:31
sean-k-mooneyjaypipes: thanks for reviewing the sriov mighration pathces :)21:33
sean-k-mooneynow if only stephenfin or maybe efried could leave the final +2+w it could merge before the ptg21:33
sean-k-mooneyor more importantly before one of the 41 conflicting patches does21:34
efriedsean-k-mooney: link please21:34
sean-k-mooneyhttps://review.opendev.org/#/c/629589/2721:34
sean-k-mooneythat and https://review.opendev.org/#/c/620115/3521:34
efriedWhen you say things like sriov and live migration I immediately think, "I hope someone else has got this."21:34
aspierssean-k-mooney, efried: BTW funnily enough Brijesh's reply when I asked why there is a limit "We have limited number of slots in memory controller, each slot gets a different encryption key."21:35
efriedhah!21:35
sean-k-mooneyya21:35
efriedMEM_ENCRYPTION_KEY_SLOTS21:35
efriedagain, pretty impl specific. Let's stick with CONTEXT21:36
aspiersOK X-D21:36
sean-k-mooneyit had to be somthign like that e.g. they what  a fix number of keys they coudl handel21:36
aspiersyeah21:36
sean-k-mooneywith this kid of thing you donw want those stored in ram which means sticking them in a register file somewhere21:37
sean-k-mooneyand you only have so much spcae on a die to add that memory so there will always be a limit like this21:37
sean-k-mooneyat least 15/16 is resonable21:38
aspiersright21:38
sean-k-mooneyi remember being asked to enable QOS policies with intel QAT devices at one point where the limit was 421:38
aspiershaha21:39
sean-k-mooneyyou could have 32 VF21:39
sean-k-mooneybut only 4 QOS policies21:39
sean-k-mooneyand it was a 1:1 maping between QOS policey and a singel VF21:39
sean-k-mooneyif  you used any of the other ones they ignroed the qos policy21:39
sean-k-mooneywe never enabeld in in openstack.21:40
openstackgerritMatt Riedemann proposed openstack/nova master: AZ list performance optimization: avoid double service list DB fetch  https://review.opendev.org/63694721:40
mriedemefried: i found and cleaned this up with test wrinkles and addressed some of my own comments ^ let me know if you think it needs more21:40
mriedembut yeah that's a really nice perf improvement from avolkov21:40
*** whoami-rajat has quit IRC21:40
efriedmriedem: ack, looking.21:41
*** tjgresha_nope has quit IRC21:41
efriedare you recusing yourself from approving at this point?21:41
sean-k-mooneymriedem: ya it look like its about half the time21:41
mriedemefried: not necessarily21:41
mriedemi just added test stuff and some code comments21:41
*** ttsiouts has joined #openstack-nova21:42
*** tjgresha has joined #openstack-nova21:43
openstackgerritMerged openstack/python-novaclient master: Updates for OpenDev transition  https://review.opendev.org/65442921:43
sean-k-mooneyya the delta for what you added is mainly non functional changes21:43
sean-k-mooneyso is the main sped up form getting the avilablity zones for jsut the enabled serivces21:45
efriedmriedem: lgtm. Why did you switch from monkeypatch to mock.patch?21:46
mriedemso i can count the calls21:46
mriedemsilly pants21:46
mriedemsean-k-mooney: it's that we don't get enabled services across 10K hosts twice21:47
aspiersdid you folks spot the pysnooper tip?21:47
sean-k-mooneyah21:48
sean-k-mooneyya21:48
sean-k-mooneyim not sure if ill end up using it or not but we will see21:48
aspiersit looks pretty nice21:48
sean-k-mooneyya but im not sure it will actully be helpful fo more complicate things. but next time i ned to debug some thing complex and im not usgin a grapical debugger ill give it a try21:52
efriedmriedem: oh, stub_out doesn't return the fixture. And apparently MonkeyPatch doesn't have a .mock attribute, like MockPatch does. Remind me why we have stub_out?21:53
efried(rhetorical)21:53
gmannmriedem: RE: on job, yes it will merge the var from parent and derived jobs.21:55
gmannmriedem: few zuul var are not append-able for example files, irrelevant-files etc which are regex not array.  but devstack var are all merged from all hierarchy  of job21:57
mriedemjaypipes: fat ol -2 on this https://review.opendev.org/#/c/625755/ if you remember it21:59
mriedemefried: to get off mox?21:59
mriedemgmann: yup thanks. i'm now just waiting a few days for ci results. :)22:00
*** imacdonn has quit IRC22:01
*** imacdonn has joined #openstack-nova22:01
sean-k-mooneymriedem: by the way did you want to review the sriov migrtion stuff or are you happy to leave it to jay and stephen22:07
mriedemwant to?22:09
mriedemwhy don't you have it in a runway if you're looking for reviews?22:09
sean-k-mooneyoh good point22:09
sean-k-mooneyits going to take w while form e to get opendev.org into fingre memory. im really glad the got the redirects in place22:12
sean-k-mooneyanyway night all o/22:12
*** slaweq has quit IRC22:12
*** slaweq has joined #openstack-nova22:14
mriedemspeaking of it looks like we have busted jobs on stable branches b/c the openstack-infra/devstack-gate change wasn't made there in our .zuul.yaml22:15
mriedemeandersson: are you going to work on backports for these? https://review.opendev.org/#/q/topic:bug/1825583+status:open+project:openstack/nova22:19
eanderssonIf I get the time this week. Got a busy schedule, and well next week is Denver.22:20
eanderssonSo can't promise that I will be able to do it before Denver.22:20
mriedemminions...22:20
eandersson=]22:20
openstackgerritMatt Riedemann proposed openstack/nova stable/pike: Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree  https://review.opendev.org/64020722:30
*** jangutter has quit IRC22:35
*** tonyb has joined #openstack-nova22:41
efriedo/22:42
*** luksky has quit IRC22:43
openstackgerritMerged openstack/nova master: Only set oslo_messaging_notifications.driver if using RPCFixture  https://review.opendev.org/65395422:50
*** tkajinam has joined #openstack-nova22:55
openstackgerritAdam Spiers proposed openstack/nova-specs master: Re-approve AMD SEV support for Train  https://review.opendev.org/64199422:56
aspiersefried, sean-k-mooney, jaypipes: patch set 12 now up, hopefully the final one https://review.opendev.org/#/c/641994/11..12//COMMIT_MSG23:02
openstackgerritIvens Zambrano proposed openstack/nova-specs master: RMD Plugin: Energy Efficiency using CPU Core P-State control The power state of a core can be setup between a minimum and the maximum frequency on the cores as defined in the factory. Each core on a CPU can be defined individually to perform at specific f  https://review.opendev.org/65102423:02
openstackgerritAdam Spiers proposed openstack/nova master: Add infrastructure for invoking libvirt's getDomainCapabilities API  https://review.opendev.org/65526823:09
openstackgerritTony Breeds proposed openstack/nova stable/pike: Move legacy-grenade-dsvm-neutron-multinode-live-migration in-tree  https://review.opendev.org/64020723:12
*** tosky has quit IRC23:13
*** ttsiouts has quit IRC23:14
*** ttsiouts has joined #openstack-nova23:15
*** gmann is now known as gmann_afk23:17
*** ttsiouts has quit IRC23:19
*** rcernin has joined #openstack-nova23:20
*** mlavalle has quit IRC23:24
openstackgerritAdam Spiers proposed openstack/nova master: Add detection of SEV support from QEMU/AMD-SP/libvirt on AMD hosts  https://review.opendev.org/63385523:27
aspierskashyap: ^^^ I finally gave you what you wanted ;-)23:32
aspiersyou'll note that the SEV patch is extremely readable now23:32
*** igordc has quit IRC23:32
* aspiers goes to bed23:32
*** gyee has quit IRC23:39
*** jangutter has joined #openstack-nova23:40
*** artom has joined #openstack-nova23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!