Friday, 2022-05-13

*** Guest0 is now known as prometheanfire00:29
opendevreviewWenping Song proposed openstack/os-traits master: Update python testing as per zed cycle testing runtime  https://review.opendev.org/c/openstack/os-traits/+/84168207:04
*** lajoskatona_ is now known as lajoskatona07:42
*** songwenping__ is now known as songwenping08:00
opendevreviewWenping Song proposed openstack/placement master: Update python testing as per zed cycle testing runtime  https://review.opendev.org/c/openstack/placement/+/84169008:42
opendevreviewWenping Song proposed openstack/placement master: Update python testing as per zed cycle testing runtime  https://review.opendev.org/c/openstack/placement/+/84169008:42
opendevreviewWenping Song proposed openstack/os-resource-classes master: Update python testing as per zed cycle testing runtime  https://review.opendev.org/c/openstack/os-resource-classes/+/84170009:20
opendevreviewWenping Song proposed openstack/os-resource-classes master: Update python testing as per zed cycle testing runtime  https://review.opendev.org/c/openstack/os-resource-classes/+/84170009:46
sean-k-mooneyricolin: +1 on the spec 10:12
sean-k-mooneyricolin: i have resotred the mlock patch too so feel free to updated it10:12
ricolinthanks sean-k-mooney :)10:12
sean-k-mooneyricolin: if you adress the extra spec name im baicly +2 ill read over it again when you respin but no rush10:15
opendevreviewRico Lin proposed openstack/nova-specs master: Add vIOMMU device support for libvirt driver  https://review.opendev.org/c/openstack/nova-specs/+/84031010:24
ricolinsean-k-mooney: done ^^^ :)10:25
sean-k-mooney+2 :) am milestone 1 i think is thursday. assuming gibi  or others approve the spec can you submit the patch to os-traits to add the traits10:28
sean-k-mooneyif we merge that before tursday then the traits can be included in the m1 relases of os-traits10:28
sean-k-mooneynova's unit tests need the traits to be in a release version of os-traits to  pass if its not merge by then its not really a big deal we will just do another release whenever they land and the nova code look good to merge10:30
stephenfinsean-k-mooney: gibi: melwitt: Reviewed that PCI in placement spec. Looks pretty good, though I'm convinced you're making an unnecessary rod for your own back in trying to port that dynamic PF/VF logic to placement land ;-)10:51
sean-k-mooneystephenfin: ya it is complicating things11:00
sean-k-mooneystephenfin: that use case is really only for sriov nics11:01
sean-k-mooneyas in for neuton ports11:01
sean-k-mooneyi dont think it really exists for pci alias11:01
sean-k-mooneyso we can kind of punt. technialy it applise to the alias too but i dont think that is a common usecase there11:02
sean-k-mooneystephenfin: you might want to look over ricolin's spec for the viommu support11:02
sean-k-mooneyits pretty short11:02
sean-k-mooneyhttps://review.opendev.org/c/openstack/nova-specs/+/84031011:03
stephenfinsure11:04
gibistephenfin: so you suggest that deciding to whitelist either the PF or its VFs are rarely used? I think the opposite. It is so easy to just whitelist everthing under 0000:81:* both PF and VF, without thingking to much, and then start using the compute for both direct and direct-physical ports, it just works, so I think there is a lot of deployment out there that might not need to whitelist both but 11:06
gibithey did 11:06
gibiif we start rejecting such config there will be a lot of pain figuring out a whitelist in these deployments that matches the current consumption 11:07
sean-k-mooneyim not sure really. im tempted to say perhaps we sould only support that if you are not trackign devices in placment11:08
sean-k-mooneyand in the mvp implement supprot for only tracking deivces as PFs or VFs11:08
sean-k-mooneyand then we could add the mixed suport after if needed11:09
stephenfinNo, I'm suggesting people wanting to use both 'direct' and 'direct-physical' on a single host is likely rare. I think it's reasonable to insist that users that use wildcard device addresses (or vendor/device IDs) choose whether they want to consume the VFs or PFs11:09
sean-k-mooneystephenfin: well your suggestiong they make that dissionc staticaly at deployment time11:09
stephenfinAt the moment, a user can dynamically choose whether they consume the PF or one or more of the VFs. I think moving to a static model makes sense now11:09
stephenfinYup11:09
sean-k-mooneyrather then dynmical as workloads are schulded11:10
stephenfinexactly11:10
sean-k-mooneyi think pf passhtough is much less common then vf11:10
gibiI have no problems forcing this to new deployments, but I still believe upgrade will be a pain11:10
stephenfin*borderline non-existent (I say, with no actual evidence either way :) However, it seems like an odd thing to do, especially when we don't support nested virt)11:11
gibibut yeah, we can push out the pain to the future by keepin the pci tracking in placement optional11:11
sean-k-mooneystephenfin: well my evidence is that live migrtation, cold migration and unshelve have basicaly been broken since the feature was added until like 2 cylces ago11:11
stephenfinI mean, it's been more than three years and people can still avoid tracking of pinned CPUs in placement11:11
stephenfinso that can can be kicked endlessly down the road11:12
sean-k-mooneyso while i know we have some customer using PFs those custoemr also have static deployments11:12
sean-k-mooneygibi: i guess its really up to you11:13
sean-k-mooneyif you want to include it in the mvp11:13
gibiOK, I see an agreement forming. Let's implement PCI tracking with placement without the dynamic selection. Keep the everything working as today if the PCI tracking in placement is disabled11:13
sean-k-mooneyi think it could be a patch at the end of the seirse by the way11:13
sean-k-mooneygibi: sound good to me that means the prefilter will not have an auto mode11:14
gibithen when everythin (except the dynamic thing works with placemnet) deprecate the old way11:14
gibiand wait for feedback11:14
gibiyeah that means no auto mode11:15
sean-k-mooneyyou will opt in on the compute with the new config option and opt in on schduler by enabling prefilter11:15
gibioperator needs to first enable tracking in the compute config11:15
gibithen enable prefiltering in the scheduler11:15
sean-k-mooneyyep11:15
gibiand if somebody only opt in on a set of computes then enables the prefilter then we say sorry you lost the non enabled computes from the PCI scheduling11:16
stephenfinI think lack of auto mode is a good thing. I called that out in the review as something weird (and I think melwitt had similar concerns)11:16
stephenfinThe less magic, the better11:16
sean-k-mooneyack its nice form an opts perspectvie if an only if it means they dont have to do anything on upgrade11:17
stephenfinsean-k-mooney: woah, that vIOMMU thing has got way bigger. I was proposing enabling by default on supported platforms with zero configurability. We now have...three knobs? :-O11:17
stephenfinI've asked ricolin to clarify the need for each knob since it's not at all obvious from reading the spec. If they're really necessary, we'll need this info to document the extra specs.11:18
gibiupgrading to Zed (if it lands) with default config will mean only the old PCI scheduling will be used by nova. So no real upgrade impact. Then operators needs to enable the new tracking11:18
sean-k-mooneystephenfin: yep. the model, bit with which i dont like exposing but is required and locked memroy wich is technialy unrealted11:18
sean-k-mooneybut they need locked memroy for there specific hardware and you can only get that with realtim or sev today11:19
sean-k-mooneygibi: yep with no config changes old behavior11:19
gibiso  we need to incentivise ops to enable the new behavior11:19
stephenfinAt risk of pre-empting discussion on the spec, why is the model necessary? Are these guest OS behavior implications or something? Is there not a sane default we can pick?11:20
sean-k-mooneyso no upgrade impact by defualt but we will ahve to document how to move form one to the other11:20
sean-k-mooneystephenfin: the sane default would be virtio but it need a very new libvirt/qemu11:20
sean-k-mooneywe cant use intel because well there usecase is for aarch64 servers11:21
sean-k-mooneywe could choose dynmically i guess11:21
* gibi gets something to eat then sit down and update the PCI spec11:21
stephenfinAnd I'm guessing we can't check what the libvirt version is and use 'virtio' if libvirt > 8.3.0 else 'intel'/'smmuv3' for Intel/ARM respectively?11:21
sean-k-mooneybased on the machine-type but i dont like the live migration implciations of that11:21
stephenfinSo long as we record the model used in system metadata, we should be fine11:22
sean-k-mooneywell we could but ya 11:22
stephenfin(i.e. so we don't change during rebuild)11:22
stephenfin*migration11:22
sean-k-mooneyrecord in system_metadta or elsewhere for migration11:22
stephenfinYup11:22
sean-k-mooneyi was hoping we woudl only expose the model extra spec initally11:22
stephenfinIn case it's not obvious, I hate exposing knobs unless they're really useful. If people want all the knobs, use libvirt/oVirt :)11:22
sean-k-mooneyyep i know it had 6 i think at one point11:23
sean-k-mooneyi have been slowing getting it down to the minimal set11:23
stephenfinwhat about aw_bits?11:25
sean-k-mooneystephenfin: by the way im not sure if we can just turn the viommu on always or if there are performance overhead even if you dont explicty try to use it for say realtime guests11:25
sean-k-mooneystephenfin: that i really did not want to expose11:26
sean-k-mooneystephenfin: mnaser  left a comment about why they needed it i dont knwo if we can always set it to the max support or if it need to be tuneable11:26
sean-k-mooneyactully it was on irc let me see if i can find it11:29
sean-k-mooneystephenfin: i actully tought you exposed that in your patch by the way11:31
sean-k-mooneythe aw_bit11:31
stephenfinNope https://review.opendev.org/c/openstack/nova/+/830646/1/nova/virt/libvirt/config.py#367611:31
sean-k-mooneyoh so ricolin  added it11:32
sean-k-mooneyhttps://review.opendev.org/c/openstack/nova/+/830646/5/nova/virt/libvirt/config.py#3700=11:33
sean-k-mooneyah you just found it11:33
sean-k-mooneystephenfin: so if this really is needed i woudl liek to tie its value to the vm.11:34
sean-k-mooneystephenfin: so either in instance_system_metadta11:34
sean-k-mooneyor as a flavor/image property11:34
sean-k-mooneyif it  was a per host option we would have to schdule on it some how11:34
sean-k-mooneyand possible extend the migration data objects if we did not store it in the instace_system_metadata11:35
stephenfinYeah, no arguments from me there. Host-level configuration for guest-specific knobs is always a problem11:35
sean-k-mooneyi was hoping we could jsut not set it and let it up ot libvirt11:35
stephenfinme too11:37
sean-k-mooney"""Use virtio if libvirt >= 8.5.0, else intel or the aarch64 equivalent""" hehe i can say the aarch64 equivalent so i cant rememeber it either11:38
sean-k-mooneystephenfin: since i have your attention do you have any feedback on https://review.opendev.org/c/openstack/oslo-specs/+/839819 im trying to poc it now11:39
stephenfinSure11:40
* sean-k-mooney spent 4 hours yesterday fixing emac. i fixed it by deleting my config directory and cloning int again...11:40
stephenfinI think I know what the problem is there already 🙃11:41
sean-k-mooneyyour right i should have stuck with nano11:42
sean-k-mooneyactully the problem was it was sudenly in vim mode i think11:42
stephenfintouché11:42
sean-k-mooneyi use the spacemecs deistribution which has the evil plugin11:42
sean-k-mooneywhich by default make it have vim key bindings11:42
sean-k-mooneyso for some reason i coudl not use any of the ones im use to or select text11:43
sean-k-mooneyanyway its back to working in emacs/nano like mode and i can use it again but that was not fun11:44
stephenfingibi: sean-k-mooney: I think this is ready to merge now https://review.opendev.org/c/openstack/nova/+/83902912:24
stephenfinalso, it passes 🎉12:24
gibistephenfin: yepp, +212:25
sean-k-mooneyoh cool let me run it locally first since i have 3.10 and then ill +w12:26
sean-k-mooneystephenfin: i could have sworne i made the fucntional evne generitive at some point12:29
stephenfinyou did, but it hasn't merged. I stole that idea sorrynotsorry :)12:29
sean-k-mooneyoh i tought that merged ages ago12:30
sean-k-mooneyi guess i should take a look at that again12:30
sean-k-mooneycool your patch is running locally12:30
sean-k-mooneythere are a bunch of deprecations warnings12:31
stephenfin3.10 specific?12:31
sean-k-mooneyhome/sean/repos/openstack/nova-3/nova/tests/fixtures/notifications.py:42: DeprecationWarning: notifyAll() is deprecated, use notify_all() instead12:31
sean-k-mooney  self._cond.notifyAll()12:31
stephenfinI know distutils is puking everywhere since that's going away in 3.1212:32
sean-k-mooneystephenfin: so yes i think that is 3.10 related12:32
sean-k-mooneyit seams to just be that actully12:32
sean-k-mooneybut its multiple times on every test12:32
sean-k-mooneyok not every test 12:34
sean-k-mooneyhttps://zuul.opendev.org/t/openstack/build/8a45e04d9dbf4703bb1fb9f7d91a549b/log/job-output.txt#881512:34
sean-k-mooneybut we see it in the ci too anything that uses that fixture12:34
sean-k-mooneythat seams to be the only one12:34
sean-k-mooneycare to fix that in a followup maybe or respin12:35
stephenfinsure12:35
opendevreviewAlexey Stupnikov proposed openstack/nova stable/wallaby: Test aborting queued live migration  https://review.opendev.org/c/openstack/nova/+/84148312:47
sean-k-mooneystephenfin: actully do you want me to just fix the notificaiton fixture in a followup or are you already working on it12:54
stephenfingo for it; I'm playing around with cinder stuff rn12:54
sean-k-mooneycool will do12:55
gibistephenfin, sean-k-mooney: should we just rename the whitelist to device_list or we make the two config totally separate and use the device_list also to enable the new feature?13:03
gibiif we make them separate then in the initial release the whitelist still needed for the neutron based sriov even if the device_list is used for the alias based PCI passthrough13:04
gibinow I think that renaming is simpler and then I would add an extra config to enable the new placement based PCI tracking13:05
sean-k-mooneyok13:05
sean-k-mooneylets rename and add a hopefully tempoery extra option for placment tracking13:05
gibishould the placement PCI tracking config be on libvirt virt driver level or on the compute level? 13:06
gibicompute level is future proof13:06
gibibut a bit confusing as it will only be used by the libvirt driver now13:06
sean-k-mooneygood question13:06
sean-k-mooneyyou could put it in the pci section13:06
gibithat is a good middle ground13:07
gibithanks13:07
sean-k-mooney[pci]/report_in_placement=True|False13:07
gibiack13:08
sean-k-mooneystephenfin: any opipion ^13:08
stephenfinAgreed. Sounds like a '[pci]' option13:09
sean-k-mooneygibi: i know there are some that want us to do the rename anyway for policital reasons so we proably should not make operators choose between fucntionality and politics13:09
gibiyeah that also a + of this approach we just deprecate the old name, but the old name still get all the new tags for now13:10
gibiso the deployer can choose when to rename it in their config13:10
sean-k-mooneyworks for me13:11
sean-k-mooneyare those all the outstanding question in the pci spec adressed then?13:12
gibiI feel like it, but I'm doing the update now so I might bump into qustions along the way13:13
bauzasfolks, limited connectivity here probably for the end of the day, my optic fiber is changed13:15
bauzasI changed the provider, but it looks like the older provider made some mistakes13:20
opendevreviewsean mooney proposed openstack/nova master: trivial: fix deprecation warning in notification fixture  https://review.opendev.org/c/openstack/nova/+/84175613:20
sean-k-mooneystephenfin: ok done ^ back to oslo dirver13:21
sean-k-mooneybauzas: ack13:21
sean-k-mooneybauzas: when i change my mobile provider recently my old provider blocked the number porting by mistake and i lost my old number that i have had for alomts 15-20 years13:22
sean-k-mooneychanging providers sucks13:22
sean-k-mooneyon the other hand no more spam calls13:23
* sean-k-mooney just because its illegal in ireland to cold call a mobile number does not stop uk investment brokers trying to offer me advice on stock inverstments....13:24
bauzassean-k-mooney: the problem here is that my FTTH ONT number was wrote wrong13:25
bauzaswritten*13:25
bauzasthe ONT number, not the phone number13:25
bauzasso, the infrastructure operator thought it was a new fiber13:26
sean-k-mooneyya apparently the same happened for me more or less the account number was apprently wrong but i got it from the bill and gave it directly to the new operator so im not sure how that could have been 13:26
opendevreviewAlexey Stupnikov proposed openstack/nova stable/wallaby: Add functional tests to reproduce bug #1960412  https://review.opendev.org/c/openstack/nova/+/84176013:26
sean-k-mooneyi actully no have 2 seperate FTTH lines and ONTs13:26
sean-k-mooneyi now have EIR FTTH i used to have SIRO form vodaphone13:27
opendevreviewAlexey Stupnikov proposed openstack/nova stable/wallaby: Clean up when queued live migration aborted  https://review.opendev.org/c/openstack/nova/+/84173613:27
bauzasbut eventually, my new fiber tech found the right ONT number, so he went to the OLT13:28
sean-k-mooneywe have 2 semi state owned fiber networks in ireland13:28
bauzassean-k-mooney: I guess that's like in France13:28
sean-k-mooneyone is owned by what used to be the nationalise telecom company and the other is owned by the state owned electic network infra ownwer13:29
bauzaswe have different networks, depending on the town13:29
sean-k-mooneyboth are then open to telecoms to offer servies over13:29
bauzaslike here, in Isere department, we have a specific network 13:29
bauzasfor this network, we have an infra operator13:29
bauzasthat creates fibers for the whole town up to either a block of flats or a street13:30
sean-k-mooneyin ireland we are trying to avoid that and have 2 state owned/semi state owned open fiber networks13:30
bauzasand then, every customer can ask a network operator to get a fiber until their either flat or house13:31
sean-k-mooneyya that is similar here13:32
bauzaslike for me, this is changing from the infra operator to the network operator 5 meters off my house13:32
bauzasin the street13:32
*** dasm|off is now known as dasm14:01
dansmithyea/query sean-k-mooney 14:51
dansmithwow14:51
opendevreviewsean mooney proposed openstack/nova master: trivial: fix deprecation warning in notification fixture  https://review.opendev.org/c/openstack/nova/+/84175615:15
sean-k-mooneystephenfin: fixed nits ^15:15
opendevreviewMerged openstack/nova master: Add Python 3.10 functional jobs  https://review.opendev.org/c/openstack/nova/+/83902915:15
opendevreviewBalazs Gibizer proposed openstack/nova-specs master: PCI device tracking in Placement  https://review.opendev.org/c/openstack/nova-specs/+/79104715:33
gibisean-k-mooney, stephenfin, melwitt: fresh and simplified ^^15:34
opendevreviewAlexey Stupnikov proposed openstack/nova stable/wallaby: Add functional tests to reproduce bug #1960412  https://review.opendev.org/c/openstack/nova/+/84176015:37
opendevreviewAlexey Stupnikov proposed openstack/nova stable/wallaby: Clean up when queued live migration aborted  https://review.opendev.org/c/openstack/nova/+/84173615:37
sean-k-mooneygibi: thanks im not sure i have the brain power left to read it fully at 5 pm on a friday16:04
sean-k-mooneybut ill defintly take a look on monday16:04
gibiI totally understand :)16:04
gibihave a nice weekend16:05
sean-k-mooneyyou too o/16:05
opendevreviewsean mooney proposed openstack/os-vif stable/xena: Use TCP keepalives for ovsdb connections  https://review.opendev.org/c/openstack/os-vif/+/84177116:09
opendevreviewsean mooney proposed openstack/os-vif stable/xena: only register tables used by os-vif  https://review.opendev.org/c/openstack/os-vif/+/84177216:09
opendevreviewsean mooney proposed openstack/os-vif stable/wallaby: Use TCP keepalives for ovsdb connections  https://review.opendev.org/c/openstack/os-vif/+/84177316:10
opendevreviewsean mooney proposed openstack/os-vif stable/wallaby: only register tables used by os-vif  https://review.opendev.org/c/openstack/os-vif/+/84177416:10
opendevreviewsean mooney proposed openstack/os-vif stable/victoria: Use TCP keepalives for ovsdb connections  https://review.opendev.org/c/openstack/os-vif/+/84177516:13
opendevreviewsean mooney proposed openstack/os-vif stable/victoria: only register tables used by os-vif  https://review.opendev.org/c/openstack/os-vif/+/84177616:13
opendevreviewsean mooney proposed openstack/os-vif stable/ussuri: Use TCP keepalives for ovsdb connections  https://review.opendev.org/c/openstack/os-vif/+/84177716:13
opendevreviewsean mooney proposed openstack/os-vif stable/ussuri: only register tables used by os-vif  https://review.opendev.org/c/openstack/os-vif/+/84177816:13
opendevreviewsean mooney proposed openstack/os-vif stable/train: Use TCP keepalives for ovsdb connections  https://review.opendev.org/c/openstack/os-vif/+/84177916:14
opendevreviewsean mooney proposed openstack/os-vif stable/train: only register tables used by os-vif  https://review.opendev.org/c/openstack/os-vif/+/84178016:14
*** gibi is now known as gibi_pto16:39
opendevreviewMerged openstack/nova master: trivial: fix deprecation warning in notification fixture  https://review.opendev.org/c/openstack/nova/+/84175616:45
mnaserhappy friday -- https://review.opendev.org/c/openstack/nova/+/840985 + https://review.opendev.org/c/openstack/nova/+/840993 are trivial nice to have fixes that could use reviews =)18:53
opendevreviewmelanie witt proposed openstack/placement stable/victoria: placement-status: check only consumers in allocation table  https://review.opendev.org/c/openstack/placement/+/84070220:04
opendevreviewmelanie witt proposed openstack/placement stable/ussuri: placement-status: check only consumers in allocation table  https://review.opendev.org/c/openstack/placement/+/84070320:05
*** dasm is now known as dasm|off21:18

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!