*** chyka_ has joined #openstack-nova | 00:02 | |
*** yamamoto has joined #openstack-nova | 00:03 | |
*** chyka has quit IRC | 00:05 | |
*** takashin has joined #openstack-nova | 00:07 | |
*** yamamoto has quit IRC | 00:07 | |
*** gouthamr has joined #openstack-nova | 00:07 | |
*** dave-mccowan has quit IRC | 00:10 | |
*** jobewan has joined #openstack-nova | 00:11 | |
*** odyssey4me has quit IRC | 00:11 | |
*** odyssey4me has joined #openstack-nova | 00:11 | |
*** mvk has quit IRC | 00:16 | |
*** rcernin has quit IRC | 00:19 | |
*** rcernin has joined #openstack-nova | 00:19 | |
*** liverpooler has joined #openstack-nova | 00:19 | |
*** jobewan has quit IRC | 00:23 | |
*** salv-orlando has joined #openstack-nova | 00:24 | |
*** Dinesh_Bhor has joined #openstack-nova | 00:24 | |
*** chyka_ has quit IRC | 00:27 | |
*** zhurong has joined #openstack-nova | 00:27 | |
*** salv-orlando has quit IRC | 00:29 | |
*** mvk has joined #openstack-nova | 00:29 | |
*** gouthamr has quit IRC | 00:31 | |
*** vipul has joined #openstack-nova | 00:34 | |
*** gcb has joined #openstack-nova | 00:36 | |
*** gcb has quit IRC | 00:36 | |
*** vipul has quit IRC | 00:40 | |
*** fragatin_ has quit IRC | 00:40 | |
*** fragatina has joined #openstack-nova | 00:41 | |
*** fragatina has quit IRC | 00:41 | |
*** fragatina has joined #openstack-nova | 00:41 | |
*** fragatina has quit IRC | 00:41 | |
*** vipul has joined #openstack-nova | 00:42 | |
*** fragatina has joined #openstack-nova | 00:43 | |
*** jichen has joined #openstack-nova | 00:44 | |
*** fragatina has quit IRC | 00:48 | |
*** jobewan has joined #openstack-nova | 00:49 | |
*** hongbin has joined #openstack-nova | 00:51 | |
*** harlowja has quit IRC | 00:53 | |
*** mriedem has quit IRC | 00:55 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Create/lookup API services in cell0 on WSGI app startup https://review.openstack.org/556670 | 00:55 |
---|---|---|
*** gjayavelu has quit IRC | 00:57 | |
*** yingjun has joined #openstack-nova | 00:58 | |
*** oanson has quit IRC | 00:58 | |
*** jobewan has quit IRC | 00:58 | |
*** gouthamr has joined #openstack-nova | 00:59 | |
*** archit has quit IRC | 00:59 | |
*** phuongnh has joined #openstack-nova | 01:00 | |
*** oanson has joined #openstack-nova | 01:04 | |
*** yangyapeng has joined #openstack-nova | 01:06 | |
*** liverpooler has quit IRC | 01:08 | |
*** tiendc has joined #openstack-nova | 01:08 | |
*** voelzmo has joined #openstack-nova | 01:09 | |
*** annp has quit IRC | 01:10 | |
*** annp has joined #openstack-nova | 01:11 | |
*** yangyapeng has quit IRC | 01:11 | |
*** hoangcx has quit IRC | 01:12 | |
*** hoangcx has joined #openstack-nova | 01:12 | |
*** yangyapeng has joined #openstack-nova | 01:13 | |
*** jobewan has joined #openstack-nova | 01:16 | |
*** Tom-Tom has quit IRC | 01:20 | |
*** trozet has quit IRC | 01:24 | |
*** salv-orlando has joined #openstack-nova | 01:25 | |
*** gyankum has joined #openstack-nova | 01:27 | |
openstackgerrit | melanie witt proposed openstack/nova master: rbd: use MAX_AVAIL stat for reporting bytes available https://review.openstack.org/556692 | 01:28 |
*** hshiina has joined #openstack-nova | 01:29 | |
*** _ix has joined #openstack-nova | 01:29 | |
*** salv-orlando has quit IRC | 01:29 | |
*** voelzmo has quit IRC | 01:32 | |
alex_xu_ | efried: I replied again https://review.openstack.org/#/c/554305/2, a long version, hope I explain clear that | 01:33 |
openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova-specs master: Complex (Anti)-Affinity Policies https://review.openstack.org/546925 | 01:35 |
*** tbachman has quit IRC | 01:36 | |
*** Tom-Tom has joined #openstack-nova | 01:41 | |
*** Tom-Tom has quit IRC | 01:42 | |
*** Tom-Tom has joined #openstack-nova | 01:42 | |
*** liverpooler has joined #openstack-nova | 01:48 | |
*** bhujay has joined #openstack-nova | 01:49 | |
*** lei-zh has joined #openstack-nova | 01:50 | |
*** jobewan has quit IRC | 01:51 | |
*** lei-zh has quit IRC | 01:51 | |
*** lei-zh has joined #openstack-nova | 01:51 | |
*** germs has joined #openstack-nova | 01:52 | |
*** germs has quit IRC | 01:52 | |
*** germs has joined #openstack-nova | 01:52 | |
*** germs has quit IRC | 01:57 | |
*** zhurong has quit IRC | 01:58 | |
*** voelzmo has joined #openstack-nova | 02:05 | |
phuongnh | join #openstack-dib | 02:09 |
*** bkopilov has quit IRC | 02:09 | |
*** suresh12 has quit IRC | 02:10 | |
*** vladikr has quit IRC | 02:12 | |
*** dave-mccowan has joined #openstack-nova | 02:15 | |
*** namnh has joined #openstack-nova | 02:16 | |
*** dikonoo has joined #openstack-nova | 02:21 | |
*** dikonoor has quit IRC | 02:21 | |
*** salv-orlando has joined #openstack-nova | 02:25 | |
*** gyee has quit IRC | 02:26 | |
*** yamamoto has joined #openstack-nova | 02:26 | |
*** yassine has quit IRC | 02:26 | |
*** yassine has joined #openstack-nova | 02:27 | |
openstackgerrit | jichenjc proposed openstack/nova master: Avoid live migrate to same host https://review.openstack.org/542689 | 02:28 |
*** liverpooler has quit IRC | 02:29 | |
*** salv-orlando has quit IRC | 02:30 | |
*** lei-zh has quit IRC | 02:31 | |
*** suresh12 has joined #openstack-nova | 02:36 | |
*** voelzmo has quit IRC | 02:38 | |
*** suresh12 has quit IRC | 02:41 | |
*** lei-zh has joined #openstack-nova | 02:42 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: NUMA-aware live migration https://review.openstack.org/552722 | 02:45 |
*** vladikr has joined #openstack-nova | 02:50 | |
*** bhujay has quit IRC | 02:50 | |
*** yingjun has quit IRC | 02:52 | |
*** gouthamr has quit IRC | 02:53 | |
*** voelzmo has joined #openstack-nova | 03:01 | |
*** fragatina has joined #openstack-nova | 03:02 | |
*** READ10 has quit IRC | 03:07 | |
*** fragatina has quit IRC | 03:08 | |
*** vladikr has quit IRC | 03:17 | |
*** vladikr has joined #openstack-nova | 03:18 | |
*** yangyapeng has quit IRC | 03:21 | |
*** yangyapeng has joined #openstack-nova | 03:21 | |
*** yangyape_ has joined #openstack-nova | 03:22 | |
*** zhurong has joined #openstack-nova | 03:23 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: NUMA-aware live migration https://review.openstack.org/552722 | 03:23 |
*** yingjun has joined #openstack-nova | 03:25 | |
*** yangyapeng has quit IRC | 03:25 | |
*** salv-orlando has joined #openstack-nova | 03:26 | |
*** takashin has quit IRC | 03:26 | |
*** fragatina has joined #openstack-nova | 03:30 | |
*** salv-orlando has quit IRC | 03:31 | |
*** yangyape_ has quit IRC | 03:34 | |
*** voelzmo has quit IRC | 03:34 | |
*** bkopilov has joined #openstack-nova | 03:40 | |
*** andreas_s has joined #openstack-nova | 03:42 | |
*** suresh12 has joined #openstack-nova | 03:43 | |
*** Tom-Tom has quit IRC | 03:44 | |
*** sree has joined #openstack-nova | 03:45 | |
*** Tom-Tom has joined #openstack-nova | 03:45 | |
*** diga has joined #openstack-nova | 03:46 | |
*** andreas_s has quit IRC | 03:46 | |
*** dave-mccowan has quit IRC | 03:49 | |
*** Tom-Tom has quit IRC | 03:50 | |
*** fragatina has quit IRC | 03:51 | |
*** voelzmo has joined #openstack-nova | 03:51 | |
*** lei-zh has quit IRC | 03:53 | |
*** germs has joined #openstack-nova | 03:53 | |
*** germs has quit IRC | 03:53 | |
*** germs has joined #openstack-nova | 03:53 | |
*** udesale has joined #openstack-nova | 03:55 | |
*** germs has quit IRC | 03:58 | |
*** fragatina has joined #openstack-nova | 03:59 | |
*** hongbin has quit IRC | 03:59 | |
*** bhujay has joined #openstack-nova | 04:01 | |
*** suresh12 has quit IRC | 04:03 | |
*** suresh12 has joined #openstack-nova | 04:06 | |
openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add host to API and Conductor https://review.openstack.org/556513 | 04:10 |
openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: [WIP] Record the host info in EventReporter https://review.openstack.org/556746 | 04:11 |
*** yingjun has quit IRC | 04:11 | |
*** voelzmo has quit IRC | 04:16 | |
*** yamahata has joined #openstack-nova | 04:24 | |
*** annp has quit IRC | 04:26 | |
*** salv-orlando has joined #openstack-nova | 04:27 | |
*** markvoelker has quit IRC | 04:27 | |
*** psachin has joined #openstack-nova | 04:27 | |
*** abhishekk has joined #openstack-nova | 04:28 | |
openstackgerrit | Jake Yip proposed openstack/nova master: Add --until_deleted_at to db archive_deleted_rows https://review.openstack.org/556751 | 04:29 |
*** salv-orlando has quit IRC | 04:32 | |
*** yingjun has joined #openstack-nova | 04:32 | |
*** AlexeyAbashkin has joined #openstack-nova | 04:33 | |
*** chyka has joined #openstack-nova | 04:34 | |
*** Tom-Tom has joined #openstack-nova | 04:34 | |
*** Tom-Tom has quit IRC | 04:36 | |
*** Tom-Tom has joined #openstack-nova | 04:36 | |
*** Tom-Tom has quit IRC | 04:37 | |
*** Tom-Tom has joined #openstack-nova | 04:37 | |
*** AlexeyAbashkin has quit IRC | 04:37 | |
*** chyka has quit IRC | 04:39 | |
*** zhurong has quit IRC | 04:39 | |
*** takashin has joined #openstack-nova | 04:40 | |
*** janki has joined #openstack-nova | 04:48 | |
*** yingjun has quit IRC | 04:50 | |
*** diga has quit IRC | 04:56 | |
*** Dinesh_Bhor has quit IRC | 05:01 | |
*** Dinesh_Bhor has joined #openstack-nova | 05:03 | |
*** yamamoto_ has joined #openstack-nova | 05:07 | |
openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Consider nested RPs in get_all_with_shared https://review.openstack.org/556450 | 05:10 |
openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: support shared and nested allocation candidates https://review.openstack.org/556514 | 05:10 |
*** yamamoto has quit IRC | 05:11 | |
*** dikonoo has quit IRC | 05:11 | |
*** dikonoo has joined #openstack-nova | 05:12 | |
*** salv-orlando has joined #openstack-nova | 05:15 | |
*** ratailor has joined #openstack-nova | 05:18 | |
*** fragatina has quit IRC | 05:20 | |
*** fragatina has joined #openstack-nova | 05:20 | |
*** ratailor is now known as rtailor | 05:21 | |
*** tetsuro has joined #openstack-nova | 05:23 | |
*** markvoelker has joined #openstack-nova | 05:28 | |
*** lei-zh has joined #openstack-nova | 05:31 | |
*** gjayavelu has joined #openstack-nova | 05:37 | |
*** yingjun has joined #openstack-nova | 05:38 | |
*** moshele has joined #openstack-nova | 05:41 | |
*** suresh12 has quit IRC | 05:42 | |
*** takashin has quit IRC | 05:47 | |
*** ttsiouts_ has joined #openstack-nova | 05:49 | |
*** lei-zh has quit IRC | 05:49 | |
*** lei-zh has joined #openstack-nova | 05:50 | |
*** ttsiouts_ has quit IRC | 05:51 | |
*** germs has joined #openstack-nova | 05:54 | |
*** germs has quit IRC | 05:54 | |
*** germs has joined #openstack-nova | 05:54 | |
*** tbachman_ has joined #openstack-nova | 05:54 | |
*** tbachman_ is now known as tbachman | 05:56 | |
*** germs has quit IRC | 05:59 | |
*** sree has quit IRC | 05:59 | |
*** sree has joined #openstack-nova | 05:59 | |
*** moshele has quit IRC | 06:00 | |
*** yamamoto has joined #openstack-nova | 06:01 | |
*** yingjun has quit IRC | 06:03 | |
*** yamamoto_ has quit IRC | 06:04 | |
*** sree has quit IRC | 06:04 | |
*** tbachman has quit IRC | 06:07 | |
*** annp has joined #openstack-nova | 06:12 | |
*** lajoskatona has joined #openstack-nova | 06:13 | |
*** psachin has quit IRC | 06:14 | |
*** sree has joined #openstack-nova | 06:15 | |
*** moshele has joined #openstack-nova | 06:16 | |
*** namnh_ has joined #openstack-nova | 06:17 | |
*** bandini has joined #openstack-nova | 06:18 | |
*** sree has quit IRC | 06:19 | |
*** namnh has quit IRC | 06:20 | |
*** namnh has joined #openstack-nova | 06:21 | |
*** namnh_ has quit IRC | 06:21 | |
*** namnh_ has joined #openstack-nova | 06:22 | |
*** chyka has joined #openstack-nova | 06:23 | |
openstackgerrit | Naichuan Sun proposed openstack/nova master: xenapi(N-R-P):Get vgpu info from `allocations` https://review.openstack.org/521717 | 06:23 |
*** psachin has joined #openstack-nova | 06:23 | |
*** moshele has quit IRC | 06:24 | |
*** namnh has quit IRC | 06:26 | |
*** sahid has joined #openstack-nova | 06:27 | |
*** chyka has quit IRC | 06:27 | |
*** Spaz-Home has joined #openstack-nova | 06:32 | |
*** Spazmotic has quit IRC | 06:34 | |
openstackgerrit | Roman Dobosz proposed openstack/nova master: Remove server group sched filter support caching https://review.openstack.org/529200 | 06:39 |
openstackgerrit | Roman Dobosz proposed openstack/nova master: get instance group's aggregate associations https://review.openstack.org/531243 | 06:39 |
openstackgerrit | Roman Dobosz proposed openstack/nova master: Support aggregate affinity filters https://review.openstack.org/529201 | 06:39 |
openstackgerrit | Roman Dobosz proposed openstack/nova master: Add nodes to group hosts to be checked against aggregation https://review.openstack.org/556761 | 06:39 |
openstackgerrit | Roman Dobosz proposed openstack/nova master: Added weight for aggregate soft (anti) affinity. https://review.openstack.org/556762 | 06:39 |
openstackgerrit | sahid proposed openstack/nova-specs master: update: isolate guests emulthreads on CONF.cpu_shared_set https://review.openstack.org/511188 | 06:42 |
*** tuanla____ has joined #openstack-nova | 06:47 | |
*** afaranha has joined #openstack-nova | 06:49 | |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: Initial change set of z/VM driver https://review.openstack.org/523387 | 06:50 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: Spawn and destroy function of z/VM driver https://review.openstack.org/527658 | 06:50 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add snapshot function https://review.openstack.org/534240 | 06:50 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add power actions https://review.openstack.org/543340 | 06:50 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add get console output https://review.openstack.org/543344 | 06:50 |
*** moshele has joined #openstack-nova | 06:51 | |
*** pcaruana has joined #openstack-nova | 06:53 | |
*** moshele has quit IRC | 06:57 | |
yikun | @jichen, https://review.openstack.org/#/c/556513/ some replies on patch, take a look when you have time. :) thanks | 06:59 |
*** takashin has joined #openstack-nova | 07:01 | |
*** gjayavelu has quit IRC | 07:01 | |
*** jaosorior has quit IRC | 07:02 | |
*** danpawlik has joined #openstack-nova | 07:03 | |
*** danpawlik has quit IRC | 07:05 | |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Allow to specify granular CPU feature flags https://review.openstack.org/534384 | 07:05 |
jichen | yikun: yes, I am looking at that patch now , thanks for notification | 07:10 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/548772 | 07:12 |
yikun | jichen, OK, thanks | 07:12 |
*** phuongnh has quit IRC | 07:14 | |
*** phuongnh has joined #openstack-nova | 07:15 | |
*** tesseract has joined #openstack-nova | 07:15 | |
*** yingjun has joined #openstack-nova | 07:17 | |
kashyap | johnthetubaguy: Morning, when you are around, would appreciate if you can take a gander: https://review.openstack.org/#/c/534384/ | 07:17 |
kashyap | johnthetubaguy: Tests pass (locally; Zuul is yet to give ACK), backport concerns addressed, release note in place. | 07:18 |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Noauth should also use request_id from compute_req_id.py https://review.openstack.org/555266 | 07:18 |
*** yamamoto_ has joined #openstack-nova | 07:22 | |
*** sshwarts has joined #openstack-nova | 07:23 | |
*** yamamoto has quit IRC | 07:25 | |
*** rcernin has quit IRC | 07:25 | |
*** damien_r has joined #openstack-nova | 07:25 | |
openstackgerrit | jichenjc proposed openstack/nova master: Avoid live migrate to same host https://review.openstack.org/542689 | 07:25 |
openstackgerrit | Takashi NATSUME proposed openstack/nova-specs master: Change a validation in creating a server group https://review.openstack.org/546484 | 07:26 |
*** cdent has joined #openstack-nova | 07:29 | |
*** ragiman has joined #openstack-nova | 07:30 | |
*** yikun_jiang has joined #openstack-nova | 07:30 | |
*** andreas_s has joined #openstack-nova | 07:33 | |
*** yikun has quit IRC | 07:35 | |
*** tianhui_ has joined #openstack-nova | 07:36 | |
*** salv-orlando has quit IRC | 07:36 | |
*** salv-orlando has joined #openstack-nova | 07:37 | |
*** tianhui has quit IRC | 07:38 | |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Filter allocation candidates by forbidden traits in db https://review.openstack.org/556660 | 07:41 |
*** salv-orlando has quit IRC | 07:42 | |
*** gongysh has joined #openstack-nova | 07:43 | |
openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Consider nested RPs in get_all_with_shared https://review.openstack.org/556450 | 07:44 |
openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Support shared and nested allocation candidates https://review.openstack.org/556514 | 07:44 |
*** psachin has quit IRC | 07:46 | |
bauzas | good morning Stackers | 07:47 |
bauzas | remember, today is a specs review day | 07:47 |
*** dims_ has joined #openstack-nova | 07:48 | |
*** sree_ has joined #openstack-nova | 07:49 | |
*** dims has quit IRC | 07:49 | |
*** sree_ has quit IRC | 07:49 | |
*** sree has joined #openstack-nova | 07:49 | |
*** xinliang has joined #openstack-nova | 07:51 | |
*** xinliang has quit IRC | 07:51 | |
*** xinliang has joined #openstack-nova | 07:51 | |
*** alexchadin has joined #openstack-nova | 07:51 | |
*** AlexeyAbashkin has joined #openstack-nova | 07:51 | |
*** alexchadin has quit IRC | 07:52 | |
*** alexchadin has joined #openstack-nova | 07:53 | |
*** ralonsoh has joined #openstack-nova | 07:53 | |
openstackgerrit | Konstantinos Samaras-Tsakiris proposed openstack/nova master: Add `hide_hypervisor_id` flavor extra_spec https://review.openstack.org/555861 | 07:53 |
*** kholkina has joined #openstack-nova | 07:53 | |
*** jaosorior has joined #openstack-nova | 07:53 | |
*** zhurong has joined #openstack-nova | 07:53 | |
*** germs has joined #openstack-nova | 07:54 | |
*** germs has quit IRC | 07:54 | |
*** germs has joined #openstack-nova | 07:54 | |
openstackgerrit | Naichuan Sun proposed openstack/nova master: (WIP)xenapi(N-R-P): Add API to support compute node resource provider update and create https://review.openstack.org/521041 | 07:56 |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Noauth should also use request_id from compute_req_id.py https://review.openstack.org/555266 | 07:57 |
*** pooja_jadhav has quit IRC | 07:57 | |
*** pooja_jadhav has joined #openstack-nova | 07:57 | |
openstackgerrit | Naichuan Sun proposed openstack/nova master: (WIP)xenapi(N-R-P): Add API to support compute node resource provider update and create https://review.openstack.org/521041 | 07:58 |
*** germs has quit IRC | 07:59 | |
*** ktibi has joined #openstack-nova | 07:59 | |
*** vladikr has quit IRC | 08:00 | |
*** lucas-afk is now known as lucasagomes | 08:01 | |
kholkina | jichen, hi! could you please take a look at https://review.openstack.org/#/c/547964/ | 08:01 |
*** mdnadeem has joined #openstack-nova | 08:01 | |
jichen | kholkina: ok, right now | 08:02 |
bauzas | jianghuaw_: naichuans: around ? | 08:03 |
*** salv-orlando has joined #openstack-nova | 08:04 | |
*** phuongnh has quit IRC | 08:07 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Parameter verification for servers.inc (1/3) https://review.openstack.org/528201 | 08:07 |
*** tssurya has joined #openstack-nova | 08:07 | |
*** derekh has joined #openstack-nova | 08:09 | |
*** tetsuro has left #openstack-nova | 08:12 | |
*** bhujay has quit IRC | 08:13 | |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: Initial change set of z/VM driver https://review.openstack.org/523387 | 08:14 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: Spawn and destroy function of z/VM driver https://review.openstack.org/527658 | 08:14 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add snapshot function https://review.openstack.org/534240 | 08:14 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add power actions https://review.openstack.org/543340 | 08:14 |
openstackgerrit | jichenjc proposed openstack/nova master: z/VM Driver: add get console output https://review.openstack.org/543344 | 08:14 |
jichen | hi sahid, can you take a look at https://review.openstack.org/#/c/523387 and see whether it's what you want? thanks | 08:15 |
*** alexchad_ has joined #openstack-nova | 08:20 | |
*** alexchadin has quit IRC | 08:20 | |
openstackgerrit | jichenjc proposed openstack/nova master: WIP: remove ec2 in service and cmd https://review.openstack.org/556778 | 08:23 |
*** ccamacho has joined #openstack-nova | 08:23 | |
*** ccamacho has quit IRC | 08:23 | |
sahid | jichen: hello, yes, thanks that looks what i had in my head i will check that deeper soon | 08:24 |
jichen | sahid: thanks for thorough review and appreciate your further comments, thanks | 08:25 |
sahid | jichen: i made a simple request | 08:27 |
*** mvk has quit IRC | 08:27 | |
sahid | why you did not have moved zVMconnectorRequestHandler code in Hypervisor? | 08:27 |
*** ccamacho has joined #openstack-nova | 08:27 | |
*** ccamacho is now known as ccamacho|PTO | 08:28 | |
gibi | bauzas: I read you discussion about matching the specific NUMA node selected by placement to the one that will be selected by the virt driver | 08:29 |
gibi | bauzas: and it sounds very similar to what I have with network RP selected by the placement vs. network selected by neutron agent on the physical level | 08:30 |
jianghuaw_ | bauzas, hi. | 08:31 |
jianghuaw_ | bauzas, what's up? | 08:33 |
gibi | bauzas: in the neturon case we think about communicating the RP selection to neutron during the port binding to ensure neutron has the necessary information | 08:33 |
bauzas | gibi: for the moment, we agreed on not having a problem | 08:33 |
bauzas | gibi: because the NUMA filter would get the same resources | 08:34 |
bauzas | jianghuaw_: just a question about vGPUs | 08:35 |
bauzas | jianghuaw_: given each pGPU is having multiple types and given each type is having specific vGPU number, I think using nested RPs for us would be something like : | 08:36 |
openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Record the host info in EventReporter https://review.openstack.org/556746 | 08:36 |
*** mdnadeem_ has joined #openstack-nova | 08:36 | |
bauzas | jianghuaw_: root RP(compute) -> child RP (pGPU) -> child RP (GPU type) | 08:36 |
bauzas | jianghuaw_: WDYF ? | 08:36 |
*** mdnadeem has quit IRC | 08:37 | |
jianghuaw_ | I think we still have to restrict each pGPU only support one vGPU type. | 08:37 |
openstackgerrit | Silvan Kaiser proposed openstack/nova master: Exec systemd-run with privileges in Quobyte driver https://review.openstack.org/554195 | 08:37 |
gibi | bauzas: OK, then good for you | 08:37 |
jianghuaw_ | so root RP(compute) -> child RP (pGPU)<vgpu inventory> | 08:37 |
bauzas | jianghuaw_: if so, we should be asking operators to say which type for each PCI device, right? | 08:38 |
bauzas | if we want to support multiple types | 08:38 |
*** alexchad_ has quit IRC | 08:38 | |
jianghuaw_ | I guess yes. | 08:38 |
bauzas | and also, I'm not sure that Intel has the same problem than nVidia where only one type is possible for each pGPU | 08:38 |
jianghuaw_ | For xen, it's root RP(compute) -> child RP (pGPU group)<vgpu inventory> | 08:39 |
bauzas | but each group is only having one type, right? | 08:39 |
bauzas | if so, how do you say which type for each group ? | 08:39 |
*** alexchadin has joined #openstack-nova | 08:39 | |
*** gyankum has quit IRC | 08:41 | |
*** gyan_ has joined #openstack-nova | 08:41 | |
jianghuaw_ | all of the same type PGPUs will be put into the same group. | 08:41 |
bauzas | not sure I understand | 08:41 |
bauzas | each physical device is having multiple possible types | 08:41 |
jianghuaw_ | so still depending the configure option to restrict each grpu only has one type enabled. | 08:41 |
*** rmart04 has joined #openstack-nova | 08:42 | |
jianghuaw_ | But if there are multiple types of PGPUs exists in single compute. it can support different types. | 08:42 |
bauzas | say I'm doing enabled_vgpu_types=nvidia-1,nvidia-2 | 08:43 |
bauzas | (or the name you use in Xen) | 08:43 |
bauzas | if I have 2 pGPUs, each one supporting both types, what Xen will have ? | 08:43 |
bauzas | two pGPU groups ? | 08:43 |
jianghuaw_ | Also we can ask operators to customize the pGPU groups. Then we can put some pgpus into group1 and then other others be group2. | 08:43 |
jianghuaw_ | and each can enable different vGPU types. | 08:43 |
*** gyan__ has joined #openstack-nova | 08:44 | |
*** yamahata has quit IRC | 08:44 | |
jianghuaw_ | switching to a meeting. | 08:45 |
*** mdbooth has joined #openstack-nova | 08:46 | |
*** gyan_ has quit IRC | 08:47 | |
jichen | sahid: sorry for delay, I replied in the patch and saw your question here | 08:47 |
bauzas | jianghuaw_: mmmm, not sure I like that | 08:47 |
*** sahid has quit IRC | 08:51 | |
*** mvk has joined #openstack-nova | 08:55 | |
kashyap | melwitt: When you're back: the ListOpt() trick works -- https://review.openstack.org/#/c/534384/ | 08:55 |
*** Dinesh_Bhor has quit IRC | 08:56 | |
*** mdnadeem_ has quit IRC | 08:59 | |
*** mdnadeem has joined #openstack-nova | 08:59 | |
*** zhurong has quit IRC | 09:00 | |
*** jogo has quit IRC | 09:03 | |
openstackgerrit | Theodoros Tsioutsias proposed openstack/nova-specs master: Add PENDING vm state https://review.openstack.org/554212 | 09:04 |
*** rmart04 has quit IRC | 09:06 | |
*** rmart04 has joined #openstack-nova | 09:06 | |
*** sahid has joined #openstack-nova | 09:09 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs https://review.openstack.org/552924 | 09:10 |
*** afaranha has quit IRC | 09:11 | |
*** hshiina has quit IRC | 09:13 | |
*** bhujay has joined #openstack-nova | 09:18 | |
openstackgerrit | Konstantinos Samaras-Tsakiris proposed openstack/nova master: Add `hide_hypervisor_id` flavor extra_spec https://review.openstack.org/555861 | 09:18 |
ktibi | Hi, how can I disable the compatibility check for live migration ? because I have two CPU model : Nehalem & Nehalem-IBRS and migration fail :/ | 09:24 |
*** XueFeng has quit IRC | 09:28 | |
*** XueFeng has joined #openstack-nova | 09:29 | |
*** takashin has left #openstack-nova | 09:31 | |
*** mvk has quit IRC | 09:36 | |
openstackgerrit | Tetiana Lashchova proposed openstack/nova-specs master: Allow modification of user-data via the server update https://review.openstack.org/547964 | 09:36 |
*** mvk has joined #openstack-nova | 09:37 | |
*** jogo has joined #openstack-nova | 09:39 | |
jianghuaw_ | bauzas, came back from a call meeting. I understood your concern. The problem is that we can't handle the dynamically changing vGPU capacities. | 09:41 |
openstackgerrit | Bhagyashri Shewale proposed openstack/nova-specs master: Disallow rotation parameter 0 for 'createBackup' API https://review.openstack.org/511825 | 09:41 |
jianghuaw_ | I mean the available vGPUs for one type will be impacted by other types belongs to the same pGPU. | 09:41 |
*** yingjun has quit IRC | 09:42 | |
jianghuaw_ | that's why we have to keep the restriction that each PGPU (or pgpu group) can only expose one vGPU type. | 09:42 |
kaisers1 | mikal: I'm not sure which direction to go in https://review.openstack.org/#/c/554195/ , could you pls review regarding Stephens comments? | 09:42 |
bauzas | jianghuaw_: sure but I don't think it's a problem | 09:43 |
bauzas | jianghuaw_: at least for libvirt, what I know is that if I'm providing multiple inventories, then when I'll create the first mdev, it'll automatically update the inventories of the other types to be total=0 | 09:44 |
*** Tom-Tom has quit IRC | 09:44 | |
jianghuaw_ | bauzas, will it cause conflict? | 09:45 |
jianghuaw_ | in the case multiple vGPUs have been allocated from multiple inventories | 09:46 |
jianghuaw_ | Before create the first mdev, all inventories will have >0 vGPUs available. Right? | 09:47 |
jianghuaw_ | If yes, it's possible to allocate vGPUs from different inventories. | 09:48 |
jianghuaw_ | Then the first mdev creation will result into the other inventories' total be 0. the allocatoin for the other vGPUs will fail. right? | 09:49 |
openstackgerrit | Matthew Booth proposed openstack/nova-specs master: Add serial numbers for local disks https://review.openstack.org/556565 | 09:50 |
*** sdague has joined #openstack-nova | 09:51 | |
openstackgerrit | Silvan Kaiser proposed openstack/nova master: Exec systemd-run with privileges in Quobyte driver https://review.openstack.org/554195 | 09:52 |
*** jichen has quit IRC | 09:55 | |
stephenfin | kashyap: This of any interest? Abandoning if not https://review.openstack.org/#/c/348394/ | 09:55 |
* kashyap clicks | 09:55 | |
*** germs has joined #openstack-nova | 09:55 | |
kashyap | stephenfin: That's another piecemeal way of fixing the current mess of handling different OVMF binaries | 09:56 |
kashyap | stephenfin: Can be abandoned, IMHO. And we should handle it globally for all distributions | 09:57 |
* kashyap proposed a spec for that | 09:57 | |
kashyap | https://review.openstack.org/#/c/506720/ | 09:57 |
kashyap | stephenfin: Which is in turn a bit predicated on this RFC I started for libvirt and QEMU: [RFC] Defining firmware (OVMF, et al) metadata format & file | 09:58 |
kashyap | https://lists.nongnu.org/archive/html/qemu-devel/2018-03/msg01978.html | 09:58 |
stephenfin | kashyap: Cool, done | 09:58 |
* kashyap goes to look about bumping min libvirt / QEMU versions for 'Solar' release | 09:58 | |
*** chyka has joined #openstack-nova | 09:58 | |
*** yassine has quit IRC | 10:00 | |
*** germs has quit IRC | 10:00 | |
stephenfin | sahid: Any chance you could abandon these, now that bauzas' vGPU series has merged? https://review.openstack.org/#/q/topic:pci-mdev-support-compaq+(status:open+OR+status:merged) | 10:01 |
*** chyka has quit IRC | 10:02 | |
*** mdnadeem_ has joined #openstack-nova | 10:03 | |
*** gyan_ has joined #openstack-nova | 10:03 | |
*** ratailor_ has joined #openstack-nova | 10:03 | |
*** abhishekk_ has joined #openstack-nova | 10:03 | |
*** udesale_ has joined #openstack-nova | 10:03 | |
*** abhishekk has quit IRC | 10:04 | |
*** rtailor has quit IRC | 10:04 | |
*** mdnadeem has quit IRC | 10:04 | |
*** gyan__ has quit IRC | 10:05 | |
*** udesale__ has joined #openstack-nova | 10:05 | |
*** gyan__ has joined #openstack-nova | 10:05 | |
*** ratailor__ has joined #openstack-nova | 10:06 | |
*** udesale has quit IRC | 10:07 | |
*** udesale_ has quit IRC | 10:08 | |
*** gyan_ has quit IRC | 10:08 | |
*** abhishekk_ has quit IRC | 10:08 | |
*** ratailor_ has quit IRC | 10:08 | |
*** mdnadeem_ has quit IRC | 10:08 | |
*** yamamoto_ has quit IRC | 10:10 | |
*** avolkov has joined #openstack-nova | 10:10 | |
*** gongysh has quit IRC | 10:11 | |
*** derekh is now known as derekh_afk | 10:13 | |
*** zhaochao has quit IRC | 10:14 | |
sahid | stephenfin: i'm expecting gerrit to abandon them automatically at some point | 10:15 |
stephenfin | Ah, it doesn't do that. Someone (typically sdague in the past) had to run a script or something | 10:16 |
*** namnh_ has quit IRC | 10:18 | |
*** mdnadeem_ has joined #openstack-nova | 10:20 | |
*** abhishekk_ has joined #openstack-nova | 10:20 | |
*** abhishekk_ is now known as abhishekk | 10:21 | |
kashyap | stephenfin: Question for you here: https://review.openstack.org/#/c/530924/ | 10:27 |
*** gongysh has joined #openstack-nova | 10:28 | |
kashyap | Also, sigh (existing fault), naming nuisance: 'disk_cachemodes' (should be 'disk_cache_modes') | 10:28 |
kashyap | Changing such things at this point would be futile (for OCD's sake), as it might break scripts, & other tools, etc :-( | 10:29 |
*** sahid has quit IRC | 10:30 | |
*** zhurong has joined #openstack-nova | 10:31 | |
*** lei-zh has quit IRC | 10:33 | |
*** gyan__ has quit IRC | 10:34 | |
*** mdnadeem has joined #openstack-nova | 10:36 | |
*** bkopilov has quit IRC | 10:36 | |
*** gyan__ has joined #openstack-nova | 10:36 | |
*** vivsoni has quit IRC | 10:37 | |
*** mdnadeem_ has quit IRC | 10:38 | |
*** vivsoni has joined #openstack-nova | 10:38 | |
*** moshele has joined #openstack-nova | 10:46 | |
*** moshele has quit IRC | 10:48 | |
*** udesale__ has quit IRC | 10:51 | |
*** yamamoto has joined #openstack-nova | 10:52 | |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Filter resource providers by forbidden traits in db https://review.openstack.org/556472 | 10:56 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Filter allocation candidates by forbidden traits in db https://review.openstack.org/556660 | 10:56 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Parse forbidden traits in query strings https://review.openstack.org/556819 | 10:56 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Support forbidden traits in API https://review.openstack.org/556820 | 10:56 |
cdent | Okay, with that done, I can brew up to start the spec dive | 10:57 |
*** abhishekk has quit IRC | 10:58 | |
*** tiendc has quit IRC | 10:58 | |
*** moshele has joined #openstack-nova | 11:00 | |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Scheduling Optimization: Remove cell0 from the list of candidates https://review.openstack.org/556821 | 11:03 |
*** AlexeyAbashkin has quit IRC | 11:05 | |
sean-k-mooney | melwitt: not that my vote counts in the actul ballot but i had assumed efried had a +/-1.5 for a while so i'd also be +1 on the proposal to add you to the core team. | 11:05 |
*** janki has quit IRC | 11:06 | |
*** AlexeyAbashkin has joined #openstack-nova | 11:07 | |
*** lucasagomes is now known as lucas-hungry | 11:08 | |
*** alexchadin has quit IRC | 11:09 | |
kashyap | sean-k-mooney: What do you mean 1.5? | 11:11 |
kashyap | (Also: IMHO, even if your vote doesn't "count" by some measure, active contributors such as you expressing their views is equally important, in my books.) | 11:12 |
sean-k-mooney | kashyap: that while efried does not have +2 right currently there are some area of the code where his +/- 1 are treated as +/- 2 due to the experience he has demonstrated in that area | 11:12 |
kashyap | I see. I guessed as much, just wanted to double-confirm. | 11:12 |
kashyap | sean-k-mooney: Unrelated bait (apologies): I think this is ready: https://review.openstack.org/#/c/534384/ | 11:13 |
sean-k-mooney | kashyap: it does not "count" because the vote is between members of the core team rather then an open vote. but yes i know its still valuble for active memeber to express there support/decent | 11:13 |
sean-k-mooney | kashyap: oh yes i skimmed it yesterday | 11:14 |
openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Record the host info in EventReporter https://review.openstack.org/556746 | 11:14 |
kashyap | sean-k-mooney: Thanks! Tests pass, backport concerns addressed, rel note in place (removed needless verbosity per dansmith's review) | 11:14 |
sean-k-mooney | i see you got the release nots job passing | 11:15 |
kashyap | Yep! I got the rST dark magic right | 11:15 |
sean-k-mooney | ill quickly review again but last time i looked it seamed fine to me | 11:15 |
kashyap | sean-k-mooney: Thanks, the core change is quite small: I limit the 'choices' keyword arg to one value (PCID) | 11:16 |
sean-k-mooney | i saw dans comment and kind of aggreed but not strongly enough to ask for a respin | 11:16 |
kashyap | sean-k-mooney: Yeah; I reworded it, and moved the CPU models info into the config file | 11:16 |
kashyap | As Operators can assume that it can be applied to all models. | 11:16 |
kashyap | As we know, some virtual CPU models include it (like the Haswell variants), some don't (Nehalem, etc). | 11:16 |
*** Zames has joined #openstack-nova | 11:18 | |
sean-k-mooney | item_type=types.String( | 11:19 |
sean-k-mooney | choices=['pcid'] | 11:19 |
sean-k-mooney | ) | 11:19 |
sean-k-mooney | thats new to me | 11:19 |
sean-k-mooney | kashyap: is that syntax documented in oslo somewhere? | 11:20 |
kashyap | sean-k-mooney: Not quote, I found it from stephenfin's use here: https://github.com/openstack/nova/blob/cd15c3d/nova/conf/vnc.py#L226,L232 | 11:20 |
kashyap | s/quote/quite/ | 11:20 |
kashyap | sean-k-mooney: I only skimmed the oslo_config/cfg.py, though | 11:20 |
kashyap | https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py | 11:21 |
sean-k-mooney | kashyap: well i have used choice elements for string fields before just never lists | 11:21 |
kashyap | sean-k-mooney: It's actually documented here: https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L52 | 11:21 |
*** sahid has joined #openstack-nova | 11:22 | |
sean-k-mooney | ya so its not that you are using a listOpt that has a choice filed, you are instead create a list of sting enums which have the chioce element | 11:24 |
sean-k-mooney | but it works | 11:24 |
*** vivsoni_ has joined #openstack-nova | 11:24 | |
kashyap | Exactly :P It's still using the 'String' type | 11:24 |
*** vivsoni has quit IRC | 11:24 | |
openstackgerrit | Elod Illes proposed openstack/nova stable/ocata: WIP: Functional test for regression bug #1713783 https://review.openstack.org/505160 | 11:26 |
openstack | bug 1713783 in OpenStack Compute (nova) ocata "After failed evacuation the recovered source compute tries to delete the instance" [High,In progress] https://launchpad.net/bugs/1713783 - Assigned to Balazs Gibizer (balazs-gibizer) | 11:26 |
*** Zames has quit IRC | 11:27 | |
*** yamamoto has quit IRC | 11:30 | |
kashyap | sean-k-mooney: Thanks for the review. Much appreciated | 11:32 |
*** sahid has quit IRC | 11:33 | |
sean-k-mooney | kashyap: no worries. i dont get enough time as i like to review but always feel free to ping me with a patch if you have one | 11:33 |
*** sahid has joined #openstack-nova | 11:33 | |
kashyap | sean-k-mooney: Understood. I don't spend 100% time here either. I think I mostly know what topics pique your interest, will do :-) | 11:33 |
*** openstackgerrit has quit IRC | 11:33 | |
*** yamamoto has joined #openstack-nova | 11:34 | |
*** dtantsur|afk is now known as dtantsur | 11:34 | |
*** gyan__ has quit IRC | 11:34 | |
*** bkopilov has joined #openstack-nova | 11:34 | |
*** yamamoto has quit IRC | 11:38 | |
*** yassine has joined #openstack-nova | 11:38 | |
*** sree has quit IRC | 11:41 | |
*** udesale has joined #openstack-nova | 11:47 | |
*** openstackgerrit has joined #openstack-nova | 11:48 | |
openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Noauth should also use request_id from compute_req_id.py https://review.openstack.org/555266 | 11:48 |
*** yamamoto has joined #openstack-nova | 11:54 | |
*** sree has joined #openstack-nova | 11:54 | |
*** gongysh has quit IRC | 11:54 | |
*** yangyapeng has joined #openstack-nova | 11:55 | |
*** germs has joined #openstack-nova | 11:56 | |
*** germs has quit IRC | 11:56 | |
*** germs has joined #openstack-nova | 11:56 | |
*** sree has quit IRC | 11:58 | |
*** tuanla____ has quit IRC | 12:00 | |
*** germs has quit IRC | 12:01 | |
*** pchavva has joined #openstack-nova | 12:03 | |
*** ratailor__ is now known as ratailor | 12:03 | |
*** openstackgerrit has quit IRC | 12:04 | |
*** sree has joined #openstack-nova | 12:04 | |
*** vladikr has joined #openstack-nova | 12:05 | |
*** alexchadin has joined #openstack-nova | 12:08 | |
*** janki has joined #openstack-nova | 12:08 | |
*** openstackgerrit has joined #openstack-nova | 12:09 | |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527 | 12:09 |
*** sree has quit IRC | 12:09 | |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Add --enable and --disable options to nova-manage update_cell https://review.openstack.org/555416 | 12:11 |
*** liverpooler has joined #openstack-nova | 12:12 | |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Update the cells FAQs and scheduler maintenance docs. https://review.openstack.org/556459 | 12:12 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Update the cells FAQs and scheduler maintenance docs. https://review.openstack.org/556459 | 12:13 |
*** edmondsw has joined #openstack-nova | 12:13 | |
*** sean-k-mooney has quit IRC | 12:14 | |
*** lucas-hungry is now known as lucasagomes | 12:16 | |
*** sree has joined #openstack-nova | 12:17 | |
*** zhurong has quit IRC | 12:27 | |
*** markvoelker has quit IRC | 12:28 | |
*** markvoelker has joined #openstack-nova | 12:28 | |
*** chyka has joined #openstack-nova | 12:28 | |
*** chyka has quit IRC | 12:33 | |
bhagyashris | johnthetubaguy: Hi, I have proposed revised (as per the discussion in Dublin PTG) spec: https://review.openstack.org/#/c/511825/2 ( | 12:35 |
bhagyashris | Disallow rotation parameter 0 for 'createBackup' API) Request you to review the same. | 12:35 |
*** yingjun has joined #openstack-nova | 12:39 | |
*** voelzmo has joined #openstack-nova | 12:40 | |
*** dave-mccowan has joined #openstack-nova | 12:41 | |
*** READ10 has joined #openstack-nova | 12:42 | |
*** odyssey4me has quit IRC | 12:43 | |
*** odyssey4me has joined #openstack-nova | 12:43 | |
* efried waves | 12:46 | |
* efried prepares to put on blinders to code reviews and focus only on specs. | 12:47 | |
*** derekh_afk is now known as derekh | 12:48 | |
*** _ix has quit IRC | 12:49 | |
*** yikun_jiang has quit IRC | 12:50 | |
*** yikun_jiang has joined #openstack-nova | 12:51 | |
* gibi waves back to efried | 12:51 | |
*** voelzmo has quit IRC | 12:53 | |
efried | alex_xu_: Responded. | 12:54 |
bauzas | sorry folks, was at sports | 12:54 |
*** moshele has quit IRC | 12:54 | |
*** gjayavelu has joined #openstack-nova | 12:57 | |
bauzas | sahid: thanks for having reviewed https://review.openstack.org/#/c/552924/ ! | 12:57 |
bauzas | sahid: you mean that page memories are different from the main memory ? | 12:58 |
efried | stephenfin: Just noticed you seem to have split personality according to stackalytics. It looks like your reviews go to one ID, your commits to the other. | 12:58 |
*** suresh12 has joined #openstack-nova | 12:59 | |
stephenfin | efried: Oh, really? | 12:59 |
efried | http://stackalytics.com/report/contribution/nova/90 -- sort by name and find thyself. | 12:59 |
bauzas | sahid: sorry, I'm not an expect for that, I just looked at https://docs.openstack.org/nova/pike/admin/huge-pages.html | 12:59 |
sahid | bauzas: i imagine when you are talking about main memroy, you are talking about small pages | 12:59 |
sahid | so if you allocate some huge pages, the small pages avail are going to decrease | 13:00 |
bauzas | sahid: okay, thanks | 13:00 |
bauzas | sahid: so, main memory and pages memories are different ? | 13:00 |
stephenfin | efried: Ah, one is using my email address, the other my launchpad ID. Guess I need to sync those somehow | 13:01 |
stephenfin | Or not. Meh | 13:01 |
bauzas | I thought it was just a size for eahc | 13:01 |
bauzas | stephenfin: efried: meh, when I see stackalytics, I'm sad :( | 13:01 |
sahid | bauzas: not sure what you mean but on a system that use hugepages that does not look right to tallk about main memory | 13:02 |
bauzas | because I don't have a lot of time for reviewing :( | 13:02 |
efried | I was like, wait, stephenfin has one commit in the last 90 days?? I *know* that ain't right. | 13:02 |
*** jaypipes has joined #openstack-nova | 13:02 | |
sahid | there are small pages and huge pages | 13:02 |
openstackgerrit | Theodoros Tsioutsias proposed openstack/nova-specs master: Enable rebuild for instances in cell0 https://review.openstack.org/554218 | 13:02 |
bauzas | sahid: okay, just to be clear, say I'm asking for a 1G page, it's different from asking to use 1GB for the main memory? | 13:02 |
openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: NUMA-aware live migration https://review.openstack.org/552722 | 13:02 |
bauzas | a different resource ? | 13:03 |
sahid | bauzas: yes | 13:03 |
bauzas | haha, thanks! | 13:03 |
*** suresh12 has quit IRC | 13:03 | |
bauzas | okay, I'll modify that then | 13:03 |
*** sree has quit IRC | 13:04 | |
artom | bauzas only ^^^ for me | 13:05 |
Spaz-Home | Good luck on your specs day, my friends. | 13:06 |
stephenfin | sahid: RE: your concerns on the NUMA-aware vSwitches, I totally agree. It's a horrible hack | 13:08 |
stephenfin | sahid: However, it's the best we can get right now. Far as I can tell, there's no deterministic way to get this information from every vSwitch (at the moment, at least) and adding that would take some neutron plugins into compute-driver territory | 13:09 |
*** lyan has joined #openstack-nova | 13:09 | |
*** lyan is now known as Guest45643 | 13:10 | |
stephenfin | NUMA affinity for the OVS case is mostly driven from the PCI devices used, as from what I can tell OVS doesn't expose a "give me the NUMA affinity for this PCI device I've attached to my bridge" API. We'd need to implement this ourselves | 13:10 |
stephenfin | Which is something we really don't want to be doing in an ML2 driver, IMO | 13:11 |
stephenfin | The better model, which is what jaypipes, gibi and I discussed, was to use placement for this and collaboratively build up this model between nova and neutron. However, placement isn't there yet | 13:11 |
stephenfin | sahid: So this is making the best of a bad situation. If I've missed something though, definitely let me know. | 13:12 |
stephenfin | I'll put all the above in the review | 13:12 |
*** eharney has joined #openstack-nova | 13:12 | |
*** Guest45643 has quit IRC | 13:15 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs https://review.openstack.org/552924 | 13:16 |
*** _ix has joined #openstack-nova | 13:18 | |
*** mriedem has joined #openstack-nova | 13:18 | |
*** liangy has joined #openstack-nova | 13:19 | |
sahid | stephenfin: we don'-t need to ask that question "give me the NUMA affinity for this PCI device I've attached to my bridge" to OVS | 13:21 |
sahid | the operator is going to configure OVS and actually DPDK based on where the device is located | 13:22 |
sahid | so basically what we need is just to retrieve where the vhu ports are created | 13:22 |
sahid | this can be done by asking OVS | 13:22 |
sahid | so the neutron agent can return such information to nova by the binding detail od the port | 13:23 |
stephenfin | So just query the PMD pinning information for a bridge? | 13:23 |
sahid | do i have mised something? | 13:23 |
sahid | hum... no sure i underatand the PMD pinning for a bridge? | 13:24 |
*** tbachman has joined #openstack-nova | 13:25 | |
stephenfin | It seemed like a big assumption to make (that PMD threads would be affined with the NIC), especially given that they don't have to be with recent releases | 13:25 |
sahid | well if operators want best perofrmance they have to do that | 13:25 |
stephenfin | Sorry, not the bridge. I'm referring to this https://developers.redhat.com/blog/2017/06/28/ovs-dpdk-parameters-dealing-with-multi-numa/ | 13:26 |
sahid | it's not our responsability (i don't think so) | 13:26 |
stephenfin | e.g. 'ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0xF0' | 13:26 |
sahid | so that is configurd by operator | 13:26 |
sahid | we are expecting that the pmd to run on the same NUMA node where the phy NIC is, right? | 13:27 |
stephenfin | Yes. There can be multiple NICs | 13:27 |
stephenfin | ...too | 13:27 |
stephenfin | So NIC X is on NUMA node 0, while NIC Y is on NUMA node 1 | 13:28 |
sahid | but NIC X and NIC Y have different network | 13:28 |
sahid | right? | 13:28 |
stephenfin | and NIC X is connected to/tagged with physnet_x, and NIC Y to physnet_y | 13:28 |
stephenfin | right | 13:28 |
sahid | so when neutron is asking to create a port for X | 13:29 |
sahid | the agent can query OVS to know where that vhu is located? | 13:29 |
stephenfin | Can they? | 13:29 |
sahid | yep | 13:30 |
sahid | let a sec to find the command | 13:30 |
stephenfin | Right, I didn't know that :D | 13:30 |
*** mikal_ has joined #openstack-nova | 13:30 | |
* stephenfin waits eagerly | 13:30 | |
alex_xu_ | efried: I missed one thing. If the resource provider X is the compute node. RP X provides VCPU and memory also. Each instance will consume the VCPU and memory. In that case, RP X will be returned | 13:31 |
sahid | stephenfin: https://software.intel.com/en-us/articles/vhost-user-numa-awareness-in-open-vswitch-with-dpdk | 13:31 |
sahid | if we can know here a port is located | 13:31 |
stephenfin | mikal: Just the man I'm looking for. Fancy jotting down your thoughts on https://review.openstack.org/554195 | 13:31 |
sahid | the agent can return this information to nova, no? | 13:31 |
stephenfin | (random aside: the OVS (-DPDK) documentation is hands-down awful. It sucks that we have to resort to random blogs for this critical information) | 13:32 |
stephenfin | sahid: If it's what we want then I don't see why not. Lemme check | 13:32 |
*** moshele has joined #openstack-nova | 13:33 | |
*** jaosorior has quit IRC | 13:33 | |
*** mikal has quit IRC | 13:33 | |
stephenfin | ooh, so 'pmd-rxq-show' does seem to be exactly what I wanted | 13:33 |
*** liangy has quit IRC | 13:34 | |
stephenfin | ...and we'd just assume that OVS was configured correctly so that there are PMD threads on all NUMA nodes, which is Red Hat's guidance at least | 13:35 |
stephenfin | sean-k-mooney[m]: If you're about, any thoughts on ^ | 13:35 |
efried | alex_xu_: Ah, then that's a mistake in modeling. | 13:36 |
*** Eran_Kuris has quit IRC | 13:36 | |
*** moshele has quit IRC | 13:37 | |
efried | alex_xu_: The RP tree should not be set up such that the compute host provides inventory of FPGA. | 13:37 |
alex_xu_ | efried: yes, I realized that also, that isn't the FPGA case now | 13:37 |
bauzas | alex_xu_: efried: which specific spec are you discussing ? | 13:37 |
stephenfin | efried: See what you made me do? https://review.openstack.org/556850 | 13:37 |
alex_xu_ | bauzas: here https://review.openstack.org/#/c/554305/ | 13:38 |
* stephenfin busts into some Tay-Tay | 13:38 | |
alex_xu_ | efried: basically, it only can happened in the compute node RP, | 13:38 |
*** gouthamr has joined #openstack-nova | 13:38 | |
bauzas | alex_xu_: ack, thanks | 13:39 |
efried | alex_xu_: Well, really any RP in the tree which provides inventory in multiple RCs that are "unrelated" to each other. | 13:39 |
*** moshele has joined #openstack-nova | 13:39 | |
efried | alex_xu_: This is actually what I wrote a bug about last year. | 13:39 |
* efried finds... | 13:39 | |
alex_xu_ | efried: ha, currently we have VGPU, but will change soon :) | 13:39 |
kashyap | "the operator is going to configure OVS and actually DPDK based on where the device is located" --> Hope the Operator has the necessary PhDs to configure DPDK et al | 13:40 |
alex_xu_ | efried: but I agree with that | 13:40 |
alex_xu_ | I feel it will be rare case in the future | 13:40 |
bauzas | alex_xu_: efried: FWIW, I discussed this morning with jianghuaw_ about some possible quirks with VGPUs | 13:41 |
bauzas | and I'd love your thoughts | 13:41 |
bauzas | because it would need to spec up | 13:41 |
bauzas | the fact is that physical GPUs support multiple types | 13:41 |
bauzas | depending on the type, you can have different number of vGPUs | 13:41 |
bauzas | so, guess what? | 13:41 |
efried | alex_xu_: I can't find the bug right offhand, but I think the scenario I described was: compute RP has HDD disk, sharing provider (or in fact another RP in the tree) has SSD disk. I ask for disk & VCPU, I ask for the HDD trait. I will get back candidates which include the SSD, because the compute RP is satisfying the HDD trait, even though it's not providing the disk resource. | 13:42 |
alex_xu_ | we need to update inventory based the user request? | 13:42 |
bauzas | alex_xu_: efried: in theory, a nested RP that would be a pGPU could have different inventories for the same resource class | 13:42 |
*** alexchadin has quit IRC | 13:42 | |
bauzas | alex_xu_: efried: but the trick is, once a vGPU is created, then all the other types but the one picked for that vGPU will have total=0 for their inventories | 13:43 |
efried | bauzas: I think we've discussed similar issues in the past. Trying to recall what we came up with as a solution... | 13:43 |
bauzas | alex_xu_: efried: so there are 2 ways to consider that | 13:43 |
bauzas | either we ask the operator to define which type he wants to support per-GPU | 13:43 |
alex_xu_ | efried: ah, I see, but I guess that bug is killed by the reality that we don't support two different storage on the host | 13:44 |
bauzas | and in that case, the enabled_vgpu_types option needs to be change | 13:44 |
bauzas | changed* | 13:44 |
efried | alex_xu_: Oh, but we do :) | 13:44 |
bauzas | or, we magically provide a 2nd level of the tree with nested RPs, meaning that a pGPU will have multiple children, each one being something like pGPU_thistype | 13:44 |
efried | bauzas: That only helps if you want to lock it down (predefine and preallocate) | 13:45 |
*** alexchadin has joined #openstack-nova | 13:45 | |
bauzas | the second solution makes the thing less impactful for operators, but it adds a second level to the tree just for VGPUs | 13:45 |
bauzas | efried: not really | 13:45 |
bauzas | efried: because at start, each inventory will provide all the possible vGPUs | 13:46 |
efried | bauzas: But you'd still be counting on an allocation from *here* affecting the inventory over *there* | 13:46 |
efried | which we can't do. | 13:46 |
bauzas | efried: in theory the allocation would be against the pGPU_type | 13:46 |
bauzas | not the pGPU itself | 13:46 |
efried | Yeah, I get that, but you would have had to pre-designate how many of each type you start with. | 13:47 |
bauzas | efried: that's the config option | 13:47 |
efried | You can't show "full" inventory for all types. | 13:47 |
efried | Okay, right, so swhat I'm saying, it's locked down ahead of time. | 13:47 |
efried | via the config option | 13:47 |
bauzas | for the moment, the config option is a ListOpt supporting types | 13:47 |
*** vivsoni_ has quit IRC | 13:48 | |
bauzas | but honestly, I think we should do something less crazy and just have an option that would do like pci passthrough | 13:48 |
*** mchlumsky has joined #openstack-nova | 13:48 | |
bauzas | ie. "for that PCI device, here is the type" | 13:48 |
*** vivsoni_ has joined #openstack-nova | 13:48 | |
bauzas | jaypipes: thoughts on that ? ^ | 13:48 |
efried | bauzas: If you want ultimate flexibility, I think you need to provide inventory of some RC that lets you represent "units of VGPU-ness", where different types consume a different number of that resource. And then your request would have to be well-behaved and ask for resources=VGPU:1,VGPU_UNITY_THINGY:4 | 13:48 |
*** yingjun has quit IRC | 13:49 | |
efried | ...where the request for VGPU_UNITY_THINGY has to be correct for the type you're requesting. | 13:49 |
openstackgerrit | Merged openstack/os-traits master: Updated from global requirements https://review.openstack.org/551599 | 13:49 |
*** moshele has quit IRC | 13:49 | |
bauzas | efried: that's where I think we're over engineering | 13:50 |
*** mlavalle has joined #openstack-nova | 13:50 | |
efried | That would be clearer as | 13:50 |
efried | resources=VGPU_TYPE_X:1,VGPU_UNITY_THINGY:4 | 13:50 |
efried | or | 13:50 |
efried | resources=VGPU:1,VGPU_UNITY_THINGY:4&required=VGPU_TYPE_X | 13:50 |
bauzas | efried: operators want flavors like VGPU=2&trait=MY_TYPE | 13:50 |
*** yingjun has joined #openstack-nova | 13:50 | |
bauzas | yeah, honestly, I feel for the short term a config option that would do a whitelist seems the most acceptable solution | 13:50 |
efried | bauzas: Yeah, I get that. This isn't the only place we've seen where it would be useful to provide a translation layer from the flavor to the actual placement request. | 13:51 |
efried | bauzas: I think perhaps we're oversimplifying/idealizing by thinking that a direct mapping of flavor to placement request is going to allow us to cover everything. | 13:51 |
bauzas | let's incrementally try to resolve the usecase | 13:52 |
bauzas | for queens, we supported a single type | 13:52 |
efried | bauzas: Basically, requiring the admin to understand the nuances and quirks of both the provider tree as modeled by the virt driver, and the syntax and semantics of the placement API query. | 13:52 |
bauzas | efried: that's where I think a whitelist could be more understandable | 13:52 |
bauzas | efried: we could have enabled_vgpu_types that would keep the existing types we agree | 13:53 |
*** hongbin has joined #openstack-nova | 13:53 | |
bauzas | and then a second option that would tell for which PCI ID which type | 13:53 |
efried | bauzas: I'm okay with that idea in general, but I'm going to be watching like a hawk to make sure we retain the separation of platform-specific syntax. | 13:53 |
bauzas | in that case, the PGPU inventory would be of one type, problem solved. | 13:53 |
efried | E.g. "NO PCI ID!" | 13:53 |
bauzas | efried: the PCI ID thingy is just within the driver | 13:54 |
* jaypipes reads back | 13:54 | |
bauzas | no crazypants about PCI tracking | 13:54 |
bauzas | litterally ask libvirt to pick that type for that pGPU | 13:54 |
efried | bauzas: If there's a PCI ID in the file, then we have to say that the file gets parsed by virt alone. Kind of thing. | 13:55 |
bauzas | efried: yeah, zactly that | 13:55 |
efried | bauzas: So... you want to do something like this for Rocky? | 13:56 |
bauzas | I guess | 13:56 |
bauzas | if nested RPs is a thing :) | 13:56 |
efried | bauzas: Cause this is edging unequivocally into Generic Device Management territory. | 13:56 |
alex_xu_ | bauzas: when your request VGPU_thistype, how do you change total=0 for the VGPU_thosetype? | 13:56 |
bauzas | alex_xu_: that's magically done by sysfs | 13:56 |
bauzas | alex_xu_: which I'm using for getting the inventory | 13:57 |
alex_xu_ | bauzas: but how the placement to know that | 13:57 |
*** germs has joined #openstack-nova | 13:57 | |
bauzas | alex_xu_: because we provide inventories of VGPU resouces as of queens :) | 13:57 |
efried | bauzas: Wait, you're talking about doing that at setup time, not at allocation time, right? | 13:57 |
bauzas | that are populated based on sysfs :) | 13:57 |
*** alexchadin has quit IRC | 13:58 | |
bauzas | efried: alex_xu_ is talking of the case where we would have pGPU types as children | 13:58 |
alex_xu_ | bauzas: if there are two requests at same time. One for VGPU_thistype, another for VGPU_thosetype. So there will be race case. Only one can successful in the host finally | 13:58 |
efried | bauzas: Bricks will be shat by several people if you start talking about modifying inventory of Y because X got allocated. | 13:58 |
bauzas | alex_xu_: yeah I considered the race condition | 13:58 |
bauzas | alex_xu_: and that's actually a good call for not doing that | 13:58 |
*** esberglu has joined #openstack-nova | 13:58 | |
bauzas | but rather doing a whitelisting on the virt driver direcrtly | 13:58 |
alex_xu_ | bauzas: so limit only one type in each host? | 13:59 |
bauzas | alex_xu_: no, one type per physical GPU | 14:00 |
alex_xu_ | ah, i got it | 14:00 |
bauzas | one type per host is already here | 14:00 |
bauzas | except the fact we don't use traits atm | 14:00 |
bauzas | anyway, will just propose something soon so we could see if that requires some spec | 14:01 |
bauzas | or at least some spec amendment | 14:01 |
efried | bauzas: Right, so you're setting up the types & inventories the first time you load up. The first time update_provider_tree runs, it reads that config file and sets up the provider tree with types & inventories according to what the admin put in there. Right? | 14:01 |
bauzas | ++ | 14:01 |
*** germs has quit IRC | 14:02 | |
efried | Cool cool | 14:02 |
alex_xu_ | the race condidates only happened one time, if people accept that, it also ok | 14:02 |
bauzas | efried: https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html describes that partially | 14:02 |
*** r-daneel has joined #openstack-nova | 14:02 | |
jaypipes | guh, it would have been nice to have more than a few hours of sleep before spec sprint day... :( | 14:02 |
efried | bauzas: I can also see a solution that goes halfway in the future... | 14:02 |
bauzas | jaypipes: take caffeine or drugs, either. | 14:02 |
*** jaosorior has joined #openstack-nova | 14:03 | |
bauzas | jaypipes: FWIW, https://review.openstack.org/#/c/541290/7/specs/rocky/approved/numa-aware-vswitches.rst will require me some paracetamol | 14:03 |
bauzas | stephenfin: ^ ;) | 14:03 |
cdent | jaypipes: start on an easy one: https://review.openstack.org/#/c/418393/ | 14:03 |
edleafe | jaypipes: have you tried the blue meth? | 14:03 |
bauzas | always take the blue pill | 14:04 |
stephenfin | I hear the blue meth is real good | 14:04 |
bauzas | never choose the red one | 14:04 |
efried | bauzas: In a fashion similar to neutron port creation, the user could create the VGPU resource before doing the boot request. The VGPU creation would eventually hit vendor-specific code (maybe virt, maybe some agent - cyborg?) which actually *would* tweak the inventories and available types. Then you would attach that VGPU to your boot request, just like you would a port. | 14:04 |
alex_xu_ | efried: any legal drugs in Florida? | 14:05 |
efried | alex_xu_: Legal schmegal. But I'm in Texas. | 14:05 |
bauzas | efried: creating the mediated devices in advance is already a possibility | 14:05 |
efried | In Florida, they wash up on the beach. | 14:05 |
*** awaugama has joined #openstack-nova | 14:07 | |
kashyap | alex_xu_: Hey there | 14:08 |
kashyap | alex_xu_: About your comment here: https://review.openstack.org/#/c/534384/16/nova/tests/unit/virt/libvirt/test_config.py | 14:09 |
efried | bauzas: But having that creation impact placement inventories would be new. | 14:09 |
bauzas | efried: no, it's already the case | 14:09 |
kashyap | alex_xu_: The test you pointed out is slightly different in that, it is testing: 'obj.extra_flags' | 14:09 |
efried | oh? | 14:09 |
bauzas | efried: if you create a mediated device, libvirt will just add that to the total | 14:09 |
bauzas | I should write a blogpost on this... | 14:10 |
bauzas | because creating a mediated device doesn't mean you *use* it for a guesrt | 14:10 |
bauzas | so, placement is getting an inventory of 'available+created' as a total already | 14:10 |
efried | cool | 14:11 |
jaypipes | stephenfin: and ack on the OVS documentation being awful. | 14:12 |
jaypipes | OVS-DPDK that is. | 14:12 |
*** archit has joined #openstack-nova | 14:13 | |
dansmith | efried: mriedem: I can never remember the moving target of translations.. we're translating exceptions now, but not translating logs _at all_ (even warning) ? | 14:14 |
bauzas | dansmith: right IIRC | 14:15 |
efried | dansmith: correct | 14:15 |
bauzas | because translating logs is too much for the i18n team | 14:15 |
bauzas | they aren't able to scale | 14:15 |
efried | Nice patch for a new contributor: remove _Lx from nova.i18n, see what breaks, fix it. | 14:16 |
dansmith | wtfever | 14:17 |
dansmith | not marking them means they _can't_ be translated which seems like a stupid step back, but whatever | 14:18 |
*** gjayavelu has quit IRC | 14:19 | |
openstackgerrit | Patricia Domingues proposed openstack/nova master: load up the volume drivers by checking architecture https://review.openstack.org/541393 | 14:21 |
jaypipes | efried: "The first time update_provider_tree runs, it reads that config file and sets up the provider tree with types & inventories according to what the admin put in there. Right?" <-- you mean, exactly like my provider-config-file spec enabled? | 14:23 |
efried | jaypipes: could be, could be. I'm not sure if I read that before it was declared moot, but if I did, I've purged it :( | 14:24 |
*** yangyapeng has quit IRC | 14:25 | |
efried | jaypipes: Anyway, it sounds like a lovely idea you had :) | 14:25 |
*** gjayavelu has joined #openstack-nova | 14:25 | |
jaypipes | efried, bauzas: I'm the person that would shit a brick if you start adding dynamic inventory creation based on whatever a user requested. | 14:26 |
efried | Yes, but not the only person. | 14:26 |
bauzas | jaypipes: I'm entering the danger timezone of me needing to go get my children at school | 14:26 |
*** gjayavelu has quit IRC | 14:27 | |
* efried now has Top Gun soundtrack stuck in head. Thanks a lot bauzas | 14:27 | |
bauzas | jaypipes: tl;dr I just want the inventory for that specific physical GPU to be config-driven | 14:27 |
*** artom has quit IRC | 14:27 | |
bauzas | jaypipes: if that sounds crazy, tell me more in 20 mins :) | 14:27 |
jaypipes | bauzas: yet another whitelist conf option, yes, I know. | 14:27 |
openstackgerrit | Dan Smith proposed openstack/nova master: Make get_allocation_candidates() honor aggregate restrictions https://review.openstack.org/547990 | 14:28 |
openstackgerrit | Dan Smith proposed openstack/nova master: Add an index on aggregate_metadata.value https://review.openstack.org/555851 | 14:28 |
openstackgerrit | Dan Smith proposed openstack/nova master: Add AggregateList.get_by_metadata() query method https://review.openstack.org/544728 | 14:28 |
openstackgerrit | Dan Smith proposed openstack/nova master: Add require_tenant_aggregate request filter https://review.openstack.org/545002 | 14:28 |
openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Honor availability_zone hint via placement https://review.openstack.org/546282 | 14:28 |
sahid | jaypipes: if you can enqueue this https://review.openstack.org/#/c/511188/ :) | 14:29 |
mriedem | efried: can we abandon https://review.openstack.org/#/c/497978/ or do you plan on updating it? | 14:29 |
efried | mriedem: Eventually. But I can abandon for now and resurrect at that time. | 14:30 |
mriedem | ok. i'm just starting backward for specs, oldest to newest and cleaning out the cruft. | 14:30 |
openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306 | 14:30 |
*** eharney has quit IRC | 14:31 | |
mriedem | johnthetubaguy: should we abandon https://review.openstack.org/#/c/438134/ for service-protected-servers? does that align with anything keystone is working on with RBAC? | 14:31 |
mriedem | edmondsw: ^ | 14:31 |
edmondsw | in a mtg, will look in a few | 14:32 |
mriedem | dansmith: this was the thing that prompted my ML thread about volume type proxy https://review.openstack.org/#/c/466595/ | 14:32 |
mriedem | i think... | 14:32 |
dansmith | ack | 14:33 |
*** mchlumsky has quit IRC | 14:34 | |
mriedem | bauzas: i'm going to start an ops list thread about https://review.openstack.org/#/c/446446/ since if it's just a bug fix for broken behavior, we don't need a spec or a microversion | 14:35 |
bhagyashris | mriedem, alex_xu_: Hi, I have proposed revised (as per the discussion in Dublin PTG) spec: https://review.openstack.org/#/c/511825/2 ( | 14:37 |
bhagyashris | Disallow rotation parameter 0 for 'createBackup' API) Request you to review the same. | 14:37 |
mriedem | bhagyashris: cool i'll review today | 14:37 |
alex_xu_ | kashyap: strange...I didn't see there is extra_flags in LibvirtConfigGuestCPU obj | 14:38 |
alex_xu_ | bhagyashris: cool, will try to reach that | 14:38 |
kashyap | alex_xu_: But, thanks to your comment, I could actually remove another line in the test | 14:38 |
alex_xu_ | kashyap: np | 14:39 |
*** ratailor has quit IRC | 14:39 | |
kashyap | alex_xu_: I'll upload a new version once I re-run my local tests. | 14:39 |
kashyap | alex_xu_: As I caught another bug from a functional test. | 14:39 |
kashyap | The casing of the config options was lost, after I moved to Oslo StrOpt(). Glad I tested locally w/ both casings | 14:39 |
kashyap | (Now fixed it) | 14:39 |
*** mchlumsky has joined #openstack-nova | 14:41 | |
jaypipes | gibi: "The backend information is needed for the NeutronBackendWeigher that tries to simulate the Neutron backend selection mechanism by implementing a preference order between backends." <-- this is what concerns me about the introduction of those NET_BACKEND_XXX traits. I don't see those traits as being germane to scheduling. Rather, I just see them as being a part of the port configuration information that we send down to os-vif. And we have | 14:41 |
jaypipes | the port binding information in Neutron for that. I still don't see why those should be traits. | 14:41 |
alex_xu_ | kashyap: actually I mean I didn't find a extra_flags field for the LibvirtConfigGuestCPU even for now... | 14:42 |
gibi | jaypipes: the backend traits are not needed for the placement query or for the scheduler filters | 14:42 |
efried | jaypipes: Do you have a spec queued up for NRP-in-a_c yet? | 14:43 |
*** lpetrut has joined #openstack-nova | 14:43 | |
gibi | jaypipes: but as the filter scheduler makes the allocation, it implicitly decides about the backend as well | 14:43 |
gibi | jaypipes: and before the bandwidth feature this decision was made by neutron | 14:43 |
gibi | jaypipes: but after it, the decision is made by the filter scheduler | 14:44 |
bhagyashris | mriedem, alex_xu_: thank you :) | 14:44 |
gibi | jaypipes: so we want to save some of the freedom of neutron here | 14:44 |
kashyap | alex_xu_: It's not in that class. But take a look at LibvirtConfigCPUFeature() | 14:44 |
gibi | jaypipes: by adding a weigher that can express backend preference order | 14:44 |
*** eharney has joined #openstack-nova | 14:45 | |
kashyap | alex_xu_: Typo, actually this one: LibvirtConfigGuestCPUFeature() | 14:45 |
jaypipes | efried: crap. haven't finished it. | 14:45 |
jaypipes | efried: I can push what I have. | 14:45 |
*** yamahata has joined #openstack-nova | 14:46 | |
efried | jaypipes: as you see fit. Just thought I'd ask, it being spec review day and all. | 14:46 |
openstackgerrit | Matthew Booth proposed openstack/nova-specs master: Add serial numbers for local disks https://review.openstack.org/556565 | 14:47 |
jaypipes | efried: ack, thx for the reminder. | 14:47 |
sean-k-mooney[m] | stephenfin: hi sorry was in a meeting you wanted me to comment on some ovs-dpdk stuff | 14:49 |
openstackgerrit | Dan Smith proposed openstack/nova-specs master: Amend the member_of spec for multiple query sets https://review.openstack.org/555413 | 14:49 |
efried | dansmith: remove -2 from ^ ? | 14:49 |
dansmith | efried: yep, I just need to fix a pep8 thing I just noticed | 14:50 |
efried | o | 14:50 |
efried | dansmith: Two spots | 14:50 |
* bauzas is back from school | 14:51 | |
stephenfin | sean-k-mooney[m]: Indeed. sahid was suggesting we could simply rely on NUMA affinity of a vhost-user interface's PMD queues to determine where to place a host | 14:51 |
bauzas | mriedem: ack, thanks | 14:51 |
bauzas | mriedem: honestly, I wasn't knowing what to do with this | 14:51 |
efried | dansmith: While you're at it, "in in" | 14:51 |
*** yangyapeng has joined #openstack-nova | 14:52 | |
dansmith | efried: ah yeah saw that before and had forgottten it | 14:52 |
dansmith | tttt | 14:52 |
*** yangyapeng has quit IRC | 14:52 | |
*** yangyapeng has joined #openstack-nova | 14:53 | |
efried | dansmith: Please clarify whether the multiple-member_of thing will also be implemented in GET /resource_providers | 14:53 |
dansmith | efried: ah, I was going to ask if we should do that too, but I expected it would be outside this spec, no? | 14:53 |
efried | dansmith: Whether it is or not, that should be stated. I could go either way. But slight preference for including it. For a couple of reasons... | 14:54 |
dansmith | ack | 14:54 |
efried | dansmith: First, a single microversion introducing multiple-member_of syntax. I like that better than one microversion introducing it for one URI, another for introducing what's effectively the same feature to another URI. | 14:55 |
openstackgerrit | Dan Smith proposed openstack/nova-specs master: Amend the member_of spec for multiple query sets https://review.openstack.org/555413 | 14:55 |
efried | dansmith: Second, it's actually going to make the code easier to write. Because we currently do the processing of member_of in common code for both; so we can continue to do that. | 14:55 |
dansmith | efried: sure | 14:56 |
*** kholkina has quit IRC | 14:56 | |
sean-k-mooney[m] | stephenfin: that wont work als ovs has no idea if there is enough memory on the numa node for the vm also the memory is not allocated until the vhost-user frontend in qemu connects to the vhost-user backend in ovs | 14:56 |
efried | dansmith: Technically, the Work Items section should be updated... | 14:56 |
dansmith | efried: for /rps? | 14:57 |
efried | dansmith: On rereading, it's sufficiently vague to be acceptable as is. | 14:57 |
dansmith | yeah, I was going to say.. | 14:57 |
efried | dansmith: Which is fine by me; I've always thought that section was pretty much redundant anyway. | 14:57 |
efried | dansmith: +1. One spec down! | 14:58 |
sean-k-mooney[m] | stephenfin: the other thing is that when the vhost-user frontend connects to the vhost-user backend in ovs it tries to allocate memory and pmd for the vhost-user interface from on of the numa nodes of the guest automatically | 14:58 |
* dansmith does fist pump | 14:58 | |
openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Handle nested providers for allocation candidates https://review.openstack.org/556873 | 14:58 |
*** yamahata has quit IRC | 14:59 | |
jaypipes | efried: ^ | 14:59 |
efried | jaypipes: ack | 14:59 |
mriedem | bhagyashris: comments inline https://review.openstack.org/#/c/511825/ | 15:00 |
*** hamzy has quit IRC | 15:02 | |
jaypipes | gibi: maybe I'm just being thick... I still don't get it. You will have multiple backends on the same compute host supporting the same physical networks that support the same vNIC types and you want to be able to choose which backend to allocate a piece of bandwidth from? | 15:04 |
bauzas | jaypipes: I'm not sure we need a spec for https://review.openstack.org/#/c/556873/1/specs/rocky/approved/nested-resource-providers-allocation-candidates.rst | 15:04 |
bauzas | jaypipes: it's just fixes we need to merge IMHO | 15:04 |
*** felipemonteiro_ has joined #openstack-nova | 15:04 | |
mriedem | bauzas: it's an api change and microversion right? | 15:05 |
mriedem | so spec is required yeah | 15:05 |
mriedem | ? | 15:05 |
gibi | jaypipes: exactly. Neutron today does it in the following way: | 15:05 |
gibi | jaypipes: Neutron has a mechnism driver config | 15:05 |
bauzas | mriedem: from the spec itself, looks like it's not changing the API | 15:05 |
stephenfin | sean-k-mooney[m]: Potentially dumb question, but in what way would lack of guest memory be related? | 15:05 |
gibi | jaypipes: Neutron iterates throught that list and try binding the port with the given driver | 15:05 |
jaypipes | mriedem: no API change, no. | 15:05 |
stephenfin | I figured if you could do 'ovs-appctl dpif-netdev/pmd-rxq-show' to show the affinity for a given interface and just return that | 15:06 |
gibi | jaypipes: the first driver that returns a positive result from that bind call will be the one Neutron use | 15:06 |
*** hamzy has joined #openstack-nova | 15:06 | |
gibi | jaypipes: so that config option defines a preference order between backends supporting the same physnet | 15:06 |
mriedem | jaypipes: bauzas: it's a behavior change for the alloc candidates api though | 15:06 |
stephenfin | Or does that even exist before the port is attached to the interface? | 15:06 |
*** voelzmo has joined #openstack-nova | 15:06 | |
bhagyashris | mriedem: thank you for review i will look into it as i am working in IST time zone it's end of day for me :) | 15:06 |
stephenfin | this would be so much easier if I have a machine to experiment on. Stupid fried motherboard is killing me :( | 15:06 |
jaypipes | mriedem: if "behaviour change" means "it will work when there are nested providers", then yes. :) | 15:07 |
bauzas | mriedem: technically, alloc-candidates doesn't work yet with nested RPs | 15:07 |
jaypipes | stephenfin: at least it's not an efried motherboard. | 15:07 |
bauzas | mriedem: it's not changing the existing | 15:07 |
mriedem | bauzas: technically volume-backed rebuild with a new image doesn't work either, | 15:07 |
* jaypipes goes back into the bad-joke cave. | 15:07 | |
mriedem | but if we make that work, it's an api change | 15:07 |
mriedem | even if the request params don't change | 15:07 |
jaypipes | does it even matter guys? the spec is up. | 15:08 |
bauzas | I agree it's a signal | 15:08 |
mriedem | yeah i'm saying it's a spec, | 15:08 |
mriedem | and likely a microversion bump | 15:08 |
bauzas | okay, I'll comment that too then | 15:08 |
*** bhujay has quit IRC | 15:09 | |
bauzas | mriedem: jaypipes: speaking of specs | 15:09 |
bauzas | jaypipes: now I'm back, can we discuss about my point with vGPU types ? | 15:10 |
*** pcaruana has quit IRC | 15:10 | |
bauzas | looks like you were sad about that | 15:10 |
sahid | stephenfin, sean-k-mooney[m] I think the issue is on the scheduling, we will have first to select a host so then we could create the ports and query them | 15:11 |
*** dikonoo has quit IRC | 15:11 | |
stephenfin | sahid: I'd be perfectly fine saying we need pre-created ports for this thing | 15:11 |
stephenfin | That's already a requirement for SR-IOV and afaik is the plan for gibi's bandwidth-aware scheduling spec | 15:11 |
*** jaosorior has quit IRC | 15:12 | |
sahid | stephenfin: the problem is then you need to raise that re-schedule excpetion is the resources are not enough | 15:12 |
jaypipes | bauzas: "sad" would be one way to put it, yes. | 15:12 |
sahid | yes i think we do that somewhere but i can't really remember | 15:12 |
stephenfin | same issue with SR-IOV though, right? | 15:12 |
bauzas | jaypipes: let's be gentlemen :p | 15:12 |
bauzas | jaypipes: so, before discussing about a solution, do you understand the problem ? | 15:13 |
sahid | stephenfin: yes probably you are right | 15:13 |
*** germs has joined #openstack-nova | 15:13 | |
*** germs has quit IRC | 15:13 | |
*** germs has joined #openstack-nova | 15:13 | |
jaypipes | bauzas: yes, I fully understand the problem. | 15:13 |
bauzas | jaypipes: in https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html we don't mention that a single pGPU can have multiple types and we just suppose a single inventory of VGPU for each | 15:13 |
jaypipes | bauzas: I had numerous conversations with jianghuaw_ about this. | 15:14 |
bauzas | the fact is, once *one* mediated device is created, then *all* the others types are not possible for that pGPU | 15:14 |
stephenfin | sahid, sean-k-mooney[m]: The bigger concern I have is that a vhost-user port is configured to a given PMD based on the guest - not the routes | 15:15 |
bauzas | jaypipes: and what are your thoughts on that ? | 15:15 |
bauzas | jaypipes: we already somehow set inventories based on config option thru enabled_gpu_types tho | 15:15 |
jaypipes | bauzas: the only thing I really don't want is on-the-fly re-configuration of providers based on *what the user requested*. | 15:16 |
stephenfin | sahid, sean-k-mooney[m]: e.g. all vhost-user ports will be handled by a random PMD until the guest is attached, when that reallocation happens | 15:16 |
bauzas | jaypipes: it's not that, if I understand correctly your concern | 15:16 |
stephenfin | sahid, sean-k-mooney[m]: Assuming vhost-user ports that aren't associated with a guest even appear in output of 'ovs-appctl dpif-netdev/pmd-rxq-show' (I can't test it, grrr) | 15:16 |
*** hemna_ has joined #openstack-nova | 15:17 | |
jaypipes | bauzas: if we want to pre-define configuration of multiple supported vGPU types using a CONF option, so be it. I would prefer to stop adding yet more CONF options and instead handle inventory of providers using a provider-config YAML file format, but that ain't gonna happen apparently, so be it. | 15:17 |
bauzas | jaypipes: if we go on a direction where each pGPU has multiple inventories, each for a GPU type, then that said, yes it would be dynamically modified on an instance creation | 15:17 |
*** lyan has joined #openstack-nova | 15:17 | |
kashyap | alex_xu_: You're right; I can remove that extra test. The main test in test_driver.py takes care of the full config. Thanks for catching. | 15:17 |
bauzas | jaypipes: I can try to spec it, you know | 15:17 |
jaypipes | bauzas: that's exactly what I *don't* want. | 15:18 |
bauzas | jaypipes: the YAML file | 15:18 |
*** lyan is now known as Guest9256 | 15:18 | |
bauzas | jaypipes: okay cool, so we're aligned | 15:18 |
edleafe | bauzas: multiple inventories is not going to work | 15:18 |
*** germs has quit IRC | 15:18 | |
bauzas | edleafe: if each inventory is behind a child RP | 15:18 |
bauzas | either way, sounds we're in agreement | 15:18 |
edleafe | yeah | 15:18 |
bauzas | I *don't* want to make things complicated | 15:18 |
bauzas | the problem is more about the specific config option | 15:19 |
bauzas | I understand jaypipes on that | 15:19 |
jaypipes | bauzas: it would be the same inventory of VGPU. But different child providers would be tagged with specific VGPU_TYPE_XXX traits, right? | 15:19 |
*** burt has joined #openstack-nova | 15:19 | |
sean-k-mooney[m] | stephenfin: if they are not associate with a guest im not sure. when they do get added to a guest they might also change. | 15:19 |
bauzas | jaypipes: you're talking of the possibility to have children, each of them being a type ? | 15:19 |
bauzas | jaypipes: if so, each inventory would be different | 15:19 |
jaypipes | bauzas: no | 15:20 |
bauzas | because the total number of vGPUs you can create depends on your type | 15:20 |
*** eharney has quit IRC | 15:20 | |
jaypipes | bauzas: I'm saying the resource class would all be "VGPU" | 15:20 |
bauzas | oh, with the conf opt solution ? | 15:20 |
stephenfin | sean-k-mooney[m]: Yeah, it seems like a lot of magic (even for ovs-dpdk) to say "we have this route that uses this NIC, therefore the vhost-user port should be processed by this PMD thread" | 15:20 |
*** sambetts|afk is now known as sambetts | 15:20 | |
bauzas | yeah, it's still VGPU resource class and a trait for that type | 15:20 |
*** eharney has joined #openstack-nova | 15:20 | |
jaypipes | bauzas: right. | 15:20 |
bauzas | so each PGPU will be a leaf | 15:20 |
sean-k-mooney[m] | stephenfin: this wont work in general however as this would only work with vhost-user. the approch you need to enable need to work for any switch backend | 15:21 |
bauzas | with one inventory about the total number of vGPUs it can create *for the type defined by the opt* | 15:21 |
jaypipes | bauzas: each pGPU group, but yes. | 15:21 |
bauzas | plus the trait telling which type it is | 15:21 |
bauzas | jaypipes: libvirt doesn't have the notion of groups | 15:21 |
jaypipes | bauzas: right. | 15:21 |
jaypipes | bauzas: I know, but xen does. | 15:21 |
bauzas | sure, but from a placement perspective, a "PGPU" RP is, from a libvirt perspective, a PCI device, and from a xen perspective, a PGPU group | 15:22 |
stephenfin | sean-k-mooney[m]: Sure, but vhost-user would be a start. Once we have a way to expose this information, we can extend other backends | 15:22 |
bauzas | but both are reconciled | 15:22 |
jaypipes | ack | 15:22 |
bauzas | it's just a driver-only thing | 15:22 |
bauzas | okay, now question | 15:23 |
bauzas | does that need to be spec'd up ? | 15:23 |
bauzas | jaypipes:^ | 15:23 |
sean-k-mooney[m] | stephenfin: if we wanted to do anything regardign the interface rx queue it likely should be an os-vif thing where we calulate teh best pmd our selves and set that not the other way around | 15:23 |
jaypipes | bauzas: yes. new CONF option, new way of behaving for the virt drivers. I would say yes. | 15:24 |
openstackgerrit | Merged openstack/nova master: tox: Remove unnecessary configuration https://review.openstack.org/556544 | 15:24 |
sean-k-mooney[m] | stephenfin: most backend wont that this info so vhost-user is not something we should build on | 15:26 |
stephenfin | sean-k-mooney[m]: What other backends would there be? | 15:26 |
*** imacdonn has quit IRC | 15:26 | |
*** imacdonn has joined #openstack-nova | 15:26 | |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Allow to specify granular CPU feature flags https://review.openstack.org/534384 | 15:26 |
sean-k-mooney[m] | stephenfin: for example kernel ovs with kernel vhost has kerne vhost-treads it spwans per interface, it dose not have pmds, so while we can taskset those threads we cant affinites them via looking a queues | 15:26 |
*** artom has joined #openstack-nova | 15:26 | |
bauzas | jaypipes: fine by me | 15:27 |
bauzas | artom: had some concerns about the virt driver "calling" the conductor in https://review.openstack.org/#/c/552722/6 | 15:27 |
*** damien_r has quit IRC | 15:27 | |
*** tianhui_ has quit IRC | 15:28 | |
artom | bauzas, replied on the patch, but we can continue here if you want. Basically, no new calls are added, it's just existing methods/returns with new data in them | 15:28 |
artom | Should I clarify the spec? | 15:28 |
bauzas | artom: maybe I misunderstood but you mean that the compute service would call back the conductor ? | 15:28 |
alex_xu_ | kashyap: np | 15:28 |
*** tianhui has joined #openstack-nova | 15:28 | |
bauzas | artom: that would help | 15:28 |
edmondsw | mriedem I don't remember ever discussing a service-protected-server case in our RBAC discussions, but I don't think you need to use service tokens there... just have a role that's given permission to manage these | 15:29 |
sean-k-mooney[m] | stephenfin: in any case queue mapping or tasksetting of vhost threads i think is out of scope of nova | 15:29 |
edmondsw | that spec needs a lot of work if anything's to happen there | 15:29 |
bauzas | artom: also, on the fact we would check the compute version | 15:30 |
bauzas | artom: does a live migration work with two different compute versions already ? | 15:30 |
openstackgerrit | Merged openstack/nova master: Move placement test cases from db to placement https://review.openstack.org/553149 | 15:30 |
mriedem | bauzas: yes | 15:30 |
mriedem | we support mixed version computes for live migration | 15:30 |
artom | bauzas, I didn't see anything in the code that would suggest it wouldn't | 15:30 |
bauzas | mriedem: context is https://review.openstack.org/#/c/552722/6/specs/rocky/approved/numa-aware-live-migration.rst@244 | 15:30 |
mriedem | we have a grenade + live migratoin job that also does this back and forth | 15:30 |
sean-k-mooney[m] | linux bridge, sriov, hardware offloaded ovs, vpp, iovision(ebpf), mini net, calico and macvtap are the main ones used beyond ovs/ovs-dpdk | 15:30 |
bauzas | mriedem: then, I was claiming we could microversion the behavioural change artom is going to introduce | 15:31 |
sean-k-mooney[m] | oh there is also snabb switch | 15:31 |
mriedem | that's a bit drastic | 15:31 |
artom | bauzas, also, we'd only fail it if the instance has "NUMA" characteristics | 15:31 |
mriedem | sec | 15:31 |
artom | Which is currently broken anyways, so not as drastic as it sounds | 15:31 |
bauzas | artom: I agree, and we don't test that AFAIK | 15:31 |
mriedem | see https://review.openstack.org/#/c/522537/13/nova/conductor/tasks/live_migrate.py | 15:31 |
mriedem | you only need to know if the source and dest computes can do the new hotness | 15:32 |
mriedem | otherwise it's a novalidhost if you can't find a pair | 15:32 |
bauzas | mriedem: yeah I remember that change | 15:32 |
*** moshele has joined #openstack-nova | 15:32 | |
sean-k-mooney[m] | stephenfin: on and NTT have a dpdk soft patch panel thing , the point is nova cant have special case code for all of these | 15:33 |
bauzas | mriedem: if the implementation of artom's spec would go into that direction, then yes we wouldn't need a microversion | 15:35 |
mriedem | i think we can agree that we don't want to add subnet to the requested network turducken in the compute api https://review.openstack.org/#/c/518227/ | 15:35 |
*** moshele has quit IRC | 15:35 | |
bauzas | mriedem: oh please, yes | 15:37 |
bauzas | just create the port in neutron and pass it to nova | 15:37 |
bauzas | aha, jinxed by mriedem's comment | 15:37 |
artom | mriedem, that link you posted in my NUMA migration spec, that's basically just checking min compute version, right? | 15:37 |
artom | So the same thing I was suggesting? | 15:38 |
artom | Just for clarity in my head, because I feel like we're talking about the same thing as though they were different | 15:38 |
gibi | jaypipes: I'm open to debate to simulate or not the neutron backend ordering preference via a scheduler weigher but for that debate we need some neutron heavy guys, like mlavalle | 15:39 |
openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Handle nested providers for allocation candidates https://review.openstack.org/556873 | 15:39 |
bauzas | artom: yeah basically | 15:39 |
bauzas | artom: it's for a different usage tho | 15:39 |
jaypipes | mriedem, bauzas: ^^ added not about microversion. | 15:39 |
artom | bauzas, right, but the mechanism is the same | 15:40 |
kashyap | Darn, lost edits to wiki.openstack.org, as it logged me out mid-way. And didn't give me a way to get the changes back. | 15:40 |
stephenfin | sean-k-mooney[m]: Well damn. That's unfortunate | 15:40 |
bauzas | artom: yup, the pattern should be the same | 15:40 |
artom | (We might eventually standardise that in a utils somewhere, since we seem to be doing a lot of it, btw) | 15:40 |
mriedem | artom: kind of, but it depends on where you're using it, | 15:40 |
mriedem | if it's the API and you're checking min compute service version across the entire cell, that's different than just comparing the source and candidate dest host | 15:41 |
bauzas | at least, we can assume we run a pre-flight check on the conductor that checks both compute versions | 15:41 |
bauzas | that could be generalized | 15:41 |
artom | mriedem, ah, so you're saying we should be more granular and just check the (source, dest) pair | 15:41 |
artom | Hrmm | 15:41 |
artom | Could we do that in the scheduler? | 15:41 |
mriedem | artom: yes, see that patch i linked | 15:41 |
bauzas | but that's an implementation detail, IHMO | 15:41 |
artom | So that straight away we have a dest that supports it? | 15:41 |
mriedem | artom: we could if we exposed the capability as a trait on the compute and added a pre-request filter | 15:42 |
bauzas | artom: the scheduler doesn't know whether it's a migration or a boot | 15:42 |
bauzas | but what mriedem wrote | 15:42 |
mriedem | artom: that would be an optimization imo | 15:42 |
mriedem | worth noting in the spec though | 15:42 |
bauzas | zactly | 15:42 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Filter resource providers by forbidden traits in db https://review.openstack.org/556472 | 15:42 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Filter allocation candidates by forbidden traits in db https://review.openstack.org/556660 | 15:42 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Parse forbidden traits in query strings https://review.openstack.org/556819 | 15:42 |
openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Support forbidden traits in API https://review.openstack.org/556820 | 15:42 |
artom | Mmmm, forbidden traits | 15:42 |
artom | /homer | 15:42 |
gibi | mriedem: I will be available during the notification subteam meeting timeslot or before that but I have nothing to really talk about so we can simply skip the meeting if you agree | 15:42 |
SamYaple | so i just ran into an annoying time consuming issue. i was going an upgrade and my nova-osapi_compute service changed to reporting its hostname to the database. this created new nova-osapi_compute services in the services table. the old service entires had version 9 and this was causing all GETs to instances to fail with instance not found | 15:43 |
SamYaple | code in question https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/compute/api.py#L2268 | 15:43 |
SamYaple | no logs anywhere describing the issue :/ | 15:43 |
mriedem | gibi: yes let's skip | 15:43 |
SamYaple | would a check for old osapi_compute service versions be appropriate to add to `nova-status upgrade check` ? | 15:44 |
gibi | mriedem: ack, let's focuse on spec reviews | 15:44 |
*** andreas_s has quit IRC | 15:44 | |
artom | mriedem, bauzas, the optimization would be the scheduler doing the version checking? And for now we let the conductor do it? | 15:44 |
*** andreas_s has joined #openstack-nova | 15:44 | |
bauzas | jaypipes: should we also discuss in https://review.openstack.org/#/c/556873/2/specs/rocky/approved/nested-resource-providers-allocation-candidates.rst about how the scheduler would pass the root RP ? | 15:44 |
bauzas | jaypipes: because atm, it just says "which compute node is having that UUID ?" | 15:45 |
openstackgerrit | Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766 | 15:45 |
openstackgerrit | Chris Dent proposed openstack/nova master: Isolate placement database config https://review.openstack.org/541435 | 15:45 |
openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Ensure that os-traits sync is attempted only at start of process https://review.openstack.org/553857 | 15:45 |
jaypipes | gibi: I'm happy to speak with mlavalle or anyone else about this. I just feel like the network-bandwidth spec has gotten way over-engineered. | 15:45 |
mriedem | SamYaple: a bit confused, | 15:45 |
mriedem | you said it started creating service entries in the db, | 15:45 |
mriedem | but that there were also old entries? | 15:45 |
mriedem | SamYaple: also, this sounds vaguely familiar to https://review.openstack.org/#/c/556670/ | 15:46 |
gibi | jaypipes: I understand that it is a long spec and this part of it seems minor | 15:46 |
mlavalle | jaypipes, gibi: I am planning to go over the spec today | 15:46 |
* bauzas goes to spec up | 15:46 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: tox: Speed things up and document them https://review.openstack.org/534382 | 15:46 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove 'tools/releasenotes_tox.sh' https://review.openstack.org/534383 | 15:46 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: tox: Make everything work with Python 3 https://review.openstack.org/556894 | 15:46 |
jaypipes | bauzas: the scheduler simply does a ComputeNode.get_all_by_uuid(), passing all the UUIDs of all providers it sees in the provider_summaries section of the allocation_candidates response. Nothing about that will be changing. | 15:46 |
*** eharney has quit IRC | 15:46 | |
mlavalle | I also have the impression that we can / should simplify a bit, jaypipes | 15:47 |
*** yamamoto has quit IRC | 15:47 | |
mriedem | SamYaple: at this point, if not CONF.cells.enable and we're in here https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/compute/api.py#L2268 we likely should be puking out a warning | 15:47 |
jaypipes | gibi: no, it's not the length of the spec that I have issues with. | 15:47 |
bauzas | jaypipes: sec | 15:47 |
jaypipes | mlavalle: it's the crossing of scope boundaries that I have issue with. | 15:47 |
SamYaple | mriedem: http://paste.openstack.org/show/715421/ | 15:47 |
gibi | mlavalle: cool, then could you please add your view about the backend selection in https://review.openstack.org/#/c/502306/21/specs/rocky/approved/bandwidth-resource-provider.rst@610 | 15:47 |
SamYaple | the first three services inthat list were the originals. i deleted them and everything started working | 15:48 |
bauzas | jaypipes: say some child RP is accepting the query, the placement API call to allocation_candidates will return the root RP UUIDs in the provider_summaries ? | 15:48 |
*** yamamoto has joined #openstack-nova | 15:48 | |
bauzas | jaypipes: is that already the case or is that requiring some implementation change ? | 15:48 |
jaypipes | gibi: the thing we are attempting to consume is a chunk of network (ingress or egress) bandwidth for a physical network. I don't really see how vNIC type nor network "backend" selection is relevant to that. | 15:48 |
bauzas | because I want to understand what's missing | 15:48 |
gibi | mlavalle: if we want to have what is described in that subsection then we have to convince jaypipes to have backend specific traits in the RP tree | 15:48 |
jaypipes | bauzas: yes. and it already does that. | 15:48 |
SamYaple | mriedem: with the old services not deleted, GETs on instances (nova show's) would fail | 15:48 |
bauzas | \o/ | 15:48 |
bauzas | I was out of nested RPs for a while | 15:49 |
mriedem | SamYaple: ok so before this, the api code just relied on getting the instances from the nova db configured in your nova-api nova.conf [database]/connection field right? | 15:49 |
gibi | jaypipes: by claiming bandwidth we implicitly select the backend, that is why backend comes into the picture | 15:49 |
bauzas | but that's a very nice point | 15:49 |
*** lajoskatona has quit IRC | 15:49 | |
mriedem | SamYaple: oh are you saying the new 15-level services are fixing the problem for you then? | 15:49 |
mriedem | but you didn't know about those until you had figured out the problem with the older service versions | 15:49 |
SamYaple | yes | 15:49 |
mriedem | ok | 15:50 |
jaypipes | gibi: but why should the nova scheduler care about that? shouldn't the nova scheduler just claim resources against the provider of network bandwidth for a specific physical network and then leave it to os-vif and the compute node to pick whatever network backend it wants? | 15:50 |
gibi | jaypipes: we failed to find a RP model that allows claiming bandwidth but not selecting a specific backend that bandwidth belongs to | 15:50 |
mriedem | SamYaple: did you just upgrade to newton? or ocata/pike? | 15:50 |
SamYaple | mriedem: newton -> ocata | 15:51 |
mriedem | ok and now you have a cell1 mapping for the nova db | 15:51 |
SamYaple | oh yea its all working now | 15:51 |
mriedem | so you're pulling instances from https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/compute/api.py#L2271 | 15:51 |
mriedem | ok | 15:51 |
mriedem | at this point i'm not sure what the nova-status version check would look for, nova-osapi_compute services with version < 15 across all cells? | 15:52 |
mriedem | and then backport that to queens, pike and ocata? | 15:52 |
SamYaple | mriedem: i think so | 15:52 |
gibi | jaypipes: could be a problem with our model | 15:52 |
SamYaple | there are no logs indicating any issues | 15:52 |
SamYaple | mriedem: i had to walk to code and db (which im not super familiar with) to findthis | 15:53 |
*** yamamoto has quit IRC | 15:53 | |
mriedem | SamYaple: yeah like i said at this point that should be a warning, but a warning in rocky doesn't help you in ocata | 15:53 |
mriedem | SamYaple: can you report a bug and i can bring it up in the cells meeting? | 15:53 |
*** derekh has quit IRC | 15:53 | |
*** yingjun has quit IRC | 15:53 | |
SamYaple | im looking to the future :) | 15:53 |
*** tianhui has quit IRC | 15:53 | |
SamYaple | sure will do | 15:53 |
*** derekh has joined #openstack-nova | 15:53 | |
*** eharney has joined #openstack-nova | 15:53 | |
mriedem | whatever we do can at least help the next poor soul that runs into this | 15:53 |
SamYaple | exactly | 15:53 |
mriedem | thanks | 15:53 |
*** tianhui has joined #openstack-nova | 15:54 | |
SamYaple | nova uses launchpad still, right? | 15:54 |
mriedem | hell yeah | 15:54 |
SamYaple | :) | 15:55 |
* gibi needs a new brain | 15:55 | |
Spaz-Home | Trade me plz. | 15:55 |
jaypipes | gibi, mlavalle: please move convo to https://etherpad.openstack.org/p/X0RboWOe7C | 15:56 |
gibi | jaypipes: ack | 15:56 |
*** ralonsoh has quit IRC | 15:57 | |
*** andreas_s has quit IRC | 15:58 | |
*** fragatina has quit IRC | 15:58 | |
*** eharney has quit IRC | 15:59 | |
*** gyee has joined #openstack-nova | 16:00 | |
*** germs has joined #openstack-nova | 16:01 | |
*** germs has quit IRC | 16:01 | |
*** germs has joined #openstack-nova | 16:01 | |
*** chyka has joined #openstack-nova | 16:01 | |
*** ktibi has quit IRC | 16:03 | |
*** yamamoto has joined #openstack-nova | 16:03 | |
*** trozet has joined #openstack-nova | 16:04 | |
SamYaple | mriedem: https://bugs.launchpad.net/nova/+bug/1759316 | 16:05 |
openstack | Launchpad bug 1759316 in OpenStack Compute (nova) "pre-cells_v2 nova-osapi_compute service in database breaks instance lookup" [Undecided,New] | 16:05 |
*** sree has joined #openstack-nova | 16:05 | |
jaypipes | mlavalle: are you light green on etherpad? | 16:06 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Scheduling Optimization: Remove cell0 from the list of candidates https://review.openstack.org/556821 | 16:06 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527 | 16:06 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Add --enable and --disable options to nova-manage update_cell https://review.openstack.org/555416 | 16:06 |
openstackgerrit | Surya Seetharaman proposed openstack/nova master: Update the cells FAQs and scheduler maintenance docs. https://review.openstack.org/556459 | 16:06 |
mlavalle | jaypipes: I am green | 16:06 |
jaypipes | mlavalle: cool, thx :) | 16:07 |
*** voelzmo has quit IRC | 16:08 | |
*** germs_ has joined #openstack-nova | 16:08 | |
sahid | gibi: i responded to your comments on the emulthreads spec, if you can let me know your rhinking on how to handle that specific point | 16:08 |
*** yamamoto has quit IRC | 16:08 | |
*** sree has quit IRC | 16:09 | |
*** germs has quit IRC | 16:10 | |
*** dave-mccowan has quit IRC | 16:10 | |
*** ragiman has quit IRC | 16:11 | |
kashyap | mriedem: Just to note, after last 30 mins of looking around, I've updated min versions for libvirt / QEMU / libguestfs for Debian, Fedora and RHEL: https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix#Distro_minimum_versions | 16:11 |
mriedem | SamYaple: ack | 16:11 |
*** mvk has quit IRC | 16:11 | |
*** germs_ has quit IRC | 16:11 | |
*** germs has joined #openstack-nova | 16:12 | |
*** germs has quit IRC | 16:12 | |
*** germs has joined #openstack-nova | 16:12 | |
kashyap | Once I do for openSUSE & SLES, then it gives a somewhat representative sample to update the code & remove backward compat cruft. | 16:12 |
mriedem | kashyap: ok. you could maybe get imacdonn or stvnoyes1 to help out with the oracle linux entry, and AJaeger or toabctl to help out with suse | 16:12 |
*** andreas_s has joined #openstack-nova | 16:13 | |
mriedem | kashyap: well, the proposed 'next' version is generally communicated on the ops mailing list for feedback | 16:13 |
kashyap | Thanks; I was just looking up things myself for the past bit. And for Debian at least I got double-checked w/ a Debian person | 16:13 |
kashyap | mriedem: Yes, that too, how could I forget that :-) | 16:13 |
kashyap | (I myself added a note about mail to Operators in the wiki. Won't miss that.) | 16:13 |
kashyap | s/Wiki/PTG Etherpad/ | 16:13 |
*** Guest9256 has quit IRC | 16:15 | |
sahid | btw mriedem or perhaps dansmith, i know that your are not so interested about the emulator threads things but if one of you can have a look at https://review.openstack.org/#/c/511188/ | 16:16 |
*** rmart04 has quit IRC | 16:17 | |
sahid | jay pipes looks to be also the one to review it but at some point i will need to have an other core | 16:17 |
*** trinaths has joined #openstack-nova | 16:17 | |
openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: NUMA-aware live migration https://review.openstack.org/552722 | 16:18 |
*** yamamoto has joined #openstack-nova | 16:18 | |
sahid | thanks in advance | 16:19 |
mriedem | sahid: that is wayyyy outside of my wheelhouse | 16:19 |
bauzas | mriedem: mmmm, we approved https://blueprints.launchpad.net/nova/+spec/vgpu-rocky but I discussed with jaypipes about how we would support multiple VGPU types by using a new conf opt and we agreed to provide a spec for that | 16:20 |
bauzas | mriedem: fine if I'm still using that blueprint ? | 16:20 |
mriedem | bauzas: ask melwitt | 16:20 |
sahid | yes yes no worries | 16:20 |
bauzas | melwitt: arouuuuund ? | 16:20 |
toabctl | kashyap, mriedem the (open)SUSE entries seems to be ok. the upcoming versions are not yet there but SLES12SP3 is the version that we need support for imo | 16:20 |
mriedem | toabctl: as the next min version you mean? | 16:21 |
bauzas | sahid: https://review.openstack.org/#/c/511188/ is on my list | 16:21 |
mriedem | for rocky, we already have the next min versions in plae | 16:21 |
mriedem | *plcae | 16:21 |
mriedem | damn it | 16:21 |
bauzas | place FTW | 16:21 |
*** derekh has quit IRC | 16:21 | |
sahid | bauzas: ah cool, thanks | 16:21 |
mriedem | https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/virt/libvirt/driver.py#L207 | 16:21 |
mriedem | toabctl: ^ | 16:21 |
*** germs has quit IRC | 16:22 | |
mriedem | kashyap: andreas_s is the person to ask for zkvm | 16:22 |
toabctl | mriedem, I'm saying the versions are up-to-date. so Rocky seems to be fine | 16:22 |
*** sahid has quit IRC | 16:23 | |
mriedem | yeah, we're talking about what the next min versions should be in S | 16:23 |
*** yamamoto has quit IRC | 16:23 | |
*** artom has quit IRC | 16:23 | |
toabctl | mriedem, so do you plan to use a higher version than currently supported on SLES/openSUSE? looks like most other distros have lower versions currently | 16:24 |
*** pcaruana has joined #openstack-nova | 16:25 | |
toabctl | mriedem, but there will be SLES15 and openSUSE Leap 15 soon. both will have newer versions. let me check these | 16:25 |
mriedem | toabctl: likely not no, the next min version should be compatible with what the various distros can support in S | 16:25 |
mriedem | the point of the next min version isn't to be the bleeding edge | 16:25 |
mriedem | it's to raise the bar, but still be a minimum that can be supported across distros | 16:25 |
imacdonn | mriedem kashyap toabctl Do keep stvnoyes1 and I in the loop, please .... QEMU/KVM version for Oracle Linux could be a bit tricky ... I think somenew newer is in the works there, but I can't make any public statements at the moment | 16:26 |
*** yassine has quit IRC | 16:26 | |
imacdonn | something* newer | 16:26 |
*** andreas_s has quit IRC | 16:26 | |
openstackgerrit | Merged openstack/nova master: add check before adding cpus to cpuset_reserved https://review.openstack.org/539865 | 16:27 |
openstackgerrit | Merged openstack/nova master: Docs: modernise links https://review.openstack.org/556024 | 16:27 |
*** derekh has joined #openstack-nova | 16:27 | |
gibi | sahid: responded in https://review.openstack.org/#/c/511188/13 | 16:29 |
*** germs has joined #openstack-nova | 16:29 | |
*** germs has quit IRC | 16:29 | |
*** germs has joined #openstack-nova | 16:29 | |
cfriesen | kashyap: just added a comment to your cpu feature flag review, but might make sense to discuss it here....seems like libvirt does allow you to set additional features when using host-model, do we want to artificially restrict that? | 16:29 |
*** germs has quit IRC | 16:32 | |
*** moshele has joined #openstack-nova | 16:33 | |
*** lucasagomes is now known as lucas-afk | 16:33 | |
*** yamamoto has joined #openstack-nova | 16:33 | |
*** andreas_s has joined #openstack-nova | 16:36 | |
jaypipes | gibi, mlavalle: gotta love etherpad conversations. even better than IRC conversations... ;) | 16:36 |
mlavalle | jaypipes: it was a good idea | 16:37 |
mlavalle | if there is a lot to discuss, it seems more orderly | 16:37 |
*** yamamoto has quit IRC | 16:37 | |
openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Add Generation to Consumers https://review.openstack.org/556971 | 16:39 |
edleafe | cdent: jaypipes: efried: ^^ | 16:39 |
efried | edleafe: noyce | 16:39 |
openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Few correction in the server filter/sort spec https://review.openstack.org/527019 | 16:39 |
cdent | edleafe: roger that | 16:39 |
*** lyan has joined #openstack-nova | 16:40 | |
*** lyan is now known as Guest59245 | 16:40 | |
*** udesale has quit IRC | 16:41 | |
efried | bauzas: ^ I think you wanted to refer to that from your spec | 16:42 |
efried | bauzas: Add Generation to Consumers https://review.openstack.org/556971 that is | 16:42 |
bauzas | mmm ? | 16:42 |
bauzas | it's 18:42 here and my brain dropped | 16:42 |
bauzas | and I still have to write a spec :p | 16:43 |
*** mdbooth has quit IRC | 16:43 | |
*** germs has joined #openstack-nova | 16:43 | |
*** germs has quit IRC | 16:43 | |
*** germs has joined #openstack-nova | 16:43 | |
*** moshele has quit IRC | 16:44 | |
*** andreas_s has quit IRC | 16:45 | |
*** felipemonteiro_ has quit IRC | 16:45 | |
*** david-lyle has joined #openstack-nova | 16:46 | |
*** yamamoto has joined #openstack-nova | 16:48 | |
*** janki has quit IRC | 16:49 | |
dansmith | bauzas: on this: https://review.openstack.org/#/c/554134/3 | 16:50 |
dansmith | bauzas: I get the desire for consistency, but it seems to me that on POST, extra_specs would always be empty, and on PUT you can't modify them anyway (right?) | 16:50 |
dansmith | I guess maybe that's a reason to do it for PUT, but POST seems completely trivial | 16:51 |
* bauzas looks | 16:51 | |
bauzas | dansmith: sec, verifying | 16:52 |
bauzas | dansmith: because IIRC, you can *create* a flavor and passing a spec | 16:52 |
dansmith | bauzas: see the comments in the spec that say no | 16:52 |
bauzas | in the same novaclient call | 16:52 |
dansmith | maybe, but not on the actual POST of /flavors | 16:53 |
*** tssurya has quit IRC | 16:53 | |
dansmith | confirmed in flavor_manage | 16:53 |
bauzas | hah | 16:53 |
*** yamamoto has quit IRC | 16:53 | |
*** sree has joined #openstack-nova | 16:53 | |
*** Guest59245 has quit IRC | 16:53 | |
bauzas | then I trampled my brain | 16:53 |
dansmith | it's not that it's a bad thing, | 16:53 |
bauzas | if so, I agree with you | 16:53 |
*** dikonoo has joined #openstack-nova | 16:53 | |
dansmith | it's just like.. I'm not sure I get the point of doing a microversion for that | 16:53 |
dansmith | mriedem: know anything about this? | 16:54 |
gibi | jaypipes, mlavalle: this etherpad discussion was really useful, thanks | 16:54 |
mlavalle | gibi, jaypipes: Thank You | 16:54 |
bauzas | dansmith: I guess the main concern is "should we accept that ?" | 16:54 |
bauzas | I mean, accepting to create a flavor with some extra specs | 16:54 |
dansmith | well, that's a different question/change than proposed by this spec addition :) | 16:55 |
*** germs_ has joined #openstack-nova | 16:55 | |
bauzas | right | 16:55 |
bauzas | honestly, that would be a separate spec then | 16:55 |
bauzas | -1 | 16:55 |
*** vivsoni has joined #openstack-nova | 16:56 | |
cfriesen | when asking placement for candidates, does nova-scheduler ask placement to limit how many candidates are returned, or does it get *all* of the possible candidates? | 16:56 |
mlavalle | gibi: so I won't review the Nova spec as is. I will wait for the next iteration | 16:56 |
*** vivsoni_ has quit IRC | 16:56 | |
*** germs has quit IRC | 16:56 | |
*** fragatina has joined #openstack-nova | 16:56 | |
mriedem | uh | 16:56 |
mriedem | never seen that spec | 16:56 |
mriedem | oh wait, | 16:57 |
*** dave-mccowan has joined #openstack-nova | 16:57 | |
gibi | mlavalle: sure, I will try to do it right now | 16:57 |
mriedem | umm, that bp is already approved | 16:57 |
mriedem | dansmith: bauzas | 16:57 |
bauzas | mriedem: because the spec was approved for just *getting* extra specs | 16:57 |
mlavalle | gibi: ah ok, then I will take a look at it later today | 16:57 |
dansmith | right, so I guess this is already going to bring in a microversion for the GET changes, | 16:57 |
mriedem | oh this is an amendment | 16:57 |
gibi | mlavalle: I will ping you | 16:57 |
*** bkopilov has quit IRC | 16:57 | |
mlavalle | perfect | 16:57 |
bauzas | mriedem: now, it asks to pass extra specs when creating | 16:57 |
dansmith | so we just do these few extra bits in the same microversion | 16:57 |
dansmith | bauzas: no it doesn't | 16:58 |
dansmith | bauzas: it's about the return of those calls | 16:58 |
mriedem | i need to read this change first | 16:58 |
dansmith | so that seems okay then | 16:58 |
dansmith | I was reading this as "a new microversion to return the must-be-empty extra specs from POST" | 16:58 |
bauzas | well, then it's impossible to get extra specs from a POST | 16:58 |
bauzas | sorry, it's late here | 16:58 |
dansmith | but if it's just done with everything else, then it's not such a big deal | 16:58 |
dansmith | bauzas: right, like I said, it's good for consistency, | 16:58 |
mriedem | you can't create a flavor today with extra specs in a single call | 16:59 |
dansmith | just not worth its own microversion IMHO, but if this is in with the useful GET changes then not a big deal | 16:59 |
dansmith | mriedem: right and this isn't aiming to change that | 16:59 |
*** derekh has quit IRC | 16:59 | |
mriedem | so if this is just saying, return extra_specs in the POST response, but it will be empty, fine | 16:59 |
dansmith | just make the return of POST consistent | 16:59 |
*** oomichi has joined #openstack-nova | 16:59 | |
*** trinaths has quit IRC | 16:59 | |
dansmith | I just didn't see a point in doing a whole new microversion for that because.. always empty, | 16:59 |
bauzas | dansmith: yeah that's what I understood when reviewing first the change | 16:59 |
dansmith | but if it's just alongside the GET changes | 16:59 |
dansmith | then that's cool | 16:59 |
bauzas | dansmith: yeah, sorry, I got confused tonight | 17:01 |
bauzas | I didn't remember it was just the returned fields | 17:01 |
mriedem | https://review.openstack.org/#/c/554134/3/specs/rocky/approved/add-extra-specs-to-flavor-list.rst@177 | 17:01 |
mriedem | exactly ^ | 17:01 |
bauzas | honestly, passing the extra specs on both POST/PUT and GET with the same microversion seems harmless to me | 17:02 |
mriedem | passing in the request or the response? | 17:02 |
mriedem | this is just the response | 17:02 |
bauzas | no, in the response | 17:02 |
mriedem | yes, it is fine | 17:02 |
bauzas | hence my original +2 | 17:03 |
mriedem | presumably yikun hit this during implementation | 17:03 |
bauzas | but I was trampled by POST tonight | 17:03 |
bauzas | and me thinking I was misunderstanding the spec | 17:03 |
* bauzas hides | 17:03 | |
*** yamamoto has joined #openstack-nova | 17:03 | |
openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Amend the "add extra-specs to flavor" for create and update API https://review.openstack.org/554134 | 17:03 |
mriedem | http://trampledbyturtles.com/ ? | 17:04 |
dansmith | mriedem: I was +W before you hit that so you send it when you're ready | 17:04 |
mriedem | already did | 17:04 |
bauzas | nice band name | 17:04 |
efried | jaypipes: Here's one for ya. I sent in separate granular resource requests for the same resource class, e.g. resources1=VF:1,BW:200&resources2=VF:1,BW:300. Per design, it's possible to get back candidates where all of that comes from one RP. So in that candidate you'll get back a *single* RP { VF: 2, BW: 500 }. Now the virt driver (or neutron, or whatever) needs to go grab/create the actual resources... | 17:06 |
mriedem | where are the john hopkins people when you need them? | 17:06 |
dansmith | man I really should have reserved my +1 on efried until the end of the waiting period, just to keep him worry'n | 17:06 |
efried | jaypipes: I guess it has to look at the flavor to figure out that the BW should be split up 200 for one and 300 for the other? | 17:06 |
*** liangy has joined #openstack-nova | 17:06 | |
*** Swami has joined #openstack-nova | 17:07 | |
*** yamamoto has quit IRC | 17:08 | |
*** fragatina has quit IRC | 17:09 | |
*** yangyapeng has quit IRC | 17:09 | |
efried | dansmith: I would have totally seen through that. | 17:09 |
*** yangyapeng has joined #openstack-nova | 17:09 | |
mriedem | there is a 1 week period right? | 17:10 |
mriedem | because efried has been nit pickin the shit out of my patches lately | 17:11 |
*** bkopilov has joined #openstack-nova | 17:11 | |
efried | mriedem: You like the abuse. | 17:11 |
efried | mriedem: I'm nitpicking your patches so dansmith will +1 me. | 17:11 |
mriedem | that reminds me, | 17:11 |
mriedem | you bastard, | 17:11 |
mriedem | i have to switch back to KSA | 17:12 |
efried | Well, we should be getting rid of neutronclient eventually anyway. | 17:12 |
mriedem | ha | 17:12 |
efried | and using raw ksa. | 17:12 |
mriedem | i will be dead and buried before that happens | 17:12 |
efried | like we're doing with ironic | 17:12 |
cfriesen | efried: why does placement combine the two requests? | 17:12 |
efried | But you don't have to do that. You can catch exceptions for error paths instead. | 17:12 |
mriedem | efried: i'd rather just say don't raise errors and check the response status code | 17:13 |
efried | mriedem: You could shim the neutronclient... | 17:13 |
efried | kidding | 17:13 |
efried | cfriesen: This is code not yet written. But if nothing else, it's because (I think) the syntax of the response has a dict keyed by resource class. So there would be no way to split them up even if we wanted to (without some new syntax). | 17:13 |
*** felipemonteiro_ has joined #openstack-nova | 17:14 | |
efried | cfriesen: But you raise a good point: since it's not written yet, we could still consider new syntax in the new microversion that introduces granular. | 17:14 |
efried | cfriesen: But that spec is already approved, so sorry, too late :P | 17:14 |
jaypipes | efried: yes | 17:15 |
mriedem | anyone know what is up with this sriov bond thing https://review.openstack.org/#/c/463526/ ? | 17:15 |
mriedem | sahid might but he's gone now | 17:15 |
openstackgerrit | Merged openstack/nova-specs master: Few correction in the server filter/sort spec https://review.openstack.org/527019 | 17:17 |
mriedem | dansmith: on the volume-backed rebuild + new image spec, i think i want to just say, add an api to cinder to re-image the volume and once that is in place, nova will use it | 17:17 |
mriedem | because cinder also has things it needs to update about the image in the volume, i.e. some image metadata stuff | 17:17 |
dansmith | mriedem: that would certainly be the nicest way yeah | 17:17 |
mriedem | because i don't think we want to do the volume create / delete swap thing, it's too messy | 17:17 |
mriedem | quotas, types, etc | 17:17 |
dansmith | I don't think create/swap/delete would be the way anyway, | 17:18 |
dansmith | we'd just lay the image down on the volume ourselves I think, | 17:18 |
dansmith | but that's definitely a whole big thing | 17:18 |
sean-k-mooney[m] | mriedem: is this yet another attempt at this or is it carring on form the previous attempts | 17:18 |
mriedem | and the volume image meta would be out of date | 17:18 |
cfriesen | efried: if you send in two granular resource requests for resources1 and resources2, wouldn't it make sense to get back a dict of {resource1:<rp>, resource2:<rp>} or similar? They could still be the same RP. | 17:18 |
*** yamamoto has joined #openstack-nova | 17:18 | |
mriedem | sean-k-mooney[m]: the sriov bond spec? it's old | 17:18 |
mriedem | and looks stale/abandoned | 17:19 |
*** dave-mccowan has quit IRC | 17:19 | |
sean-k-mooney[m] | mriedem: oh that is the bound spec that wanted to use neutron config element to configure bonds. ya i hated that part of it | 17:20 |
sean-k-mooney[m] | mriedem: ya there was also https://review.openstack.org/#/c/182242/ and there are 2-3 other ones dateing back to mitaka | 17:20 |
*** AlexeyAbashkin has quit IRC | 17:21 | |
mriedem | dansmith: ok done https://review.openstack.org/#/c/532407/ | 17:21 |
efried | gibi: In case you're in the middle, just finished review of https://review.openstack.org/#/c/502306/21 | 17:21 |
sean-k-mooney[m] | mriedem: but yes no one has touched it since december so i think https://review.openstack.org/#/c/463526/43 is abandoned. its still proposed against queens | 17:22 |
*** elmaciej has joined #openstack-nova | 17:22 | |
*** sambetts is now known as sambetts|afk | 17:23 | |
dansmith | mriedem: so, we'll need to detach the volume in order for cinder to be able to do that, which is basically equivalent to root-detach, which we already failed to do in the past.. just.. sayin' | 17:23 |
mriedem | i abandoned https://review.openstack.org/#/c/182242/ today | 17:23 |
gibi | efried: thanks, I'm currently working on an update. I will try to address your comments in that as well | 17:23 |
*** yamamoto has quit IRC | 17:23 | |
mriedem | dansmith: detach or disconnect from the host? | 17:23 |
mriedem | we need to at least keep the volume reserved for the instance | 17:24 |
dansmith | mriedem: can we do those separately? | 17:24 |
mriedem | yes | 17:24 |
dansmith | okay | 17:24 |
dansmith | then, disconnect I guess | 17:24 |
mriedem | it's basically shelve | 17:24 |
dansmith | as long as cinder doesn't have a fit with that internally | 17:24 |
mriedem | shelve the root volume | 17:24 |
dansmith | mriedem: ...which you can't do with bfv | 17:24 |
mriedem | cinder would only care about the attachments | 17:24 |
dansmith | well, | 17:24 |
dansmith | you can maybe, | 17:24 |
dansmith | but not without detaching the root, which I guess is your point | 17:25 |
*** artom has joined #openstack-nova | 17:25 | |
mriedem | honestly i don't know if volume-backed shelve works | 17:25 |
dansmith | I guess the root detach thing was mostly hard because of the desire to attach it to something else in the mean time, | 17:25 |
mriedem | since snapshot for a volume-backed instance is different in the api | 17:25 |
mriedem | and the shelve snapshot happens in hte compute | 17:25 |
dansmith | which wouldn't be the same problem here | 17:25 |
dansmith | so I think that makes more sense then yeah | 17:26 |
sean-k-mooney[m] | mriedem: for volumes backed instance i would not expect you to snapshot it at all jsut keep the volume | 17:26 |
dansmith | just want to make sure we're not sending them off on a mission that, upon completion, leaves more hard nova problems we might punt on | 17:26 |
mriedem | shelve supports volume-backed instances, it just casts directly to offload | 17:26 |
mriedem | dansmith: feel free to comment on the spec | 17:26 |
sean-k-mooney[m] | mriedem: for image backed guest we have too so we can free up the space on teh compute node but for volumes there is no reason to clean it up in the backing store just to put it into an image | 17:27 |
mriedem | i just can't imagine this is all better done inside nova | 17:27 |
dansmith | mriedem: no definitely not, just thinking through it | 17:28 |
dansmith | mriedem: you know, given all the many quagmires that came from nova-cinder interaction in the recent past | 17:28 |
sean-k-mooney[m] | mriedem: dansmith shelve for a volume backed instance should be jsut, shotdown instance, detach volume and clean up host resouces for vm no? then unshelve is jsut select host to boot on, set up entworking etc and attach volume and boot form it? | 17:29 |
mriedem | yeah so for volume-backed shelve, there is no snapshot, on unshelve we just re-attach the volumes to the instance on the new host | 17:30 |
dansmith | mriedem: you're saying that works today right? | 17:30 |
mriedem | we don't detach the volume, it stays reserved so someone else can't take it while the instance is shelved | 17:30 |
mriedem | dansmith: looks like it should from the code, | 17:30 |
*** fragatina has joined #openstack-nova | 17:30 | |
mriedem | would need to test it of course | 17:30 |
dansmith | it's the disconnect that is the important bit for detaching it from the current host | 17:30 |
dansmith | yeah | 17:30 |
sean-k-mooney[m] | mriedem: well i dont know if thats what happens today but i think that is what we should be doing | 17:30 |
mriedem | on shelve offload we delete the instance on the host so that does the disconnect | 17:31 |
mriedem | sean-k-mooney[m]: yes that's what happens today | 17:31 |
mriedem | we terminate connections on shelve offload, and then re-attach on unshelve | 17:31 |
kashyap | imacdonn: I'll write an e-mail to the Operators-List / Dev this week, it's better if we discuss the version stuff there. | 17:33 |
*** yamamoto has joined #openstack-nova | 17:33 | |
*** dave-mccowan has joined #openstack-nova | 17:34 | |
imacdonn | kashyap: OK | 17:34 |
*** mdnadeem has quit IRC | 17:34 | |
mriedem | dansmith: left a note in https://review.openstack.org/#/c/532407/ about the order of operations so we don't forget | 17:34 |
kashyap | cfriesen: Hey, was AFK (and I will be again in 10 mins). About your question -- | 17:34 |
dansmith | mriedem: cool | 17:35 |
mriedem | same guys are pushing for volume-backed rescue https://review.openstack.org/#/c/532410/ | 17:35 |
kashyap | cfriesen: As it stands, we are allowing this one choice to alleviate the existing problem. The allowing flags for 'host-model' thing, we can work it out when we lift the restriction | 17:35 |
openstackgerrit | Merged openstack/nova-specs master: Amend the "add extra-specs to flavor" for create and update API https://review.openstack.org/554134 | 17:35 |
sean-k-mooney[m] | mriedem: this is a different topic so lets not rat hole on it but if i shelve an insatce with an active multi attach volume will one of the other attachment become active. i guess you cant boot from a multi attach volume either right? | 17:35 |
kashyap | cfriesen: Does that sound reasonable to you? | 17:35 |
mriedem | sean-k-mooney[m]: you can boot from a multiattach volume | 17:36 |
mriedem | volume attachments aren't like port bindings, there isn't an active one and an inactive one | 17:36 |
mriedem | they do have attach modes, so r/o and r/w | 17:36 |
openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: NUMA-aware live migration https://review.openstack.org/552722 | 17:37 |
mriedem | which is related to https://review.openstack.org/#/c/552078/ | 17:37 |
*** yamamoto has quit IRC | 17:38 | |
mriedem | ildikov: heh look familiar? https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3554 | 17:38 |
kashyap | cfriesen: I'll respond on the review. | 17:38 |
sean-k-mooney[m] | mriedem: yes its the modes that i was think of. i was wonder if one of the other nodes would becomre r/w but i guess that would be an explcit call if you wantted that to happen not somethign that magically happend if you shutdown a r/w instacne or shelved it | 17:38 |
mriedem | correct, there is no auto-change of the mode when one attachment goes away | 17:39 |
*** elmaciej has quit IRC | 17:39 | |
mriedem | that is discussed as an option for attachment counting in https://review.openstack.org/#/c/552078/ | 17:39 |
*** yamamoto has joined #openstack-nova | 17:40 | |
*** yamamoto has quit IRC | 17:40 | |
sean-k-mooney[m] | mriedem: lol how may volume specs do we have for this cycle :) | 17:40 |
*** tesseract has quit IRC | 17:40 | |
*** yangyapeng has quit IRC | 17:40 | |
sean-k-mooney[m] | i guess thats only 2 i just feels like more | 17:41 |
*** artom has quit IRC | 17:41 | |
mriedem | volume-backed rebuild, rescue, and backup | 17:41 |
mriedem | from the same company | 17:41 |
mriedem | plus mine | 17:41 |
mriedem | that's nowhere near the number of placement specs | 17:41 |
cfriesen | kayshap: sorry, distracted by local stuff. yeah, I'm fine with it for the backport. I don't really care personally (mostly use specific cpu models for live migration) but wanted to bring it up just so it was explicitly considered. | 17:43 |
sean-k-mooney[m] | mriedem: true, am for https://review.openstack.org/#/c/552078/1/specs/rocky/approved/volume-multiattach-enhancements.rst in general do we want to add more multiboot apis to nova? and if so what is the main delta between X servers with volume Y and X servires with Y volumes each? | 17:43 |
*** arvindn05 has quit IRC | 17:43 | |
sean-k-mooney[m] | mriedem: that was the other main discussion we had with cinder right. should nova provide a way to consume teh fact that several hadware backend support creating multiple volumes at once to create many servers each with volumes in one call | 17:44 |
ildikov | mriedem: heh, I guess that's more of a workaround than a leftover... | 17:45 |
kashyap | cfriesen: No problem. So quick point: 'host-model' + PCID doesn't make sense anyway: | 17:46 |
kashyap | cfriesen: If QEMU already supports PCID, it would be enabled by 'host-model'. And if it's not supported, adding it doesn't make it magically appear :-) | 17:47 |
*** suresh12 has joined #openstack-nova | 17:48 | |
efried | cdent: "Steal an extra space from efried and put it here." Dick. | 17:49 |
*** dikonoo has quit IRC | 17:49 | |
edleafe | efried: I wouldn't *dream* of ever breaking up your double spaces! | 17:50 |
efried | edleafe: That's right. They're MINE. | 17:50 |
* jroll applauds cdent | 17:50 | |
*** pcaruana has quit IRC | 17:51 | |
*** tssurya has joined #openstack-nova | 17:51 | |
edleafe | efried: Hey, I don't want to be the only one who people think grew up on typewriters instead of computers | 17:52 |
efried | These young whippersnappers don't understand us edleafe | 17:53 |
mriedem | sean-k-mooney[m]: i don't know what you're saying | 17:54 |
mriedem | sean-k-mooney[m]: this isn't nova creating multiple servers and multiple multiattach volumes, nova doesn't create multiattach volumes, | 17:54 |
sean-k-mooney[m] | mriedem: your spec is suggesting allow boot 10 instance with this multiattach volume | 17:54 |
mriedem | it's boot from volume with an existing multiattach volume that can be attached to more than one server | 17:55 |
mriedem | sean-k-mooney[m]: if an admin doesn't want to allow multiattach bfv, they can control that via policy volume:multiattach_bootable_volume | 17:56 |
sean-k-mooney[m] | mriedem: in the cindier cross project meeting they where also a request for can i create 10 instance each with 1 voume each | 17:56 |
mriedem | sean-k-mooney[m]: that was multicreate | 17:56 |
mriedem | as far as i know, you can create 10 volume-backed instances in a single server create request today | 17:57 |
mriedem | as long as nova is creating the volumes | 17:57 |
mriedem | if it's a pre-existing volume, we don't allow that | 17:57 |
sean-k-mooney[m] | mriedem: for your spec you would create the multi attach volume in cinder first the ask nova to boot 10 instance each using that multi attach volume | 17:57 |
mriedem | because that flow does not (yet) support multiattach volumes | 17:57 |
mriedem | yes | 17:58 |
sean-k-mooney[m] | mriedem: yes the other usecase was multi create | 17:58 |
mriedem | so with attach_mode in the bdm, | 17:58 |
mriedem | well, nvm | 17:58 |
mriedem | bdms aren't indexed per server in the create request | 17:59 |
mriedem | but i was going to say, i could create 3 instances in a single request, with 3 bdms to the same multiattach volume, but only one has the r/w attachment, and the other two are r/o | 17:59 |
sean-k-mooney[m] | ya ok well its looks like a parity thing to me not a really large change in the semantics of the api you are just allowing seting the volume and attachment mode when making the multi boot request | 18:00 |
mriedem | any boot request, including multi | 18:00 |
mriedem | yes | 18:00 |
mriedem | but as noted ^ the bdms aren't indexed in a multicreate request, | 18:00 |
mriedem | so the bdm attach_mode would be the same for all of them | 18:01 |
sean-k-mooney[m] | mriedem: is that what teh sepc what to allow or what you can do today. today you would have to do that with 3 nova boot requests correct? | 18:01 |
mriedem | if you wanted them to each have different attach modes, yes | 18:02 |
mriedem | i'm not proposing that we change the bdm semantics in multicreate server to be indexed | 18:02 |
sean-k-mooney[m] | mriedem: and do you want ot allow somthing like boot 10 instance and have 4 r/w and 6 r/o or is that out of scope | 18:02 |
mriedem | that's out of scope | 18:02 |
sean-k-mooney[m] | ah they would all have the same mode | 18:02 |
sean-k-mooney[m] | ya mixing modes would be messy on the comandline and in the api | 18:02 |
mriedem | the list of bdms in the server create request are copied per instance that gets created | 18:02 |
sean-k-mooney[m] | so is it basically just a check in the api that says you can pass a volume on multi boot today? | 18:04 |
efried | edleafe: commented on https://review.openstack.org/#/c/556971/ | 18:04 |
mriedem | sean-k-mooney[m]: can't | 18:04 |
mriedem | but yes | 18:04 |
mriedem | it's linked from the spec i believe | 18:04 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: add a note about volume-backed rescue not being supported https://review.openstack.org/556996 | 18:04 |
sean-k-mooney[m] | mriedem: cool ill read it properly later. it makes sense to me how you discibe it | 18:06 |
edleafe | efried: thanks. Will update shortly | 18:08 |
*** suresh12 has quit IRC | 18:09 | |
*** ttsiouts_ has joined #openstack-nova | 18:09 | |
*** psachin has joined #openstack-nova | 18:10 | |
*** suresh12 has joined #openstack-nova | 18:10 | |
mriedem | stephenfin: we don't need this mypy spec do we? https://review.openstack.org/#/c/538217/ i thought you already started making those changes, much to my chagrin | 18:11 |
mriedem | s/chagrin/vexation/ | 18:12 |
mriedem | thanks google | 18:12 |
*** gjayavelu has joined #openstack-nova | 18:13 | |
efried | cfriesen: "efried: if you send in two granular resource requests for resources1 and resources2, wouldn't it make sense to get back a dict of {resource1:<rp>, resource2:<rp>} or similar? They could still be the same RP." Sorry, I let this scroll by... | 18:14 |
efried | cfriesen: Yes, something like that would make sense. But we don't have that in the plan at the moment. | 18:14 |
efried | cdent, edleafe, jaypipes: How do you feel about the granular microversion changing the response payload to something like ^ to preserve the division of the requests? | 18:16 |
*** rmart04 has joined #openstack-nova | 18:16 | |
*** rmart04 has quit IRC | 18:16 | |
efried | gibi has already identified a place where that would be useful. | 18:17 |
cdent | efried: which response payload are you talking about? | 18:17 |
efried | cdent: The response from GET /a_c | 18:17 |
edleafe | efried: the purpose of the granular request is to ensure that each of those requirements is met by a single RP. What would be the advantage of splitting the out if it turns out that they are the same RP? | 18:17 |
*** elmaciej has joined #openstack-nova | 18:17 | |
cdent | a_r or p_s? | 18:17 |
efried | cdent: The format of an allocation_request, I think. | 18:17 |
efried | a_r, definitely not p_s | 18:18 |
*** voelzmo has joined #openstack-nova | 18:18 | |
cdent | efried: I'm not visualizing what you're really suggesting, could you paste something somewhere, or link me to some context? | 18:18 |
cdent | (I'm stuck on "how is this different from what we already do") | 18:18 |
efried | cdent: edleafe: The example I gave above was: resources1=VF:1,BW:200&resources2=VF:1,BW:300 may result in an allocation_request like RP_X: { VF: 2, BW: 500 }. Without consulting the original request (which, as in gibi's neutron case, may not be available), the caller can't tell that one VF should get BW:200 and the other should get 300. | 18:19 |
*** voelzmo_ has joined #openstack-nova | 18:20 | |
efried | cdent, edleafe: So the suggestion is that the response should instead look like: { resources1: { RP_X: { VF: 1, BW: 200} }, resources2: { RP_X: { VF: 1, BW: 300 } } } | 18:20 |
cdent | efried: and in this case we're on the same pf, and it is the pf that is being the resource provider, yes? | 18:21 |
*** felipemonteiro__ has joined #openstack-nova | 18:21 | |
efried | cdent: yes | 18:21 |
efried | RP_X is a PF provider of VF and BW resources. | 18:21 |
cdent | efried: does the caller still know the request it made? | 18:21 |
cdent | or still have whatever info it used to make the call? | 18:21 |
efried | cdent: That depends on the scenario. But in general ima say no. | 18:22 |
sean-k-mooney[m] | mriedem: regarding https://review.openstack.org/#/c/532410/4/specs/rocky/approved/volume-backed-server-rescue.rst i think they just want to do a normal image based nova rescue but for boot from volume instance. so the image size bits of the spec are just saying the rescue image is not empty | 18:22 |
gibi | cdent: neutron provide resource request pieces via the neutron port API, nova combines them to one big a_c request | 18:22 |
*** fragatina has quit IRC | 18:22 | |
*** voelzmo has quit IRC | 18:22 | |
cdent | In general ima say that changing the allocation format again would make me sad, especially to represent granular stuff which seems icky somehow (will have to think a bit longer on why). Which is not "god no, I hate that" but rather "hmm, I'm not immediately in love with this" | 18:23 |
efried | cdent: I think the compute manager is the thing that builds the request, and the report client sends it down and gets the response, but the thing that needs to correlate the allocations to real resources is the virt driver (or in gibi's case, neutron (maybe a neutron agent)), and we're not giving that information to those guys. | 18:23 |
cdent | I'm uncomfortable with placement being use as state manager for the interaction between nova and neutron | 18:24 |
efried | eh, state manager? I don't see that. | 18:24 |
efried | All we're doing is providing the caller of GET /a_c with more detailed information about how the allocation_request was fulfilled. | 18:24 |
sean-k-mooney[m] | cdent: well its not state management. we just need to know how to correlate placement RP inventory to phyical resouce on the host | 18:24 |
efried | I.e. I took *this* request group to make *this* chunk of allocations. | 18:25 |
*** psachin has quit IRC | 18:25 | |
*** felipemonteiro_ has quit IRC | 18:25 | |
*** dtantsur is now known as dtantsur|afk | 18:25 | |
cdent | sounds like communicating state by proxy to me but maybe I'm peculiar | 18:25 |
efried | cdent: In the case of the virt driver, he can *probably* glean that information by looking back at the flavor... but now we also have resources/traits in the image, and gibi will also have them in the port, and future us may have... So asking virt to reconstruct all of that, which we currently do in nova, seems like a really bad idea. | 18:25 |
*** suresh12 has quit IRC | 18:26 | |
* cdent scrunches his brow | 18:26 | |
efried | cdent: It's not state. I see it as the difference between, "You asked for X and Y. Here's XY," and "You asked for X and Y. Here's X, Y." | 18:26 |
cdent | who is making the request for allocations in the example you gave? | 18:27 |
cdent | sorry candidates | 18:27 |
efried | the scheduler report client is the thing making the placement API call. | 18:27 |
cdent | right so nova is asking placement to construct data in a particular way so it can communicate something to nova | 18:28 |
cdent | s/nova$/neutron/ | 18:28 |
efried | No, if that's the part that's bothering you, let's remove neutron from the equation. | 18:28 |
efried | Just looking at nova talking to nova. | 18:28 |
efried | nova synthesizes resource+trait info from flavor, image, request spec, whatever sources. It comes up with a ResourceRequest (which is a set of RequestGroup) by the time it's ready to ask for allocation candidates. | 18:29 |
sean-k-mooney[m] | cdent: we have the same problem with numa in nova alone | 18:29 |
sean-k-mooney[m] | cdent: placement has 2 numa RP both with cpus. placement claims against one of the RPs how do i know which phyical numa node that alloaction is against | 18:30 |
*** suresh12 has joined #openstack-nova | 18:30 | |
*** suresh12 has quit IRC | 18:30 | |
cdent | so you have some state in the flavor, you use it to make a request and because you don't want to look back at that state in the flavor, you want to extend placement to transmit that state (again I'm not saying this is the worst thing ever, just trying to identify what's being done) | 18:30 |
efried | sean-k-mooney[m]: Not that, no. | 18:30 |
*** suresh12 has joined #openstack-nova | 18:30 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306 | 18:30 |
efried | How is what's in the flavor "state"? It's a set of resource requests. | 18:31 |
efried | As currently conceived, we'd be summing up resource requests that happen to land in the same RP. | 18:31 |
cfriesen | does anyone know why we plug vifs at nova-compute startup, but then also plug them again (but don't wait for them) when powering on the instance? (for libvirt anyway) | 18:31 |
efried | That takes away information. | 18:31 |
sean-k-mooney[m] | cfriesen: linux bridge | 18:32 |
efried | If they happened to be assigned from different RPs, we would *have* that information. | 18:32 |
efried | But since they happened to be assigned from the same RP, they get rolled together, and we lose that information. | 18:32 |
*** suresh12 has quit IRC | 18:32 | |
sean-k-mooney[m] | cfriesen: the plug on startup is because on a reboot the tap wont be added to linux bridge because linux bridge does not preseits state. we then need to do it as part of boot a second time | 18:33 |
efried | Certainly when the virt driver is trying to grab real resources corresponding to an allocation, it needs to be able to correlate the RC+RP back to something real. | 18:33 |
gibi | mlavalle, jaypipes: I updated the bandwidth spec based on our discussion. | 18:34 |
cdent | I think may be using the term "state" far more generally than you are, but that's probably not all that germane | 18:34 |
jaypipes | gibi: ty gibi. | 18:34 |
*** eharney has joined #openstack-nova | 18:35 | |
*** AlexeyAbashkin has joined #openstack-nova | 18:35 | |
gibi | efried: I don't have the brainpower any more to think through you mapping comments in the bandwidth spec today. I will get back to that tomorrow | 18:35 |
efried | cdent: looklook, here's another example. Today I have no way of requesting two different disks from placement. With granular, I could. I want a 1G disk and a 2G disk, so I would say: resources1=DISK_GB:1024&resources2=DISK_GB:2048. | 18:35 |
gibi | efried: thanks for pushing the discussion forward with cdent right now :) | 18:35 |
efried | cdent: In a scenario where I have sharing providers, or maybe multiple disk providers in my tree, or whatever, I may get back a candidate like { STOR_RP1: { DISK_GB: 1024 }, STOR_RP2: {DISK_GB: 2048 } } | 18:35 |
* gibi goes home | 18:35 | |
* efried waves at gibi | 18:35 | |
* gibi waves back | 18:36 | |
cdent | efried: I'm still trying to think this through, but it's slow going because I need to get past a fundamental issue for me: I really don't want to expose granularity in allocations... | 18:36 |
edleafe | efried: in that scenario, there isn't anything preventing placement from getting both "disks" from the same disk RP | 18:36 |
efried | cdent: And that would be fine, cause now my virt driver can tell it needs to get 1024 from STOR_RP1 and 2048 from STOR_RP2. | 18:36 |
efried | edleafe: Exactly. In which case, as currently conceived, we would report a candidate like: { STOR_RP1: { DISK_GB: 3072 } } | 18:36 |
cfriesen | sean-k-mooney[m]: I get the boot-time one...why do we need to do it again at instance boot if vifs_already_plugged=True ? | 18:36 |
cdent | didnt we have a discussion (or was it in my mind) recently about the difference between contiguous and non-contiguous resource providers? | 18:37 |
efried | cdent: Somehow virt needs to figure out that that's really a 1G disk and a 2G disk. How does he figure that out? Where does he get that information? | 18:37 |
efried | cdent: Yes, this is related to that discussion. It's why we couldn't e.g. split a VCPU:2 request across two numa nodes. | 18:37 |
edleafe | efried: what I'm saying is that there is no way to express "these must be two separate RPs" | 18:37 |
efried | edleafe: Yes, but in this case that's not something I need to express. | 18:38 |
edleafe | what granular gets you is "everything in this group must be on the same RP tree" | 18:38 |
efried | edleafe: What I *am* trying to express is that "these must be two separate *disks*". | 18:38 |
*** ttsiouts_ has quit IRC | 18:38 | |
sean-k-mooney[m] | cfriesen: i dont think vifs_already_plugged will be true on the boot case. just on soft reboot | 18:38 |
mriedem | are people ok with me just self approving this spec update to match the actual multiattach implementation? https://review.openstack.org/#/c/544152/ | 18:38 |
efried | edleafe: ...which I expressed by putting them into separate granular groups. | 18:38 |
sean-k-mooney[m] | cfriesen: are you using hybrid plug? | 18:38 |
sean-k-mooney[m] | cfriesen: e.g. are you using the iptables firewall driver or conntrack | 18:39 |
efried | mriedem: Given the +1s that are on it, fine by me. | 18:39 |
cfriesen | sean-k-mooney[m]: for power-on vifs_already_plugged is true | 18:39 |
*** ttsiouts_ has joined #openstack-nova | 18:39 | |
edleafe | efried: that's not what granular is for, is it? It's to ensure that all the requirements in a single group come from the same RP tree | 18:39 |
*** AlexeyAbashkin has quit IRC | 18:39 | |
edleafe | not that every group must come from different trees | 18:39 |
mriedem | efried: done | 18:39 |
mriedem | thnaks | 18:39 |
mriedem | thanks even | 18:39 |
efried | edleafe: "what granular gets you is "everything in this group must be on the same RP tree"" -- no, "everything in this group must be on the same RP". But also it allows you to request different resources of the same class. | 18:40 |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Allow to specify granular CPU feature flags https://review.openstack.org/534384 | 18:40 |
sean-k-mooney[m] | if vifs_already_plugged is true then we should not be calling plug at least in the os-vif case | 18:40 |
*** yamamoto has joined #openstack-nova | 18:40 | |
efried | edleafe: The canonical example being two VFs on different networks. resources1=VF:1&required1=PHYSNET_A&resources2=VF:1&required2=PHYSNET_B | 18:40 |
efried | edleafe: No way to do that in a single group. | 18:40 |
sean-k-mooney[m] | cfriesen: if libvirt is doing the plugging (kernel ovs with conntrack or noop security gorup driver) then its likely a side effect of libvirts ovs code | 18:41 |
edleafe | efried: of course | 18:41 |
efried | edleafe: And in that case, you would be assured that the resources came from different providers (because presumably one PF can't be on two physnets at once), so you would be fine. | 18:41 |
cdent | at one point I thought we had decided that if we wanted to do that the vf's had to be resource providers | 18:41 |
edleafe | efried: but if you have resources1=VF:1&required1=PHYSNET_A&resources2=VF:1&required2=PHYSNET_A you don't necessarily get different PFs | 18:41 |
efried | cdent: No, the PFs can still be providers. | 18:41 |
cfriesen | sean-k-mooney[m]: LibvirtDriver._create_domain_and_network() plugs the vifs | 18:41 |
efried | edleafe: Yes, correct. | 18:41 |
efried | edleafe: In the case of VF resources, where one VF is one resource, that's okay. | 18:42 |
edleafe | efried: but you're claiming that separate groups return separate RPs | 18:42 |
efried | edleafe: But now let's add bandwidth to the mix. | 18:42 |
*** oomichi has quit IRC | 18:42 | |
cfriesen | sean-k-mooney[m]: unconditionally, then decides whether or not to wait for them based on vifs_already_plugged | 18:42 |
efried | edleafe: No, only if traits make them split. | 18:42 |
cdent | efried: I know they _can_ be, but I thought we said that for the use case you are describing, making it work requires cn .... pf -> vf all as providers | 18:42 |
mriedem | sean-k-mooney[m]: it appears the sriov bond port stuff moved to neutron specs https://review.openstack.org/#/c/506066/ and talks about new configs in nova.conf for how that bonding actually gets configured on the compute host... | 18:42 |
*** suresh12 has joined #openstack-nova | 18:42 | |
sean-k-mooney[m] | cfriesen: yes. for a new instance that will be the only time the vif is pluged | 18:43 |
*** voelzmo_ has quit IRC | 18:43 | |
*** ttsiouts_ has quit IRC | 18:43 | |
cfriesen | sean-k-mooney[m]: right, but that happens on a resize, or a poweron, so we're plugging the vif again unnecessarily | 18:43 |
sean-k-mooney[m] | mriedem: :( really that just makes me sad. we should not have any config options needed for bonding | 18:43 |
edleafe | efried: you get separate RPs if the requirements cannot be met by a single RP. But if a single RP can satify all groups, there is nothing preventing that from being selected | 18:43 |
*** ttsiouts_ has joined #openstack-nova | 18:43 | |
efried | cdent: That helps a little, but still leaves us no way to model bandwidth. | 18:43 |
edleafe | That's what I'm trying to say | 18:43 |
efried | edleafe: Yes, you are correct. | 18:44 |
*** avolkov has quit IRC | 18:44 | |
efried | edleafe: And in that case, in the VF example, if you add bandwidth - which is, how did cdent put it, a "contiguous" resource? - then we can no longer figure out how much bandwidth each VF should get just from the allocation_request. | 18:44 |
sean-k-mooney[m] | cfriesen: well not nessicaily. on power we should not need to call LibvirtDriver._create_domain_and_network() the domain should still exist. it should just be "virsh start" effectivly | 18:45 |
*** fragatina has joined #openstack-nova | 18:46 | |
cfriesen | sean-k-mooney[m]: power_on calls hard_reboot which calls create_domain_and_network | 18:46 |
sean-k-mooney[m] | cfriesen: on resize we will be resizeing to a new host no? so we need to set up the vif there | 18:46 |
cfriesen | sean-k-mooney[m]: can resize to same host depending on config option | 18:47 |
cdent | efried: I think I'm too tapped out to really form any solid opinion on this today. Can we rejoin in progress another time, or perhaps do it in writing? | 18:47 |
*** yamamoto has quit IRC | 18:47 | |
efried | cdent: Yeah, I think I'll write it up as a delta to the granular spec, gods help me. | 18:47 |
cfriesen | sean-k-mooney[m]: _hard_reboot() undefines the domain | 18:47 |
sean-k-mooney[m] | cfriesen: uh why. im sure there is a reason but thats wrong | 18:47 |
efried | cdent: Maybe a ML post would be more expeditious. | 18:47 |
*** pchavva has quit IRC | 18:48 | |
sean-k-mooney[m] | cfriesen: but for resize we go via the scheduler so we cant assume we will | 18:48 |
cdent | I think email would be an easier starting point perhaps? | 18:48 |
efried | ight | 18:48 |
sean-k-mooney[m] | cfriesen: yes it does but we should not be hard rebooting when we resize or power on a vm | 18:49 |
cfriesen | sean-k-mooney[m]: hmm, does the fact that we undefine and then define a new domain affect the need to plug the vifs again? | 18:49 |
*** suresh12 has quit IRC | 18:49 | |
sean-k-mooney[m] | cfriesen: actully so again for linux bridge we would need to plug again | 18:49 |
sean-k-mooney[m] | when qemu exits the tap would be removed form the kernel and the linux bridge | 18:50 |
cfriesen | sean-k-mooney[m]: from the code comments: "We use _hard_reboot here to ensure that all backing files, network, and block device connections, etc. are established and available before we attempt to start the instance." | 18:50 |
cfriesen | sean-k-mooney[m]: okay, that explains it then | 18:50 |
sean-k-mooney[m] | so when we power on we have to add it back | 18:50 |
sean-k-mooney[m] | for ovs this is persited in the db and not needed | 18:50 |
*** moshele has joined #openstack-nova | 18:51 | |
cfriesen | sean-k-mooney[m]: persisted over a compute node reboot? | 18:51 |
sean-k-mooney[m] | cfriesen: is this causeing a bug for you or jsut interested in why its working this way? | 18:51 |
cfriesen | sean-k-mooney[m]: we're having performance issues and races on compute node startup in a small edge node. | 18:51 |
cfriesen | sean-k-mooney[m]: trying to figure out what my options are | 18:52 |
sean-k-mooney[m] | cfriesen: ya the port is stored in the ovs db so when the tap shows up the revalidator treads in the ovs-vswitchd will detect it and add it | 18:52 |
openstackgerrit | Merged openstack/nova-specs master: Amend volume multi-attach spec https://review.openstack.org/544152 | 18:52 |
sean-k-mooney[m] | cfriesen: i assume you have configured nova to start all instance on server reboot | 18:53 |
cfriesen | sean-k-mooney[m]: nope, there's another component that's in charge of that. but nova will plug all the vifs on nova-compute startup even if the instances are powered off. | 18:53 |
*** ttsiouts_ has quit IRC | 18:54 | |
*** moshele has quit IRC | 18:54 | |
cfriesen | that slows down neutron, which causes it to be delayed responding to something else, which times out and does exponential backoff...and the end result is that one or two instance take a long time to become pingable | 18:54 |
*** trozet has quit IRC | 18:55 | |
sean-k-mooney[m] | cfriesen: ah ok. i still think its a bug that nova plugs the vifs on compute agent start up. we should proably revisiti if its really needed. as you said we unconditionally plug the vifs when starting a vm so not sure what edge case still requires it | 18:55 |
sean-k-mooney[m] | the current bevior dates back from nova-networks with linuxbridge but we do several things differently now | 18:56 |
cfriesen | sean-k-mooney[m]: if we did get rid of the vif plugging at startup, would we have to wait for the network events in _create_domain_and_network() when powering on? | 18:57 |
sean-k-mooney[m] | cfriesen: we cant wait because only ovs with the neutron agents sends those events correctly | 18:58 |
sean-k-mooney[m] | cfriesen: linux bridge does not send them reliably and odl only send them when the port is first bound not when port are plugged | 18:59 |
cfriesen | sean-k-mooney[m]: we do wait on instance spawn though, is that different? | 19:00 |
*** germs_ has quit IRC | 19:00 | |
sean-k-mooney[m] | cfriesen: we have no ideay what happens for all the rest of the backend but that is why we dont wiat currently | 19:01 |
*** yamamoto has joined #openstack-nova | 19:01 | |
*** suresh12 has joined #openstack-nova | 19:01 | |
cfriesen | sean-k-mooney[m]: kay, thanks for the info. | 19:02 |
sean-k-mooney[m] | on instance spawn you meen on first boot. and yes that would be different as all backend send a vif plugged event in that case | 19:02 |
sean-k-mooney[m] | cfriesen: by the way its oke for us to wait for os-vif to plug the vif we just cant wait for neutron to say its wired up. we would like to but all the ml2 dirvers do different things | 19:03 |
mlavalle | thanks gibi :-) | 19:03 |
*** voelzmo has joined #openstack-nova | 19:03 | |
*** tssurya has quit IRC | 19:07 | |
*** yamamoto has quit IRC | 19:08 | |
*** lpetrut has quit IRC | 19:11 | |
openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Non-unique network names in Servers IPs API response https://review.openstack.org/521392 | 19:15 |
mriedem | sean-k-mooney[m]: that reminds me, you know how for the new port binding live migration spec we talked about adding a wait on the source host for the vifs to be plugged on the dest host during pre_live_migration? | 19:16 |
mriedem | i think that is likely not possible now because of ODL | 19:16 |
mriedem | since ODL won't send a vif plugged event until the port binding changes, right? | 19:16 |
mriedem | which currently happens in post live migration | 19:16 |
sean-k-mooney[m] | mriedem: yes didnt you already remove that in your code. we were talking about this last week i think? | 19:17 |
mriedem | i'm almost inclined to add a [workarounds] config option to disable that wait for vif plugged, but enable the wait by default | 19:17 |
mriedem | it's a todo in the code right now | 19:17 |
mriedem | so for ovs and LB we'd default to wait, | 19:17 |
mriedem | but if you're using ODL, you disable the wait | 19:17 |
sean-k-mooney[m] | ya odl wont send the event until there is a neutron port update which will only happen when we activate the new building as a result of the change to the host-id in the vif binding_details dicts | 19:18 |
sean-k-mooney[m] | mriedem: we cant tell if its odl or ovs from nova so we cant wait | 19:19 |
*** voelzmo has quit IRC | 19:19 | |
sean-k-mooney[m] | we can wait for the compute to tell use it has plugged the interface we just cant wait for neutron to say its wired it up | 19:19 |
*** voelzmo has joined #openstack-nova | 19:19 | |
mriedem | sean-k-mooney[m]: i know we can't | 19:19 |
mriedem | but the operator can | 19:19 |
*** voelzmo_ has joined #openstack-nova | 19:19 | |
mriedem | so for live migration, we can default to wait, assuming you're using a networking backend that doesn't suck | 19:20 |
mriedem | and if you are, then you need to configure nova-compute to not wait for vif plugged events during live migration | 19:20 |
*** voelzmo_ has quit IRC | 19:20 | |
mriedem | to other specs cores, i think https://review.openstack.org/#/c/521392/ is a no-brainer | 19:20 |
*** voelzmo has quit IRC | 19:20 | |
sean-k-mooney[m] | oh yes we could have a per compute config option or i guess a could wide one for the conductor | 19:20 |
mriedem | conductor doesn't wait | 19:21 |
mriedem | compute does | 19:21 |
*** gouthamr has quit IRC | 19:21 | |
sean-k-mooney[m] | mriedem: so in the prelive migrate would we have the dest read the config value and stick it in the migration data | 19:21 |
mriedem | umm | 19:22 |
mriedem | i figured the source would read the config since the source host is what's waiting | 19:22 |
mriedem | if the wait has to depend on the dest, then yeah maybe that has to go into the migrate data object | 19:22 |
mriedem | if the dest host is using ODL but the source is using OVS | 19:22 |
sean-k-mooney[m] | well since the dest might have a different network backend to the souce we should get the dest value no? | 19:23 |
mriedem | probably | 19:23 |
sean-k-mooney[m] | we proabbly also want to default to not waiting to not break upgrades as that is the default behavior today | 19:24 |
sean-k-mooney[m] | we can then change the default after 1 release | 19:24 |
mriedem | oh i suppose | 19:25 |
sean-k-mooney[m] | or put somthing in the neutorn port binding so neutron can tell use if we should wait or not in the future | 19:25 |
mriedem | older dest computes wouldn't send the value anyway | 19:25 |
mriedem | neutron telling us if waiting is safe would be great, but i'm not driving that on the neutron side | 19:26 |
*** itlinux has joined #openstack-nova | 19:26 | |
sean-k-mooney[m] | well the sepc said we would only use the new flow if they both supported it but ya old dest would not send it so its safer to assume dont wait | 19:27 |
sean-k-mooney[m] | mriedem: honestly i dont think that would be too hard to enable in neutron. i dont know if i have time to look at it but it should be a minor enough api extention | 19:28 |
mriedem | i mention the old dest compute thing because we had also talked about doing this separately from the blueprint as a bug fix so we could backport it | 19:28 |
mriedem | but if it requires changes to the versioned object, we can't really backport it | 19:28 |
mriedem | that was also before we knew that ODL didn't send events | 19:28 |
sean-k-mooney[m] | basicaly i think we could have a singel key in the portbining dict. vif_plugged_event_policy that could be never,on_bind or on_plug then just hard code it for each of the ml2 drivers | 19:29 |
sean-k-mooney[m] | mriedem: ah ya that makes sense | 19:30 |
*** vivsoni has quit IRC | 19:32 | |
sean-k-mooney[m] | mriedem: do you want me to open an rfe but for the event stuff and we can see what the neutron team says? it would not be a hard dependecy but it would allow us to know in the future if we can wait | 19:32 |
cdent | edleafe: responded to your coments on forbidden traits in rp in db. I think you're missing a point or I've explained things very poorly | 19:33 |
*** vivsoni has joined #openstack-nova | 19:33 | |
openstackgerrit | Merged openstack/nova-specs master: Non-unique network names in Servers IPs API response https://review.openstack.org/521392 | 19:33 |
mriedem | sean-k-mooney[m]: sure | 19:34 |
*** harlowja has joined #openstack-nova | 19:34 | |
*** mvk has joined #openstack-nova | 19:34 | |
efried | jaypipes, cdent, edleafe: Is there anything preventing two records in the allocations table having the same consumer+rp+rc? | 19:37 |
cdent | efried: in the table itself, no, but the interface on the object does a full wipe before it does any set (for a given consumer) | 19:39 |
efried | cdent: duly noted. | 19:40 |
mriedem | do we have a bp or anything anymore for supporting shared storage pools in nova? there was a bp but i can't find it now, maybe it was marked obsolete? | 19:40 |
edleafe | cdent: ok, but it seems like overkill. No one would be sending !TRAIT now, so I don't quite get the need to support it | 19:41 |
efried | mriedem: Since we descoped it in Q, and didn't bring it back up in Dublin... | 19:41 |
cdent | once that full stack is in place, someone can accidentally use the wrong microversion, and they need an appropriate error | 19:41 |
mriedem | efried: yeah i'm trying to list it as an alternative in https://review.openstack.org/#/c/551927/ | 19:42 |
*** gouthamr has joined #openstack-nova | 19:42 | |
cdent | efried, mriedem: tetsuro is still dilligently trying to make shared, correct, yes? | 19:42 |
efried | mriedem: IIRC, the shared storage provider spec was approved even before Q as one of those vague "we're headed in this direction" things, which led to the inception of aggregates and MISC_SHARES_VIA_AGGREGATE trait, but was never fully tied off. | 19:43 |
efried | mriedem: And after that we never (re)proposed anything on it. | 19:43 |
cdent | edleafe: did you see [t 18ze]? | 19:45 |
purplerbot | <cdent> once that full stack is in place, someone can accidentally use the wrong microversion, and they need an appropriate error [2018-03-27 19:41:56.981348] [n 18ze] | 19:45 |
cdent | edleafe: because I don't understand why you think it is unnecessary, and would like to | 19:45 |
mriedem | efried: ok, replied in https://review.openstack.org/#/c/551927/ | 19:46 |
mriedem | dansmith: jaypipes: ^ you might have other ideas | 19:46 |
mriedem | it's the age old "how can i support migration w/o globally shared ssh keys" | 19:46 |
mriedem | if you know a set of computes are in a shared storage pool, you could do the shared storage aggregate thing and then we could look that up from nova-compute rather than do the ssh check | 19:47 |
*** gouthamr has quit IRC | 19:47 | |
efried | mriedem: I think this is the closest we came to a shared storage pool spec https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/generic-resource-pools.html | 19:47 |
*** gouthamr has joined #openstack-nova | 19:48 | |
mriedem | before that, we had https://specs.openstack.org/openstack/nova-specs/specs/mitaka/approved/migrate-libvirt-volumes.html | 19:48 |
openstackgerrit | Merged openstack/nova master: api-ref: Parameter verification for servers.inc (1/3) https://review.openstack.org/528201 | 19:48 |
mriedem | that got held up on the imagebackend refactor o' doom | 19:49 |
*** lajoskatona has joined #openstack-nova | 19:50 | |
*** lajoskatona has quit IRC | 19:50 | |
*** awaugama has quit IRC | 19:51 | |
jaypipes | efried: yes, the primary key is on rp+rc+consumer. | 19:54 |
jaypipes | efried: so the table itself is preventing that. | 19:54 |
efried | jaypipes: How hard to change? | 19:54 |
jaypipes | efried: why would we want to change that? | 19:54 |
efried | To preserve granular groups in allocation records. | 19:54 |
efried | jaypipes: Composing an email on it. | 19:55 |
jaypipes | efried: honestly, I'm not interested in doing that. | 19:55 |
dansmith | that's going to be a poblem | 19:55 |
dansmith | unless placement handles all the atomicity of working that into a consistent view from the api | 19:55 |
*** germs has joined #openstack-nova | 19:55 | |
*** germs has quit IRC | 19:55 | |
*** germs has joined #openstack-nova | 19:55 | |
dansmith | we depend on allocations being per consumer | 19:55 |
jaypipes | right | 19:55 |
efried | per consumer isn't a problem. | 19:55 |
*** rmart04 has joined #openstack-nova | 19:55 | |
dansmith | presumably you want multiple allocations per consumer against a single RP right? | 19:56 |
efried | dansmith: Yes. I'm saying two records with e.g. | 19:57 |
efried | CONSUMER_A, RP_1, DISK_GB, 1024 | 19:57 |
efried | CONSUMER_A, RP_1, DISK_GB, 2048 | 19:57 |
efried | meaning "one 1G disk, one 2G disk, both from the same provider, allocated to the same consumer" | 19:57 |
cdent | jaypipes: you sure about that primary key? I see it as just on the id column? | 19:57 |
cdent | and no uniq anywhere | 19:57 |
dansmith | efried: how do you know which is which? | 19:57 |
efried | dansmith: I contend it doesn't matter which is which. | 19:57 |
dansmith | efried: and what is the difference between that and one 3G allocation | 19:57 |
efried | ditto | 19:57 |
efried | oh, sorry | 19:57 |
efried | dansmith: The difference is between "one 1G disk, one 2G disk" and "one 3G disk" | 19:58 |
*** edmondsw has quit IRC | 19:58 | |
dansmith | unless you add a group name into that key I don't see how you can reconstruct the original structure | 19:58 |
dansmith | oh I see | 19:58 |
*** vivsoni has quit IRC | 19:58 | |
dansmith | if it's the same RP, I don't think that's really placement's job to keep track of that | 19:58 |
*** vivsoni_ has joined #openstack-nova | 19:58 | |
efried | dansmith: Then whose? | 19:58 |
jaypipes | efried: if there isn't one, that's a terrible mistake. | 19:59 |
dansmith | that'd be a nova problem, looking at the fact that we have 3G of allocation and, oh hey, the flavor said two disks" | 19:59 |
dansmith | placement needs only to know that you got 3G of storage from that provider | 19:59 |
efried | dansmith: It was placement that took in a request of the form resources1=DISK_GB:1024&resources2=DISK_GB:2048 and responded with DISK_GB:3072. | 19:59 |
dansmith | if we decide to cut it up that's a VM management thing (i.e. nova) | 19:59 |
*** edmondsw has joined #openstack-nova | 19:59 | |
efried | dansmith: Yup, that'll work, but only for a little while. | 19:59 |
dansmith | encoding that you have two disks into an allocation in placement is teaching placement too much about nova, IMHO | 20:00 |
*** edmondsw_ has joined #openstack-nova | 20:01 | |
dansmith | efried: if you didn't ask (or imply) placement for those disks to be on different providers, then I think replying with a merged allocation or rejecting it as antithetical is legit | 20:01 |
*** trozet has joined #openstack-nova | 20:01 | |
sean-k-mooney[m] | dansmith: you will have to do that for things like bandwith requests on two different nics that could be allocated form the same pf | 20:01 |
*** bkopilov has quit IRC | 20:01 | |
sean-k-mooney[m] | or tor | 20:01 |
sean-k-mooney[m] | so its not jsut a nova thing | 20:01 |
efried | dansmith: Well, that's a separate issue - asking for them to be on different providers can't be expressed currently (assuming traits are the same etc.) | 20:01 |
*** germs has quit IRC | 20:02 | |
efried | dansmith: The problem moving forward is that we're going to have agents other than nova responsible for mapping allocation into actual resources. Plus the fact that we're going to be synthesizing that request string from not just the flavor, but also from image metadata, neutron, eventually cyborg, cinder... | 20:02 |
sean-k-mooney[m] | dansmith: so are you saying we should be explcit and have some way of saying its ok to merge? | 20:02 |
dansmith | sean-k-mooney[m]: if the resource comes from a different pile, then it needs to be a different RP (even if the same pf) | 20:02 |
*** germs has joined #openstack-nova | 20:02 | |
*** germs has quit IRC | 20:02 | |
*** germs has joined #openstack-nova | 20:02 | |
dansmith | sean-k-mooney[m]: and if it's not then placement doesn't need to know about it | 20:02 |
efried | dansmith: That doesn't let you express bandwidth on the PFs though. | 20:03 |
dansmith | sean-k-mooney[m]: rather decide whether not stating them as separate means they're merge-able or if it means it's a 400 | 20:03 |
edleafe | cdent: So you see a case where someone would create flavors with traits that are prefixed with '!', but not specify a microversion that supports it? I guess I can't see that. | 20:03 |
*** lpetrut has joined #openstack-nova | 20:03 | |
edleafe | But if it's necessary, so be it | 20:03 |
sean-k-mooney[m] | efried: well it kind of does | 20:04 |
cdent | edleafe: no I see a non-nova client of the placement api make requests to get resource providers filtered by traits, forgetting to set the microversion | 20:04 |
*** edmondsw has quit IRC | 20:04 | |
jaypipes | efried, sean-k-mooney[m]: I think you are both over-thinking this. | 20:04 |
efried | dansmith: Example that sean-k-mooney[m] is talking about: resources1=VF:1,NET_EGRESS_BW:200&resources2=VF:1,NET_EGRESS_BW:300 -- you can get back a candidate like { PF_RP1: { VF: 2, NET_EGRESS_BW: 500 } } | 20:04 |
*** edmondsw_ has quit IRC | 20:04 | |
cdent | edleafe: as in: a human | 20:04 |
*** edmondsw has joined #openstack-nova | 20:04 | |
sean-k-mooney[m] | the case that is not covered today would be anti afintiy of PFs for VFs for ha bonds | 20:04 |
*** edmondsw has quit IRC | 20:04 | |
dansmith | efried: right, it's still nova asking neutron for the fine-grained quota on each port though | 20:04 |
efried | neutron is responsible for carving out two PFs. How does it know that one needs BW:200 and the other needs 300? | 20:05 |
dansmith | efried: placement doesn't need more information than that 500 is committed to that consumer | 20:05 |
*** yamamoto has joined #openstack-nova | 20:05 | |
dansmith | efried: it doesn't imply it from placement, IMHO | 20:05 |
jaypipes | dansmith: zactly. | 20:05 |
efried | then from where? | 20:05 |
dansmith | from where what? | 20:05 |
dansmith | either in the port you created ahead of time, or from the flavor details when nova goes to create it for you | 20:06 |
jaypipes | efried: the allocation is the amount of that resource that is being provided by that specific resourc eprovider. it's 500. | 20:06 |
efried | Where does it get the information that that request entails one VF with BW:200 and one with BW:300? | 20:06 |
efried | neutron has access to the flavor? And the image? | 20:06 |
sean-k-mooney[m] | efried: if there are no different traits between resouce1 and resouce2 its ok to merge | 20:06 |
dansmith | efried: are you being difficult now or what? | 20:06 |
dansmith | efried: neutron ports don't just magically get created | 20:06 |
dansmith | efried: normally nova creates ports for you based on what you asked for in terms of --nic arguments, presumably influenced by the flavor if this sort of bw quota was encoded there | 20:07 |
dansmith | nova uses that to decide what to ask placement for, | 20:07 |
dansmith | and then uses it to talk to neutron to create the ports for you | 20:07 |
dansmith | placement is not the Windows Registry of information about instances for other services to mine :) | 20:07 |
dansmith | jaypipes: quote of the day for you ^ | 20:07 |
*** shaohe_feng has quit IRC | 20:08 | |
efried | dansmith: Okay, so for this case we get it from here, for that case we get it from there, for this other case maybe we have to invent something convoluted to pass it through... Whereas if we split up the allocations in placement, it could all come from the same place in a consistent way. | 20:08 |
efried | and no, I'm not *trying* to be difficult. | 20:08 |
*** liverpooler has quit IRC | 20:08 | |
dansmith | efried: the answer to the above leading question is not "mine it from placement" | 20:08 |
efried | I'm predicting that we will suffer a death of a thousand cuts by solving this problem differently every time we encounter it. | 20:08 |
sean-k-mooney[m] | efried: neutron has acess to neither the flavor or image | 20:08 |
dansmith | efried: haaaaave you met us? | 20:08 |
dansmith | either way, | 20:09 |
dansmith | placement is not the database of things for neutron to imply random things about your quota or policy from | 20:09 |
jaypipes | WHAT dansmith SAID. | 20:09 |
sean-k-mooney[m] | the enutron port case i was thinking of is i create two sriov ports in enutron tell nova to use both and want to express anti afinity for those at the pf level so in the granualr request to placment we need some what to tell placement they cannont come form the same RP even though they have not other differences | 20:10 |
jaypipes | this is precisely the conversation I had with gibi and mlavalle earlier today. | 20:10 |
dansmith | nova could, for example, be configured to ask placement for bandwidth resource, but never tell neutron about it, if we want general recordkeeping and kinda soft quotas for distribution, but we don't need actual administrative limiting of bandwidth, for example | 20:10 |
dansmith | if you mine it out of placement, you have no idea the semantics of the data you're examining | 20:10 |
efried | if this was a neutron-specific or nova-specific thing, okay. I don't see it that way, though. | 20:10 |
jaypipes | sean-k-mooney[m]: that is what granular request groups are for. | 20:10 |
efried | jaypipes: granular request groups don't solve it. | 20:10 |
dansmith | efried: the port case is a nova/neutron conversation and placement should C its way out | 20:10 |
dansmith | efried: for disks, it's nova/nova or nova/cinder or whatever | 20:11 |
*** yamamoto has quit IRC | 20:11 | |
jaypipes | efried: how is granular request groups not for solving the case of "gimme 2 of these resources from different providers"? | 20:11 |
sean-k-mooney[m] | placement is the thing to model all quantitive resouces with and qualitive tratis however so i may want to express affities between resouce in the future. for now i gues a filter could be used but i would be somewhat concered that we would get no valid host becaue of the limit we pass into placement | 20:11 |
efried | jaypipes: We have no way to say "from different providers". | 20:11 |
openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: tox.ini: remove the stale 'minversion = 1.4' https://review.openstack.org/530776 | 20:12 |
efried | jaypipes: Unless there happen to be traits we can exploit. | 20:12 |
dansmith | efried: that's what I was saying, | 20:12 |
jaypipes | that's the whole dang point behind granular request groups. | 20:12 |
dansmith | efried: I think it's implied that if you ask for two DISK_GB allocations you expect them to be from different RPs | 20:12 |
sean-k-mooney[m] | jaypipes: the thing that granualr resouce groups is missing is i cant say Resouce1 and resouce2 can be from the same RP or the inverse the must be form the same RP | 20:12 |
efried | dansmith: Oh, definitely not that. | 20:12 |
jaypipes | dansmith: zactly my thought. | 20:12 |
dansmith | efried: and thus we should decide whether that is the case and reject, or if we should expect those to come back merged | 20:12 |
dansmith | efried: well, I would have thought exactly that, fwiw | 20:12 |
efried | What if you only have one disk provider? and still want two disks? | 20:12 |
dansmith | efried: that's not a placement thing dude | 20:13 |
dansmith | efried: that's a nova thing | 20:13 |
dansmith | efried: it will ask for how much resource from a specific provider it needs, | 20:13 |
dansmith | and it may cut it up or not | 20:13 |
dansmith | placement has no bidness knowing or caring about that | 20:13 |
jaypipes | sean-k-mooney[m]: that was the whole point of granular request groups. to allow the caller to say "give me a total of 2 of these things, but make sure I get 1 each from different providers". | 20:13 |
efried | jaypipes: No, not that. | 20:13 |
jaypipes | yes, yes that | 20:13 |
dansmith | efried: yes that | 20:13 |
dansmith | exactly that | 20:13 |
efried | jaypipes: "Give me two of these things, but each one has different traits" I'll grant you. | 20:14 |
efried | ...which has the effect of dividing them across two providers. | 20:14 |
dansmith | efried: if you ask for two things with no trait difference, I think the assumption is you want them from different places | 20:14 |
efried | But if they *don't* have different traits, putting them in different groups does *not* give you different RPs. | 20:14 |
dansmith | you're saying that, | 20:14 |
dansmith | but I disagree that that is how it should be | 20:14 |
dansmith | so if you're talking about your patches as of now, then okay.. but then we should find and -2 those :) | 20:15 |
*** bkopilov has joined #openstack-nova | 20:15 | |
efried | dansmith: http://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/granular-resource-requests.html#semantics third bullet. | 20:15 |
jaypipes | agree wholeheartedly with dansmith. I don't see any logic in granular request groups *ever* meaning "give me two of these exact same resources, but allow them both to be from the same provider" | 20:15 |
*** shaohe_feng has joined #openstack-nova | 20:15 | |
dansmith | efried: okay so that's what I was saying above.. we either decide if they should be different (what I've been assuming) or they're not and they're effectively merged | 20:16 |
dansmith | efried: your bullet says the latter | 20:16 |
dansmith | which, if we've decided, then cool, but I think that was/is a mistake | 20:16 |
efried | jaypipes: You would never want two VFs from the same PF? | 20:16 |
jaypipes | agreed it was a mistake. | 20:16 |
dansmith | efried: two VFs from one PF is VF=2 | 20:16 |
jaypipes | efried: if you did, you would say resources:SRIOV_NET_VF=2 | 20:16 |
dansmith | efried: not VF=1,VF=1 | 20:16 |
sean-k-mooney[m] | jaypipes: it does simply one case which is merging resouce request form the image/flavor/port/volume | 20:16 |
sean-k-mooney[m] | jaypipes: it also has implcation for you cpu spec | 20:17 |
dansmith | what jaypipes said | 20:17 |
efried | How do you say, "Give me 8 VCPUs, but I don't care how you split them across NUMA nodes"? | 20:17 |
sean-k-mooney[m] | today if i have a guest with no numa toplogy and it cant fit on one numa node it will be spread. | 20:17 |
jaypipes | IMO, that third bullet should absolutely be amended to say "requesting granular request groups of the same resource means that the resulting candidates will be from different providers" | 20:17 |
dansmith | efried: if I just want multiple VFs and ensure they're on different PFs, but I don't care about any other reason, then VF=1,VF=1 means that, different from VF=2 | 20:17 |
dansmith | efried: VCPUS=8 | 20:18 |
sean-k-mooney[m] | if we say the non numbered resouce request must also be form one RP we cannot support that now | 20:18 |
efried | dansmith: VCPUS=8 will get you all 8 from the same RP. That's not even related to this discussion. | 20:18 |
jaypipes | sean-k-mooney[m]: the non-numbered request group specifically says "use_same_provider=False" | 20:19 |
dansmith | efried: how does VPUS=4,VCPUS=4 make it more flexible? | 20:19 |
*** ttsiouts_ has joined #openstack-nova | 20:19 | |
dansmith | efried: that's not saying "do as you with" | 20:19 |
dansmith | *wish | 20:19 |
efried | dansmith: It doesn't. But VCPUS=1,VCPUS=1,VCPUS=1,VCPUS=1,VCPUS=1,VCPUS=1,VCPUS=1,VCPUS=1 does. | 20:19 |
sean-k-mooney[m] | dansmith: what about the case of all you severs have 2 numa node with 10 cores each and i want 16 cores but dont care about numa in my guest | 20:19 |
efried | dansmith: Now we can spread 'em any which way they fit. | 20:20 |
dansmith | efried: you can't be serious | 20:20 |
dansmith | efried: and if I want 32G of memory allocated anywhere? 32*24 query params? :) | 20:20 |
jaypipes | sean-k-mooney[m]: then you would request VCPU=16 | 20:20 |
sean-k-mooney[m] | jaypipes: so non granular can be form multiple RPs and granular resuest each request is guarteeed to be form differnt RPs | 20:20 |
efried | dansmith: I imagine step sizes come into play there | 20:20 |
jaypipes | sean-k-mooney[m]: yes. | 20:21 |
dansmith | efried: okay but that's still a damn lot of params for silly reasons | 20:21 |
dansmith | tbh, I'm kinda frustrated by this conversation at this stage in the game and I'm late leaving for something else | 20:21 |
efried | dansmith: How else do you ensure generic spreadability? Is that a silly reason? | 20:21 |
*** rmart04 has quit IRC | 20:21 | |
efried | Okay. Y'all have made your position clear. | 20:21 |
dansmith | so I shall bow out, but jaypipes I agree about that third bullet change, or at least putting a stake in the ground about needing to revisit it | 20:21 |
dansmith | back later | 20:22 |
jaypipes | efried: what is generic spreadability? is that like I can't believe it's not butter? | 20:22 |
efried | Oh, you better believe it. | 20:22 |
jaypipes | heh | 20:22 |
*** yassine has joined #openstack-nova | 20:22 | |
*** moshele has joined #openstack-nova | 20:23 | |
*** vivsoni_ has quit IRC | 20:23 | |
efried | jaypipes: It means, "I know I want 4 VCPUs. I don't care about NUMA affinity. I just want my instance." Some hosts have no NUMA (all VCPU on the compute RP). Some have two NUMA nodes, some have four. It's a fullish cloud. I sure would like it if I could land my instance, whether there's a host with 4 VCPUs in one NUMA node, or one with 3,1, or one with 2,1,1,0, etc. | 20:24 |
*** gouthamr has quit IRC | 20:25 | |
*** vivsoni has joined #openstack-nova | 20:25 | |
sean-k-mooney[m] | jaypipes: so just to confim. non granuarl resouce resutes can from from multiple RP. each Granular resouce request of the same resouce class is guarenteed to come form different RPs and Granular Resouce request can or cannot spread across multiple RP im assuming Cannot? | 20:25 |
jaypipes | efried: resources:VCPU=4 | 20:25 |
jaypipes | simply as that. | 20:25 |
efried | jaypipes: Then we're treating VCPU special. | 20:25 |
efried | jaypipes: Cause you definitely can't treat DISK_GB=1024 the same way. | 20:25 |
cdent | efried: are you saying in the case where the vcpu inventory is registered on the numa nested provider, or something else? | 20:26 |
jaypipes | sean-k-mooney[m]: a single granular request group of an amount of a resource class cannot spread that bucket across multiple providers, no. | 20:26 |
efried | sean-k-mooney[m]: As currently designed [1], separate request groups with the same resource class *may* or *may not* come from the same RP. | 20:26 |
efried | [1] http://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/granular-resource-requests.html#semantics | 20:26 |
sean-k-mooney[m] | efried: that would be covered by just Resource:VCPU=4 | 20:26 |
efried | sean-k-mooney[m]: Oh, also what jaypipes says. | 20:26 |
efried | sean-k-mooney[m]: No - see above about DISK_GB. | 20:27 |
jaypipes | efried: how so? all of those resource providers have 4 VCPU available. | 20:27 |
sean-k-mooney[m] | efried: why? | 20:27 |
efried | jaypipes: No, 3,1 means "3 in NUMA node 0, 1 in NUMA node 1" etc. | 20:27 |
efried | sean-k-mooney[m]: If VCPU=4 means I can spread VCPU across multiple numa node RPs, then DISK_GB=1024 means I can spread... individual gigabytes? across multiple storage RPs. | 20:28 |
jaypipes | oh, in that case then no, no providers would be returned (since none have 4 VCPU available) | 20:28 |
jaypipes | efried: ^ | 20:28 |
jaypipes | efried: and that is A-OK in my book. | 20:28 |
efried | jaypipes: Exactly. I get NoValidHosts. But there *were* hosts with available procs. | 20:28 |
jaypipes | no there were not. | 20:28 |
jaypipes | efried: there were no resource providers that had 4 VCPU. | 20:28 |
efried | But there were *hosts* that had 4 VCPu. | 20:29 |
jaypipes | efried: the host isn't the provider of the VCPU in your scenario, though. | 20:29 |
efried | why not just say you can't have the same RC in different RPs then? | 20:29 |
sean-k-mooney[m] | jaypipes: well in his case there were 3cpus on 1 numa node an 1 on the second | 20:30 |
efried | jaypipes: But I didn't ask for my instance to land on a RP. I asked for it to land on a host. Which has a tree of RPs. But I don't know (and shouldn't care in this case) where the VCPUs are in that tree. | 20:30 |
*** moshele has quit IRC | 20:30 | |
jaypipes | sean-k-mooney[m]: exactly. there were no providers that had 4 VCPU available. | 20:30 |
sean-k-mooney[m] | but there was not host with 1 RP with 4 VCPUS | 20:30 |
*** germs has quit IRC | 20:30 | |
efried | tbc, I'm agreeing that VCPU=4 should give you no hits in this case. | 20:31 |
efried | I'm saying we need a way to express a request that *does* land this instance. | 20:31 |
jaypipes | efried: I pretty specifically remember you telling me that my proposed "sum the inventories for like resource classes within a provider tree" was absolutely the wrong way to handle nested providers. | 20:31 |
sean-k-mooney[m] | jaypipes: right so today. before placement i can have a host with 2 socket each with 10 cores and i can boot a vm with no numa topology with 16 cores | 20:31 |
jaypipes | efried: and I'm saying I don't care about that. | 20:32 |
efried | jaypipes: Yes, exactly, because the DISK_GB case breaks it unequivocally. | 20:32 |
sean-k-mooney[m] | that would now break | 20:32 |
efried | ^^ this. | 20:32 |
jaypipes | I really don't care. | 20:32 |
efried | With the granular syntax as designed, we have a way to ask for this ^ that will work in *any* scenario. | 20:32 |
efried | namely: resources1=VCPU:1,...,resources16=VCPU:1 | 20:33 |
sean-k-mooney[m] | efried: well diskGB only breaks in some cases | 20:33 |
efried | I'll grant you that I don't want the admin/operator to have to say that. | 20:33 |
jaypipes | efried: that is just over-engineering IMHO. | 20:33 |
jaypipes | for a use case that just isn't particularly attractive to me. | 20:33 |
openstackgerrit | Merged openstack/nova-specs master: tox.ini: remove the stale 'minversion = 1.4' https://review.openstack.org/530776 | 20:33 |
sean-k-mooney[m] | efried: that has the opisite problem | 20:34 |
efried | sean-k-mooney[m]: Not if separate granular groups can land on the same RP. | 20:35 |
sean-k-mooney[m] | now i cant say thes have to come from different RPs | 20:35 |
efried | correct. | 20:35 |
efried | without traits. | 20:35 |
sean-k-mooney[m] | efried: even with traits | 20:36 |
sean-k-mooney[m] | traits would artifically nanorow your selection | 20:36 |
sean-k-mooney[m] | you basically need a way to express "use_same_provider=True/False" relationships between Resouce1... | 20:37 |
sean-k-mooney[m] | e.g. Resouce1,Resouce2:use_same_provider=True | 20:37 |
* jaypipes wonders whether resource requests in kubernetes support this level of crazy. | 20:37 | |
efried | sean-k-mooney[m]: Agree we don't want to restrict artificially via traits. Which is why that's not a solution, just a side effect. | 20:38 |
jaypipes | oh wait, no, no they don't. at all. | 20:38 |
efried | jaypipes: kubernetes is just a babe. Give it a couple of years. | 20:38 |
efried | It'll either be dead, or supporting this level of crazy. | 20:38 |
jaypipes | efried: intel has been trying desperately for 2+ years to add this level of crazy to resource management in k8s. | 20:38 |
sean-k-mooney[m] | jaypipes: not yet. give intel time :P | 20:38 |
jaypipes | sorry sean-k-mooney[m], but it's true. | 20:38 |
sean-k-mooney[m] | jaypipes: haha i know one of our k8s teams sits 10 feet form me | 20:39 |
jaypipes | connor doyle? | 20:40 |
sean-k-mooney[m] | jaypipes: am i dont recognise that name but they have been working on multus ant the multi nic support + cpu pinning, hugepages and somthing else | 20:42 |
cfriesen | jaypipes: sean-k-mooney[m]: in the VCPU and 4KB pages case we currently let the instance float across the whole compute node....I'm of the opinion that we *shouldn't* let it, and *should* limit it to a single host NUMA node. | 20:43 |
mriedem | i saw in the latest k8s release notes that they now support cpu pinning and huge pages | 20:44 |
jaypipes | cfriesen: ok. nothing about the proposals would prevent that. | 20:44 |
efried | cfriesen: Fine by me, but you're going to bounce a lot of spawn requests that way. | 20:44 |
mriedem | good for them | 20:44 |
cfriesen | efried: if we don't restrict it, we have no idea how many 4KB pages are left on each host numa node. | 20:44 |
sean-k-mooney[m] | cfriesen: that breaks existing behavior where it cant fit in one numa node | 20:44 |
cfriesen | sean-k-mooney[m]: yes. and I think we have no option. | 20:45 |
efried | cfriesen: except splitting into individual pages, one per request group. | 20:45 |
efried | or... inventing something new. | 20:45 |
cfriesen | efried: I'm not sure we can do that with qemu. | 20:46 |
efried | cfriesen: I'm not talking about qemu splitting. I'm talking about the request being split. Then placement will give you back summed-up allocations per RP. | 20:46 |
efried | NUMA_0: PAGES=64, NUMA_1: PAGES=1024 or whatever | 20:47 |
efried | cfriesen: because btw, jaypipes and dansmith came down hard on the idea of placement tracking separate request groups in any way. | 20:47 |
cfriesen | efried: when you start up qemu and it's allowed to float across the whole host, we do not know how much it will consume from each host numa node | 20:47 |
sean-k-mooney[m] | jaypipes: so how would you feel about somthing like Resouce:VCPU:use_same_provider:true and then just define that all Rsource provider groups will not overlap with others | 20:48 |
efried | cfriesen: Oh, you have to let qemu have its head completely? Bogus. | 20:48 |
cfriesen | efried: at least the way we do it now, yes. for hugepages we map a file and tell it to use that, but for 4KB pages we just say "you're allowed to use up to X memory" | 20:48 |
*** archit has quit IRC | 20:48 | |
sean-k-mooney[m] | efried: well no the said seperate request groups guarenteeed different RPs | 20:49 |
efried | cfriesen: Then wouldn't the hugepages be inventory on the compute RP? | 20:49 |
sean-k-mooney[m] | efried: that means the allocation candiates for those request groups will have to be reported sepreately | 20:49 |
efried | sean-k-mooney[m]: Which I don't agree with. Which was already discussed and decided in the original spec in Q. | 20:49 |
cfriesen | efried: hugepages are fine, they imply that we're limited to a numa node. instances with "shared" cpus and 4KB pages are allowed to float across the whole compute node currently | 20:49 |
*** archit has joined #openstack-nova | 20:49 | |
sean-k-mooney[m] | cfriesen: for 4k we can map a file too if we want | 20:49 |
cfriesen | sean-k-mooney[m]: can we map multiple files for an instance with a single virtual numa node? | 20:50 |
sean-k-mooney[m] | cfriesen: we just dont but we can numa afinites 4k pages | 20:50 |
cfriesen | sean-k-mooney[m]: yes, but can we allocate 3GB of 4K pages from host numa node 0 and 1GB from host numa node 1 for an instance with a single virtual numa node? | 20:51 |
sean-k-mooney[m] | cfriesen: i think so. why would you want too | 20:52 |
sean-k-mooney[m] | to allow the memory to come form multiple host numa nodes | 20:52 |
cfriesen | sean-k-mooney[m]: If I have only 3GB memory free on one numa node and 1GB on the other, and I want to keep the current behaviour of letting the instance float across the whole compute node. | 20:53 |
*** elmaciej has quit IRC | 20:53 | |
jaypipes | sean-k-mooney[m]: again, I think that granular request groups should mean that the resources in each request group are provided by different resource providers. | 20:54 |
sean-k-mooney[m] | i would have to check. we added the abiltiy to use file desciptor memory by setting the souce elemet of this https://libvirt.org/formatdomain.html#elementsMemoryBacking | 20:54 |
jaypipes | sean-k-mooney[m]: I do not care about the use case of "general spreadability". | 20:54 |
efried | jaypipes: but only if they're the same resource class | 20:54 |
jaypipes | efried: yes. | 20:54 |
sean-k-mooney[m] | i know that backing file can be affinites but i dont know if we can create two backing files attach to the same guest numa node | 20:54 |
efried | jaypipes: That's gonna be tough to implement, just for starters. | 20:55 |
jaypipes | efried: though I don't see a reason why you would separate request groups where one request group does *not* contain a resource class... | 20:55 |
efried | jaypipes: So that they aren't forced to land on the *same* RP. | 20:55 |
sean-k-mooney[m] | cfriesen: i think it would work at the qemu level but i need to dig deeper | 20:55 |
jaypipes | efried: say wha? | 20:55 |
cfriesen | sean-k-mooney[m]: unless we can do that, we can't let a single-numa-node guest use memory from multiple host numa nodes. | 20:56 |
sean-k-mooney[m] | jaypipes: ya i actully prefer that each group is a different RP. that is what i had originally assumed | 20:56 |
jaypipes | efried: I mean, I don't see a use case for doing, for example, this: resources1=VCPU:1,required2=HW_NIC_OFFLOAD_GENEVE | 20:56 |
efried | jaypipes: It wouldn't make sense for me to say DISK_GB:1024,VF:1. Maybe I misunderstood your statement. | 20:56 |
jaypipes | efried: that doesn't make sense to me. | 20:56 |
sean-k-mooney[m] | cfriesen: well thats the thing they are not really singel numa guests | 20:57 |
efried | jaypipes: resources1=VF:1,BW:200&required1=PHYSNET_A&resources2=VF:1,BW:300&required2=REALLY_FAST | 20:57 |
sean-k-mooney[m] | they are guest that did no specify a numa topology so there is no reason nova could not make them multi numa guests | 20:58 |
jaypipes | efried: ok, and? | 20:58 |
jaypipes | efried: that makes total sense to me. | 20:58 |
cfriesen | sean-k-mooney[m]: ooh, fun. that won't surprise anyone. :) | 20:58 |
efried | Contrived example. I don't care which physnet that second VF is on. I just want it to be fast. Why can't it land on the same RP as the first one? | 20:58 |
cfriesen | sean-k-mooney[m]: don't forget we'd have to preserve the numa topology over live migration | 20:58 |
sean-k-mooney[m] | cfriesen: if we made them multi numa guest when there resouces were split across host numa nodes it would fix the accounting issue and improve the performance of the guest | 20:59 |
*** edmondsw has joined #openstack-nova | 20:59 | |
jaypipes | efried: because it's just crazy to reason about for the user, frankly. | 20:59 |
sean-k-mooney[m] | cfriesen: your relying on an implentation deatil that i vift-driver specific | 20:59 |
cfriesen | sean-k-mooney[m]: it could also make the guest slower if it's OS isn't numa-aware. | 20:59 |
jaypipes | efried: if a user requests two groups of a resource, the user expects those groups to be separate. | 21:00 |
jaypipes | efried: and "separate" means "provided by different providers" in my and dansmith's book. | 21:00 |
sean-k-mooney[m] | cfriesen: the guest os | 21:00 |
*** edmondsw_ has joined #openstack-nova | 21:00 | |
cfriesen | sean-k-mooney[m]: yep | 21:00 |
efried | jaypipes: disagree completely. IMO it's way easier for the user to reason about the groups individually, without regard for their interrelations. | 21:00 |
sean-k-mooney[m] | cfriesen: how would it make it slow when before it would have been spread on the phyical host but it never would have knonw | 21:01 |
*** dave-mccowan has quit IRC | 21:01 | |
jaypipes | efried: I'm afraid we'll just have to agree to disagree. | 21:01 |
efried | jaypipes: Fact is, either way we decide this will leave a hole, cases we can't express. | 21:01 |
cfriesen | sean-k-mooney[m]: code running in the guest on two separate cpus that wants to share a lot of memory between the two CPUs. now you've got cross-NUMA latencies | 21:01 |
jaypipes | efried: true. | 21:01 |
efried | jaypipes: The way it's currently written is simpler, more flexible, and easier to implement; and IMO easier to reason about. But that may just be because it's what I've had in my head since early Q. | 21:02 |
cfriesen | sean-k-mooney[m]: you're correct that it's no worse than it is now | 21:03 |
sean-k-mooney[m] | cfriesen: yes and today that can happen too. the guest sees 1 numa node but on the host ist ram can be allocate form multiple because it floats | 21:03 |
jaypipes | My brain is, frankly, pretty fried. | 21:03 |
efried | I've thought through all the things you can't express with it, and it boils down to just one thing: you can't express "separate these request groups". | 21:03 |
jaypipes | I really need to eat and refresh my brain juices. | 21:03 |
efried | ...but adding that feature would be relatively straightforward with the addition of some new syntax. | 21:03 |
*** edmondsw has quit IRC | 21:04 | |
efried | Yeah, for my part I really ought to go review some more specs that aren't already approved. | 21:04 |
*** edmondsw_ has quit IRC | 21:05 | |
*** moshele has joined #openstack-nova | 21:05 | |
*** salv-orl_ has joined #openstack-nova | 21:06 | |
sean-k-mooney[m] | cfriesen: :) so i was suggesting was if you did not spcify a numa topology then we leave it up to the virt-driver but it then has to reflect the topolgy to the guest. it would be no wose then today but actully fix the 4k pages issue and technically i dont think we are breaking any guartees as i dont hink we define that if numa_nodes is not set its 1 numa node | 21:06 |
*** salv-orlando has quit IRC | 21:06 | |
*** yamamoto has joined #openstack-nova | 21:07 | |
sean-k-mooney[m] | cfriesen: anyway that an issue for another day | 21:07 |
*** eharney has quit IRC | 21:08 | |
*** shaohe_feng has quit IRC | 21:11 | |
openstackgerrit | Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247 | 21:11 |
openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Add Generation to Consumers https://review.openstack.org/556971 | 21:11 |
edleafe | cdent: efried: jaypipes: ^^ updated with your suggestions | 21:12 |
*** yamamoto has quit IRC | 21:12 | |
efried | edleafe: Check yer whitespace | 21:14 |
edleafe | efried: ugh, do I have to? <whine> | 21:15 |
efried | edleafe: But hold, there's another typo | 21:15 |
*** itlinux has quit IRC | 21:15 | |
efried | edleafe: And another thing that needs to be said. | 21:16 |
*** liangy has quit IRC | 21:16 | |
edleafe | well, I wasn't gonna push a rev for whitespace until everyone had a crack at it | 21:16 |
*** archit is now known as amodi | 21:19 | |
*** itlinux has joined #openstack-nova | 21:20 | |
efried | edleafe: Never mind that last thing. I was gonna ask if the generation in PUT /allocations/{c} was going to quit being ignored. But that's a RP generation, and not really relevant here. | 21:20 |
*** amodi_ has joined #openstack-nova | 21:21 | |
efried | edleafe: Consider me cracked. | 21:21 |
*** amodi has quit IRC | 21:21 | |
*** amodi_ is now known as amodi | 21:21 | |
*** vivsoni has quit IRC | 21:21 | |
sean-k-mooney[m] | you know that feeling when you find the souce of a bug you hit and you wish you had not... | 21:22 |
sean-k-mooney[m] | here we convert form the disk bus the use asked for to a prefix https://github.com/openstack/nova/blob/ef0ce4d692d28a7f5a0079e24acdbfe7d2767e8b/nova/virt/libvirt/blockinfo.py#L124-L143 | 21:23 |
cdent | cracked | 21:23 |
sean-k-mooney[m] | here we convert form the prefix to the disk bus https://github.com/openstack/nova/blob/ef0ce4d692d28a7f5a0079e24acdbfe7d2767e8b/nova/virt/libvirt/blockinfo.py#L297-L304 | 21:23 |
*** lpetrut has quit IRC | 21:23 | |
sean-k-mooney[m] | we map sata scis and usb to sd then hard code sd = scsi | 21:23 |
edleafe | efried: I always have | 21:24 |
sean-k-mooney[m] | that mean on kvm you can request sata or usb disk which is why i had to go to IDE to fix my centos image | 21:24 |
sean-k-mooney[m] | *cant request | 21:25 |
*** r-daneel has quit IRC | 21:25 | |
openstackgerrit | Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247 | 21:26 |
*** vivsoni has joined #openstack-nova | 21:26 | |
sean-k-mooney[m] | setting the disk_bus for kvm host has apreantely been broken for 5 years since https://github.com/openstack/nova/commit/7be531fe9462f2b07d4a1abf6687f649d1dfbb89 was merged ... | 21:27 |
*** felipemonteiro__ has quit IRC | 21:28 | |
openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Add Generation to Consumers https://review.openstack.org/556971 | 21:28 |
openstackgerrit | Merged openstack/nova-specs master: Amend the migration paging spec for uuid in server migrations response https://review.openstack.org/532904 | 21:29 |
edleafe | cdent: efried: jaypipes: ^^ fixed those ghastly errors | 21:31 |
efried | edleafe: +1 | 21:31 |
*** edmondsw has joined #openstack-nova | 21:35 | |
mriedem | jaypipes: i'll come back to the mirror aggregates in placement spec, but not today, burned out | 21:35 |
*** fragatina has quit IRC | 21:35 | |
*** fragatina has joined #openstack-nova | 21:35 | |
*** moshele has quit IRC | 21:36 | |
*** edmondsw has quit IRC | 21:36 | |
openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Ensure that os-traits sync is attempted only at start of process https://review.openstack.org/553857 | 21:37 |
cdent | thanks for putting some eyes on the error code spec jaypipes | 21:39 |
*** shaohe_feng has joined #openstack-nova | 21:39 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Proposes Multiple GPU types https://review.openstack.org/557065 | 21:41 |
bauzas | jaypipes: mriedem: just created the spec we discussed about multiple types | 21:41 |
bauzas | it's a very simple spec, but needs agreement | 21:42 |
*** cdent has quit IRC | 21:42 | |
openstackgerrit | Merged openstack/nova-specs master: Address review comments from afdc828db3c9d0205b6ded268db24f5cdf857fa6 https://review.openstack.org/554251 | 21:43 |
*** burt has quit IRC | 21:45 | |
mriedem | bauzas: i don't remember talking about that at all | 21:46 |
melwitt | bauzas: on https://blueprints.launchpad.net/nova/+spec/vgpu-rocky it looks like you can just use that bp to link with your spec since it was meant to cover the multiple vgpu types part anyway. the spec is just needed to facilitate discussion on the details, right? | 21:46 |
mriedem | i might have been in ireland when it happened, but didn't talk about it | 21:46 |
*** mriedem is now known as mriedem_afk | 21:47 | |
*** esberglu has quit IRC | 21:47 | |
*** felipemonteiro has joined #openstack-nova | 21:47 | |
melwitt | yeah, I don't remember particulars about the vgpu discussion other than, there's more work to do this cycle and we're agreed to do it and review it | 21:47 |
*** esberglu has joined #openstack-nova | 21:48 | |
bauzas | mriedem: no worries, I pinged you a while ago just about whether I should use another BP for it, and you told me to ping melwitt | 21:48 |
*** esberglu has quit IRC | 21:48 | |
bauzas | melwitt: yup, zactly | 21:48 |
bauzas | melwitt: I linked that BP to the spec | 21:48 |
melwitt | okay, cool. thanks | 21:48 |
bauzas | so, I just want to make clear that the spec only covers the problem in it | 21:49 |
*** esberglu has joined #openstack-nova | 21:49 | |
bauzas | which requires a consensus, hence a spec | 21:49 |
bauzas | other feature patches won't need it | 21:49 |
melwitt | right, just have to agree how to implement the multiple types | 21:49 |
melwitt | yeah | 21:49 |
bauzas | I'd love jianghuaw_ to voice on that spec too | 21:50 |
*** esberglu has quit IRC | 21:53 | |
bauzas | anyway, calling it a day \o | 21:54 |
*** gouthamr has joined #openstack-nova | 21:58 | |
*** trozet has quit IRC | 21:59 | |
*** amodi has quit IRC | 22:00 | |
*** salv-orl_ has quit IRC | 22:01 | |
*** esberglu has joined #openstack-nova | 22:01 | |
*** salv-orlando has joined #openstack-nova | 22:01 | |
*** gouthamr has quit IRC | 22:04 | |
*** itlinux has quit IRC | 22:05 | |
*** esberglu has quit IRC | 22:05 | |
*** salv-orlando has quit IRC | 22:06 | |
*** yamamoto has joined #openstack-nova | 22:08 | |
jaypipes | mriedem_afk: you and me both, man :) | 22:11 |
*** ttsiouts_ has quit IRC | 22:12 | |
*** mchlumsky has quit IRC | 22:12 | |
*** yamamoto has quit IRC | 22:14 | |
*** hemna_ has quit IRC | 22:14 | |
*** mikal has joined #openstack-nova | 22:15 | |
*** trozet has joined #openstack-nova | 22:15 | |
*** lbragstad has quit IRC | 22:15 | |
*** rcernin has joined #openstack-nova | 22:16 | |
*** mikal_ has quit IRC | 22:18 | |
openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Add Generation to Consumers https://review.openstack.org/556971 | 22:21 |
edleafe | ^^ damn rST double colons! | 22:21 |
*** fragatina has quit IRC | 22:21 | |
*** fragatina has joined #openstack-nova | 22:21 | |
*** artom has joined #openstack-nova | 22:24 | |
*** germs has joined #openstack-nova | 22:31 | |
*** germs has quit IRC | 22:31 | |
*** germs has joined #openstack-nova | 22:31 | |
*** yamahata has joined #openstack-nova | 22:31 | |
*** lbragstad has joined #openstack-nova | 22:32 | |
*** lbragstad has quit IRC | 22:32 | |
*** germs has quit IRC | 22:36 | |
sean-k-mooney[m] | bauzas: can you review the bug fix i have up for the libvirt mtu here https://review.openstack.org/#/c/553072/ when you have time. i think this is also a good backport candiate. | 22:37 |
sean-k-mooney[m] | i also found another lovely bug today https://bugs.launchpad.net/nova/+bug/1759420 | 22:38 |
openstack | Launchpad bug 1759420 in OpenStack Compute (nova) "nova does not correctly support HW_DISK_BUS=sata or usb for kvm/qemu" [Undecided,New] | 22:38 |
*** felipemonteiro has quit IRC | 22:38 | |
sean-k-mooney[m] | because of ^ i had to for a centos vm that was given to me to run with disks attached to ide because it was freaking out with scsi or virtio disks. the virtio_blk and virtio_scsi drivers were causeing the kernel to lock up | 22:40 |
*** andreas_s has joined #openstack-nova | 22:47 | |
openstackgerrit | Eric Fried proposed openstack/nova master: Remove usage of [placement]os_region_name https://review.openstack.org/557086 | 22:47 |
efried | edleafe: ^ | 22:47 |
*** andreas_s has quit IRC | 22:52 | |
openstackgerrit | Eric Fried proposed openstack/nova master: Slugification utilities for placement names https://review.openstack.org/556628 | 22:56 |
*** yassine has quit IRC | 22:56 | |
*** claudiub|2 has quit IRC | 22:57 | |
openstackgerrit | Michael Still proposed openstack/nova master: Move configurable mkfs to privsep. https://review.openstack.org/551921 | 22:58 |
openstackgerrit | Michael Still proposed openstack/nova master: Move xenapi xenstore_read's to privsep. https://review.openstack.org/552241 | 22:58 |
openstackgerrit | Michael Still proposed openstack/nova master: Move xenapi disk resizing to privsep. https://review.openstack.org/552242 | 22:58 |
openstackgerrit | Michael Still proposed openstack/nova master: Move xenapi partition copies to privsep. https://review.openstack.org/553605 | 22:58 |
openstackgerrit | Michael Still proposed openstack/nova master: Move image conversion to privsep. https://review.openstack.org/554437 | 22:58 |
openstackgerrit | Michael Still proposed openstack/nova master: We no longer need rootwrap. https://review.openstack.org/554438 | 22:58 |
openstackgerrit | Michael Still proposed openstack/nova master: We don't need utils.trycmd any more. https://review.openstack.org/554439 | 22:58 |
*** fragatina has quit IRC | 22:58 | |
*** fragatina has joined #openstack-nova | 22:58 | |
*** hongbin has quit IRC | 22:58 | |
*** gouthamr has joined #openstack-nova | 22:59 | |
mikal | efried: I've replied to your comments on the first patch, but the only changes were in comments. I'm not sure that was worth a -1. | 22:59 |
sean-k-mooney[m] | mikal: no more rootwap will be quite nice to see :) | 23:01 |
sean-k-mooney[m] | mikal: also i like you blueprint url | 23:01 |
mikal | sean-k-mooney[m]: ta, I do try. Other blueprints you might enjoy include execs-ive-had-a-few... | 23:01 |
*** salv-orlando has joined #openstack-nova | 23:02 | |
*** yassine has joined #openstack-nova | 23:06 | |
mlavalle | jaypipes: could we structure this RP tree https://review.openstack.org/#/c/502306/22/specs/rocky/approved/bandwidth-resource-provider.rst@432 so an allocation can be satisfied by ANY of the networks in the tree? | 23:06 |
*** felipemonteiro has joined #openstack-nova | 23:07 | |
*** salv-orlando has quit IRC | 23:08 | |
*** yamamoto has joined #openstack-nova | 23:10 | |
jaypipes | mlavalle: not sure I'm following your question... | 23:11 |
*** AlexeyAbashkin has joined #openstack-nova | 23:12 | |
mlavalle | jaypipes: I'll pose my question in the spec be cause I have to run in 15 minutes | 23:12 |
mlavalle | thanks for replying | 23:12 |
*** fragatina has quit IRC | 23:14 | |
*** fragatina has joined #openstack-nova | 23:14 | |
*** yamamoto has quit IRC | 23:15 | |
*** AlexeyAbashkin has quit IRC | 23:17 | |
*** suresh12 has quit IRC | 23:19 | |
*** awaugama has joined #openstack-nova | 23:19 | |
*** suresh12 has joined #openstack-nova | 23:19 | |
*** suresh12 has quit IRC | 23:24 | |
*** suresh12 has joined #openstack-nova | 23:25 | |
*** felipemonteiro has quit IRC | 23:25 | |
*** harlowja has quit IRC | 23:32 | |
*** mlavalle has quit IRC | 23:32 | |
*** gouthamr has quit IRC | 23:35 | |
*** chyka has quit IRC | 23:35 | |
*** germs has joined #openstack-nova | 23:35 | |
*** germs has quit IRC | 23:35 | |
*** germs has joined #openstack-nova | 23:35 | |
*** chyka has joined #openstack-nova | 23:35 | |
*** germs has quit IRC | 23:36 | |
*** germs has joined #openstack-nova | 23:36 | |
*** germs has quit IRC | 23:36 | |
*** germs has joined #openstack-nova | 23:36 | |
*** chyka has quit IRC | 23:39 | |
*** mriedem_afk has quit IRC | 23:43 | |
*** takashin has joined #openstack-nova | 23:44 | |
*** awaugama has quit IRC | 23:44 | |
*** yamahata has quit IRC | 23:44 | |
*** hshiina has joined #openstack-nova | 23:55 | |
*** germs has quit IRC | 23:58 | |
*** germs has joined #openstack-nova | 23:59 | |
*** germs has quit IRC | 23:59 | |
*** germs has joined #openstack-nova | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!