14:59:50 #startmeeting gantt 14:59:51 Meeting started Tue May 6 14:59:50 2014 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:59:52 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:59:55 The meeting name has been set to 'gantt' 15:00:09 hi bauzas 15:00:10 hi all 15:00:15 hi 15:00:19 hi all 15:00:25 n0ano is off today, so I'm chairing this one 15:00:45 just before we start, I would like to add into 'Open' a topic 15:00:53 on a new blueprint that we proposed 15:00:55 toan-tran: sure, go ahead 15:01:12 we can even create a dedicated topic for your needs ;) 15:01:19 :) 15:01:35 could I just have the title ? 15:01:46 cpu allocation per flavor 15:01:57 cpu allocation ratio per flavor 15:02:01 :) 15:02:06 ok, keep me aware if I'm forgetting it :) 15:02:16 will do :) 15:02:18 let's start 15:02:30 hello all 15:02:44 * johnthetubaguy waves 15:02:46 #topic Actions items from previous meetings 15:03:02 (argh, made a typo) 15:03:19 #link http://eavesdrop.openstack.org/meetings/gantt/2014/gantt.2014-04-29-15.00.html 15:03:37 a few actions were created 15:03:42 YorikSar, you there ? 15:03:59 bauzas: o/ 15:04:05 YorikSar: cool 15:04:08 an action was open for you 15:04:14 to create the nova-specs bp 15:04:16 I saw it 15:04:18 I've put first version of the spec to the Gerrit. 15:04:23 cool, nice work 15:04:31 could you please give us the link ? 15:04:37 I've added those who I could find in last meeting logs to reviewers 15:04:41 cool 15:04:46 YorikSar: nice catch 15:04:53 #link https://review.openstack.org/92128 15:04:55 thanks 15:05:04 (I hope that'll work for meeting bot) 15:05:08 it will 15:05:33 let's discuss it on a separate topic 15:05:50 I'm looking forward to reviews. I've never put together a blueprint with this new template in repo 15:05:54 ok, n0ano also had action to look over https://review.openstack.org/82778 15:05:56 bauzas: Sure 15:06:13 I guess he had no time 15:07:02 IMHO, prolonging this action doesn't make sense as we're 1 week close to the summit 15:07:16 so, let's move to the next topic 15:07:24 unless someone is feeling different :) 15:07:44 no, please proceed :) 15:07:50 ok 15:07:52 #topic Status on forklift efforts 15:08:13 well, far less progress here, I was really busy by my vacations 15:08:36 but still, there 3 things you need to know 15:08:57 #link https://review.openstack.org/82133 is merged 15:09:20 so I'll focus my attention on delivering first implementation by the next weeks 15:09:32 of course, progress will be impacted by Summit 15:09:39 bauzas: super 15:09:49 implementation of this blueprint can be found here as draft : 15:10:02 #link https://review.openstack.org/82778 15:10:21 bauzas: just a question 15:10:24 sure 15:10:36 this blueprint is targeted for Juno-1 15:10:43 in the srt you said that only aggregate tables & vm tables are concerned 15:10:50 what about host_state? 15:10:59 computes_* tables? 15:11:26 I haven't said that in this blueprint ;,) 15:11:40 that's another blueprint, targeted for Juno-2 15:11:50 oops, sorry, wrong blueprint :) 15:11:55 #link https://review.openstack.org/#/c/89893/ 15:12:04 but indeed, there is room for discussion here 15:12:18 so, yes, HostState is a class, not a DB table 15:12:33 bauzas: the table is compute_* 15:12:37 HostState is persisted by compute_nodes 15:13:01 At least for now... 15:13:03 johnthetubaguy: I'm pretty well interested in your feedback for https://review.openstack.org/#/c/89893/ 15:13:21 YorikSar: at least until no-db-scheduler is merged yes :) 15:13:24 bauzas: yeah, I will take a look, interested to get this reviewed before the summit 15:13:43 johnthetubaguy: I'm planning to discuss over it by the Gantt session 15:13:55 johnthetubaguy: because there are some performance concerns as reviews 15:14:16 johnthetubaguy: if you look at the discussion in the previous patchset 15:14:38 johnthetubaguy: so, I think that's a good fit for trying to get opinions before Summit 15:15:15 bauzas: agreed, its a good topic to discuss, and there are a few performance things that worry me at least, but I feel there are some good options to fix those 15:15:36 indeed, and that's good place for discussing it at Summit 15:15:44 yup 15:15:56 ok, any other questions about these 2 blueprints ? 15:16:04 some filters use compute_nodes tables 15:16:14 which is updated by nova periodic tasks 15:16:24 how to separate them? 15:16:35 compute_nodes is planned to stay where it is :) 15:16:56 bauzas: then how corefilter & ramfilter work then? 15:17:01 thanks to the first bp (sched-lib), RT will update compute_nodes seamlessly 15:17:33 compute_nodes is planned to stay on the Gantt side of the whiteline 15:17:42 so filters won't be impacted 15:17:54 RT will be 15:18:02 hence the sched-lib blueprint 15:18:22 ok 15:18:46 another question: how this bp & no-db work together? 15:18:58 toan-tran: that's an excellent question 15:19:15 toan-tran: I guess the nova-spec review for no-db should sort that out, it changes things slightly 15:19:21 toan-tran: and as for all excellent questions, it doesn't have an answer yet :) 15:19:34 bauzas: can anything that this bp pull out be used in no-db? 15:19:53 toan-tran: AIUI, nope 15:19:57 meaning: instead of putting it in a separated db, put it in memcache? 15:20:15 toan-tran: ideally, access to DB should be proxified 15:20:35 toan-tran: so whatever the backend is, the calls would stay the same 15:20:40 bauzas: +1 15:20:43 so I think the no-db scheduler will be a new scheduler-lib, if we get it right 15:21:03 woah 15:21:19 good place for discussion over here then 15:21:51 YorikSar: what do you think? 15:21:54 I was just thinking that instead of calling db_api.whatever() we were calling nodb.whatever() 15:22:19 bauzas: well, it's more complicated than that 15:22:30 well, I love complications :) 15:22:36 db_api has its own model 15:22:52 I don't know if memcache has similar thing 15:22:57 no-db scheduler only delivers host stats to the scheduler. I guess it can be a start for a scheduler-lib 15:23:01 hence the word 'proxy' :) 15:23:56 YorikSar: have you validated scheduler-lib interfaces ? 15:23:57 In current state it "simply" replaces calls like db_api.put_my_state and db_api.get_all_states with requests to synchronizer. 15:24:32 bauzas: Nope. 15:24:39 but calls to db_api.put_my_stats will now go to sched_lib.update_stats() 15:24:55 I guess I should take a closer look to all those blueprints. 15:25:07 so I think you could take use of sched-lib as facade for your own implem 15:25:33 so that the interface would stay the same 15:25:43 bauzas: Then we'll replace whatever is inside sched_lib.update_stats with calls with call to synchronizer. 15:25:50 bauzas: Yes, exactly. 15:25:59 that's my idea yes 15:26:15 well, at least it should be optional and flag-driven :D 15:26:34 as I said, a common facade for 2 implems 15:26:36 bauzas: We'll see ;) 15:27:00 YorikSar: I suggest you to look at https://review.openstack.org/#/c/82133/19/specs/juno/scheduler-lib.rst 15:27:14 you'll find the specs for the interface 15:27:15 I think once people take a look at what those changes actually do they won't be afraid of switching on-the-fly. 15:27:42 maybe it's worth moving from this topic to the no-db sched one ? 15:27:53 so we could easily discuss on this 15:28:04 unless someone wants to talk about sched forklift ? 15:28:17 bauzas: I'll take a look at blueprints tomorrow. 15:28:42 and I also have to review yours 15:29:17 #topic no-db scheduler 15:29:35 #action bauzas to review https://review.openstack.org/92128 15:30:08 YorikSar: do you have other things to raise over this topic ? 15:30:23 I mean, about design or queries? 15:30:44 bauzas: I guess, I'll be waiting for reviews on spec. 15:31:13 I've came onto one problem though. It's about compute_nodes you've mentioned. 15:31:20 YorikSar: sure ? 15:31:28 YorikSar: what's your problem ? 15:31:37 I think they should go. :) 15:31:45 lol 15:31:55 They hold mostly info used for scheduling only. 15:32:14 then, I would have to say to leave them where it is :) 15:32:40 bauzas: I remember Boris discussed it long ago 15:32:54 And when I took a look at what should be moved to one big JSON state, it seemed to me that all fields there have nothing (little) to do with the rest of Nova. 15:32:56 saying that the Synchronizer will take care of that 15:33:07 mmm interesting view 15:33:23 if you look at the Juno-2 bp for isolating DB scheduler 15:33:30 I have a problem over here 15:34:00 I mean, we say that scheduler should not access other tables but the ones he manages for persistence 15:34:08 There are places where compute_nodes are requested along with stats just for couple fields stored in them. I'm not sure how to handle that. 15:34:25 but the problem is that scheduler is directly reading other Nova objects states, like aggregates 15:34:30 * YorikSar adding another blueprint to the reading list... 15:34:51 so, if we say that compute_nodes should get rid off 15:35:02 with a big JSON state 15:35:22 that means that related objects should also be considered as this 15:35:30 aggregates and instances at least 15:36:02 well, to be honest, JSON and DB are 2 edges of the same thing 15:36:05 bauzas: I think we should separate data that relates to hosts from data that relates to instances. 15:36:13 except the persistence thing of course 15:36:43 So aggregates are for hosts, so they should go to (dark) scheduler side. 15:36:46 YorikSar, is the instance state you are thinging of independent of host 15:36:49 could you please keep it in mind for your blueprint ? 15:37:44 I'm also wondering how no-db sched will manage extensible RT 15:38:06 as the bp is now validated, unless I'm wrong PaulMurray? 15:38:17 bauzas, waiting for a second +2 15:38:41 PaulMurray: ok cool 15:38:56 PaulMurray: I guess scheduler will have to manage a list of resources anyway. So at least list of instances on the host will remain in scheduler. 15:38:56 bauzas, most code is done - just the spec to go :) 15:39:11 PaulMurray: I saw that RT using objects is also validated as bp 15:39:21 bauzas, yp 15:39:25 PaulMurray: yep, I had no chance to review latest pachsets 15:39:27 s/yp/yep/ 15:39:38 (about extensible RT I mean) 15:40:31 bauzas, my biggest problem was I had a url longer than 79 characters 15:40:35 so, YorikSar that means I think that the no-db sched bp should only focus on access to DB and proxy them 15:41:04 PaulMurray: oh bad, have you had -1 for this ? 15:41:13 from jenkins 15:41:18 PaulMurray: dammit 15:41:25 PaulMurray: that's pretty much the problem with jenkins 15:41:34 :D 15:41:40 no worries - I made the world pep8 compliant so I could use a shorter url to refer to 15:41:59 is there a PEP8 gate now for nova-specs ? 15:42:04 wasn't aware of 15:42:06 bauzas: Yes. It's just proxying calls that used to target DB to wibbly-wobbly vortex that just delivering data to scheduler. 15:42:19 bauzas, no, not really, just 79 character width limit 15:42:34 YorikSar: ok cool so that should be not impacting other bps 15:42:51 provided the DB API remains the same :) 15:42:55 PaulMurray: I've pasted some long URLs to my spec and Jenkins never complained. Mb you should have only URL on the line to make it pass?.. 15:43:15 YorikSar: +1 15:43:27 YorikSar: I never went thru this problem 15:43:30 YorikSar, maybe - the check only failed today - it past last week 15:43:37 strange 15:43:53 ok, could we consider to move to the Design sessions topic ? 15:43:55 Maybe they've changed coins in cointosser 15:44:23 bauzas: Surt 15:44:29 #topic Juno Summit design sessions 15:45:10 #link https://etherpad.openstack.org/p/Gantt-summit-sessions 15:45:35 here is the final planning for scheduler-related sessions 15:46:31 I please ask proposers to fill in https://wiki.openstack.org/wiki/Summit/Juno/Etherpads#Nova 15:46:53 I began to put some ideas in https://etherpad.openstack.org/p/juno-nova-gantt-apis 15:46:59 that's draft of course 15:47:08 is there a link for all the session plan? 15:48:01 toan-tran: which kind of link do you need ? 15:48:19 well, a link that shows all the date & time of the sessions 15:48:29 ah 15:48:33 http://junodesignsummit.sched.org/ only shows presentation session, not degisn sessions 15:48:37 take the sched.org things 15:48:47 toan-tran: you made confusion ;) 15:49:06 toan-tran, https://wiki.openstack.org/wiki/Summit/Juno 15:49:10 toan-tran: http://junodesignsummit.sched.org/ is the Design sumit 15:49:22 bauzas: ok :D 15:49:59 http://openstacksummitmay2014atlanta.sched.org/ is for the regular Openstack Summit 15:50:08 bauzas: right :) 15:50:53 just a quick reminder that you can see other people's attendance by looking over each person agenda 15:51:31 like mine http://openstacksummitmay2014atlanta.sched.org/ 15:51:33 oops 15:51:36 http://openstacksummitmay2014atlanta.sched.org/sbauza 15:51:41 etc. 15:51:55 bauzas, also if you put a picture in it makes it a lot easier for people to find you (if you want to be found :) 15:52:04 PaulMurray: +1 :) 15:52:15 PaulMurray: even if the picture is 4 years old... 15:52:17 bauzas, by "you" I mean "one" of course 15:52:27 bauzas, all mine are 10 years old 15:52:31 :) 15:52:33 I still look the same :) 15:52:46 ok, we still have 2 topics to discuss and 8 mins left 15:52:54 sorry for rushing this 15:53:10 #topic cpu allocation ratio per flavor 15:53:18 toan-tran: you're up 15:53:25 thanks 15:53:37 Here is the spec: https://review.openstack.org/#/c/87213/ 15:53:42 #link https://review.openstack.org/#/c/87213/ 15:53:56 currently the cpu_allocation_ratio is put into the aggregate 15:54:20 thus a host will accept the VMs within its ratio 15:54:36 however 15:54:56 it is sometimes prefereable that the we want to put it in the flavor 15:55:14 for instance: 15:55:35 Flavor 1 has cpu_ratio 1 15:55:42 Flavor 2 has cpu_ratio 2 15:56:05 meaning that the first flavor will not accept sharing its core with others 15:56:21 while flavor 2 can share up to 1 more VM in the same core 15:56:40 so if a host has 12 cores 15:57:07 he can accept 12 VMs of flavor 1, or 24 VMs of flavor 2, or 6 from flavor 1 + 12 from flavor 2 15:57:18 ok, I see 15:57:24 that's what we want to realize 15:57:32 we're running out of time, I can propose you to discuss over this first in the bp 15:57:38 toan-tran, have you looked at extensible RT 15:57:50 https://review.openstack.org/#/c/86050/ 15:57:57 PaulMurray: +1 15:58:09 PaulMurray: not yet, 15:58:10 toan-tran, could be useful 15:58:16 yeah 15:58:24 it sounds like what we want in conjunction 15:58:25 ok, we're definitely eating time 15:58:43 any other opens to place before Summit ? 15:58:45 actually there are several methods, including cgroup & quota:cpu 15:59:40 guys, we're running out of time 16:00:01 well, I guess I can put back the bp for another time then :) 16:00:03 I can propose to discuss over other topics outside here 16:00:14 cool thanks 16:00:16 #endmeeting