00:01:05 <ekcs> #startmeeting congressteammeeting
00:01:06 <openstack> Meeting started Thu Apr 20 00:01:05 2017 UTC and is due to finish in 60 minutes.  The chair is ekcs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:01:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:01:09 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:14 <ekcs> hi ramineni_ . welcome back!
00:02:31 <ekcs> Hi all! as usual, the proposed topics are here. please take a few moments to take a look and feel free to add more.
00:02:39 <ekcs> #link https://etherpad.openstack.org/p/congress-meeting-topics
00:03:04 <masahito> hi
00:05:00 <ekcs> hi masahito !
00:06:19 <ekcs> ok let’s get started then.
00:06:34 <ekcs> #topic announcements
00:07:21 <ekcs> Pike-1 was released last week. We got a fair amount of stuff in there, but definitely a lot more to be done in the next two milestones =)
00:07:32 <ekcs> Not much more to say there.
00:07:35 <ekcs> Any other announcements?
00:08:01 <ramineni_> ekcs: when is pike2 targted?
00:08:47 <ekcs> ramineni_: week of Jun 5
00:09:18 <ramineni_> ekcs: ohok
00:10:08 <ekcs> ok moving on then.
00:10:15 <ekcs> #topic patches
00:11:09 <ekcs> ramineni_: we discussed your datasource page layout patch last time.
00:11:13 <ekcs> #link https://review.openstack.org/#/c/445371/
00:11:16 <ekcs> we can continue today.
00:11:27 <ramineni_> ekcs: yes
00:11:58 <ekcs> To summarize, I think most of us thought it’s great to separate the datasource tables into their individual datasource pages.
00:12:47 <ekcs> ekcs and thinrichs also think that the current Service Status portion of the datasources page is important and it’d be great to keep that information around in one page.
00:13:37 <ramineni_> sounds good. until I find a way to add service status into the first table, i can keep the service status table also listed
00:14:32 <ekcs> Both thinrichs and ekcs suggested that if it’s not easy to rework the new datasource listing to display the status info like before, we can just display the old Service Status table on top, in addition to the new datasouce listing.
00:15:05 <ekcs> Right.
00:15:15 <ekcs> So anything else to discuss there?
00:15:20 <ramineni_> ekcs: ok, sure, will update that
00:15:39 <ramineni_> ekcs: no
00:16:02 <ekcs> awesome.
00:16:11 <ekcs> ok any other patches we’d like to discuss here?
00:16:39 <ramineni_> ekcs: need one help
00:17:28 <ekcs> yups.
00:17:35 <ramineni_> ekcs: devstack patch I have added for dashboard project, works fine for me . Can anyone test it out the same for confirmation before merging , Ill update the patch today
00:18:13 <ekcs> This one? #link https://review.openstack.org/#/c/453474/
00:18:24 <ramineni_> ekcs: yes
00:18:51 <ramineni_> ill fix the jenkins today
00:19:38 <ekcs> Silly question: how do you test it?
00:20:10 <ekcs> If I ./stack in devstack, it’ll grab the plugin from github repo right?
00:20:10 <ramineni_> ekcs: :)
00:20:28 <ekcs> But we want it to grab the plugin from this unmerged code.
00:20:44 <ramineni_> ekcs: i did with OFFLINE=True, it took the unmerged code
00:21:34 <ramineni_> ekcs: If you dont keep RECLONE=True, it desnt checkout the new code i guess, so takes the changes
00:21:42 <ekcs> I see. Could you add a comment or commit message in gerrit telling someone exactly how to do it? it’ll save like at least half an hour of someone else trying to figure it out.
00:22:06 <ramineni_> ekcs: sure, will do that
00:22:14 <ekcs> thanks!
00:22:37 <ekcs> any other patches to discuss?
00:23:08 <ramineni_> nothing from my side
00:23:56 <ekcs> ok let’s move on then.
00:24:02 <ekcs> #topic policy library
00:24:12 <ekcs> ok back at policy library again!
00:24:41 <ekcs> I’ve been working on a spec for the policy library. Here are some issues/questions that came up as I did that.
00:24:55 <ekcs> I’ll put them down here and if there are thoughts now, please discuss.
00:25:03 <ekcs> If not, we can move on and leave discussion to gerrit.
00:25:37 <ekcs> policy library spec #link https://review.openstack.org/#/c/457880/
00:25:52 <masahito> I'll check the spec
00:26:00 <ekcs> So the questions are:
00:26:16 <ekcs> 1. Assuming we want the policy files in yaml format, should we make the API itself accept yaml? or let GUI, client, and other users of the API convert yaml to json (straightforward)?
00:26:38 <ekcs> 2. If we decide for the API to accept yaml, when POSTing to ../policies?format=yaml, should the whole body be in yaml? or just the rules part?
00:26:47 <ekcs> 3. If we want policies in the library to be automatically updated in response to changes in the library directory, what do we do with multi-PE conflicts?
00:27:33 <masahito> For #1 it's better to only accept json format in server side.
00:28:00 <ekcs> masahito: makes sense.
00:28:05 <masahito> GUI and client parse the yaml to json and send a request.
00:28:19 <ekcs> Is thinrichs around? He suggested accepting yaml in API.
00:28:24 <thinrichs> Yep
00:29:21 <thinrichs> I guess I hadn't thought that far along
00:29:36 <thinrichs> The main thing to me was that people could write the policies in YAML
00:29:51 <thinrichs> But JSON seems a good approximation of that
00:30:13 <thinrichs> One thing I'm unclear on skimming thru the spec...
00:30:52 <thinrichs> If someone adds a policy to the library via the API, is that getting saved to files on disk?
00:31:28 <thinrichs> Or are the files on disk only useful for devstack/etc. to load the policy library?
00:31:33 <ekcs> So assume we write the policy files in YAML, there’s a choice between letting the API accept YAML, or letting the GUI/client/API caller convert the YAML to JSON. The main consideration is whether there would be an API caller that can’t easily to that conversion before calling.
00:32:10 <ramineni_> and one question, how are users going to custoimize the policicies in library, just editing the files in library?
00:32:20 <thinrichs> I think the YAML question might be answered by understanding how the files on disk relate to the policies in servers
00:32:34 <thinrichs> s/policies/policy libraries/
00:33:14 <thinrichs> One option is to create DB tables that store the policy library, and the files on disk are just used to populate that library.
00:33:54 <thinrichs> So that would involve persisting the policy library via the DB, just like we persist the regular policies (and rules) today.
00:34:16 <ekcs> thinrichs: ok. I’m not sure yet. If we want a changed/added file to be reflected in the server library (as bryan_att suggested), then it seems we would also want to sync the changes to the server library back to the files.
00:34:56 <thinrichs> I could imagine giving people an API that lets them download the current policy library and store them as YAML files.
00:35:22 <thinrichs> But esp. with multiple policy servers, persisting to files sounds unwieldy
00:35:59 <ekcs> But the syncing can lead to complications when there are multiple PEs. So I’m not sure we should really do the syncing at this point. (we all see dropbox and services like that occassionally encountering conflicts that required manual resolution)
00:36:14 <ekcs> right.
00:37:06 <ekcs> Personally I see the automatic update from changed files as a second/third order feature we could dispense with for now.
00:37:07 <thinrichs> I'd lean toward a model where we have APIs that make it easy to upload/download the files, but that the source of truth is always the DB
00:37:30 <ekcs> thinrichs: I’m thinking the same way.
00:38:02 <masahito> I haven't read the spec yet, but the libraries store policy as a file?
00:39:20 <masahito> I imagine DB only has a map, library name to policy/rule id and vice verse. then API only returns the collection of the policy/rule.
00:39:21 <ekcs> Here’s a good place to look at first real quick. #link https://docs.google.com/document/d/12f1VciulhT9yCYOc7jiulGiLT-tFpffLxNOpr-2QX2I/edit#
00:39:27 <ekcs> thinrichs put down proposed workflows.
00:39:51 <masahito> oops, ok. I'll also check.
00:40:20 <ekcs> masahito: that’s kind of like the short version of the spec. I’m writing the spec based on that short sketch.
00:43:30 <ramineni_> i think yaml accepts the customizable input through command line also?
00:43:55 <ramineni_> but may be ill also check the spec first, confused why storing library in DB is required
00:43:56 <ekcs> masahito: What I put in the spec is there are two new DB tables.
00:44:15 <ekcs> * A librarypolicies table to store the policies in the policies library.
00:44:15 <ekcs> * A libraryrules table to store the rules in the policies in the policies library.
00:45:00 <ekcs> The question is whether to keep that in sync with the files on disk in the policy-library directory.
00:46:32 <masahito> I thought congress server only keeps the uri of the libraries. like file:///foo or http://bar
00:46:45 <ekcs> ramineni_: I suppose we don’t technically need both DB and file. Perhaps we could just have one of the other.
00:47:39 <thinrichs> One thing I just realized: in the library we're thinking about policy+rules as the thing we are CRUDing, but in the existing policy API it's just the policy (not the rules) that we CRUD
00:48:03 <masahito> and reads the file from the uri, then it only create the library id in the table and use current policy/rule.
00:49:34 <ramineni_> and library policies are only templates , storing in file makes sense? and loading them with actual values can be stored in DB
00:50:03 <ekcs> masahito: I’m a littel confused. maybe tell us what tables and column you envision in your thinking?
00:51:03 <ekcs> ramineni_: in thinrichs ’s sketch, the idea is to forgo the concept of templates for now, but store actual concrete policy rules in the policy library. then let the user simply edit the policy rules before importing into policy engine.
00:52:43 <masahito> ekcs: new tables are "policy library" table and "map of a policy library to a rule" to handle the new APIs. and using existing 'policies' table and 'policy_rules' table to handle rules in the library.
00:53:01 <masahito> ekcs: I'll comment details in the spec
00:53:55 <ekcs> thinrichs: Not exactly sure what you mean. We do have CRUD of rules. #link https://docs.google.com/document/d/14hM7-GSm3CcyohPT2Q7GalyrQRohVcx77hxEx4AO4Bk/edit# The difference is before when you create a policy, it’s always empty. Now we want to allow creation of a policy with rules also provided in the request. Right?
00:54:54 <ekcs> masahito: ok. one quick clarification, in what you’re describing, every rule in the policy library is also stored in the policy_rules table right? So it needs to have a flag saying it’s not enabled?
00:55:20 <masahito> ekcs: yes.
00:55:32 <thinrichs> Right.  But we need to be careful when saying "policy": are we talking about the policy metadata that's stored in the DB today in the policy table, or are we talking about the policy metadata plus the rules and their metadata
00:56:04 <ekcs> Ok. glad we’re having this discussion because we have like 3-4 different understanding of how it’s done.
00:56:21 <ekcs> thinrichs: right.
00:56:28 <ekcs> great start. let’s continue in gerrit =)
00:56:36 <ekcs> last 4 minutes anything else to talk about?
00:56:39 <ekcs> #topic open discussion
00:59:55 <ekcs> ok then =)
01:00:02 <ekcs> More next week?
01:00:08 <ekcs> bye all.
01:00:20 <thinrichs> Bye
01:00:23 <ekcs> #endmeeting