00:00:31 <ekcs> #startmeeting congressteammeeting
00:00:32 <openstack> Meeting started Thu Mar 23 00:00:31 2017 UTC and is due to finish in 60 minutes.  The chair is ekcs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:00:35 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:05 <ekcs> hi all! good to have everyone back.
00:01:29 <masahito> hi
00:01:59 <ekcs> here are some items for today. please take a look and add yours if you have something to add =)
00:02:03 <ekcs> #link https://etherpad.openstack.org/p/congress-meeting-topics
00:02:07 <ekcs> hi masahito !
00:07:29 <ekcs> ok then let’s get started.
00:07:55 <ekcs> feel free to continue adding to the items as we talk.
00:08:14 <ekcs> #topic Backward compatibility with new table columns
00:08:37 <ekcs> I was working on this task #link https://bugs.launchpad.net/congress/+bug/1674537
00:08:37 <openstack> Launchpad bug 1674537 in congress "add to cinder_driver encryption and attachment info" [Undecided,Confirmed]
00:09:24 <ekcs> And I realized that it’s problem when every time we add a new data column to an existing datasource table we can break compatibility with existing policy rules.
00:10:20 <ekcs> to avoid breaking compatibility, we can either 1. don’t add new columns. add any new data to a new table.
00:10:57 <ekcs> 2. allow too few positional arguments in reference to datasource tables. eg. nova:flavors(x)
00:12:04 <ekcs> I have a little more discussion in the bug link above. anyone have thoughts on this? bringing it up on this meeting since it affects congress overall if we decide to allow too few positional args.
00:13:23 <masahito> option#1 looks good for me.
00:13:54 <masahito> because #2 is hard to reduce the table column.
00:14:55 <ekcs> right we can only add not remove. but I don’t think that’s a consideration here, because whether we go 1 or 2, we can’t remove a table column without breaking existing policies.
00:16:08 <masahito> And additional option is that we introduce datasource driver version to manage the table column.
00:18:26 <masahito> when creating datasource users can specify datasource driver version, like creating cinder_driver version 1.
00:19:09 <ekcs> I personally don’t like option 1 because we end up with a very messy schema. like I’m working on a patch that grabs several attributes about cinder volumes that wasn’t captured before (encryption, availability_zone, replication status, etc.). If we choose option 1, then either we add many new tables (volumes_encryption, volumes_availability_zone, etc.) one for each new piece of data. or we add one new table like volumes_additional_info. and then nex
00:19:10 <ekcs> time we add more things it’d be like volumes_more_additional_info.
00:21:10 <ekcs> versioning can definitely work. but also add to the things we need to maintain.
00:21:42 <ekcs> are you thinking like cinder_driver_v1_1.py or like just cinder_driver.py but it understands version?
00:22:45 <masahito> I thought later one. just cinder_driver.py
00:24:23 <ekcs> hmm yea that’d be the cleanest approach I think. but also more things to maintain over time. if we anticipate needing to alter tables in many ways not just adding new columns, then that definitely seems like the right approach.
00:25:02 <ekcs> but if we don’t, it may be overkill, unless we have other good reasons for it anyway. is that something we’ve discussed much before?
00:26:14 <ekcs> anyway we’d probably need more input here to make progress. I can summarize and discuss on ML later.
00:30:33 <bryan_att> Hi all - joining late
00:30:54 <ekcs> hi bryan_att
00:30:57 <ekcs> hi thinrichs
00:31:03 <thinrichs> Hi all.  Sorry I'm late.
00:31:09 <ekcs> here are the topics. https://etherpad.openstack.org/p/congress-meeting-topics
00:31:14 <ekcs> feel free to add to it.
00:32:13 <ekcs> we were just talking about what to do when we want to add new data from a datasource. say encryption, availabality zone, etc. from cinder.
00:32:46 <ekcs> if we add a data column, it would break compatibility with existing policy rules. unless we do something:
00:33:10 <ekcs> a. allow too few positional args. just pad with anynomous variables similar to how column references are handled.
00:33:29 <ekcs> b. masahito mentioned we can go all the way and version the drivers.
00:34:40 <ekcs> go ahead if you guys have anything to add, or we could also move on =)
00:34:55 <bryan_att> not being able to extend the datasource columns seems like a major limitation
00:35:26 <ekcs> bryan_att: agreed.
00:35:38 <bryan_att> even if new columns were added at the end, eventually we would need to deprecate some columns, right?
00:36:32 <bryan_att> the fixed positioning of columns (and not easily knowing what those are), has been one of the tough things about writing rules
00:37:09 <bryan_att> unless I don't know all the options yet (probable)
00:37:41 <bryan_att> API/datasource versioning though seems like a good thing in general
00:37:56 <ekcs> bryan_att: positional arguments are hard to use yes. but have you used named column reference?
00:38:09 <bryan_att> ala name-value?
00:38:16 <bryan_att> name=value
00:38:22 <ekcs> eg. p(id) :- nova:flavors(id,vcpus=x)
00:38:55 <ekcs> p(id) :- nova:flavors(id=id,vcpus=x)
00:38:59 <bryan_att> yes, but I also run into warnings/errors if i use them the wrong way. and figuring out why is hard sometimes
00:39:39 <bryan_att> in general that's how I would like to write rules - only ref the columns I need
00:40:07 <ekcs> i see. yea I think that’s the intent.
00:40:25 <ekcs> with hard to understand errors, I would love to see some examples next time you run into them?
00:40:54 <bryan_att> sure, I will take notes
00:41:10 <ekcs> thanks. that’d help us improve things for sure.
00:41:19 <ekcs> ok let’s move on to other topics then.
00:41:47 <ekcs> #topic Congress support in TryStack
00:42:13 <ekcs> bryan_att: you wanna go ahead?
00:42:27 <bryan_att> I checked out TryStack, and updated the issue - it's based upon Liberty and there's no clear way to enhance it
00:42:46 <bryan_att> I sent a note to the RDO team about that on OPNFV, and will update it further with the answer
00:43:05 <bryan_att> I asked when do they plan to update e.g. to Newton - seems we need a recent version for it to be useful
00:43:09 <bryan_att> that's all for now
00:43:24 <ekcs> bryan_att: got it thanks a lot! I saw your update on the bug.
00:43:32 <ekcs> awesome that’d be good to know.
00:43:48 <bryan_att> Also I'll try out the other issue (HTTPS support) asap
00:44:04 <ekcs> thanks lots!
00:44:22 <ekcs> let’s go back to policy library then.
00:44:33 <ekcs> #topic policy library
00:45:09 <ekcs> here are some of the policies thinrichs and others have collected. https://docs.google.com/document/d/12f1VciulhT9yCYOc7jiulGiLT-tFpffLxNOpr-2QX2I/edit#
00:45:26 <ekcs> some of them seem suitable for being in a policy library as is.
00:45:50 <ekcs> we discussed previously how we want to implement the library.
00:47:10 <thinrichs> I pushed up a candidate start of the policy library (the directory of files that contain policies, not the code implementing the functionality of the library).
00:47:11 <thinrichs> https://review.openstack.org/#/c/448785/
00:47:37 <thinrichs> It includes just one policy for now
00:47:58 <thinrichs> I figured we could settle on that one, and then add a couple more.  And then we'd have a starting point for writing the code.
00:48:39 <ekcs> thath’s great.
00:48:56 <thinrichs> I know we discussed storing the library in YAML vs. JSON.  I did it with YAML, but we could support JSON too (or alternatively only support JSON).
00:48:57 <bryan_att> looks good to me
00:49:05 <ekcs> the bare minimal is to ship the library as a collection of files. and provide CLI & GUI to add policies from file into the active system.
00:49:38 <bryan_att> I like the file approach, that allows customization in deployment..
00:49:51 <thinrichs> ekcs: +1.  That's what I was thinking too.  API design is in the google doc.
00:50:13 <thinrichs> Near the top of ..
00:50:15 <thinrichs> #link https://docs.google.com/document/d/12f1VciulhT9yCYOc7jiulGiLT-tFpffLxNOpr-2QX2I/edit#
00:50:24 <thinrichs> Look for the tables
00:52:54 <thinrichs> The Library in that proposal is just a collection of policies; they never get evaluated.
00:53:27 <ekcs> looks like a great proposal.
00:53:33 <thinrichs> If the user wants to use a library policy, the API makes it easy to download it as YAML and then add that policy into the normal /policy part of the API
00:53:54 <ekcs> the CLI/API workflow doesn’t even require congress to know about a library.
00:54:03 <thinrichs> Right
00:54:12 <ekcs> but the GUI workflow requires congress (at least the GUI) to have some idea of what’s in the library.
00:54:57 <thinrichs> The CLI/API would know about the library.  But people could use their own YAML/json-encoded policies too.
00:55:14 <ekcs> I see okay.
00:55:33 <thinrichs> The GuI would clearly only let people use policies that are in the library
00:55:50 <thinrichs> but people could add their own policies into the library
00:56:19 <ekcs> so what should we do next on this? if the proposal sounds good to people then we can start breaking it into tasks?
00:56:39 <thinrichs> Probably we should push in the API change as a spec so people can comment.
00:57:11 <ekcs> great.
00:57:14 <thinrichs> But if people are roughly in agreement then breaking it into tasks  seems right.
00:57:21 <thinrichs> We could do the 2 in parallel.
00:57:25 <ekcs> masahito, bryan_att any thoughts on the API and workflow proposal?
00:58:18 <masahito> Overall, I agree the list of new APIs.
00:58:24 <bryan_att> yes, I agree, both sound fine as a start
00:59:09 <bryan_att> I assume we can sync the file at any time, and the GUI will pick up any changes the next time the user browses the library?
00:59:35 <bryan_att> this would simplify distribution of policies across a set of clouds
01:00:05 <thinrichs> bryan_att: the GUI should pick up changes to the library.  But if someone pulled a policy from the library and inserted it into the existing policy collection, the inserted-policy wouldn't be updated.
01:00:22 <bryan_att> understood and agreed as a limitation
01:00:22 <thinrichs> Does that distinction make sense?
01:00:41 <bryan_att> that would need to be a user guide note
01:00:55 <thinrichs> I expect that most of the time people will pull down a library policy and then Change it at least a little before inserting
01:01:05 <thinrichs> We're out of time.
01:01:17 <ekcs> yup.
01:01:25 <ekcs> continue later then.
01:01:45 <ekcs> great discussions today. thanks all!
01:02:10 <ekcs> #endmeeting