00:00:51 <ekcs> #startmeeting congressteammeeting
00:00:52 <openstack> Meeting started Thu May 11 00:00:51 2017 UTC and is due to finish in 60 minutes.  The chair is ekcs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:00:53 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:00:56 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:03 <ramineni_> ekcs: hi
00:01:27 <ekcs> hi all! welcome back to another meeting. collaborative list of topics are kept here as usual: https://etherpad.openstack.org/p/congress-meeting-topics
00:01:30 <ekcs> hi ramineni_
00:02:50 <ramineni_> ekcs: howz the summit going on?
00:04:40 <ekcs> it’s going well. got to participate in some interesting discussions around rbac, hierarchical quotas, api microversioning, deprecating postgres, etc.
00:06:18 <ramineni_> ekcs: ohk, great
00:06:41 <ekcs> ok let’s move along then =)
00:06:51 <ekcs> #topic gate failure
00:07:21 <ekcs> looks like you’ve resolved/worked around the two main issues, devstack and keystone v2?
00:07:46 <ramineni_> ekcs: yes, but still one issue is pending
00:07:55 <ekcs> thanks a ton!
00:08:00 <ekcs> what’s the pending issues ramineni_ ?
00:08:17 <ekcs> dealing with the remaining keystone v2 stuff?
00:08:23 <ramineni_> ekcs: HA tests and keystone v2 tests are disabled as of now
00:09:20 <ramineni_> ekcs: yes , with the removal of v2 v3 in the auth_url, v3 is working staright forward , but still need to figure out identifying v2
00:09:46 <ramineni_> HA test problem is also we are using idnetity v2 clients for creation of replica
00:10:26 <ekcs> ah I didn’t notice the HA stuff. so is that just a matter of changing to v3 in the way replicas are created?
00:10:37 <ramineni_> ekcs: and also, you remember making the local copy of manager file which used to be in tempest?
00:10:48 <ekcs> yes
00:11:27 <ramineni_> ekcs: any idea, if can remove this and use directly
00:11:29 <ramineni_> https://github.com/openstack/congress/blob/master/congress_tempest_tests/tests/scenario/manager.py
00:13:17 <ramineni_> ekcs:  this part is failing in HA , https://github.com/openstack/congress/blob/master/congress_tempest_tests/tests/scenario/congress_ha/test_ha.py#L45
00:13:53 <ramineni_> ekcs: taking v2 here , have to use right client here and test it ..ill get to that this week
00:14:28 <ekcs> thanks so much for figuring stuff out!
00:14:58 <ekcs> ok ramineni_ said: “any idea, if can remove this and use directly https://github.com/openstack/congress/blob/master/congress_tempest_tests/tests/scenario/manager.py”
00:15:18 <ekcs> I’m confused what “this” refers to. what are you proposing to remove?
00:15:48 <ramineni_> ekcs: local copy, the link i mentioned
00:17:19 <ramineni_> can we remove this https://github.com/openstack/congress/blob/master/congress_tempest_tests/tests/scenario/manager.py”
00:17:21 <ramineni_> ?
00:17:25 <ekcs> ok yea mean remove the local copy you linked and use tempest library directly.
00:17:31 <ramineni_> ekcs: yes
00:18:15 <ekcs> I would think no because the idea is tempest team is going to take some time to refactor things and create stable interfaces.
00:18:32 <ekcs> when they’re done we’ll rework things to use their new interfaces.
00:18:42 <ramineni_> ekcs: ok
00:18:50 <ekcs> in the mean time, if we use their library directly, things will break as they make changes.
00:19:05 <ekcs> unless they’re already done, but i haven’t seen any notice on ML.
00:19:24 <ekcs> just did a search again to be sure. nothing on ML.
00:19:38 <ramineni_> ekcs: ok, got it.
00:19:41 <ekcs> does it help us/you to remove?
00:20:24 <ramineni_> ekcs: no
00:20:51 <ekcs> oh ok. well i’ll monitor the ML and when i hear update from tempest team I’ll let people know.
00:20:54 <ramineni_> ekcs: I better not create new failures by touching that file :)
00:21:00 <ekcs> hehe.
00:21:12 <ramineni_> ekcs: great, thanks
00:21:31 <ramineni_> ekcs: In the meanwhile , can we merge  to resolve the gate failure as of now.. ?
00:21:56 <ekcs> ok one last thing. in what you’ve looked at so far, do you know if keystone v2 is supposed to work (we just use it wrong) or not supposed to work at all?
00:22:03 <ramineni_> ekcs: local devstack failing because of the issues, so we need that patch to be merged asap
00:22:21 <ekcs> yup I think we should merge it. I already +1ed.
00:22:38 <ekcs> you can go ahead and +2/+1
00:22:58 <ramineni_> ekcs: i couldnt get anything, i have checked the keystone latest commits, nothing of deprecation related
00:23:23 <ramineni_> ekcs: change in auth_url , may be we have to change the way we are detetcting its v2
00:23:47 <ekcs> ok hmm. maybe i’ll ask one of the keystone folks at the summit hehe. ok thanks!
00:24:04 <ekcs> do you have the keystone patch handy that changed the auth_url expected?
00:25:51 <ramineni_> ekcs: you meant this one https://review.openstack.org/#/c/458449/
00:25:52 <patchbot> patch 458449 - openstack-dev/devstack - try to use unversioned keystone endpoints everywhere (MERGED)
00:27:22 <ekcs> hmm ok. yea I see that devstack patch, was wondering if we know the keystone patch that made this necessary.
00:27:52 <ekcs> from there we may be able to figure things out.
00:28:13 <ekcs> there must have been a change on keystone that made the versioned auth_url not work.
00:28:35 <ramineni_> ekcs: yes, ok, couldnt find it as of now
00:28:37 <ekcs> not necessarily easy to track down tho.
00:28:41 <ekcs> yea. ok. great
00:28:54 <ekcs> if i get hold of a keystone guy i’ll ask =)
00:29:14 <ramineni_> ekcs: great :)
00:29:59 <ekcs> ok well lets move on then?
00:30:45 <ramineni_> yes
00:32:03 <ekcs> #topic policy library
00:32:28 <ekcs> ok too bad no thinrichs and masahito, but let’s see what we can do.
00:32:33 <ekcs> thanks for the comments.
00:32:43 <ekcs> on #link https://review.openstack.org/#/c/457880/6/specs/pike/policy-library.rst@96
00:32:44 <patchbot> patch 457880 - congress-specs - policy library spec
00:33:15 <ramineni_> ekcs: yep, i was just wondering if we have explored that option ?
00:34:28 <ramineni_> ekcs: and still couldnt get why we need to store template policies in DB  ? and if we want to store, can u update inthe spec, what all details we are storing?
00:34:44 <ramineni_> ekcs: is it just the id, or all details?
00:35:11 <ekcs> ok so there are two things.
00:36:17 <ekcs> for whether to go actual templates instead of plain policy that can be editted: thinrichs came up with that design. I think the idea is we may not have time this cycle to figure out all the details needed to deal with templates (what syntax, would it interfere with existing rule syntax? do we need new parser grammer?, what gui, etc)
00:36:32 <ekcs> so we can get almost all the benefit without explicitly doing placeholders.
00:36:49 <ekcs> just use normal variables/constants, and choose names that make it clear users can edit them.
00:37:47 <ekcs> do you see a major benefit of using templates instead of actual policies users can edit?
00:38:26 <thinrichs> Hi all.  Sorry I'm late—got caught up (again)
00:38:41 <ramineni_> thinrichs: hi
00:38:57 <ekcs> ah you’re here just in time for the pol lib discussion.
00:39:28 <ekcs> we’re talking about ramineni_ ’s comment here: https://review.openstack.org/#/c/457880/6/specs/pike/policy-library.rst@96
00:39:28 <patchbot> patch 457880 - congress-specs - policy library spec
00:40:02 <ramineni_> ekcs: hmm , i saw using jinja2 is easier , so proposed that option , but may be for next cycle as you mentioned
00:40:18 <ekcs> ekcs: for whether to go actual templates instead of plain policy that can be editted: thinrichs came up with that design. I think the idea is we may not have time this cycle to figure out all the details needed to deal with templates (what syntax, would it interfere with existing rule syntax? do we need new parser grammer?, what gui, etc)
00:40:18 <ekcs> [5:36pm] ekcs: so we can get almost all the benefit without explicitly doing placeholders.
00:40:35 <ekcs> that’s what I said. anything correction/addition thinrichs ?
00:42:06 <thinrichs> ekcs: nope—that was the thought.  That a template is a lot of work, and people will want to modify things more than a template provides (eventually) anyway.
00:42:40 <thinrichs> There's nothing stopping us from adding the functionality of templates later
00:42:47 <ramineni_> thinrichs: ya, ok
00:43:45 <ekcs> ok the other issue is why do we persist the policy library in DB.
00:44:56 <ekcs> what model do you have in mind ramineni_ ?
00:45:01 <thinrichs> My thought there was if we store them in the DB then it's easy to make them accessible via the API
00:45:21 <thinrichs> which we'd need to do, esp for Horizon
00:45:50 <ramineni_> thinrichs: after modifying or before modfying?
00:46:27 <thinrichs> Both.  The policy "library" has its own API and policies contained within it are not active.  (Eventually these could be raw policies or templates.)
00:47:08 <ramineni_> thinrichs: if user wants to use only 2/3 policies of it? still we load all the policies into DB
00:47:17 <thinrichs> Then the user wants to apply/instantiate a policy from the library and make it active.
00:47:48 <thinrichs> The library DB tables would be completely separate from the current Policy/Rule tables.
00:48:17 <thinrichs> The library DB tables have their own CRUD operations to add new policies, look at the ones that exist, update, and delete.
00:48:33 <thinrichs> All those operations would be completely independent of our current CRUD operations on Policies/Rules.
00:48:59 <ramineni_> thinrichs: what details are we storing in DB for policy libraries? same as policy table?
00:49:29 <thinrichs> I believe so.  Not sure if there are any extra fields we'd want.  The spec might cover that.
00:49:54 <ekcs> ramineni_: https://review.openstack.org/#/c/457880/6/specs/pike/policy-library.rst@96
00:49:55 <patchbot> patch 457880 - congress-specs - policy library spec
00:50:13 <ekcs> here is the table I propose in the spec
00:50:23 <ekcs> ugh nvm
00:50:24 <ekcs> wrong link
00:50:27 <thinrichs> The difference is most obvious with execute[] rules.  The execute[] rules in the Library tables would never fire; but the execute[] rules in the Policy/Rules tables would fire.
00:50:58 <thinrichs> Said another way, the Policy Engine would never see any of the Library policies (until they were inserted into the Policy/Rules tables).
00:51:04 <ekcs> https://review.openstack.org/#/c/457880/6/specs/pike/policy-library.rst@209
00:51:05 <patchbot> patch 457880 - congress-specs - policy library spec
00:51:27 <ramineni_> thinrichs: hmm, so user would edit the polciies in files or using API?
00:52:18 <ramineni_> thinrichs: sample policies**
00:52:38 <ekcs> the API is how congress server sees it.
00:52:50 <ekcs> but CLI would support referencing a file.
00:53:07 <ekcs> and passing it along to server using API.
00:53:19 <ekcs> horizon would have edit box. but again pass it along to server using API.
00:53:55 <ekcs> in the current spec proposal anyway.
00:54:16 <thinrichs> Looking at the sample policies, I'd think most of the time people will want to edit them before instantiating them (templating might make that easier).
00:54:29 <thinrichs> In Horizon, that editing could be done in an edit box, as ekcs suggests.
00:54:54 <thinrichs> From the CLI it's unclear to me, but I'd imagine pulling the policy into a file, having the user edit it, and uploading the file.
00:55:14 <thinrichs> The CLI could also stitch together the download + upload APIs into a single command
00:55:30 <thinrichs> for those cases when the user doesn't want to edit
00:56:41 <thinrichs> When we support templates, I'd imagine the user would want to instantiate the template with given parameters.
00:56:49 <ekcs> in either `policy create` or `library policy create` the CLI can accept either the policy with all rules in json/yaml as well as path to file.
00:57:27 <ekcs> in `policy create`, an additional option is to create directly from a policy stored in the library, by referencing it’s name.
00:57:50 <ekcs> a deployer may have already edited that policy in the library to be suitable for the particular setup. then an admin can simply activate it this way.
00:58:25 <thinrichs> So the CLI would be something like "openstack congress library install <policy name> <values>", which would under the hood (a) pull down the template, (b) run the template, (c) upload the instantiated policy.  (We could move some/all of that to the server too)
00:58:46 <thinrichs> (The above was for the templated case.)
00:58:53 <thinrichs> down the road
00:59:00 <ramineni_> thinrichs: yes, thats what i had in mind
00:59:00 <ekcs> Here’s the proposed API for activating directly from library without editing. https://review.openstack.org/#/c/457880/6/specs/pike/policy-library.rst@609
00:59:01 <patchbot> patch 457880 - congress-specs - policy library spec
00:59:37 <ekcs> well we’re out of time. let’s continue on gerrit =)
01:00:08 <ramineni_> ekcs: ok
01:00:15 <thinrichs> Ok
01:00:25 <ekcs> or we could contine on #congress if interested.
01:00:34 <ekcs> anyway i’ll be on congress.
01:00:38 <ekcs> see you next time!
01:00:41 <ekcs> #endmeeting