18:01:04 <elmiko> #startmeeting sahara
18:01:05 <openstack> Meeting started Thu Feb  4 18:01:04 2016 UTC and is due to finish in 60 minutes.  The chair is elmiko. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:09 <openstack> The meeting name has been set to 'sahara'
18:01:20 <NikitaKonovalov> o/
18:01:21 <elmiko> #chair SergeyLukjanov
18:01:21 <openstack> Current chairs: SergeyLukjanov elmiko
18:01:23 <esikachev> hi
18:01:23 <huichun> hello
18:01:24 <elmiko> hi
18:01:25 <AndreyPavlov> hi
18:01:28 <Akanksha08> Hi
18:01:37 <mionkin> Hi
18:01:38 <elmiko> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
18:01:58 <elmiko> let's give folks a minute or 2 to arrive
18:02:11 <vgridnev> hi!
18:02:48 <tmckay> hi folks
18:03:24 <SergeyLukjanov> my ZNC seems broken...
18:03:26 <SergeyLukjanov> hey guys
18:03:32 <elmiko> hey
18:03:41 * elmiko hands microphone to SergeyLukjanov
18:03:44 <SergeyLukjanov> #topic News / updates
18:03:50 <elmiko> tmckay: i added your topic to the agenda
18:03:56 <tmckay> elmiko, nice, thanks
18:04:04 <SergeyLukjanov> elmiko thx for starting meetings, seems like it's my tradition to be few mins late ;)
18:04:10 <tellesnobrega> hi
18:04:15 <elmiko> SergeyLukjanov: hehe, no problem at all =)
18:04:16 <tmckay> I've been working on substring searches ... have a few words about that
18:04:30 <elmiko> mainly api v2 and security bugs for me
18:04:33 <vgridnev> I am working on the cluster health checks, basic things are almost done, waiting for the reviews
18:04:59 <SergeyLukjanov> I've participated PTLs interview series and talk a short story about v2 API there and security improvements
18:05:00 <tmckay> question, it looks to me like heat is still broken in the gate, any update on that?
18:05:00 <Akanksha08> I am working on improving anti-affinity spec
18:05:05 <NikitaKonovalov> working on HDP HA for ResoureceManager and Name, need to make a final push on it
18:05:16 <tmckay> (at least last evening we had a lot of broken tests still)
18:05:26 <huichun> Working on EDP improvement
18:05:44 <mionkin> working on updating of spark sahara plugin for using latest spark version (1.6.0)
18:05:46 <SergeyLukjanov> #info We have 2 more weeks before the final releases for non-client libraries for this cycle, and 3 weeks before the final releases for client libraries
18:05:54 <SergeyLukjanov> #info We have 3 more weeks before the Mitaka-3 milestone and overall
18:05:55 <SergeyLukjanov> feature freeze.
18:06:04 <vgridnev> tmckay, it looks like there keystone v3 stuff was reverted in devstack, so everything should be ok, basically
18:06:06 <SergeyLukjanov> #undo
18:06:07 <openstack> Removing item from minutes: <ircmeeting.items.Info object at 0x9750d90>
18:06:10 <SergeyLukjanov> #info We have 3 more weeks before the Mitaka-3 milestone and overall feature freeze.
18:06:16 <elmiko> mionkin: is that for the cdh plugin?
18:06:27 <SergeyLukjanov> vgridnev how is it going with client? are we ready to release it
18:06:28 <SergeyLukjanov> ?
18:06:33 <SergeyLukjanov> AndreyPavlov ^^
18:06:52 <AndreyPavlov> https://review.openstack.org/#/c/272503/ is on review
18:06:56 <tmckay> SergeyLukjanov, what is "non-client libraries" vs "client libraries" ?
18:07:03 <AndreyPavlov> the last thing i guess
18:07:10 <SergeyLukjanov> tmckay oslo.msg vs python-saharaclient
18:07:13 <AndreyPavlov> import/export spec will be added to backlog
18:07:24 <vgridnev> SergeyLukjanov, there is an update on health checks stuff, but we can release that it in the final release I guess
18:07:25 <mionkin> elmiko: it is for spark plugin
18:07:26 <SergeyLukjanov> we don't actually have non-client libs
18:07:26 <tmckay> ah, thanks
18:07:53 <elmiko> mionkin: ah, got it. thanks!
18:08:15 <elmiko> and yay spark 1.6.0 \o/
18:08:45 <vgridnev> elmiko, I bet Spark will not work with Swift again
18:08:48 <SergeyLukjanov> elmiko yay!
18:09:11 <elmiko> vgridnev: most likely, but that probably has more to do with the inclusion of the hadoop-openstack.jar
18:09:15 <elmiko> at a guess
18:09:24 <SergeyLukjanov> elmiko do we need "API v2 progress" topic today?
18:09:51 <elmiko> SergeyLukjanov: i'd like to keep it on the agenda until we get it out of experimental. (kinda like how we kept sahara@horizon)
18:09:59 <elmiko> just to help keep the train rolling
18:10:58 <elmiko> unless there are objections to having it there for awhile
18:11:25 <SergeyLukjanov> elmiko sure
18:11:26 <SergeyLukjanov> #topic API v2 progress
18:11:43 <elmiko> hey!
18:11:48 <SergeyLukjanov> I'm absolutely ok with and it's an important complex part to be tracked well
18:11:57 <elmiko> cool, thanks
18:12:14 <elmiko> i've updated the initial commit POC #link https://review.openstack.org/#/c/273316/
18:12:26 <elmiko> basically addressing some comments that SergeyLukjanov had
18:12:32 <elmiko> i do have a question though
18:12:55 <elmiko> SergeyLukjanov: you had mentioned on review dropping the project id stuff in the new AuthValidatorV2. what did you mean by that?
18:13:04 <SergeyLukjanov> elmiko I'll re-iterate on it
18:13:07 <elmiko> cool
18:13:22 <elmiko> also, i've broken out all the endpoints hierarchies into separate python modules
18:13:39 <elmiko> i'd like a few opions on whether this seems like a good solution
18:13:48 <SergeyLukjanov> elmiko I mean that probably not need the AuthValidator at all, it was a hacky way to handle something and I honestly don't remember what was the real issue
18:14:03 <SergeyLukjanov> elmiko I think with this ACLs based auth we don't need it
18:14:10 <elmiko> SergeyLukjanov: seemed to me the issue was assuring that the requested project id is the same as the one in the token
18:14:20 <elmiko> ok
18:14:52 <elmiko> i need to update the wiki page with a few more work items, but otherwise the initial commit seems to be working well so far
18:15:02 <elmiko> that's all i have, unless there are questions
18:15:51 <SergeyLukjanov> elmiko I expect that it'll be checked by ACL code as well :)
18:16:00 <elmiko> most likely
18:16:12 <elmiko> i should review the acl stuff a little deeper
18:18:23 <elmiko> that's all for api v2
18:19:23 <elmiko> #topic Anti-affinity spec
18:19:35 <elmiko> #link https://review.openstack.org/#/c/269202/
18:19:43 <elmiko> Akanksha08
18:19:58 <Akanksha08> elmiko, yes
18:20:15 <Akanksha08> I have added a comment in the spec https://review.openstack.org/#/c/269202/5
18:20:36 <Akanksha08> vgridnev, you had some questions there
18:20:48 <vgridnev> yes, I do
18:20:53 <SergeyLukjanov> sorry, I've been looking on a v2 patch :)
18:21:01 <vgridnev> and I still have them
18:21:26 <Akanksha08> vgridnev, did you read the comment I made
18:22:01 <vgridnev> The general problem: all instances of the node group are combined into one ResourceGroup, and we can't specify Server group for each instance separately
18:22:14 <SergeyLukjanov> I've not yet looked at the spec
18:22:20 <SergeyLukjanov> but we must keep backward compat
18:22:28 <vgridnev> Any ideas how to resolve that?
18:22:46 <Akanksha08> but I think server group should not be tied to the node group
18:23:08 <Akanksha08> I did not understand why this has been done - https://github.com/openstack/sahara/commit/8f965bfd5e5ada9cdf6a26dd150ddb0294817ee1
18:24:31 <Akanksha08> we can specify server group for each instance separately in the scheduler hints
18:24:40 <Akanksha08> I have explained that in the comment
18:24:53 <vgridnev> We can't do this
18:25:43 <tmckay> vgridnev, is it a restriction in heat?
18:25:43 <Akanksha08> vgridnev,  will it have any negative impact on current functionalities?
18:25:45 <vgridnev> All instances of the node group are combined into one OS::Heat::ResourceGroup
18:25:49 <Akanksha08> yes
18:25:57 <Akanksha08> I understand that
18:26:33 <vgridnev> ok, and how to specify server group for each instance then?
18:27:02 <Akanksha08> If the value of the get_resource is a variable say "SERVER_GROUP_NAME" + instance_index
18:27:47 <Akanksha08> instance_index or any value like for server_group_1 it will be 1 and for server_group_2 it will be 2
18:29:17 <Akanksha08> vgridnev, elmiko does that look okay
18:29:25 <vgridnev> Akanksha08, can we discuss that a little bit later? I need to make some research on heat again
18:29:39 <Akanksha08> vgridnev, okay
18:29:41 <vgridnev> in #openstack-sahara maybe
18:29:46 <elmiko> Akanksha08: from my understanding of what you have defined, yes
18:30:19 <Akanksha08> vgridnev, I still have a question on why this change was made -  https://github.com/openstack/sahara/commit/8f965bfd5e5ada9cdf6a26dd150ddb0294817ee1
18:31:01 <Akanksha08> here what I understand is the server group param is being tied to a node group
18:31:17 <Akanksha08> or should we discuss this as well on #openstack-sahara
18:31:50 <vgridnev> Akanksha08, server group was defined outside  of the ResourceGroup, so that heat can't find definition of the server group
18:33:03 <Akanksha08> so does heat expects the same to be defined inside the ResourceGroup
18:35:15 <vgridnev> Akanksha08, no, we can pass server group as property of the ResourceGroup, so it can be defined outside
18:36:41 <Akanksha08> okay
18:37:08 <elmiko> are we good on this topic for now?
18:37:34 <Akanksha08> for me yes. I wanted to clear vgridnev's doubts
18:37:41 <elmiko> k, thanks
18:37:47 <Akanksha08> and clear my doubts as well
18:37:53 <elmiko> #topic Substring search options
18:38:00 <elmiko> #link https://review.openstack.org/#/q/status:open+project:openstack/sahara+branch:master+topic:bug/1503345
18:38:03 <elmiko> tmckay
18:38:12 <elmiko> thanks Akanksha08 and vgridnev
18:39:19 <Akanksha08> :) thanks elmiko and vgridnev
18:39:31 <elmiko> tmckay: this is your topic =)
18:39:31 <tmckay> elmiko, thanks
18:39:35 <elmiko> k
18:39:54 <tmckay> okay, here is the summary. I was using Nova as a model for how to do substring matching
18:40:22 <tmckay> I looked at the code quickly and initially assumed nova was using a LIKE filter with % chars for simple matching
18:40:50 <tmckay> But, it isn't. It's actually using regular expressions for certain fields, like name (other fields are exact match)
18:40:57 <tmckay> So here is what I propose:
18:41:10 <tmckay> we use regular expressions, too, for certain defined fields
18:41:16 <tmckay> we support this for mysql and postgres
18:41:27 <tmckay> if you use a different database, you only get exact matches
18:41:45 <tmckay> Nova uses LIKE as a fallback for other databases, but I'm not sure I see the point ....
18:41:59 <tmckay> (they also use regex for sqlite, which we could do)
18:42:24 <tmckay> so, the initial patches are trying to use LIKE, but I need to modify them to use regex.  Pretty easy change.
18:43:14 <elmiko> i'm +1 for the regex approach
18:43:39 <tmckay> so, any questions, objections? We will have regex matching for things like name, description, plugin_name, etc and exact matching for things like uuids and version numbers
18:44:00 <tmckay> only mysql and postgres will have this. Other stuff ... so sorry :)
18:44:37 <tmckay> I set initial patches to WIP to modify ...
18:45:13 <huichun> tmckay:  for database we use like, for Sahara level we use regex?
18:45:14 <elmiko> sounds good to me
18:45:54 <tmckay> huichun, there is one place LIKE is being used currently, it could be modified (this is in counting data source references in a certain case with proxy users)
18:46:22 <tmckay> huichun, so all of the pattern-matching get_all() methods would use regular expressions
18:46:46 <huichun> Clear for me
18:47:02 <tmckay> huichun, this will be hardwired for any calls through the REST api.  Internal calls which use the conductor will have pattern matching turned off by default, but could turn it on with a flag
18:48:04 <tmckay> it turns out that using regex is actually simpler :) using like raises some questions
18:48:18 <tmckay> those questions go away with regex ;-)
18:49:19 <NikitaKonovalov> tmckay: do we need to add any indexes to db, to make this work properly?
18:49:49 <tmckay> NikkitaKonovalov, no, we just have to use the correct query filters for certain columns
18:50:04 <tmckay> oops, too many ks
18:51:34 <elmiko> 9min left
18:53:12 <elmiko> any other questions on this topic?
18:53:39 <SergeyLukjanov> #topic Open discussion
18:53:49 <SergeyLukjanov> 6 mins left
18:54:35 <vgridnev> We have two specs on the review, both have 2 +2: https://review.openstack.org/#/c/254956/ and https://review.openstack.org/#/c/266557/ . Let's concentrate and make additional reviews to approve them!
18:55:13 <elmiko> vgridnev: +1
18:59:17 <elmiko> i suppose we're done?
18:59:26 <huichun> Bye
18:59:28 <elmiko> have a good weekend all ;)
18:59:43 <huichun> We have spring festival this week
18:59:56 <elmiko> ooh, that's right new year is this weekend right?
19:00:00 <elmiko> \o/
19:00:04 <huichun> yes
19:00:04 <SergeyLukjanov> out of time :(
19:00:07 <elmiko> gong xi fasai
19:00:10 <SergeyLukjanov> #endmeeting