13:30:34 <dangtrinhnt> #startmeeting openstack search
13:30:35 <openstack> Meeting started Mon Feb 11 13:30:34 2019 UTC and is due to finish in 60 minutes.  The chair is dangtrinhnt. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:30:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:30:38 <openstack> The meeting name has been set to 'openstack_search'
13:30:52 <dangtrinhnt> thuydang, sapd1_ meeting time :)
13:31:13 <thuydang> Hi
13:31:48 <dangtrinhnt> hi, sapd1_ are you there?
13:34:18 <dangtrinhnt> ok, let's start with the #topic Denver summit demos
13:34:24 <dangtrinhnt> https://etherpad.openstack.org/p/search-team-meeting-agenda
13:34:38 <dangtrinhnt> #topic Denver summit demos
13:35:17 <dangtrinhnt> thuydang, do you have any update on this?
13:35:22 <dangtrinhnt> sapd1, hi
13:35:34 <dangtrinhnt> https://storyboard.openstack.org/#!/story/2004840
13:35:36 <thuydang> There is no update for the demo on my side. I was quite busy.
13:35:46 <sapd1> hi
13:35:49 <dangtrinhnt> ok, me too for the last 2 weeks.
13:36:11 <dangtrinhnt> I will work on it this week
13:36:34 <thuydang> Ah, I added some task to the storyboard link above.
13:36:36 <sapd1> yes. Because I was on holiday last week.
13:36:48 <dangtrinhnt> sapd1, could you please give me some screenshots of what you mentioned in the story? https://storyboard.openstack.org/#!/story/2004840
13:37:21 <dangtrinhnt> thuydang, sapd1, yeah It
13:37:35 <dangtrinhnt> It's the Lunar New Year for us :D
13:38:27 <dangtrinhnt> no worries, as long as I have a better understanding of what sapd1's idea, I can start working on it, properly before the Stein-3 milestone
13:38:29 <thuydang> @dangtrinhnt which screenshots?
13:39:02 <sapd1> dangtrinhnt: You mean mock screenshot.
13:39:10 <dangtrinhnt> sapd1, yeah
13:39:20 <dangtrinhnt> or something like that
13:40:04 <sapd1> dangtrinhnt: I will draw a diagram for that. maybe tomorrow, Have you guys tried setup keystone-to-keystone yet?
13:40:44 <thuydang> not yet
13:40:54 <thuydang> I could start 1 devstack
13:40:56 <dangtrinhnt> sapd1, thanks. No, I don't have the resource for that, maybe trying to borrow some server somewhere.
13:41:14 <thuydang> do you think testing with 2 devstack is a good idea?
13:41:27 <dangtrinhnt> thuydang, that's good enough for the demo I think
13:41:38 <sapd1> dangtrinhnt: You can try with 2 small server, with only keystone and glance are installed.
13:41:45 <sapd1> that's enough
13:41:53 <dangtrinhnt> ok, sounds reasonable. Thanks.
13:42:21 <thuydang> I have to increase timeout when installing on my laptop
13:42:22 <sapd1> #link https://docs.openstack.org/keystone/pike/advanced-topics/federation/configure_federation.html
13:42:45 <dangtrinhnt> thuydang, what timeout you're talking about?
13:42:55 <thuydang> devstack has a timeout setting
13:43:13 <thuydang> otherwise it crashes when some service takes too long to start up
13:43:23 <thuydang> glance in my case
13:43:44 <dangtrinhnt> ah, ok, cause normally devstack still eats lots of memory. I have to run inside a 16GB machine
13:44:20 <thuydang> I used a VM with 8G
13:44:22 <sapd1> with keystone-2-keystone I think We don't need to use devstack.
13:44:38 <sapd1> I will write Dockerfile and docker-compose for that.
13:44:48 <sapd1> 2 keystone containers and 2 mariadb containers
13:45:00 <sapd1> I think
13:45:26 <dangtrinhnt> sapd1, but how can you have Searchlight and other needed services?
13:45:27 <thuydang> just test the api and access token right?
13:46:44 <dangtrinhnt> sapd1, your proposed method just for testing keystone-2-keystone federation right?
13:46:45 <sapd1> We can run in container too :D
13:47:13 <sapd1> We can run every services in container.
13:47:32 <dangtrinhnt> I may try with Kolla but Kolla need at least 2 network interfaces
13:47:49 <dangtrinhnt> and it crashes every time :)
13:48:05 <dangtrinhnt> ok, let me try and get back to you guys at the end of this week.
13:48:24 <thuydang> let's draw a diagram for understanding
13:48:38 <sapd1> dangtrinhnt: I will give you my Dockerfile of searchlight and keystone too
13:48:47 <dangtrinhnt> sapd1, that would be awesome
13:49:13 <dangtrinhnt> you can even commit a patch set to put it inside searchlight repo
13:49:25 <dangtrinhnt> anything else about the demos?
13:49:52 <dangtrinhnt> thuydang, sapd1
13:49:52 <sapd1> let discuss about keystone-2-keystone.
13:49:53 <thuydang> I think we should provide diagrame of the workflow
13:50:44 <sapd1> assume we have 3 keystone services - let's say: keystone-idp, keystone-sp1, keystone-sp2
13:50:59 <dangtrinhnt> ok
13:51:01 <sapd1> idp - identity provider , sp - service provider
13:51:24 <thuydang> e.g., for sp is glance, etc?
13:51:40 <sapd1> no,
13:51:56 <sapd1> sp is another openstack cluster
13:52:43 <sapd1> so on keystone-idp we can see sp1 and sp2 then we can search in 3 openstack clusters -
13:52:45 <thuydang> I thought each openstack cluster has 1 keystone holding OS services endpoints
13:53:16 <thuydang> ok, that means 1 OS will be the idp
13:53:28 <dangtrinhnt> thuydang, this will explain https://docs.openstack.org/keystone/pike/advanced-topics/federation/configure_federation.html
13:53:47 <sapd1> but on sp1 and sp2, they only can see idp
13:54:26 <sapd1> so the problem here is when user switch to sp1 or sp2, how he/shee can search in other service provider.
13:55:07 <dangtrinhnt> sapd1, ok, I think there is a way in Searchlight to indicate the cluster that you're searching
13:55:43 <sapd1> *she
13:56:29 <sapd1> in my environment, we don't create users in service provider.
13:56:51 <thuydang> Tenant should be able to add her other SP right?
13:57:17 <sapd1> So I think this user can re-authenticate with idp and search.
13:57:54 <sapd1> what do you mean? thuydang
13:58:12 <dangtrinhnt> If we use the centralized authentication like this, things should be simple. Just need to indicate the tenant ID when querying resources
13:58:44 <sapd1> dangtrinhnt: yeah.
13:59:02 <sapd1> but with k2k, We have local tenant id and remote tenant id
13:59:19 <dangtrinhnt> I'm not sure if it's possible right now but logically, it's the way. Something likes searchlight.query(resource_type='NOVA_SERVER', tenant_id='abc')
13:59:19 <thuydang> so the idp knows about a tenant's resources in all SP clusters?
13:59:56 <sapd1> dangtrinhnt: Cool, tenant_id='local_abc|remote_cdf'
14:00:02 <dangtrinhnt> thuydang, no
14:00:29 <sapd1> thuydang: It's a mapping.
14:00:30 <dangtrinhnt> idp just stores the identities, for tenant resources, searchlight will use its plugins to query
14:00:56 <sapd1> thuydang: map tenant_A in idp with tenant_B in sp1
14:01:05 <sapd1> s/with/to
14:01:59 <thuydang> that's whtat I mean
14:02:29 <dangtrinhnt> sapd1, I think what you meant is Idp for Searchlight to authenticate against the tenants
14:02:38 <thuydang> how can the mapping be done? somewhere we must specify that tenant_A in idp is tenant_B in sp1
14:03:17 <sapd1> dangtrinhnt: sorry, as design, we will run searchlight service and ES as well in all openstack clusters. So we will search through searchlight-api
14:04:05 <thuydang> Let's put it like this:
14:04:16 <thuydang> we have 2 clusters with searchlight installed
14:04:50 <thuydang> the problem is using searchlight in 1 cluster to search in the other, right?
14:05:31 <sapd1> thuydang: yep
14:05:43 <dangtrinhnt> so you have multiple ES instances?
14:05:57 <thuydang> I thinks so
14:06:23 <thuydang> because the clusters belonging to multiple SPs may have nothing to do with each other
14:06:33 <thuydang> OS- AWS for example
14:06:44 <thuydang> and so is OS - OS
14:06:59 <dangtrinhnt> Then, each SL instance will use the other tenant's SL-api to query the other tenant's resources?
14:07:18 <thuydang> Yes, I thinks so
14:07:29 <dangtrinhnt> wow, that not scales :)
14:07:38 <thuydang> why?
14:08:10 <sapd1> thuydang: https://github.com/openstack/keystone/blob/master/keystone/federation/backends/sql.py mapping table in sql database
14:08:22 <sapd1> dangtrinhnt: Yep. I understand
14:08:41 <dangtrinhnt> for every new tenant you want to search, you have to say somewhere the searchlight-api endpoints
14:08:54 <sapd1> thuydang: When we have many many openstack clusters, So we have to go to every endpoint to search.
14:09:03 <thuydang> I don't think so
14:09:46 <thuydang> at worst case we only search in the cluster the tenant has access right?
14:10:40 <sapd1> we have to try to authenticate. Because the mapping is created in SP side.
14:12:33 <dangtrinhnt> ok, I think we will have a better view with drawing :)
14:12:36 <thuydang> sure, the tennant must provide credential for each of his cluster, which is used by SL to authenticates
14:13:05 <thuydang> yes, let's have anothe meeting for this discussion
14:13:37 <thuydang> let's move on with other topics
14:13:48 <sapd1> yeah
14:13:59 <dangtrinhnt> ok, looks like the topic gets some interesting points :)
14:14:14 <dangtrinhnt> #topic Functional tests for Searchlight in Py3
14:14:39 <dangtrinhnt> ok, the TC is checking our progress with python 3 functional tests
14:15:12 <dangtrinhnt> Over the last couple months, I added the python 3 tests for unit and functional tests
14:15:54 <dangtrinhnt> looks like it's acceptable compare to other projects
14:15:55 <dangtrinhnt> #link https://wiki.openstack.org/wiki/Python3
14:16:12 <dangtrinhnt> I updated the documents yesterday
14:16:37 <thuydang> do we have to migrate searchlight api to python3?
14:17:04 <dangtrinhnt> I think SL is py3 compatible
14:17:15 <dangtrinhnt> it passes the python3 tests
14:17:30 <dangtrinhnt> but the thing here is our test coverage is low, only 70 something
14:18:09 <dangtrinhnt> I would like to increase the test coverage to 90 something in the RC milestone
14:18:35 <dangtrinhnt> it means we need to add more tests and maybe we can separate the functional tests with Zuul
14:18:39 <dangtrinhnt> and tempest
14:19:45 <thuydang> I'm not familiar with testing and will have to learn first :-)
14:19:56 <dangtrinhnt> something like this https://github.com/openstack/tacker/blob/master/.zuul.yaml
14:20:18 <dangtrinhnt> with zuul, we can set our test env and dependencies :)
14:20:32 <dangtrinhnt> anyway, it's for the next milestone
14:20:37 <dangtrinhnt> and only if we have time :D
14:20:47 <sapd1> thuydang: me too
14:21:11 <dangtrinhnt> no worries, it's just a good-to-have feature.
14:21:16 <thuydang> we will try to cover testing with our new features
14:21:27 <dangtrinhnt> ok, I guess we can move on to the next topic?
14:21:31 <thuydang> ok
14:21:39 <dangtrinhnt> #topic TC vision reflection:
14:22:19 <dangtrinhnt> as you may now the OpenStack leadership has published a vision at #link https://governance.openstack.org/tc/reference/technical-vision.html
14:22:49 <dangtrinhnt> It's good to align our works with the TC vision
14:22:58 <dangtrinhnt> good practice
14:23:05 <dangtrinhnt> we can learn from other projects
14:23:06 <thuydang> I agree
14:23:20 <dangtrinhnt> https://etherpad.openstack.org/p/nova-tc-vision-self-eval
14:23:27 <dangtrinhnt> https://review.openstack.org/#/c/629060/
14:23:33 <dangtrinhnt> https://review.openstack.org/#/c/630216/
14:23:43 <dangtrinhnt> they're doing the reflection
14:24:32 <sapd1> dangtrinhnt: I will read it later.
14:24:52 <dangtrinhnt> ok, I started a new etherpad
14:24:54 <dangtrinhnt> https://etherpad.openstack.org/p/-tc-vision-self-eval
14:25:34 <dangtrinhnt> we will do about 3 weeks to 9 Mar. to put ideas to the pad and I will commit a doc change
14:26:12 <dangtrinhnt> Please help us to evaluate our works against the TC vision. :)
14:27:45 <sapd1> dangtrinhnt: yeah.
14:28:12 <dangtrinhnt> You're welcomed to put your concerns, ideas, issues, solution, etc...
14:28:50 <dangtrinhnt> ok, time up. Anything else you want to discuss?
14:29:07 <sapd1> nope :D
14:29:19 <dangtrinhnt> ah, btw, we will have the Denver summit vote results in mid of Feb (around 15th I guess)
14:29:45 <thuydang> for the SIG
14:29:47 <sapd1> PTG too.
14:29:55 <dangtrinhnt> ok
14:30:12 <thuydang> Let's do some survey on related groups
14:30:22 <dangtrinhnt> +1
14:30:32 <thuydang> then we will have better view of the vision and scope
14:30:50 <dangtrinhnt> ok, I noted down
14:31:14 <thuydang> there is probably overlaping scope
14:31:29 <sapd1> +1
14:31:34 <dangtrinhnt> one promising benefit of the SIG is we will make SL more relevant and more reason to live on
14:31:41 <dangtrinhnt> thuydang, agree
14:32:58 <dangtrinhnt> ok, time up. We can discuss about it more later.
14:33:05 <thuydang> ok
14:33:22 <dangtrinhnt> Thanks for joining the meeting today :)
14:33:53 <dangtrinhnt> Bye
14:33:54 <dangtrinhnt> #endmeeting