| *** sapd1 has joined #openstack-searchlight | 01:38 | |
| dangtrinhnt | Guys, today we will have the team meeting at 13:30 UTC today in this channel. Please look at the etherpad for the agenda: https://etherpad.openstack.org/p/search-team-meeting-agenda | 02:03 |
|---|---|---|
| dangtrinhnt | Everyone are welcomed to put ideas | 02:09 |
| sapd1 | dangtrinhnt: yeah | 03:04 |
| *** sapd1 has quit IRC | 11:37 | |
| *** thuydang has joined #openstack-searchlight | 12:33 | |
| dangtrinhnt | #startmeeting openstack search | 13:30 |
| openstack | Meeting started Mon Feb 11 13:30:34 2019 UTC and is due to finish in 60 minutes. The chair is dangtrinhnt. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:30 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:30 |
| *** openstack changes topic to " (Meeting topic: openstack search)" | 13:30 | |
| openstack | The meeting name has been set to 'openstack_search' | 13:30 |
| dangtrinhnt | thuydang, sapd1_ meeting time :) | 13:30 |
| thuydang | Hi | 13:31 |
| dangtrinhnt | hi, sapd1_ are you there? | 13:31 |
| dangtrinhnt | ok, let's start with the #topic Denver summit demos | 13:34 |
| dangtrinhnt | https://etherpad.openstack.org/p/search-team-meeting-agenda | 13:34 |
| dangtrinhnt | #topic Denver summit demos | 13:34 |
| *** openstack changes topic to "Denver summit demos (Meeting topic: openstack search)" | 13:34 | |
| *** sapd1 has joined #openstack-searchlight | 13:34 | |
| dangtrinhnt | thuydang, do you have any update on this? | 13:35 |
| dangtrinhnt | sapd1, hi | 13:35 |
| dangtrinhnt | https://storyboard.openstack.org/#!/story/2004840 | 13:35 |
| thuydang | There is no update for the demo on my side. I was quite busy. | 13:35 |
| sapd1 | hi | 13:35 |
| dangtrinhnt | ok, me too for the last 2 weeks. | 13:35 |
| dangtrinhnt | I will work on it this week | 13:36 |
| thuydang | Ah, I added some task to the storyboard link above. | 13:36 |
| sapd1 | yes. Because I was on holiday last week. | 13:36 |
| dangtrinhnt | sapd1, could you please give me some screenshots of what you mentioned in the story? https://storyboard.openstack.org/#!/story/2004840 | 13:36 |
| dangtrinhnt | thuydang, sapd1, yeah It | 13:37 |
| dangtrinhnt | It's the Lunar New Year for us :D | 13:37 |
| dangtrinhnt | no worries, as long as I have a better understanding of what sapd1's idea, I can start working on it, properly before the Stein-3 milestone | 13:38 |
| thuydang | @dangtrinhnt which screenshots? | 13:38 |
| sapd1 | dangtrinhnt: You mean mock screenshot. | 13:39 |
| dangtrinhnt | sapd1, yeah | 13:39 |
| dangtrinhnt | or something like that | 13:39 |
| sapd1 | dangtrinhnt: I will draw a diagram for that. maybe tomorrow, Have you guys tried setup keystone-to-keystone yet? | 13:40 |
| thuydang | not yet | 13:40 |
| thuydang | I could start 1 devstack | 13:40 |
| dangtrinhnt | sapd1, thanks. No, I don't have the resource for that, maybe trying to borrow some server somewhere. | 13:40 |
| thuydang | do you think testing with 2 devstack is a good idea? | 13:41 |
| dangtrinhnt | thuydang, that's good enough for the demo I think | 13:41 |
| sapd1 | dangtrinhnt: You can try with 2 small server, with only keystone and glance are installed. | 13:41 |
| sapd1 | that's enough | 13:41 |
| dangtrinhnt | ok, sounds reasonable. Thanks. | 13:41 |
| thuydang | I have to increase timeout when installing on my laptop | 13:42 |
| sapd1 | #link https://docs.openstack.org/keystone/pike/advanced-topics/federation/configure_federation.html | 13:42 |
| dangtrinhnt | thuydang, what timeout you're talking about? | 13:42 |
| thuydang | devstack has a timeout setting | 13:42 |
| thuydang | otherwise it crashes when some service takes too long to start up | 13:43 |
| thuydang | glance in my case | 13:43 |
| dangtrinhnt | ah, ok, cause normally devstack still eats lots of memory. I have to run inside a 16GB machine | 13:43 |
| thuydang | I used a VM with 8G | 13:44 |
| sapd1 | with keystone-2-keystone I think We don't need to use devstack. | 13:44 |
| sapd1 | I will write Dockerfile and docker-compose for that. | 13:44 |
| sapd1 | 2 keystone containers and 2 mariadb containers | 13:44 |
| sapd1 | I think | 13:45 |
| dangtrinhnt | sapd1, but how can you have Searchlight and other needed services? | 13:45 |
| thuydang | just test the api and access token right? | 13:45 |
| dangtrinhnt | sapd1, your proposed method just for testing keystone-2-keystone federation right? | 13:46 |
| sapd1 | We can run in container too :D | 13:46 |
| sapd1 | We can run every services in container. | 13:47 |
| dangtrinhnt | I may try with Kolla but Kolla need at least 2 network interfaces | 13:47 |
| dangtrinhnt | and it crashes every time :) | 13:47 |
| dangtrinhnt | ok, let me try and get back to you guys at the end of this week. | 13:48 |
| thuydang | let's draw a diagram for understanding | 13:48 |
| sapd1 | dangtrinhnt: I will give you my Dockerfile of searchlight and keystone too | 13:48 |
| dangtrinhnt | sapd1, that would be awesome | 13:48 |
| dangtrinhnt | you can even commit a patch set to put it inside searchlight repo | 13:49 |
| dangtrinhnt | anything else about the demos? | 13:49 |
| dangtrinhnt | thuydang, sapd1 | 13:49 |
| sapd1 | let discuss about keystone-2-keystone. | 13:49 |
| thuydang | I think we should provide diagrame of the workflow | 13:49 |
| sapd1 | assume we have 3 keystone services - let's say: keystone-idp, keystone-sp1, keystone-sp2 | 13:50 |
| dangtrinhnt | ok | 13:50 |
| sapd1 | idp - identity provider , sp - service provider | 13:51 |
| thuydang | e.g., for sp is glance, etc? | 13:51 |
| sapd1 | no, | 13:51 |
| sapd1 | sp is another openstack cluster | 13:51 |
| sapd1 | so on keystone-idp we can see sp1 and sp2 then we can search in 3 openstack clusters - | 13:52 |
| thuydang | I thought each openstack cluster has 1 keystone holding OS services endpoints | 13:52 |
| thuydang | ok, that means 1 OS will be the idp | 13:53 |
| dangtrinhnt | thuydang, this will explain https://docs.openstack.org/keystone/pike/advanced-topics/federation/configure_federation.html | 13:53 |
| sapd1 | but on sp1 and sp2, they only can see idp | 13:53 |
| sapd1 | so the problem here is when user switch to sp1 or sp2, how he/shee can search in other service provider. | 13:54 |
| dangtrinhnt | sapd1, ok, I think there is a way in Searchlight to indicate the cluster that you're searching | 13:55 |
| sapd1 | *she | 13:55 |
| sapd1 | in my environment, we don't create users in service provider. | 13:56 |
| thuydang | Tenant should be able to add her other SP right? | 13:56 |
| sapd1 | So I think this user can re-authenticate with idp and search. | 13:57 |
| sapd1 | what do you mean? thuydang | 13:57 |
| dangtrinhnt | If we use the centralized authentication like this, things should be simple. Just need to indicate the tenant ID when querying resources | 13:58 |
| sapd1 | dangtrinhnt: yeah. | 13:58 |
| sapd1 | but with k2k, We have local tenant id and remote tenant id | 13:59 |
| dangtrinhnt | I'm not sure if it's possible right now but logically, it's the way. Something likes searchlight.query(resource_type='NOVA_SERVER', tenant_id='abc') | 13:59 |
| thuydang | so the idp knows about a tenant's resources in all SP clusters? | 13:59 |
| sapd1 | dangtrinhnt: Cool, tenant_id='local_abc|remote_cdf' | 13:59 |
| dangtrinhnt | thuydang, no | 14:00 |
| sapd1 | thuydang: It's a mapping. | 14:00 |
| dangtrinhnt | idp just stores the identities, for tenant resources, searchlight will use its plugins to query | 14:00 |
| sapd1 | thuydang: map tenant_A in idp with tenant_B in sp1 | 14:00 |
| sapd1 | s/with/to | 14:01 |
| thuydang | that's whtat I mean | 14:01 |
| dangtrinhnt | sapd1, I think what you meant is Idp for Searchlight to authenticate against the tenants | 14:02 |
| thuydang | how can the mapping be done? somewhere we must specify that tenant_A in idp is tenant_B in sp1 | 14:02 |
| sapd1 | dangtrinhnt: sorry, as design, we will run searchlight service and ES as well in all openstack clusters. So we will search through searchlight-api | 14:03 |
| thuydang | Let's put it like this: | 14:04 |
| thuydang | we have 2 clusters with searchlight installed | 14:04 |
| thuydang | the problem is using searchlight in 1 cluster to search in the other, right? | 14:04 |
| sapd1 | thuydang: yep | 14:05 |
| dangtrinhnt | so you have multiple ES instances? | 14:05 |
| thuydang | I thinks so | 14:05 |
| thuydang | because the clusters belonging to multiple SPs may have nothing to do with each other | 14:06 |
| thuydang | OS- AWS for example | 14:06 |
| thuydang | and so is OS - OS | 14:06 |
| dangtrinhnt | Then, each SL instance will use the other tenant's SL-api to query the other tenant's resources? | 14:06 |
| thuydang | Yes, I thinks so | 14:07 |
| dangtrinhnt | wow, that not scales :) | 14:07 |
| thuydang | why? | 14:07 |
| sapd1 | thuydang: https://github.com/openstack/keystone/blob/master/keystone/federation/backends/sql.py mapping table in sql database | 14:08 |
| sapd1 | dangtrinhnt: Yep. I understand | 14:08 |
| dangtrinhnt | for every new tenant you want to search, you have to say somewhere the searchlight-api endpoints | 14:08 |
| sapd1 | thuydang: When we have many many openstack clusters, So we have to go to every endpoint to search. | 14:08 |
| thuydang | I don't think so | 14:09 |
| thuydang | at worst case we only search in the cluster the tenant has access right? | 14:09 |
| sapd1 | we have to try to authenticate. Because the mapping is created in SP side. | 14:10 |
| dangtrinhnt | ok, I think we will have a better view with drawing :) | 14:12 |
| thuydang | sure, the tennant must provide credential for each of his cluster, which is used by SL to authenticates | 14:12 |
| thuydang | yes, let's have anothe meeting for this discussion | 14:13 |
| thuydang | let's move on with other topics | 14:13 |
| sapd1 | yeah | 14:13 |
| dangtrinhnt | ok, looks like the topic gets some interesting points :) | 14:13 |
| dangtrinhnt | #topic Functional tests for Searchlight in Py3 | 14:14 |
| *** openstack changes topic to "Functional tests for Searchlight in Py3 (Meeting topic: openstack search)" | 14:14 | |
| dangtrinhnt | ok, the TC is checking our progress with python 3 functional tests | 14:14 |
| dangtrinhnt | Over the last couple months, I added the python 3 tests for unit and functional tests | 14:15 |
| dangtrinhnt | looks like it's acceptable compare to other projects | 14:15 |
| dangtrinhnt | #link https://wiki.openstack.org/wiki/Python3 | 14:15 |
| dangtrinhnt | I updated the documents yesterday | 14:16 |
| thuydang | do we have to migrate searchlight api to python3? | 14:16 |
| dangtrinhnt | I think SL is py3 compatible | 14:17 |
| dangtrinhnt | it passes the python3 tests | 14:17 |
| dangtrinhnt | but the thing here is our test coverage is low, only 70 something | 14:17 |
| dangtrinhnt | I would like to increase the test coverage to 90 something in the RC milestone | 14:18 |
| dangtrinhnt | it means we need to add more tests and maybe we can separate the functional tests with Zuul | 14:18 |
| dangtrinhnt | and tempest | 14:18 |
| thuydang | I'm not familiar with testing and will have to learn first :-) | 14:19 |
| dangtrinhnt | something like this https://github.com/openstack/tacker/blob/master/.zuul.yaml | 14:19 |
| dangtrinhnt | with zuul, we can set our test env and dependencies :) | 14:20 |
| dangtrinhnt | anyway, it's for the next milestone | 14:20 |
| dangtrinhnt | and only if we have time :D | 14:20 |
| sapd1 | thuydang: me too | 14:20 |
| dangtrinhnt | no worries, it's just a good-to-have feature. | 14:21 |
| thuydang | we will try to cover testing with our new features | 14:21 |
| dangtrinhnt | ok, I guess we can move on to the next topic? | 14:21 |
| thuydang | ok | 14:21 |
| dangtrinhnt | #topic TC vision reflection: | 14:21 |
| *** openstack changes topic to "TC vision reflection: (Meeting topic: openstack search)" | 14:21 | |
| dangtrinhnt | as you may now the OpenStack leadership has published a vision at #link https://governance.openstack.org/tc/reference/technical-vision.html | 14:22 |
| dangtrinhnt | It's good to align our works with the TC vision | 14:22 |
| dangtrinhnt | good practice | 14:22 |
| dangtrinhnt | we can learn from other projects | 14:23 |
| thuydang | I agree | 14:23 |
| dangtrinhnt | https://etherpad.openstack.org/p/nova-tc-vision-self-eval | 14:23 |
| dangtrinhnt | https://review.openstack.org/#/c/629060/ | 14:23 |
| dangtrinhnt | https://review.openstack.org/#/c/630216/ | 14:23 |
| dangtrinhnt | they're doing the reflection | 14:23 |
| sapd1 | dangtrinhnt: I will read it later. | 14:24 |
| dangtrinhnt | ok, I started a new etherpad | 14:24 |
| dangtrinhnt | https://etherpad.openstack.org/p/-tc-vision-self-eval | 14:24 |
| dangtrinhnt | we will do about 3 weeks to 9 Mar. to put ideas to the pad and I will commit a doc change | 14:25 |
| dangtrinhnt | Please help us to evaluate our works against the TC vision. :) | 14:26 |
| sapd1 | dangtrinhnt: yeah. | 14:27 |
| dangtrinhnt | You're welcomed to put your concerns, ideas, issues, solution, etc... | 14:28 |
| dangtrinhnt | ok, time up. Anything else you want to discuss? | 14:28 |
| sapd1 | nope :D | 14:29 |
| dangtrinhnt | ah, btw, we will have the Denver summit vote results in mid of Feb (around 15th I guess) | 14:29 |
| thuydang | for the SIG | 14:29 |
| sapd1 | PTG too. | 14:29 |
| dangtrinhnt | ok | 14:29 |
| thuydang | Let's do some survey on related groups | 14:30 |
| dangtrinhnt | +1 | 14:30 |
| thuydang | then we will have better view of the vision and scope | 14:30 |
| dangtrinhnt | ok, I noted down | 14:30 |
| thuydang | there is probably overlaping scope | 14:31 |
| sapd1 | +1 | 14:31 |
| dangtrinhnt | one promising benefit of the SIG is we will make SL more relevant and more reason to live on | 14:31 |
| dangtrinhnt | thuydang, agree | 14:31 |
| dangtrinhnt | ok, time up. We can discuss about it more later. | 14:32 |
| thuydang | ok | 14:33 |
| dangtrinhnt | Thanks for joining the meeting today :) | 14:33 |
| dangtrinhnt | Bye | 14:33 |
| dangtrinhnt | #endmeeting | 14:33 |
| *** openstack changes topic to "OpenStack Searchlight - https://wiki.openstack.org/wiki/Searchlight" | 14:33 | |
| openstack | Meeting ended Mon Feb 11 14:33:54 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:33 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_search/2019/openstack_search.2019-02-11-13.30.html | 14:33 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_search/2019/openstack_search.2019-02-11-13.30.txt | 14:33 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_search/2019/openstack_search.2019-02-11-13.30.log.html | 14:34 |
| sapd1 | bye | 14:34 |
| dangtrinhnt | sapd1, please put your mock figure about keystone-2-keystone on the story: https://storyboard.openstack.org/#!/story/2004840 | 14:38 |
| dangtrinhnt | one thing I'm worrying about your design is that It will work only for OpenStack clouds, when we want to integrate other cloud platforms, we may have to design another architecture. In a scalable and extensible system design perspective, Searchlight needs to have an universal interface to plug in any cloud platform it want. Something like patterns or unified architecture, so that we don't have to reinvent the wheel every time. Anyway, | 14:43 |
| dangtrinhnt | that's too much for today. Let's see how it goes :) | 14:43 |
| thuydang | Let's go from a simple solution and improve it :-) | 14:45 |
| dangtrinhnt | Thank you guys for putting the fire into this project, It's a little bit quiet these days :) | 14:45 |
| *** sapd1 has quit IRC | 16:01 | |
| *** thuydang has quit IRC | 16:36 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!