08:00:00 <eranrom> #startmeeting storlets
08:00:00 <openstack> Meeting started Tue Nov  1 08:00:00 2016 UTC and is due to finish in 60 minutes.  The chair is eranrom. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
08:00:04 <openstack> The meeting name has been set to 'storlets'
08:00:17 <eranrom> Hi
08:00:26 <takashi> o/
08:00:50 <eranrom> takashi: Hi. Shell we start?
08:01:07 <takashi> kota_: around?
08:01:50 <eranrom> takashi: Do you know if he planned on joining?
08:02:13 <kota_> here
08:02:21 <eranrom> kota_: Hi.
08:02:26 <eranrom> ok, lets start.
08:02:40 <kota_> hi eranrom, just finished the previous meeting
08:02:40 <takashi> eranrom: yes :-)
08:02:46 <eranrom> I have one topic for today:
08:03:00 <eranrom> #topic use destack for s2aio
08:03:34 <eranrom> At the summit, we agreed that the next step in the big-tent work would be to get rid of 'apt-get install keystone'
08:04:18 <eranrom> I figured out that the best way to do that is to use devstack (with keystone and swift only) instead of the current 'proprietary' swift/keystone Ansible installation scripts
08:04:26 <takashi> eranrom: Do you mind if I explain some context for that to kota_?
08:04:38 <eranrom> takashi: sure, please do
08:04:48 <takashi> eranrom: thanks
08:05:05 <takashi> akihito: hi
08:05:24 <takashi> kota_: As I told you before I'm hitting a permission problem in my packaging work
08:05:24 <eranrom> akihito: hi
08:05:27 <kota_> akihito!!!
08:05:30 <akihito> Sorry for the delay
08:05:40 <akihito> Hi!
08:06:08 <kota_> takashi: liberasurecode?
08:07:50 <takashi> takashi: no. I forgot the precise name, but something like crypto
08:08:00 <takashi> ^^^^ not for me, but for kota_
08:09:02 <takashi> which is caused by duplicate install about the package by apt and pip
08:09:42 <takashi> As you advised me, I stoped to install requirement.txt for functional testing, but it can not be the solution because it seems to be required by python-keystoneclient, which is in test-requirements.txt
08:10:00 <takashi> s/it seems/that crypto package seems/g
08:11:02 <takashi> That's why we are currently trying to get rid of apt installation.
08:11:29 <kota_> I didn't get yet why that causes using devstack?
08:11:49 <kota_> I don't have opposite but wondering
08:12:18 <eranrom> kota_: It just seems the easiest way to install keystone without taking care of many dependenies.
08:12:54 <eranrom> trying to do "pip install ." from a freshly checkout keystone master does not seem to work.
08:13:22 <eranrom> that is, the install finished successfully, but nothing actually runs due to missing deps
08:13:23 <kota_> eranrom: how about stable/xxx release?
08:13:36 <eranrom> of keystone?
08:14:06 <kota_> yup
08:14:20 <kota_> i think we have a couple of problems
08:14:30 <kota_> A. installation, B. setup
08:15:02 <eranrom> kota_: you mean installation and setup of keystone?
08:15:09 <kota_> eranrom: yes
08:15:43 <eranrom> kota_: well, devstack takes care of both (with some config nobs as well)
08:15:53 <kota_> eranrom: maybe?
08:16:08 <takashi> kota_: takling about the test for stable release, you can sepcify which tag/branch for each project do you checkout in devstack
08:16:27 <kota_> i think devstack should work for both keystone installation and setup independently
08:17:18 <eranrom> regarding stable/release, the keystone docs specifically state that the keystone repo does not take care of deps. see note here: http://docs.openstack.org/developer/keystone/installing.html
08:17:26 <kota_> but the problem takashi got seems virtual env installation on the keystone-installed environment???
08:18:22 <kota_> that doesn't seem to be related to the way (whether using apt or devstack).
08:18:24 <eranrom> kota_: oh, so you were saying that if takashi used stable/XXX for keystone that conflict between apt and pip would have been gone?
08:18:38 <kota_> eranrom: yup
08:18:59 <kota_> exactly, using devstack is an easy way to intallation/setup
08:19:15 <kota_> but the environment should be similar with installed by apt
08:19:41 <kota_> the keystone will be setup-ed in the out of virtualenv?
08:20:31 <eranrom> kota_: Apologies the ignorance, but does devstack install in a virtual env?
08:20:32 <kota_> I cannot cleary say which way is the best though.
08:20:40 <kota_> idk
08:20:59 <takashi> devstack creates virtual env for each services, so keystone is installed in one virtual env, and swift is installed in the other env (if we also install swift in devstack)
08:21:25 <kota_> takashi: you're wise
08:21:55 <eranrom> takashi: that is the idea. with devstack we also get for free additional distros, and no longer need to maintain our own install scripts
08:21:59 <kota_> that mean, storlets also cannot use the keystone virtualenv?
08:22:32 <kota_> takashi:^^
08:23:19 <takashi> kota_: still need some more look, but maybe no
08:23:31 <kota_> interesting
08:23:33 <eranrom> kota_: takashi: does it matter?
08:23:55 <eranrom> that is if storlets can use the keystone virtual env?
08:24:10 <kota_> i think what we need is to install keystone-client module in the virtual env, right?
08:24:26 <kota_> s/the virtual env/storlets virtual env/
08:24:29 <eranrom> kota_: right
08:25:40 <kota_> ah, it might work if duplicated installation for keystone-client to both keystone server virtual env and storlets virtual env
08:26:00 <kota_> and no installtion to outside python.
08:26:13 <kota_> takashi: that's what you'd like to do?
08:27:05 <takashi> kota_: yes, and the important point here is that if we install keystone in virtualenv, we don't need 'sudo' anymore for installing keystone
08:27:53 <takashi> The current problem seems to be caused when we install one package in root (by apt), and again install the same package in non-root (by pip)
08:28:27 <kota_> takashi: that looks a bit far from the fact, that avoids the installation from storlets anymore, right? IIRC?
08:28:40 <kota_> s/IIRC/IIUC/
08:31:09 <takashi> kota_: oh, yes.
08:31:26 <kota_> (or just, keeping as reference scripts)
08:32:27 <eranrom> So are we in agreement for going the devstack way?
08:33:00 <kota_> could try
08:33:18 <kota_> but we have some tasks to consider
08:33:39 <kota_> how we're migrating the test environment to devstack gate
08:34:39 <eranrom> kota_: Well, currently, I am not looking at moving the storlets installation to devstack, only using devstack to install swift and storlets as part of s2aio.
08:34:48 <eranrom> kota_: unless I am missing here something.
08:35:35 <kota_> eranrom: k
08:35:44 <kota_> eranrom: my point is
08:35:59 <kota_> how to set up the gate job
08:36:32 <eranrom> kota_: we already have a gate job that calls s2aio followed by calling the functional tests
08:36:33 <kota_> now we're using s2io setup script in the functional test gate job and it stops takashi's work
08:37:15 <takashi> kota_: At the first point, we can call stack.sh from s2aio.sh
08:37:19 <kota_> the next step is adding devstack job to run functional? I'm not sure it works or not.
08:37:41 <takashi> and keep using current func test setting
08:38:00 <kota_> takashi: ah, you mean, we are including devstack dependency to current storlets master?
08:38:33 <takashi> kota_: yes
08:39:05 <eranrom> Is that a problem?
08:39:50 <kota_> to be honest, i don't like that way because it increases extra dependencies which we don't need in production
08:40:45 <kota_> i thought, just running storlets setup and functional on devstack vm instead of adding dependency to storlets repo.
08:40:58 <eranrom> kota_: this is only used in the s2aio script, which IMO, is not actually used in production
08:41:13 <kota_> that should be crearly separated from our repo.
08:41:52 <takashi> kota_: FYI, some openstack projects like magnum has a devstack plugin in their repo, which is used for testing purpose
08:41:59 <takashi> s/has/have/g
08:42:10 <kota_> takashi: but swift doesn't
08:42:34 <takashi> kota_: That's because swift is totally cared in devstack and don't need any additional plugin
08:42:53 <eranrom> ok, what do we mean by having dependency on devstack?
08:43:01 <kota_> takashi: what's additional plugin?
08:43:33 <takashi> kota_: something like this https://github.com/openstack/magnum/tree/master/devstack
08:44:19 <takashi> Nova, Cinder or other 'core' projects do not have this kind of things, because they are taken care by devstack totally.
08:45:01 <takashi> But young projects sometimes have this kind of additonal install framework for testing in their own repo
08:46:32 <takashi> So for me, it seems reasonable to have installation framework for testing in our repo. Of cause we should mark them for 'testing purpose', not for production.
08:47:02 <kota_> takashi: I'm now confusing
08:47:13 <yuanying_> Recently, devstack plugin is main stream. Heat moved devstack plugin from devstack itself
08:47:44 <kota_> takashi: you said, we need to call stack.sh in the s2io.sh but the plugin looks to be prepared when devstack setuping
08:48:11 <kota_> takashi: I think just making preparation for setuping for devstack, that looks good
08:48:42 <eranrom> yuanying_: and I think we should do the same as a next step. However, I think that we are currently trying to settle some other misunderstanding.
08:49:01 <kota_> takashi: but I'm feeling that's bad to make install script via devstack (meaning fetch devstack and calling stack.sh) .
08:49:27 <kota_> eranrom: that's what I'm feeling > misunderstanding.
08:50:28 <eranrom> kota_: sorry. I think this is a first step that should be followed by having a storlet devstack plugin
08:50:48 <kota_> eranrom: sounds reasonable
08:50:54 <kota_> maybe the steps are
08:51:05 <takashi> eranrom: +1
08:51:28 <kota_> 1. making plugin, 2. making devstack gate which can run with plugin (this should be successful) and then
08:51:45 <kota_> 3. making a package which can run in the env 2
08:52:28 <kota_> maybe 4. set the 2, 3 to voting job and deprecate current env or set as non-voting?
08:52:40 <takashi> kota_: do you mean to add another gate job which runs over devstack installation?
08:52:49 <kota_> takashi: sur
08:52:50 <kota_> sure
08:53:23 <kota_> which can run successfuly with your new packaging?
08:53:54 <kota_> otherwise, current gate job should fail forever?
08:53:59 <kota_> am i missing something?
08:54:04 <eranrom> kota_: 2 questions.
08:54:24 <kota_> because current gate envrionment for functests is not on devstack, IIRC.
08:54:34 <kota_> eranrom: go ahead.
08:54:35 <eranrom> 1. This means that we cannot land Takashi's work until we are done with step 2
08:55:20 <kota_> eranrom: for 1, i think so.
08:55:28 <eranrom> 2. In the suggested steps, where do we place the functional tests?
08:56:17 <takashi> eranrom: are you talking about the way to run functional tests over devstack?
08:56:38 <takashi> (the way to kick our functional tests over devstack
08:56:39 <eranrom> takashi: yes.
08:56:39 <kota_> eranrom: for 2, functests will run in current env until step 2, and since step 2 functests will run both environment
08:57:01 <kota_> and then, after step3 or at the same time, we will stop to run functests in the current environment.
08:57:42 <kota_> i think
08:58:05 <eranrom> ok, I think I gotcha. I think we have two options to go forward:
08:58:53 <eranrom> 1. replace current swift/keystone installation with devstack installation, land takashis packaging work, and then proceed to adding a storlet devstack plugin and move the gate job
08:59:05 <eranrom> 2. follow kota's steps above.
08:59:30 <eranrom> I am afraid that getting a storlet plugin into devstack might take a long time.
08:59:42 <kota_> eranrom: true
08:59:55 <takashi> +1 for 1.
09:00:21 <eranrom> time is up, shell we move to #openstack-storlets?
09:00:34 <takashi> eranrom: yes
09:00:34 <eranrom> #endmeeting