16:00:42 <rakhmerov> #startmeeting Mistral
16:00:42 <openstack> Meeting started Mon Mar  6 16:00:42 2017 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:46 <openstack> The meeting name has been set to 'mistral'
16:01:04 <rakhmerov> hello
16:01:11 <fultonj> hello
16:01:26 <d0ugal> Hey!
16:01:39 <rakhmerov> :)
16:02:20 <rakhmerov> mgershen: hi, you here?
16:02:31 <rakhmerov> d0ugal: 1 min
16:02:37 <d0ugal> ure
16:02:39 <d0ugal> sure
16:03:31 <rakhmerov> ok, I'm back
16:03:55 <rakhmerov> is there anybody else on the meeting? Name yourself
16:04:55 <fultonj> just us three then?
16:05:08 <rakhmerov> d0ugal: I wonder if not sending out reminders was a bad decision :)
16:05:20 <rakhmerov> fultonj: yes, seems so
16:05:21 <d0ugal> fultonj: it might be!
16:05:31 <rbrady> o/
16:05:33 <d0ugal> rakhmerov: hmm, good question.
16:05:38 <fultonj> yay
16:05:41 <rakhmerov> ooh, Ryan, hi
16:05:55 <d0ugal> rakhmerov: I did remind rbrady 40 mins ago, so an email probably wouldn't have helped either :P
16:05:57 <rakhmerov> fultonj: I'm not sure we know each other
16:06:05 <rakhmerov> fultonj: what brought you to our meeting?
16:06:12 <fultonj> hi rakhmerov I'm working on tripleo ceph integration and want to use d0ugal's mistral-ansible-actions
16:06:21 <rbrady> sorry, I guess I need to set myself a calendar invite
16:06:28 <d0ugal> rbrady: np
16:06:31 <rakhmerov> fultonj: ook, I see
16:06:33 <rakhmerov> good
16:06:43 <rakhmerov> yeah
16:06:55 <d0ugal> rakhmerov: and that raises the question of where the ansible actions should live - it seems like mistral-extra would be best
16:07:13 <d0ugal> (but we might put them in tripleo-common until the mistral-lib/mistral-extra repos are ready)
16:07:15 <rakhmerov> you know what, I already have a calendar event, I'm going to send it out to people who need to participate (at least potentially)
16:07:32 <d0ugal> Good idea.
16:08:04 <rakhmerov> #action rakhmerov: send an invitation to team members to a weekly meeting
16:08:27 <rakhmerov> ok, we'll skip the usual topic about previous action items today
16:08:29 <rakhmerov> no action items
16:09:11 <rakhmerov> honestly, I was hoping to sort out all the notes by today's meeting after the PTG (I also took a short vacation last week) but I didn't do that
16:09:16 <rakhmerov> only started
16:09:36 <rakhmerov> so, let's quickly report our statuses as usually and then we can discuss anything
16:09:49 <rakhmerov> I'll share my impression about the PTG also
16:10:10 <rakhmerov> #topic Current status (roadblocks, what was done last week, plans)
16:10:44 <d0ugal> I have been focusing on reviewing the mistral-lib work and spent quite a bit of time making our tests faster. Then today I spent a long time trying to understand the coverage gate and eventually failing. I plan to just continue with various things and to start to move the openstack actions to mistral-extra
16:11:28 <mgershen> Hi all!
16:11:29 <rbrady> I iterated on the custom actions api patch (it got smaller, yaay) and started working on the keystone_utils port to mistral-extra
16:11:29 <rakhmerov> my status: got back from the PTG last Thu, mostly was doing reviews (there was a big queue of them), started working on workflow global context BP, started sorting out PTG notes and adjusting LP based on them
16:11:37 <rakhmerov> mgershen: hi Michal!
16:11:40 <rakhmerov> glad you joined
16:11:49 <d0ugal> mgershen: hey
16:11:58 <toure> mgershen hello
16:12:11 <rakhmerov> d0ugal: awesome job on the tests!
16:12:21 <rakhmerov> toure: hi Toure!
16:12:24 <mgershen> rakhmerov: was here, but didn't notice the time
16:12:35 <rakhmerov> yep :)
16:12:41 <d0ugal> rakhmerov: they are no longer really slow - but I still want to try and make them faster.
16:12:50 <rakhmerov> so, already have a crowd here :)
16:12:51 <rakhmerov> it's good
16:13:01 <toure> rakhmerov hi Renat :)
16:13:09 <rakhmerov> yep
16:13:46 <rakhmerov> toure, fultonj: we just started talking about our statuses, that's the usually thing we do on the meetings
16:14:05 <mgershen> status: working on adding include_output to python-mistralclient (not to cli). good progress, working on tests. we don't have any for client.
16:14:13 <rakhmerov> just to tell what we've been working on, what kind of issues we have etc.
16:14:29 <rakhmerov> fultonj, toure: if you have anything to share pls feel free
16:14:41 <rakhmerov> mgershen: good
16:15:08 <fultonj> status: I was at PTG but spent time in tripleo and I regret not stopping by to meet any of you.
16:15:23 <rakhmerov> d0ugal: do you have any estimations on how faster they can even be?
16:15:26 <toure> status: just started working on writing the spec / blueprint for error analysis
16:15:45 <rakhmerov> d0ugal: I'm just trying to understand if it's something that we need to keep doing
16:16:10 <rakhmerov> fultonj: that's ok, we may have a chance still to meet in person
16:16:16 <d0ugal> rakhmerov: we probably don't, I doubt there are any other big wins - but I find it fun :)
16:16:31 <d0ugal> rakhmerov: but I wont spend much time on it, so don't worry about that.
16:16:34 <rakhmerov> toure: ooh, cool. So do you think I can assign this BP to you?
16:16:54 <toure> rakhmerov sure, I would like to work on it
16:16:58 <d0ugal> is there a blueprint already?
16:17:16 <rakhmerov> d0ugal: yeah, I agree, it's fun. So maybe my suggestion then is to keep doing it on the background?
16:17:30 <rakhmerov> meaning that if you notice something you can go ahead and fix it
16:17:36 <rakhmerov> that's what I usually do
16:17:43 <rakhmerov> refactorings, renamings etc.
16:17:47 <d0ugal> rakhmerov: yup, that's the plan.
16:17:53 <rakhmerov> toure: ok
16:18:06 <toure> rakhmerov is it the seperation of error?
16:18:15 <rakhmerov> #action rakhmerov: assign "Error analysis BP" to Toure
16:18:32 <rakhmerov> toure: what do you mean?
16:18:55 <toure> rakhmerov I didn't know if you had one in progress
16:19:18 <rakhmerov> d0ugal: yeah, there is a BP but it's not detailed enough I think
16:19:22 <d0ugal> k
16:19:34 <rakhmerov> nope, no one is working on that now
16:19:38 <rakhmerov> at least actively
16:19:41 <rakhmerov> you can take it
16:19:44 <toure> cool
16:19:46 <toure> ack
16:19:48 <rakhmerov> gimme a sec, I'll find it
16:20:40 <rakhmerov> ok, here it is: https://blueprints.launchpad.net/mistral/+spec/mistral-debug
16:20:50 <rakhmerov> but I think we need to rename it properly
16:21:09 <rakhmerov> "Workflow error analysis" probably sounds better
16:21:21 <toure> :) was just about to write that
16:21:22 <toure> :)
16:21:56 <rbrady> rakhmerov: that spec is the endpoint and json result, correct?
16:22:15 <rakhmerov> toure: ok, just assigned and renamed
16:22:19 <rakhmerov> it's yours :)
16:22:26 <rakhmerov> rbrady: correct
16:22:42 <rbrady> rakhmerov: ack.  thanks
16:22:57 <toure> rakhmerov thanks
16:23:01 <rakhmerov> toure: I'm gonna put some more details into it tomorrow but you feel free to put yours
16:23:14 <toure> ack
16:23:31 <rakhmerov> rbrady: what about your status?
16:23:52 <rbrady> I iterated on the custom actions api patch (it got smaller, yaay) and started working on the keystone_utils port to mistral-extra
16:24:11 <rakhmerov> :) I looked at it quickly, looks good so far
16:24:17 <rakhmerov> awesome
16:24:57 <rbrady> rakhmerov: thanks.  I'm hoping the custom actions is pretty close to done.  the keystone_utils patch may take a little more time as I'm trying to work with / around using oslo_cfg
16:24:57 <rakhmerov> d0ugal: I still didn't have a chance to talk to you and rbrady about some suggestions you made regarding this BP
16:25:16 <rakhmerov> d0ugal: do you think we can do now? Or it will take too long?
16:25:30 <rakhmerov> rbrady: yep
16:25:43 <rakhmerov> I'll open a new topic
16:25:49 <rakhmerov> #topic Open discussion
16:25:50 <d0ugal> rakhmerov: it might take a while, can we quickly discuss the ansible actions first? Then I can try and explain it
16:26:06 <rakhmerov> d0ugal: np
16:26:20 <rakhmerov> d0ugal, fultonj: go ahead
16:26:26 <fultonj> #topic mistral-ansible-actions
16:26:36 <fultonj> this cycle i'm going to be implementing tripleo + ceph-ansible integration https://review.openstack.org/#/c/387631
16:26:38 <rakhmerov> #topic mistral-ansible-actions
16:26:44 <fultonj> I am planning to use d0ugal's mistral-ansible-actions
16:26:51 <fultonj> My understanding is that there will be a mistral-extra where this might live in the future
16:26:52 <d0ugal> https://github.com/d0ugal/mistral-ansible-actions
16:27:11 <rakhmerov> yes
16:27:12 <fultonj> My plan in the meantime is to put in in tripleo-common where other custom mistractions actions live
16:27:23 <fultonj> When mistral-lib is ready perhaps mistral-ansible-actions could be added in the future
16:27:54 <d0ugal> I had mentioned this to rakhmerov before briefly I think but wanted to make sure we shared the idea in the meeting
16:28:11 <rakhmerov> small correction: mistral-extra is probably the right place for them
16:28:14 <d0ugal> It is the first time Mistral will have actions for integrating to something externally (beyond OpenStack)
16:28:32 <rakhmerov> yes, that's true
16:28:58 <rakhmerov> so, generally, I think it's OK unless these actions are not generic enough
16:28:58 <d0ugal> so we may want to this about how we do that generally and what the requirements are for adding more actions
16:29:14 <rakhmerov> e.g. if they are specific to TripleO or something like that
16:29:30 <rakhmerov> d0ugal: yes
16:29:55 <d0ugal> right, and I think it is important that we have some way to automate testing - we should be able to do some basic testing with ansible (call a playbook that doesn't do anything maybe)
16:30:29 <fultonj> do you want to require someone adding something to mistral-extra to submit a unit test?
16:30:42 <rakhmerov> rbrady: Ryan, at the PTG when we were discussion what will go to mistral-extra, we decided that it will be OpenStack related stuff. I'm trying to understand if it should be ONLY OpenStack specific stuff or not
16:30:45 <mgershen> in PTG it was said mistral-extra is for openstack related things.
16:31:01 <d0ugal> fultonj: I think we need unit tests as a minimum, but it would be better if we could have some sort of integration test - even if it is very simple.
16:31:22 <d0ugal> fultonj: at the moment this is just one action, but it will be hard over time for a mistral team to verify the actions all work otherwise.
16:31:42 <d0ugal> fultonj: I guess tripleo-ci might test it eventually - but that is a special case :)
16:31:45 <rakhmerov> mgershen: yes, right. That's what we said. But is it a problem if we put ansible actions in there too?
16:32:05 <fultonj> d0ugal: sounds good to me. i think it should be tested on mistral's side too
16:32:21 <fultonj> e.g. i have a test i run to make sure i can run the hello world playbook
16:32:32 <rakhmerov> d0ugal, fultonj: yes, testing action is our pain now, we'd like to address this issue
16:32:40 <rbrady> rakhmerov: if we are going to constrain all openstack stuff to one repo, it would be better named as something like mistral-openstack.  if we're going to put all upstream supported 3rd party integration code there, mistral-extra seems okay
16:32:41 <mgershen> rakhmerov: maybe there is a better place for a more "standard" actions.
16:32:45 <rakhmerov> it's an important part of all action development
16:33:16 <mgershen> rakhmerov: and about ansible - it might be a security issue for us.
16:33:19 <rbrady> if an action or something to support an action is general in nature, mistral-lib seems like a good place
16:33:56 <d0ugal> +1
16:33:58 <rakhmerov> rbrady: I'd rather disagree, because it's just a library, not actuall action packs
16:34:03 <rakhmerov> I mean mistral-lib
16:34:19 <rakhmerov> I'd separate actions from the lib that they use
16:34:41 <rakhmerov> that's why we wanted to have actions in a separate repo (mistral-extra)
16:34:41 <rbrady> rakhmerov: are you saying that you'd like to create an additonal repo for such things?
16:34:50 <rakhmerov> so that we can test them separately etc.
16:35:05 <rakhmerov> rbrady: for what things?
16:35:12 <rakhmerov> sorry, maybe I misunderstood you
16:35:33 <rakhmerov> I was talking about why we decided to have OpenStack actions in mistral-extra, not in mistral-lib
16:35:50 <d0ugal> mgershen: in what way would it be a security issue?
16:36:08 <d0ugal> Maybe mistral-extra needs a configuration - somebody might want only specific openstack projects, or they might only want ansible (or not) etc.
16:36:21 <rakhmerov> d0ugal: 100% agree
16:36:41 <rakhmerov> it should be an option used at installation time what action packs to use
16:36:49 <d0ugal> Then mistral-extra can have a wide selection of actions (which IMO will be great for mistral adoption) but not everyone needs them all
16:36:50 <rakhmerov> to install
16:37:00 <rakhmerov> yes
16:37:04 <rbrady> rakhmerov: yes - we decided that openstack integration should be placed in mistral-extra.  From your question, I don't think we should constrain to only openstack integration in mistral-extra.
16:37:25 <rakhmerov> rbrady: yes
16:37:28 <rbrady> rakhmerov: if we constrain a library to be openstack only, it should have openstack in the name
16:37:42 <d0ugal> I think you both agree :)
16:37:43 <rakhmerov> rbrady: no-no, that's fine
16:37:57 <mgershen> d0ugal: ansible can create files on the VM it is running on. it's look std.ssh that you need the password or key in order to get access to the VM mistral is running on.
16:38:07 <rakhmerov> I mean, I was just saying that there shouldn't be any actual actions in mistral-lib, that's all I was protecting
16:38:21 <d0ugal> mgershen: good point - but it can only access what the Mistral user can access.
16:38:36 <rbrady> rakhmerov: I'm saying that a general use action, e.g. base actions should be in mistral-lib
16:38:55 <d0ugal> mgershen: would making it optional in mistral-extra via a config be okay with you?
16:39:06 <rakhmerov> rbrady: yes!
16:39:08 <rakhmerov> :)
16:39:12 <rakhmerov> we're on the same page
16:39:45 <rbrady> rakhmerov: ack.
16:39:48 <fultonj> mgershen: i was assuming the user would need to set that those things up, e.g. configure SSH keys etc.
16:40:19 <d0ugal> fultonj: but ansible can operate on localhost. so it gives you a way to run a command on the machine running mistral
16:40:22 <fultonj> the action would need to enable the mistral user beyond the default
16:41:05 <mgershen> d0ugal: I think optional is a good approach and we can talk about more details later.
16:41:11 <d0ugal> ok
16:41:53 <rakhmerov> yes, I think it's just a matter of prerequisites for a certain action pack
16:42:01 <rakhmerov> that needs to be documented etc.
16:42:10 <mgershen> fultonj: what d0ugal said
16:42:31 <fultonj> ack
16:42:33 <rakhmerov> so, then is everyone ok with ansible actions living in mistral-extra?
16:42:47 <rakhmerov> along with OpenStack actions
16:43:09 <d0ugal> I guess this should be documented somewhere? In mistral-extra maybe?
16:43:29 <rakhmerov> and I agree with d0ugal, we'll need to have flexibility on what to install from the whole bunch of actions
16:43:42 <mgershen> if we can make them optional, I think it's fine
16:43:44 <rakhmerov> d0ugal: documented what?
16:43:55 <d0ugal> rakhmerov: what mistral-extra is and what it should contain :)
16:44:01 <rakhmerov> sure
16:44:17 <rakhmerov> it's a part of this whole work
16:44:33 <rakhmerov> we're just in the beginning
16:44:41 <toure> +1 mistral-extra should contain various actions
16:44:47 <rakhmerov> ok
16:45:17 * rakhmerov renat needs to learn how to use voting system in IRC ;)
16:45:27 <d0ugal> :)
16:45:31 <rakhmerov> ok
16:45:46 <rakhmerov> so, anything else you'd like to bring up?
16:46:33 <mgershen> coverage efforts?
16:46:43 <rakhmerov> FYI: once I'll update LP I want to hold a planning meeting for this milestone which ends on Apr 14th
16:47:01 <d0ugal> mgershen: hah, and our coverage tracking is broken :)
16:47:02 <rakhmerov> we can devote the next meeting to that
16:47:16 <rakhmerov> d0ugal: yeah, can you tell about it?
16:47:31 <rakhmerov> sharatass is not here
16:47:37 <d0ugal> the coverage report changes between test runs
16:47:49 <rakhmerov> d0ugal: I saw that you found something today but didn't read into it
16:48:03 <rakhmerov> d0ugal: you mean w/o any rebases?
16:48:10 <d0ugal> rakhmerov: Yeah
16:48:25 <d0ugal> rakhmerov: if you run "python setup.py testr --coverage" twice (or maybe a few times) you will sometimes get different results
16:48:29 <d0ugal> I don't fully understand why
16:48:55 <rakhmerov> wooow
16:48:58 <rakhmerov> how come? :)
16:49:21 <d0ugal> I really don't know
16:49:41 <d0ugal> If I were to guess, I would say that some code only runs under specific race conditions? but that doesn't make sense to me
16:50:01 <d0ugal> This patch for example, it doesn't change anything
16:50:02 <d0ugal> https://review.openstack.org/#/c/441902/
16:50:34 <d0ugal> and Jenkins ran against patchset 3 twice (I rechecked it) and got different results
16:50:40 <rakhmerov> weird
16:51:08 <d0ugal> it might be something to do with eventlet and limitations of coverage?
16:51:15 <d0ugal> but I am making wild guesses at this point :)
16:51:35 <rakhmerov> don't know really :)
16:51:38 <mgershen> there were also weird results here: https://review.openstack.org/#/c/438913/
16:52:16 <mgershen> we all reviewed it and I complained about unexpected coverage result in the comments.
16:53:35 <rakhmerov> d0ugal, mgershen: do I understand right that it's not only about delta between two revisions? But even for one patchset it may be different?
16:53:45 <mgershen> to tests d0ugal theory I think we can remove some tests (on devstack only) and see if the unsuitability stops when running coverage locally.
16:53:48 <d0ugal> rakhmerov: yup, even for one patch set
16:54:06 <rakhmerov> woow
16:54:41 <d0ugal> rakhmerov: if you edit the cover.sh file locally and remove the git commands - so it always runs the same code you will find it still changes sometimes
16:54:42 <rakhmerov> so that makes me think that until we figure out what's wrong with it it doesn't make a lot of sense to rely on this gate
16:54:53 <d0ugal> Agreed
16:55:23 <d0ugal> I hear murano is using the same script and it works for them - so it must be something "special" about mistral
16:55:54 <rakhmerov> ok
16:56:22 <d0ugal> mgershen: so keep removing tests to find where the problem is?
16:56:35 <mgershen> d0ugal: yes.
16:56:50 <d0ugal> good idea. I wonder if I could script that
16:57:09 <mgershen> d0ugal: starting from tests that we thing test the files that keep their coverage keeps changing
16:57:21 <d0ugal> delete each test file one by one, running the tests 4 times each time until the coverage stops changing
16:57:37 <rakhmerov> :)
16:57:38 <mgershen> d0ugal: nice idea
16:57:41 <rakhmerov> yep
16:57:54 <d0ugal> mgershen: oh, that is probably a better approach - but I am lazy, so I might try and automate it all :D
16:57:55 <mgershen> d0ugal: really nice!!
16:57:57 <rakhmerov> otherwise you'll spend a month to find the reason
16:58:02 <d0ugal> lol
16:58:14 <rakhmerov> ok, it's about time to finish
16:58:16 <d0ugal> Okay, I'll try and do that
16:58:27 <d0ugal> #action d0ugal to find tests that change coverage randomly
16:59:43 <rakhmerov> ok
16:59:47 <rakhmerov> let's stop here
16:59:51 <rakhmerov> thanks for joining
17:00:00 <rakhmerov> bye
17:00:01 <d0ugal> Thanks!
17:00:05 <mgershen> bye
17:00:12 <rakhmerov> #endmeeting