16:02:22 <rakhmerov> #startmeeting Mistral
16:02:23 <openstack> Meeting started Mon Sep 26 16:02:22 2016 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:02:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:02:26 <openstack> The meeting name has been set to 'mistral'
16:02:29 <rakhmerov> hi everyone
16:02:38 <mgershen1> hi
16:02:49 <ddeja> o/
16:03:30 <rakhmerov> hi
16:03:38 <rakhmerov> let's wait 1 more minute
16:03:44 <rakhmerov> someone else may join
16:04:42 <rakhmerov> ddeja: in the meantime I can answer your question that you asked in our channel
16:05:12 <rakhmerov> it's because one of the 'join' dependencies has action std.async_noop which never ends on its own
16:05:19 <ddeja> oh, yes
16:05:29 <rakhmerov> it ends only if we give it a result manually
16:05:30 <rakhmerov> yep
16:05:32 <ddeja> know it is clear
16:05:36 <ddeja> now*
16:05:37 <rakhmerov> ok
16:05:46 <rakhmerov> ok, let's start
16:06:04 <rakhmerov> d0ugal: I'm not sure if you're available but welcome to join our meeting
16:06:17 <rakhmerov> #topic Review Action items
16:06:41 <d0ugal> Hey all
16:06:44 <d0ugal> Sorry I am late
16:07:06 <rakhmerov> no problem
16:07:11 <mgershen1> hi d0ugal
16:07:11 * d0ugal went for a run and worked a bit hard, now he is recovering slowly
16:07:23 <rakhmerov> :)
16:07:26 <rakhmerov> 1. ddeja will check if it is safe to change from sync to async in default executor while returning action results
16:07:35 <rakhmerov> this is done obviously
16:07:46 <ddeja> rakhmerov: not really
16:07:49 <rakhmerov> ooh, no!!
16:07:59 <rakhmerov> yeah, I confused it with a different thing
16:08:03 <ddeja> well, it is done, but not obviously :)
16:08:08 <rakhmerov> :)
16:08:13 <ddeja> I've looked into the code
16:08:15 <rakhmerov> can you provide more details?
16:08:32 <ddeja> it should be totaly safe to change it in deafult executor
16:08:50 <ddeja> but _on_action_complete method from EngineClient is used in 3 more places
16:08:56 <ddeja> (despite tests)
16:09:01 <rakhmerov> yeah
16:09:15 <ddeja> in 2 of them it also should be safe to change into async
16:09:26 <ddeja> but in one it have to be 'sync'
16:09:33 <rakhmerov> ddeja: the only thing I'm not sure about is how synchronous action running will work
16:09:37 <rakhmerov> yes
16:09:48 <rakhmerov> if we do 'mistral action-run' in sync mode I guess
16:09:50 <rakhmerov> right?
16:09:58 <ddeja> we do it in sync mode right now
16:10:06 <rakhmerov> hm..
16:10:22 <ddeja> what we also do in sync mode, is sending result back to engine
16:10:44 <ddeja> engine send request to start action in async manner (if action is part of workflow)
16:10:46 <rakhmerov> ddeja: as far as this one I'm not sure it's good
16:11:07 <ddeja> but executor send result back in sync mode, which I guess can be changes
16:11:09 <ddeja> changed*
16:11:13 <rakhmerov> I would say why do we need to send a result synchronously?
16:11:25 <rakhmerov> yes, I see no reason why not
16:11:27 <ddeja> there is no reason
16:11:35 <rakhmerov> ok
16:11:44 <rakhmerov> so what about running action in sync mode?
16:11:50 <ddeja> but there is one place when we want to use _on_action_complete in synchronous mode
16:11:57 <rakhmerov> can it be just customized somehow for this specific case?
16:12:10 <ddeja> so we need to change the method so we can decide how we would do use it - in sync or async mode
16:12:14 <ddeja> I'll send patch tommorow
16:12:20 <rakhmerov> yeah, ok
16:12:22 <d0ugal> I always felt like sync action calls broke the models a bit :)
16:12:26 <rakhmerov> I think we're on the same page
16:12:36 <rakhmerov> d0ugal: 100% true statement
16:12:38 <rakhmerov> me too
16:12:39 <ddeja> about running action
16:12:42 <rakhmerov> yeah
16:12:53 <ddeja> run action needs to be send in sync mode
16:13:07 <rakhmerov> when I was refactoring it in summer I was about to twist my mind because this specific function
16:13:07 <ddeja> API is waiting for the reply
16:13:40 <rakhmerov> ddeja: yes, but seems like we can somehow determine that on the executor side and decide how to send a result
16:13:43 <rakhmerov> sync or async
16:13:47 <rakhmerov> right?
16:13:54 <d0ugal> Can we change it to run in async, but still return the result?
16:14:06 <d0ugal> Would that solve the problem?
16:14:12 <ddeja> yes, but run-action case is special
16:14:28 <ddeja> executor runs slightly different when run-action is executed
16:14:56 <rakhmerov> :)
16:15:02 <ddeja> and in that case, executor sends results back in async mode
16:15:02 <rakhmerov> ok
16:15:14 <ddeja> but we can still have a deadlock
16:15:20 <rakhmerov> ddeja: so what's your conclusion? Do you think it's doable?
16:15:44 <ddeja> OK, first let's distinguish two things, to make sure that we are on the same page
16:16:10 <ddeja> it is totaly doable to change how executor sends results back, I'll send patch tommorow
16:16:37 <ddeja> it is very hard to get rid of deadlock in case run-action is used to start another action
16:16:37 <rakhmerov> ok
16:17:07 <ddeja> so, with the easy change we would get rid of almost all deadlocks, but this one will still remain
16:17:14 <rakhmerov> hm.. I'm confused, I thought using 'eventlet' RPC on executor helps, no?
16:17:26 <ddeja> not for this one
16:17:48 <ddeja> as long as we have one executor, mistral run action 'run action' would deadlock on engine
16:17:53 <ddeja> one engine*
16:18:26 <rakhmerov> ok
16:18:40 <ddeja> I think the safest thing to do is:
16:18:49 <rakhmerov> wait a sec
16:18:55 <rakhmerov> just want to clarify
16:18:56 <ddeja> ok
16:19:07 <rakhmerov> this is what you discussed recently with Thomas, right?
16:19:12 <ddeja> yup
16:19:22 <rakhmerov> the corner case when we use mistral action to run action?
16:19:27 <ddeja> yes
16:19:37 <rakhmerov> ok, my suggestion: let's postpone it
16:19:49 <ddeja> I have one easy fix
16:20:01 <d0ugal> Can we make the mistral actions async only?
16:20:27 <rakhmerov> I agree this is an issue, but 1) no need to shoot our own legs 2) it's not the only issue like that
16:20:50 <rakhmerov> d0ugal: maybe, need to think
16:21:08 <rakhmerov> d0ugal: I guess sync behavior can be immitated just on the client
16:21:09 <ddeja> on executor side - if mistralclient is used in executor, we can easly check if it would be used to create action-execution and raise an exception
16:21:45 <ddeja> I can prepare a patch for this also
16:22:00 <rakhmerov> ok, if it's not too time consuming
16:22:09 <ddeja> it should not
16:22:15 <rakhmerov> please add all of us as reviewers + Thomas Herve
16:22:24 <rakhmerov> at least we can discuss it there
16:22:25 <ddeja> sure
16:22:29 <ddeja> ok
16:22:30 <d0ugal> +1
16:22:44 <mgershen1> +1
16:22:59 <rakhmerov> ok, let's move on? I suggest we take discuss it more tomorrow if needed
16:23:06 <ddeja> yes, we can move on
16:23:13 <rakhmerov> #topic Current status
16:24:14 <rakhmerov> my status: fixed one pretty simple bug (patches on review), made some more optimizations in engine (removed environment from task inbound contexts) and mostly reviewing patches, writing emails and cleaning up launchpad
16:24:23 <rakhmerov> and also backporting patches to stable/newton
16:25:06 <d0ugal> I posted a patch today to save the result of sync actions, but given the other discussion on this I am even less sure about it :) https://review.openstack.org/376366
16:25:11 <mgershen1> status 1/2: waiting for a review in rally https://review.openstack.org/#/c/358352/
16:25:35 <mgershen1> 2/2: opened a bug, but will probably work on another issue: https://bugs.launchpad.net/mistral/+bug/1627711
16:25:36 <openstack> Launchpad bug 1627711 in Mistral "on-complete of a default on-error tasks sometimes doesn't run" [Undecided,New]
16:25:42 <ddeja> my status: On monday I've spend some time on fixing the deadlock bug, but then nothing, needed to speed up work on my master thesis, I'm terriblly sorry that I didn't have time for reviews.
16:25:51 <rakhmerov> d0ugal: ok, I'll take a look
16:26:23 <d0ugal> I have also been looking into the security bug ( https://bugs.launchpad.net/mistral/+bug/1627689 ).
16:26:24 <openstack> Launchpad bug 1627689 in Mistral "Openstack service client caching breaks security and backward compatibility" [Critical,Confirmed] - Assigned to Jeff Peeler (jpeeler-z)
16:26:48 <d0ugal> but we will probably be talking about that in a bit anyway :)
16:26:50 <rakhmerov> ddeja: that's ok, no worries
16:27:10 <rakhmerov> d0ugal: yes, Jeff replied on that bug
16:27:23 <d0ugal> ddeja: I greatly appreciate all the help you have been able to give :)
16:27:34 <d0ugal> rakhmerov: he did? I can't see it.
16:28:24 <rakhmerov> in a private exchange :)
16:28:35 <rakhmerov> he is on sick leave today
16:28:52 <d0ugal> ah, I did see that
16:29:08 <rakhmerov> as far as I understand he actually might not have time to fix it
16:29:13 <rakhmerov> this week
16:29:22 <rakhmerov> so somebody else has to
16:29:29 <d0ugal> rakhmerov: Yeah, I think that is true. I would like to help.
16:29:37 <rakhmerov> d0ugal: can you?
16:29:52 <d0ugal> rakhmerov: Yeah, I think so
16:30:08 <d0ugal> rakhmerov: Given this is urgent, I would suggest reverting the patch and I'll propose an updated version ASAP
16:30:21 <rakhmerov> this is an option, yes
16:30:45 <rakhmerov> I just don't know how much important this caching is for TripleO
16:30:52 <d0ugal> rakhmerov: Not very.
16:30:59 <d0ugal> It is more of a "nice to have"
16:31:11 <rakhmerov> the solution seems to be just to build cache keys correctly
16:31:23 <d0ugal> rakhmerov: and also handle invalidated tokens.
16:31:27 <rakhmerov> so that they contain not only tenant id but a user name
16:31:31 <rakhmerov> yes
16:31:35 <rakhmerov> ooh, yes, tokens
16:31:38 <rakhmerov> forgot about them
16:31:39 <d0ugal> I think the tokens will be trickier to fix.
16:31:44 <rakhmerov> true
16:31:45 <d0ugal> The cache key should be easy
16:31:51 <rakhmerov> yep
16:32:01 <d0ugal> I'll see what I can do tomorrow.
16:32:02 <rakhmerov> as far as tokens, yep, it might be tricky
16:32:07 <rakhmerov> ok
16:32:14 <d0ugal> I'll assign it to me
16:32:16 <rakhmerov> then I'll assign it to you, okay?
16:32:22 <d0ugal> Already done :)
16:32:22 <rakhmerov> or you can do it, yes
16:32:24 <rakhmerov> ok
16:32:48 <rakhmerov> as far as tokens, we can probably parse it and see its expiration time
16:33:07 <rakhmerov> but I have never done this (with Keystone tokens)
16:33:16 <rakhmerov> ok
16:33:28 <d0ugal> No, I've not looked at them either - but I will investigate
16:33:49 <rakhmerov> ok
16:34:00 <rakhmerov> maybe it's not that hard as we imagine
16:34:11 <rakhmerov> ok, let's move on
16:34:15 <rakhmerov> #topic RC2 release readiness
16:34:26 <rakhmerov> so essentially, two things here
16:34:54 <rakhmerov> 1. we need to backport needed patches (partially done)
16:35:09 <rakhmerov> 2. Fix this caching issue
16:36:07 <rakhmerov> I also have 3 patches on review that would be nice to have in RC2
16:36:20 <rakhmerov> especially two about removing delayed calls
16:36:41 <rakhmerov> because it's really a bug and sometimes it may be severe
16:36:53 <rakhmerov> other than that seems like we're OK
16:36:57 <d0ugal> When are you planning to release RC2?
16:37:10 * d0ugal is juggling Mistral and TripleO critical bugs now :)
16:37:12 <rakhmerov> this Thursday is a deadline
16:37:18 <rakhmerov> :)
16:37:22 <d0ugal> okay, that's the same day. damn.
16:37:26 <rakhmerov> yep
16:37:38 <rakhmerov> but we need to be ready on Wed
16:37:54 <rakhmerov> and btw, I won't be available most of this Wed
16:38:05 * ddeja also
16:38:11 <d0ugal> hah, I'll be here :)
16:38:12 <rakhmerov> but I think we'll do it
16:38:43 <rakhmerov> I again realized a couple of days ago that our docs are pretty terrible
16:38:44 <rakhmerov> :)
16:38:52 <rakhmerov> honestly
16:39:09 <rakhmerov> and it's kind of sad that we again didn't have time to improve them in this cycle
16:39:24 <rakhmerov> hoping we'll be able to do it soon though
16:39:30 <d0ugal> rakhmerov: haha, I don't think they are *that* bad. Some projects are worse.
16:39:36 <rakhmerov> maybe d0ugal could help :)
16:39:47 <d0ugal> but once people stop me writing code for newton, I'd be happy to spend some time working on docs
16:39:57 <mgershen1> I really wanted to improve them too, maybe next time
16:40:01 <d0ugal> (I will be doing something similar for TripleO)
16:40:03 <rakhmerov> d0ugal: yeah, it's probably about my expectations. They are pretty high
16:40:24 <rakhmerov> I see some ideal docs in my head, and we are far away from that picture :)
16:40:25 <d0ugal> rakhmerov: Can you open a bug and write down some of the issues you have and what you would like to see?
16:40:37 <d0ugal> rakhmerov: Then assign that to me so I have something to start with
16:40:58 <rakhmerov> d0ugal: yes, definitely
16:41:17 <rakhmerov> what's more, somebody is already working to identify gaps in our docs
16:41:24 <rakhmerov> but I'll add more to that
16:41:27 <d0ugal> oh, good
16:41:33 <rakhmerov> style, structure etc. etc.
16:41:58 <rakhmerov> #action rakhmerov: file a ticket on what needs to be improved in Mistral docs and assign to d0ugal
16:42:03 <rakhmerov> ok
16:42:24 <rakhmerov> #topic Maintaining Launchpad projects (mistral and python-mistralclient)
16:43:17 <rakhmerov> I just decided to include this topic because I was trying to clean up our LP today :)
16:43:47 <rakhmerov> I know that it's been an issue for a while that we don't really use 'python-mistralclient' LP project
16:44:03 <rakhmerov> I think it's time to start doing it
16:44:04 <d0ugal> I didn't even know it existed!
16:44:31 <d0ugal> I would suggest using one project and making better use of tags in Launchpad.
16:44:42 <rakhmerov> d0ugal: ok :) Which makes me think that it's good that I included this topic
16:44:54 <rakhmerov> you think so?
16:45:03 <rakhmerov> well, wait a sec
16:45:18 <rakhmerov> the reason we were using two project is versioning
16:45:27 <rakhmerov> it's different between server and client
16:45:31 <d0ugal> This is how TripleO handles this: https://github.com/openstack/tripleo-specs/blob/master/specs/policy/bug-tagging.rst
16:45:33 <rakhmerov> as for most of other projects
16:45:49 <d0ugal> Ah, I see. I didn't think about that.
16:46:08 <rakhmerov> as far as tags, I fully agree
16:46:16 <rakhmerov> it's just about versioning
16:46:36 <d0ugal> Okay, I am not familiar enough with versioning to help there :)
16:46:46 <d0ugal> I guess in TripleO we just have the same version for everything
16:46:48 <rakhmerov> long ago we used the same versioning schema but then there came an OpenStack decision to version clients differently
16:46:52 <rakhmerov> so did I :)
16:46:57 <d0ugal> aha
16:47:03 <mgershen1> I think if we have another project for client we should use it.
16:47:22 <d0ugal> Yeah, if the versioning is different then I think it makes sense
16:47:27 <rakhmerov> yeah, we version server as 1,2,3 dev milestones, then RC1, RC3 (maybe more) and final release
16:47:27 * d0ugal has some client bugs he needs to move over
16:47:37 <rakhmerov> but for client we don't need that
16:49:25 <rakhmerov> it's not needed for client because 1) we assumingly we don't change client that much 2) many versions of client may be compatible with one server API
16:49:36 <rakhmerov> anyway, let's try to use it
16:49:55 <rakhmerov> so I'm essentially asking just to keep in mind that we have it
16:50:07 <ddeja> ok
16:50:23 <d0ugal> Sounds good. Is there a way to move bugs over?
16:50:37 <rakhmerov> not that I'm aware of
16:51:09 <rakhmerov> d0ugal: sorry :)
16:51:26 <rakhmerov> that's all that I have from my side for today
16:51:43 <d0ugal> no problem!
16:52:01 <rakhmerov> d0ugal: do you have many of them to move?
16:52:14 <d0ugal> rakhmerov: No, only one that I remember.
16:52:28 <d0ugal> and I need to check the others
16:52:34 <mgershen1> d0ugal: "Also affects project" might do it
16:53:09 <d0ugal> mgershen1: good point, that should work. I'll take a look later :)
16:53:12 <mgershen1> d0ugal: if you send me a link of such bug, I will try
16:53:25 <d0ugal> I can't even find it now, so I am confused. 1 min
16:53:27 <mgershen1> d0ugal: cool
16:53:56 <mgershen1> d0ugal: there is a + icon near it
16:55:13 <rakhmerov> ok
16:55:34 <rakhmerov> can we end the meeting?
16:55:42 <ddeja> I guess so
16:56:13 <mgershen1> fine by me
16:56:14 <d0ugal> Yup!
16:56:37 <rakhmerov> ok
16:56:43 <rakhmerov> thanks for joining!
16:56:44 <rakhmerov> bye
16:56:44 <d0ugal> hah, I couldn't find it because somebody fixed the bug already \o/
16:56:51 <mgershen1> bye!
16:56:52 <ddeja> bye
16:56:55 <d0ugal> Thanks, bye!
16:57:00 <rakhmerov> d0ugal: it might have been me ;)
16:57:01 <rakhmerov> #endmeeting