16:01:31 <rakhmerov> #startmeeting Mistral
16:01:32 <openstack> Meeting started Mon May 16 16:01:31 2016 UTC and is due to finish in 60 minutes.  The chair is rakhmerov. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:35 <openstack> The meeting name has been set to 'mistral'
16:02:05 <rakhmerov> hi
16:02:19 <rakhmerov> who is here?
16:02:37 <rbrady> o/
16:02:59 <mgershenzon> O/
16:03:06 <rakhmerov> hi guys
16:03:34 <rbrady> :)
16:03:41 <rakhmerov> 1 sec
16:04:13 <rakhmerov> ok, let's start
16:04:17 <rakhmerov> #topic Review action items
16:04:21 <rakhmerov> no items )
16:04:39 <rakhmerov> #Current status (progress, issues, roadblocks, further plans)
16:04:52 <rakhmerov> let's quickly tell our updates, if any
16:05:48 <rakhmerov> my status: working on several blueprints at the same time
16:05:49 <rakhmerov> https://blueprints.launchpad.net/mistral/+spec/mistral-engine-error-handling
16:06:44 <rakhmerov> https://blueprints.launchpad.net/mistral/+spec/mistral-action-result-processing-pipeline
16:07:01 <rakhmerov> it all came to the need to do pretty serious refactoring in Mistral engine
16:07:13 <rakhmerov> will get it done this week,
16:07:40 <ddeja> hey guys
16:07:50 <rakhmerov> besides that, other small activities like fixing gates, dealing with oslo.messaging issue (again)
16:07:53 <rakhmerov> hi Dawid
16:08:07 <rakhmerov> please let us know if you have any updates
16:08:52 <rakhmerov> I'm planning to work on custom actions API after my current tasks
16:09:01 <ddeja> unfortunetely no - I spend last week on sick leave but I'm starting to feeling better so I hope to be back at work tommorow or on Wednesday
16:09:15 <rbrady> rakhmerov: I'd like to contribute to that bp as well
16:09:25 <rakhmerov> ddeja: ooh, ok, I hope get better soon
16:09:32 <rakhmerov> rbrady: ok, sure
16:09:59 <rakhmerov> rbrady: do you think we need to create a spec for that?
16:10:16 <rakhmerov> we can discuss what we do together and you can actually take it
16:10:20 <rakhmerov> or part of it
16:10:20 <rbrady> rakhmerov: it might be a good idea and allow us a way to solicit more feedback
16:10:34 <rbrady> rakhmerov: ack
16:11:10 <rbrady> rakhmerov: I had wanted to start on it by now, but I ran into a bug that's taking my time (more on that in the open discussion)
16:11:16 <rakhmerov> rbrady: yes, my only concern is that I think when writing a spec it's usually hard to predict all cool things that you can come up with when actually implementing it
16:11:43 <rakhmerov> so maybe we need a spect but w/o an attempt to make it too detailed
16:11:57 <rbrady> rakhmerov: agreed.  in a agile world the spec is a minimum viable product plan
16:12:06 <rakhmerov> only reflect main things and leave opportunities for extention
16:12:22 <rakhmerov> rbrady: we're on the same page then
16:12:23 <rakhmerov> perfect
16:13:01 <rakhmerov> let's move to the following topic then
16:13:06 <rakhmerov> #Discuss versioning issue
16:13:24 <rakhmerov> subtitle: The latest one is 2.0.0 but "pip install mistral' installs 2015.1.0 since it's greater
16:13:53 <rakhmerov> I just wanted to discuss with the team the best approach to solve this issue
16:14:19 <rakhmerov> we have multiple versions on PyPi: including 2015.1.0 which is actually older than 2.0.0
16:14:30 <rakhmerov> because we used to have year based versioning
16:14:40 <rakhmerov> but then we switched to regular schema
16:15:10 <rbrady> rakhmerov: do we know of anyone currently still depending on 2015.1.0?
16:15:16 <rakhmerov> and now if people use "pip install mistral" they get 2015.1.0 version because it's greater
16:15:26 <rakhmerov> rbrady: that's the exact issue :)
16:15:45 <rakhmerov> If I knew that nobody was using it I'd just remove it
16:16:06 <rakhmerov> so maybe you came across a similar situation?
16:16:09 <rbrady> rakhmerov: removing it is one way to conduct a survey of who's using it
16:16:13 <rakhmerov> what do people usually do?
16:16:21 <rakhmerov> survey?
16:16:23 <rakhmerov> how?
16:16:37 <rakhmerov> in ML?
16:17:11 <rbrady> rakhmerov: if you remove it from pypi, people may reach out to ask where it went.
16:17:19 <rbrady> rakhmerov: mailing list is probably a good option
16:17:25 <rakhmerov> yeah
16:17:43 <ddeja> maybe we could just rename the old versions?
16:17:53 <rakhmerov> there's also an option to remove 2015.1.0 and re-release it under say 1.2.0
16:18:16 <rakhmerov> ddeja: I need to check if PyPi allows that
16:19:07 <rakhmerov> but generally what rbrady says makes sense to me, if they find that it's removed they would likely just ask where it is
16:19:16 <rakhmerov> and what they should use instead
16:19:38 <rakhmerov> and if it's possible to keep it but under a different name then it would be ok
16:19:39 <ddeja> yeah, I'm ok with that
16:19:40 <rakhmerov> I think
16:19:58 <rakhmerov> hm.. ok, let me check it after it
16:20:06 <rbrady> what policy does mistral have for supporting previous releases?
16:20:22 <rakhmerov> rbrady: only previous stable version
16:20:45 <rakhmerov> 2015.1.0 is not supported already
16:20:53 <rakhmerov> it's a year old
16:21:14 <mgershenzon> I think if we rename that release, the new name should look similar to humans.
16:21:32 <rakhmerov> mgershenzon: ideally yes, but how?
16:21:58 <rakhmerov> somehow we need to transform 2015.1.0 to something like 1.1.1
16:22:17 <rakhmerov> don't know how
16:22:31 <rbrady> is there a specific place on pypi where it displays older releases? I do not see it at https://pypi.python.org/pypi/mistral
16:22:41 <rakhmerov> #action rakhmerov: check if PyPi allows to rename releases
16:22:57 <mgershenzon> Maybe with concatenation? Adding "0." at the beginning. Is that a legal version?
16:23:05 <rakhmerov> yes, there's an admin interface
16:23:19 <rakhmerov> mgershenzon: not sure, need to check
16:23:56 <rakhmerov> you know, seems like it's possible to rename it
16:24:02 <rakhmerov> via PyPi UI
16:24:07 <rakhmerov> I just checked it
16:24:17 <rbrady> just as an average user, I'm not sure if they are able to see the release through the web ui at all.  using the search I can only find mistralclient
16:24:37 <mgershenzon> 0.2015.1looks similar to me, if we can't concatenate the full version.
16:24:38 <rakhmerov> rbrady: yeah, it's a bug in PyPi
16:24:44 <rakhmerov> they are working on it
16:24:45 <ddeja> rbrady: you can list all of the version of package after adding "/json" to the end of pypi URL
16:24:58 <rakhmerov> a bunch of packages are not searchable although they exist
16:25:17 <rakhmerov> ddeja: ooh really?
16:25:22 <rakhmerov> that's cool
16:25:33 <ddeja> it gives pure json
16:25:45 <rakhmerov> ok
16:25:54 <ddeja> with a lot of stuff like all package versions ;)
16:26:29 <rakhmerov> alright, I think I'll just rename it. Just need to pick a good name and announce it in ML I guess
16:26:42 <rbrady> with the policy of only supporting the previous stable version, removing anything prior to 1.0.0 should be okay then
16:27:17 <rakhmerov> and be ready to get piled with rotten tomatoes from our senior community members )
16:28:06 <rakhmerov> rbrady: yes, I was just thinking about potential users who might be using it in some production clouds of older versions (kilo)
16:28:11 <mgershenzon> If we can rename, it is better than deleting
16:28:16 <rbrady> ack
16:28:17 <rakhmerov> yep
16:28:30 <rakhmerov> ok, I think it's kind of solved
16:28:49 <rakhmerov> #topic Discuss what to do with "at-least-once"
16:29:08 <rakhmerov> some background
16:29:30 <rakhmerov> oslo team recently released oslo.messaging 5.0.0 :)
16:29:47 <rakhmerov> and it broke our dirty hack in rpc.py
16:29:47 <ddeja> Oh
16:29:56 <rakhmerov> that was needed to implement at-least-once delivery
16:30:01 <rakhmerov> ddeja: yeah :(
16:30:22 <rakhmerov> but that's the consequence that we were aware of from the very beginning
16:30:52 <ddeja> (I was hoping for the oslo team to not be this fast)
16:31:09 <rakhmerov> the bad thing is that their classes and interfaces changed significantly and we need to spend time again to understand how to apply a similar hack
16:31:20 <rakhmerov> ddeja: yeah, me too
16:31:25 <rakhmerov> but it is what it is
16:31:55 <rakhmerov> so for now I sent a patch https://review.openstack.org/#/c/316578/ to remove that hack until we find a solution
16:32:45 <rakhmerov> options: 1) research their new code and do a similar hack 2) finally implement our rpc layer based on direct access to amqp
16:33:06 <rakhmerov> there's the 3rd: keep pushing that change to oslo.messaging to support at-least-once
16:33:40 <rakhmerov> here's my opinion: I don't like 1) and would prefer to go with 2)
16:34:10 <ddeja> +1
16:34:13 <mgershenzon> What about 3? Do we have a chance?
16:34:15 <rakhmerov> on my estimation it would take ~1 month to implement it, especially given that we already have patches that do that mostly
16:34:29 <rakhmerov> we can reexamine them and reuse if possible
16:34:54 <rakhmerov> mgershenzon: I think yes, but it will be a long process and we should be doing it in parallel
16:35:23 <ddeja> yup, using own rpc layer is still kind of hacky
16:35:30 <rakhmerov> ddeja: actually I was going to suggest you be working on that RPC impl )
16:35:42 <rakhmerov> ddeja: why?
16:35:45 <rakhmerov> why hacky?
16:36:13 <ddeja> I was just going to say that after we have our own implementation we still should push on oslo
16:36:22 <rakhmerov> yes, true
16:36:25 <ddeja> so we can eventually start using it fully
16:36:53 <ddeja> becouse from my perspective it's better if openstack project uses only oslo messaging, not their own solution
16:37:06 <ddeja> but i totaly agree that option 2 is better than 1 :)
16:37:08 <mgershenzon> +1
16:37:18 <rakhmerov> but here's my thinking: we could make it configurable so that we explicitly declare that "Ok, we give you a choice to use either oslo.messaging based RPC impl but you won't get at-least-once or you can use AMQP directly with all bells and wistles"
16:37:34 <ddeja> yup
16:37:36 <rakhmerov> ddeja: yeah, right
16:38:10 <rakhmerov> another reason I'm saying this: we already spent a year trying to push it into oslo :)
16:38:14 <rakhmerov> w/o any luck
16:38:29 <ddeja> I'm only thinking if we are going to support only rabbit in our layer?
16:38:34 <rakhmerov> new people come to oslo, new ptls etc. and we have to start over again and again
16:39:22 <rakhmerov> ddeja: I think we should start with a flexible design in Mistral itself
16:39:22 <rakhmerov> what I mean is basically:
16:39:27 <rakhmerov> our RPC layer is nothing else but ~10 methods
16:39:35 <rakhmerov> im rpc.py
16:39:40 <ddeja> sure
16:39:51 <rakhmerov> it could be just an interface (or abstract class)
16:40:34 <ddeja> ok
16:40:58 <rakhmerov> and for it we could have different implementations like: amqp (could be actually splitted by driver types: say pika and kombu), zero mq, qpid
16:41:10 <ddeja> ok, cool
16:41:19 <rakhmerov> in practice, most of the time we'll need just rabbit (e.g. with pika)
16:41:29 <rakhmerov> but we'll have a pattern how to easily create new ones
16:41:49 <rakhmerov> ddeja: btw, that would allow us to implement that per-task delivery much easier
16:42:05 <rakhmerov> we could just have it in our design in the first place
16:42:05 <ddeja> I guess I can start with implementation, starting later this week, or in worst case scenario - from Monday
16:42:17 <rakhmerov> it's basically just another param in our rpc methods
16:42:24 <ddeja> rakhmerov: yup
16:42:28 <rakhmerov> ddeja: great!
16:42:32 <rakhmerov> ddeja: just one question
16:42:50 <rakhmerov> I remember that it was pretty urgent for you to get that hack in
16:43:32 <rakhmerov> and I wonder if it's critically important to keep it in the code for that month or so while we're working on a good solution?
16:43:38 <rakhmerov> I mean for you
16:43:41 <ddeja> not really
16:43:53 <ddeja> it was more about beeing in release
16:43:58 <rakhmerov> ok, then that's good )
16:43:59 <rakhmerov> yeah
16:44:33 <rakhmerov> ddeja: as an option, I'd suggest to restore that hack (in a different form) in your own fork, if needed
16:44:47 <rakhmerov> if it's badly needed I can even help with it
16:44:57 <rakhmerov> but it'd be better not to have in master
16:44:58 <ddeja> there is no need for it on master branch ;)
16:45:04 <rakhmerov> yes
16:45:11 <ddeja> Mitaka is OK
16:45:16 <rakhmerov> ok
16:45:28 <rakhmerov> I feel a relief )
16:45:48 <ddeja> the only thing is to make sure this feature would be also present in Newton :)
16:46:23 <rakhmerov> ddeja: ok, then ping me when you start working on it, we could first discuss those patches that we have and think over the plan
16:46:40 <rakhmerov> ddeja: it's feasible, I'm confident
16:46:45 <rakhmerov> one way or another
16:46:53 <rakhmerov> we just need to keep working on it
16:47:13 <hparekh__> hi
16:47:14 <rakhmerov> okey, deal
16:47:18 <rakhmerov> hparekh__: hey )
16:47:21 <ddeja> rakhmerov: I'll look at those patches first and I will for sure let you know after
16:47:41 <hparekh__> my status is working on that bp
16:47:55 <hparekh__> filtering one
16:47:58 <rakhmerov> ddeja: yeah, it's not that trivial and I already have some experience with that so please communicate
16:48:12 <hparekh__> will post patch soon
16:48:14 <rakhmerov> hparekh__: ok, just wanted to ask you about your status )
16:48:29 <rakhmerov> hparekh__: I reviewed your patch partially
16:48:40 <rakhmerov> sorry for the delay, will complete tomorrow
16:49:09 <hparekh__> ok no issue
16:49:23 <rakhmerov> hparekh__: and I cherry picked that patch you pointed to
16:49:24 <rakhmerov> https://review.openstack.org/#/c/316961/
16:49:35 <rakhmerov> I hope it'll help
16:49:41 <rakhmerov> to fix stable/liberty
16:49:51 <hparekh__> oh ok :)
16:50:02 <mgershenzon> Do we have a spec to show the Oslo guys?
16:50:39 <rakhmerov> mgershenzon: there is an abandoned spec but honestly I haven't fully read it myself )
16:50:42 <mgershenzon> I'm talking about our implementation of at least once
16:50:45 <rakhmerov> I'll find a patch
16:50:52 <rakhmerov> mgershenzon: yeah, right
16:51:01 <rakhmerov> I'll find it and give a link to you
16:51:13 <rakhmerov> let me do it right away actually...
16:51:16 <rakhmerov> 20 sec
16:51:58 <rakhmerov> here it is: https://review.openstack.org/#/c/256342/
16:52:32 <rakhmerov> as you can see there was a huge huge huge discussion
16:52:39 <rakhmerov> and as a result it got abandoned
16:53:24 <rakhmerov> #topic Open Discussion
16:53:30 <rbrady> https://bugs.launchpad.net/mistral/+bug/1581649
16:53:31 <openstack> Launchpad bug 1581649 in Mistral "Action Execution Response Timeout" [Undecided,New]
16:53:52 <mgershenzon> We can create a spec on mistral
16:54:09 <rbrady> TripleO devs have run into this issue with Mistral.  I saw it at summit and thought it might just be affecting me
16:54:10 <rakhmerov> mgershenzon: yep
16:54:33 <rakhmerov> rbrady: yeah, I remember you told me
16:54:39 <rbrady> but it's essentially blocking our integration efforts atm
16:54:52 <rakhmerov> let me assign it to Newton-1
16:55:29 <rakhmerov> ok, we can check it tomorrow
16:55:38 <ddeja> rbrady: is theere a workflow definition included in the bug?
16:55:40 <rakhmerov> it must be something simple
16:55:54 <rbrady> it's affecting all workflow calls
16:56:05 <ddeja> hm, interesting
16:56:19 <rbrady> often the api isn't responding in time
16:56:45 <ddeja> can it be due to ongoing executions?
16:56:51 <rbrady> because we're getting the same error with the mistral cli, the api directly with curl or javascript calls
16:57:05 <rakhmerov> ok
16:57:41 <rbrady> ddeja: how many calls would I have to make to stop mistral from executing?  at this point it's a handful of calls
16:58:27 <rbrady> ddeja: I've tried restarting the mistral services and it has no apparent effect.  I still cannot get a consistent result
16:59:01 <rakhmerov> rbrady: it's really weird, I've never seen people encounter this..
16:59:20 <rakhmerov> rbrady: make sure to specify as much info as possible about how to reproduce it
16:59:22 <ddeja> rbrady: I don't know. I'm just wondering if this may be result of lack of resources on the physical servers
16:59:24 <rakhmerov> and your environment
16:59:48 <rakhmerov> ddeja: yeah, that's what I thought, just a matter of environment or something
17:00:06 <rakhmerov> but it needs to be looked at more closely..
17:00:06 <rbrady> I can setup a bluejeans session or google hangout and share my screen if anyone wants to inspect the environment
17:00:37 <rakhmerov> rbrady: yeah, I'd be happy too but it's the midnight for me now :)
17:00:47 <rakhmerov> maybe tomorrow
17:00:53 <rakhmerov> ooh, shoot...
17:00:59 <rakhmerov> we need to end the meeting
17:01:10 <rakhmerov> rbrady: we'll take care of this issue
17:01:24 <rakhmerov> have to close the meeting
17:01:29 <rakhmerov> thanks for joining!
17:01:31 <rakhmerov> bye
17:01:36 <hparekh__> thanks
17:01:42 <mgershenzon> Bye
17:01:45 <rakhmerov> #endmeeting