16:04:14 <akuznetsova> #startmeeting Mistral
16:04:15 <openstack> Meeting started Mon Nov 23 16:04:14 2015 UTC and is due to finish in 60 minutes.  The chair is akuznetsova. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:04:17 <nmakhotkin> hi everyone!
16:04:19 <openstack> The meeting name has been set to 'mistral'
16:04:24 <bhaskarduvvuri> hi
16:04:27 <_gryf> hi
16:04:31 <xylan_kong> hi, guys
16:04:31 <akuznetsova> hi everyone
16:04:31 <^Gal^> hi
16:04:41 <LimorStotland> hi
16:05:20 <akuznetsova> give me a sec, I'll find last meeting minutes
16:05:20 <ddeja> hello
16:05:28 <nmakhotkin> :)
16:05:54 <xylan_kong> seems we have some new faces today
16:06:44 <akuznetsova> yeah, it's cool
16:06:54 <akuznetsova> let's start
16:07:17 <akuznetsova> #action items review
16:07:58 <akuznetsova> #topic
16:08:10 <akuznetsova> #topic action items review
16:08:29 <xylan_kong> :-)
16:08:36 <akuznetsova> 1. rakhmerov: create Mitaka series and milestone in
16:08:36 <akuznetsova> python-mistralclient LP
16:09:06 <akuznetsova> nmakhotkin, do you know, Renat did it or not?
16:09:48 <nmakhotkin> I see mistralclient ver 1.2.0
16:09:57 <xylan_kong> yeah, me too
16:10:14 <akuznetsova> ok, it means that this ai is done
16:10:23 <nmakhotkin> AFAIK client versions differ from version of main project
16:10:41 <akuznetsova> 2. rakhmerov, melisha: design "Mistral HA" and "Multi-region support" during M-1
16:11:10 <nmakhotkin> is melisha with us?
16:11:13 <akuznetsova> I am not sure that I see any docs/etherpads
16:11:29 <LimorStotland> I can update on it : they didn't do it yet
16:11:46 <xylan_kong> someone asked me about this topic today
16:11:50 <akuznetsova> LimorStotland, thanks
16:11:51 <nmakhotkin> so, let's forward action to next meeting
16:11:59 <akuznetsova> yes
16:12:05 <nmakhotkin> #action rakhmerov, melisha: design "Mistral HA" and "Multi-region support" during M-1
16:12:06 <LimorStotland> ok
16:12:16 <akuznetsova> thanks, nmakhotkin
16:12:21 <nmakhotkin> np :)
16:12:45 <akuznetsova> that's all action items from previous meeting
16:12:58 <rakhmerov> hi guys
16:13:09 <akuznetsova> hi rakhmerov
16:13:11 <rakhmerov> I reviewed OpenStack HA meeting log btw
16:13:29 <akuznetsova> #topic Current status (progress, issues, roadblocks, further plans)
16:13:43 <rakhmerov> please keep going...
16:13:45 <rakhmerov> I'll catch up
16:14:14 <nmakhotkin> my status: did a lot of review last week
16:14:16 <^Gal^> I'm on mistraldashboard pagination in execution screens
16:14:41 <LimorStotland> my status: i am working on internal alcatel-lucent issues + mistral reviews
16:14:47 <nmakhotkin> btw, nice to see ceilometer and trove actions in mistral :)
16:15:02 <xylan_kong> - completed the implementation of mistral workflow sharing, https://review.openstack.org/244155 [READY FOR REVIEW]
16:15:02 <xylan_kong> - fixed some bugs found during workflow sharing feature testing. [MERGED]
16:15:02 <xylan_kong> - completed mistral-spec initialization, I think it's a big improvement for Mistral as an official project. [MERGED]
16:15:03 <xylan_kong> - had an experience with Reno, gave a sharing to the whole team via email. [DONE]
16:15:03 <xylan_kong> - review work [ALWAYS]
16:15:28 <xylan_kong> nmakhotkin: yeah, I'll do zaqar integration in next few days
16:15:30 <^Gal^> xylan_kong: nice!
16:15:43 <rakhmerov> my status: reviews, backporting (not finished) and some email exchanges
16:15:43 <nmakhotkin> xylan_kong: awesome!
16:15:44 <akuznetsova> my status: worked on improving our Jenkins ci, split one gate into two separate gates, I am planning to continue this work and make gates voting
16:16:08 <xylan_kong> nmakhotkin: thanks
16:16:13 <LimorStotland> xylan_kong nice !!!
16:16:31 <akuznetsova> xylan_kong, oh, you came prepared )
16:16:50 <xylan_kong> akuznetsova: yes, of course
16:16:58 <akuznetsova> xylan_kong, :-)
16:17:01 <xylan_kong> a habit
16:17:26 <akuznetsova> does anybody have any roadblocks?
16:17:48 <xylan_kong> i have one, but can be discussed in open discussion
16:18:01 <xylan_kong> about how we identify resource
16:18:12 <akuznetsova> xylan_kong, ok, so let's move to open discussion !
16:18:17 <xylan_kong> akuznetsova: ok
16:18:25 <akuznetsova> #topic Open Discussion
16:18:43 <akuznetsova> xylan_kong, please, go ahead
16:18:56 <rakhmerov> I also wanted to discuss status of backporting but not sure if we need to do it now
16:19:03 <xylan_kong> the problem was found during the implementation of workflow sharing feature
16:19:18 <rakhmerov> about a half of bugs that ALU wants got merged into stable/liberty
16:19:19 <xylan_kong> #link https://blueprints.launchpad.net/mistral/+spec/use-workflow-id-in-rest-api
16:19:47 <xylan_kong> I want to talk about the bp/bug, for fixing that, maybe we need to consider re-design our Mistral API
16:19:49 <rakhmerov> some of them can't be merged because we need to repair a sequence of commits in stable/liberty
16:19:58 <rakhmerov> I'm planning to work on it tomorrow first thing
16:21:12 <LimorStotland> xylan_kong : i am not sure i understand... can you please give example or something ?
16:21:26 <xylan_kong> Without uuid, we can hardly distinguish different workflows with same name, especially when we use 'public' workflows or we want to get benifits from workflow sharing feature I have been implementing these days.
16:21:32 <akuznetsova> xylan_kong, what do you mean by re-design ? api v3 ?
16:21:40 <xylan_kong> .akuznetsova maybe
16:22:19 <xylan_kong> since it seems a big change for mistral, especially related to API
16:22:35 <xylan_kong> I have already met some bugs related to that, please see https://bugs.launchpad.net/mistral/+bug/1518655 and https://bugs.launchpad.net/python-mistralclient/+bug/1518276
16:22:35 <openstack> Launchpad bug 1518655 in Mistral "Cron-trigger running failed when workflow's scope is updated to 'private' from 'public'" [Undecided,New]
16:22:36 <openstack> Launchpad bug 1518276 in python-mistralclient "Can not distinguish private workflow and public workflow with the same name" [Medium,Fix committed] - Assigned to Lingxian Kong (kong)
16:22:37 <bhaskarduvvuri> that sounds like a nice idea xylan_kong
16:23:13 <xylan_kong> currently, different tanants can have workflows with same name
16:23:20 <rakhmerov> xylan_kong: yes, I fully understand you
16:23:31 <xylan_kong> rakhmerov :-)
16:23:32 <rakhmerov> I think it's a subject of API v3 design
16:23:43 <akuznetsova> I don't remember why we decided to use names instead on uuid
16:23:52 <rakhmerov> but also agree that we need to fix it in a backwards compatible way in v2
16:24:01 <xylan_kong> rakhmerov: yes, absolutely
16:24:13 <xylan_kong> we can both support name and uuid
16:24:17 <bhaskarduvvuri> or maybe have v2.1
16:24:17 <nmakhotkin> yes, we can use name or uuid
16:24:30 <bhaskarduvvuri> for backport
16:24:38 <rakhmerov> akuznetsova: because we wanted save API calls when we needed to do something with workflows
16:24:57 <rakhmerov> so that you don't need to lookup a workflow by name first in order to find its id
16:25:05 <rakhmerov> and then, for example, to execute it
16:25:32 <rakhmerov> assuming that names are unique you can run them immediately without having to know their ids
16:25:37 <rakhmerov> that was the idea
16:26:00 <akuznetsova> oh, I see
16:26:07 <nmakhotkin> names are human-readable :)
16:26:16 <nmakhotkin> and rememberable
16:26:16 <rakhmerov> nmakhotkin: indeed
16:26:34 <akuznetsova> but now in case of public wfs it became a problem
16:26:40 <xylan_kong> nmakhotkin: but it is also changable
16:27:00 <LimorStotland> xylan long do you have any design for it?
16:27:01 <xylan_kong> and it's part of ReST API
16:27:07 <nmakhotkin> xylan_kong: really?
16:27:12 <rakhmerov> actually, it can be solved also if we print out their "scope" field and "name + scope" is a unique constraint
16:27:16 <xylan_kong> LimorStotland: yes, i think it's not a hard thing todo
16:27:25 <nmakhotkin> I don't remember there is a possibility to change a name of workflow
16:27:47 <rakhmerov> xylan_kong: yes, I'm actually not sure we can change it
16:27:50 <rakhmerov> let me see..
16:28:03 <xylan_kong> we can't change the name
16:28:42 <rakhmerov> we can't
16:28:45 <rakhmerov> that's right
16:29:04 <rakhmerov> because /workflows/name PUT accepts plain/text only
16:29:20 <rakhmerov> which is YAML
16:29:27 <LimorStotland> so can please publish the design... its a bit hard (at least for me to see the problem, for me it's sound like we only need to add workflow-id to the get ....)
16:29:52 <xylan_kong> rakhmerov: that works. but as a user, maybe I will be confused that i can't design a workflow with a name i wanted if you add the constraint
16:30:08 <rakhmerov> yes
16:30:14 <rakhmerov> well, after all I think it may have been a mistake to identify workflows by names
16:30:16 <xylan_kong> LimorStotland: yes, it deserves a spec for review
16:30:29 <LimorStotland> xylan_kong  thanks
16:30:38 <xylan_kong> so, what's you suggestion about when we can start to do that?
16:30:49 <xylan_kong> in M? or N?
16:30:56 <rakhmerov> on one hand, it's a little bit more convenient in certain cases but on the other hands ids are much more flexible
16:31:06 <rakhmerov> xylan_kong: doing what?
16:31:15 <xylan_kong> rakhmerov: use uuid
16:31:26 <xylan_kong> or adding uuid support
16:31:30 <xylan_kong> in rest api
16:31:38 <rakhmerov> xylan_kong: whenever API v3 comes through, in my opinion
16:31:55 <rakhmerov> which I hope will happen in M
16:32:07 <xylan_kong> since i have to solve the problem in the resource sharing feature design
16:32:16 <rakhmerov> well, it's just my thought, I'm not sure about this suggestion
16:32:29 <akuznetsova> probably will be more changes in api v3, so I guess that we need to design it first
16:32:30 <rakhmerov> it really should be designed first and reviewed like LimorStotland said
16:32:48 <xylan_kong> rakhmerov: yes, agreed
16:32:51 <rakhmerov> akuznetsova: yes
16:32:54 <rakhmerov> xylan_kong: yes
16:33:23 <rakhmerov> xylan_kong: so you actually sent some patches with showing id, right?
16:33:23 <xylan_kong> so maybe i can have the first commit in mistral-spec :-)
16:33:30 <xylan_kong> rakhmerov: yes
16:33:32 <rakhmerov> I voted +2 for them
16:33:43 <xylan_kong> rakhmerov: but that's just some workaround for nwo
16:33:56 <xylan_kong> can not solve the whole problem totally
16:34:05 <rakhmerov> xylan_kong: ok, but do we need anything else for now?
16:34:11 <rakhmerov> I know that's a workaround
16:34:53 <xylan_kong> the problems still exist
16:35:13 <akuznetsova> yes
16:35:18 <xylan_kong> and we have to wait until we support uuid in rest api
16:35:36 <akuznetsova> let's create etherpad with new API design
16:35:58 <xylan_kong> akuznetsova: no need etherpad, just specification in mistral-spec
16:36:00 <rakhmerov> xylan_kong: I mean is it a roadblock for resource sharing implementation now?
16:36:08 <akuznetsova> xylan_kong, oh, yes
16:36:58 <xylan_kong> rakhmerov: yes, when i share a workflow to a tenant, he can't see the workflow details if he has a workflow with the same name
16:37:24 <rakhmerov> why?
16:37:26 <xylan_kong> in implementation, we don't know which workflow detail the tenant want to show
16:37:33 <rakhmerov> give me a sec.. thinking..
16:37:39 <xylan_kong> with the only workflow name
16:38:00 <xylan_kong> given we have 2 tenant2, A and B.
16:38:05 <rakhmerov> yes
16:38:09 <xylan_kong> A creats a workflow, and shars with B
16:38:14 <rakhmerov> what do you do to share a workflow?
16:38:15 <rakhmerov> yes
16:38:30 <xylan_kong> blablabla....
16:38:43 <xylan_kong> rakhmerov: you already know , right?
16:38:47 <rakhmerov> what's the sharing algorithm?
16:38:52 <rakhmerov> no I don't
16:39:08 <rakhmerov> more precisely, I remember what we discussed at the summit but didn't look at your impl yet..
16:39:17 <xylan_kong> rakhmerov: https://blueprints.launchpad.net/mistral/+spec/mistral-resource-sharing
16:39:45 <xylan_kong> yes, you can take a look at the code first, including the sharing api design
16:40:00 <xylan_kong> especially the test cases
16:40:16 <LimorStotland> i think that what xylan_kong  is saying is that it can create a situation where a user can run 2 different wf with the same name
16:40:28 <rakhmerov> xylan_kong: yes, I saw the BP, just please explain your algo in 2 words
16:40:36 <rakhmerov> you create a permissions table?
16:40:51 <xylan_kong> i added a new table to record the sharing relationship
16:41:12 <xylan_kong> the table can be used for workflow, workbook, and action
16:41:27 <xylan_kong> and any other resources in future that can be sharable
16:41:43 <rakhmerov> ooh, ok. I see now the problem
16:41:53 <rakhmerov> correct me if I'm wrong..
16:42:01 <xylan_kong> ok
16:42:55 <rakhmerov> so even if we shared a workflow from A to B and users from B can see the WF in listings they still can't use it because they need to use unique names
16:43:10 <xylan_kong> rakhmerov: yes, exactly
16:43:20 <rakhmerov> that are not unique anymore because A and B can contain wfs with same names
16:43:25 <rakhmerov> yep...
16:43:29 <rakhmerov> f...k
16:43:39 <xylan_kong> calm down :-)
16:43:40 <rakhmerov> it's a serious issue :)
16:44:18 <rakhmerov> it definitely need to change but I just wonder if we can workaround it for now somehow
16:44:41 <xylan_kong> rakhmerov: yes, we can support name and uuid at the same time
16:44:50 <xylan_kong> name is just for backward compatible
16:45:00 <xylan_kong> and we recommend users to use uuid in future
16:45:05 <xylan_kong> maybe in M release
16:45:16 <rakhmerov> you mean we can change all that methods taking names to also support ids?
16:45:24 <xylan_kong> rakhmerov: yes
16:45:27 <rakhmerov> yeah, that should work
16:45:36 <akuznetsova> I guess that it is good variant
16:45:44 <rakhmerov> I actually I know some technologies that use that approach
16:45:52 <rakhmerov> for example, StackStorm does that
16:46:11 <rakhmerov> xylan_kong: so that is what you're going to work on?
16:46:22 <rakhmerov> create a spec for it?
16:46:26 <xylan_kong> and in release notes, we must be honest to tell users what's the restriction about using name
16:46:33 <xylan_kong> rakhmerov: sure, i can go with that
16:46:44 <rakhmerov> xylan_kong: yes, please do
16:46:50 <rakhmerov> because anyway it's related to your task
16:46:56 <xylan_kong> rakhmerov: yes
16:47:00 <rakhmerov> ok, deal
16:47:09 <xylan_kong> i can start with a specification
16:47:56 <akuznetsova> yes, please
16:48:16 <rakhmerov> ok
16:48:17 <xylan_kong> will call for review when i finish
16:48:21 <xylan_kong> :-)
16:48:29 <akuznetsova> anything else to discuss ?
16:48:47 <akuznetsova> rakhmerov, ?
16:48:50 <rakhmerov> I'm done
16:48:54 <ddeja> akuznetsova, rakhmerov: I have a short question, do you have any progress on https://bugs.launchpad.net/mistral/+bug/1513456 Today I was able to dummy fix it (using sleep), but have idea for better solution. I'm wondering if I can reassigne it to myself.
16:48:54 <openstack> Launchpad bug 1513456 in Mistral "task stuck in RUNNING state when all action executions are finished" [Critical,Triaged] - Assigned to Renat Akhmerov (rakhmerov)
16:49:04 <rakhmerov> and I would even say "I'm completely finished" :)
16:49:13 <nmakhotkin> AFAIK we don't
16:49:25 <nmakhotkin> rakhmerov?
16:49:25 <xylan_kong> rakhmerov :-)
16:49:26 <akuznetsova> yes, we don't fix it
16:49:28 <rakhmerov> ddeja: please do
16:49:32 <rakhmerov> feel free
16:49:45 <ddeja> ok, will do so :)
16:49:54 <rakhmerov> ddeja: just one question
16:50:17 <rakhmerov> were you just able to reproduce it or you were able to understand the cause of the issue?
16:50:30 <ddeja> both
16:50:34 <rakhmerov> I'd be rather interested to know where it comes from
16:50:35 <_gryf> rakhmerov, we were working on it last week
16:50:41 <rakhmerov> ddeja: ooh, awesome!
16:51:07 <rakhmerov> ddeja: can you please explain briefly what this issue is caused by?
16:51:16 <ddeja> It's a race condition in default engine
16:51:24 <rakhmerov> ok, where exactly?
16:51:30 <ddeja> in method 'on_action_complete'
16:51:33 <_gryf> rakhmerov, ddeja, it could be good, to make a comment under the bug on lp
16:51:44 <rakhmerov> yes, definitely
16:51:59 <rakhmerov> ok, let me put it this way: I'd like to know all the guts of this issue
16:52:01 <rakhmerov> :))
16:52:12 <ddeja> when transaction is started, it sometimes (in my case - always) didn't finish
16:52:27 <ddeja> before transaction is started in second thread
16:52:27 <nmakhotkin> transaction hangs??
16:52:46 <ddeja> so, both transactions didn't see that the other action has ended
16:53:10 <nmakhotkin> yep, I also investigated that a bit
16:53:47 <rakhmerov> hm..
16:53:48 <rakhmerov> weird
16:53:55 <ddeja> it can be fixed by not checking in code if every action has ended, but leave it to database
16:53:58 <noa_koffman> I saw the same thing...
16:54:01 <rakhmerov> why didn't that TX finish?
16:54:24 <ddeja> it did finished
16:54:31 <rakhmerov> ddeja: as far as checks it makes sense, yes
16:54:32 <ddeja> but before it happend, second started
16:54:44 <rakhmerov> oooh! I think I got it
16:55:03 <rakhmerov> it's because an ORM model in memory differs from DB state..
16:55:07 <rakhmerov> right?
16:55:09 <ddeja> yes
16:55:14 <rakhmerov> gotcha
16:55:17 <akuznetsova> folks, we have only 3 mins left
16:55:23 <ddeja> OK :)
16:55:25 <xylan_kong> ddeja: good job! welcome your contribution btw
16:55:29 <rakhmerov> then the fix should be pretty simple
16:55:43 <rakhmerov> ddeja: yeah, awesome job! Thanks
16:55:55 <akuznetsova> let's wrap our meeting
16:55:59 <rakhmerov> reassign it to yourself and ask any help if needed
16:56:00 <ddeja> xylan_kong, rakhmerov, thanks :-)
16:56:07 <ddeja> OK
16:56:11 <akuznetsova> we can continue in our usual channel
16:56:12 <rakhmerov> it's an extremely important issue
16:56:29 <rakhmerov> ok, now I'm completely finished
16:56:35 <rakhmerov> ok to end the meeting
16:56:38 <nmakhotkin> bye!
16:56:41 <akuznetsova> bye
16:56:42 <xylan_kong> ok, productive meeting
16:56:43 <rakhmerov> let'd meet in our channel
16:56:44 <rakhmerov> bye
16:56:44 <xylan_kong> bye, guys
16:56:49 <_gryf> bye
16:56:53 <noa_koffman> bye
16:56:54 <ddeja> bye
16:56:57 <akuznetsova> #endmeeting