08:00:06 <d0ugal> #startmeeting mistral
08:00:07 <openstack> Meeting started Fri Apr 20 08:00:06 2018 UTC and is due to finish in 60 minutes.  The chair is d0ugal. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
08:00:11 <openstack> The meeting name has been set to 'mistral'
08:00:30 <d0ugal> It is the Friday office hour!
08:00:32 <d0ugal> yay!
08:01:06 <d0ugal> d0ugal,rakhmerov,apetrich,bobh, mcdoker18: ping :)
08:01:10 <d0ugal> https://etherpad.openstack.org/p/mistral-office-hours
08:01:18 <apetrich> here we are
08:01:24 <rakhmerov> d0ugal: yep, here
08:02:09 <d0ugal> So, the only interesting news I have is that Mistral R-1 has been released!
08:02:23 <rakhmerov> d0ugal: cool )
08:03:01 <d0ugal> mistral-lib and python-mistralclient releases will happen soon, I made a mistake in the original patches.
08:03:51 <d0ugal> With R-1 out, I am going to spend this hour moving all the R-1 bugs to R-2
08:03:58 <d0ugal> and if I get time after that I am going to try and do some triage
08:03:59 <apetrich> nice
08:04:23 <rakhmerov> yep
08:05:30 <d0ugal> I made a copy/paste fail - I tried to release mistral-lib with a mistralclient sha and a mistralclient release with a mistral-extra sha :/
08:05:42 <d0ugal> Thankfully the tools block me.
08:06:10 <d0ugal> If anyone has anything to discuss, do let me know - there is time for that.
08:06:39 <d0ugal> My goal with the office hours is to take care of project/PTL duties so that I am here and active, but if somebody else is here and wants to chat then that is more important.
08:07:24 <rakhmerov> d0ugal: I don't have much, you've probably seen my memory/performance improvements that have already been merged. That's all I have. Plus I'm investigating other issues now that we're seeing in our env
08:07:39 <d0ugal> rakhmerov: other performance issues?
08:07:44 <rakhmerov> well, yes
08:07:51 <rakhmerov> I can tell briefly
08:08:01 <d0ugal> Sure, if you think they would be interesting
08:08:34 <rakhmerov> one thing is that we do deletion of resources (wf execs and defs) and for some reason get huge memory footprint during that
08:08:53 <rakhmerov> I have no idea yet why this happens but it's consistently reproducible
08:09:21 <rakhmerov> the other thing is that we run a workflow with some subworkflows, so totally we have 4-5 wf executions
08:09:43 <d0ugal> rakhmerov: when deleting an object, do we load the full model from the database?
08:09:45 <rakhmerov> BUT the internal caches that cache WF specifications by executions get full, all 100 entries
08:09:51 <rakhmerov> which is I'm sure a bug
08:10:11 <rakhmerov> d0ugal: no, we shouldn't. We just do a delete query
08:10:18 <rakhmerov> I'll check that anyway
08:10:31 <d0ugal> I guess we don't, but I can't think why else memory usage would be high
08:10:43 <rakhmerov> d0ugal: yeah, me too
08:11:12 <d0ugal> I'm very interested to hear what you find.
08:11:24 <rakhmerov> yep
08:11:37 <rakhmerov> I need some more time though, hopefully today I'll have more info
08:12:02 <rakhmerov> because I can't reproduce those issues locally on my mac
08:12:09 <rakhmerov> only in our distributed env in the lab
08:12:40 <rakhmerov> that's all I have I guess )
08:12:44 <rakhmerov> keep working..
08:12:47 <d0ugal> Thanks for sharing :)
08:13:25 <d0ugal> So in R-1 we fixed 13 bugs, but we have 8 in progress and 25 more triaged
08:13:26 <rakhmerov> but https://review.openstack.org/#/c/562554 makes a really big difference memory wise )
08:13:35 <rakhmerov> d0ugal: ok
08:13:49 <rakhmerov> let me take a look at LP
08:14:02 <d0ugal> R-2 already has 15 bugs assigned. We either need to reduce the bug for R-2 or solve lots of things :)
08:14:12 <d0ugal> s/solve/fix/
08:14:40 <rakhmerov> :)
08:14:48 <rakhmerov> so looking at R-1..
08:14:51 <rakhmerov> https://bugs.launchpad.net/mistral/+bug/1757966
08:14:52 <openstack> Launchpad bug 1757966 in Mistral "Mistral server consumes more memory over time" [Critical,In progress] - Assigned to Renat Akhmerov (rakhmerov)
08:15:03 <rakhmerov> my latest patches were addressing it
08:15:17 <rakhmerov> but I'd prefer not to close it yet and do some more investigation
08:15:22 <d0ugal> Sure
08:15:25 <rakhmerov> so let's move it to R-2
08:15:27 <rakhmerov> ok for you?
08:15:28 <rakhmerov> ok
08:16:01 <rakhmerov> https://bugs.launchpad.net/mistral/+bug/1720797
08:16:02 <openstack> Launchpad bug 1720797 in Mistral "Migrate Mistral CI jobs to new Zuul v3" [High,In progress] - Assigned to Brad P. Crochet (brad-9)
08:16:13 <rakhmerov> I thought this was done, no? :)
08:16:15 <d0ugal> There has been some recent activity there
08:16:24 <rakhmerov> ok
08:16:29 <d0ugal> I thought... let me check.
08:16:46 <d0ugal> https://review.openstack.org/#/c/509184/
08:16:56 <d0ugal> so it seems that mistralclient isn't finished.
08:17:18 <rakhmerov> ooh, yeah
08:17:18 <rakhmerov> ok
08:17:35 <rakhmerov> d0ugal: I'll step out for 5 mins.. Sorry.
08:17:39 <d0ugal> rakhmerov: np
08:23:17 <rakhmerov> i'm here
08:24:05 <d0ugal> apetrich: Did you ever hear more from Winson about documentation for the execution events?
08:24:29 <apetrich> d0ugal, not really
08:24:41 <d0ugal> apetrich: oops
08:24:46 <d0ugal> rakhmerov: ^ that question was for you?
08:24:47 <rakhmerov> nope
08:24:49 <d0ugal> :)
08:24:58 <rakhmerov> i can ping him
08:25:01 <d0ugal> Thanks
08:25:17 <d0ugal> Even if he could provide something rough we can then polish that would be useful
08:25:25 <rakhmerov> yes
08:27:42 <rakhmerov> sent an email to him..
08:27:47 <d0ugal> Thanks!
08:29:13 <rakhmerov> yeah, a number of bugs for R-2 is impressive )
08:29:24 <d0ugal> Yeah :/
08:29:33 <rakhmerov> https://bugs.launchpad.net/mistral/+bug/1664030 is marked as Fixed
08:29:34 <openstack> Launchpad bug 1664030 in Mistral "docker image build crashing" [High,Fix released] - Assigned to Brad P. Crochet (brad-9)
08:29:34 <d0ugal> At some point I am going to go through that list and remove some
08:29:39 <rakhmerov> why is it in R-2?
08:30:25 <d0ugal> rakhmerov: oh, my mistake.
08:30:41 <d0ugal> Moved it back
08:30:50 <d0ugal> I didn't realise thrash|g0ne had opened a new bug.
08:31:29 <d0ugal> https://bugs.launchpad.net/mistral/+bug/1765421
08:31:30 <openstack> Launchpad bug 1765421 in Mistral "docker image build job not working" [Undecided,In progress] - Assigned to Brad P. Crochet (brad-9)
08:31:37 <d0ugal> That is the new one, I'll mark that for R-2
08:31:43 <rakhmerov> ok
08:32:24 <d0ugal> Okay, I have moved everything that wasn't complete to R-2
08:32:38 <d0ugal> I am going to send an email to the list asking people to check the bugs and blueprints that they care about
08:32:45 <d0ugal> and make sure they are all correctly assigned
08:33:39 <rakhmerov> ok
08:34:47 <rakhmerov> I can confirm that I should be able to complete those that are assigned to me
08:34:53 <d0ugal> Great, thanks
08:34:57 <d0ugal> rakhmerov: the blueprints also? :)
08:35:07 <rakhmerov> and their statues are up to date
08:35:12 <rakhmerov> on BPs, let me check..
08:35:54 <rakhmerov> https://blueprints.launchpad.net/mistral/+spec/mistral-async-workflows probably won't be in my focus
08:36:00 <rakhmerov> I'd move it to R-3
08:36:09 <d0ugal> Sure
08:36:12 <rakhmerov> I want to get it done in Rocky, but not in R-2
08:36:19 <rakhmerov> I'll be working on Scheduler
08:36:26 <d0ugal> Great
08:36:27 <rakhmerov> (not even sure if we have a ticket for that)
08:36:35 <rakhmerov> yes, as we talked at the PTG
08:36:42 <rakhmerov> and some docs that we also discussed
08:36:55 <d0ugal> Good point. We should open a bug/blueprint - whichever you think is most sensible.
08:37:19 <rakhmerov> ok, I'll do
08:37:41 <rakhmerov> #action rakhmerov: open a ticket for the Scheduler refactoring work
08:37:49 <rakhmerov> https://blueprints.launchpad.net/mistral/+spec/mistral-readonly-db-transactions
08:38:10 <rakhmerov> this one btw is a low hanging fruit that could be given to someone who's getting up to speed
08:38:30 <rakhmerov> fairly easy but also internally (code wise) useful
08:38:55 <d0ugal> rakhmerov: https://review.openstack.org/#/c/529513/
08:39:02 <d0ugal> it is completed alreay? :)
08:39:18 <d0ugal> Or is there something else we need to do?
08:39:21 <rakhmerov> Hah!
08:39:23 <d0ugal> Looks like apetrich done it in Queens!
08:39:24 <rakhmerov> looking...
08:39:35 <rakhmerov> I even approved it :))
08:39:40 <d0ugal> haha
08:39:43 <apetrich> lol
08:39:50 <d0ugal> We are good at this.
08:39:54 <d0ugal> :P
08:39:56 <rakhmerov> ok, but then the status is not up to date :) (who to blame? :)) )
08:40:09 <d0ugal> everything in this meeting :-D
08:40:16 <d0ugal> everyone* /me is tired
08:40:27 <apetrich> probably me. I must have forgotten to add the blueprint for the automation to pick it up
08:40:37 <rakhmerov> ok, np )
08:40:43 <d0ugal> apetrich: blueprints are not automatically closed AFAIK
08:40:58 <apetrich> really? I assumed they were
08:41:01 <rakhmerov> let's mark it Implemented though? I don't we we can move it back to a closed milestone
08:41:07 <d0ugal> and I guess when you did it you didn't have permission on launchpad
08:41:13 <d0ugal> I'll update it
08:41:19 <apetrich> oh that's true
08:41:44 <d0ugal> I'll just have to leave it as rocky-1
08:41:54 <d0ugal> apetrich: so now you can claim credit for it in queens and rocky.
08:41:59 <rakhmerov> ok
08:42:01 <rakhmerov> :)
08:42:06 <d0ugal> brb, 2 mins
08:42:44 <rakhmerov> d0ugal: I have reasonable doubts that someone will get https://blueprints.launchpad.net/mistral/+spec/mistral-secure-sensitive-data done )
08:48:16 <d0ugal> back - sorry about that
08:48:24 <d0ugal> rakhmerov: Indeed, I agree.
08:48:34 <rakhmerov> )
08:49:43 <d0ugal> So we actually fixed 22 bugs in R-1, I found a number that had merged but didn't have a milestone
08:50:01 <rakhmerov> ooh
08:50:27 <rakhmerov> can we assign a milestone to them now?
08:50:41 <d0ugal> Yes, I have done that
08:50:51 <d0ugal> https://launchpad.net/mistral/+milestone/rocky-1
08:50:56 <d0ugal> The bug list is longer!
08:51:10 <rakhmerov> ok
08:51:12 <rakhmerov> awesome
08:51:31 <rakhmerov> I guess we need to remind someone to set a milestone :)
08:51:40 <d0ugal> Indeed!
08:51:40 <rakhmerov> Hardik probably?
08:51:48 <d0ugal> A number of people :)
08:51:50 <rakhmerov> ok
08:52:00 <rakhmerov> I'll tell Hardik (my colleague)
08:52:02 <d0ugal> but I am not sure that everyone can do that?
08:52:07 <d0ugal> bobh had missed a few
08:52:31 <rakhmerov> I thought for bugs it should be possible, hm.. but may be not
08:52:44 <d0ugal> This one for example didn't have a milestone: https://bugs.launchpad.net/mistral/+bug/1761246
08:52:45 <openstack> Launchpad bug 1761246 in Mistral "cut_list() needs to work the same as cut_dict()" [Low,Fix released] - Assigned to Bob Haddleton (bob-haddleton)
08:53:20 <rakhmerov> ok
08:53:53 <rakhmerov> d0ugal: if only a limited circle of people can do assignments maybe we just need to check it on review
08:54:38 <d0ugal> Yup, we should look into that.
08:54:53 <d0ugal> https://bugs.launchpad.net/mistral/+bug/1765349
08:54:54 <openstack> Launchpad bug 1765349 in Mistral "doc: missing minor version release notes" [Undecided,New]
08:55:11 <d0ugal> We have had a request to go and write historical release notes :/
08:55:52 <d0ugal> We have been getting better are release notes slowly, but I don't think it is realistic to write and backport release notes for previous cycles.
08:56:26 <rakhmerov> d0ugal: yes, Hardik confirmed that he can't assign a milestone
08:56:50 <rakhmerov> d0ugal: ooh gosh, no!
08:56:53 <d0ugal> rakhmerov: I'll add milestone reviewing to the regular lost of tasks on the etherpad.
08:57:02 <rakhmerov> I'll eat an entire milestone probably :)
08:57:06 <d0ugal> lol
08:57:20 <rakhmerov> I'll = it'll
08:58:45 <pgaxatte> rakhmerov: I can't imagine what you take for breakfast :)
08:59:00 <rakhmerov> :)
08:59:20 <rakhmerov> pgaxatte: reviews mostly )
08:59:49 <hardikjasani> Haha
09:00:37 <d0ugal> lol
09:00:47 <d0ugal> Okay, we are our of time! That was a good R-1 cleanup
09:00:51 <d0ugal> Thanks for the help
09:01:15 <d0ugal> Next week I'll start looking at the R-2 goals and making sure they seem realistic. Hopefully I can reduce the work we have planned to someting almost possible.
09:01:18 <d0ugal> #endmeeting