08:00:31 <licanwei> #startmeeting Watcher
08:00:32 <openstack> Meeting started Wed Aug 14 08:00:31 2019 UTC and is due to finish in 60 minutes.  The chair is licanwei. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
08:00:34 <Dantalion> Hello
08:00:35 <openstack> The meeting name has been set to 'watcher'
08:00:44 <licanwei> hi~
08:01:13 <licanwei> Dantalion: hi
08:01:49 <licanwei> #topic Announcements
08:02:56 <licanwei> Shanghai Project Update is coming
08:03:13 <chenke> I will go there.
08:03:34 <Dantalion> I will not be there, is there anything planned for Watcher?
08:03:42 <licanwei> I want to submit Watcher Project Update
08:04:38 <Dantalion> Sounds good, do you need help?
08:05:56 <licanwei> Dantalion: maybe some specs about CERN need your help
08:06:22 <Dantalion> Okaj, If so you can send me email anytime
08:07:20 <chenke> Dantalion If you want to come, there will be free tickets
08:08:52 <licanwei> Dantalion: thanks
08:09:55 <licanwei> #topic Reviews
08:10:25 <licanwei> https://review.opendev.org/#/c/676103/ Don't revert Migrate action needs review
08:10:37 <chenke> +1
08:11:33 <chenke> I agree that we don't need to revert migrate if it has been done successfully.
08:11:45 <licanwei> yes
08:11:54 <Dantalion> I agree in general but I think it can depend on the strategy
08:12:33 <chenke> Yes, I have consider about defferernt stategy.
08:13:49 <chenke> But after thinking about it, for each strategy, after the migration is successful, there is no need to revoke it.
08:15:07 <Dantalion> For now this is fine if we ever need it in the future we can consider different types of migrate actions or passing a parameter to the migrate action
08:15:22 <chenke> Will some strategies be special?
08:16:23 <licanwei> Now no strategy need to revert migration
08:16:52 <licanwei> but it's better if we can set based strategy
08:17:25 <licanwei> maybe need to add a new config option
08:18:24 <licanwei> another way is to add a new action
08:18:26 <chenke> Yes, choose by one param.
08:18:50 <licanwei> move on
08:19:12 <licanwei> https://review.opendev.org/#/c/676110/ Add Python 3 Train unit tests needs review
08:20:08 <Dantalion> +1 for this patch
08:20:13 <chenke> +1
08:20:24 <licanwei> It's Train python3-updates goal
08:20:43 <licanwei> but i don't know if tempest-plugin need to update
08:21:31 <Dantalion> Me neither but I think it makes sense to have all projects of Watcher use the same Python versions with unit tests
08:22:02 <licanwei> ok, agree
08:22:37 <licanwei> https://review.opendev.org/#/c/675871/ node resource consolidation spec needs review
08:22:58 <licanwei> It's new bp spec
08:23:46 <Dantalion> This will take time to review
08:23:59 <licanwei> thanks
08:24:21 <licanwei> https://review.opendev.org/670936 rollback node status needs review
08:25:17 <chenke> I quickly read it and I understood the meaning of this spec. Overall, this is not the same as the previous load balancing and concentrating algorithms.
08:25:44 <chenke> But for the detailed implementation, I have not had time to see it.
08:25:50 <licanwei> #topic  Discussion
08:26:07 <licanwei> Parallelism implementations threadpool vs taskflow
08:26:36 <licanwei> Dantalion: What's the result?
08:26:47 <Dantalion> I have tried both threadpool and taskflow implementations available on https://review.opendev.org/#/c/671556/ and https://review.opendev.org/#/c/671264/
08:27:22 <Dantalion> threadpool is slightly faster but not substantially, however, taskflow consumes much more memory around 480 instead of 135 megabytes for the same scope
08:27:59 <Dantalion> I think taskflow is to complicated supporting many features such as task order, zookeeper remote workers and revert / fallback actions
08:29:04 <Dantalion> With a 16 thread threadpool the performance is between 15 and 21 times faster
08:29:21 <licanwei> https://review.opendev.org/#/c/671264/8/watcher/decision_engine/model/collector/nova.py line 287
08:29:21 <Dantalion> from 316 seconds to just 11 for a 44 node cell for example
08:30:28 <licanwei> you can also set executor parameter
08:30:45 <licanwei> to test diffrent executor type
08:32:38 <Dantalion> That's true I think I remember the default being greenthreads
08:33:41 <licanwei> Do you want to merge these codes?
08:36:09 <Dantalion> Not in these states, I want to write a spec. I prefer the threadpool implementation but I am interested in others opinion as well
08:37:30 <chenke> The effect of the thread pool depends on the time of the IO
08:37:59 <chenke> Python has a GIL will affect the Parallelism time.
08:39:04 <licanwei> ok
08:40:37 <licanwei> no more need to be dicussed
08:41:12 <licanwei> thank you Corne and chenke
08:41:22 <licanwei> I'll end the meeting
08:41:28 <chenke> okay.
08:41:35 <Dantalion> Thank you and until next time
08:41:41 <licanwei> bye
08:41:45 <chenke> See you~
08:41:48 <licanwei> #endmeeting