| *** thorst has joined #openstack-watcher | 00:12 | |
| *** thorst has quit IRC | 00:13 | |
| *** thorst has joined #openstack-watcher | 00:43 | |
| *** hidekazu has joined #openstack-watcher | 00:47 | |
| *** thorst has quit IRC | 01:02 | |
| *** thorst has joined #openstack-watcher | 01:11 | |
| *** thorst has quit IRC | 01:11 | |
| *** sanfern has quit IRC | 01:51 | |
| *** zhurong has joined #openstack-watcher | 01:53 | |
| *** karthikpr has joined #openstack-watcher | 02:04 | |
| *** thorst has joined #openstack-watcher | 02:12 | |
| *** thorst has quit IRC | 02:17 | |
| *** karthikpr has quit IRC | 03:00 | |
| *** karthikpr has joined #openstack-watcher | 03:04 | |
| *** karthikpr has quit IRC | 03:08 | |
| *** thorst has joined #openstack-watcher | 03:13 | |
| *** Tamayo has quit IRC | 03:32 | |
| *** thorst has quit IRC | 03:33 | |
| *** sanfern has joined #openstack-watcher | 03:40 | |
| *** zhurong has quit IRC | 03:57 | |
| openstackgerrit | licanwei proposed openstack/watcher master: exception when running 'watcher audit start XXX' https://review.openstack.org/450555 | 03:58 |
|---|---|---|
| *** karthikpr has joined #openstack-watcher | 04:05 | |
| *** hidekazu has quit IRC | 04:09 | |
| *** karthikpr has quit IRC | 04:10 | |
| *** hidekazu has joined #openstack-watcher | 04:13 | |
| *** thorst has joined #openstack-watcher | 04:30 | |
| *** thorst has quit IRC | 04:34 | |
| *** adisky_ has joined #openstack-watcher | 04:42 | |
| openstackgerrit | Hidekazu Nakamura proposed openstack/watcher-specs master: Add spec for Cinder model integration https://review.openstack.org/450173 | 04:43 |
| *** zhurong has joined #openstack-watcher | 04:55 | |
| *** karthikpr has joined #openstack-watcher | 04:57 | |
| *** karthikpr has quit IRC | 06:24 | |
| *** thorst has joined #openstack-watcher | 06:31 | |
| *** thorst has quit IRC | 06:35 | |
| *** vincentfrancoise has joined #openstack-watcher | 07:22 | |
| openstackgerrit | Jeremy Zhang proposed openstack/watcher master: Use HostAddressOpt for opts that accept IP and hostnames https://review.openstack.org/450596 | 07:22 |
| *** danpawlik has joined #openstack-watcher | 07:22 | |
| *** alexchadin has joined #openstack-watcher | 07:29 | |
| alexchadin | vincentfrancoise: hi | 07:29 |
| vincentfrancoise | alexchadin: morning | 07:29 |
| alexchadin | vincentfrancoise: how are you? | 07:29 |
| vincentfrancoise | alexchadin: great and you? | 07:30 |
| alexchadin | vincentfrancoise: I'm ok :) | 07:30 |
| alexchadin | vincentfrancoise: I'd like to discuss background-jobs-ha with you | 07:30 |
| vincentfrancoise | alexchadin: ok (I have 15mn before my daily scrum) | 07:30 |
| alexchadin | vincentfrancoise: https://review.openstack.org/#/c/449659/1/watcher/db/sqlalchemy/job_store.py line 72 | 07:31 |
| vincentfrancoise | alexchadin: what about the pickling? | 07:31 |
| alexchadin | vincentfrancoise: job_state contains 'func' key that is only reference to the method of class | 07:32 |
| alexchadin | vincentfrancoise: so when we add our jobs to db, apscheduler saves only link, not object | 07:32 |
| alexchadin | vincentfrancoise: it means that when we get job from db, we get the following error: TypeError: unbound method launch_audits_periodically() must be called with ContinuousAuditHandler instance as first argument (got nothing instead) | 07:33 |
| alexchadin | vincentfrancoise: fair enough. Cause ContinuousAuditHandler object is not initialised | 07:33 |
| alexchadin | vincentfrancoise: I propose to make ContinuousAuditHandler like singleton | 07:34 |
| vincentfrancoise | alexchadin: hum let me fully understand the problem | 07:34 |
| vincentfrancoise | alexchadin: I kind of get what you mean but I need to read the code to make more sense out of what you told me just now :p | 07:35 |
| alexchadin | vincentfrancoise: we save only link like watcher.decision_engine.audit.continuous:ContinuousAuditHandler.launch_audits_periodically | 07:35 |
| alexchadin | vincentfrancoise: ok ok | 07:35 |
| vincentfrancoise | alexchadin: ok so why is it not an object when it gets serialized? | 07:41 |
| vincentfrancoise | alexchadin: the way I understand is that what seems to be serialized is the actual class instead of the instance | 07:41 |
| alexchadin | apscheduler don't save instance or class, only a link to them | 07:42 |
| alexchadin | vincentfrancoise: here is launch of function after getting it from db: https://github.com/agronholm/apscheduler/blob/master/apscheduler/executors/base.py#L125 | 07:43 |
| vincentfrancoise | alexchadin: oh ok so can't we re-initialize them when we unpickle them? | 07:43 |
| alexchadin | vincentfrancoise: as you can see, it tries to launch function without init | 07:43 |
| alexchadin | vincentfrancoise: I thought about it, but IMHO it isn't right way | 07:44 |
| alexchadin | vincentfrancoise: We will get a lot of ContinuousAuditHandler instances | 07:44 |
| alexchadin | vincentfrancoise: IMHO, ContinuousAuditHandler should be init once | 07:44 |
| alexchadin | vincentfrancoise: I need to go so let's meet later | 07:50 |
| *** alexchadin has quit IRC | 07:51 | |
| openstackgerrit | Santhosh Fernandes proposed openstack/watcher master: Add gnocchi support in workload_balance strategy https://review.openstack.org/450615 | 08:05 |
| *** acabot_ has joined #openstack-watcher | 08:06 | |
| *** acabot has quit IRC | 08:11 | |
| *** acabot_ has quit IRC | 08:11 | |
| *** acabot has joined #openstack-watcher | 08:11 | |
| *** acabot has quit IRC | 08:13 | |
| *** pksingh has joined #openstack-watcher | 08:20 | |
| *** acabot has joined #openstack-watcher | 08:30 | |
| *** acabot has left #openstack-watcher | 08:30 | |
| *** acabot has joined #openstack-watcher | 08:30 | |
| *** thorst has joined #openstack-watcher | 08:33 | |
| *** alexchadin has joined #openstack-watcher | 08:36 | |
| alexchadin | vincentfrancoise: ping | 08:36 |
| vincentfrancoise | alexchadin: pong | 08:36 |
| vincentfrancoise | alexchadin: sorry I left for my meeting and forgot to tell you | 08:37 |
| alexchadin | vincentfrancoise: no worries | 08:37 |
| vincentfrancoise | alexchadin: ok so if I resume from where we left | 08:40 |
| vincentfrancoise | alexchadin: I agree that creating many instances isn't a good option | 08:41 |
| alexchadin | vincentfrancoise: DE instantiate ContinuousAuditHandler only once for now | 08:42 |
| vincentfrancoise | alexchadin: since they would all instantiate a self.scheduler | 08:42 |
| alexchadin | vincentfrancoise: +1 | 08:42 |
| vincentfrancoise | alexchadin: so IMHO, we either make the ContinuousAuditHandler a singleton | 08:43 |
| vincentfrancoise | alexchadin: or we make the BackgroundScheduler a singleton | 08:43 |
| vincentfrancoise | alexchadin: then the other problem I saw | 08:45 |
| vincentfrancoise | alexchadin: the "from apscheduler.schedulers import background" import in continuous.py should not be done like this | 08:45 |
| alexchadin | vincentfrancoise: I vote for the first option since rewriting BackgroundScheduler may call some collisions | 08:46 |
| vincentfrancoise | alexchadin: all background schedulers in Watcher should instead inherit from https://github.com/openstack/watcher/blob/master/watcher/common/scheduling.py#L26 | 08:46 |
| vincentfrancoise | alexchadin: not rewriting | 08:46 |
| vincentfrancoise | alexchadin: IMHO, they should all rely on DecisionEngineSchedulingService https://github.com/openstack/watcher/blob/master/watcher/decision_engine/scheduling.py#L35 | 08:48 |
| vincentfrancoise | alexchadin: and then DecisionEngineSchedulingService should actually be a singleton | 08:48 |
| alexchadin | vincentfrancoise: you suggest to inherit DecisionEngineSchedulingService? | 08:49 |
| openstackgerrit | Hidekazu Nakamura proposed openstack/watcher master: [WIP] Cinder model integration https://review.openstack.org/450642 | 08:52 |
| vincentfrancoise | alexchadin: this https://github.com/openstack/watcher/blob/master/watcher/decision_engine/audit/continuous.py#L46 | 08:52 |
| *** thorst has quit IRC | 08:52 | |
| vincentfrancoise | alexchadin: should be a DecisionEngineSchedulingService instance | 08:52 |
| alexchadin | vincentfrancoise: ok | 08:52 |
| vincentfrancoise | alexchadin: and we should modify DecisionEngineSchedulingService to make it a singleton | 08:52 |
| vincentfrancoise | alexchadin: because right now it seems we already have 2 background scheduler running at the same time | 08:53 |
| alexchadin | vincentfrancoise: ah, yeah I see that BackgroundSchedulerService inherit background.BackgroundScheduler | 08:53 |
| vincentfrancoise | alexchadin: yeah that's why | 08:53 |
| alexchadin | vincentfrancoise: good point, let's see what we can do with it:) | 08:53 |
| vincentfrancoise | alexchadin: problem is that it will probably break many tests | 08:54 |
| vincentfrancoise | alexchadin: because adding singletons usually creates test dependencies | 08:54 |
| alexchadin | vincentfrancoise: but how we can run job then? it will still tries to run launch_audits_periodically without instantiating ContinuousAuditHandler | 08:55 |
| vincentfrancoise | alexchadin: so maybe you'll have to add a cleanup that would destroy the background scheduler singleton instance everytime to avoid that | 08:55 |
| vincentfrancoise | alexchadin: ah yeah so once this is done, I suggest we make launch_audits_periodically a classmethod | 08:57 |
| vincentfrancoise | alexchadin: the self.scheduler can then be replaced by an instantiation that would do nothing if the singleton is already created | 08:57 |
| alexchadin | vincentfrancoise: launch_audits_periodically adds self.execute_audit as method to execute | 08:57 |
| vincentfrancoise | alexchadin: ah I missed that one... | 08:58 |
| alexchadin | vincentfrancoise: there are dependencies after dependencies... | 08:58 |
| vincentfrancoise | alexchadin: ok so let me think about another solution then | 09:00 |
| alexchadin | I'll be back soon | 09:00 |
| vincentfrancoise | alexchadin: ok | 09:01 |
| *** alexchadin has quit IRC | 09:16 | |
| *** necuser__ has quit IRC | 09:20 | |
| *** gabor_antal_ is now known as gabor_antal | 09:23 | |
| *** karthikpr has joined #openstack-watcher | 09:26 | |
| *** karthikpr has quit IRC | 09:31 | |
| *** acabot has quit IRC | 09:32 | |
| *** acabot has joined #openstack-watcher | 09:34 | |
| *** necuser__ has joined #openstack-watcher | 09:34 | |
| *** thorst has joined #openstack-watcher | 09:47 | |
| aspiers | jwcroppe: we don't package it yet but I would really like to | 10:03 |
| openstackgerrit | Santhosh Fernandes proposed openstack/watcher master: Add gnocchi support in VM-Workload-Consolidation strategy https://review.openstack.org/450693 | 10:05 |
| openstackgerrit | Zhong Luyao proposed openstack/watcher-specs master: Noisy Neighbor Strategy renamed cache occupancy strategy to noisy neighbor https://review.openstack.org/398162 | 10:06 |
| *** vincentfrancoise has quit IRC | 10:16 | |
| *** thorst has quit IRC | 10:19 | |
| *** zhurong has quit IRC | 10:20 | |
| *** zhurong has joined #openstack-watcher | 10:28 | |
| *** sanfern has quit IRC | 10:28 | |
| *** acabot has quit IRC | 10:28 | |
| *** acabot has joined #openstack-watcher | 10:29 | |
| *** pksingh has quit IRC | 10:33 | |
| *** acabot has quit IRC | 10:41 | |
| *** alexchadin has joined #openstack-watcher | 10:43 | |
| *** alexchadin has quit IRC | 10:43 | |
| *** alexchadin has joined #openstack-watcher | 10:44 | |
| *** alexchadin has quit IRC | 10:48 | |
| aspiers | hi all, is there a concise summary anywhere of the difference between Watcher and Neat? If not, it would be really helpful to create one | 10:51 |
| *** alexchadin has joined #openstack-watcher | 10:52 | |
| aspiers | it seems that Neat is only focusing on VM consolidation, right? but seems to have done some great work in that area | 10:58 |
| aspiers | it seems a shame that they implemented it as a separate project though | 10:59 |
| aspiers | I wonder why that happened | 10:59 |
| openstackgerrit | Zhong Luyao proposed openstack/watcher-specs master: Noisy Neighbor Strategy renamed cache occupancy strategy to noisy neighbor https://review.openstack.org/398162 | 11:02 |
| openstackgerrit | licanwei proposed openstack/watcher master: Add action description https://review.openstack.org/450720 | 11:11 |
| *** thorst has joined #openstack-watcher | 11:20 | |
| aspiers | also I get the impression they take a decentralized approach | 11:23 |
| *** zhurong has quit IRC | 11:25 | |
| *** thorst has quit IRC | 11:25 | |
| *** vincentfrancoise has joined #openstack-watcher | 11:27 | |
| *** alexchadin has quit IRC | 11:30 | |
| *** openstackgerrit has quit IRC | 11:33 | |
| vincentfrancoise | aspiers: hi | 11:39 |
| aspiers | hi vincentfrancoise | 11:39 |
| vincentfrancoise | aspiers: just read about the comparison with openstack neat | 11:40 |
| vincentfrancoise | aspiers: basically they were focusing on the consolidation whereas watcher has taken a more generic path to address any goal | 11:41 |
| vincentfrancoise | aspiers: also I didn't test it by their only algorithm is bin packing, which is also the case for basic_consolidation in Watcher | 11:42 |
| vincentfrancoise | aspiers: and at the time, the solution wasn't really well maintained and it seemed the project was dying out | 11:44 |
| vincentfrancoise | aspiers: so it made more sense for us to start from a clean slate (at least that's what what I was told when I also asked the question) | 11:45 |
| *** thorst has joined #openstack-watcher | 11:45 | |
| aspiers | oh, so Neat came first? interesting | 11:45 |
| vincentfrancoise | aspiers: as for explaining the difference, since they were not within the openstack big tent or even partially integrated within the community, we didn't really bother | 11:46 |
| vincentfrancoise | aspiers: if you look closely into the readme, you can see this is a PhD project and I assume the guy just dropped the project at the end of it | 11:48 |
| aspiers | vincentfrancoise: seems that there has been very little activity since 2013 https://github.com/beloglazov/openstack-neat/graphs/contributors | 11:50 |
| aspiers | the guy now works for Google, so I assume he's not working on OpenStack any more | 11:50 |
| aspiers | that's a shame | 11:50 |
| aspiers | not much activity here either https://groups.google.com/forum/#!forum/openstack-neat | 11:51 |
| *** openstackgerrit has joined #openstack-watcher | 11:58 | |
| openstackgerrit | aditi sharma proposed openstack/watcher master: WIP: run watcher-api with mod-wsgi https://review.openstack.org/450740 | 11:58 |
| *** karthikpr has joined #openstack-watcher | 13:27 | |
| *** jwcroppe has quit IRC | 13:28 | |
| *** karthik__ has joined #openstack-watcher | 13:29 | |
| *** karthikpr has quit IRC | 13:31 | |
| *** karthik__ has quit IRC | 13:34 | |
| *** jwcroppe has joined #openstack-watcher | 13:42 | |
| *** sanfern has joined #openstack-watcher | 13:43 | |
| *** karthikpr has joined #openstack-watcher | 13:47 | |
| *** karthikpr has quit IRC | 13:52 | |
| *** karthikpr has joined #openstack-watcher | 14:05 | |
| *** karthikpr has quit IRC | 14:09 | |
| *** Tamayo has joined #openstack-watcher | 14:10 | |
| *** gabor_antal has quit IRC | 15:05 | |
| *** karthikpr has joined #openstack-watcher | 15:10 | |
| *** karthikpr has quit IRC | 15:15 | |
| *** vincentfrancois1 has joined #openstack-watcher | 15:18 | |
| *** sanfern has quit IRC | 15:19 | |
| *** karthikpr has joined #openstack-watcher | 15:27 | |
| *** sanfern has joined #openstack-watcher | 15:31 | |
| *** karthikpr has quit IRC | 15:32 | |
| *** alexchadin has joined #openstack-watcher | 15:45 | |
| alexchadin | vincentfrancoise: ping | 15:45 |
| alexchadin | aspiers: hi | 15:47 |
| aspiers | hi | 15:47 |
| alexchadin | aspiers: I agree with you about Neat achievements, Anton did a great work | 15:47 |
| aspiers | yes it looks impressive | 15:48 |
| aspiers | I met this guy https://cs.uwaterloo.ca/~mkarsten/ earlier this year - he is interested in this field and was asking about Neat, Watcher etc. | 15:49 |
| alexchadin | aspiers: I've studied Anton's work, some of his suggests was used to build Nova-LoadBalancer, which then has been integrated into Watcher | 15:50 |
| alexchadin | aspiers: Martin is welcome here! | 15:51 |
| aspiers | alexchadin: I suggested him to join :) | 15:51 |
| alexchadin | aspiers: great! We are appreciate fresh looks and minds :) | 15:52 |
| aspiers | I thought so ;) | 15:52 |
| alexchadin | aspiers: back to Neat, unfortunately this project was made in different code-style unlike OpenStack | 15:53 |
| alexchadin | aspiers: each great ideas should be wrapped in appropriate form | 15:54 |
| alexchadin | idea* | 15:54 |
| aspiers | ah | 15:54 |
| *** vincentfrancoise has quit IRC | 15:56 | |
| *** alexchadin has quit IRC | 16:00 | |
| *** alexchadin has joined #openstack-watcher | 16:19 | |
| *** gabor_antal has joined #openstack-watcher | 16:43 | |
| *** gabor_antal_ has joined #openstack-watcher | 16:55 | |
| *** gabor_antal has quit IRC | 16:55 | |
| *** alexchadin has quit IRC | 16:59 | |
| *** gabor_antal_ is now known as gabor_antal | 17:14 | |
| *** wootehfoot has joined #openstack-watcher | 19:04 | |
| *** hidekazu has quit IRC | 20:26 | |
| *** wootehfoot has quit IRC | 20:30 | |
| *** wootehfoot has joined #openstack-watcher | 20:37 | |
| *** ianychoi has quit IRC | 21:07 | |
| *** ianychoi has joined #openstack-watcher | 21:08 | |
| *** thorst has quit IRC | 21:51 | |
| *** wootehfoot has quit IRC | 22:11 | |
| *** jwcroppe has quit IRC | 22:49 | |
| *** jwcroppe has joined #openstack-watcher | 22:49 | |
| *** jwcroppe has quit IRC | 22:50 | |
| *** jwcroppe has joined #openstack-watcher | 22:51 | |
| *** thorst has joined #openstack-watcher | 22:51 | |
| *** thorst has quit IRC | 22:55 | |
| *** jwcroppe has quit IRC | 22:56 | |
| *** thorst has joined #openstack-watcher | 23:08 | |
| *** jwcroppe has joined #openstack-watcher | 23:22 | |
| *** thorst has quit IRC | 23:25 | |
| *** thorst has joined #openstack-watcher | 23:25 | |
| *** thorst has quit IRC | 23:29 | |
| *** thorst has joined #openstack-watcher | 23:56 | |
| *** hidekazu has joined #openstack-watcher | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!