15:00:55 <peschk_l> #startmeeting cloudkitty
15:00:56 <openstack> Meeting started Fri Feb  1 15:00:55 2019 UTC and is due to finish in 60 minutes.  The chair is peschk_l. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:59 <openstack> The meeting name has been set to 'cloudkitty'
15:01:18 <peschk_l> hello everybody
15:01:27 <jferrieu> hi
15:01:38 <erolg> o/
15:01:42 <peschk_l> the agenda for today's meeting can be found at https://etherpad.openstack.org/p/cloudkitty-meeting-topics
15:02:29 <peschk_l> we'll proceed with the topics in the order theyy are added. If there is anything you'd like to address, fell free to add your topic to the etherpad
15:02:47 <peschk_l> hello erolg, nice to see new people around
15:03:00 <peschk_l> alright, let's start
15:03:08 <peschk_l> #topic v2 API
15:04:21 <peschk_l> The spec for the v2 API is still under review. We'd like to merge it as soon as possible. I'll push work that has already been done on it as soon as the spec is accepted
15:05:39 <peschk_l> jferrieu, huats, could you please have a look at the spec again ? If you both approve the spec, we'll merge it. I already integrated Linkid's remarks
15:05:54 <peschk_l> #link https://review.openstack.org/#/c/614275/
15:06:31 <jferrieu> I am reviewing it
15:06:45 <peschk_l> I believe that's it for the v2 API, not much to say appart from that
15:06:50 <peschk_l> jferrieu: thanks!
15:07:02 <peschk_l> #topic Prometheus collector spec
15:08:27 <peschk_l> as you may know, jferrieu has been working on the Prometheus collector lately. He has submitted a spec. It looks good to me, but additional reviews are of course required. If anybody wants to read it, please do so!
15:08:45 <peschk_l> #link https://review.openstack.org/#/c/626142/
15:09:06 <peschk_l> jferrieu: anything to add about this ?
15:10:25 <jferrieu> I have started to setup a dev environment looking forward to implement unit tests
15:10:42 <jferrieu> I will push them as soon as the spec is validated by the community
15:10:55 <peschk_l> all right, nice
15:11:32 <peschk_l> #topic state of the code cleanup
15:12:44 <peschk_l> as we discussed during last meeting, it has been decided to remove the unmaintained parts of cloudkitty's code. ie some fetchers, the v2 gnocchi storage (which was only for testing purposes), some collectors and the gnocchi transformer
15:13:12 <peschk_l> things are going forward on this, as you can see on the associated story:
15:13:17 <peschk_l> # link https://storyboard.openstack.org/#!/story/2004400
15:14:01 <peschk_l> A patch is under review for the removal of the v2 storage backend, and the writer will be removed in T, S being the deprecation release
15:15:20 <peschk_l> No help is required on this, but we definitely need some for the next topic!
15:15:31 <peschk_l> #topic documentation updates
15:16:22 <peschk_l> the associated story can be found here
15:16:26 <peschk_l> #link https://storyboard.openstack.org/#!/story/2004179
15:17:46 <peschk_l> Three patches are currently under review: One is about collectors (from and admin point of view), another is some developper documentation about how to write a new collector, and the last one updates the hashmap documentation
15:17:55 <peschk_l> #link https://review.openstack.org/#/c/633035/
15:18:06 <peschk_l> @link https://review.openstack.org/#/c/625924/
15:18:18 <peschk_l> #link https://review.openstack.org/#/c/628393/
15:19:20 <peschk_l> It would be nice to have reviews on these from users, as they are the target audience of these changes
15:19:53 <peschk_l> jferrieu and Linkid have already made some reviews on these, but it would be nice to have other point of views
15:20:07 <peschk_l> huats: if you have some time to read these, it would be great
15:21:07 <peschk_l> just a note for people who are not familiar with gerrit: you can have an html version of the generated documentation by clicking on "openstack-tox-docs" under "Zuul check" in the gerrit interface for a given patch
15:21:43 <peschk_l> a fourth patch about storage backend configuration is also WIP
15:22:32 <peschk_l> Does anybody have something to add on this ?
15:24:23 <peschk_l> all right, moving on
15:24:42 <peschk_l> #topic state of the dashboard
15:25:34 <peschk_l> As we have already discussed several times, we are trying to get rid of minified javascript in the dashboard
15:27:11 <peschk_l> this will be done in three steps:
15:27:11 <peschk_l> 1. Add non-minified versions of the javascript files next to the minified ones
15:27:11 <peschk_l> 2. Get rid of D3-pie, as it is not packaged in XStatic
15:27:11 <peschk_l> 3. Make the dashboard depend on Xstatic packages for javascript dependencies, rather than having them in the repo
15:27:32 <peschk_l> step 1 has been managed by zigo and has already been merged
15:28:30 <peschk_l> I am affected to step 2 and have submitted a patch for it: https://review.openstack.org/#/c/627994/. Reviews welcome!
15:29:20 <peschk_l> step 3 is Linkid's job. It is currently WIP, but he has already submitted a patch he's working on, in case you want to have a look:
15:29:36 <peschk_l> #link https://review.openstack.org/#/c/628318/
15:30:05 <peschk_l> I'm confident that this will be done by the Stein release
15:30:42 <peschk_l> There are two more things in the dashboard that must be solved before we release Stein
15:31:21 <peschk_l> 1. The hashmap service used for predictive pricing is harcoded in the dashboard (it's "compute"). This should be configurable by admins.
15:31:45 <peschk_l> I'm currently working on this and I'll keep you updated
15:33:04 <peschk_l> 2. Some imports are done in ways which are only compatible with django 1.x. This will be an issue when we'll switch to python3/django2. A patch has been submitted this morning to address this:
15:33:18 <peschk_l> #link https://review.openstack.org/#/c/634393/
15:34:20 <peschk_l> These should be the last patches we land before the Stein release. It seems to be a reasonable toarget
15:34:24 <peschk_l> *target
15:35:03 <peschk_l> Does anybody have questions about the dashboard ?
15:36:43 <peschk_l> moving on
15:37:00 <peschk_l> #topic Change in Gnocchi collector's behavior concerning resource metadata
15:38:02 <peschk_l> This should be quick. We encoutered the same scenario in several deployments. Sometimes, gnocchi resources do have measures in their associated metrics after their deletion date
15:38:26 <peschk_l> due ot its design, the gnocchi collector fails to retrieve the associated metadata
15:38:41 <peschk_l> more details are available in the followiong story:
15:38:57 <peschk_l> https://storyboard.openstack.org/#!/story/2004895
15:39:03 <peschk_l> #link https://storyboard.openstack.org/#!/story/2004895
15:39:25 <peschk_l> The proposed fix can be found here:
15:39:28 <peschk_l> #link https://review.openstack.org/#/c/634199/
15:40:30 <peschk_l> jferrieu has already reviewed it, at least one additional review would be nice
15:41:09 <peschk_l> That's also something which would be nice to have in Stein
15:42:07 <peschk_l> let's move on to the final topic. there will be some time for q &a at the end of the meeting
15:42:26 <peschk_l> #topic Adding collector + fetcher to the state database
15:44:11 <peschk_l> I'm currently working on a spec for this. The problem is basically the following: Let's say that you use cloudkitty to rate an OpenStack cloud. You have one collector collecting data from gnocchi and another one from monasca
15:44:59 <peschk_l> if both of these collectors are using the same scope (let's say project_id), the behavior of cloudkitty will be unpredictable
15:46:10 <peschk_l> or rather: The first collector trying to process a project will lock it and update the state until the current day. This would lead to missing data
15:47:33 <peschk_l> More details on this problem and the proposed solution will be available in the spec once it is submitted
15:48:36 <peschk_l> but in a nutshell: we could add the collector (ie the datasource) and the fetcher which retrieved the scope into the state management workflow
15:49:49 <peschk_l> This would allow to fix the case described above, and would avoid hash collisions between UUIDs from different sources (even if they are very unlikely to occur)
15:50:08 <peschk_l> Does anybody have questions on this ?
15:51:17 <peschk_l> #topic Q & A
15:52:04 <peschk_l> we have 10 minutes left. If anybody wants to ask some questions on any of today's topics, he/she can do it now
15:53:45 <openstackgerrit> Luka Peschke proposed openstack/cloudkitty master: Delete v2 gnocchi storage  https://review.openstack.org/625911
15:53:59 <jferrieu> you were speaking of the state management workflow
15:54:39 <jferrieu> can I have some details on this? Or at least some resource to help at understanding it?
15:54:52 <peschk_l> jferrieu: of course
15:56:08 <peschk_l> For each running processor the orchestrator retrieves the different scopes (project/domain IDs / namespaces / whatever) to rate
15:57:25 <peschk_l> for each scope, a worker is spawned. This worker tries to acquire a lock. If it succeeds, it will process the scope it was given from the state (ie timestamp) it has been left at until the current timestamp - X collect periods
15:58:21 <peschk_l> currently, only the scope_id is stored, next to its timestamp. But in the case described above, the following would happen:
15:58:55 <jferrieu> yeah I now see the problem I tink
15:59:16 <jferrieu> it would happen on lock acquiring
15:59:24 <peschk_l> - gnocchi collector locks the tenant XXX, queries its state, get None. In consequence, it processes data from the beginning of the month until timestamp H and releases the lock
16:00:35 <peschk_l> - monasca collector tries to acquire the lock, fails, processes some other tenants, and tries again. When it asks for the scope's state, it gets "H". So the monasca collector will only collect data from H to H+1
16:01:07 <peschk_l> adding the collector next to the scope_id would solve this
16:01:28 <peschk_l> *the colelctor and the scope fetcher
16:03:20 <jferrieu> ok I see now
16:03:23 <jferrieu> thanks
16:03:46 <peschk_l> you're welcome :-)
16:04:51 <peschk_l> We're runnig out of time, I'll end the meeting. Thanks everybody for attending!
16:05:02 <peschk_l> #endmeeting