02:00:32 <dangtrinhnt> #startmeeting openstack telemetry
02:00:33 <openstack> Meeting started Thu May 23 02:00:32 2019 UTC and is due to finish in 60 minutes.  The chair is dangtrinhnt. Information about MeetBot at http://wiki.debian.org/MeetBot.
02:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
02:00:36 <openstack> The meeting name has been set to 'openstack_telemetry'
02:00:59 <dangtrinhnt> zhurong, lxkong, are you here?
02:01:13 <lxkong> hey
02:01:24 <dangtrinhnt> shaoman, adriant
02:01:27 <dangtrinhnt> lxkong hi
02:01:41 <adriant> o/
02:01:47 <dangtrinhnt> Let's look at the meeting agenda today #link https://etherpad.openstack.org/p/telemetry-meeting-agenda
02:01:51 <dangtrinhnt> adriant hi :)
02:01:58 <zhurong> Hi dangtrinhnt
02:02:05 <dangtrinhnt> zhurong hi
02:02:51 <dangtrinhnt> okie, our focus today is the Train-1 goals and Telemetry roadmap
02:03:20 <dangtrinhnt> #topic Telemetry roadmap (cont.)
02:03:34 <dangtrinhnt> #link https://etherpad.openstack.org/p/telemetry-train-roadmap
02:04:24 <dangtrinhnt> After the last meeting, joadavis helped us to gather some options for Telemetry in Train
02:05:18 <dangtrinhnt> I would like to discuss a little bit about them before moving forward
02:05:40 <dangtrinhnt> +Proposal A: Try to maintain current course, Increase community involvement and communication
02:05:50 <dangtrinhnt> +Proposal B: Gather new feature requirements, dedicate a team to implementing them
02:06:01 <dangtrinhnt> +Proposal C: Work with each OpenStack project to provide its own telemetry data
02:06:10 <dangtrinhnt> +Proposal D: Define an end of life for the Telemetry project, tell the community to switch to Prometheus
02:06:25 <dangtrinhnt> +Proposal E: Fold Telemetry in to the Monasca project => I could expect an optimistic future for this proposal but after 2-3 more cycles
02:06:26 <dangtrinhnt> What do you think?
02:06:38 <dangtrinhnt> lxkong, zhurong, adriant?
02:07:49 <lxkong> the pA and pB are more realistic
02:07:51 <zhurong> I vote for Proposal B
02:07:59 <adriant> part of me wonders if a mix of ABC is possible
02:08:11 <dangtrinhnt> I'm ok with A,B,C. But, D, E I guess are hard to choose because there are too many people still rely on Telemetry.
02:08:49 <dangtrinhnt> lxkong, +1
02:09:05 <dangtrinhnt> adriant, I have the same thought with you.
02:09:15 <adriant> in theory, would prometheus be able to give us all the same metrics?
02:09:32 <adriant> and it would act as a datastore too, so no gnocchi
02:10:05 <adriant> the problem is no keystone integration
02:10:16 <adriant> and within that per project multi-tenancy
02:10:44 <adriant> and a lot of of useful features of telemetry rely on the concept of per project metrics
02:11:07 * adriant isn't 100% sure he's making sense
02:11:46 <dangtrinhnt> like lxkong and joadavis has pointed out, there are some issues with timestamp in Prometheus that will not provide reliable data for billing
02:11:54 <dangtrinhnt> am I right lxkong?
02:12:16 <lxkong> i'm only convinced if the person who recommends prometheus could provide some PoC demos, or at least do some actual work to compare those two
02:12:30 <lxkong> otherwise, i'm not considering that as an option
02:12:40 <zhurong> I am agree with lxkong
02:12:47 <lxkong> sorry, should be "i'm only convinced unless"
02:13:50 <dangtrinhnt> So am I.
02:13:59 <lxkong> maybe prometheus is much more promising than ceilometer, i don't know. So, show me something
02:14:57 <dangtrinhnt> Okie, so I think we can put Proposal D aside. How about E?
02:15:17 <dangtrinhnt> Monasca?
02:16:27 <adriant> does it do polling of resource data?
02:16:36 <adriant> I don't actually know enough about it
02:17:00 <adriant> https://github.com/openstack/monasca-agent so I guess it does collection of a sort
02:17:18 <dangtrinhnt> I think so #link https://wiki.openstack.org/wiki/Monasca
02:18:02 <lxkong> hmm...it replies on Apache Spark and kafaka?
02:18:13 <dangtrinhnt> It is an agent-based monitoring system. Ah, yes
02:18:19 <adriant> it sounds like it can be used for telemetry, but it does a lot more than just that
02:18:41 <lxkong> adriant: i don't think our ops will like it :-)
02:18:55 <adriant> folding telemetry for billing into monitoring sounds like a potential nightmare
02:19:12 <adriant> and yeah... even more complex dependencies :(
02:19:57 <dangtrinhnt> What they are trying to do is to add scalability to monitoring. And, considering the architecture, I guess It would be a huge change if people want to replace telemetry with monasca
02:20:15 <dangtrinhnt> I don't think it's a good idea for this cycle or the next one.
02:20:32 <lxkong> i don't know much about Monasca either, i'll leave it to the Mosasca team. Hopefully after several dev cycles, there are more and more users deploy Monasca. We will see.
02:20:43 <dangtrinhnt> +1
02:20:59 <lxkong> at least for us, we won't consider using it in the next 1 or 1.5 years
02:21:30 <dangtrinhnt> it makes sense considering the amount of resource and effort you have put into Telemetry
02:21:42 <lxkong> but i'm not against that idea
02:22:40 <dangtrinhnt> okie, I think we need to work with joadavis and the monasca team over the next several cycles I guess :)
02:23:08 <dangtrinhnt> I guess for Train, we will do something like a mix of A,B, and C.
02:23:18 <dangtrinhnt> any comments?
02:25:00 <dangtrinhnt> I will draft a proposal roadmap for Telemetry patch on gerrit in the next couple days.
02:25:07 <dangtrinhnt> Okie, next topic
02:25:27 <dangtrinhnt> #topic Train-1 progress update
02:26:37 <dangtrinhnt> self-healing with AODH is done, isn't it? lxkong?
02:26:48 <lxkong> dangtrinhnt: yes
02:26:49 <dangtrinhnt> #link https://review.opendev.org/#/c/659046/
02:26:54 <dangtrinhnt> great, thanks
02:26:58 <lxkong> we are working on the Heat support internally
02:27:17 <dangtrinhnt> that's cool
02:27:21 <lxkong> dangtrinhnt: you may notice i was still creating the alarm manually in the demo
02:27:37 <lxkong> after heat support, we could define the alarm in heat template
02:27:50 <dangtrinhnt> awesome
02:28:01 <zhurong> It is a great job lxkong
02:28:04 <dangtrinhnt> Please keep us updated with that, thanks
02:28:22 <lxkong> sure
02:29:40 <dangtrinhnt> zhurong "switch to oslo.privileps" is merged
02:30:25 <zhurong> yeah, next part we want to add container metrics
02:30:27 <dangtrinhnt> #link https://review.opendev.org/#/c/656098/
02:30:42 <dangtrinhnt> Great, thanks.
02:31:00 <lxkong> zhurong: container metrics, you mean Magnum or Zun?
02:31:23 <zhurong> with Zun
02:31:27 <lxkong> ah, ok
02:31:37 <lxkong> zhurong: have you deployed Zun in production?
02:32:03 <zhurong> yeah, we have Zun in our production
02:32:12 <dangtrinhnt> That's cool
02:33:12 <lxkong> cool, good to know
02:33:55 <zhurong> we did a lot of work in Zun
02:34:46 <dangtrinhnt> okie, next feature, the "event subsystem": I would want to wait for rafaelweingartne to update on this. He cannot attend today meeting.
02:35:42 <dangtrinhnt> In the last meeting, we're thinking about bringing back cpu_util to telemetry. Do we have resource to do that?
02:36:14 <lxkong> unfortunately, i don't have time to do that currently
02:36:53 <lxkong> but we definitely need that after upgrading Ceilometer
02:37:06 <dangtrinhnt> Okie, I guess we can put that on hold and maybe re-open it at Train-2 or 3
02:37:17 <dangtrinhnt> yeah
02:38:10 <dangtrinhnt> I will trying to spend something fix/update the docs in the next couple days.
02:38:39 <dangtrinhnt> Anything you want to discuss about Train-1 milestone?
02:39:34 <lxkong> nothing from me
02:39:56 <zhurong> me too
02:40:08 <dangtrinhnt> okie, next topic
02:40:21 <dangtrinhnt> #topic Shanghai summit
02:40:42 <dangtrinhnt> Anyone want to attend the next OpenInfra Summit in Shanghai?
02:40:59 <zhurong> haha lxkong welcome back, come on, 加西亚
02:41:07 <dangtrinhnt> :)
02:41:13 <lxkong> dangtrinhnt: we don't know yet.
02:41:24 <lxkong> but i will be kubecon Shanghai in June
02:41:38 <dangtrinhnt> We need to show the world what we're doing this cycle :)
02:41:47 <dangtrinhnt> zhurong, will you? :)
02:42:17 <lxkong> btw, i'm in the Programming Committee for the Container Infrastructure Track for Shanghai Openinfra Summit
02:42:18 <zhurong> I will be there, this time I don't need the visa, haha
02:42:30 <zhurong> cool lxkong
02:43:03 <lxkong> zhurong: would much appreciate if you could share the work you've done in Zun
02:44:07 <zhurong> we have a lighter topic about zun in Denver summit, but due to the visa, we can not attend.
02:44:19 <dangtrinhnt> lxkong yeah, that's cool
02:44:44 <dangtrinhnt> zhurong, same with me, I could not attend Denver because of VISAAA
02:44:46 <dangtrinhnt> :)
02:44:59 <joadavis_> We did miss telemetry at Denver summit.
02:45:17 <zhurong> sure, we will have topic about zun in shanghai lxkong
02:45:18 <dangtrinhnt> joadavis_ hi :)
02:45:41 <joadavis_> Dont mind me, I'm only partly here
02:46:28 <joadavis_> I would like to come to Shanghai summit and meet you all, but budget constraints...
02:47:24 <dangtrinhnt> okie, np. Neither do I, I just changed job and don't think I can go there.
02:47:53 <dangtrinhnt> Do you have anything else to discuss?
02:48:04 <dangtrinhnt> we have 12m left.
02:48:38 <lxkong> i have to leave now, has an appointment later on
02:48:48 <joadavis_> As a late attendee, I will say proposal E is somewhat already done/doable
02:49:10 <joadavis_> Witek can talk about it in the meeting at 0800 too
02:49:12 <dangtrinhnt> lxkong, np. Take your time.
02:49:32 <dangtrinhnt> joadavis_ , so let do it in the next meeting session today. Thanks
02:49:46 <zhurong> bye lxkong
02:49:49 <dangtrinhnt> zhurong, adriant, do you have anything to add?
02:51:00 <zhurong> dangtrinhnt: nothing from my side today
02:51:17 <adriant> no, mostly just a fly on the wall today.
02:51:39 <dangtrinhnt> btw, zhurong, could you and shaoman attend the next meeting today?
02:51:51 <dangtrinhnt> with the other guys
02:52:03 <dangtrinhnt> I guess it's too much for you.
02:52:31 <dangtrinhnt> adriant, your inputs/questions are valuable :)
02:52:32 <zhurong> yeah, we will, today shaoman have a internal meeting this time
02:53:07 <adriant> dangtrinhnt: ty! Just busy running around between a lot of tasks, so mostly writing/testing code while watching IRC a bit today.
02:53:39 <dangtrinhnt> zhurong: great, thanks :) see you then.
02:54:02 <dangtrinhnt> adriant, so do I :) haha
02:54:11 <dangtrinhnt> okie, I guess we can end the meeting now
02:54:19 <dangtrinhnt> Thanks everyone for joining.
02:54:41 <zhurong> thanks, bye all
02:54:53 <dangtrinhnt> #endmeeting