08:00:31 <dangtrinhnt> #startmeeting openstack telemetry
08:00:32 <openstack> Meeting started Thu May 23 08:00:31 2019 UTC and is due to finish in 60 minutes.  The chair is dangtrinhnt. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
08:00:36 <openstack> The meeting name has been set to 'openstack_telemetry'
08:01:26 <dangtrinhnt> mrunge, zhurong, joedavis, hyang, witek, shaoman are you here for the meeting?
08:01:39 <witek> morning
08:02:14 <dangtrinhnt> witek, hi :)
08:02:31 <mrunge> hi
08:02:36 <witek> hi
08:02:46 <dangtrinhnt> I'm looking for Tobias Urdin's IRC handle
08:02:57 <dangtrinhnt> anyone know it?
08:03:14 <mrunge> good question, I may remember it, when I see it
08:03:48 <witek> tobiao?
08:04:00 <dangtrinhnt> I guess, tobias-urdin
08:04:15 <dangtrinhnt> anyway
08:04:44 <m0zart89> Hello.Is this channel strictly for meetings and discussions ?
08:04:46 <mrunge> https://launchpad.net/~tobias-urdin
08:04:58 <mrunge> tobias-urdin, as already said
08:05:37 <mrunge> m0zart89: it's not. but we're having a meeting right now. if you have a question, can that wait till after the meeting?
08:05:37 <dangtrinhnt> anyway, the focus of this meeting is "Cross-project discussion (e.g., monasca merged/integration/replacement) + roadmap of Telemetry in Train"
08:05:58 <m0zart89> mrunge: of course ;)
08:06:12 <dangtrinhnt> m0zart89, :) in about 1 hour, we're free :)
08:06:52 <dangtrinhnt> Please look at the meeting agenda #link https://etherpad.openstack.org/p/telemetry-meeting-agenda
08:07:10 <witek> I have added a URL to Monasca publisher review to the agenda
08:07:40 <witek> oh, you moved it, it's OK
08:07:58 <dangtrinhnt> witek, yes, I would like to discuss a little bit about our roadmap before moving forward
08:08:04 <dangtrinhnt> #topic Telemetry roadmap
08:08:28 <dangtrinhnt> #link https://etherpad.openstack.org/p/telemetry-train-roadmap
08:08:58 <dangtrinhnt> joadavis had helped us put together several options
08:09:35 <dangtrinhnt> A: Try to maintain current course, Increase community involvement and communication
08:09:42 <dangtrinhnt> B: Gather new feature requirements, dedicate a team to implementing them
08:09:48 <dangtrinhnt> C: Work with each OpenStack project to provide its own telemetry data
08:09:57 <dangtrinhnt> D: Define an end of life for the Telemetry project, tell the community to switch to Prometheus
08:10:07 <dangtrinhnt> E: Fold Telemetry in to the Monasca project => I could expect an optimistic future for this proposal but after 2-3 more cycles
08:11:48 <witek> I think C adds real value to OpenStack
08:11:49 <dangtrinhnt> In this morning session, we agreed that a mix of A,B,C could be a doable solution. D would need a solid proof that Prometheus can replace Telemetry. E. Is also a good option but would be in 2-3 cycles since many people depends on Telemetry
08:12:02 <dangtrinhnt> witek, I think so too
08:12:41 <dangtrinhnt> Any comments?
08:13:03 <witek> regarding E, I think it should be a free choice for companies to decide which approach they want to choose
08:13:44 <mrunge> I'd think, we should keep the community honest and talk about deprecation
08:14:33 <mrunge> regarding to D: I don't see a feature parity there yet
08:15:32 <dangtrinhnt> +1
08:16:46 <mrunge> other thoughts?
08:17:06 <witek> what is your opinion dangtrinhnt ?
08:17:12 <dangtrinhnt> witek, monasca seems more flexible and give users more choice so I think it's inevitable to use monasca.
08:17:54 <dangtrinhnt> Looking at what Monasca offers so far, I would want to design a future for Billing and Monitoring.
08:18:25 <mrunge> do you really want to dive into billing?
08:18:35 <mrunge> there used to be a project for that
08:18:54 <mrunge> and everyone else tried to avoid it like hell
08:19:15 <dangtrinhnt> Cloud Kitty you mean?
08:19:18 <mrunge> right
08:19:33 <mrunge> there was a single company behind it, iirc
08:19:55 <dangtrinhnt> lxkong and adriant are mostly use ceilometer for their billing system
08:19:59 <dangtrinhnt> tobias-urdin, hi
08:20:12 <mrunge> with regards to merging monasca and telemetry: which community is larger?
08:20:20 <tobias-urdin> hi o/ sorry if i missed the beginning, been working nightshift
08:20:26 <mrunge> i.e. maybe it'd make sense to merge monasca into telemetry?
08:21:05 <dangtrinhnt> mrunge, I would love to here some stats of monasca from witek
08:21:35 <witek> stats are all public, you can look them up at stackalytics
08:21:42 <dangtrinhnt> But like I said, It would be in 2-3 more cycles.
08:21:56 <witek> we have Fujitsu, SUSE, StackHPC and NEC on board
08:22:40 <dangtrinhnt> so currently, they are all active contributors?
08:23:11 <witek> yes, these are companies actively contributing
08:23:15 <dangtrinhnt> great
08:23:45 <mrunge> looking at the stats, telemetry is about twice as big?
08:24:16 <mrunge> hmm, that is derailing here and doesn't lead anywhere, let's table that
08:24:50 <witek> mrunge: I think Telemetry's mission never had been to provide operational monitoring
08:25:14 <witek> with sampling rates of 15s and below
08:25:16 <dangtrinhnt> but right now Telemetry only has a couple active contributors like shaoman, lxkong, zhurong and myself (mostly review and merge code)
08:25:27 <dangtrinhnt> witek, true
08:25:53 <mrunge> right witek. /me is looking more for sampling rates in 1s intervals
08:26:09 <mrunge> or below
08:27:41 <dangtrinhnt> okie, regarding the merge, how about wait until the end of Train?
08:27:41 <tobias-urdin> im somewhat conflated on treating monitoring and metrics collection as the same thing, we explicitly kept away from monasca to not provide monitoring and only collect metrics
08:27:48 <tobias-urdin> to skip deploying eight more services, and skipping java
08:28:04 <witek> dangtrinhnt: coming back to your idea, I'd be happy to help working on such design
08:28:17 <mrunge> right. we don't do monasca either
08:29:16 <dangtrinhnt> witek, great thanks, let's start an etherpad and see what could be done.
08:29:28 <dangtrinhnt> tobias-urdin, makes sense, I would skip java
08:30:07 <witek> on Java topic, we planned for this cycle to work on removing the last Java component
08:30:17 <witek> thresholding engine
08:30:39 <dangtrinhnt> Let's put together a draft design if the two (telemetry, monasca) could be matched.
08:32:30 <dangtrinhnt> okie, anything else on this?
08:32:58 <mrunge> not from my side
08:33:23 <witek> not from me either
08:34:20 <dangtrinhnt> okie
08:35:33 <dangtrinhnt> That's great. I will draft a vision for Telemetry in Train (published to the docs) based on what we discussed here including mention the possibility or merging with Monasca
08:35:42 <dangtrinhnt> in the next couple days
08:35:52 <dangtrinhnt> next topic
08:35:53 <mrunge> ok, thank you dangtrinhnt
08:36:05 <dangtrinhnt> #topic Monasca publisher
08:36:17 <dangtrinhnt> #link https://review.opendev.org/#/c/562400/
08:36:27 <dangtrinhnt> It has been a while
08:36:37 <witek> this is an old review we have tried to contribute over one year ago
08:36:54 <witek> it adds Monasca publisher to Ceilometer
08:37:13 <witek> and allows pushing Ceilometer measurements to Monasca API
08:37:37 <dangtrinhnt> I don't see any reason not to merge it.
08:37:55 <witek> great, that's what I wanted to check on
08:38:18 <dangtrinhnt> So please resolve the merge conflicts and let us review it again.
08:38:20 <witek> the last PS is from October, so I think it should be revisited again
08:38:42 <witek> great, we'll update you on this
08:38:46 <dangtrinhnt> Okie, thanks.
08:39:28 <dangtrinhnt> mrunge, any comment?
08:40:28 <mrunge> uhm, regarding the publisher? nope
08:40:39 <dangtrinhnt> okie
08:41:07 <dangtrinhnt> Do you have anything else to discuss today?
08:41:54 <mrunge> uhm, I still have a pile of openstack emails to read, so I probably skipped the announcement for this meeting so far
08:42:08 <mrunge> are we going to have this meeting in this time slot regularly?
08:42:42 <witek> what about the idea of new instrumentation code for OpenStack projects?
08:42:43 <dangtrinhnt> Yes, for now
08:42:53 <witek> Idea C from your list
08:43:04 <mrunge> dangtrinhnt: thank you, sounds good to me.
08:43:45 <mrunge> I would rather document integration or exposing of metrics for other projects
08:44:01 <mrunge> i.e. document, how to send metrics to the metrics subsystem
08:44:12 <dangtrinhnt> witek, I'm ok with that and would combine it with A and B.
08:44:46 <dangtrinhnt> #topic Any other things
08:44:57 <dangtrinhnt> mrunge, +1
08:45:35 <tobias-urdin> mrunge: +1 from what i understood it telemetry never wanted to build ceilometer with this polling mechanism to work around projects not exposing metrics
08:45:47 * mrunge nods
08:45:56 <tobias-urdin> when it first started, would be interesting to know the optimal end result if it started from the beginning now
08:46:02 <mrunge> I've been contacted e.g by the manila team on exactly that question
08:46:28 <mrunge> ceilometer has a bad name for being slooooow
08:46:46 <tobias-urdin> any idea how one reliably would solve the push of metrics instead of polling? while polling is resource intensive it somewhat reliable on what it does
08:46:46 <dangtrinhnt> :)
08:47:07 <mrunge> I'd have ideas :)
08:47:13 <tobias-urdin> please don't say use existing message buss (rabbitmq) :D
08:47:20 * mrunge nods
08:47:31 <mrunge> nope. do not use rabbit
08:48:05 <mrunge> I would also like to see a message mesh instead of a single bus
08:48:31 <witek> shouldn't every service instrument their own code and expose it in the standardized way?
08:48:53 <mrunge> witek we're talking about different things
08:48:57 <tobias-urdin> i'll just out; im for any A,B,C approach, we use it primaryly for billing and autoscaling, gnocchi for storage and have invested a lot of time to have a third party integrate their billing engine with gnocchi API
08:49:39 <tobias-urdin> i heard some mention on octavia metrics, we have a simple one that collects usage, i can clean that up and push upstream
08:49:49 <mrunge> that sounds like you invested some effort, tobias-urdin
08:50:09 <mrunge> kudos!
08:50:25 <dangtrinhnt> tobias-urdin, great
08:52:17 <dangtrinhnt> Since we only have less than 10m left, I will put your question on the agenda for the next meeting and maybe open a discussion on the ML
08:52:20 <mrunge> so, any immediate actions for now?
08:52:45 <mrunge> or a plan what to do next wrt telemetry here?
08:53:43 <dangtrinhnt> I don't have any idea for now. So let bring that up to the ML.
08:53:59 <dangtrinhnt> if you talking about what tobias-urdin just asked
08:55:00 <mrunge> I haven't seen any question from tobias-urdin, did I miss that?
08:55:20 <dangtrinhnt> ah, about the metric polling/pushing
08:55:28 <mrunge> No I meant next steps for us to move forward here (with train)
08:55:39 <mrunge> ah! got you
08:56:54 <mrunge> what about pushing metrics via amqp to something presenting it to a component to pick it up?
08:57:12 <mrunge> i.e. you'd be moving the load off the collectors
08:57:47 <witek> isn't this what Monasca is doing right now?
08:57:57 <witek> s/amqp/kafka
08:58:22 <dangtrinhnt> Okie, for Train I wouldn't expect big design/architecture or changes.
08:59:18 <dangtrinhnt> So next steps are:
08:59:23 <tobias-urdin> the C approach is interesting to say the least, A and B more realistic. Then there is probably a lot of things that Monasca and Telemetry could work togther on
08:59:47 <tobias-urdin> sorry dangtrinhnt, go on :)
09:00:10 <dangtrinhnt> 1. Focus on community engament via ML, meeting, events
09:00:21 <dangtrinhnt> 2. Fix existing bugs, documents
09:01:13 <dangtrinhnt> 3. Designing the vision for Telemetry for the next 2-3 cycles (e.g., monasca work together)
09:01:30 <dangtrinhnt> Yeah, basically what tobias-urdin said.
09:01:50 <dangtrinhnt> Is that clear enought?
09:01:52 <dangtrinhnt> okie, time up
09:02:09 <mrunge> yupp
09:02:14 <mrunge> thank you, dangtrinhnt
09:02:16 <witek> thanks dangtrinhnt
09:02:18 <tobias-urdin> yes, thanks for clearing everything up dangtrinhnt!
09:03:10 <dangtrinhnt> okie, thanks everyone for the meeting
09:03:13 <dangtrinhnt> #endmeeting