Tuesday, 2019-12-03

« Monday, 2019-12-02 Index Wednesday, 2019-12-04 »
*** ivve has quit IRC00:16
*** joadavis has quit IRC04:07
*** joadavis has joined #openstack-monasca04:07
*** witek_ has joined #openstack-monasca07:36
*** trident has quit IRC07:51
*** johanssone has quit IRC07:51
*** trident has joined #openstack-monasca07:51
*** johanssone has joined #openstack-monasca07:52
*** ivve has joined #openstack-monasca08:51
*** chaconpiza has joined #openstack-monasca08:58
*** dougsz has joined #openstack-monasca09:03
*** pcaruana has joined #openstack-monasca10:08
*** witek_ has quit IRC10:41
*** arseni-lipinski has quit IRC10:45
*** vishalmanchanda has joined #openstack-monasca11:34
*** trident has quit IRC11:40
*** trident has joined #openstack-monasca11:43
*** witek has joined #openstack-monasca12:06
*** arseni-lipinski has joined #openstack-monasca12:37
witekCourtesy Monasca meeting reminder in #openstack-monasca: witek, jayahn,iurygregory,ezpz,igorn,haad,sc,joadavis, akiraY,tobiajo,dougsz,fouadben, amofakhar, haru5ny (haruki),kaiokmo,pandiyan,guilhermesp,chaconpiza,hosanai,Wasaac,brtknr12:59
witek#startmeeting monasca13:00
openstackMeeting started Tue Dec  3 13:00:18 2019 UTC and is due to finish in 60 minutes.  The chair is witek. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
*** openstack changes topic to " (Meeting topic: monasca)"13:00
openstackThe meeting name has been set to 'monasca'13:00
witekhello everyone13:00
dougszhey all13:00
witekhi Doug13:01
witekthe agenda is very light13:02
witekhttps://etherpad.openstack.org/p/monasca-team-meeting-agenda13:02
witekdougsz: do you have anything else to discuss today?13:02
dougszI started revamping the Prometheus plugin for Monasca Agent13:03
witekplease add to the agenda if applicable13:04
witeklet's start13:04
dougszyeah - probably not worth discussing in detail this week - will wait until more testing has been done13:04
witek#topic logs CI job13:04
*** openstack changes topic to "logs CI job (Meeting topic: monasca)"13:04
witeksmall progress on logs CI job13:04
witekI've added DEBUG logging level for ES13:04
witekhttps://review.opendev.org/69688913:05
witekand from what I understand the logs look OK13:05
dougszGood idea13:05
witekit's just that the logs don't get posted to ES13:06
witekso we have to investigate further which component earlier in the pipeline stops working correctly13:07
witekI've confirmed already earlier that the logs get published to Kafka13:07
witekso the next candidate would be logs-persister (Logstash) or Kafka itself13:08
witek#topic monasca-agent Kafka plugin bug13:09
*** openstack changes topic to "monasca-agent Kafka plugin bug (Meeting topic: monasca)"13:09
dougszWe saw one issue with logstash output plugin not posting logs recently, but it turned out to be related to /etc/hosts config13:09
dougszso it's unlikely to be that, given it sometimes passes13:09
witekyou mean logstash-monasca output plugin?13:10
witeklogs-persister uses ES output plugin13:10
dougszsorry, logstash kafka output plugin - which I am not sure applied here13:10
witekno13:11
witekwhen looking at Kafka I've noticed it's not monitored correctly on DevStack13:12
witekthe agent's plugin need a refresh13:12
witekKafka plugin13:12
witekI've put some notes together:13:13
witekhttps://storyboard.openstack.org/#!/story/200697713:13
dougszThanks, I've added myself as a watcher13:14
dougszI figure there are going to be a lot of py3 related issues this cycle13:14
witekyes, on the agent side most of all I think13:14
witek#topic AOB13:15
*** openstack changes topic to "AOB (Meeting topic: monasca)"13:15
witekfor Prometheus plugin, have you looked at DataDog plugin?13:16
witekhttps://github.com/DataDog/integrations-core/tree/master/prometheus13:17
dougszHmm, no, thanks for that13:17
witeknot sure if it helps, but may be worth checking13:18
dougszYeah, looks like it's missing some of the bits I need13:18
dougsz1. Division of metrics for calculating percentages13:18
dougsz2. Converting counters to rates - looks like it can send deltas which isn't quite what I need13:19
witekit would be nice to have these, indeed13:19
dougszI see there is some control for not scraping everything, that is 3) for me13:20
dougszI commonly see 1MB + scrapes on prometheus endpoints which is crazy13:20
brtknri have a question if i may after you dougsz13:22
dougszgo for it brtknr13:22
brtknrI was wondering if monasca-api supports metrics aggrgation at source13:22
brtknre.g. 30 min average13:23
brtknror daily average13:23
witekno13:23
witekwe could extend the API, and send queries to InfluxDB13:23
brtknri know InfluxDB supports this but wasnt sure if monasca provides a route to this functionality13:23
witekright, that would be easy to implement and very useful13:24
brtknris there demand for this feature?13:24
witekthe downside, we cannot use it for alerting13:24
witekI think we should add this if we have developers who could and want to work on it13:25
dougsz+113:27
brtknrdougsz: we should probably ask our customers if they'd like us to develop this feature13:27
dougszit won't hurt, but i'm not aware of any demand13:28
dougszI suppose I am slightly worried about tighter integration with InfluxDB13:29
brtknrdougsz: are there no devs who want to implement cassandra backend?13:31
brtknrdougsz: there are a few loose ends with timerange scoping too13:31
brtknrwhich simply emit warning in the logs and carry on with previous behaviour13:32
witekI wouldn't be worried about tighter integration with InfluxDB13:34
witeksome functionality (like avg, or rate) could be available on InfluxDB only13:34
dougszThat's true. I am tempted to try TimescaleDB one day13:35
dougszIf would at least be nice to know if it is any good before putting more effort into InfluxDB13:35
brtknrIts significantly faster to query summary metrics vs every single data point13:37
brtknrA quick test for this is to use the influxdb backend directly and use set intervals to calculate averages13:38
witekbrtknr: for averages you can use `metric-statistics`13:38
brtknrfrom gravana13:38
witekhttps://github.com/openstack/monasca-api/blob/master/docs/monasca-api-spec.md#get-v20metricsstatistics13:38
brtknrfrom grafana13:38
brtknrwitek: cool13:39
witeknot sure how performant it is for longer time ranges13:40
witekon another topic, we still have a number of open reviews13:42
witekI just copied the links over from last week13:44
witekanything else for today?13:44
witekthanks for joining13:45
witekand see you next week13:45
witekbye13:45
witek#endmeeting13:45
*** openstack changes topic to "OpenStack Monitoring as a Service | https://wiki.openstack.org/wiki/Monasca"13:45
openstackMeeting ended Tue Dec  3 13:45:37 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)13:45
openstackMinutes:        http://eavesdrop.openstack.org/meetings/monasca/2019/monasca.2019-12-03-13.00.html13:45
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/monasca/2019/monasca.2019-12-03-13.00.txt13:45
openstackLog:            http://eavesdrop.openstack.org/meetings/monasca/2019/monasca.2019-12-03-13.00.log.html13:45
dougszthanks all, bye13:45
*** trident has quit IRC14:24
*** trident has joined #openstack-monasca14:25
*** pcaruana has quit IRC14:33
*** pcaruana has joined #openstack-monasca15:04
openstackgerritArseni Lipinski proposed openstack/monasca-api master: Upgrade Elkstack in new API  https://review.opendev.org/69712316:01
*** ivve has quit IRC16:15
openstackgerritArseni Lipinski proposed openstack/monasca-api master: Upgrade Elkstack in new API  https://review.opendev.org/69712316:32
*** witek has quit IRC17:18
*** vishalmanchanda has quit IRC18:21
*** vishalmanchanda has joined #openstack-monasca18:21
*** gmann is now known as gmann_afk19:00
*** ivve has joined #openstack-monasca19:28
*** gmann_afk is now known as gmann20:25
*** dougsz has quit IRC20:52
*** pcaruana has quit IRC22:06
*** ivve has quit IRC23:47
« Monday, 2019-12-02 Index Wednesday, 2019-12-04 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!