15:00:39 <gordc> #startmeeting telemetry
15:00:39 <openstack> Meeting started Thu Feb  4 15:00:39 2016 UTC and is due to finish in 60 minutes.  The chair is gordc. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:41 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:43 <openstack> The meeting name has been set to 'telemetry'
15:01:50 <llu-laptop> o/
15:01:52 <ildikov> o/
15:01:57 <idegtiarov> o/
15:02:04 <_nadya_> o/
15:02:24 <liusheng> o/
15:02:40 <sileht> o/
15:02:41 <gordc> ok, let's start this
15:02:44 <ityaptin-tablet> O/
15:02:56 <gordc> #topic roadmap items (new/old/blockers) https://wiki.openstack.org/wiki/Telemetry/RoadMap
15:03:07 <gordc> reminder: http://docs.openstack.org/releases/mitaka/schedule.html
15:03:31 <gordc> feb 29 is d-day
15:03:35 <jd__> o/
15:03:51 <gordc> we basically have 3+ weeks before everything must be in
15:04:05 <gordc> if there's any pressing items (especially specs) please raise now
15:04:49 <gordc> i would probably point people to review https://review.openstack.org/#/c/162167/
15:05:14 <pradk> o/
15:05:19 <gordc> this is probably the spec with most design questions
15:05:47 <idegtiarov> gordc, thank you! want to ask folks to review it
15:06:20 <gordc> _nadya_: i assume ityaptin is still working on the polling cache patch?
15:06:59 <_nadya_> gordc: yep, he is. Anyway, we've already had POC
15:07:11 <_nadya_> gordc: so it should be not too much work
15:07:30 <gordc> ack, i will start looking at it soon.
15:07:39 <_nadya_> gordc: also, we did some testing. Looks very good. 5 sec polling and no load on Nova API
15:07:50 <gordc> _nadya_: cool stuff.
15:07:59 <gordc> any other items?
15:08:03 <ildikov> _nadya_: sounds cool \o/ :)
15:08:05 <llu-laptop> idegtiarov: any reply to the comment on https://review.openstack.org/#/c/162167/16/specs/mitaka/events-pipeline-transformers.rst ?
15:08:47 <_nadya_> llu-laptop: regarding multiple agents?
15:09:06 <llu-laptop> that and the 'emit'
15:09:13 <_nadya_> llu-laptop: actually, the main idea is to use the cache
15:09:35 <_nadya_> llu-laptop: probably, Redis or oslo.cache in general
15:09:57 <idegtiarov> llu-laptop, we cannot use event-to-sample because we expecting to emit event after transformation not sample
15:10:08 <llu-laptop> how about put those ideas in the spec?
15:10:23 <gordc> llu-laptop: ++
15:10:41 <gordc> there's a lot of unwritten design eleements
15:10:47 <llu-laptop> idegtiarov: but in your example listed in spec, it emits sample, not events
15:10:55 <idegtiarov> llu-laptop, I've done update today please take a look
15:11:33 <idegtiarov> no in my spec emit of sample was optional
15:11:40 <liusheng> does the event-timeout-alarm proposal depend on this ?
15:11:43 <_nadya_> agreed, sorry guys. The reason why we started to work on this again is that several customers asked about that. Looks like, it's a real use case indeed
15:12:02 <_nadya_> I was not sure about this
15:12:09 <llu-laptop> i'm ok with transformers tansfrom one event to another
15:12:44 <llu-laptop> but not very convinced for the event transformer to emit samples
15:12:50 <_nadya_> llu-laptop: have you seen the example about latency metric?
15:12:50 <sileht> /query jd__
15:12:58 <ildikov> I would assume those samples would actually be statistics
15:13:01 <llu-laptop> I think emit samples from event is the job of event-to-sample publisher
15:13:59 <idegtiarov> llu-laptop, agree with you and it seems that I mentioned it in spec
15:14:05 <_nadya_> llu-laptop:  +1, I thought that the spec about event-to-samples publishing were done mostly for idegtiarov's spec
15:14:27 <idegtiarov> probably I forgot to remove this option from schema
15:14:41 <llu-laptop> ok, i might be misguided by the line 65 of https://review.openstack.org/#/c/162167/16..17/specs/mitaka/events-pipeline-transformers.rst
15:14:43 <gordc> idegtiarov: is the requirement that it be done inline rather than post-storage?
15:15:54 <idegtiarov> gordc, it done you mean transformation of new event&
15:16:14 <idegtiarov> gordc, it done you mean transformation of new event?
15:16:42 <_nadya_> gordc: sure it may be done in post-storage. the same is true for current transformers. But people use some external storage, transformation-based metrics will be not available
15:16:54 <gordc> _nadya_: ack. got it
15:18:04 <llu-laptop> idegtiarov: b.t.w the url at line 90 is not valid
15:18:16 <idegtiarov> llu-laptop, thanks will improve
15:19:41 <llu-laptop> then i'm ok with the spec after you add the descriptiono of multi-agent support :)
15:19:53 <gordc> ok, let's continue to track spec (anyone feel free to jump in there)
15:20:09 <gordc> let's run through the projects
15:20:09 <ljxiash> Hi, we have a spec
15:20:16 <gordc> #topic aodh topics
15:20:44 <gordc> just an announcement, as we had a vacancy...
15:21:11 <_nadya_> ljxiash: could you please send a link?
15:21:29 <ljxiash> gordc: ok, I can wait. the link is https://review.openstack.org/#/c/273475/
15:21:56 <gordc> ljxiash: let's revisit when we get to ceilometer topics
15:22:18 <gordc> Liusheng has stepped up and volunteered to be lead/liaison for Aodh
15:22:44 <ljxiash> gordc: :) ok
15:22:50 <gordc> i'll announce it on list, but basically he'll just be more focused on Aodh and helpe me out tracking bugs/specs against Aodh
15:22:57 <gordc> thanks Liusheng.
15:23:01 <llu-laptop> liusheng: thumb up ++
15:23:04 <liusheng> yes, I am now sure I can do well, but just want to have a try :-)
15:23:24 <ljxiash> thumb up
15:23:25 <_nadya_> liusheng: cool!
15:23:25 <ildikov> liusheng: congrats and thanks for taking the responsibility for that module :)
15:23:44 <liusheng> thanks, need all your helps :)
15:23:59 <idegtiarov> liusheng, congrats! that is a good news
15:24:37 <gordc> cool cool. the one other topic i have for aodh, is aodhclient functional tests currently need gnocchi-api to work. does anyone know how get around this? or are we confident unit tests are suffice?
15:24:53 <liusheng> idegtiarov, ildikov , _nadya_ llu-laptop thanks
15:25:08 <liamji> liusheng, congrats
15:25:40 <liusheng> liamji: :)
15:25:55 <gordc> sileht: ^ since you did gnocchiclient
15:26:03 <llu-laptop> gordc: personally, i'm not that confident though.
15:26:29 <llu-laptop> sorry, I misread the info
15:26:38 <llu-laptop> no opinion on that
15:26:45 <sileht> gordc, it's really easy to run gnocchi for functionnal
15:27:17 <gordc> sileht: i guess the question is do we have an issue that to test aodhclient i need aodh and gnocchi running?
15:27:21 <pradk> sileht, the aodh api for gnocchi rule expects gnocchi running, i think thats the issue
15:27:37 <pradk> sileht, mainly to lookup get_aggregation_methods
15:27:53 <gordc> or do we create a fall back where if gnocci isn't running aodh will provide a default set of rules
15:28:51 <pradk> gordc, rules? or maintain a list of aggregation methods as default?
15:29:08 <sileht> gordc, pradk to test gnocchi rules I'm afraid that you need gnocchi
15:29:08 <gordc> right. instead of going to check gnocchi for the potential set of rules.
15:30:17 <pradk> gordc, imo to get this client functionality in, if the unit test is validating the client with mocked return values, that is good to get it in
15:30:30 <pradk> we can work on functional with gnocchi running subsequently
15:30:40 <gordc> sileht: hm. seems like a weird design to check gnocchi for rules but still have aodh know what needs to be in those rules
15:31:31 <pradk> yea it would be nice if the aodh api dint enforce gnocchi endpoint
15:31:45 <sileht> gordc, the fact is that I wouldn't rewrite the query schema validation in aodh
15:32:06 <sileht> gordc, so aodh pass the thing to gnocchi to ensure the query written by the user is correct
15:33:23 <gordc> sileht: but aodh already knows the rules becaues it already validates the input is correct for the rule
15:33:42 <sileht> gordc, no doesn't valid the 'query' field of the rule
15:34:07 <gordc> but it knows you need threshold and some other fields...
15:34:54 <sileht> gordc, I'm talking about the complex structure for searching resource http://docs.openstack.org/developer/gnocchi/rest.html#searching-for-resources
15:35:56 <sileht> gordc, this one is valid by gnocchi instead of aodh: https://github.com/openstack/aodh/blob/master/aodh/api/controllers/v2/alarm_rules/gnocchi.py#L125
15:36:44 <gordc> sileht: right. but from aodhclient pov, we already define all the possible gnocchi rules, it's odd that aodh asked gnocchi for rules (https://github.com/openstack/aodh/blob/master/aodh/api/controllers/v2/alarm_rules/gnocchi.py#L62-L69)
15:37:04 <gordc> not just validating query but validating the rule is actually real.
15:37:29 <sileht> gordc, the aggregation function avialable is pluggable in Gnocchi, we can't hardcode the list
15:37:55 <gordc> but yeah, i guess if it's validating query part, we need gnocchi no matter what
15:38:03 <sileht> gordc, aodhclient is just one possible client
15:38:37 <gordc> one possible client of aodh?
15:38:55 <sileht> gordc, I still use curl as client ;)
15:38:56 <gordc> pradk: i guess we'll need to start gnocchi as well
15:39:18 <pradk> yea sounds like it
15:39:20 <sileht> gordc, ceilometerclient is still a client too
15:39:57 <ildikov> will that be something we maintain longer term?
15:40:13 <ildikov> I mean the support in ceilometerclient
15:40:22 <liusheng> sileht: same question
15:40:22 <gordc> sileht: it still seems weird i hardcode rule in clients and know how to validate gnocchi alarms. but the server itself has no idea how to handle gnocchi alarm and needs connection to gnocchi
15:40:44 <sileht> gordc, I don't see the issue
15:40:49 <gordc> ildikov: alarms in ceilometerclient is deprecated (as of aodhclient)
15:40:59 <gordc> it won't be removed... it's just there.
15:41:05 <ildikov> gordc: ok, cool, tnx for confirming
15:41:19 <gordc> sileht: why can't we hardcode it in aodh server since it's already hardcoded in aodhclient?
15:42:03 <gordc> sileht: *shrugs* i use client more than curl. :) no biggie
15:42:04 <sileht> gordc, the rules type can be hardcoded, but the list of aggregation methods must not
15:42:31 <gordc> but the rules are based on aggregation_methods?
15:42:38 <sileht> gordc, and if you want to remove the query validation, we need to duplicate all the validation from Gnocchi into aodh
15:43:08 <gordc> we can skip.lol i'm just asking random questions now
15:43:25 <gordc> any other aodh topics?
15:43:48 <gordc> #topic ceilometer topics
15:43:50 <pradk> gordc, so whats the consensus here? should we proceed without functional tests now and add them subsequently?
15:44:03 <gordc> pradk: let's enable gnocchi in aodhclient functional tests too
15:44:19 <gordc> ljxiash: still there?
15:44:25 <ljxiash> ye
15:44:27 <ljxiash> yes
15:44:44 <pradk> k i'll poke sileht to see how to enable gnocchi in functional tests
15:44:56 <sileht> pradk, sure
15:46:06 <gordc> ljxiash: want to discuss your spec?
15:46:28 <gordc> or you just making people aware of it?
15:46:53 <ljxiash> yes, I think I should first invite some neutron member to review this spec.
15:47:09 <_nadya_> gordc: let me ask neutron folks to take a look
15:47:09 <gordc> ljxiash: agreed. i don't know enough myself.
15:47:14 <gordc> _nadya_: thanks!
15:47:20 <gordc> #link https://review.openstack.org/#/c/273475/
15:47:23 <ljxiash> _nadya_: thank you!
15:47:24 <gordc> if you understand neutron
15:47:54 <gordc> ljxiash: my only comment afterwards is try to list what you plan to accomplish this cycle. i'm not sure if we can get it all in (but maybe we can)
15:47:56 <ljxiash> I really appreciate any help.
15:48:30 <liusheng> I have asked a neutron guy to take a look, if he have time.
15:48:44 <gordc> cool cool. anything else ceilometer related?
15:48:47 <_nadya_> liusheng: all neutron team will come. lol
15:48:56 <gordc> _nadya_: more the better.
15:49:06 <liusheng> _nadya_: welcome!
15:49:18 <gordc> #topic gnocchi topics
15:49:31 <gordc> sileht: jd__: anything we need to discuss?
15:49:36 <gordc> 2.0 on hold?
15:49:39 <jd__> we don't discuss, we execute
15:49:45 <llu-laptop> sileht: jd__: one question, i'm working on bug #1518338 of adding new resource types for snmp related metrics, when I see sileht's patches for resource-type-api, should I go on or should I leverage the new dynamic resource type support?
15:49:47 <openstack> bug 1518338 in Gnocchi "Ceilometer doesn't build the data correclty for Gnocchi when you send hypervisors HW metrics using snmpd and agent-central" [High,In progress] https://launchpad.net/bugs/1518338 - Assigned to Lianhao Lu (lianhao-lu)
15:49:47 <jd__> yeah we still have some work for 2.0
15:49:57 <jd__> but it should comme soon :)
15:50:10 <jd__> llu-laptop: depends if you want it in 2.0 or 2.1?
15:50:18 <jd__> llu-laptop: sileht branch is not going in for 2.0
15:50:50 <llu-laptop> what's the 2.0 cut date?
15:51:20 <llu-laptop> and what's the expected date for 2.1?
15:51:22 <gordc> tbd
15:51:23 <llu-laptop> :)
15:51:29 <gordc> and tbd
15:52:42 <gordc> i imagine once we get some feedback from testers, we can move forward.
15:53:16 <gordc> #topic open discussion
15:53:41 <llu-laptop> anyone noticed many gate failure today?
15:53:48 <gordc> llu-laptop: every day
15:54:02 <gordc> lol is it neutron job?
15:54:35 <gordc> llu-laptop: i did notice this https://bugs.launchpad.net/ceilometer/+bug/1541555
15:54:36 <openstack> Launchpad bug 1541555 in oslo.messaging "dispatch errors with oslo.messaging 4.1.0" [Undecided,New]
15:54:51 <gordc> sileht: i didn't start debugging it htough
15:54:55 <gordc> though*
15:54:59 <_nadya_> llu-laptop: I saw a lot of tempest failures about too long notification procession. Is it still happening?
15:55:41 <llu-laptop> don't know I just saw a neutron one, and don't know what's going wrong in http://logs.openstack.org/46/273346/8/gate/gate-tempest-dsvm-ceilometer-postgresql-full/96cca2d/
15:56:26 <dims_> gordc : that one still uses 4.0.0 http://logs.openstack.org/70/274170/3/gate/gate-telemetry-dsvm-integration-aodh/4ef96b5/logs/pip-freeze.txt.gz
15:56:28 <sileht> gordc, hum interesting
15:56:47 <dims_> upper constraints change has not gone in yet
15:57:09 <gordc> dims_: oh... well it's happening for 4.0.0 then. :)
15:57:40 <dims_> :) 4.1.0 is going in as we speak - https://review.openstack.org/#/c/276058/
15:58:04 <dims_> gordc : so you may want to wait a day and blame 4.1.0 :)
15:58:08 <gordc> ack. i'll keep an eye out to see if anything changes
15:58:43 <dims_> gordc : aye aye
15:58:50 <gordc> anythiing else?
15:59:28 <gordc> thanks folks. remember, don't plan for anything to merge last week of cycle becuase of gate flood
15:59:32 <gordc> #endmeeting