Wednesday, 2014-02-12

openstackgerritgordon chung proposed a change to openstack/ceilometer: Alembic migrations not tested
*** thomasem has quit IRC00:15
openstackgerritgordon chung proposed a change to openstack/ceilometer: rename meter table to sample
*** promulo has joined #openstack-ceilometer00:22
*** xmltok has quit IRC01:11
_cjones_Anyone know why I don't see any events on the ceilometer-collector.01:21
*** promulo has quit IRC01:27
*** promulo has joined #openstack-ceilometer01:27
*** _cjones_ has quit IRC01:45
*** xianghui has joined #openstack-ceilometer02:13
openstackgerritShuangtai Tian proposed a change to openstack/ceilometer: Use the module units to refer bytes type
*** flwang has quit IRC03:00
*** flwang has joined #openstack-ceilometer03:04
*** promulo has quit IRC03:22
openstackgerritDazhao Yu proposed a change to openstack/ceilometer: Make sure use IPv6 sockets for ceilometer in IPv6 environment
*** xianghui has quit IRC05:13
*** xianghui has joined #openstack-ceilometer05:30
*** gordc has quit IRC05:39
*** sayali has joined #openstack-ceilometer05:50
*** ildikov_ has quit IRC05:54
*** _nadya_ has joined #openstack-ceilometer06:01
*** saju_m has joined #openstack-ceilometer06:03
openstackgerritJenkins proposed a change to openstack/ceilometer: Imported Translations from Transifex
openstackgerritJia Dong proposed a change to openstack/ceilometer: Implement meter query by 'counter_volume' field
*** jaypipes has quit IRC06:12
*** jaypipes has joined #openstack-ceilometer06:15
*** _nadya_ has quit IRC06:24
openstackgerritZhang Jinnan proposed a change to openstack/ceilometer: Remove blank line in docstring
*** eglynn has joined #openstack-ceilometer07:01
*** eglynn has quit IRC07:09
*** ildikov_ has joined #openstack-ceilometer07:14
*** eglynn has joined #openstack-ceilometer07:25
*** eglynn has quit IRC07:32
*** eglynn has joined #openstack-ceilometer07:47
*** eglynn has quit IRC08:07
*** julienvey_ has joined #openstack-ceilometer08:17
*** boris-42_ has joined #openstack-ceilometer08:30
*** eglynn has joined #openstack-ceilometer08:34
*** mihgen has joined #openstack-ceilometer09:02
*** yassine has joined #openstack-ceilometer09:18
*** flwang has quit IRC09:42
*** mihgen has quit IRC09:55
*** mihgen has joined #openstack-ceilometer10:02
eglynnildikov_: good morning!10:04
ildikov_eglynn: good morning :)10:04
ildikov_eglynn: is it a nice day for pipelines?10:04
eglynnhi, just a couple of quick thoughts on your project filtering in the pipeline idea10:04
eglynnildikov_: ... not here in Dublin it's not! ;)10:05
eglynnildikov_: ... back to our usual rain 24/710:05
ildikov_eglynn:... I haven't seen the sun since I arrived back...10:05
eglynn... /me digs for ML reference10:05
ildikov_eglynn: yep, that's it10:06
ildikov_eglynn: at the end we agreed on having a projects list in the pipeline config, like we have resources now10:06
eglynnildikov_: yeah ... so some off-the-cuff ideas10:06
eglynnpulling back a little first to see the big picture10:07
eglynn... there are two kinda separate cases here10:07
eglynn1. notification handling10:07
eglynn2. pollster polling10:07
eglynnin the case of #1, we can't AFAIK restrict which projects we get notified about10:08
eglynnso regardless of the pipeline config, we're gonna be told about all projects anyway10:08
eglynnso in that case, the filtering really is more like ...10:08
eglynn"throw the samples related to all but these projects on the floor"10:09
eglynn... as opposed to ...10:09
eglynn"completely ignore projects not on this list"10:09
eglynn(we can't completely ignore as we get the notifications anyway)10:09
ildikov_ this filtering is more about specify project specific settings and for restricting projects from getting info10:09
eglynnso here's a wacky thought ...10:09
eglynn... this kind of discarding of samples could easily be done as things stand in a simple transformer10:10
eglynn... but just park that for a second10:10
eglynnthen there's the case #2, actual polling10:11
eglynnhere I'd imagine the goal is not to incurr the work of polling resources that relate to projects we're not interested in10:11
eglynnassuming that is so ... there are two sub-cases, I think10:12
ildikov_how do you mean incur the work?10:12
eglynn... well I mean ceilometer agents (compute, central etc.) not incurring the workload of say polling a resource belonged to a non-interesting project10:13
ildikov_the whole idea here is to fine grain the piepline config, by making it possible to specify project specific settings10:13
ildikov_ah, ok, yes10:13
ildikov_so we could specify different polling inetrvals for instance10:14
eglynnsure, there's that10:14
eglynnor you could restrict to only metering certain projects10:14
eglynnand for others do no metering at all10:14
eglynnsay an internal cloud is spun up by the engineering dept10:15
ildikov_yes, it is a possibility too, but it is still the admin's choice as the config is still in pipeline.yaml10:15
eglynnengineers get un-billed access, but cloud is also shared with other departments, who are billed for their usage10:16
eglynnyeap, sure ... all driver by the admin's choices as expressed in the pipeline.yaml10:16
eglynnall *driven by10:16
ildikov_yes, it's a good example for the non-metering10:16
eglynnso, two two sub-cases for polling that I mentioned above10:17
ildikov_and it also means to have less data in the DB10:17
eglynn2.(a) polling REST APIs that don't allow easy constraint to a set of project IDs10:17
eglynne.g. I don't think glance API allows you to say:10:17
eglynn"gimme all the image IDs that belong to projectA OR projectB OR projectC" in a single query10:18
ildikov_hm, it's time to improve their query too ;)10:18
eglynnso in that case maybe easiest to create samples for *all* images and then just discard uninteresting samples in a transformer10:18
ildikov_but it is still our choice that which data we will store in the db10:19
eglynnuninteresting == belonging to projects not on our list of project IDs10:19
ildikov_yes, it's a vali option10:19
eglynnsure ... I'm suggesting that transformers provide us an existing mechanism for doing the discard10:20
eglynnthen the other sub-case of polling10:20
eglynn2.(b) "two phase polling" ... i.e. resource discovery followed by per-resource polling10:20
eglynnexample: compute agent discovers local instances first via nova-api, then polls the local hypervisor for each instance10:21
eglynnin that case you prolly wanna avoid the work of polling the hypervisor about instances owned by non-interesting projects, right?10:21
eglynn... /me is thinking that avoidance could be done by chaining resource discovery extensions10:22
eglynnfor context ...10:22
eglynnintroduces the idea of a resource discover extension10:22
eglynne.g. the compute agent's call out to the nova-api will be encapsulated in a new "local_instances" extensions10:23
eglynn*extension, singular10:23
eglynnsimilarly for say disocvering baremetals via ironic API10:24
eglynnso what if these discovery extensions could be chained together?10:24
eglynn... e.g. "local_instances" followed by "project_filter"10:25
eglynn... the resources discovered by one extension would be fed into the next on the chain10:25
eglynn... the next could add or subtract10:26
eglynnreason I got thinking along these lines was10:26
eglynnthat it seemed project filtering could be done in a transformer in every other case10:27
eglynnapart from what I called 2(b) above10:27
eglynnwell it could be done in 2(b) as well10:27
eglynnbut only sub-optimally10:27
ildikov_but if I would set the interval too, then it is not the transformer's business10:28
ildikov_or I'm still missing somethnig from the big picture :)10:30
eglynntrue, but I don't think setting the interval on a per-project basis really helps much in any case other than 2(b)10:30
eglynnso the interval is meaningless in the case of notifications, right?10:30
eglynn(... because we don't get to control when service emit notifications)10:30
ildikov_in case of notifications yes10:31
eglynnsometimes that's driven by async lifecycle event10:31
eglynnsometimes by a cadence defined in the other service10:31
ildikov_what about polling?10:31
eglynnyep, so in the polling case, /me was thinking 2(a) and 2(b) should considered separately10:32
eglynnin 2(a) it don't really help us much to only call out to glance for images owned by certain projects at a particular polling interval10:33
eglynnbecause the glance API will tell us about *all* images anyway10:33
eglynnso might as well just create samples for them all for every polling interval, then discard in the transform chains as necessary10:34
eglynnso it seems to me that only case 2(b) allows us to save any real work by restricting our attention to certain projects10:35
ildikov_saving work is not the only benefit of the idea, I think :)10:35
eglynnyeah true10:37
ildikov_Ceilometer have to deal with a large amount of data and if we do not store samples that are not needed, it can also be a benefit10:37
*** flwang has joined #openstack-ceilometer10:37
eglynnyeap, though discarding samples in a transformer would give that benefit also, right?10:38
eglynn... i.e. the discard would happen in the originating agent10:38
eglynn... central, compute, whatever10:38
eglynn... before any samples are stored10:38
ildikov_the interval also affects the amount of stored data10:38
eglynnnot if the unneeded data is discarded before it stored?10:39
eglynnsimilarly restrictive resource discovery would avoid the samples being created in the 1st place, so again not stored10:40
ildikov_we can have projects, of which for instance the cpu util is not interesting in every 10 seconds, just in every two hours10:42
ildikov_or cpu samples itself10:42
eglynnone sec10:43
eglynnildikov_: ... sorry, back now10:45
eglynnso yeah, let's drill down into that example10:45
eglynn(projectA, projectB) => gimme cpu_util every 10s10:45
eglynn(projectX, projectY) => gimme cpu_util every 7200s10:46
eglynnso we need two pipelines10:46
eglynnone restricted to (projectA, projectB) with interval set to 1010:46
ildikov_yes, with the same transformer10:47
eglynnone restricted to (projectX, projectY) with interval set to 720010:47
*** xianghui has quit IRC10:49
ildikov_so in your bp, we would need two sources, right?10:49
eglynnbut let's stick to old unified (source+sink) pipeline for the second10:51
eglynn... just trying to think thru how the per-project filtering would work10:51
eglynnso we'd need to do the restriction somewhere in the pipeline, right?10:53
eglynnthe question is, where in the current scheme would that make most sense to do?10:54
ildikov_yes, I thought it before the transformers, but as you mentioned earlier, in some cases it could be done there10:54
eglynnso before the transformers, is also after the samples have been initially created by the pollsters10:54
ildikov_but not in other case we could limit the polling itself10:55
ildikov_if we do not have to fight for instance with the limitations of Glance API10:55
eglynnyep in the example of (projectX, projectY) we'd want to ensure that we don't hit the hypervisors for info about those instances every 10s10:55
eglynnwhen we're only gonna use that info every 7200s10:56
*** saju_m has quit IRC10:57
ildikov_so what's the next step from here?10:58
eglynnok so consider how the compute agent currently works10:58
eglynnsorry /me was lost in a thought there for a sec! ;)10:59
eglynnso for each unique pipeline interval10:59
eglynnthe compute agent sets up one of these puppies  ...10:59
ildikov_np, it happens to me too ;)10:59
eglynnthis task goes out and grabs *all* instances running on the local host11:00
ildikov_do we have a chance to identify the instance-project relation there?11:01
eglynnand feeds each of these to each of the pollsters that match the meters filter on the pipeline in question11:01
ildikov_can we make the puppy to know about which instance it should grab?11:01
eglynnyes, we could potentially check the 'projects' element of the pipeline within that outter for loop11:02
eglynnwell actually it's a tad messier that that /me thinks11:04
eglynnthe task is not per-pipeline11:04
ildikov_yes, I've just had a thought that it may not per-pipeline11:05
eglynnthe task can have multiple pipelines, all added to the publish_context11:05
eglynnso the task is really per-*interval*11:05
eglynnso we'd have to pick out the pipelines from the publish context that have a 'projects' element that matches the current instance and only publish samples on that11:07
ildikov_you mean one task runs periodically?11:07
eglynnconsider ...11:07
eglynn(projectA, projectB) => gimme cpu_util every 10s11:07
eglynn(projectX, projectY) => gimme cpu_util every 7200s11:07
eglynn(projectQ, projectR) => gimme cpu_util every 7200s11:07
eglynnthen we'd have *two* polling tasks11:08
eglynnone for interval 10s (single pipeline)11:08
eglynn one for interval 7200s (two pipelines)11:08
eglynneach task runs periodically11:09
eglynnjust a different period in each case11:09
eglynnso it seems like this logic might be a tad intrusive to the pollster manager11:11
eglynnto have to recognize which instances should be excluded in each case11:11
eglynne.g. maintain multiple publish contexts with different non-disjoints subsets of the the pipeline in each case11:12
eglynnnow fast-forward to i3 when we have sources and sinks decoupled in the pipeline config11:12
eglynn... /me hopes!11:12
ildikov_if that would help, than /me hope too ;)11:13
eglynn... then the interval would be defined on just the source, not the entrire pipeline11:13
eglynnBTW this per-project filtering is for Juno, right?11:14
eglynn(as opposed to i3)11:14
ildikov_I wasn't from the beginning sure that the current solution will support the idea without any change11:14
ildikov_yes, it is definitely for Juno11:14
eglynnmeh! ... reading back /me realizes /me can't type11:15
ildikov_we still have work and some waiting tasks with the complex query, so that definitely would be too much now11:15
eglynnI meant ... maintaining multiple publish_contexts each with a different non-disjoints subset of the set of pipelines with the same interval11:16
ildikov_(I'm still a bit sad that Juno is not Jekyll...:) )11:17
ildikov_np, I more or less got, what you wanted to write there :)11:18
eglynnI was torn on Jekyll11:18
eglynn... wasn't sure whether the tie in with Mr Hyde was a net positive11:19
eglynnMr. Hyde == AWS ?11:19
eglynn... anyhoo, back on topic11:19
ildikov_anyway, my vote was on Jekyll :)11:20
ildikov_BTW, I planned before your pipeline bp to figure out something for a better structure of pipeline def11:20
ildikov_so we are at the point of figuring out what will be changed by the sources and sinks, right?11:22
eglynnso I'm thinking rather than having that complexity in the polling tasks11:22
eglynn(as described above)11:22
eglynninstead it's pushed into the resource discovery phase11:23
eglynnso we filter out those uninteresting resources before the sink does any work with them11:23
eglynnfor polling sinks without explicit resource discovery11:23
eglynnor for notification-driven sinks11:24
eglynnwe filter out in the transformer chain11:24
ildikov_hm, will this solve the interval related didfficulties?11:25
ildikov_and what do you mean here by transformer chain?11:26
ildikov_the transformer section in each sink?11:26
eglynnwell not in every sink11:26
ildikov_sorry, I'm a bit lost :)11:27
eglynnok, forget transformers for a sec11:27
ildikov_not in every, but the ones in the sink definition part11:27
*** mihgen has quit IRC11:27
eglynnI think we could add a new abstraction in the source to simplify things11:28
eglynnin fact that would be much better from the source/sink separation PoV11:29
ildikov_yes, I had a similar idea, when I saw your design11:29
ildikov_but one bad news, is that I need to run first, to get my spine in shape again11:30
*** ilyashakhat has joined #openstack-ceilometer11:30
ildikov_first=soon :)11:30
eglynnI'm up against the shot-clock too11:30
eglynnso let's park it there for the moment11:30
eglynndo you feel progress?11:30
eglynnthat's forward progress11:31
eglynnas opposed to backward progress ;)11:31
ildikov_it seemed to me that my idea is not impossible, even with your design11:31
ildikov_I hope you agree :)11:32
eglynnright, certainly not impossible11:32
eglynnwe just need to figure out where exactly to do the filtering11:32
eglynnand think thru a few typical cases11:32
ildikov_and I think it would be usefull too, just needs more work to figure out how it should work exactly11:32
eglynnit's a bit clearer to me now, I think11:32
ildikov_yes, definitely11:33
eglynnthe discarding transformer idea was a blind alley I think11:33
eglynnso we need to work out a point in te source handling that allows filtering to occur11:33
eglynnin as generic a way as possible11:33
ildikov_yes, this discussion helped me too11:33
eglynnyep, I think that's do-able11:33
ildikov_yes, that's the next step in the process11:34
eglynnI'll keep it in mind as I dig into the source/sink impl over the next few days11:34
ildikov_cool! :)11:34
eglynnlet's aim to circle back on this before EoW11:34
ildikov_that would be great, we can continue the discussion, if you will have some time and also, if you reach a point during the implementation that could be inetersting from this PoV11:35
ildikov_ok, sounds good, tanks11:35
eglynnI *hope* to have a WIP patch posted by EoW, that should help focus further discussion11:35
eglynnwell thank you too, the discussion made it all clearer for me as well11:36
ildikov_k, great :)11:36
ildikov_it's good to hear that I not just wasted your time, with crazy ideas and stupid questions :)11:37
eglynnno its really good to tease these things out on IRC before much code is written11:38
ildikov_yes, that's absolutely true, it makes coding life much easier in most of the cases :)11:38
ildikov_k, I need to run now, I'll be super late again ;)11:39
*** saju_m has joined #openstack-ceilometer11:40
ildikov_thanks again and then continue this week11:40
ildikov_laters :)11:40
*** ildikov_ is now known as ildikov_afk11:40
*** saju_m has quit IRC11:49
*** mihgen has joined #openstack-ceilometer11:55
*** julienvey_ has quit IRC11:58
*** julienvey_ has joined #openstack-ceilometer12:04
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements in operator for complex query functionality
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements "not" operator for complex query
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements metadata query for complex query feature
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements field validation for complex query functionality
*** julienvey_ has quit IRC12:09
*** julienvey_ has joined #openstack-ceilometer12:18
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements in operator for complex query functionality
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements "not" operator for complex query
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements metadata query for complex query feature
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Implements field validation for complex query functionality
*** eglynn has quit IRC12:33
*** promulo has joined #openstack-ceilometer12:40
*** Alexei_987 has joined #openstack-ceilometer12:45
*** sayali_ has joined #openstack-ceilometer12:47
*** sayali has quit IRC12:49
*** eglynn has joined #openstack-ceilometer12:58
*** saju_m has joined #openstack-ceilometer13:05
*** saju_m has quit IRC13:06
*** saju_m has joined #openstack-ceilometer13:07
*** jdob has joined #openstack-ceilometer13:28
*** prad_ has joined #openstack-ceilometer13:29
*** thomasem has joined #openstack-ceilometer13:31
*** thomasem_ has joined #openstack-ceilometer13:32
*** thomasem has quit IRC13:32
*** ildikov_afk is now known as ildikov_13:34
ildikov_sileht: hi. are you around?13:34
*** julienvey_ has quit IRC13:45
*** julienvey_ has joined #openstack-ceilometer13:47
*** saju_m has quit IRC14:21
*** mihgen has quit IRC14:27
*** tongli has joined #openstack-ceilometer14:27
tongli@eglynn, ping.14:28
eglynntongli: sup?14:28
tongli@eglynn, thanks so much for your comments. can you point me to the place where the alarm action takes place?14:28
tongli@eglynn, the alarm_notifier agent actually does the alarm action? right?14:30
eglynntongli: for webhooks ^^^ or for logging actions vvv14:30
eglynntongli: yep
tongli@eglynn, they do not change the alarm state in anyway, even after the action has been performed?14:31
eglynntongli: exactly, as I stated on gerrit14:32
tongliin that case, how does notifier know when to perform the action? and know if the action has been performed?14:32
tongli@eglynn, I think that is where I was lost.14:33
eglynntongli: last comment on
eglynntongli: the evaluator tells the notifier to carry out the action via an RPC message14:33
eglynntongli: the notifier is told the old state and the new state14:34
eglynntongli: but the state transformation is not the notifier's concern14:34
eglynntongli: that's the concern of the evaluator14:34
tonglievaluator does that.14:34
*** gordc has joined #openstack-ceilometer14:34
tongliok. make sense.14:34
tongli@eglynn, now how will notification alarm maintain its state?14:35
eglynntongli: one sec14:35
tongli@eglynn, see something, tell the notifier to take action, seems to me does not even need to update the state14:35
tongli@eglynn, I mean the evaluator monitors the notification, when conditions met, simply tell the notifier to take action, the alarm state does not even need to be persisted or updated.14:36
eglynntongli: alarm action and alarm state are separate concepts14:37
eglynntongli: the current alarm state is maintained persistently14:37
eglynntongli: the action is a notification that the state has changed14:37
tongli@eglynn, yeah, what I mean is that seems to me that the notification alarm state does not even matter.14:38
eglynntongli: no the state does matter, as the action includes information about the state14:38
*** saju_m has joined #openstack-ceilometer14:38
eglynntongli: however the action is decoupled from the state transformation14:39
eglynntongli: the fact that the action has occured *does not* mean that the state flips back14:39
tongli@eglynn, yeah, I got that , but what I am not clear about is how the state get changed from one to another.14:39
tongli@eglynn, especially in the notification alarm case.14:40
*** jmckind has joined #openstack-ceilometer14:40
eglynntongli: as I said above, the alarm evaluator is responsible for detecting when state transitions occur14:40
eglynntongli: in the case of the threshold alarm, it refers to the state as determined by the observed trend in the statistics14:41
eglynntongli: ... outside threshold => state flips to ALARM14:41
eglynntongli: ... inside threshold => state flips to OK14:41
tongli@eglynn, notification alarm is bit different, it is like for each notification comes in, evaluator knows if the condition is met or not.14:41
eglynntongli: that's why I suggested in gerrit the idea of a time window14:41
tongli@eglynn, yeah, I got how threshold logic work.14:42
eglynntongli: so the ALARM state does not mean "I saw this notification at some time in the past"14:42
tongli@eglynn, so you mean when the specified time elapsed, the state changed to OK.14:42
eglynntongli: instead the ALARM state means "I saw this notification with the configured time window"14:42
eglynn*within the configured time window14:43
eglynntongli: is that clear?14:43
tongli@eglynn, yeah, or even combined with threshold, like within specified windows, these many notifications.14:44
eglynntongli: yeah, could be useful too14:44
tongli@eglynn, that is why I was thinking these should be sort of combined with threshold alarm.14:44
eglynntongli: seems to me sufficiently different from threshold alarm14:45
tonglijust with a time window?14:45
eglynntongli: ... i.e. threshold alarm are oriented to statistics14:45
eglynntongli: ... and periods14:45
eglynntongli: whereas this refers to notifications over a time window14:45
eglynntongli: vaguely similar, but still quite distinct14:45
tongli@eglynn, well, it could be over a time window (also period) with certain conditions (query).14:46
eglynntongli: now a time window would point you back to the events API for the alarm evaluator14:46
eglynntongli: instead of consuming the notifications as they arise14:46
eglynntongli: otherwise you'd need to maintain a persistent timestamp as to when the notification was seen14:47
tongli@eglynn, yeah, true,14:47
eglynntongli: ... otherwise wouldn't survive restarts14:47
silehteglynn, tongli sorry but for me having alarm stuff in the ceilometer-notification-agent looks weirds,14:47
eglynnsileht: agree14:47
tongli@eglynn, the time window just means starting now, I will watch these notifications.14:47
silehteglynn, tongli we don't know who is the owner of the notification at this step of the processing14:47
eglynnsileht: so I was suggesting using the events API instead of consuming notifications directly14:48
silehteglynn, +114:48
tongli@sileht, I do not think I understand what you mean by "do not know who is the owner..."14:49
*** mihgen has joined #openstack-ceilometer14:50
silehtIn ceilometer this is the role of a plugin.NotificationBase to parse a notification payload to retreive the user/project/resource id14:50
tongli@sileht, are we talking about same thing?14:51
silehttongli, how do we ensure tenant X cannot trigger an alarm on a notificaiton owned by tenant Y14:52
tongli@sileht, it should be in the evaluator.14:53
silehttongli, where is located the tenant_id in a notification ? that depends of the notification type14:54
tongliit becomes one of the condition. should the condition be any kind of the notification or the condition that requires match userid, ctenatn_id, etc.14:54
tongliu'_context_project_id': u'df58a6666d5440aca0582287494fde5a',  u'_context_tenant_id': u'df58a6666d5440aca0582287494fde5a',  u'_context_user': u'e532f7d9bdb341d0a4b41098d524743e',  u'_context_user_id': u'e532f7d9bdb341d0a4b41098d524743e',14:54
tonglinotice the information in the raw notification,14:55
tongliany of these can be used as condition to match it up.14:55
tongliit should not be limited to a specific tenant, or project or whatever.14:55
tonglithe alarm definition should dictate that.14:56
silehttongli, I don't think you can use the context for that14:57
silehttongli, Imagine the case of a ReserllerAdmin do something on behalf an other user14:57
silehtthe context have the RessellerAdmin ids, but the payload in the notification have the 'on behalf user' ids14:58
tongli@sileht, then in your alarm definition you define what kind of notification you would like to match up.14:59
tongliif you are looking for notification coming from that ResellerAdmin, then the notification should have some information of these.15:00
silehttongli, sure15:00
tongli@sileht, if the raw notification does not have information, there is no way that the event can fabricate15:01
tongli@sileht going with the raw notification is a lot flexible, IMHO.15:02
Alexei_987jd__: Hi what's the meeting agenda for today?15:02
jd__Alexei_987: the one on the wiki page15:02
Alexei_987jd__: it's seems that we have outdated agenda on the wiki15:02
jd__let me check15:03
jd__no that's our default agenda15:03
Alexei_987jd__: I propose to add 1 topic
silehttongli, the raw notification have and all Notification plugin already have the code to parse some kind notification to extract this information15:03
jd__Alexei_987: you think this is a worth a meeting?15:04
Alexei_987jd__: I propose to discuss "I think that we should be ready to tolerate redundant data. Ceilometer  is meant to be used at large scale and in such condition data redundancy  is much better than low performance."15:04
tongli@sileht, yeah, the notification alarm simply takes the data and use it. Not do duplicate work.15:04
Alexei_987jd__: and possible changes in data structure15:04
Alexei_987jd__: but I guess it's better to be discussed on the summit15:05
jd__Alexei_987: I'm having a hard time how this can be discussed in 20 minutes on IRC honestly15:05
jd__yeah or a mailing list thread15:05
Alexei_987jd__: agreed15:05
silehttongli, I just want to be sure you are aware about 'Tenant Y' should not allowed to set alarm on 'Tenant X' notification (except if Tenant Y is a admin)15:05
Alexei_987jd__: btw should I start publishing patches related to real backends stuff?15:06
tongli@jd__, can you shed some lights on how notification alarm state should be changed from one to another?15:06
jd__Alexei_987: hm why do you even ask? :)15:06
Alexei_987jd__: I still have some test failures :)15:06
jd__tongli: not sure, likely sileht or eglynn might have a better idea than me15:06
jd__Alexei_987: well you can still publish them and open bugs to track the issues that can be fixed in different patches15:06
Alexei_987jd__: and I cannot make Postegres 100% working15:06
Alexei_987jd__: ok I'll updload first series today15:07
tongli@jd__, ok, in that case, we go with the time window.15:07
eglynntongli: my suggestion I made above ^^^15:08
tongli@eglynn, yeah the time window, right?15:08
eglynntongli: ... yeap, evaluated against the events API (as opposed to notifications directly consumed)15:09
tongli@eglynn, in that case, it will be part of the evaluator agent and loops forever?15:10
tongli@eglynn, lot of load on the api server.15:10
tongli@eglynn, I like the time window thing, but keep calling the event api (second hand) and add extra load for API servers,15:11
tongli@eglynn, I am not sure I like that.15:11
eglynntongli: depends on how many notification alarms, how constrained their conditions are15:11
tongli@eglynn, true, but with raw notification,we do not have that problem.15:12
eglynntongli: otherwise you'll have to remember received timestamps for the notifications somehow15:12
eglynntongli: ... in a way that survives across restarts15:12
*** eglynn is now known as eglynn-afk15:13
tongli@eglynn, should we pay that much for it though. do not really know.15:15
ildikov_sileht: will you have a few minutes for me? I have just one short question regarding to the sphinxcontrib-pecanwsme project.15:15
silehtildikov_, sure15:15
ildikov_sileht: thanks15:15
ildikov_sileht: I have a fix for this project, which removes the trailing slash from the end of the URLs (webprefix)15:16
ildikov_sileht: what is the way of proposing this fix to that project?15:16
ildikov_sileht: contributing there was kind of out of scope for me until now15:17
silehtildikov_, clone the repo on github, make your change and do a pull request15:18
silehtildikov_, this lib is not yet in stackforge so, this is the old github way for this one15:18
ildikov_sileht: ok, thanks15:19
ildikov_sileht: what is the lifecycle of the project?15:19
ildikov_sileht: I mean I need this fix for a Ceilometer bugfix15:19
ildikov_sileht: if the fix is accepted, will this be available for Ceilometer too, or is there a release mechanism?15:20
silehtildikov_, it depends on dhellmann_ , but in the past it have release a new version each time a pull request have been merged15:20
ildikov_sileht: thanks very much, then I will follow the old github process and wait for response15:21
*** mihgen has quit IRC15:22
openstackgerritA change was merged to openstack/ceilometer: Updated from global requirements
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: store recording timestamp
nprivalovajd__, hi! Please approve . I'm afraid that I will be ought to rebase again and all votes disappear15:46
jd__nprivalova: done15:47
nprivalovajd__, thanks!15:47
*** prad_ has quit IRC15:49
*** sayali_ has quit IRC15:53
*** prad_ has joined #openstack-ceilometer15:56
*** jaypipes has quit IRC15:56
*** sayali_ has joined #openstack-ceilometer15:56
*** jaypipes has joined #openstack-ceilometer16:02
*** xmltok has joined #openstack-ceilometer16:14
*** dhellmann_ is now known as dhellmann16:17
dhellmanngordc: ping16:24
*** thomasem_ is now known as thomasem16:24
*** prad__ has joined #openstack-ceilometer16:26
*** prad_ has quit IRC16:28
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Refactored fake connection URL classes
gordcdhellmann: thanks for the heads up.16:45
gordcdhellmann: i've approved dims patch so it should be merged in.16:46
gordcis that a gate test for Nova?16:47
*** _cjones_ has joined #openstack-ceilometer16:51
dhellmanngordc: I'm actually not sure16:57
dhellmanngordc: dims had a patch? fungi brought this up in #openstack-infra a bit ago16:58
gordcdhellmann: yeah. we rolled it back this morning:
dhellmanngordc: ah16:58
gordcdhellmann: i'm assuming this fixes the issue. (it was the only change i checked in recently)16:59
dhellmanngordc: likely16:59
gordcdhellmann: i'll check with dims where that gate is being run. i assume all was ok since i didn't notice anything failing constantly.17:01
dhellmanngordc: ok, if that does fix the problem you can obviously close the bug :-)17:01
*** thomasem_ has joined #openstack-ceilometer17:03
*** thomasem_ has quit IRC17:04
*** thomasem has quit IRC17:05
*** gordc has quit IRC17:48
*** _nadya_ has joined #openstack-ceilometer17:48
*** saju_m has quit IRC17:50
*** saju_m has joined #openstack-ceilometer17:51
*** ildikov_ has quit IRC17:53
*** saju_m has quit IRC17:58
*** saju_m has joined #openstack-ceilometer18:00
*** eglynn-afk has quit IRC18:07
*** ildikov_ has joined #openstack-ceilometer18:31
*** _nadya_ has quit IRC18:37
*** Alexei_987 has quit IRC18:49
*** saju_m has quit IRC18:51
*** _nadya_ has joined #openstack-ceilometer18:52
*** eglynn-afk has joined #openstack-ceilometer18:53
*** saju_m has joined #openstack-ceilometer19:01
*** julienvey_ has quit IRC19:03
_nadya_jd__, ping. or eglynn-afk19:05
openstackgerritJenkins proposed a change to openstack/ceilometer: Updated from global requirements
*** eglynn-afk has quit IRC19:05
promulohey guys, did any of you ever enabled healthnmon on a devstack installation?19:17
_nadya_I hope my message will not be lost. I cannot visit meeting today, so my update from tempest is the following19:22
_nadya_We have good progress here . But notification tests cannot pass through Jenkins. It's very strange that dsvm-postgress-full pass but dsvm-full doesn't. I'm talking mostly about . If someone has comments about this please let me know. Or just share on meeting, I will read the logs19:22
_nadya_lsmola: jd__, dhellmann, llu, sileht, ^^19:25
*** thomasem has joined #openstack-ceilometer19:27
*** _nadya_ has quit IRC19:45
*** eglynn-afk has joined #openstack-ceilometer19:47
*** saju_m has quit IRC19:48
*** julienvey_ has joined #openstack-ceilometer19:51
*** julienvey_ has quit IRC19:53
*** Alexei_987 has joined #openstack-ceilometer19:54
*** gordc has joined #openstack-ceilometer19:58
openstackgerritgordon chung proposed a change to openstack/ceilometer: sample table contains redundant/duplicate data
openstackgerritgordon chung proposed a change to openstack/ceilometer: Alembic migrations not tested
openstackgerritgordon chung proposed a change to openstack/ceilometer: rename meter table to sample
*** eglynn-afk has quit IRC20:28
*** promulo has quit IRC20:33
*** eglynn-afk has joined #openstack-ceilometer20:33
*** boris-42_ has quit IRC20:55
*** jdob has quit IRC20:58
*** jdob has joined #openstack-ceilometer20:58
*** boris-42_ has joined #openstack-ceilometer20:59
*** thomasem has quit IRC21:01
*** eglynn-afk is now known as eglynn21:01
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Removed pecan app ref to fix test tearDown cleanup process
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Fixed connection pooling in tests
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: * OSLO db.session fix
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Run unit tests against MySQL
*** jdob has quit IRC21:08
*** jdob has joined #openstack-ceilometer21:08
gordcdhellmann: so this is related to notifier middleware which doesn't work in Nova anymore because of oslo.messaging upgrade.21:26
jd__I saw that thread21:26
gordcdhellmann: when notifier middleware and audit  middleware are graduated into oslo.notifier and oslo.middleware... how are they called from nova/neutron/?21:26
jd__I think I'll work on porting it21:26
jd__gordc: I don't think middleware is really going to be graduated as it is21:27
jd__the notifier middleware should go in oslo.messaging IMHO21:27
gordcjd__: yeah, i wasn't sure what to do. i'm open to porting it over to oslo.messaging as well but not sure where to start.21:27
jd__if it's not already there (I don't recall)21:27
gordcjd__: yep. i don't think it's there but dhellmann said it will be moved there.21:27
jd__I'll try to do that21:28
*** nealph_ has joined #openstack-ceilometer21:28
gordcjd__: i was really just curious how the middleware would work after all this? do we still have audit middleware code replicated everywhere?21:28
jd__gordc: not just importing in paste.ini the right egg/module and that should work21:29
gordcjd__: if we pull out the code currently. to call notifier middlweare just add oslo.messaging.(middleware.)notifier to paste.ini and we're good?21:30
gordcjd__: and then the audit middleware would be... dhellmann said it would be moved to oslo.middleware lib but if not how would that work?21:31
gordcjd__: lol you gave me 50% of the answer :)21:33
dhellmanngordc: sorry, got distracted by a jenkins failure21:33
*** eglynn has quit IRC21:33
gordcdhellmann: hope it isn't my fault again.21:33
*** nealph_ has quit IRC21:33
dhellmanngordc: jd__ is right, the notifying middleware will go in oslo.messaging and the projects will import it from there instead of from their projects21:33
dhellmanngordc: no, one of my patches21:34
gordcdhellmann: cool cool.21:34
*** nealph has joined #openstack-ceilometer21:34
gordcdhellmann: so for audit middleware. if/when that goes to oslo.middleware. the projects would just need to add oslo.middlware to their requirements?21:34
dhellmannsileht: if ceilometer is blocked on an oslo review, please open a bug or blueprint for me so I can make sure it is on our review priority list21:35
silehtdhellmann, you mean ?
dhellmanngordc: I don't know the dependencies of that middleware off the top of my head, but probably21:36
dhellmannsileht: if that's the one, yes :-)21:36
dhellmannsileht: I was following up on the comment you made in the meeting while I looked away -- we have had other projects waiting for patches that we didn't know were blocking them21:36
gordcdhellmann: audit middleware just inherits notifier middleware right now...and overloads a few methods.21:36
silehtdhellmann, how I can make the link ? just a comment in the whtieboarD ?21:37
dhellmannsileht: that blueprint is already high and approved, so you've done what you need to -- I need to point it out to the oslo devs to encourage them to review the patches21:38
silehtdhellmann, ok cool :)21:38
_cjones_Hey guys, quick question. I'm using ceilometer, Havana release with mongodb. Why do I not see any network.outgoing.packets in Horizon?21:40
gordc_cjones_: do you see incoming.packets?21:41
_cjones_gordc: Negative, at least not in the gui. I see them in the db though.21:41
_cjones_Those choices just seem abseent. So I'm wondering if it is just an oversight, or if I haven't configured something correctly.21:42
*** promulo has joined #openstack-ceilometer21:42
gordc_cjones_: i'd need to take a look at horizon code, i would think they'd grab the entire list of meters but maybe they're just picking some off selectively.21:43
*** promulo has quit IRC21:43
*** promulo has joined #openstack-ceilometer21:43
_cjones_gordc: I suspect just a bug. No sense having you dig. I'll search and open one if need be.21:45
gordc_cjones_: ok, sure. have at it.21:46
openstackgerritA change was merged to openstack/ceilometer: Adds additional details to alarm notifications
openstackgerritRichard Su proposed a change to openstack/ceilometer: Add qpid-python to requirements.txt
*** tongli has quit IRC22:29
*** jdob has quit IRC22:37
*** jmckind has quit IRC22:43
*** promulo has quit IRC22:50
*** promulo has joined #openstack-ceilometer22:50
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Fixed DateTime in PostgreSQL
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Fix for get_statistics with postgresql
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Run tests against PostgreSQL
*** rwsu has quit IRC23:06
*** rwsu has joined #openstack-ceilometer23:08
_cjones_gordc: Are you around for another technical question?23:20
gordc_cjones_: i'll give it a try23:21
_cjones_What is the virt_inspector. And why/how is that used.23:23
gordc_cjones_:  this?
_cjones_Ah, I think I got it. I was just looking at the base class. yes.23:24
_cjones_It is just an inspector agent to retrieve information about the instances running in the virtual env.23:25
_cjones_Or am I wrong?23:25
gordcyeah. it's used by our compute pollster.
gordcyou'll see there it grabs a hypervisor inspector... depending on what you choose it'll use that inspector to gather metrics (libvirt by default)23:25
_cjones_Right. I'm just trying to repurpose Horizon for a project and wanted to know where to put my code.23:25
gordccool cool23:26
_cjones_I thought I could get away with just writing my own pollsters, but it looks like it will be a bit more work.23:26
_cjones_gordc Last one. You mentioned (libvirt by default), is there a setting in Ceilometer to change this?23:28
gordcbasically in ceilometer.conf add hypervisor_inspector = hyperv option... it's the only other one (vmware is coming)23:31
*** yassine has quit IRC23:41
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Fixed DateTime in PostgreSQL
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Run unit tests against MySQL
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Fix for get_statistics with postgresql
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Fixed connection pooling in tests
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: * OSLO db.session fix
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Run tests against PostgreSQL
Alexei_987how can one find out version of mysqld that is used on jenkins?23:47

Generated by 2.14.0 by Marius Gedminas - find it at!