Tuesday, 2014-01-28

*** openstack has joined #openstack-ceilometer00:02
*** eglynn-afk has quit IRC00:21
openstackgerritDoug Hellmann proposed a change to openstack/ceilometer: Use stevedore's make_test_instance  https://review.openstack.org/6949100:32
openstackgerritA change was merged to openstack/ceilometer: Correct spelling of logger for dispatcher.file  https://review.openstack.org/6885400:36
*** xianghui has joined #openstack-ceilometer00:53
*** xmltok has quit IRC00:56
openstackgerritZhiQiang Fan proposed a change to openstack/python-ceilometerclient: Remove unused import for print_function  https://review.openstack.org/6951801:03
*** _ruhe is now known as ruhe01:16
openstackgerritYuuichi Fujioka proposed a change to openstack/ceilometer: fix column name to Unit from Volume  https://review.openstack.org/6952401:17
openstackgerritYuuichi Fujioka proposed a change to openstack/ceilometer: Implements monitoring-network  https://review.openstack.org/6047301:20
*** gordc has quit IRC01:31
*** yfujioka has joined #openstack-ceilometer02:02
*** ruhe is now known as _ruhe02:03
*** urulama has quit IRC02:12
*** rwsu has quit IRC03:18
*** xianghui has quit IRC03:27
openstackgerritJia Dong proposed a change to openstack/python-ceilometerclient: Modify ceilometer client cmd line help info  https://review.openstack.org/6398403:51
*** sayali_ has joined #openstack-ceilometer04:10
*** sayali has joined #openstack-ceilometer04:10
openstackgerritA change was merged to openstack/ceilometer: Use stevedore's make_test_instance  https://review.openstack.org/6949104:13
*** sayali_ has quit IRC04:13
*** sayali_ has joined #openstack-ceilometer04:16
*** sayali_ has quit IRC04:21
*** sayali_ has joined #openstack-ceilometer04:22
*** ildikov_ has quit IRC05:57
openstackgerritJenkins proposed a change to openstack/ceilometer: Imported Translations from Transifex  https://review.openstack.org/6280806:02
*** sayali_ has quit IRC06:17
*** sayali_ has joined #openstack-ceilometer06:21
*** xianghui has joined #openstack-ceilometer06:41
*** eglynn-afk has joined #openstack-ceilometer06:57
*** flwang has quit IRC07:04
*** xianghui has quit IRC07:10
*** xianghui has joined #openstack-ceilometer07:13
openstackgerritJenkins proposed a change to openstack/ceilometer: Updated from global requirements  https://review.openstack.org/6823707:16
openstackgerritliusheng proposed a change to openstack/python-ceilometerclient: Reduce redundant parameter of some commands in CLI  https://review.openstack.org/6677607:24
openstackgerritliusheng proposed a change to openstack/python-ceilometerclient: Reduce redundant parameter of some commands in CLI  https://review.openstack.org/6677607:27
openstackgerritliusheng proposed a change to openstack/python-ceilometerclient: Reduce redundant parameter of some commands in CLI  https://review.openstack.org/6677607:28
*** flwang has joined #openstack-ceilometer07:34
*** ildikov_ has joined #openstack-ceilometer08:05
*** yfujioka has quit IRC08:07
openstackgerritA change was merged to openstack/ceilometer: Clean .gitignore  https://review.openstack.org/6936608:15
*** eglynn-afk has quit IRC08:17
*** eglynn-afk has joined #openstack-ceilometer08:20
*** s2r2 has quit IRC08:20
*** yassine has joined #openstack-ceilometer08:23
*** s2r2 has joined #openstack-ceilometer08:26
openstackgerritZhiQiang Fan proposed a change to openstack/ceilometer: Remove unnecessary code from alarm test  https://review.openstack.org/6955708:31
*** xianghui has quit IRC08:35
*** _nadya_ has joined #openstack-ceilometer08:42
*** _ruhe is now known as ruhe08:51
*** tian has joined #openstack-ceilometer08:52
tianhi, anyone can help me ?  how  to  use  the function notify in  ceilometer.computer.nova_notifier.py ?08:55
*** _nadya_ has quit IRC08:55
*** eglynn-afk is now known as eglynn08:59
*** lsmola_ has joined #openstack-ceilometer09:08
*** SergeyLukjanov_ is now known as SergeyLukjanov09:08
*** _nadya_ has joined #openstack-ceilometer09:13
*** jsuchome has joined #openstack-ceilometer09:20
*** ruhe is now known as _ruhe09:20
openstackgerritA change was merged to openstack/python-ceilometerclient: Raise traceback on error when using CLI and -debug  https://review.openstack.org/6923609:25
openstackgerritA change was merged to openstack/python-ceilometerclient: Update client to display data type of traits  https://review.openstack.org/6722409:25
*** Alexei_987 has joined #openstack-ceilometer09:31
*** _nadya_ has quit IRC09:40
*** _nadya_ has joined #openstack-ceilometer09:48
*** _nadya_ has quit IRC09:52
openstackgerritAlexei Kornienko proposed a change to openstack/ceilometer: Refactored session access  https://review.openstack.org/6785009:54
*** lsmola_ has quit IRC09:58
*** flwang has quit IRC10:03
*** ityaptin has quit IRC10:11
*** lsmola_ has joined #openstack-ceilometer10:13
*** lsmola_ has quit IRC10:43
*** flwang has joined #openstack-ceilometer10:46
*** lsmola_ has joined #openstack-ceilometer10:56
*** boris-42 has quit IRC11:11
*** SergeyLukjanov is now known as SergeyLukjanov_a11:19
*** SergeyLukjanov_a is now known as SergeyLukjanov11:19
openstackgerritIldiko Vancsa proposed a change to openstack/ceilometer: Add documentation for pipeline configuration  https://review.openstack.org/6935011:20
openstackgerritShuangtai Tian proposed a change to openstack/ceilometer: Use flavorid instead of id in the nova_notifier  https://review.openstack.org/6958212:00
*** boris-42 has joined #openstack-ceilometer12:31
openstackgerritA change was merged to openstack/ceilometer: Remove unnecessary code from alarm test  https://review.openstack.org/6955712:42
openstackgerritA change was merged to openstack/ceilometer: use common code for migrations  https://review.openstack.org/6945912:42
*** _ruhe is now known as ruhe12:51
*** jdob has joined #openstack-ceilometer13:04
*** flwang has quit IRC13:10
*** dhellmann_ is now known as dhellmann13:16
*** flwang has joined #openstack-ceilometer13:33
*** dhellmann is now known as dhellmann_13:36
*** dhellmann_ is now known as dhellmann13:37
gibilsmola_: are you around?13:44
lsmola_gibi: yes, hello13:44
gibilsmola_: hello! We did some thinking with ildikov_ about the instance statistic query you need13:45
*** prad has joined #openstack-ceilometer13:45
ildikov_lsmola_: hi13:45
lsmola_ildikov_: hello13:45
lsmola_gibi: ildikov_ have you figured it out ? :-)13:45
gibilsmola_: we made some progress...13:46
lsmola_gibi: seems like it will be quite a complex query13:46
gibilsmola_: I think in sql you would need something like the following13:46
ildikov_lsmola_: we tried to figure out, how could we solve your problem and we also tried to figure out a generic way, to achieve this13:46
gibiSELECT count(*), project_id FROM ( SELECT DISTINCT resource_id, project_id FROM Meter) GROUP BY project_id13:46
gibilsmola_: I think this would give you the number of resources that was active at a given period13:47
gibigrouped by project_id13:47
lsmola_gibi: ok13:47
lsmola_gibi: but how to run it for each period?13:47
gibithe current old query handles period as spearate querys in a for loop :)13:48
lsmola_gibi: oh god :-)13:48
gibilooping over the time window with the given period13:48
Alexei_987jd__: question about eventlet. Recently a patch landed that disabled it. Are we going to use this moment to enable py3 tests for ceilometer?13:48
lsmola_gibi: though i think I saw something like this13:48
jd__Alexei_987: I wish but it's not enough yet13:48
jd__the main event loop still depends on eventlet13:49
gibithe reason is that sql time handling is dialect specific13:49
lsmola_gibi: I know elastic search has feature called facets for this kind of query13:49
Alexei_987jd__: ah I see13:49
*** viktors has joined #openstack-ceilometer13:49
lsmola_gibi: also in postgre, you could do something like that with WINDOW I suppose13:49
lsmola_gibi: not sure about what SQL alchemy supports though13:50
lsmola_gibi: and not sure what mongo has13:50
gibilsmola_: I'm also not sure Mysql has WINDOW :)13:50
gibiso I think we have to keep the looping to be compatible with multiple SQL dialect13:51
lsmola_gibi: would be nice to have it in one query, cause the overview pages will shoot a lot of queries13:51
lsmola_gibi: that sounds pretty evil to me :-)13:51
lsmola_gibi: it will be sooo slow, nobody will actually use it :-(13:51
gibiit is, the other way would be to have different implementation for different SQL dialect13:51
lsmola_gibi: well ceilometer supports only mysql and postgree AFAIK13:52
lsmola_gibi: or more?13:52
lsmola_gibi: for queries like this, SQL alchemy is just not good enough13:52
gibilsmola_: good question :) I would think that it can suppot whatever SQL engine that SQL Alchemy supports13:53
lsmola_gibi: I think I saw they do the aggregation in the loop also13:53
lsmola_gibi: i believe Ceilometer has only those two in doc13:53
*** JCxMLnblFl has joined #openstack-ceilometer13:54
*** JCxMLnblFl has left #openstack-ceilometer13:54
gibilsmola_: I'm not sure which aggregation you refer to. Anyhow the period handling is on side of the problem13:54
gibilsmola_: the other is that for your query we need subqueries13:54
gibilsmola_: allowing generic support for subqueries makes the API very complex13:55
lsmola_gibi: yeah that is true13:55
lsmola_gibi: well the other option would be to expose a new meter13:56
gibilsmola_: for example you might want to define a WHERE clause (a filter expression) at every subquery separately13:56
lsmola_gibi: yeah13:57
gibilsmola_: separate meter is something we have to think about but it sounds a good idea as it would hide the complexity behind the descriptive name of the new meter13:57
ildikov_lsmola_: I haven't found a reference in sqlalchemy that it would have support for the period13:58
openstackgerritA change was merged to openstack/ceilometer: Updated from global requirements  https://review.openstack.org/6823713:58
lsmola_ildikov_: yeah you would probably have to implement it with WINDOW13:59
ildikov_lsmola_: we had a kind of similar idea, like this new meter, we could have an aggregated meter, or something like, we just weren't sure if it will be accepted as a solution for your need13:59
*** gordc has joined #openstack-ceilometer13:59
lsmola_ildikov_: does it support window?13:59
lsmola_ildikov_: the prepared aggregated meter could work14:00
lsmola_ildikov_: at least for project overview page14:00
lsmola_ildikov_: so having samples e.g. total_nodes, total_nodes_up, percentage of nodes up for project14:02
lsmola_ildikov_: then i could show it in timeline any way i want14:02
gibilsmola_: postgres supports WINDOW but sqlalchemy is not. :/14:03
*** eglynn is now known as eglynn-lunch14:03
ildikov_lsmola_: I could not find WINDOW in sqlalchemy, but I'm not an expert14:03
lsmola_ildikov_: yeah it probably doesn't support it14:03
gibilsmola_: total_nodes meter is possible to do with the current data but total_nodes_up needs extra data to be collected as the VM status is not inculded in the metadata of the "instance" meter14:04
lsmola_gibi: I believe it has value 1 when it is up, and 0 when suspended14:05
lsmola_gibi: not sure though14:05
gibibasically the total_nodes would be total_allocated_nodes as the current "instance" meter has value 1 if it is ACTIVE, PAUSED, STOPPED and there os no sample with 0 value14:05
ildikov_lsmola_: I'm not against of the aggregated meters idea for those values that requires subqueries for count it, as to support the generation of queries like the one you need is a complex problem14:05
lsmola_gibi: I was planning to write Documentation of the meters14:05
lsmola_gibi: cause nobody know how they act :-)14:06
lsmola_gibi: hm, ok14:06
ildikov_lsmola_: we've tried before ping you, just to be sure :)14:06
gibilsmola_: we just tried. There is no 0 reported for stopped VMs14:06
gibilsmola_: I think we can write a blueprint for this new meters to be added and let's see how the other thinks about this direction14:07
lsmola_gibi: i thought I saw 0 :-)14:07
ildikov_lsmola_: we had the instance meter, I do not think that we missed any configuration, which would change the content of that meter14:07
lsmola_gibi: ok14:07
lsmola_gibi: i will put there links to overview pages wireframes14:08
lsmola_gibi: so we can see what everything we need14:08
gibilsmola_: I shortly checked you pdf, but I think I will do a deeper look in it14:08
lsmola_ildikov_: yeah, suspending of machine probably restart some meter14:09
lsmola_ildikov_: what measures uptime?14:09
ildikov_lsmola_: that would be good, if we would have everything in place that is needed14:09
lsmola_ildikov_: I believe we have blueprint for admin and project overview pages, there should be wireframes for it14:09
gordcdhellmann: do you know if there are any settings i need to set to get oslo.rootwrap to work? i keep getting an import error "from oslo.rootwrap.cmd import main\nImportError: No module named rootwrap.cmd" when running devstack.14:10
lsmola_ildikov_: and we are preparing some more wireframes14:10
ildikov_lsmola_:I do not think that we currently measure uptime14:10
ildikov_lsmola_:ok, I will check that bp14:11
lsmola_gibi: somebody deleted list of nova meters :-o http://docs.openstack.org/developer/ceilometer/measurements.html14:11
ildikov_lsmola_:will the additional wireframs will be linked there too14:11
lsmola_ildikov_: yes, it should be there14:11
ildikov_lsmola_:yes, eglynn had a not sphinx compatible change in that table, the fix is up for review already :)14:12
*** prad has quit IRC14:12
lsmola_ildikov_: ok14:12
gibilsmola_: thanks for the info, we will try to draft a bp for this new meters14:17
lsmola_gibi: ildikov_ excellent, thank you :-)14:18
ildikov_lsmola_:in that table there is instance and instance:<type> meters and the description is duration for both, but it is not duration, the text is not correct there14:18
lsmola_ildikov_: there is some duration counter somewhere, as cumulative meter14:19
*** CephFan1 has joined #openstack-ceilometer14:21
gibilsmola_: there is cpu meter that measures cpu time in nanoseconds14:21
CephFan1Does Ceilometer have Ceph Object storage integration?  I keep seeing this error in central manager Account HEAD failed: 400 Bad Reques14:21
lsmola_gibi: yeah, that is probably it14:21
lsmola_gibi: not sure how it is related, but the counter is being restarted after suspend14:22
ildikov_lsmola_: I saw cumulative counters only for cpu and disk and network I/O rate meters14:22
lsmola_gibi: so getting toatl uptime is pretty hard when you suspend it in the middle14:22
gibilsmola_: yes, I saw that same that cpu counter is reset if the VM suspended so you are right is is not easy to calculate total uptime if that is required14:24
ildikov_lsmola_: I think there is a need periodically for having uptime meter, maybe it would be good to have an uptime and total uptime to handle somehow when the instance and/or the hypervisor is down14:24
ildikov_lsmola_: I do not know how we can get the info from libvirt, I need to check what can be done to have an uptime meter14:25
lsmola_ildikov_: yeah as I heard it is just showing libvirt counter, which is being restarted14:27
openstackgerritIlya Tyaptin proposed a change to openstack/ceilometer: Add table prefix for unit tests with hbase  https://review.openstack.org/6869814:30
*** sayali_ has quit IRC14:30
ildikov_lsmola_: we need to investigate how we can gather the required information and I will also check if there is any bug or bp registered for this uptime topic14:31
openstackgerritA change was merged to openstack/ceilometer: Use explicit http error code for api v2  https://review.openstack.org/6877514:43
*** RelayChatInfo has joined #openstack-ceilometer14:47
*** RelayChatInfo has left #openstack-ceilometer14:47
openstackgerritA change was merged to openstack/python-ceilometerclient: Remove unused import for print_function  https://review.openstack.org/6951814:51
*** prad has joined #openstack-ceilometer14:55
*** ruhe is now known as _ruhe14:56
*** jmckind has joined #openstack-ceilometer15:05
*** urulama has joined #openstack-ceilometer15:06
*** eglynn-lunch is now known as eglynn-call15:07
ildikov_lsmola_:I've found a blueprint for VM state: https://blueprints.launchpad.net/ceilometer/+spec/state-meter15:12
*** rwsu has joined #openstack-ceilometer15:12
*** _ruhe is now known as ruhe15:30
gibilsmola_: just checked the cpu meter but that is the calculated time that the VM used not wall clock time. So if the VM loads the cpu 50% for a minute then that meter reports 30 sec cpu time15:33
lsmola_gibi: interesting15:33
lsmola_gibi: do you think you could put those facts into doc somewhere? :-)15:34
gibilsmola_: that is a fair request :)15:35
eglynn-callgibi: yes, the cpu meter measure cumulative CPU time actually used by an instance (not the wall-clock duration for which the CPU is associated with an instance)15:36
gibieglynn-call: you can help with the documentation :)15:37
ildikov_eglynn-call: will you available for a short statistics talk today? :)15:38
eglynn-callgibi: some new documentation, or just clarifying what's already described in https://github.com/openstack/ceilometer/blob/master/doc/source/measurements.rst ?15:41
gibieglynn-call: I think that needs clarification as it was not clear for me at first. I had to check libvirt to see15:43
gibiI will propose a patch for it15:43
eglynn-callildikov_: I'm on a call right now, but if you want to take the stats question off-line to an email I'll try to get to it before EoD15:43
eglynn-callgibi: cool enough, put me down as a reviewer pls15:43
gibieglynn-call: OK, I will do15:44
eglynn-callgibi: thank you sir!15:44
ildikov_eglynn-call:I can send some initials via mail, but it would be better, if we could have an online more-or-less brainstorming session about this topic, if it is ok for you15:48
ildikov_eglynn-call: we had a discussion today with gibi about distinct and some generic way of creating statistics, but we haven't reached a consensus yet15:50
eglynn-callildikov_: k, IRC discussion at circa 4:30 work for you?15:51
ildikov_eglynn-call: it's good for us, thanks15:53
*** boris-42 has quit IRC16:07
*** SergeyLukjanov is now known as SergeyLukjanov_16:09
openstackgerritBalazs Gibizer proposed a change to openstack/ceilometer: Enhance the documentation of the cpu measurement  https://review.openstack.org/6963716:10
*** eglynn-call is now known as eglynn16:12
eglynnildikov_, gibi: k, small digression first ...16:16
eglynnso currently we have say 'GET /v2/meters/instance/statistics?period=p'16:17
eglynnwhich returns (sum, max, min, avg, count)16:17
eglynnfor the 'instance' meter, aggregated for each period p16:17
gibieglynn: go ahead :)16:17
eglynnyep, so we get the *same* set of aggregate function applied each time16:18
eglynnmy idea was to allow individual aggregate functions to be selected16:18
eglynn*and* for these functions to be parameterized16:18
eglynnso for example ...16:18
eglynnGET /v2/meters/instance/statistics?aggregate-by=stddev&period=p16:18
eglynnto force the computation of stddev instead of the normal (sum, max, min, avg, count)16:19
eglynnthen to extend this to allow the aggregate function itself be parameterized16:19
eglynnsay for example ...16:19
eglynnGET /v2/meters/instance/statistics?aggregate-by=quantile&aggregate-on=0.99&period=p16:20
jd__does anyone recall how we handle NotImplemented from the storage driver in the API?16:20
eglynnto allow the 99th percentile to calculated16:20
eglynnjd__: doesn't it just throw the NotImplemented back to the caller?16:20
eglynn... so what's the relevance of these paremeterized aggregates?16:21
eglynn... well it occurred to me that *distinct* could itself be though of as an aggregate function16:22
gibidistint in SQL is equvivalent to a groupby without agggregation function16:22
eglynn... where the aggregate-on is the attribute we're counting the distinct values for16:23
eglynngibi: in sqlalchemy, distinct can be counted via func.count(distinct(Model.attribute)), no?16:24
eglynnso say ...16:24
eglynnGET /v2/meters/instance/statistics?aggregate-by=distinct&aggregate-on=resource_id&period=p16:24
eglynnwould map to ...16:24
eglynnat or around line ... https://github.com/openstack/ceilometer/blob/master/ceilometer/storage/impl_sqlalchemy.py#L56416:25
eglynn(instead of the usual func.avg, func.sum, func.max ... etc.)16:25
eglynnfor horizon per-tenant grouping, we'd have ...16:25
eglynnGET /v2/meters/instance/statistics?aggregate-by=distinct&aggregate-on=resource_id&period=p&groupby=project_id16:26
eglynngibi: ... does that make sense?16:26
jd__eglynn: well I got a 500 in _one_ test because of that it seems, not sure why16:27
eglynnI'm no SQL expert, but it doesn't seem any worse than the current aggregation cost in the sqlalchemy driver16:27
gibieglynn: well you made a trick here I guess16:28
eglynnjd__: that's strange, don't the scenario tests have a strategy for swallow the not NotImplementedErrors and skipping those tests?16:28
gibieglynn: I tend to agree that it would work16:29
eglynngibi: cool ... I was about to ask whether a "trick" was a good thing or a bad thing ;)16:29
gibieglynn: I don't know if this trick with the distinct and group by is generic enough16:30
eglynngibi: have you a usecase in mind?16:30
eglynn(that it wouldn't address)16:30
jd__eglynn: I thought so but I'm lost in the code :)16:31
*** ruhe is now known as _ruhe16:31
eglynnjd__: ... I know the feeling16:31
gibieglynn: In general we might need to group by on multiple fields16:31
gibieglynn: for example16:31
eglynngibi: ... would that not still work?16:32
eglynngibi: ... i.e. we could still groupby on both foo and bar while counting distinct snafus?16:33
gibieglynn: let's assume we want to calculate an avg(counter_volume) group by resource_id and then a sum() group by project_id16:33
eglynnin separate queries, or?16:33
gibiin one query.16:34
* gibi try to make a real example...16:34
openstackgerritIlya Tyaptin proposed a change to openstack/ceilometer: Skip unit tests with mongo or db2 when environment variables aren't set  https://review.openstack.org/6964416:34
eglynngibi: well as things stand, surely groupby is per-query?16:35
eglynngibi: the ability to conflate different groupbys in a single query seems to me like a different problem16:36
gibilet's assume we have a proper memory meter that reports actual memory usage of VMs16:36
eglynnk... maybe even like a hammer looking for a nail ;)16:37
gibiand we need a chart with project level memory usage16:37
gibithen we first need to calc an average group by resource_id on a period then a sum group by project id16:38
openstackgerritIlya Tyaptin proposed a change to openstack/ceilometer: Skip unit tests with mongo or db2 when environment variables aren't set  https://review.openstack.org/6964416:38
gibiif we want to do it in one query then we need a subquery16:38
eglynngibi: sure, but sounds like 3 separate queries is the most natural way of doing it though16:38
* jd__ found his problem16:39
eglynngibi: ... i.e. I'm not sure I get why the conflation into a single query would be important?16:39
eglynnjd__: \o/ ... enlighten us16:39
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: samples: fix test case status code check  https://review.openstack.org/6964816:40
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: Use swift master  https://review.openstack.org/6815016:40
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: bases of a Cassandra driver  https://review.openstack.org/6277916:40
jd__eglynn: https://review.openstack.org/6964816:40
gibieglynn: actually it is only important on API level as in SQLAlchemy we can have subqueries for sure16:40
gibi(and in mongo we can have multiple map-reduce)16:40
eglynnjd__: a-ha16:41
gibieglynn: your API proposal is good in a sense that it does not allow too complex things like multiple group bys16:41
gibieglynn: and it is problematic in the same sense :) as it does not allow querying the mem usage on project level because distinct is not enough there16:42
eglynngibi: k, so if I understand you right, you're thinking that different groupbys are *possible* to do in the storage drivers (via sub-queries, multiple M-Rs, ...)16:43
eglynngibi: ... but does that make them desirable to support in the API?16:43
gibieglynn: yes, I'm affraid of the API complexity16:44
eglynngibi: me too :)16:44
gibieglynn: I mean if we allow multiple group bys, but on the other hand multiple group by is needed by horizon for the mem usage, or the disk usage statistics16:44
eglynngibi: ... also I don't think I fully understand your mem usage example16:45
gibieglynn: here is the overview page plane for horizon http://people.redhat.com/~lsurette/OpenStack/Horizon%20Admin%20Overview%20Pages_2.0.pdf16:45
gibieglynn: ther you see something like network usage calculated on project level16:46
gibito do that you have to first aggregate the average on resource level on a period then summ it on project level16:46
gibiand for that we need two group bys and that is something that opens the road to a complex AP16:47
gibiIf we allow two groupbys then we basically allow subquery and then then we end up supporting filtering on subquery level16:49
eglynngibi: I'm still confused about that example16:50
eglynngibi: why would make sense to sum the averages?16:50
eglynngibi: why not just sum the underlying raw data?16:50
eglynngibi: wouldn't you get the same answer either way?16:51
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: samples: fix test case status code check  https://review.openstack.org/6964816:51
openstackgerritJulien Danjou proposed a change to openstack/ceilometer: storage: bases of a Cassandra driver  https://review.openstack.org/6277916:51
gibiyou have two VMs that report every minute the average network usage in B/s.16:53
gibi(the two VM is in the same project)16:53
gibiand now we need to draw a chart that show the average network usage on project level and in one hour period16:54
eglynngibi: k, when you said average I thought you meant the average aggregate function applied to some sample data16:54
eglynngibi: (not a meter that captures a rate per second)16:54
openstackgerritIlya Tyaptin proposed a change to openstack/ceilometer: Replace non-ascii symbols in docs  https://review.openstack.org/6965816:55
gibieglynn: ok. :) so we both need to make average on a longer time period than the sample are reported and in the same time sum up the VM in the same project16:55
gibieglynn: I don't know if it is clear enough. :)17:01
eglynngibi: I'm struggling to understand by the averaging per-VM is even required17:02
eglynngibi: i.e. why you can't just average over the entire project and multiply that by the number of VMs?17:02
eglynns/struggling to understand by/struggling to understand *why*/17:05
*** boris-42 has joined #openstack-ceilometer17:07
*** flwang has quit IRC17:10
gibieglynn: you are right we can rearrange the things to a sum and a multiplication. Do we want that the user express the query in this way or we need to translate it to this less complex from?17:11
* gibi needed to do some basic math. :)17:11
eglynngibi: well my preference would be to only give the user as much hammer as they really need17:12
eglynngibi: (as opposed to exposing a lot of complexity in API, even if an equivalent query can be expressed more simply from the get-go)17:12
openstackgerritPradeep Kilambi proposed a change to openstack/ceilometer: Fix docs on what an instance meter represents  https://review.openstack.org/6674617:13
gibieglynn: I agree that complexity is a bad thing.17:14
*** _ruhe is now known as ruhe17:15
gibieglynn: lsmola_ needs this instance statistics that can be solved with distinct and one group by, he also needs the network statistics I used as an example above17:15
gibiin that usecase your API proposal with distinct is not enough17:15
jaypipesjd__: question for you... looking at your cassandra patches, I didn't know that Cassandra depended on swift... is that the case?17:17
gibiwe either need to allow multiple groupby or allow to express the multiplication17:17
gibior the client needs to do two separate queries and then do the multiplication on the client side17:17
eglynngibi: ... /me is not seeing that as unreasonable burden on the client TBH17:18
gibieglynn: OK. Understood. So only the instance statistics is special due to the fact that that meter only reports when a VM exists17:23
gibitherefore in this case we cannot move the problem to the client side in a same way as in the network statistics case17:23
openstackgerritVictor Sergeyev proposed a change to openstack/ceilometer: WIP: Add test for check sync models and migrations  https://review.openstack.org/6967417:25
gibieglynn: a bit different question. When you introduce the distinct in the query then what changes needed to be done on the Statistics object used in the API as a return value for the query?17:26
eglynngibi: yeah, the distinct instance isn't suitable for moving to the client side as there wouldn't be much prior distillation of the data within the API layer17:26
eglynngibi: yeah, we probably need a generic name-value pair on the Statistics API representation also17:27
*** ruhe is now known as _ruhe17:28
eglynngibi: (as that's currently hardcoded with the current set of aggregate labels)17:28
jd__jaypipes: no17:28
gibieglynn: Yes, I agree that we need to change that part to something more generic. I guess the parameterised aggregation functions also needs some extra care there17:29
openstackgerritPradeep Kilambi proposed a change to openstack/ceilometer: Fix measurement docs around various meters to represent Existance instead of Duration  https://review.openstack.org/6967517:29
eglynngibi: yep, that's a good point ... overall though are we ad idem that distinct as a new statistics aggregate function is a workable idea?17:30
jaypipesjd__: is there any reason the cassandra patch is dependent on the swift git master patch?17:33
jaypipesjd__: sorry, when I see a dependent patch in gerrit, I believe that the patches are related...17:34
*** yassine has quit IRC17:34
jd__jaypipes: yeah no relation, just the result of my incremental bug fixing in that branch17:35
jaypipesjd__: k, no worries, was just checking with you. thx.17:35
gibieglynn: I think distinct will work for this specific usecase. However it only works in this case because we can assume that one resource is always belongs to onyl one project17:35
jaypipesjd__: figured easier to ask you in IRC than have you get mad at me for asking a question in review ;)17:35
*** sayali has joined #openstack-ceilometer17:36
*** SergeyLukjanov_ is now known as SergeyLukjanov17:36
jd__I never get mad17:36
gibieglynn: I don't know that if we allow distinct in general then what kind of other assumptions we need to make17:37
*** viktors has left #openstack-ceilometer17:38
*** Alexei_987 has quit IRC17:38
eglynngibi: I think distinct should be allowed in general, but probably most useful in practice for grouped-by queries17:39
eglynngibi: gotta drop off shortly (... kids to feed)17:39
eglynngibi: let's chat further tmrw if you any more doubts on that approach17:39
gibieglynn: OK. Thanks for the discussion. Let's continue it tomorrow17:40
eglynngibi: cool17:41
*** eglynn has quit IRC17:54
*** ildikov_ has quit IRC18:04
*** xmltok has joined #openstack-ceilometer18:11
*** kwhitney has left #openstack-ceilometer18:18
openstackgerritA change was merged to openstack/python-ceilometerclient: Modify ceilometer client cmd line help info  https://review.openstack.org/6398418:19
*** Alexei_987 has joined #openstack-ceilometer18:33
openstackgerritPradeep Kilambi proposed a change to openstack/ceilometer: Fix measurement docs to correctly represent Existance meters  https://review.openstack.org/6967518:52
*** ildikov_ has joined #openstack-ceilometer18:58
*** dperaza has joined #openstack-ceilometer19:00
*** jsuchome has quit IRC19:33
*** _ruhe is now known as ruhe19:34
*** rwsu has quit IRC19:36
*** rwsu has joined #openstack-ceilometer19:40
*** hewbrocca has joined #openstack-ceilometer19:41
*** hewbrocca has left #openstack-ceilometer19:41
*** tongli has joined #openstack-ceilometer20:17
*** ruhe is now known as _ruhe20:54
*** _ruhe is now known as ruhe21:00
*** yassine has joined #openstack-ceilometer21:12
*** urulama has quit IRC21:23
*** CephFan1 has quit IRC21:30
*** tongli has quit IRC21:44
*** dperaza has quit IRC21:50
*** SergeyLukjanov is now known as SergeyLukjanov_21:59
*** ryanpetrello has joined #openstack-ceilometer22:32
*** jdob has quit IRC22:33
dhellmanndoes anyone have any idea why I see "ImportError: No module named middleware.proxy_logging" when running the tests from trunk?22:51
dhellmannswift is installed in my .tox/py27 directory but it does not include a middleware package22:52
dhellmannthe import line is actually "from swift.common.middleware.proxy_logging import InputProxy"22:52
dhellmannjd__, eglynn: ^^22:53
jd__dhellmann: you need this: https://review.openstack.org/#/c/68150/22:55
*** prad has quit IRC22:55
dhellmannjd__: looks good, shall I approve it?22:55
jd__dhellmann: I think so yeah22:56
dhellmannoh, you submitted it, that's why you haven't approved it22:56
* dhellmann is starting to get crossed eyes from reviewing code22:56
dhellmannjd__: +2 a22:56
jd__dhellmann: while you're around, any idea what I need to do to have a new pbr released?22:57
dhellmanntechnically I can do it, but I would want to check with mordred first22:58
jd__dhellmann: then, would you be kind enough to check with him once https://review.openstack.org/#/c/63236/ is merged?22:58
jd__I would then be able to go forward with Oslo portage22:59
dhellmannjd__: sure22:59
dhellmannactually, I'm not 100% sure I have that ACL, but I can check22:59
jd__at least I hope we don't have a SPOF on that, as mordred hasn't be really responsive recently23:00
dhellmannwe can get it fixed if we do have that23:01
*** openstack has joined #openstack-ceilometer23:04
*** ruhe is now known as _ruhe23:10
openstackgerritYuuichi Fujioka proposed a change to openstack/ceilometer: fix column name and alignment  https://review.openstack.org/6952423:54
*** dperaza has joined #openstack-ceilometer23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!