Tuesday, 2016-12-06

*** donghao has quit IRC00:02
*** ljxiash_ has joined #openstack-telemetry00:04
*** ljxiash has quit IRC00:08
*** dave-mccowan has quit IRC00:08
*** thorst has joined #openstack-telemetry00:12
*** thorst has quit IRC00:15
*** thorst has joined #openstack-telemetry00:16
*** thorst has quit IRC00:25
*** catintheroof has quit IRC00:25
*** Guest76526 is now known as ioni00:31
*** ioni is now known as Guest9725800:31
*** ljxiash_ has quit IRC00:46
*** hfu has joined #openstack-telemetry00:52
*** diogogmt has quit IRC00:55
*** rcernin has quit IRC01:01
*** leitan has joined #openstack-telemetry01:06
*** ljxiash has joined #openstack-telemetry01:08
*** ljxiash has quit IRC01:08
*** thorst has joined #openstack-telemetry01:09
*** thorst has quit IRC01:09
*** thorst has joined #openstack-telemetry01:10
*** zhurong has joined #openstack-telemetry01:12
*** thorst has quit IRC01:14
*** thorst has joined #openstack-telemetry01:23
*** thorst has quit IRC01:24
*** zhurong has quit IRC01:25
*** zhurong has joined #openstack-telemetry01:25
*** leitan has quit IRC01:27
openstackgerrityaoyuan proposed openstack/ceilometer: filtered out the phynical nics when query vm nics with VMware API  https://review.openstack.org/40680701:31
*** Guest97258 is now known as ioni01:32
*** ioni is now known as Guest6205001:32
openstackgerrityaoyuan proposed openstack/ceilometer: filtered out the phynical nics when query vm nics with VMware API  https://review.openstack.org/40680701:39
*** Jack_Iv has joined #openstack-telemetry01:40
*** Jack_Iv has quit IRC01:45
*** sam_nowitzki has joined #openstack-telemetry01:50
*** lhx_ has joined #openstack-telemetry01:51
sam_nowitzkiHi, I'm using Gnocchi for storing Ceilometer metric, but some metrics Ceilometer collects but Gnocchi show no data, though I config it in gnocchi.resources.yaml. For instance: network.outgoing.bytes.rate01:57
sam_nowitzkimy gnocchi_resources.yaml01:57
sam_nowitzkihttp://pastebin.com/icjJE7NK01:57
*** thorst has joined #openstack-telemetry02:05
*** thorst has quit IRC02:05
*** thorst has joined #openstack-telemetry02:06
*** thorst has quit IRC02:15
*** hfu has quit IRC02:15
*** Guest62050 is now known as ioni02:32
*** ioni is now known as Guest178202:33
*** Ephur has quit IRC02:44
*** hfu has joined #openstack-telemetry02:44
*** thorst has joined #openstack-telemetry03:12
*** thorst has quit IRC03:20
*** hfu has quit IRC03:31
*** Guest1782 is now known as ioni03:33
*** ioni is now known as Guest3734103:34
*** hfu has joined #openstack-telemetry03:36
*** hfu has quit IRC03:40
*** hfu has joined #openstack-telemetry03:40
*** hfu has quit IRC03:44
*** hfu has joined #openstack-telemetry03:45
*** hfu has quit IRC03:47
*** hfu has joined #openstack-telemetry03:50
*** links has joined #openstack-telemetry03:52
*** hfu has quit IRC03:54
*** masber has joined #openstack-telemetry04:07
*** hfu has joined #openstack-telemetry04:08
*** hfu has quit IRC04:15
*** thorst has joined #openstack-telemetry04:18
*** hfu has joined #openstack-telemetry04:20
*** thorst has quit IRC04:25
sorrisonAny gnocchi devs around? Have a question about datetime formats04:25
*** Jack_Iv has joined #openstack-telemetry04:25
*** afazekas has quit IRC04:26
*** hfu has quit IRC04:27
sorrisonBasically in test_storage.py the tests pass in different types of datetime objects. Not sure why and if the storage driver should have to concern itself with datatime format04:27
sorrisonsometimes they pass in pure datetime objects. And other times they pass in utils.dt_to_unix_ns(2014, 1, 1, 12, 0, 1) which is a mumpy.datetime64 object04:28
*** Jack_Iv has quit IRC04:29
*** Guest37341 is now known as ioni04:34
*** ioni is now known as Guest9334904:35
*** afazekas has joined #openstack-telemetry04:58
*** zhangguoqing has joined #openstack-telemetry05:07
*** anteaya has quit IRC05:16
*** thorst has joined #openstack-telemetry05:24
*** anteaya has joined #openstack-telemetry05:29
*** thorst has quit IRC05:30
*** sanchitmalhotra has joined #openstack-telemetry05:31
*** Guest93349 is now known as ioni05:35
*** ioni is now known as Guest8547205:35
*** hfu has joined #openstack-telemetry05:37
*** hfu has quit IRC05:40
*** hfu has joined #openstack-telemetry05:42
*** sanchitmalhotra1 has joined #openstack-telemetry05:43
*** sanchitmalhotra has quit IRC05:44
*** sanchitmalhotra1 is now known as sanchitmalhotra05:44
*** hfu has quit IRC05:47
*** hfu has joined #openstack-telemetry05:47
*** adriant has quit IRC05:53
*** nadya has joined #openstack-telemetry05:55
*** Jack_Iv has joined #openstack-telemetry05:56
*** hfu has quit IRC06:06
*** nadya has quit IRC06:07
*** hfu has joined #openstack-telemetry06:15
*** Jack_Iv has quit IRC06:16
*** hfu has quit IRC06:27
*** thorst has joined #openstack-telemetry06:29
*** Jack_Iv has joined #openstack-telemetry06:30
*** hfu has joined #openstack-telemetry06:32
*** rcernin has joined #openstack-telemetry06:34
*** thorst has quit IRC06:34
*** Guest85472 is now known as ioni06:36
*** hfu has quit IRC06:36
*** ioni is now known as Guest7255006:36
*** sanchitmalhotra has quit IRC06:39
*** hfu has joined #openstack-telemetry06:46
*** hfu has quit IRC06:48
*** hfu has joined #openstack-telemetry06:53
*** Jack_Iv has quit IRC06:56
*** Jack_Iv has joined #openstack-telemetry06:57
*** hfu has quit IRC06:57
*** hfu has joined #openstack-telemetry06:57
*** Jack_Iv has quit IRC07:03
openstackgerritShengping Zhang proposed openstack/ceilometer: Allow rss collected as memory usage when polling cirros  https://review.openstack.org/40735407:10
*** Jack_Iv has joined #openstack-telemetry07:11
*** rcernin has quit IRC07:12
*** Jack_Iv_ has joined #openstack-telemetry07:17
*** g3ek has quit IRC07:20
*** Jack_Iv has quit IRC07:21
*** g3ek has joined #openstack-telemetry07:21
*** Guest66666 has quit IRC07:27
*** Guest66666 has joined #openstack-telemetry07:29
*** thorst has joined #openstack-telemetry07:32
*** yprokule has joined #openstack-telemetry07:33
*** rcernin has joined #openstack-telemetry07:34
*** Guest72550 is now known as ioni07:37
*** ioni is now known as Guest1733207:37
*** thorst has quit IRC07:40
*** pcaruana has joined #openstack-telemetry07:42
*** hfu has quit IRC07:48
*** hfu has joined #openstack-telemetry07:49
*** hfu has quit IRC07:53
*** hfu has joined #openstack-telemetry07:53
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: fix oslo.db 4.15.0 breakage  https://review.openstack.org/40688507:59
*** nadya has joined #openstack-telemetry08:05
openstackgerritShengping Zhang proposed openstack/ceilometer: Allow rss collected as memory usage when polling cirros  https://review.openstack.org/40735408:15
*** sanchitmalhotra has joined #openstack-telemetry08:16
*** ljxiash has joined #openstack-telemetry08:20
*** ljxiash has quit IRC08:24
*** Guest17332 is now known as ioni08:38
*** ioni is now known as Guest2462108:38
*** thorst has joined #openstack-telemetry08:38
*** thorst has quit IRC08:44
*** zhangguoqing has quit IRC08:47
*** shardy has joined #openstack-telemetry09:11
*** larainema has quit IRC09:14
*** Jack_Iv_ has quit IRC09:14
*** larainema_ has joined #openstack-telemetry09:15
*** thorst has joined #openstack-telemetry09:24
*** thorst has quit IRC09:31
*** yassine has joined #openstack-telemetry09:34
*** yassine is now known as Guest1745009:35
*** Guest24621 is now known as ioni09:38
*** ioni is now known as Guest673009:39
*** shardy is now known as shardy_mtg09:45
*** zhurong has quit IRC10:03
*** sam_nowitzki has quit IRC10:23
*** thorst has joined #openstack-telemetry10:28
*** cdent has joined #openstack-telemetry10:29
*** egonzalez90 has joined #openstack-telemetry10:31
*** egonzalez90 has left #openstack-telemetry10:32
*** thorst has quit IRC10:36
*** Guest6730 is now known as ioni10:39
*** egonzalez90 has joined #openstack-telemetry10:39
*** ioni is now known as Guest3902210:40
*** Ephur has joined #openstack-telemetry10:47
*** Ephur has quit IRC10:52
*** hfu has quit IRC10:58
*** Jack_Iv_ has joined #openstack-telemetry11:15
*** Jack_Iv_ has quit IRC11:19
*** shardy_mtg has quit IRC11:30
*** thorst has joined #openstack-telemetry11:34
*** Guest39022 is now known as ioni11:40
*** vint_bra has joined #openstack-telemetry11:41
*** thorst has quit IRC11:41
*** ioni is now known as Guest1310911:41
lhx_sileht, is the measurements up-to-date? http://docs.openstack.org/admin-guide/telemetry-measurements.html11:44
*** egonzalez90 has left #openstack-telemetry11:44
lhx_some placed marks 'Meters added in the Kilo release', so the measures are all Kilo release?11:46
*** hfu has joined #openstack-telemetry12:11
*** nadya has quit IRC12:14
*** thorst has joined #openstack-telemetry12:30
*** nadya has joined #openstack-telemetry12:38
*** Guest13109 is now known as ioni12:41
*** ioni is now known as Guest1228412:41
*** larainema_ has quit IRC12:46
*** larainema_ has joined #openstack-telemetry12:46
*** Kevin_Zheng has quit IRC12:46
*** Kevin_Zheng has joined #openstack-telemetry12:47
*** csatari has joined #openstack-telemetry12:48
csatariHi I'm looking for some help with Panko and devstack. Can someone help me to figure out if I need to add other projects to devstack then Panko?12:49
*** nadya has quit IRC12:49
*** nadya has joined #openstack-telemetry12:51
*** lhx_ has quit IRC13:05
*** hfu has quit IRC13:07
*** dave-mccowan has joined #openstack-telemetry13:19
*** shardy has joined #openstack-telemetry13:22
*** pradk has joined #openstack-telemetry13:22
*** hfu has joined #openstack-telemetry13:23
*** hfu has quit IRC13:29
*** hfu has joined #openstack-telemetry13:29
*** lhx_ has joined #openstack-telemetry13:31
*** hfu has quit IRC13:38
*** fguillot has joined #openstack-telemetry13:41
*** Guest12284 is now known as ioni13:42
*** ioni is now known as Guest1947513:42
*** catintheroof has joined #openstack-telemetry13:43
*** swamireddy has quit IRC13:43
*** hfu has joined #openstack-telemetry13:53
*** hfu has quit IRC13:58
*** swamireddy has joined #openstack-telemetry13:58
*** nadya has quit IRC13:59
*** nadya has joined #openstack-telemetry14:03
*** nadya has quit IRC14:03
*** nadya has joined #openstack-telemetry14:03
*** nadya has quit IRC14:04
*** gordc has joined #openstack-telemetry14:04
*** nadya has joined #openstack-telemetry14:06
*** hfu has joined #openstack-telemetry14:08
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: storage: split s3 driver  https://review.openstack.org/40215414:16
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: storage: split swift driver  https://review.openstack.org/40215314:16
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: storage: split ceph driver  https://review.openstack.org/40203314:16
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: storage: split file driver  https://review.openstack.org/40203214:16
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: Introduce new storage groups for storage  https://review.openstack.org/40232014:16
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: storage: remove temporary incoming setup  https://review.openstack.org/40232114:16
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: metricd: move metricd options in metricd group  https://review.openstack.org/40234114:16
*** swamireddy has quit IRC14:17
*** donghao has joined #openstack-telemetry14:24
*** hfu has quit IRC14:28
*** hfu has joined #openstack-telemetry14:29
*** swamireddy has joined #openstack-telemetry14:32
*** hfu has quit IRC14:35
*** hfu has joined #openstack-telemetry14:38
*** hfu has quit IRC14:40
*** links has quit IRC14:40
*** nadya has quit IRC14:41
*** Guest19475 is now known as ioni14:42
*** nadya has joined #openstack-telemetry14:43
*** ioni is now known as Guest4749314:43
*** nadya has quit IRC14:46
*** nadya has joined #openstack-telemetry14:46
*** Ephur has joined #openstack-telemetry14:49
*** Ephur has quit IRC14:53
*** rbak has joined #openstack-telemetry14:56
lhx_gordc, should the measurements be up-to-date?15:03
lhx_http://docs.openstack.org/admin-guide/telemetry-measurements.html15:03
*** hfu has joined #openstack-telemetry15:04
gordclhx_: they aren't up to date.15:06
gordcyou can update them if you have time15:06
gordci think you need to delete a lot of things15:07
lhx_gordc, some places mark 'Meters added in the Kilo release', so the measures are in Kilo release?15:07
lhx_or measures were just added when kilo release? and we should update with the latest measures?15:09
*** larainema_ is now known as larainema15:11
gordclhx_: it means they were added in Kilo (2nd option)15:13
gordclhx_: but we dropped a lot of them15:13
gordcie. no more instance meter15:13
gordcthe magnetodb stuff is gone becuase that project doesn't exist anymore15:13
gordcthe sahara (data processing) stuff is gone as well15:14
gordci would just try to grep meter in code, if you can't find it, it's gone15:14
lhx_gordc, some measures like IPMI based meters, how can I get the latest?15:16
gordclhx_: https://github.com/openstack/ceilometer/tree/master/ceilometer/ipmi15:17
gordcsileht: you know why we require testresourcs now?15:19
gordcit an oslo.db thing?15:19
silehtgordc, because this is a extra dep of oslo.db15:19
gordci see. so everything that uses it needs it now?15:19
silehtgordc, yes15:19
gordckk, adding to aodh then15:20
gordcthrowing error for me15:20
silehtgordc,when we depends on oslo.db>=4.14.0 we could install oslo.db with somthing like oslo.db[fixture,mysql]15:20
silehtgordc, and remove pymysql/pscyco2/testresources from our requirements15:21
gordcyeah, that seems cleaner. i'll let you do it :P15:21
silehtlol15:21
*** jwcroppe has joined #openstack-telemetry15:21
silehtraise oslo.db to 4.14 instead 4.8 will break newton compat15:22
silehtI dunno if we want that now15:22
gordcprobably not. i guess we wait until next cycle15:22
openstackgerritgordon chung proposed openstack/aodh: add testresources req  https://review.openstack.org/40756415:23
*** dotplus has joined #openstack-telemetry15:25
dotplusis openstack/gnocchi on-topic for this channel? or should I be elsewhere?15:30
gordcdotplus: yep. go for it. no promises on good advice though15:31
openstackgerritgordon chung proposed openstack/aodh: add testresources req  https://review.openstack.org/40756415:34
dotplusI'm wondering whether there any public examples of using gnocchi? In particular, I'd love to see a demo or even just graphs based on gnocchi-stored TS data that show ... well, anything. Something that is actually useful. I'm considering a side project to help our OS eng folk get a better understanding of how to improve various performance metrics and I'd love to use an OS project if it makes sense instead more obvious choices like ...15:35
dotplus... something built on influxdb or whatever.15:35
gordcdotplus: nebeliu has a few demos where they use cloudkitty+gnocchi for their rating system. they've done a few demos.15:37
gordcthey have their own ui though.15:37
jd__sileht: gordc: well we can keep Newton and 3.0 together and move on as far as I'm concern15:37
gordci have it in our environment with grafana, but i don't have any public demos.15:38
gordcjd__: oh. i was thinking more about aodh15:38
jd__gordc: hm so the answer is even more obvious now? we never had any compat' between different cycles of other openstack projects15:39
jd__dotplus: did you check the website http://gnocchi.xyz/ as a first step?15:39
gordcjd__: i won't lie, i don't understand how this whole requirements stuff works in openstack with this false upper-constraints bs.15:39
gordci'll just nod and agree15:40
jd__gordc: works for me :p15:40
silehtgordc, it's simple, when a project want to stay in the past it blocks the whole openstack15:42
*** Guest47493 is now known as ioni15:43
*** ioni is now known as Guest2962115:44
gordcsileht: lol15:44
gordci just read that thread yesterday.15:45
silehtthe fact is oslo.messaging can't work with the old version and monasca can't with the new one...15:46
* lhx_ thinking about the issue what you are talking15:48
gordcsileht: sucks how mutually exclusive projects must share same requirements15:49
silehtgordc, ceilometer already requires kafka>=1.0.0 but don't follow the requirements stuffs :)15:50
*** nadya has quit IRC15:50
gordclhx_: you want to fix requirements? you will need a lot of time.15:51
lhx_gordc, sorry, it isn't in my plan15:53
gordc:)15:54
dotplusgordc: ok I have the nubeliu demo queued. what sort of graphs do you have in grafana? traditional system-level stuff like load averages etc.? or OS-specific stuff like instance creation timing, instance/network/volume count over time?15:54
*** Jack_Iv_ has joined #openstack-telemetry15:54
gordcdotplus: if using just opnestack services, it's whatever measurements ceilometer generates.15:56
lhx_gordc, these days I have done some preparing to use ceilometer in hardware, it costs much of my times15:57
gordcdotplus: my environment has a few more metrics... gnocchi just stores time-series and you can configure grafana to graph whatever you want.15:58
openstackgerritgordon chung proposed openstack/aodh: Replace retrying with tenacity  https://review.openstack.org/36760015:58
gordcsileht:  i blocked oslo.db in this patch too. https://review.openstack.org/#/c/407564/15:59
silehtgordc, sure16:00
openstackgerritgordon chung proposed openstack/aodh: Replace retrying with tenacity  https://review.openstack.org/36760016:01
lhx_will this be remove?16:03
lhx_https://github.com/openstack/ceilometer/blob/master/setup.cfg#L11516:03
gordclhx_: yes https://review.openstack.org/#/c/400973/16:07
lhx_gordc, all instance on compute and central node?16:08
lhx_I see you just remove the central node instance16:09
gordclhx_: the deuce... there's two?lol16:11
gordclhx_: you should -1 my patch then. i'll remove it from both.16:11
lhx_gordc, maybe I can propose one, lol16:12
gordclhx_: sure. i think another patch i'm depending on polls from central. so maybe nothing is needed16:13
gordcmy patch only shows one and master only shows one16:13
gordcbrb16:13
*** shardy has quit IRC16:14
*** catinthe_ has joined #openstack-telemetry16:14
*** yprokule has quit IRC16:16
*** catintheroof has quit IRC16:16
dotplusgordc: ok. so you would say gnocchi/grafana might well be suitable for storing/visualizing metrics in general? For example, my immediate use case is Jenkins job build steps - I want to store how long various build steps take (and possibly even profiling each command within a buildstep that is a shell command) and how that changes over time.16:16
*** rcernin has quit IRC16:17
*** catinthe_ has quit IRC16:18
dotplusjd__: sorry, yes. I did look over the gnocchi docs briefly. Also, some of the material on your blog16:19
*** pcaruana has quit IRC16:19
*** cdent has quit IRC16:22
*** cdent has joined #openstack-telemetry16:26
dotplusjd__: perhaps you could comment on why you started gnocchi (as opposed to using an existing collector/tsdb)? to ease integration with ceilometer and OS more generally? for project org reasons such as foundation control instead of external? for some technical reason(s) why the alternatives were not suitable? All of the above or something else?16:32
dotplus(I think any one or combination of the above could be valid, I don't want to come across as combative here - I'm seeking to understand the thinking to help decide direction for *our* team's effort)16:33
*** fguillot has quit IRC16:33
*** jmlowe has joined #openstack-telemetry16:35
jd__dotplus: there might be some context written on my slides https://julien.danjou.info/talks/storing-metrics-at-scale-with-gnocchi.pdf16:35
dotplusaha, I'll look there. thanks16:36
jd__dotplus: no, that's good questions :) don't worry16:36
jmlowegordc: Can you give me some hints on what to look at for this? http://paste.openstack.org/show/591563/16:36
*** Jack_Iv_ has quit IRC16:42
gordcdotplus: yep. that's a good use case. basically it's designed to measure anything over time. our system we're using it to partially visualise load, and partially to do some pattern recognition stuff.16:43
gordcjmlowe: weird. you know what version you have running?16:44
gordcjd__: ^ not sure if you've seen this? seems it's not doing split properly.16:44
*** Guest29621 is now known as ioni16:44
jmlowewas 3.0.1 now 3.0.2 as of an hour ago16:44
*** ioni is now known as Guest9999016:45
jmlowepulled from cheeseshop16:45
jd__gordc: what makes you say that?16:46
gordcNoneType' object has no attribute 'serialize'16:46
*** nadya has joined #openstack-telemetry16:46
jmlowethis ceph object strikes me as odd gnocchi_b53d78b8-6312-4eda-a28f-3b764f5e2421_none_v3, is that right?16:47
jd__gordc: right… that is weird16:48
jd__jmlowe: no that's right16:48
jd__jmlowe: it's the unaggregated timeserie containing the current back window of points16:48
jd__"none" is for unaggregated, obviously (sigh :p)16:49
jmloweok, is there one of those objects I should dump and examine?16:49
gordcwe are very good at naming.16:49
jd__gordc: yeah ts.split() returns None, wtf?16:49
gordcjmlowe: this is thrown after you upgraded to 3.0.2?16:50
jd__right knowing it's a problem in 3.0.2 and not in 3.0.1 might help :p16:51
gordcdid we require upgrade between 3.0.1 to 3.0.2? i feel like we might have16:51
jmloweit was doing it on 3.0.1, saw there was some message packing changes in 3.0.2 so I went for it16:51
gordcoh ok.16:51
jmlowewhat's the worst that could happen16:51
jd__ok so the problem is in the rewrite code16:51
jd__jmlowe: it's just not treating any new measure for that metric currently, until we fix the bug :p16:52
jmlowethat's usually the start of how I get into trouble16:52
jd__(if it's a bug, though it looks like one)16:52
jmloweis there anything I can get for you that would make it easier to diagnose and fix?16:53
jd__jmlowe: can you send me a copy of the files?16:55
gordcjust looking through code now. there a few scenarios which could trigger it it seems16:55
jd__gordc: I'm trying to find those scenarios yeah, I'm not seeing it clearly yet :/16:55
jmloweall of the ones listed in the paste?16:55
jd__jmlowe: yep16:55
jmloweok, I'll get cracking16:55
jd__if we can find the precise case you're in it should be easy to fix16:55
jd__likely the new measures are very far away of the latest measure or something like that16:56
*** nadya has quit IRC16:56
jd__something that might be not tested in our test I imagine?16:56
jd__(wild guess)16:56
gordcjmlowe: did you change what aggregations are calculated by any chance? (add a new aggregation?)16:58
jd__gordc: this is very weird because the code says basically that there's a key like e.g. 1477440000.0 that is old and should be compressed, but then when it tries to get it from Ceph, it gets no data, so it gets None16:59
jd__gordc: yeah that's a good question ^^16:59
jmloweI don't think I did16:59
gordcjd__: yeah, those seem like the two obvious places that could return None16:59
jmloweso the history is ceilometer liberty -> mitaka, gnocchi 1.x->2.x->3.x17:00
*** nadya has joined #openstack-telemetry17:01
jmlowethe move from gnocchi 2.x to 3.x had some measures that had to be thrown away, the api got started before metricd finished conversion if I remember correctly17:01
jd__wah, you mean the upgrade code actually worked, somehow? lol17:01
jmlowemore or less17:01
gordcjd__: apparently not.lol17:01
jd__I'm impressed :)17:01
jmlowesometimes less17:01
jd__:D17:01
jd__sorry to hear that17:02
jd__it's too bad you did not report the bug(s) though17:02
jmloweeh, it's not critical data17:02
jd__jmlowe: can you give us the details of the archive policy used by metric b53d78b8-6312-4eda-a28f-3b764f5e2421 ?17:02
jmloweI don't think I realized there was a bug, there was at least some degree of operator error17:02
jmloweI have those files tar'ed up, how to you want them, ~4k17:03
*** Ephur has joined #openstack-telemetry17:03
*** efoley has joined #openstack-telemetry17:03
gordcjd__: there's  std_300.0 missing17:03
*** catintheroof has joined #openstack-telemetry17:03
*** catintheroof has quit IRC17:03
gordcor not in paste17:04
jmlowehttp://paste.openstack.org/show/591568/17:04
*** catintheroof has joined #openstack-telemetry17:04
jd__gordc: here you are eye of lynx17:04
jd__jmlowe: so where is std_300.0? did you hide it somewhere? monster!17:05
jd__;)17:05
jd__FREE STD_300.017:05
jmloweI don't tolerate deviation17:05
jd__haha17:05
jd__even if it's a standard one? c'mon17:05
jmlowerados get -p gnocchi gnocchi_b53d78b8-6312-4eda-a28f-3b764f5e2421_1478520000.0_std_300.0_v3 gnocchi_b53d78b8-6312-4eda-a28f-3b764f5e2421_1478520000.0_std_300.0_v317:06
jmloweerror getting gnocchi/gnocchi_b53d78b8-6312-4eda-a28f-3b764f5e2421_1478520000.0_std_300.0_v3: (2) No such file or directory17:06
jmloweso it's really missing?17:06
jd__yeah so that's the problem17:07
jd__we need to log a message if that happens17:07
jd__jmlowe: I imagine there was a problem at some stage? which one I could not say, upgrade maybe?17:07
jd__sudden shutdown, data corruption, a random file eraser? who knows17:07
gordci hope it's not because it's deleting too aggressively... although it doens't make sense the other aggreagtes are there.17:07
jd__that should not be /fatal/ though, we could just ignore it and log an warning17:08
*** lhx_ has quit IRC17:08
gordcjmlowe: you can switch this to 'return' https://github.com/openstack/gnocchi/blob/master/gnocchi/storage/_carbonara.py#L241 that may be the quick hack17:09
gordcassuming it throws aggregationdoesnotexist and not some granularitydoesnotexist exception17:10
jmloweso just to verify s/pass/return/ in line 24117:11
gordci think? lol that would've been my patch. jd__ ?17:11
gordcconfidence inspiring.17:12
jmlowehttp://paste.openstack.org/show/591569/17:13
jd__gordc: jmlowe: yes this is what I'm doing17:14
jd__well you need an if split is None: return just before L249 to be exact17:14
jmlowesame, unless of course I should have whacked the pyc17:14
jd__wait a sec I'm pushing a patch17:14
*** nadya has quit IRC17:15
jd__jmlowe: gordc https://review.openstack.org/#/c/407614/17:16
gordcjmlowe: hmm... maybe add a try/except around that and print out key_as_str, aggregation, archive_policy_def.granularity)17:17
gordcor that.17:17
jmloweok, firing up that patch17:18
jd__I hope I did not make a formatting mistake in the LOG.warning lol17:19
jd__that would be embarassing17:19
jd__:P17:19
gordcjd__: no test. cannot be verified.lol17:20
gordcor jmlowe is the test17:20
jd__jmlowe will be the test for now, then it should be easy to add a test17:21
jd__just create a metric and delete a file :p17:21
jd__bbl17:24
jmloweI've got to go, be back in 2 hours or so17:25
jmlowelooks promising though http://paste.openstack.org/show/591571/17:25
jmlowelike I might converge on zero eventually17:25
gordcjmlowe: :) cool cool. thanks for raising and verifying17:26
jmlowenp17:26
jmlowethanks to both of you17:26
*** rcernin has joined #openstack-telemetry17:26
jmloweand there we go, down to 1 metric 1 measure17:27
*** jmlowe has quit IRC17:28
gordcyay17:28
*** diogogmt has joined #openstack-telemetry17:32
*** Guest99990 is now known as ioni17:45
*** ioni is now known as Guest9789617:46
*** fguillot has joined #openstack-telemetry17:46
*** leitan has joined #openstack-telemetry17:47
*** Guest17450 has quit IRC18:01
*** jwcroppe has quit IRC18:03
jd__cool!18:12
jd__dotplus: see, that's why you should use Gnocchi, an awesome support team fixing bugs :p18:12
jd__so what's wrong with our integration tests? gordc, sileht ?http://logs.openstack.org/85/406885/5/check/gate-telemetry-dsvm-integration-gnocchi-ubuntu-xenial/a728f36/console.html18:14
*** leitan has quit IRC18:16
*** rbak has quit IRC18:20
*** efoley has quit IRC18:23
gordcjd__: https://review.openstack.org/#/c/407622/1 i think that18:26
jd__ok, that's a good change at least18:27
gordcbut no one in heat channel will confirm so i'm just going to run with it.18:27
jd__I should yell at therve a bit for a change18:27
gordcgo for it :)18:27
gordcdisclaimer: 100% of my patches to project-config are guesses.lol18:28
dotplusjd__: very good, decision made.18:29
jd__gordc: I feel you bro18:29
*** jwcroppe has joined #openstack-telemetry18:32
openstackgerritJulien Danjou proposed openstack/gnocchi: carbonara: log a message and do not fail if a file is missing  https://review.openstack.org/40761418:38
*** Guest97896 is now known as ioni18:46
*** ioni is now known as Guest4246118:46
*** Jack_Iv_ has joined #openstack-telemetry19:01
*** Jack_Iv_ has quit IRC19:03
*** Jack_Iv_ has joined #openstack-telemetry19:04
*** Jack_Iv_ has quit IRC19:10
*** Jack_Iv_ has joined #openstack-telemetry19:11
*** catinthe_ has joined #openstack-telemetry19:14
*** bapalm_ has quit IRC19:14
*** nadya has joined #openstack-telemetry19:15
*** Jack_Iv_ has quit IRC19:16
*** catintheroof has quit IRC19:16
*** bapalm has joined #openstack-telemetry19:18
*** Jack_Iv_ has joined #openstack-telemetry19:20
*** Jack_Iv_ has quit IRC19:20
*** nadya has quit IRC19:22
*** AlexeyAbashkin has joined #openstack-telemetry19:22
*** AlexeyAbashkin has quit IRC19:23
*** nadya has joined #openstack-telemetry19:25
*** nadya has quit IRC19:26
*** rbak has joined #openstack-telemetry19:40
*** Guest42461 is now known as ioni19:47
*** ioni is now known as Guest622219:47
*** jmlowe has joined #openstack-telemetry19:50
*** jwcroppe has quit IRC19:54
*** rwsu has quit IRC20:02
*** catintheroof has joined #openstack-telemetry20:02
*** catinthe_ has quit IRC20:04
*** Marx314 has joined #openstack-telemetry20:05
*** jmlowe has quit IRC20:09
*** hfu has quit IRC20:12
*** rwsu has joined #openstack-telemetry20:22
*** adriant has joined #openstack-telemetry20:25
*** jwcroppe has joined #openstack-telemetry20:25
*** leitan has joined #openstack-telemetry20:34
*** leitan has quit IRC20:37
*** leitan_ has joined #openstack-telemetry20:37
*** fguillot has quit IRC20:47
*** Guest6222 is now known as ioni20:48
*** diogogmt has quit IRC20:48
*** ioni is now known as Guest2741720:48
*** david-lyle has quit IRC20:55
*** catintheroof has quit IRC21:06
*** Jack_Iv_ has joined #openstack-telemetry21:08
*** dave-mccowan has quit IRC21:10
openstackgerritgordon chung proposed openstack/ceilometer: add polling.yaml docs  https://review.openstack.org/40771121:13
*** gordc has quit IRC21:16
*** leitan_ has quit IRC21:25
*** Jack_Iv_ has quit IRC21:29
*** chlong has joined #openstack-telemetry21:43
*** gordc has joined #openstack-telemetry21:47
*** Guest27417 is now known as ioni21:48
*** ioni is now known as Guest1732021:49
sorrisonjd__ you around? Got a quick question about gnocchi and datetime formats21:53
sorrisonor any other gnocchi devs?21:54
gordcsorrison: whatsup?21:54
gordcthis might end up with me telling you i don't know and to ask on mailing list.lol21:55
sorrisonIn the gnocchi storage tests some tests pass in datetimes others pass in numpy.datetime6421:55
sorrisonJust seems strange that the storage driver needs to worry about converting datetimes21:55
sorrisonwriting a storage driver for influxdb and got it down to one failing test and wondering how to handle21:56
sorrisonhttp://logs.openstack.org/60/390260/20/check/gate-gnocchi-tox-db-py27-mysql-ubuntu-xenial/6dcfcb9/console.html21:56
gordcyou have some examples? iirc, we're storing neither technically.21:56
sorrisonIn gnocchi/tests/test_storage.py21:57
gordckk, taking look at code now21:58
sorrisonthe date conversion/handling to the right format happens at a higher level I think it's just the tests that test the storage driver itself that seem to pass in different formats21:58
sorrisoneg https://github.com/openstack/gnocchi/blob/master/gnocchi/tests/test_storage.py#L39721:59
gordcyeah, it should be converting it to numpy.datetime6421:59
sorrisonhttps://github.com/openstack/gnocchi/blob/master/gnocchi/tests/test_storage.py#L57221:59
gordcthe api that is21:59
sorrisonI'm just not sure what format I should be handling21:59
*** Ephur has quit IRC22:01
gordcthe actual timestamps passed for storage should be nanosecond epoch time22:01
gordcthe code you're failing at is i think the range timestamps22:02
sorrisonthe tests for get_measures pass in datetimes and the tests for get_cross_metric_measures pass in numpy.datetime6422:03
*** leitan has joined #openstack-telemetry22:03
gordchmm. so i don't think it matters in carbanara driver because it ends up getting normalised by pandas.Timestamp (which handles both)22:06
gordcfor get_cross_metric_measures specifically22:06
*** chlong has quit IRC22:06
gordcseems like it's passing in datetime.datetime.utcfromtimestamp22:07
*** chlong has joined #openstack-telemetry22:07
*** thorst has quit IRC22:09
*** thorst has joined #openstack-telemetry22:09
gordci'm pretty sure datetime is not the format api is passing so we can probably change tests... i just need to figure out what datetime.datetime.utcfromtimestamp22:12
gordcis22:12
*** chlong has quit IRC22:12
gordcoh. it's a datetime. nm then22:13
gordcdatetime it is22:13
*** jmlowe has joined #openstack-telemetry22:14
sorrisonok so you think all the test_storage tests should pass in datetimes?22:15
*** thorst has quit IRC22:18
gordcsorrison: yeah, it seems like that's what is being passed in by api.22:18
gordcthey all validate using utils.to_datetime which eventually just return https://github.com/openstack/gnocchi/blob/411626f871984f0f0776f6a4fd37bb3a47c05a62/gnocchi/utils.py#L121-L12722:19
sorrisonok cool, thanks gordc I'll have a play changing that and see how it goes22:19
gordci'll push a patch to switch to datetime. won't merge since our gate is sort of blocked right now22:20
gordcor you can push a patch.22:20
gordcif you have it already22:20
sorrisonna I haven't done anything22:20
gordckk. i'll do it now and give it a test. seems fine after i changed one22:20
*** vint_bra has quit IRC22:22
*** david-lyle has joined #openstack-telemetry22:24
sorrisonGreat thanks, I'll rebase my change once its in gerrit22:24
*** mfranc213_ has joined #openstack-telemetry22:29
*** dave-mccowan has joined #openstack-telemetry22:32
mfranc213_hello.  i am trying to fine where ceilometer sets the volume for a cpu_util request for libvirt.  i understand that the rate_of_change transformation actually produces the % based on current and previous cpu_times, but i cannot see where s.volume is set in this case.  would someone be able to assist?22:32
openstackgerritgordon chung proposed openstack/gnocchi: use datetime when defining series range  https://review.openstack.org/40773722:32
mfranc213_s/fine/find :)22:32
*** dave-mcc_ has joined #openstack-telemetry22:33
gordcsorrison: let me know if that works for you ^22:34
gordcmfranc213_: it's done here https://github.com/openstack/ceilometer/blob/master/ceilometer/transformer/conversions.py#L208-L21122:35
*** dave-mccowan has quit IRC22:37
mfranc213_gordc: thank you.  i guess my question is: where is s.volume set when it's used, for example, here: https://github.com/openstack/ceilometer/blob/master/ceilometer/transformer/conversions.py#L20422:37
*** thorst_ has joined #openstack-telemetry22:38
*** thorst_ has quit IRC22:38
gordcmfranc213_: when it's used?22:39
sorrisonso far so good gordc22:39
mfranc213_gordc: sorry that i'm not being clear.  L204 shows that s.volume is already set.  where was it set?22:40
gordcbasically it calculates rate of change and then if you follow _convert() part, you should see it starts doing stuff there.22:40
gordcline 204 is just extracting the volume from current sample (which it takes from cpu sample)22:41
*** david-lyle has quit IRC22:41
mfranc213_gordc: the rate of change based on difference fetches of cpu_time.  where does that fetch happen?22:41
gordcmfranc213_: oh. there's a compute agent which polls hypervisor for cputime22:41
mfranc213_gordc: i looked at sample.Sample and elsewhere (and everywhere) but not in the right place :(22:41
mfranc213_gordc: i looked in the agent code also.22:42
gordcsorrison: good to hear22:42
mfranc213_gordc: i'm probably just being obtuse.22:42
gordcmfranc213_: no comment :P j/k22:42
gordcit starts here: https://github.com/openstack/ceilometer/blob/master/ceilometer/compute/pollsters/cpu.py#L36-L4922:43
mfranc213_gordc: looking ...22:43
gordcthat will run on compute agent and generate cputime samples.22:43
*** jmlowe has quit IRC22:43
*** harlowja has quit IRC22:43
gordcthat will push onto a queue and get picked up by notificaiton agent which does the transformation into cpu_util22:43
*** harlowja has joined #openstack-telemetry22:43
sorrisongordc wrt gate brokenness, are you talking about the oslo.db 4.15.0 breakage?22:44
mfranc213_gordc: ah yes, but that's CPUPollster, and setup.cfg sends the caller to CPUUtilPollster, whose get_samples calles inspect_cpu_util, which is not implemented for libvirt.22:44
mfranc213_gordc: you have having at least 2 conversations simultaneously.  this is very impressive! :)22:45
gordcsorrison: that... and heat also switched to devstack plugins recently so we need to upgrade our gate to explicitly enable heat https://review.openstack.org/#/c/407622/22:45
gordcthen we can merge the oslo.db stuff22:45
sorrisonah ok I see, lovely little tangle22:46
gordcmfranc213_: cpuutilpollster will not generate anything if on libvirt since it doesn't provide that functionality22:46
openstackgerritSam Morrison proposed openstack/gnocchi: WIP Resurrect influxdb backend driver  https://review.openstack.org/39026022:47
gordcsorrison: i'm hoping that's all the issues for blocked gate.22:47
mfranc213_gordc: yes, so how does a call to cpu_util end up producing the desired result in the case of libvirt?  i understand what you said above about the notification agent but don't see the tie from the cpu_util request to the result.  i think this will be my last question since i've hit my allotment.22:48
*** Guest17320 is now known as ioni22:49
*** ioni is now known as Guest189022:50
*** cdent has quit IRC22:53
gordcmfranc213_: no worries. in the libvirt path, there is no call to cpu_util. it generates cputime22:54
gordcif you specify cpu_util transformation in pieline.yaml (which it does by default), it will end up deriving cpu_util samples as well22:55
mfranc213_gordc: thank you very much.  i'm going to go away and think about what you said, and go exploring again. i really appreciate your help.22:56
gordcnp22:58
*** rcernin has quit IRC23:02
sorrisongordc you mind if I rebase your change ontop of "fix oslo.db 4.15.0 breakage" change?23:03
gordcsorrison: sure. go ahead.23:08
*** dave-mcc_ has quit IRC23:08
*** leitan_ has joined #openstack-telemetry23:16
*** leitan has quit IRC23:18
sorrisonhmm gerrit won't let me do that for some reason23:21
openstackgerritSam Morrison proposed openstack/gnocchi: use datetime when defining series range  https://review.openstack.org/40773723:21
sorrisonweb interface lets me23:21
openstackgerritSam Morrison proposed openstack/gnocchi: WIP Resurrect influxdb backend driver  https://review.openstack.org/39026023:22
openstackgerritgordon chung proposed openstack/aodh: Replace retrying with tenacity  https://review.openstack.org/36760023:23
*** jwcroppe has quit IRC23:26
gordcsorrison: usually it's other way around.23:26
gordcheading off. ttyl23:26
*** gordc has quit IRC23:26
*** jwcroppe has joined #openstack-telemetry23:26
*** Jack_Iv has joined #openstack-telemetry23:29
*** jwcroppe has quit IRC23:31
*** Jack_Iv has quit IRC23:34
*** masber has quit IRC23:37
*** thorst_ has joined #openstack-telemetry23:39
*** jmlowe has joined #openstack-telemetry23:41
*** rbak has quit IRC23:45
jmlowegordc: the ceilometer compute agent generates the thing I'm interested in storing in gnocchi, hypervisor cpu load, for some reason I have to report this number to the NSF23:45
*** rbak has joined #openstack-telemetry23:45
*** rbak has quit IRC23:46
*** thorst_ has quit IRC23:48
*** Guest1890 is now known as ioni23:50
*** ioni is now known as Guest8170623:51
*** lhx_ has joined #openstack-telemetry23:53
*** jmlowe has quit IRC23:56
*** catintheroof has joined #openstack-telemetry23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!