Friday, 2015-08-21

« Thursday, 2015-08-20 Index Saturday, 2015-08-22 »
*** jwcroppe has quit IRC00:15
*** thorst has joined #openstack-ceilometer00:17
*** thorst has quit IRC00:30
*** thorst has joined #openstack-ceilometer00:32
*** thorst has left #openstack-ceilometer00:33
*** _cjones_ has quit IRC00:47
*** haigang has joined #openstack-ceilometer01:00
*** qiaowei has joined #openstack-ceilometer01:04
openstackgerritRen Qiaowei proposed openstack/ceilometer: xenapi: support the session when xenserver is master  https://review.openstack.org/21539301:15
*** ahilan has joined #openstack-ceilometer01:30
*** jwcroppe has joined #openstack-ceilometer01:31
openstackgerritliusheng proposed openstack/ceilometer: Avoid recording whole instance info in log  https://review.openstack.org/21493101:36
*** ahilan has quit IRC01:46
*** ahilan has joined #openstack-ceilometer01:49
*** ankita_w_ has quit IRC01:58
*** ankita_wagh has joined #openstack-ceilometer02:14
*** changbl_ has joined #openstack-ceilometer02:14
*** ankita_wagh has quit IRC02:15
*** ankita_wagh has joined #openstack-ceilometer02:16
*** ankita_wagh has quit IRC02:18
*** ankita_wagh has joined #openstack-ceilometer02:19
openstackgerritCan ZHANG proposed openstack/ceilometer: Add HTTP location header to POST request  https://review.openstack.org/21540402:20
*** jwcroppe has quit IRC02:20
*** ddieterly has quit IRC02:33
*** ankita_w_ has joined #openstack-ceilometer02:47
*** ankita_wagh has quit IRC02:50
*** ahilan has quit IRC02:59
*** admin0 has quit IRC03:01
*** changbl_ has quit IRC03:14
*** tiny-hands has quit IRC03:25
*** ddieterly has joined #openstack-ceilometer03:34
*** ddieterly has quit IRC03:38
*** fawadkhaliq has joined #openstack-ceilometer04:02
*** ankita_w_ has quit IRC04:15
*** ankita_wagh has joined #openstack-ceilometer04:16
*** ankita_wagh has quit IRC04:17
*** ankita_wagh has joined #openstack-ceilometer04:18
*** haigang has quit IRC04:27
openstackgerritMerged openstack/ceilometer: Corrected test_fallback_meter_path test case  https://review.openstack.org/21500104:30
*** ddieterly has joined #openstack-ceilometer04:35
*** ddieterly has quit IRC04:40
*** julim has quit IRC04:47
*** fawadkhaliq has quit IRC04:51
*** fawadk has joined #openstack-ceilometer04:51
*** yprokule has joined #openstack-ceilometer04:58
openstackgerritOpenStack Proposal Bot proposed openstack/ceilometer: Updated from global requirements  https://review.openstack.org/21387405:08
*** jwcroppe has joined #openstack-ceilometer05:21
*** ddieterly has joined #openstack-ceilometer05:36
*** ddieterly has quit IRC05:40
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: DO NOT MERGE: jsonpath_rw_ext test  https://review.openstack.org/21546105:59
*** gzhai has joined #openstack-ceilometer06:23
*** ankita_wagh has quit IRC06:24
*** nadya has joined #openstack-ceilometer06:27
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Use jsonpath_rw_ext instead of jsonpath  https://review.openstack.org/21546106:35
*** ddieterly has joined #openstack-ceilometer06:36
openstackgerritCan ZHANG proposed openstack/ceilometer: Return sample links when POST /v2/meters  https://review.openstack.org/21540406:36
openstackgerritMerged openstack/ceilometer: handle list payloads in notifications  https://review.openstack.org/21320206:37
*** ddieterly has quit IRC06:41
openstackgerritCan ZHANG proposed openstack/ceilometer: Return sample links when POST /v2/meters  https://review.openstack.org/21540406:46
*** shardy has joined #openstack-ceilometer07:27
*** lsmola has joined #openstack-ceilometer07:31
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: integration tests  https://review.openstack.org/21518907:35
*** ddieterly has joined #openstack-ceilometer07:37
*** ddieterly has quit IRC07:42
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: next upstream jsonpath_rw  https://review.openstack.org/21546107:54
*** gzhai has quit IRC07:57
*** qiaowei has quit IRC08:03
*** fawadk has quit IRC08:06
*** fawadkhaliq has joined #openstack-ceilometer08:06
*** yassine has joined #openstack-ceilometer08:09
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: failfast  https://review.openstack.org/21518908:21
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: adjusts timeout  https://review.openstack.org/21551008:21
*** fawadkhaliq has quit IRC08:21
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: TEST: gate tests  https://review.openstack.org/21458008:24
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: failfast  https://review.openstack.org/21518908:24
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: adjusts timeout  https://review.openstack.org/21551008:24
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: failfast  https://review.openstack.org/21518908:33
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: adjusts timeout  https://review.openstack.org/21551008:33
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: TEST: gate tests  https://review.openstack.org/21458008:33
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: failfast  https://review.openstack.org/21518908:35
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: adjusts timeout  https://review.openstack.org/21551008:35
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: TEST: gate tests  https://review.openstack.org/21458008:36
*** ddieterly has joined #openstack-ceilometer08:38
*** ddieterly has quit IRC08:43
*** fawadkhaliq has joined #openstack-ceilometer08:43
*** jwcroppe has quit IRC08:58
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: Add jsonpath logging  https://review.openstack.org/21546109:08
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: use jsonpath_rw_ext  https://review.openstack.org/21552909:08
*** cdent has joined #openstack-ceilometer09:36
*** ddieterly has joined #openstack-ceilometer09:39
_gryfhi.09:39
_gryfyesterday I was asked about adding aodh to my devstack setup09:40
_gryfi've added line "enable_service aodh" to my local.conf, cleand up and stack the thing09:40
_gryfhowever there is no aodh presence09:40
_gryfi have an impression, that i'm doing it wrong ;)09:43
sileht_gryf, your local.conf should have:09:43
sileht      [[local|localrc]]09:43
sileht      # Enable the aodh alarming services09:43
sileht      enable_plugin aodh https://github.com/openstack/aodh master09:43
sileht      enable_service aodh-evaluator,aodh-notifier,aodh-api09:43
sileht      disable_service ceilometer-alarm-notifier ceilometer-alarm-evaluator09:43
*** ddieterly has quit IRC09:43
_gryfsileht, I'll try that, thanks09:44
*** nadya has quit IRC09:44
sileht_gryf, I will update the doc :p09:44
_gryfsileht, excellent :D09:44
openstackgerritMehdi Abaakouk (sileht) proposed openstack/aodh: doc: Fix the devstack configuration  https://review.openstack.org/21555009:45
sileht_gryf, ^09:45
_gryfsileht, great :)09:45
*** nadya has joined #openstack-ceilometer09:49
*** fawadkhaliq has quit IRC09:57
*** guy_rodrigue has joined #openstack-ceilometer10:01
openstackgerritMerged openstack/ceilometer: Improve comments in notification.py  https://review.openstack.org/20819210:10
*** idegtiarov_ has quit IRC10:26
*** jwcroppe has joined #openstack-ceilometer10:36
*** fawadkhaliq has joined #openstack-ceilometer10:36
*** ddieterly has joined #openstack-ceilometer10:39
*** ddieterly has quit IRC10:44
*** jwcroppe has quit IRC10:44
*** jwcroppe has joined #openstack-ceilometer10:45
_gryfhm.10:49
*** jwcroppe has quit IRC10:49
_gryfsileht, i stacked the thing, it pull the aodh, but i cant see any aodh-* services in screeen…10:50
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: DO NOT MERGE: integration test  https://review.openstack.org/21458010:50
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: devstack: fix gnocchi url with aodh  https://review.openstack.org/21557810:50
*** fawadkhaliq has quit IRC10:51
*** haigang has joined #openstack-ceilometer10:52
sileht_gryf, are you sure you don't override ENABLED_SERVICES in your local.conf ? or perhaps you have an old localrc file.10:57
openstackgerritJulien Danjou proposed openstack/gnocchi: indexer: raise an error if deleting a non-existent metric  https://review.openstack.org/21421710:57
_gryfsileht, nope10:57
_gryfit's a brand new local.conf10:57
*** fawadkhaliq has joined #openstack-ceilometer10:59
sileht_gryf, perhaps your can create a paste with your ./devstack.sh log10:59
_gryfsileht, sure11:00
*** cdent has quit IRC11:06
*** cdent has joined #openstack-ceilometer11:14
openstackgerritChris Dent proposed openstack/gnocchi: Rudimentary support for influxdb in devstack plugin  https://review.openstack.org/21531311:18
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: use jsonpath_rw_ext  https://review.openstack.org/21552911:22
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: Add jsonpath logging  https://review.openstack.org/21546111:22
*** thorst has joined #openstack-ceilometer11:31
*** ddieterly has joined #openstack-ceilometer11:40
gordcsileht: does jamielennox's patch fix your keystone issue or is this needed? https://review.openstack.org/#/c/215339/11:45
*** ddieterly has quit IRC11:45
*** jwcroppe has joined #openstack-ceilometer11:45
*** fawadkhaliq has quit IRC11:57
*** fawadkhaliq has joined #openstack-ceilometer11:57
gordcsileht:  i like your testing techniques :) https://review.openstack.org/#/c/21552911:58
silehtgordc, no that an other issue :)12:03
_gryfsileht, http://paste.openstack.org/show/423355/12:03
silehtgordc, my jsonpath-rw-ext lib pass all upstream tests, so I try to understand what is wrong with tempest12:04
*** pradk has quit IRC12:04
sileht_gryf, this is not a complete log12:04
cdent_gryf: can you share your local.conf too?12:05
_gryfsileht, hm. that's the output of the stack command12:05
_gryfcdent, sure12:05
cdentit looks like paste ate some of your data12:05
sileht\o/ gate-ceilometer-dsvm-integration SUCCESS in 31m 08s12:05
cdentsileht: woot!12:06
_gryfcdent, you're right :/12:06
cdentsileht: now for the sad part: you have to scrutinize the logs to see that it actually did what it thinks it did12:06
cdentwhen I was doing the grenade stuff it would "pass" but only because it wasn't actually doing anything with ceilometer12:07
silehtcdent, it did I follow the log since this morning12:07
cdentcool12:07
_gryfcdent, http://paste.openstack.org/show/423360/ - local.conf12:07
silehtcdent, http://logs.openstack.org/80/214580/5/experimental/gate-ceilometer-dsvm-integration/804c7a8/console.html#_2015-08-21_11_52_34_05612:08
cdentaw, how pretty12:08
cdenti'll have to let FND know12:08
nadyafolks, why event-alarm-evaluator is being implemented in Ceilo but not in Aodh?12:09
silehtnadya, oO12:10
silehtnadya, it's on aodh: https://review.openstack.org/#/q/status:open+project:openstack/aodh+branch:master+topic:bp/event-alarm-evaluator,n,z12:11
silehthttps://review.openstack.org/#/q/project:openstack/aodh+branch:master+topic:bp/event-alarm-evaluator,n,z12:11
silehtceilometer code have been abandoned: https://review.openstack.org/#/q/project:openstack/ceilometer+branch:master+topic:bp/event-alarm-evaluator,n,z12:11
*** jwcroppe has quit IRC12:12
nadyasileht: yep, right. sorry :)12:12
_gryfsileht, logs have around 12k lines. seems like there is a limit on paste.openstack.org to around 1000 lines :/12:12
cdent_gryf: there's nothing obviously wrong with your local.conf so I guess the next step is to comb through your stack.sh.log. Do you have some other way you can share it (dropbox, url somewhere, etc)?12:12
nadyasileht: I was looking at https://review.openstack.org/#/c/195376/12:12
_gryfcdent, gimme a sec12:13
nadyasileht: didn't see a red cross12:13
silehtno problem12:13
*** guy_rodrigue has left #openstack-ceilometer12:14
*** prad_ has joined #openstack-ceilometer12:18
*** alejandrito has joined #openstack-ceilometer12:20
openstackgerritMehdi Abaakouk (sileht) proposed openstack/aodh: doc: Fix the devstack configuration  https://review.openstack.org/21555012:20
*** yassine has quit IRC12:22
*** ahilan has joined #openstack-ceilometer12:23
_gryfcdent, sileht, here are logs: https://www.dropbox.com/s/ulpd7k1jzj2tndw/devstack_logs.tgz?dl=012:23
* cdent looks12:24
*** ddieterly has joined #openstack-ceilometer12:24
silehtto finish the telemetry integrated job: I have 3 pending fixes, reviews are welcome: https://review.openstack.org/#/c/215189/ , https://review.openstack.org/#/c/215578/ , https://review.openstack.org/#/c/215339/12:27
*** idegtiarov_ has joined #openstack-ceilometer12:27
sileht_gryf, your devstack installation doesn't success12:28
_gryfsileht, sort of yes12:28
sileht_gryf, and fail to setup 'tempest'12:28
_gryfsileht, right12:28
sileht_gryf, nothing related to ceilo/aodh :p12:28
_gryfsileht, nevertheless the screen is started with all the services but the aodh12:28
sileht_gryf, that normal tempest is setuped before some services are run12:29
_gryfsileht, hm.12:29
*** yassine has joined #openstack-ceilometer12:30
cdentyeah _gryf I'd suggest disabling tempest and see where that gets you12:30
_gryfcdent, ok. how can I do that?12:31
cdentI read through the log and the aodh parts appear to be running without failing12:31
cdentdisable_service tempest12:31
_gryfcdent, yup, nothing suspicious12:31
*** edmondsw has joined #openstack-ceilometer12:31
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: devstack: fix gnocchi url with aodh  https://review.openstack.org/21557812:32
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: DO NOT MERGE: integration test  https://review.openstack.org/21458012:32
_gryfcdent, stacking…12:32
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: failfast  https://review.openstack.org/21518912:37
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: adjusts timeout  https://review.openstack.org/21551012:37
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration: downscale the stack  https://review.openstack.org/21560412:37
gordcsileht: why do we need to fail fast if it's just one test?12:37
silehtgordc, it one 'gabbi test', but that's still many tests from the unittest runner PoV12:38
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: DO NOT MERGE: integrated downscaling test  https://review.openstack.org/21560612:40
prad_sileht, lemme know how this one turns out https://review.openstack.org/#/c/215529 .. might solve the mystery i'm facing12:40
prad_sileht, though you need to update the version in global requirements i think12:40
*** ddieterly has quit IRC12:40
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration: downscale the stack  https://review.openstack.org/21560412:42
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: failfast  https://review.openstack.org/21518912:42
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: integration test: adjusts timeout  https://review.openstack.org/21551012:42
nadyagordc: Gordon, looks like it's implemented https://blueprints.launchpad.net/ceilometer/+spec/mandatory-limit? I'm walking through all bps and decided to change some states...12:42
gordcnadya: there's still one patch12:42
gordchttps://review.openstack.org/#/c/199180/12:42
gordci'm looking to update tempest tests just in case this patch breaks stuff12:43
gordci don't think it will...12:43
cdentsileht: I guess your tasty jsonpath bait worked...12:43
_gryfcdent, sileht, thanks guys. indeed, failing tempest was the culprit12:43
cdentglad to hear it _gryf12:43
sileht_gryf, welcome12:43
nadyagordc: ok, I'll run experimental job there because of postgres failure. Hope it was cause by the fact that the job was not completed yet...12:45
gordcnadya: good point. yeah, we should gate on postgres when we can.12:47
prad_sileht, gordc, same error https://jenkins07.openstack.org/job/gate-tempest-dsvm-full/18231/consoleFull12:47
prad_as i was getting12:47
prad_so looks like the new library is having issues on devstack?12:48
silehtI confirm12:49
*** julim has joined #openstack-ceilometer12:49
gordcprad_: nah, you're just going crazy. :)12:49
prad_i banged my head against the wall trying to find bugs in my code and finally gave up last night12:50
silehtprad_, I'm so sorry12:50
prad_gordc, that i can confirm :)12:50
prad_sileht, no worries, so we have a fallback ?12:51
prad_sileht, i was using the new library as from jsonpath_rw_Ext import parser and doing parser.parse12:51
prad_sileht, you were using it differently in your patch ? jsonpath_rw_ext.parse ? you changed that in newer versions i assume12:52
*** exploreshaifali has joined #openstack-ceilometer12:52
*** prad_ is now known as pradk12:53
*** tiny-hands has joined #openstack-ceilometer12:54
silehtpradk, 0.1.5 only introduce this shortcut12:55
pradkcool12:56
openstackgerritMerged openstack/aodh: Revert "Use generic keystone uri in devstack ..."  https://review.openstack.org/21533912:58
*** tiny-hands has quit IRC13:00
*** haigang has quit IRC13:10
gordcsileht: i'll try to get you one more review on keystonemiddleware patch later today.13:13
silehtpradk, cdent gordc it's really wierd I see the processing of samples that tempest doesn't see13:13
cdentsileht: have you checked timestamps?13:14
cdentmaybe they are being seen after the tempest tests have timed out13:14
gordcsileht: is it beyond 1000 samples? the swift tests don't really filter much and we now default to 1000 return samples (lowering to 100)13:17
*** nadya has quit IRC13:17
gordccdent: it's kind of weird that it only happens on those jsonpath patch... maybe it's encoding it in something else?13:17
silehtcdent, gordc this is a timing issue13:18
*** nadya has joined #openstack-ceilometer13:19
silehtcdent, gordc for example 'memory.*50b7f27c-dc50-4cd9-9105-cb524d87f995' is processed at 2015-08-21 12:44:14.789 when tempest stop trying at  2015-08-21 12:38:28,36713:19
cdentyeah, that's what I figured13:19
cdentthere are some _very_ weird forces at work controlling the flow of time on the gate nodes13:20
gordchow is it only jsonpath_rw_ext patch though? that's very weird.13:20
silehtgordc, in ceilometer we recreate the json parser  for each field of a notification13:22
silehtgordc, my change adds many of new token to jsonpath_rw, that mean many new class to instanciante each times13:23
*** jwcroppe has joined #openstack-ceilometer13:23
gordcsileht: ah i see... so our processing is slow as hell with new lib? is it slow with current lib as well?13:24
silehtgordc, that my guess13:24
cdentthat's pretty badly slow :(13:27
cdentif that's what it actually is13:27
cdentdo we need a new parser?13:27
gordcsileht: does creating an endpoint per definition parallelise work or does it still loop through each? right now we have one endpoint and it runs through all the definitions.13:28
*** nadya has quit IRC13:29
*** jwcroppe has quit IRC13:30
gordccdent: i think it only happens with the new lib... the exist lib seems to process relatively efficiently (not much diff between when api first polls for sample, and when we process it at notification agent)13:32
gordcor we are just getting super lucky and somehow the queue isn't backed up each time.13:33
cdentgordc: what I mean was: why are we recreating a new parse with each go, rather than reusing the existing parser13:33
cdentthe jsonpath-rw has identified a couple of problems in sileht code, dunno if that would be a contributing factor13:33
* sileht writting a change that prepare parser when agent load13:33
gordccdent: ah, i thought you were talking about the parser in general, not initialising new one13:33
*** haigang has joined #openstack-ceilometer13:38
*** r-mibu has quit IRC13:45
*** r-mibu has joined #openstack-ceilometer13:45
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: use jsonpath_rw_ext  https://review.openstack.org/21552913:48
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: TEST: Add jsonpath logging  https://review.openstack.org/21546113:48
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: test: cache jsonpath_rw parser  https://review.openstack.org/21564013:48
silehtcdent, an alternative to jsonpath is perhaps yaql, it's still completly undocumented...13:50
silehtthat a project done for murano13:50
*** rbak has joined #openstack-ceilometer13:50
silehtit have some small syntax differences:13:50
sileht* foo.[*].bar is not supported as alias of foo[*].bar13:51
pradki ran into issues using yaql actually.. it doesnt seem very robust and not to mentioned no docs at all13:51
sileht* all jsonpath must start be $.13:51
sileht* it allows many arithmetic operators13:52
pradksileht, looking at the code all you are really doing is inheriting the lexar class from jsonpath_rw and adding the filter [parsing.. why is there such a performance difference?13:52
pradkare we instantiating something more than needed?13:52
pradkunless jsonpath_rw is performance beast and we are just tipping it off the cliff13:53
silehtit doesn't use jsonpath_rw parser it have it's own implementation (that looks similar)13:53
gordcpradk: jsonpath_rw seems fine.13:54
pradksileht, ah you changed it? i see class Expression(jsonpath_rw.JSONPath):13:54
silehtfrom the performance point of view I don't think yaql is better than jsonpath_rw, they use the same parsing method13:54
pradkgordc, ok13:54
silehtpradk, I have fixed the upstream remarks only in the PR13:54
pradksileht, hmm ok13:55
silehtpradk, they are only cosmetic changes13:55
pradkok13:55
silehtpradk, gordc cdent I have preload all parsers to see if we have a performance difference: https://review.openstack.org/#/c/215640/13:55
pradkgordc, mind looking at https://review.openstack.org/#/c/213832/ .. i removed it from the other dep chain, should be good to go13:56
pradkgordc, also has the lookup renaming etc in it13:56
pradksileht, cool13:57
*** ahilan has quit IRC13:58
*** nadya has joined #openstack-ceilometer14:00
gordcsileht:  you think yaql is a good alternative? a problem with switching is we need to code support for existing schema...14:00
silehtgordc, I don't known, the pain point is clearly the inexistance of the documentation14:01
cdenti'd be very surprised if jsonpath is the sole source of the problem here14:01
cdentif so that's really weird14:02
gordcsileht: yeah... :(14:02
gordccdent: originally i thought it was just the gate. when prad told me last week, there were a few other jobs that ran (all) tests really slow.14:03
gordccdent: but it seems like it's only jsonpath-rw-ext patch that is slow constantly... and significantly slower14:03
silehtcdent, with jsonpath_rw_ext, the notification agent process 5000 samples less than without but run 7 minutes more14:04
cdenthuh. clearly you did something wrong :)14:05
gordchahah14:05
pradkhmm so only reason for this new library is for cpu meters, i'm worried if its woth taking on this much performance hit for just converting these14:06
silehtif the part that take many times is 'build the node tree', preloading the parser will be a boost of performance14:07
silehtif this is the message parsing, we are fucked :(14:07
pradkyea preloading seems like a good idea for sure .. if the impact is significant it would save us, but if we just end up barely passing with periodic failures i would worry keeping this in14:08
gordci wonder if we're the channel with most f bombs... we must be.lol14:10
gordcpradk: i agree. ^14:10
pradkand now you just declared it and its logged permanently14:10
pradk:)14:11
*** ahilan has joined #openstack-ceilometer14:11
silehtahaha14:11
*** ityaptin has quit IRC14:11
gordcdisclaimer: for new devs, we are a friendly bunch. we just like to say 'fuck' when appropriate. see: all previous use cases.14:13
*** jasonamyers has joined #openstack-ceilometer14:20
gordcpradk: tbh, prefixing paths with $ is pretty good idea.14:20
*** pradk has quit IRC14:21
*** haigang has quit IRC14:21
gordcand then we can just say, does this start with '$'? and does it contain '*' and then we won't need that multi attr at all.14:22
*** prad_ has joined #openstack-ceilometer14:22
*** prad_ is now known as pradk14:23
nadyagordc: https://review.openstack.org/#/c/215485/14:30
*** pradk has quit IRC14:31
*** pradk has joined #openstack-ceilometer14:32
gordcnadya: cool! thanks!14:32
*** fawadkhaliq has quit IRC14:41
*** exploreshaifali has quit IRC14:43
*** jwcroppe has joined #openstack-ceilometer14:57
*** jwcroppe has quit IRC14:59
gordcpradk: http://eavesdrop.openstack.org/irclogs/%23openstack-ceilometer/latest.log.html#t2015-08-21T14:22:0214:59
*** jwcroppe has joined #openstack-ceilometer14:59
gordcwhat do you think of that idea?14:59
gordcnot the f-bomb part.15:00
*** r-daneel has joined #openstack-ceilometer15:02
silehtgordc, pradk, cdent tests pass when we preload parser: https://jenkins01.openstack.org/job/gate-tempest-dsvm-full/24738/console15:03
cdenthuh15:03
cdentcomputers are weird15:03
* cdent becomes a potter15:03
gordcelectrician is more useful.15:04
cdenttoo much tech15:06
cdentI can do clay with water, dirt, fire15:06
cdentyou can record that in your cdent book as well, gordc: I made some progress on a bfa in ceramics15:07
gordcno ones needs fancy pots.15:07
cdentno one _needs_ ceilometer15:07
gordcdamn. that hurts bro15:08
gordcwe all need lasers in the sky.15:08
gordcsileht: hmm. that actually worked pretty well15:10
* cdent just got called bro15:10
cdentI think I need find a better tshirt15:11
gordccdent: my speech hasn't changed since eighth grade15:11
gordcpradk: ^ want to change your patch to that?15:12
gordcand see if we can consistently achieve speed?15:12
pradksileht, cool15:13
pradkgordc, sure, does it consistently pass?15:14
silehtI cleanup the patch to make it mergeable15:14
pradksileht, cool, i can depend my patch on yours15:14
gordcsileht: ok... should we be patching something in jsonpath-rw-ext? or at least opening a bug?15:15
silehtgordc, https://review.openstack.org/#/c/215640/15:15
pradksileht, and what version should we be using?15:16
pradki'll update global requirements15:16
silehtpradk, I guess 0.1.5 to have the interface as jsonpath_rw15:16
pradk0.1.2 is what we have now15:16
pradkok15:16
openstackgerritMerged openstack/gnocchi: devstack: fix gnocchi url with aodh  https://review.openstack.org/21557815:17
*** haigang has joined #openstack-ceilometer15:18
*** fawadkhaliq has joined #openstack-ceilometer15:19
gordc... i don't understand why putting it in init is faster.15:20
cdentcreating the parser?15:20
cdentoh I see15:21
cdentisn't the idea here that the parsing is being done one instead of over and over again?15:21
gordccdent: it looks like it's does parsing for all attr in init... but it's still looks like it's parsing it one by one...15:24
silehtgordc, create the parser looks expensive15:24
gordci will switch to plumber.15:25
cdentit's pyparser-based, yeah?15:25
cdentor do I mean ply?15:25
silehtits ply naed15:25
sileht*based*15:25
gordcsileht: don't we create a parser per attr? on line 6715:25
cdentyeah, I had a similar experience with a ply thing I made: best to only make it once15:25
silehtyaql for example cache all parsers if you use the simplier API yaql.eval(...)15:27
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Use jsonpath_rw_ext for meter definitions  https://review.openstack.org/21552915:30
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: preload jsonpath_rw parsers  https://review.openstack.org/21564015:30
cdentsileht: if you user parse() from jsonpath_rw it is creating a new JsonPathParser class with each call15:33
silehtyes15:33
cdentyou ought to be able to make a parse_class = JsonPathParser()15:33
cdentonce15:33
cdentand then call parse_class.parse(<data>) whenever you like15:34
silehtcdent, good catch I will do that15:34
* cdent changes gabbi :)15:35
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Move profiler meters to yaml  https://review.openstack.org/21383215:35
gordcsileht: comment in patch... but i like the idea.15:36
gordcand i hate i have to rebase massively.15:36
gordcsileht: you may have to rebase on this https://review.openstack.org/#/c/212815/ if it merges15:38
gordcbut we can deal with that if/when it happens.15:38
silehtgordc, shouldn't be a big deal15:38
*** ankita_wagh has joined #openstack-ceilometer15:38
gordcsileht: cool cool. good optimisation15:39
gordci should look at events code15:39
silehtgordc, event code already do that in __init__15:40
gordcsileht: oh cool.15:40
* gordc goes back to sleep15:40
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Use jsonpath_rw_ext for meter definitions  https://review.openstack.org/21552915:43
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: preload jsonpath_rw parsers  https://review.openstack.org/21564015:43
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Use jsonpath_rw_ext for meter/event definitions  https://review.openstack.org/21552915:44
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Use jsonpath_rw_ext for meter/event definitions  https://review.openstack.org/21552915:46
*** _cjones_ has joined #openstack-ceilometer15:47
*** _cjones_ has quit IRC15:47
*** _cjones_ has joined #openstack-ceilometer15:47
silehtcdent, fyi wsme should be released on monday15:47
cdentawesome15:47
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: DO NOT MERGE: integration test  https://review.openstack.org/21458015:49
openstackgerritMehdi Abaakouk (sileht) proposed openstack/gnocchi: DO NOT MERGE: integrated downscaling test  https://review.openstack.org/21560615:49
*** shardy_ has joined #openstack-ceilometer15:54
*** haigang has quit IRC15:55
*** shardy has quit IRC15:56
*** shardy_ has quit IRC16:00
*** shardy has joined #openstack-ceilometer16:01
openstackgerritMerged openstack/ceilometer: Introduce Guru Meditation Reports into Ceilometer  https://review.openstack.org/19636916:06
*** thorst has quit IRC16:11
*** ankita_wagh has quit IRC16:11
openstackgerritMerged openstack/ceilometer: full multi-meter support  https://review.openstack.org/21281516:13
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: Use jsonpath_rw_ext for meter/event definitions  https://review.openstack.org/21552916:31
openstackgerritMehdi Abaakouk (sileht) proposed openstack/ceilometer: preload jsonpath_rw parsers  https://review.openstack.org/21564016:31
*** exploreshaifali has joined #openstack-ceilometer16:34
*** thorst has joined #openstack-ceilometer16:35
*** rbak has quit IRC16:37
*** AlexeyElagin has quit IRC16:41
*** ankita_wagh has joined #openstack-ceilometer16:42
*** yprokule has quit IRC16:47
*** ankita_wagh has quit IRC16:49
*** ankita_wagh has joined #openstack-ceilometer16:50
*** ankita_wagh has quit IRC16:54
*** nadya has quit IRC17:04
*** ankita_wagh has joined #openstack-ceilometer17:08
*** ankita_wagh has quit IRC17:09
*** lsmola has quit IRC17:13
*** ankita_wagh has joined #openstack-ceilometer17:15
*** pradk has quit IRC17:26
*** admin0 has joined #openstack-ceilometer17:27
*** nadya has joined #openstack-ceilometer17:29
*** prad_ has joined #openstack-ceilometer17:31
openstackgerritgordon chung proposed openstack/ceilometer: make telemetry sample payloads dictionaries  https://review.openstack.org/21385117:42
*** nadya has quit IRC17:44
*** nadya has joined #openstack-ceilometer17:45
*** jasonamyers has quit IRC17:46
*** nadya has quit IRC17:50
*** annasort has joined #openstack-ceilometer17:53
*** prad_ is now known as pradk17:55
pradkgordc, sileht, the tests are still running slow with update patch for me17:55
gordcpradk:  https://review.openstack.org/#/c/215640/ this?17:57
pradkgordc, so i applied those new patches locally and even the regular unit tests seem to be going slow17:58
pradklemme look at how jenkins is doing for that17:59
gordchttps://jenkins03.openstack.org/job/gate-tempest-dsvm-full/20841/consoleFull17:59
gordcseems decent17:59
pradkhmm this is failing  https://review.openstack.org/#/c/215640/18:00
pradk215529 i mean18:02
pradkhttps://review.openstack.org/#/c/21552918:03
gordcpradk: grenade?18:06
pradkah now tempest looks like it passed18:07
*** ankita_wagh has quit IRC18:08
pradkfor some reason the meter tests seem to be running slower than normal18:11
pradkmay be its just me18:11
gordclet me try and see if my devstack fixed itself18:16
* cdent glances at gordc 18:17
*** boris-42 has quit IRC18:20
gordcbroke...18:22
* gordc goes back to sleep again18:22
cdentI've just created a pollster that records the current temperature from wherever18:29
cdentjust to see if I could18:30
cdenttaught me a lot18:30
* gordc waits for cdent's ip to get blocked18:30
cdentapparently I can make 1200 queries per minute18:31
cdenti've only got two resources set up18:32
cdentshould be okay18:32
cdentthis is what we should be telling people to do18:32
cdentno more code in ceilo, write your own. cheers thanks alot18:32
*** cdent has quit IRC18:34
gordcwhat are you polling?18:36
gordcoh, he left... meh18:36
gordcseems my devstack works... will test18:38
*** jfluhmann has quit IRC18:45
*** yassine has quit IRC18:49
*** ankita_wagh has joined #openstack-ceilometer18:53
*** fawadkhaliq has quit IRC18:58
*** ahilan has quit IRC19:08
pradkgordc, i updated this addressing your comments https://review.openstack.org/#/c/213832/19:12
pradkgordc, looking at rebasing my filter patch to use Mehdi's logic now19:12
pradkone issue i see with preloading is ..19:12
pradkhe's using partial to pre parse the values19:13
pradkwhich wont work for say say scale or expression for example19:13
pradkso we might still have to load it after preloading in these cases19:14
gordcpradk: i don't quite understand19:17
gordcpradk: can you take a look at the last comment on 21383219:18
pradkgordc, so if you look at the way the parse of values is done self._field_getter[name] = functools.partial(19:19
pradk                    self._parse_jsonpath_field, parts)19:19
pradkand it gets preloaded, so we're not doing the match with message here19:20
pradkin case of scale or expression, we will need to do that when the message is there19:21
pradkas we cant really do a functools.partial object * 100 here19:21
pradkgordc, sure will look19:22
pradkat your comment i mean19:22
pradkhmm thought i removed it in setup.. perhaps it fell into another commit.. will remove here19:22
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Move profiler meters to yaml  https://review.openstack.org/21383219:24
pradkgordc, the way partial works is its returning a partial function here19:26
pradkgordc, and when called later, the rest of the arguments get appended as keyward arguments19:27
*** Redcavalier has joined #openstack-ceilometer19:27
pradkso that works fine for a single value, but for a combination string etc, we cant really preload until the match is done to perform the special operation19:27
pradkgordc, see what i mean?19:30
pradkbrb19:31
RedcavalierHi, I understand this is a channel for team discussion, but I've got a very specific question regarding ceilometer. Basically, we want to use ceilometer's metrics for our cloud billing system. However, we want to be able to clear these metrics after a while (like 3 months). Is that even possible to do in ceilometer? All info on the web explain how to drop the database to erase ceilometer data, which is extremely messy.19:34
gordcRedcavalier: you need to set a time to live (ttl) value.19:38
gordcwhich db are you using?19:38
gordchttp://docs.openstack.org/kilo/config-reference/content/section_ceilometer.conf.html19:39
gordcthere is a metering_time_to_live and an event_time_to_live19:39
gordcif you are running sql backend, you'll need to run ceilometer-expirer via a cron job periodically19:40
gordcif mongodb, it'll do that itself.19:40
gordcpradk: looking19:41
Redcavaliermongodb19:41
Redcavalierthank you for the pointers that will help19:41
gordcno problem. you'll need to run ceilometer-dbsyc to upgrade your db if it already exists.19:41
RedcavalierAlright, I'll keep that in mind. We'll flush the db first and then set the time to live and then db-sync.19:43
gordcRedcavalier: cool cool.19:43
gordcbest of luck19:43
Redcavalierthx19:44
openstackgerritMerged openstack/ceilometer: Avoid recording whole instance info in log  https://review.openstack.org/21493119:45
*** alejandrito has quit IRC19:48
*** nadya has joined #openstack-ceilometer19:49
*** ankita_w_ has joined #openstack-ceilometer19:56
*** admin0 has quit IRC19:56
*** ankita_w_ has quit IRC19:56
*** ankita_w_ has joined #openstack-ceilometer19:57
openstackgerritMerged openstack/ceilometer: Updated from global requirements  https://review.openstack.org/21387419:57
*** ankita_w_ has quit IRC19:57
*** ankita_w_ has joined #openstack-ceilometer19:58
*** boris-42 has joined #openstack-ceilometer19:58
*** ankita_w_ has quit IRC19:58
*** ankita_w_ has joined #openstack-ceilometer19:59
*** ankita_wagh has quit IRC19:59
*** ankita_w_ has quit IRC19:59
*** ankita_wagh has joined #openstack-ceilometer19:59
*** fawadkhaliq has joined #openstack-ceilometer20:03
gordcpradk: is the suggestino to not do it in init?20:06
*** idegtiarov_ has quit IRC20:10
*** jwcroppe has quit IRC20:10
*** jwcroppe has joined #openstack-ceilometer20:13
pradkgordc, i'm just saying its not easily extendable for complex cases like caching operations like scaling or expressions etc20:19
gordcpradk: if we move it out of the init, do we lose all the optimisation?20:21
pradkyea probably, i'm not suggesting that20:25
pradkthe problem is not really where its done, its more what we're caching20:26
gordci see. yeah, we need it as fast as possible. so if it's more code, i guess it is what it is...20:26
*** Redcavalier has quit IRC20:41
*** exploreshaifali has quit IRC20:47
*** ankita_wagh has quit IRC20:48
*** nadya has quit IRC20:49
*** ankita_wagh has joined #openstack-ceilometer20:49
*** ankita_w_ has joined #openstack-ceilometer20:50
*** ankita_wagh has quit IRC20:50
*** fawadkhaliq has quit IRC20:58
pradkgordc, interesting the new parser doesnt like %20:58
pradkin cpu units the unit: '%'20:58
pradkJsonPathLexerError: Error on line 1, col 0: Unexpected character: %20:59
gordcpradk: yeah, i can see it. it expects all strings to be paths21:00
gordcmaybe we should prefix paths with $21:02
gordcwhat i was mentioning befoer.21:03
gordcto avoid multi/lookup attr21:03
pradkgordc, parser would automatically check for that? or we do something like value.startswith21:04
gordcwe do something21:04
gordcsimilarly for [*] cases21:05
pradkyea probably save us from this head ache .. i can look into that once i'm done with this21:08
gordcwhich headache?21:08
gordcit should probably go in before Mehdi's patch.21:09
pradkgetting confused between a static string vs path21:09
gordcpradk: word. yeah. my multi thing was super hacky21:09
pradkagree21:09
gordcluckily it's not that big a change i think.21:10
gordci think '$.' is a good prefix. maybe keep it close to yaql in case we want to adopt21:11
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Add filter support to meter definitions in yaml  https://review.openstack.org/21061921:11
pradkk i have the operations working with mehdi's patch ^^ .. it would cleaner if we had this support in the parser itself21:12
pradkbut this should do for now i think21:12
pradksince we already have our own parser, it might be worth looking into supporting syntax like 'payload.volume * 100' in the parser itself.. mehdi said he might be able to look into that next week21:15
pradkthat would clean up all the attribute syntax from the yaml21:15
pradkgordc, i'll look into the $. chnage on monday21:16
gordcsurfe21:17
*** ddieterly has joined #openstack-ceilometer21:19
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Add cpu meters to yaml  https://review.openstack.org/21536721:20
pradkk hopefully these will pass21:20
openstackgerritgordon chung proposed openstack/ceilometer: make telemetry sample payloads dictionaries  https://review.openstack.org/21385121:21
pradkk i'm outta here21:26
pradkgordc, have a good weekend21:26
gordclaters21:27
openstackgerritPradeep Kilambi proposed openstack/ceilometer: Add support to handle operations in meter definitions  https://review.openstack.org/21061921:31
*** annasort has quit IRC21:33
*** edmondsw has quit IRC21:34
*** r-daneel has quit IRC21:42
*** thorst has quit IRC21:42
*** ankita_wagh has joined #openstack-ceilometer21:51
*** ankita_w_ has quit IRC21:51
*** jruano has joined #openstack-ceilometer21:56
*** gordc has quit IRC21:58
*** ddieterly has quit IRC22:04
*** ddieterly has joined #openstack-ceilometer22:07
*** _cjones_ has quit IRC22:30
*** ankita_w_ has joined #openstack-ceilometer22:31
*** ankita_wagh has quit IRC22:34
*** ddieterly has quit IRC22:43
*** alejandrito has joined #openstack-ceilometer22:46
*** ddieterly has joined #openstack-ceilometer23:10
*** ddieterly has quit IRC23:15
openstackgerritMerged openstack/ceilometer: Implement consuming metrics from Magnum  https://review.openstack.org/21356223:21
*** ankita_w_ has quit IRC23:27
*** promulo has joined #openstack-ceilometer23:31
*** promulo__ has quit IRC23:32
*** ankita_wagh has joined #openstack-ceilometer23:34
*** jruano has quit IRC23:37
*** jruano has joined #openstack-ceilometer23:55
« Thursday, 2015-08-20 Index Saturday, 2015-08-22 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!