Monday, 2017-01-30

*** thorst_ has joined #openstack-telemetry00:00
*** thorst_ has quit IRC00:04
*** thorst_ has joined #openstack-telemetry00:13
*** thorst_ has quit IRC00:13
*** thorst_ has joined #openstack-telemetry01:44
*** thorst_ has quit IRC01:49
*** sferna1_ has quit IRC02:03
*** thorst_ has joined #openstack-telemetry02:09
*** thorst_ has quit IRC02:12
*** thorst_ has joined #openstack-telemetry02:57
*** thorst_ has quit IRC02:57
*** thorst_ has joined #openstack-telemetry03:07
*** thorst_ has quit IRC03:07
*** rcernin has joined #openstack-telemetry03:11
*** rcernin has quit IRC03:13
*** rcernin has joined #openstack-telemetry03:14
*** rcernin has quit IRC03:25
*** hfu has joined #openstack-telemetry03:31
*** links has joined #openstack-telemetry03:56
*** links has quit IRC03:57
*** sanfern has joined #openstack-telemetry04:18
*** sanfern has quit IRC04:49
*** sanfern has joined #openstack-telemetry04:50
*** sudipto has joined #openstack-telemetry04:59
*** sudipto_ has joined #openstack-telemetry04:59
*** r-mibu has quit IRC05:09
*** r-mibu has joined #openstack-telemetry05:14
*** sanfern has quit IRC05:26
*** sanfern has joined #openstack-telemetry05:27
*** sferna1_ has joined #openstack-telemetry05:32
*** sanfern has quit IRC05:34
*** swamireddy has joined #openstack-telemetry05:43
*** thorst_ has joined #openstack-telemetry05:49
*** nadya has joined #openstack-telemetry05:50
*** adriant has quit IRC05:54
*** nadya has quit IRC05:54
*** thorst_ has quit IRC05:55
*** Jack_Iv has joined #openstack-telemetry05:55
*** nadya has joined #openstack-telemetry05:58
*** swamireddy has left #openstack-telemetry06:00
*** jvarlamova has joined #openstack-telemetry06:01
*** swamireddy_ has joined #openstack-telemetry06:01
*** Jack_Iv has quit IRC06:30
*** rcernin has joined #openstack-telemetry07:11
*** rcernin has quit IRC07:12
*** rcernin has joined #openstack-telemetry07:12
*** nadya has quit IRC07:47
*** tesseract has joined #openstack-telemetry07:50
*** thorst_ has joined #openstack-telemetry07:51
*** nadya has joined #openstack-telemetry07:56
*** thorst_ has quit IRC07:56
*** sferna1__ has joined #openstack-telemetry08:06
*** sferna1_ has quit IRC08:08
*** pcaruana has joined #openstack-telemetry08:15
*** shardy has joined #openstack-telemetry08:18
*** yprokule has joined #openstack-telemetry08:28
*** sudipto_ has quit IRC09:10
*** sudipto has quit IRC09:11
*** sudipto_ has joined #openstack-telemetry09:11
*** sudswas__ has joined #openstack-telemetry09:13
*** yassine has joined #openstack-telemetry09:27
*** yassine is now known as Guest4885209:28
*** chopmann has joined #openstack-telemetry09:28
*** rcernin has quit IRC09:31
*** rcernin has joined #openstack-telemetry09:33
*** efoley__ has joined #openstack-telemetry09:33
*** thorst_ has joined #openstack-telemetry09:52
*** Jack_Iv has joined #openstack-telemetry09:55
*** thorst_ has quit IRC09:57
*** Jack_Iv has quit IRC09:57
*** Jack_Iv has joined #openstack-telemetry10:00
*** Jack_Iv has quit IRC10:00
*** Jack_Iv has joined #openstack-telemetry10:00
*** cdent has joined #openstack-telemetry10:35
*** Jack_Iv has quit IRC10:44
*** eglynn has joined #openstack-telemetry10:52
*** Jack_Iv has joined #openstack-telemetry11:05
*** Jack_Iv has quit IRC11:08
*** Jack_Iv has joined #openstack-telemetry11:09
*** Jack_Iv has quit IRC11:13
jd__sileht: gnocchi reviews appreciated so we can release11:15
jd__also there's a bunch of job for telemetry integration failing no idea why11:15
silehtjd__, looks like a compute test that fail ...11:22
jd__ok so for UUID reencoding it's normal11:22
jd__we need a gnocchiclient that stops encoding11:22
jd__i'll do a release today11:22
jd__the incoming conf i think is just a bug11:22
sileht+111:23
jd__sileht: can u check https://review.openstack.org/#/c/425759/ while you're at it11:23
jd__if gordc is around today I hope I can push a release for gnocchi itself by the end of the day :(11:23
*** Jack_Iv has joined #openstack-telemetry11:24
*** dims has joined #openstack-telemetry11:26
cdentjd__: if there's something I could make sense of that you want to push me at, go for it11:37
openstackgerritMerged openstack/gnocchi: noauth: force user to be provided  https://review.openstack.org/42469611:45
openstackgerritMerged openstack/gnocchi: tests: use GNU sed where needed  https://review.openstack.org/42557711:46
*** sferna1__ has quit IRC11:48
*** thorst_ has joined #openstack-telemetry11:53
*** vint_bra has joined #openstack-telemetry11:54
*** thorst_ has quit IRC11:57
openstackgerritMerged openstack/ceilometer: drop api and storage references from install-guide  https://review.openstack.org/42630412:05
openstackgerritMerged openstack/ceilometer: update verify to use gnocchi  https://review.openstack.org/42638912:05
jd__cdent: what do you mean? :)12:08
cdentjd__: if you need something reviewed point me at it, but only if it is something that I'll understand, given how little I pay attention these days12:09
jd__cdent: oh sure thanks :)12:09
jd__cdent: you can check https://review.openstack.org/#/c/425974 :)12:10
* cdent looks12:10
*** Jack_Iv has quit IRC12:14
cdentjd__: those functional test failures in live.yaml are real: http://logs.openstack.org/74/425974/1/check/gate-gnocchi-dsvm-functional-file-mysql-ubuntu-xenial/232d003/testr_results.html.gz12:14
cdentbut not clear why they wouldn't have failed before12:14
jd__ah12:14
jd__we had so many false negative due to devstack, I stopped checked errors for a week12:15
*** Guest48852 has quit IRC12:16
openstackgerritMerged openstack/gnocchi: tests: factorize measure generation to speed up test  https://review.openstack.org/42626812:19
openstackgerritMerged openstack/gnocchi: fix live gabbi test  https://review.openstack.org/42596212:19
*** catintheroof has joined #openstack-telemetry12:25
*** Guest48852 has joined #openstack-telemetry12:28
*** Jack_Iv has joined #openstack-telemetry12:29
*** Jack_Iv has quit IRC12:35
*** Jack_Iv has joined #openstack-telemetry12:35
*** Jack_Iv has quit IRC12:35
*** leitan has joined #openstack-telemetry12:35
*** Jack_Iv has joined #openstack-telemetry12:36
*** shardy is now known as shardy_lunch12:37
*** Jack_Iv has quit IRC12:38
*** Jack_Iv has joined #openstack-telemetry12:39
*** Jack_Iv_ has joined #openstack-telemetry12:40
*** thorst_ has joined #openstack-telemetry12:41
*** Jack_Iv has quit IRC12:43
*** Jack_Iv_ has quit IRC12:45
*** Jack_Iv has joined #openstack-telemetry12:46
*** Jack_Iv has quit IRC12:46
*** Jack_Iv has joined #openstack-telemetry12:48
*** Jack_Iv has quit IRC12:52
*** Jack_Iv has joined #openstack-telemetry12:54
*** dave-mccowan has joined #openstack-telemetry13:00
*** chopmann has quit IRC13:00
*** Jack_Iv has quit IRC13:05
*** Jack_Iv has joined #openstack-telemetry13:09
openstackgerritMerged openstack/ceilometer: gnocchi: don't rely on gnocchi uuid5  https://review.openstack.org/41910613:14
*** Jack_Iv has quit IRC13:14
*** shardy_lunch is now known as shardy13:18
*** Jack_Iv has joined #openstack-telemetry13:20
*** Jack_Iv has quit IRC13:22
*** cdent has quit IRC13:30
openstackgerritgordon chung proposed openstack/gnocchi: modernise gabbi tests  https://review.openstack.org/42597413:33
openstackgerritgordon chung proposed openstack/gnocchi: supporting disabling reporting process  https://review.openstack.org/42652813:38
*** nadya has quit IRC13:42
*** pradk has joined #openstack-telemetry13:47
*** narasimha_SV has joined #openstack-telemetry13:47
narasimha_SVhttp://paste.openstack.org/show/596883/13:48
narasimha_SVwhen any resource is getting created13:48
narasimha_SVin ceilometer-collector I am seeing this log13:48
silehtnarasimha_SV, looks into gnocchi api log (or the apache one)13:48
narasimha_SVhttp://paste.openstack.org/show/596885/13:51
narasimha_SVapache log13:51
*** tlian has joined #openstack-telemetry13:52
silehtarf, again13:52
silehtnarasimha_SV, can you give the exact mysql version ?13:52
silehtnarasimha_SV, mysqld --version13:53
narasimha_SV5.6.2913:54
silehtnarasimha_SV, you have only that in the backtrace, I don't see any Gnocchi reference13:54
sileht?13:54
silehtnarasimha_SV, ubuntu or Centos ? or something else13:57
narasimha_SVhttp://paste.openstack.org/show/596886/ this I can see ani gnocchi-api log13:57
sileht?13:57
narasimha_SVcentOS13:57
*** nadya has joined #openstack-telemetry13:57
silehtnarasimha_SV, can you provide mysql server log too ?13:58
narasimha_SVsileht: if I am not wrong entire code to update metric details to database is present in gnocchi/cli.py right ?13:58
silehtnarasimha_SV, gnocchi/cli.py is just the cli entrypoint14:00
silehtnarasimha_SV, can I have "mysql -e 'show errors;'" ?14:00
narasimha_SVno errors14:02
narasimha_SV show errors; Empty set (0.00 sec)14:02
silehtnarasimha_SV, it looks like you have a mysql issue when "constraint fail" it should report a column name but we got 'Unknown error code' instead14:04
*** cdent has joined #openstack-telemetry14:04
narasimha_SVok14:06
narasimha_SVcan you point me the metric updation code in gnocchi14:06
narasimha_SVwhere this query is getting executed14:07
silehtnarasimha_SV, https://github.com/openstack/gnocchi/blob/master/gnocchi/indexer/sqlalchemy.py#L656-L67314:08
silehtnarasimha_SV, with a mysql that works, we got an exception.DBReferenceError in your case with the fk name.14:09
narasimha_SVok14:11
silehtnarasimha_SV, maybe something in "mysql -e 'show warnings;'" ?14:12
narasimha_SVno warnings tooo14:14
sileht:(14:14
narasimha_SVwhen a metric is getting created for a newly added image/ volume or other14:15
narasimha_SVwhen will the resource will be processed ?14:15
*** nadya has quit IRC14:15
openstackgerritMerged openstack/python-gnocchiclient: Fix data display when listing resources  https://review.openstack.org/42575914:17
silehtnarasimha_SV, in your case, you never go to that part.14:27
silehtnarasimha_SV, when mysql return the correct error, we built we list of resource that doesn't exists: https://github.com/openstack/gnocchi/blob/master/gnocchi/rest/__init__.py#L1371-L140114:28
silehtnarasimha_SV, and return HTTP 400 to ceilometer with the list of non-existing resource ids14:28
silehtnarasimha_SV, then Ceilometer create all missing resources14:29
*** fguillot has joined #openstack-telemetry14:29
jd__narasimha_SV: u're not using Oracle version anymore?14:46
*** rcernin has quit IRC14:48
openstackgerritMerged openstack/python-gnocchiclient: add note when using OS_AUTH_TYPE  https://review.openstack.org/42537314:49
*** rbak_ has joined #openstack-telemetry14:54
akrzostrying to run gnocchi benchmark i'm getting "PicklingError: Can't pickle <type 'thread.lock'>: attribute lookup thread.lock failed"14:58
akrzoslooks like this https://bugs.launchpad.net/python-gnocchiclient/+bug/154895614:59
openstackLaunchpad bug 1548956 in python-gnocchiclient "PicklingError: Can't pickle <type 'thread.lock'>" [Undecided,Incomplete] - Assigned to zhangguoqing (474751729-o)14:59
*** rcernin has joined #openstack-telemetry15:00
*** ddyer has quit IRC15:10
*** shardy has quit IRC15:13
*** Jack_Iv has joined #openstack-telemetry15:18
*** ddyer has joined #openstack-telemetry15:21
*** gordc has joined #openstack-telemetry15:28
gordcakrzos: i don't have bug handy but i believe it's related to authentication15:40
akrzosyeah i can change to threadpool but thats obiviously less desirable than a process pool for benchmarking purposes and it works but let me see if i can skip auth to get benchmark working15:46
gordcakrzos: yeah, you can only run it without keystone. that's how i've been benchmarking as well... that or you hack to threadpool.15:49
akrzosgot it15:50
*** nicodemus_ has joined #openstack-telemetry15:51
akrzosi almost always use a tripleo deployed cloud so most of that configuration is already done for me so this will be interesting seeing how to remove the keystone auth to get this working15:51
akrzosfyi here was those first results with gnocchi with cephstorage15:53
akrzoshttp://docs-draft.openstack.org/61/421461/4/check/gate-performance-docs-docs-ubuntu-xenial/4a7e9e4//doc/build/html/test_results/telemetry_gnocchi_with_ceph/index.html#telemetry-gnocchi-with-ceph-report-1k-instances15:53
gordcakrzos: cool stuff. i like the pretty graphs16:00
akrzosgordc: Thanks16:03
akrzosnext step 10k instances16:03
akrzos:D16:03
gordcakrzos: lol let me know how that goes for you. specifically how many firetrucks to put out the fire.16:05
akrzoshaha will do16:05
openstackgerritMerged openstack/gnocchi: storage: Read incoming config correctly  https://review.openstack.org/42559016:14
*** shardy has joined #openstack-telemetry16:14
openstackgerritgordon chung proposed openstack/ceilometer: add ceilometer-upgrade step to install guide  https://review.openstack.org/42683116:18
*** leitan_ has joined #openstack-telemetry16:19
*** leitan has quit IRC16:20
*** rcernin has quit IRC16:24
*** chopmann has joined #openstack-telemetry16:33
openstackgerritJulien Danjou proposed openstack/python-gnocchiclient: client: allow to pass auth directly  https://review.openstack.org/42683916:36
*** cdent has quit IRC16:42
*** Jack_Iv has quit IRC16:49
openstackgerritgordon chung proposed openstack/gnocchi: paginate ceph report generation  https://review.openstack.org/42684616:55
*** tesseract has quit IRC16:58
*** chopmann has quit IRC17:09
*** nadya has joined #openstack-telemetry17:09
*** Guest48852 has quit IRC17:17
*** cdent has joined #openstack-telemetry17:17
openstackgerritgordon chung proposed openstack/ceilometer: make http publisher equivalent to dispatcher  https://review.openstack.org/39677117:24
openstackgerritgordon chung proposed openstack/ceilometer: make connection pool configurable  https://review.openstack.org/39796317:24
*** yprokule has quit IRC17:27
*** Guest48852 has joined #openstack-telemetry17:34
*** Jack_Iv has joined #openstack-telemetry17:40
*** nadya has quit IRC17:40
*** nadya has joined #openstack-telemetry17:47
akrzosgot no auth working for this environment too17:51
akrzosbut looks like you can only push emasures to a single metric17:52
akrzosmeasures*17:54
akrzosgordc / jd__ is that true only to a single metric with gnocchi benchmark?17:55
gordcyep. when i did benchmark i ran multiple gnocchi benchmark commands in parallel17:55
gordci don't think a single command will saturate gnocchi.17:56
akrzosbut if your limited to pushing measures to a single metric per command that is running to get multiple measures feeding to multiple metrics concurrently you'll need something to orchestrate running all of those commands and then grabbing their outputs etc17:57
akrzosmight not be as realistic as spawning bunch of instances or resources to generate measures for collection on17:58
akrzossince each instance/resource will have a number of metrics17:58
*** rbak_ has quit IRC17:58
gordcakrzos: i don't really understand "that is running to get multiple measures feeding to multiple metrics concurrently"18:00
*** leitan has joined #openstack-telemetry18:00
akrzoswell in a real cloud we have ceilometer feeding unprocessed measures into gnocchi-api then gnocchi metricd processing those into aggregated measures afterwards, i would wonder if the benchmark command would be better if you could tell it push measures into more than a single metric18:02
*** harlowja has joined #openstack-telemetry18:02
akrzosor if thats outside the scope of what gnocchi benchmark should be doing or it isn't helpful to you all18:03
*** leitan_ has quit IRC18:03
*** efoley_ has joined #openstack-telemetry18:04
*** Jack_Iv has quit IRC18:06
gordci've only used to benchmark API and nothing. i guess it could do more? not sure how accurate benchmarking the aggregation step from client is though... especially since the aggregation intervals are delayed based on whatever you set scheduling interval at18:07
gordcs/nothing/nothing else/18:08
*** efoley__ has quit IRC18:08
*** nadya has quit IRC18:09
akrzosyeah looking at it i figured i could load up the cloud18:10
akrzosthen use the gnocchi benchmark measures show to check responsiveness of api to return aggregated telemetry data18:11
akrzosand i could continue to use my collectd plugin to report if the backlog is growing or sustaining the load18:11
akrzosi realize that the show would return non-realtime data if the bakclog was growing conitnously (unless you sent --refresh)18:12
*** efoley_ has quit IRC18:12
akrzospoint would be to show under high load (a lot of instances sitting in the cloud generating telemetry load) the api to get aggregated metrics on your instances is still fast18:13
akrzos(Or maybe it isn't and we can identify an unaddressed bottleneck)18:14
gordcyeah, that seems like it could work to test that case.18:14
*** sudipto_ has quit IRC18:15
*** sudswas__ has quit IRC18:15
*** htruta` is now known as htruta18:23
akrzosi can probably wipe up a rally scenario that can do this and pick different metrics each time to grab18:24
*** nadya has joined #openstack-telemetry18:26
*** cdent has quit IRC18:30
*** Gleb has joined #openstack-telemetry18:31
*** Gleb has quit IRC18:52
*** mnaser has joined #openstack-telemetry19:11
*** Jack_Iv has joined #openstack-telemetry19:48
*** Jack_Iv has quit IRC19:54
*** Jack_Iv has joined #openstack-telemetry19:56
*** shardy has quit IRC20:01
*** leitan has quit IRC20:28
*** leitan has joined #openstack-telemetry20:29
openstackgerritgordon chung proposed openstack/gnocchi: modernise gabbi tests  https://review.openstack.org/42597420:31
*** leitan has quit IRC20:34
*** rbak_ has joined #openstack-telemetry20:34
*** nadya has quit IRC20:38
openstackgerritJulien Danjou proposed openstack/python-gnocchiclient: client: allow to pass auth directly  https://review.openstack.org/42683920:43
*** adriant has joined #openstack-telemetry20:43
*** narasimha_SV has quit IRC20:49
*** Jack_Iv has quit IRC21:04
*** Jack_Iv has joined #openstack-telemetry21:05
*** eglynn has quit IRC21:05
openstackgerritgordon chung proposed openstack/ceilometer: polling definition file  https://review.openstack.org/40568221:20
openstackgerritgordon chung proposed openstack/gnocchi: paginate ceph report generation  https://review.openstack.org/42684621:21
*** vint_bra has quit IRC21:23
*** catintheroof has quit IRC21:40
openstackgerritMerged openstack/ceilometer: make http publisher equivalent to dispatcher  https://review.openstack.org/39677121:41
openstackgerritMerged openstack/ceilometer: make connection pool configurable  https://review.openstack.org/39796321:41
*** catintheroof has joined #openstack-telemetry21:42
*** catinthe_ has joined #openstack-telemetry21:52
*** catintheroof has quit IRC21:53
openstackgerritMerged openstack/ceilometer: Don't load many times the same publisher  https://review.openstack.org/42065521:57
*** nicodemus_ has quit IRC22:03
*** catintheroof has joined #openstack-telemetry22:04
openstackgerritMerged openstack/ceilometer: use hyphen-less option name sql_expire_sample_only  https://review.openstack.org/40917722:05
*** catinthe_ has quit IRC22:07
*** eglynn has joined #openstack-telemetry22:09
openstackgerritgordon chung proposed openstack/ceilometer: update multi-publisher image  https://review.openstack.org/42695222:11
*** Guest48852 has quit IRC22:15
*** vint_bra has joined #openstack-telemetry22:17
*** catinthe_ has joined #openstack-telemetry22:17
*** eglynn has quit IRC22:18
*** catinth__ has joined #openstack-telemetry22:18
*** catintheroof has quit IRC22:18
*** catinthe_ has quit IRC22:22
*** Jack_Iv has quit IRC22:25
*** catinth__ has quit IRC22:29
*** thorst_ has quit IRC22:36
openstackgerritMerged openstack/gnocchi: mysql: retry on table changes  https://review.openstack.org/42593822:49
openstackgerritMerged openstack/gnocchi: tests: do not create a resource with a custom resource type in 2.2  https://review.openstack.org/42628922:49
openstackgerritMerged openstack/gnocchi: modernise gabbi tests  https://review.openstack.org/42597422:49
*** thorst_ has joined #openstack-telemetry23:06
*** gordc has quit IRC23:09
*** thorst_ has quit IRC23:10
*** lionel has quit IRC23:37
*** sileht has quit IRC23:37
*** lionel has joined #openstack-telemetry23:39
*** sileht has joined #openstack-telemetry23:41
*** catintheroof has joined #openstack-telemetry23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!