Tuesday, 2017-09-19

*** chlong has joined #openstack-telemetry00:20
*** edmondsw has quit IRC00:22
*** thorst has joined #openstack-telemetry00:26
*** tovin07_ has joined #openstack-telemetry00:41
*** edmondsw has joined #openstack-telemetry00:46
*** edmondsw has quit IRC00:50
*** lhx__ has quit IRC01:08
openstackgerritzhangxuanyuan proposed openstack/ceilometer master: Deprecate OPT nova_control_exchange  https://review.openstack.org/50498201:13
*** thorst has quit IRC01:27
*** donghao has joined #openstack-telemetry01:29
*** lhx__ has joined #openstack-telemetry01:36
*** lhx__ has quit IRC01:55
*** lhx__ has joined #openstack-telemetry01:55
*** chlong has quit IRC02:03
*** cumseeker has joined #openstack-telemetry02:27
*** cumseeker has quit IRC02:31
*** dave-mccowan has quit IRC02:35
-openstackstatus- NOTICE: Gerrit is being restarted to feed its insatiable memory appetite02:41
*** Tom has joined #openstack-telemetry03:11
*** lhx__ has quit IRC03:23
*** Tommy-Tom has joined #openstack-telemetry03:23
*** lhx__ has joined #openstack-telemetry03:23
*** thorst has joined #openstack-telemetry03:28
*** Tommy-Tom has quit IRC03:31
*** Tommy-Tom has joined #openstack-telemetry03:31
*** thorst has quit IRC03:33
*** Tommy-Tom has quit IRC03:40
*** Tommy-Tom has joined #openstack-telemetry03:41
*** Tommy-Tom has quit IRC03:45
*** Tommy-Tom has joined #openstack-telemetry04:02
*** Tommy-To_ has joined #openstack-telemetry04:15
*** Tommy-Tom has quit IRC04:16
*** Tommy-To_ has quit IRC04:27
*** Tommy-Tom has joined #openstack-telemetry04:27
*** thorst has joined #openstack-telemetry04:29
*** thorst has quit IRC04:34
*** Tom has quit IRC04:37
*** Tommy-Tom has quit IRC04:37
*** Tommy-Tom has joined #openstack-telemetry04:38
*** Tommy-Tom has quit IRC04:42
*** Tom has joined #openstack-telemetry04:56
*** Tom has quit IRC05:00
*** zhurong has joined #openstack-telemetry05:24
*** thorst has joined #openstack-telemetry05:30
*** thorst has quit IRC05:34
*** Tom has joined #openstack-telemetry05:46
*** Tommy-Tom has joined #openstack-telemetry05:50
*** Tommy-To_ has joined #openstack-telemetry05:51
*** Tommy-T__ has joined #openstack-telemetry05:53
*** Tom has quit IRC05:53
*** Tommy-___ has joined #openstack-telemetry05:54
*** Tom has joined #openstack-telemetry05:54
*** Tommy-Tom has quit IRC05:55
*** Tom has quit IRC05:55
*** Tom has joined #openstack-telemetry05:55
*** Tommy-Tom has joined #openstack-telemetry05:56
*** Tommy-To_ has quit IRC05:56
*** Tommy-To_ has joined #openstack-telemetry05:57
*** Tommy-T__ has quit IRC05:58
*** Tommy-___ has quit IRC05:59
*** Tommy-Tom has quit IRC06:00
*** Tom has quit IRC06:05
*** afazekas has quit IRC06:05
*** harlowja has quit IRC06:05
*** dr_gogeta86 has quit IRC06:05
*** afazekas has joined #openstack-telemetry06:05
*** dr_gogeta86 has joined #openstack-telemetry06:05
*** dr_gogeta86 has quit IRC06:05
*** dr_gogeta86 has joined #openstack-telemetry06:05
*** Tom has joined #openstack-telemetry06:10
*** Tom has quit IRC06:25
*** Tommy-To_ has quit IRC06:26
*** Tom has joined #openstack-telemetry06:27
*** thorst has joined #openstack-telemetry06:31
*** Tom has quit IRC06:33
*** Tommy-Tom has joined #openstack-telemetry06:35
*** thorst has quit IRC06:35
*** Tom has joined #openstack-telemetry06:36
*** Tom has quit IRC06:43
*** Tom has joined #openstack-telemetry06:45
*** zhurong has quit IRC06:54
*** Tommy-Tom has quit IRC06:57
*** Tom has quit IRC07:04
*** Tommy-Tom has joined #openstack-telemetry07:05
*** Tommy-Tom has quit IRC07:09
openstackgerritChenghui Yu proposed openstack/ceilometer master: Update os-api-ref>=1.4.0  https://review.openstack.org/50507307:10
*** Tommy-Tom has joined #openstack-telemetry07:10
*** Tom has joined #openstack-telemetry07:11
*** Tommy-To_ has joined #openstack-telemetry07:11
*** yprokule has joined #openstack-telemetry07:14
*** Tommy-Tom has quit IRC07:15
*** iranzo has joined #openstack-telemetry07:15
*** rcernin has joined #openstack-telemetry07:18
*** zhurong has joined #openstack-telemetry07:19
*** pcaruana has joined #openstack-telemetry07:22
-openstackstatus- NOTICE: Post jobs are not executed currently, do not tag any releases07:24
*** ChanServ changes topic to "Post jobs are not executed currently, do not tag any releases"07:24
*** tesseract has joined #openstack-telemetry07:24
*** thorst has joined #openstack-telemetry07:32
*** thorst has quit IRC07:36
*** Tommy-To_ has quit IRC07:39
*** Tommy-Tom has joined #openstack-telemetry07:40
*** alexchadin has joined #openstack-telemetry07:44
*** Tommy-Tom has quit IRC07:44
*** Tommy-Tom has joined #openstack-telemetry07:46
*** xiaofandh12 has joined #openstack-telemetry07:54
*** donghao has quit IRC07:56
openstackgerritHanxi Liu proposed openstack/ceilometer master: Clean up diect publihser  https://review.openstack.org/50510408:04
*** shardy has joined #openstack-telemetry08:04
*** Tom__ has joined #openstack-telemetry08:07
*** Tom has quit IRC08:10
*** Tommy-Tom has quit IRC08:12
*** Tommy-Tom has joined #openstack-telemetry08:13
*** Tommy-Tom has quit IRC08:17
*** efoley has joined #openstack-telemetry08:20
*** Tommy-Tom has joined #openstack-telemetry08:21
*** zhurong has quit IRC08:24
*** thorst has joined #openstack-telemetry08:33
*** thorst has quit IRC08:37
*** Tommy-Tom has quit IRC08:41
*** Tommy-Tom has joined #openstack-telemetry08:41
*** iranzo has quit IRC08:57
*** Tommy-Tom has quit IRC09:00
*** Tommy-Tom has joined #openstack-telemetry09:01
*** alexchadin has quit IRC09:04
*** alexchadin has joined #openstack-telemetry09:05
*** zhurong has joined #openstack-telemetry09:05
*** Tommy-Tom has quit IRC09:05
*** Tommy-Tom has joined #openstack-telemetry09:15
*** thorst has joined #openstack-telemetry09:33
*** thorst has quit IRC09:38
*** Tommy-Tom has quit IRC09:38
*** Tommy-Tom has joined #openstack-telemetry09:39
*** Tommy-Tom has quit IRC09:39
*** Tommy-Tom has joined #openstack-telemetry09:39
*** Tommy-Tom has quit IRC09:42
*** Tommy-Tom has joined #openstack-telemetry09:42
*** Tommy-Tom has quit IRC09:47
*** Tommy-Tom has joined #openstack-telemetry09:49
*** Tommy-Tom has quit IRC09:53
*** Tommy-Tom has joined #openstack-telemetry09:53
*** Tommy-Tom has quit IRC09:57
*** Tommy-Tom has joined #openstack-telemetry09:58
*** Tommy-Tom has quit IRC10:09
*** Tommy-Tom has joined #openstack-telemetry10:09
*** tovin07_ has quit IRC10:09
*** Tommy-Tom has quit IRC10:10
*** Tommy-Tom has joined #openstack-telemetry10:11
*** alexchadin has quit IRC10:13
*** Tommy-Tom has quit IRC10:18
*** Tommy-Tom has joined #openstack-telemetry10:18
*** thorst has joined #openstack-telemetry10:18
*** thorst has quit IRC10:19
*** Tommy-Tom has quit IRC10:30
*** alexchadin has joined #openstack-telemetry10:31
openstackgerritChenghui Yu proposed openstack/ceilometer master: Update os-api-ref>=1.4.0 and reno>=2.5.0  https://review.openstack.org/50507310:33
*** alexchadin has quit IRC10:36
*** Tom__ has quit IRC10:40
openstackgerritChenghui Yu proposed openstack/ceilometermiddleware master: Update reno>=2.5.0  https://review.openstack.org/50516310:44
*** alexchadin has joined #openstack-telemetry10:55
openstackgerritOpenStack Proposal Bot proposed openstack/aodh master: Imported Translations from Zanata  https://review.openstack.org/50359511:16
*** thorst has joined #openstack-telemetry11:19
*** thorst has quit IRC11:26
*** alexchadin has quit IRC11:33
*** alexchadin has joined #openstack-telemetry11:34
*** alexchadin has quit IRC11:35
*** alexchadin has joined #openstack-telemetry11:36
*** Tom__ has joined #openstack-telemetry11:41
*** gordc has joined #openstack-telemetry11:43
*** Tom__ has quit IRC11:45
*** zhurong has quit IRC11:49
*** Drankis has joined #openstack-telemetry11:50
*** alexchadin has quit IRC11:56
*** alexchadin has joined #openstack-telemetry11:57
*** efoley has quit IRC12:03
*** efoley_ has joined #openstack-telemetry12:03
*** thorst has joined #openstack-telemetry12:04
*** edmondsw has joined #openstack-telemetry12:09
*** alexchadin has quit IRC12:13
*** alexchadin has joined #openstack-telemetry12:13
*** Tom__ has joined #openstack-telemetry12:13
*** efoley has joined #openstack-telemetry12:32
*** efoley_ has quit IRC12:32
*** dave-mccowan has joined #openstack-telemetry12:46
*** leitan has joined #openstack-telemetry12:48
*** catintheroof has joined #openstack-telemetry12:49
*** dave-mcc_ has joined #openstack-telemetry12:51
*** dave-mccowan has quit IRC12:51
*** swamireddy has quit IRC12:52
openstackgerritHanxi Liu proposed openstack/ceilometer master: Make sure partition_coordinator is valid to get hashring  https://review.openstack.org/50521312:55
openstackgerritHanxi Liu proposed openstack/ceilometer master: partition_coordinator value should be valid to get hashring  https://review.openstack.org/50521312:57
*** swamireddy has joined #openstack-telemetry13:08
*** lhx__ has quit IRC13:11
*** chlong has joined #openstack-telemetry13:19
*** alexchadin has quit IRC13:28
*** Tommy-Tom has joined #openstack-telemetry13:28
*** gordc has quit IRC13:36
*** jobewan has joined #openstack-telemetry13:45
*** Drankis has quit IRC13:51
*** kswiatek has joined #openstack-telemetry13:54
kswiatekhi, is here anyone who could help me with gnocchi on devstack?13:55
jd_kswiatek: just ask13:58
*** Tommy-Tom has quit IRC13:58
*** Tommy-Tom has joined #openstack-telemetry13:58
kswiatekI run devstack with ceilometer, aodh, gnocchi, heat using simple local.conf13:59
kswiatekeverything working except gnocchi, which return 'You don't have permission to access /v1/metric on this server'13:59
kswiatekafter I run 'ceilometer-upgrade --skip-metering-database'14:00
kswiatek2-3 weeks ago with the same local.conf gnocchi was fine14:00
*** efoley has quit IRC14:01
*** efoley has joined #openstack-telemetry14:01
kswiatekmy local.conf:   http://paste.openstack.org/show/621431/14:02
kswiatekthis is exact error I got after ceilometer-upgrade: http://paste.openstack.org/show/621432/14:05
kswiatekI tried find solution but https://ask.openstack.org is down14:06
kswiatekalso I tried this doc: but without success: https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.html14:08
*** Tommy-Tom has quit IRC14:10
*** Tommy-Tom has joined #openstack-telemetry14:10
*** Tommy-Tom has quit IRC14:15
jd_kswiatek: is OS_AUTH_TYPE set?14:24
jd_ah the error is at upgrade14:25
jd_kswiatek: check that Gnocchi API auth mode is set to keystone and correctly configured14:25
jd_kswiatek: ah I know actually waiiiii14:26
jd_t14:26
jd_kswiatek: https://github.com/gnocchixyz/gnocchi/issues/31714:26
jd_kswiatek: that might be it because your 403 comes from Apache, not Gnocchi14:26
*** efoley has quit IRC14:26
kswiatekjd_: i think it's working :D thanks14:32
*** Tommy-Tom has joined #openstack-telemetry14:35
*** efoley has joined #openstack-telemetry14:36
jd_kswiatek: feel free to chime in in the GitHub issue14:36
jd_kswiatek: I don't think it's clear why it's not working by default14:37
*** gordc has joined #openstack-telemetry14:38
*** Tommy-Tom has quit IRC14:48
*** Tommy-Tom has joined #openstack-telemetry14:48
*** Tommy-Tom has quit IRC14:52
*** Tom__ has quit IRC15:27
*** ddyer has quit IRC15:32
*** ddyer has joined #openstack-telemetry15:37
*** Tommy-Tom has joined #openstack-telemetry15:48
*** Tommy-Tom has quit IRC15:53
*** Tommy-Tom has joined #openstack-telemetry15:54
*** sbezverk has quit IRC16:03
*** yprokule has quit IRC16:03
*** Tommy-Tom has quit IRC16:06
*** Tommy-Tom has joined #openstack-telemetry16:07
*** Tommy-Tom has quit IRC16:11
*** xiaofandh12 has quit IRC16:11
*** donghao has joined #openstack-telemetry16:18
*** donghao has quit IRC16:22
*** rwsu has quit IRC16:28
*** rwsu has joined #openstack-telemetry16:30
*** rwsu has quit IRC16:30
*** rwsu has joined #openstack-telemetry16:30
*** ddyer has quit IRC16:33
*** pcaruana has quit IRC16:38
*** ddyer has joined #openstack-telemetry16:40
*** ddyer has quit IRC16:50
*** ddyer has joined #openstack-telemetry16:51
*** rcernin has quit IRC16:52
*** tesseract has quit IRC16:52
*** ddyer has quit IRC16:56
*** ddyer has joined #openstack-telemetry17:00
*** ddyer has quit IRC17:00
*** ddyer has joined #openstack-telemetry17:13
*** efoley has quit IRC17:28
*** rbak has joined #openstack-telemetry17:40
*** shardy has quit IRC17:44
*** rbak has quit IRC17:46
*** harlowja has joined #openstack-telemetry17:49
*** rbak has joined #openstack-telemetry18:02
*** iranzo has joined #openstack-telemetry18:23
*** iranzo has quit IRC18:35
*** rbak has quit IRC19:01
*** itlinux has joined #openstack-telemetry19:02
*** spilla has joined #openstack-telemetry19:19
*** rbak has joined #openstack-telemetry19:32
*** NostawRm has quit IRC19:33
itlinuxhello all.. any suggestions on this http://paste.openstack.org/show/621472/19:36
itlinuxthanks!19:36
*** rbak has quit IRC19:48
*** aagate has quit IRC19:59
*** spilla has quit IRC20:01
vkmcitlinux, is it a meter you defined?20:07
itlinuxno20:07
*** rcernin has joined #openstack-telemetry20:14
-openstackstatus- NOTICE: Zuul and Gerrit are being restarted to address issues discovered with the Gerrit 2.13 upgrade. review.openstack.org will be inaccessible for a few minutes while we make these changes. Currently running jobs will be restarted for you once Zuul and Gerrit are running again.20:27
gordcitlinux: is that flooding your logs?20:37
gordcthat actually should appear once every time you create a new instance20:37
itlinuxif that's ok then great..20:38
itlinuxyes everytime I create an instance.. but if I use cinder volume that does not do it.. since it's not ephemeral20:38
gordcwe build metrics off messages from nova and nova sends a message when it first starts building instance  -- before it's placed on a host -- so it doesn't give us a host originally20:38
itlinuxok thanks.. just wanted to make sure it's all good!20:39
gordcoh. hmm.. didn't realise it only affected ephemeral instances.20:39
gordcgood to know20:39
itlinuxwell I do not see it when I do create it from volume.. so that's my guess ..20:39
gordcstrange. i'll just open a bug to see if we can somehow make that issue less jarring20:40
itlinuxok..20:41
itlinuxmore than an error I would call it something diff!20:41
itlinuxinitial state or something..20:41
gordcyeah... not sure what best solution is.20:43
itlinuxok thanks gordc:20:43
gordcnp20:43
*** leitan has quit IRC20:47
*** leitan has joined #openstack-telemetry20:48
*** leitan has quit IRC20:48
*** leitan has joined #openstack-telemetry20:48
*** leitan has quit IRC20:49
*** leitan has joined #openstack-telemetry20:49
*** catintheroof has quit IRC20:50
*** catintheroof has joined #openstack-telemetry20:50
*** catintheroof has quit IRC20:51
*** leitan has quit IRC20:53
*** itlinux has quit IRC20:56
*** itlinux has joined #openstack-telemetry20:56
*** itlinux has quit IRC20:58
*** itlinux has joined #openstack-telemetry20:59
itlinuxgordc: any tips on this requests.packages.urllib3.connectionpool [-] Connection pool is full, discarding connection:21:05
itlinuxI see this on one of the controllers21:06
gordcitlinux: in notification-agent logs?21:09
itlinuxno this is in collector.log21:09
itlinuxagent looks fine21:10
gordchmm... i imagine it has something to do with default 64threads collector enables. can't remember what default connectionpool size requests library uses21:11
gordcif you're using anything newer than newton, you could technically just disable collector service.21:11
itlinuxhere is the error, I am able to telnet to ip 8041 now not sure..21:12
itlinuxhttp://paste.openstack.org/show/621481/21:12
itlinuxyes this is ocata21:12
itlinuxhow do I disable that just thanks gordc:21:12
gordcyou'll need to change your pipeline.yaml file so all the publishers are 'gnocchi://' rather than 'notifier://'21:13
itlinuxyou mean this systemctl stop openstack-ceilometer-collector.service21:13
*** thorst has quit IRC21:14
gordcrestart your notification-agent serivce after.... and then yeah, you can stop openstack-ceilometer-collector.service21:14
itlinuxis this in /etc/ceilomter or /etc/gnocchi21:14
gordcthis won't necessarily solve your issue though (depends if you have workload_partitioning enabled.21:14
itlinuxahh ok..21:14
gordcyou can control pool size in pike: https://github.com/openstack/ceilometer/blob/master/ceilometer/gnocchi_client.py#L3121:14
gordcdoesn't seem like it's available in ocata (unless we backport it)21:15
itlinuxahh!21:15
itlinuxwill it be?21:15
gordc*shrugs* if someone proposes it ;)21:16
*** thorst has joined #openstack-telemetry21:16
itlinuxI see the notifier in the pipeline.yaml..21:16
itlinuxI guess I need to do that in the 3 controllers LOL!21:17
itlinuxand restart and then stop that services in each.21:17
gordcyeah... :( i guess keep your collector on until you've restarted all of them so you don't lose (as much) stuff21:17
gordcsomethign similar to this: https://github.com/openstack/ceilometer/blob/master/ceilometer/pipeline/data/pipeline.yaml#L3821:18
gordcyou'll need to do it eventually. because the collector will be gone soon https://review.openstack.org/#/c/504244/21:18
itlinuxcould I use this then..21:19
*** thorst has quit IRC21:20
gordcwhat's "this"?21:21
*** itlinux has quit IRC21:21
*** itlinux has joined #openstack-telemetry21:25
itlinuxahh I mean the git link you sent ..21:26
itlinuxfor the changes or I could just do :%s/notifiyer/gnocchi :)21:26
gordcyeah, you should probably just do the replace all... i think we changed the default meters ceilometer captures recently so it's probably different from what you have deployed.21:27
itlinuxok.. this is two weeks old!21:28
itlinuxLOL ocata latest build from ci...21:28
gordcoh... *shrugs* i guess you can take a look and see what the diff is. :)21:29
itlinuxyes will do a diff now21:29
itlinuxhere is the diff http://paste.openstack.org/show/621482/21:30
gordc:) looks like you're good regardless of what you choose to do then.21:31
itlinuxyes just pulling them down and then restart which services did you say?21:32
gordcdo you have a polling.yaml file? if not the pipeline.yaml is technically used by polling agents as well but it doesn't actually look at the publisher part of file.21:34
gordcso long story short, you just need to restart notification agents.21:34
itlinuxI have stop openstack-ceilometer-collector.service and restarted notification21:37
itlinuxand yes there is polling.yaml file21:37
gordccool. you should be good... (also, i take zero responsibility of things break)21:38
itlinux:)21:38
itlinuxof course!21:39
itlinuxtesting now..21:39
gordcif you check rabbit queue, and see message on metering.sample queue (or whatever queue your collector was listening to) you shoudl probably restart collector temporarily.21:39
itlinuxlooks like its' working21:39
gordccool!21:40
itlinuxstil shows the fancy error!21:40
gordcthats a shame.lol21:40
gordcdo you have workload_partitioning enabled? i imagine you do since you have notification agent on multiple controllers21:41
itlinuxnot show how do I check this?21:42
gordcit'll be in your ceilometer.conf file.21:42
itlinuxlet me check..21:42
itlinuxI do not have have workload_partitioning21:43
itlinuxsorry # Its value may be silently ignored in the future.21:43
itlinux#workload_partitioning = false21:43
itlinuxdo I need to have that enabled?21:43
gordcyou need it if you have transformations in your pipeline.21:44
itlinuxok so #workload_partitioning = true?21:44
gordcand you have multiple notification agent/workers21:44
itlinuxwhat is the option21:44
gordcyou'll need to drop the comment '#' as well.21:44
gordcyou'll probably want to configure pipeline_processing_queues as well21:45
itlinuxok..21:46
gordcit defaults to 10. you probably don't need. higher the number, the better distribution but your memory usage goes up with it.21:46
gordcif you were using master, i'd tell you to get rid of transformations and let gnocchi calculate it for you so you can avoid workload_partitioning but *shrugs* i guess you'll have to deal with it for now.21:48
*** gordc has quit IRC22:08
*** chlong has quit IRC22:21
itlinuxok..22:29
itlinuxcan you check htis out..22:29
itlinuxhttp://paste.openstack.org/show/621483/22:30
*** dave-mcc_ has quit IRC22:33
-openstackstatus- NOTICE: Gerrit is being restarted to address some final issues, review.openstack.org will be inaccessible for a few minutes while we restart22:34
itlinuxgordc: are you there?22:38
*** vint_bra has quit IRC22:45
*** catintheroof has joined #openstack-telemetry22:45
*** openstackgerrit has quit IRC22:47
*** itlinux has quit IRC22:49
*** edmondsw has quit IRC22:50
*** edmondsw has joined #openstack-telemetry22:55
*** edmondsw has quit IRC22:59
*** rcernin has quit IRC23:11
*** edmondsw has joined #openstack-telemetry23:23
*** hoonetorg has quit IRC23:23
*** edmondsw has quit IRC23:27
*** jobewan has quit IRC23:36
*** hoonetorg has joined #openstack-telemetry23:36
*** ChanServ changes topic to "#openstack-telemetry is OpenStack Telemetry | http://wiki.openstack.org/Telemetry"23:41
-openstackstatus- NOTICE: Gerrit is once again part of normal puppet config management. Problems with Gerrit gitweb links and Zuul post jobs have been addressed. We currently cannot create new gerrit projects (fixes in progress) and email sending is slow (being debugged).23:41

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!