*** weechat4 has joined #openstack-monasca | 02:17 | |
*** Neptu_ has joined #openstack-monasca | 02:19 | |
*** Neptu has quit IRC | 02:19 | |
*** sc has quit IRC | 02:19 | |
*** jgu has joined #openstack-monasca | 02:21 | |
*** jgu has quit IRC | 04:20 | |
*** jgu has joined #openstack-monasca | 04:25 | |
openstackgerrit | Tuan Do Anh proposed openstack/monasca-agent master: Trivial fix typo of description https://review.openstack.org/581187 | 04:27 |
---|---|---|
*** jgu has quit IRC | 04:33 | |
*** jgu has joined #openstack-monasca | 04:38 | |
*** jgu__ has joined #openstack-monasca | 04:41 | |
*** jgu has quit IRC | 04:43 | |
*** jgu__ has quit IRC | 05:00 | |
*** amofakhar has joined #openstack-monasca | 07:29 | |
openstackgerrit | Merged openstack/monasca-agent master: Trivial fix typo of description https://review.openstack.org/581187 | 07:54 |
*** rodolof has joined #openstack-monasca | 08:11 | |
*** dougsz has joined #openstack-monasca | 08:13 | |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 08:42 |
*** toabctl has quit IRC | 08:48 | |
*** toabctl has joined #openstack-monasca | 08:54 | |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 09:30 |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 09:33 |
*** amofakhar has quit IRC | 10:05 | |
*** amofakhar has joined #openstack-monasca | 10:06 | |
*** amofakhar has quit IRC | 10:10 | |
*** amofakhar has joined #openstack-monasca | 10:18 | |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 10:24 |
openstackgerrit | Merged openstack/monasca-log-api master: Ignore changes in docs when running tempest tests https://review.openstack.org/580623 | 10:28 |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 10:50 |
openstackgerrit | Witold Bedyk proposed openstack/monasca-common master: Minor language changes and added license headers https://review.openstack.org/580654 | 10:56 |
*** amofakhar has quit IRC | 11:10 | |
dougsz | Is monasca-thresh considered legacy at this stage? | 11:11 |
dougsz | Or in other words, is there supposed to be a tarball here: https://tarballs.openstack.org/monasca-thresh/ ? | 11:12 |
*** amofakhar has joined #openstack-monasca | 11:25 | |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 11:28 |
*** amofakhar has quit IRC | 11:30 | |
openstackgerrit | Adrian Czarnecki proposed openstack/monasca-api master: [WIP] enable python3 in devstack https://review.openstack.org/576100 | 11:46 |
*** dpawlik_ has quit IRC | 11:50 | |
*** dpawlik has joined #openstack-monasca | 11:55 | |
*** amofakhar has joined #openstack-monasca | 12:00 | |
*** amofakhar has quit IRC | 12:11 | |
openstackgerrit | Merged openstack/monasca-log-api master: Add architecture diagram https://review.openstack.org/568834 | 12:38 |
*** amofakhar has joined #openstack-monasca | 12:50 | |
openstackgerrit | Witold Bedyk proposed openstack/monasca-common master: Minor language changes and added license headers https://review.openstack.org/580654 | 13:09 |
witek | dougsz: https://tarballs.openstack.org/ci/monasca-thresh/ | 13:14 |
dougsz | witek: Thanks - that's good to have for the binary Kolla image. I poked infra about that missing tar file and they think something's broken. | 13:18 |
guilhermesp | hello witek ! Is my "explanation" above comprehensive or is there another way to expose my problem? | 13:20 |
witek | guilhermesp: sorry, haven't had time to look at it | 13:20 |
guilhermesp | no problem, as you can :) I'm trying to debug to have more information | 13:21 |
*** amofakhar has quit IRC | 13:23 | |
*** ammo has joined #openstack-monasca | 13:23 | |
*** ammo is now known as Guest42906 | 13:23 | |
*** rodolof has quit IRC | 13:31 | |
*** rodolof has joined #openstack-monasca | 13:32 | |
witek | guilhermesp: is it the review you're referring to https://review.openstack.org/578721 ? | 13:37 |
guilhermesp | witek: yep | 13:38 |
guilhermesp | I've been doing some fixes and clean up in this role that we already use in production. We just need to push it to openstack-ansible itself | 13:39 |
guilhermesp | the only component that I change was our grafana-ansible role to ansible-grafana role from cloudalchemy. Not changes in the roles of kafka, storm, zookeeper and influxdb at all. They are the same that we use in production with netwon and pike | 13:40 |
witek | could you point me to tempest test logs from the Zuul CI job? | 13:45 |
witek | guilhermesp: ^^ | 13:45 |
guilhermesp | yes, but there isn't too much information there. What can I do also is to point you the tempest report that is in my test vm. Just need to put a temporary floating ip so you can access. Is that ok? | 13:53 |
guilhermesp | actually, there is information in the zuul ci job, but is really messy :P | 13:55 |
guilhermesp | here: http://logs.openstack.org/21/578721/14/check/openstack-ansible-functional-ubuntu-xenial/837cd97/job-output.txt.gz#_2018-07-06_13_40_32_276764 | 13:55 |
guilhermesp | tell me if you prefer a html formatted output | 13:56 |
openstackgerrit | Christian Brandstetter proposed openstack/monasca-log-api master: Add tooling for building Docker image https://review.openstack.org/579092 | 13:56 |
guilhermesp | i.e: https://imgur.com/a/NUBiIws | 13:58 |
*** jgu has joined #openstack-monasca | 14:26 | |
*** jgu__ has joined #openstack-monasca | 14:27 | |
*** jgu has quit IRC | 14:31 | |
witek | guilhermesp: from persister logs I can see that no message was published/consumed to/from alarm-state-transitions topic | 14:34 |
witek | kafka logs look clean | 14:34 |
witek | I guess I'd check thresholding configuration | 14:34 |
witek | and storm | 14:34 |
guilhermesp | hum, I made some assumptions yesterday about monasca-thresh when I saw this story: https://storyboard.openstack.org/#!/story/2001533 | 14:37 |
guilhermesp | so, we can assume that monasca-tempest-plugin has no issue at all, right? | 14:38 |
*** Guest42906 has quit IRC | 14:48 | |
witek | I cannot really confirm that, but the fact that all alarm related tests fail indicate possible problem with Kafka topic or thresholding engine | 14:48 |
witek | the logs you've posted here, http://paste.openstack.org/show/725352/ are expected | 14:48 |
guilhermesp | witek: good to know, so all of these log are not related to the real issue. That's a good clue | 14:49 |
witek | after creating alarm-definitions, can you list the alarms? | 14:50 |
guilhermesp | via monasca cli, when I create the same alarm as the tests, with "monasca alarm-definition-create ... " I can list them via "monasca alarm-definition-list" | 14:50 |
guilhermesp | during the tests, I also made a query in mysql db, and I saw the alarms createds in the tests | 14:52 |
guilhermesp | I'll be afk for just some minutes | 14:52 |
witek | alarm definitions are persisted directly in mysql db | 14:55 |
*** kaiokmo has joined #openstack-monasca | 14:56 | |
witek | alarms are evaluated by thresholding engine based on alarm definitions | 14:57 |
witek | alarm state transition events are generated there and published to 'alarm-state-transitions' topic | 14:57 |
*** bobh has joined #openstack-monasca | 15:08 | |
*** rodolof has quit IRC | 15:22 | |
*** bobh has quit IRC | 15:24 | |
*** rodolof has joined #openstack-monasca | 15:37 | |
*** jgu__ has quit IRC | 15:38 | |
*** jgu has joined #openstack-monasca | 15:39 | |
*** rodolof has quit IRC | 15:42 | |
*** rodolof has joined #openstack-monasca | 15:43 | |
guilhermesp | witek: sorry for the delay (lunch time here). Well, I will take a better look at the 3 componentes: kafka, storm and monasa-threash to have some more clues | 15:44 |
guilhermesp | maybe there is something different from my two deployments (newton and pike), I will compare the test deployment with these production deployments | 15:45 |
*** dougsz has quit IRC | 16:59 | |
*** bobh has joined #openstack-monasca | 17:45 | |
openstackgerrit | jacky06 proposed openstack/python-monascaclient master: Remove the duplicated word https://review.openstack.org/581437 | 17:51 |
*** bobh has quit IRC | 19:00 | |
guilhermesp | witek: probably it is a problem with storm workder. There's a loop of killikng worker's process. Trying to find the cause in the logs | 19:29 |
guilhermesp | here's the output of one of the storm workers: http://paste.openstack.org/show/725461/ | 20:14 |
*** jgu has quit IRC | 20:37 | |
*** aagate has joined #openstack-monasca | 22:42 | |
*** aagate has quit IRC | 22:48 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!