15:00:44 #startmeeting ceilometer 15:00:45 Meeting started Thu Aug 28 15:00:44 2014 UTC and is due to finish in 60 minutes. The chair is gordc. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:46 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:49 The meeting name has been set to 'ceilometer' 15:00:56 hi folks. 15:01:00 o/ 15:01:09 o/ 15:01:10 o/ 15:01:10 o/ 15:01:15 o/ 15:01:22 o/ 15:01:27 o/ 15:01:50 o/ 15:02:12 O/ 15:02:32 we'll say that's quorum. 15:02:59 #topic Juno-3 planning 15:03:11 o / 15:03:17 <_nadya_> o/ 15:03:19 so we have code up for all the bps we're targetting for j-3 15:03:43 https://launchpad.net/ceilometer/+milestone/juno-3 15:04:02 please prioritise your reviews for the bps we're targetting. 15:04:17 nsaje: any update on the partitioning work? 15:04:34 i know you've been putting a lot work there... much appreciated 15:04:46 gordc: yes, I'm working on a way to migrate the rest of the central agent pollsters to discoveries 15:05:05 nsaje: do you need any help there? 15:05:20 i'd assume quite a few pollsters don't use resources. 15:05:27 gordc: yes, I'd like to quickly verify my thoughts 15:05:52 gordc: I'm thinking of using fabiog_ 's idea of getting endpoints from keystone (for those pollsters that poll general stuff, like for example swift, which polls account info) 15:06:07 gordc: and then treating those endpoints as sort of 'resources' for the pollster 15:06:47 nsaje: is this going to be a separate patch? 15:06:53 gordc: so they'd get balanced around using the same logic, thus ensuring there's always someone polling and at the same time if there's multiple endpoints, multiple agents can share the work 15:07:01 fabiog_: yes 15:07:07 nsaje: that sounds like an option. i believe swift pollsters and glance pollsters already try to authenticate endpoint first so there should be code there 15:07:15 gordc: there is 15:07:16 nsaje: thanks it will be easier to follow 15:07:27 * nealph nods 15:07:51 fabiog_: yeah. gordc : I guess we could put this as a bugfix, if we fail to get it in for j-3? Since it's actually a bug in a way, pollsters not using discoveries :) 15:08:22 o/ 15:08:25 gordc: I'm a bit worried, j-3 is wednesday, right? 15:08:42 nsaje: that's an option. i'm not sure eglynn set a hard FF date yet so i'll let him figure it out when he gets back tomorrow 15:09:12 nsaje: do you expect it'll take long? i should be able to help out if needed. 15:09:24 gordc: no, I think I'll have a patch up tomorrow 15:09:26 gordc: DinaBelova and I have tested HA on a tree node devstack. Our results were shared with nsaje and cdent 15:10:03 idegtiarov: gordc : yeah, the partitioning code itself works. The problems with the central agent are due to the pollster<->discovery issue 15:10:09 nsaje: sounds good. i think as long as patch is up we should be good. eglynn can make final decision. 15:10:49 gordc: cool. One other thing is a subsequent partitioning patch (since it looks like the actual patritioning code is working) at https://review.openstack.org/#/c/115237/ 15:11:11 please take a look at it guys, so we can get it rolling 15:11:18 that's it from me :) 15:11:31 nsaje: will do 15:11:41 cool cool. i'll have a look. again thanks for work and don't hesitate to ask for help 15:11:51 gordc: no problem, will do! 15:11:52 gordc: nsaje: I don't think that eglynn would aim for that hard FF, especially in case of this patch 15:12:00 cdent: we all good on your end re: grenade survivability? 15:12:23 ildikov: agreed. 15:12:27 Yeah, at this point it is just a matter of getting reviews and getting the gate to settle down; 15:12:45 swiftclient and sarah dashboard have caused some trouble in the last day or so 15:12:52 cdent: the gate will be the difficult part. 15:13:39 s/sarah/sahara/ 15:13:58 those were the two items i've been tracking... 15:14:05 any other concerns regarding j-3 items? 15:14:39 moving on i guess. 15:14:42 gordc: the last part of the admin guide is in WIP state on gerrit, I will try to finish it next week 15:14:55 ildikov: you have a link handy? 15:14:56 gordc: but it's external from the Launchad PoV 15:15:48 gordc: the last patch: https://review.openstack.org/116909 15:16:03 ildikov: cool cool. i'll have a look. 15:16:08 gordc: can you confirm that the doc build is stable and passing? I've been following the ml but wanted to verify for the sake of https://review.openstack.org/#/c/113396/ 15:16:08 gordc: but it is far from being ready for review now 15:16:39 ildikov: ok. i'll give it a quick pass then. 15:16:58 nealph: the gate should be stabilised now... (haven't seen any random failures past few days) 15:17:19 gordc: cool, thanks. 15:17:24 nealph: gordc's patch seemes to solve the wsme issue with the docs build 15:17:51 gordc: cool, thanks, if you have any idea for the troubleshooting guide content, it's welcomed :) 15:17:57 ildikov: cdent and i are trying to figure out the wsme issues but we should be ok from ceilometer pov 15:18:13 gordc did you see the changes I committed last night? 15:18:32 folks, o/ sorry, was unavailable to attend the beginning 15:18:44 gordc: yeap, I saw your discussion here yesterday, it would be good to know the root cause, but until the gate is stable we're fine 15:19:06 https://review.openstack.org/#/c/116371/ 15:19:19 cdent: yep seems good. dhellmann, jd__: wsme patch we're working on ^ 15:19:28 not a blocker since we have workaround 15:19:43 ok. so moving on 15:19:52 that's passing in the gate and it passes, consistently locally with multiple PYTHONHASHSEED variations 15:19:52 * dhellmann adds to his review list 15:19:59 #topic Tempest status 15:20:05 dhellmann: thanks! 15:20:16 DinaBelova: is this your update? 15:20:33 well, kind of :) 15:20:41 DinaBelova: it is now :) 15:20:44 go for it. 15:21:01 as for the my unskipping change, it's failing now due to the issues with the glance, I guess 15:21:18 problem like 'we want to see image in one state, it's in other one' 15:21:27 DinaBelova: yeah, i noticed... seems like a different failure every sigle recheck. 15:21:30 I guess that's only logical blocker here 15:21:37 gordc, yes, indeed 15:21:53 I hope to see green lines soon 15:22:02 DinaBelova: i can't see it being related to unskipping telemetry test though. 15:22:07 as I'll them ping Sean to take a look on that 15:22:13 yeah, it has no actual link 15:22:25 ok, I'll ping him once again today 15:22:33 possibly it'll make some result 15:22:33 :) 15:22:44 as the original issue was just gone 15:22:51 DinaBelova: that's what i think too. 15:23:09 hopefully we can get that test back up and running soon. 15:23:12 as for the Tempest, I guess that's it actually 15:23:14 yeah 15:23:25 cool cool 15:23:34 anyone else have items relating to Tempest? 15:23:45 a-ha one moment 15:24:27 just forgot about some things we need to backport in ceilo and devstack to allow https://review.openstack.org/#/c/115971/ to be merged 15:24:41 that's change by cdent to tempest 15:25:11 currently it's blocked due to the fact swift + ceilo was disabled in the icehouse 15:25:24 that whole stack of stuff seems to be hung up in the usual way: needs review, needs the gate to unsuck 15:25:25 gordc, one of this changes is yours https://review.openstack.org/#/c/116229/ 15:25:51 https://review.openstack.org/#/c/116230/ - and this one to the devstack 15:26:00 devstack change has +2 already 15:26:39 ok, i don't have stable release powers so you'll need to check with jd__ and eglynn to confirm ceilomter patch. 15:26:43 ceilo team needs to backport our change before the devstack 15:26:52 otherwise we'll have some problems with Sean :D 15:26:56 ok 15:27:12 will do 15:27:19 and that's all for tempest i'll assume 15:27:34 #topic TSDaaS/gnocchi status 15:27:45 jd__: any updates here? 15:27:50 just +2a on https://review.openstack.org/#/c/116229 15:28:01 so I'm still working on archive policies 15:28:08 I've pushed a few patches but it's not complete yet 15:28:16 jd__: you can kill me in front of everyone, not too much progress on my side 15:28:27 I hope to finish soon, waiting for some feedbacks too 15:28:27 jd__, ildikov - about the dispatcher 15:28:27 ildikov, hehe 15:28:27 I noticed ildikov :) 15:28:34 that's all I go 15:28:34 t 15:28:42 ildikov: we do that behind closed doors 15:29:01 probably idegtiarov might do that, but only if it won't be done next week :) 15:29:12 idegtiarov has the vacation next week :) 15:29:28 gordc: well, from my PoV, it does not make the situation better, I will be dead anyway by the end ;) 15:29:28 if it'll be still no progress there, he might take this role on him 15:29:29 DinaBelova: that's for openTSDB dispatcher? 15:29:30 :) 15:29:42 ildikov: no comment. :) 15:30:15 gordc, no I'm about finishing this one https://review.openstack.org/#/c/98798/ 15:30:34 ildikov: o, yep i will be glad to help with the dispatcher 15:30:34 collector dispatcher to plug gnocchi in the ceilo 15:30:36 DinaBelova: uh gotcha, i dont know why i just read dispatcher as driver 15:30:44 no-no :) 15:31:03 sounds good. ildikov, DinaBelova i'll let you guys sync up on whether we want to handover work item. 15:31:09 ok :) 15:31:12 DinaBelova: ok, if I'll not have anything working by the end of next week, we can discuss about that 15:31:20 ildikov, ok, good 15:31:23 ildikov: cool cool thanks. 15:31:27 DinaBelova: cool, thanks 15:31:37 no more pasta news i assume? 15:32:17 #topic Open discussion 15:33:00 * cdent has nothing 15:33:10 * DinaBelova neither 15:33:14 short meeting? 15:33:16 :D 15:33:19 \o/ 15:33:19 don't jinx it. 15:33:31 gordc: LOL :) 15:33:45 i'll give it a minute... 15:34:08 everyone watch the dial of your clock tick. 15:34:09 just a quick question, what's the status of Events in Ceilometer atm? 15:34:17 jinx'd 15:34:35 since they're coming up quite a lot lately, but I don't know if there's anyone actively maintaining that part of ceilo 15:34:37 :) 15:34:52 nsaje: it's partially there? rax is heads down on stacktach so i don't believe we have an owner for it. 15:35:05 sounds right 15:35:08 nsaje, all implemented for the mongo, hbase, and long ago for the sql 15:35:19 there was some interest here locally on picking it up but i don't think we have anyone officially working on it. 15:35:20 idegtiarov implemented them for mongo and hbase 15:35:28 <_nadya_> nsaje: idegtiarov is an expert with events :) 15:35:31 we have now full db driver coverage for the current events functionality 15:35:32 yeah :) 15:35:35 if anyone wants to start working on it that'd be awesome 15:35:54 nsaje: Events are almost implemented this patch needs to be landed https://review.openstack.org/#/c/112582/ 15:35:55 gordc: I agree, it needs someone working on them, good topic for the summit I guess 15:36:15 idegtiarov: on my list, almost there :) 15:36:21 folks, what do you mean by working on them? 15:36:28 lots of work has been done by idegtiarov :) 15:36:33 nsaje: yeah. i think it's pretty important still. just wasn't on juno track because of other fires. 15:36:37 ildikov: just for folks fyi 15:36:41 expanding functionality to bring it on par with stacktach I guess 15:36:52 DinaBelova: ^ 15:36:58 DinaBelova: there's a few events related bps still up that were never implemented/started 15:37:05 nsaje, a-ha, got it 15:37:19 just missed the stacktach word :) 15:37:29 i think we need to start testing it a bit more as well since i'm not sure how well events works tbh. 15:37:37 gordc: exactly, me neither 15:37:49 :) 15:38:11 <_nadya_> ok, if you decided to continue the meeting I ask one thing :) 15:38:15 DinaBelova: yep. i mean stacktach would be an option for people looking for richer events support for short term... but it'd be nice to have events a bit richer in ceilometer as well 15:38:55 _nadya_: go for it :) 15:39:20 <_nadya_> Migration in HBase is rather complicated thing. And may take a lot of time. What do you think if we have 2 migration scripts: for development version and for production? 15:39:58 <_nadya_> the problem is that we need map-reduce job for production I guess... 15:40:02 _nadya_: how have we been handling migration in dev so far? none i suppose? 15:40:10 _nadya_: how would these two look like? 15:40:18 gordc, ityaptin is working on it 15:40:20 <_nadya_> gordc: yep 15:40:58 * DinaBelova searching the change - I supposed it was a draft 15:41:05 <_nadya_> for development python script is ok, ityaptin has already provided a change request 15:41:08 DinaBelova: _nadya_ : i see... not a hbase expert myself so it'd be good to have hbase driver as a viable optoin 15:41:24 gordc, indeed 15:42:03 <_nadya_> but if we are talking about gigabytes or terabytes we cannot just read-change-writeback in python script 15:42:38 <_nadya_> it should be map-reduce job and possibly .jar-file that will be executed using Hadoop 15:43:11 hmm... i had a feeling the production version might require something unique 15:43:12 <_nadya_> I believe that if somebody would have production HBase then Hadoop will be available 15:43:56 is the production version something reusable or would it be dependent on the deployers environment? 15:44:21 gordc, well, it's very logical to use hadoop jobs to perform actions on the hbase data 15:44:27 that's kind of common pattern 15:44:37 <_nadya_> it's reusable 15:44:50 as people usually have hbase close to the hadoop 15:44:56 DinaBelova: cool cool. i'm just wondering where we'd store it. 15:45:10 gordc - some tools folder? 15:45:12 <_nadya_> btw ityaptin script #link https://review.openstack.org/#/c/115615/ 15:45:17 oh! 15:45:20 _nadya_, thanks 15:45:30 I have bad connection to the review.openstack.org 15:45:32 :( 15:45:51 <_nadya_> gordc: yep, have the same question about where to store java (?) code 15:46:03 or some separated repo 15:46:06 * DinaBelova thinking 15:46:24 _nadya_: yeah i don't know if there's precedence... 15:46:44 nealph: is monasca on stackforge? 15:47:00 gordc: yes 15:47:13 fabiog_, oh, really? 15:47:19 I did not see their repo 15:47:21 <_nadya_> anyway, we have time to think about it :) I just wanted to note that python migration script is for dev only 15:47:22 >_< 15:47:29 _nadya_, ++ 15:47:32 fabiog_: cool cool. _nadya_ i guess stackforge could be an option if we decided to go down that route 15:47:33 DinaBelova: yeap, it's been there for a while now 15:47:39 DinaBelova: they are moving it, not sure they completed the transfer 15:47:45 _nadya_: agreed. start with dev version first. 15:47:47 hehe, ok 15:48:03 <_nadya_> ok! thanks for discussion! 15:48:08 gordc, so please take a look on the https://review.openstack.org/#/c/115615/ 15:48:10 ;) 15:48:19 _nadya_: thanks for heads up 15:48:19 _nadya_: gordc: we needed the migration because of a separator patch IIRC 15:48:33 gordc: here it is the link: https://launchpad.net/monasca 15:48:48 _nadya_: gordc: which version should it be dependent on? 15:49:20 <_nadya_> ildikov: dev. it should depend on https://review.openstack.org/#/c/115615/ 15:50:13 _nadya_: ok, cool, thanks 15:50:28 <_nadya_> ildikov: #link https://review.openstack.org/#/c/106376/ . idegtiarov could you please create a dependency? 15:51:18 anything else? 15:51:26 _nadya_, I'll ping idegtiarov directly 15:51:59 wall of text inc. 15:51:59 last call... 15:52:01 I have one more question regarding events: currently, the 'event' notification handler uses a dispatcher directly. Shouldn't it use a publisher to publish it to the collector, especially now that we'll be using notifications for publisher->collector communication and thus we have guarantees about not losing messages. We then even have an option to run a separate collector for events, with a different queue, that does retries if 15:52:01 saving to db fails. 15:52:05 _nadya_: k, thanks 15:52:05 jd__: ^^ 15:52:38 I seem to be ruining your plans today gordc :-) 15:52:43 no comment 15:52:50 nsaje: what's the point to forward notifications again? 15:53:11 collector is only handling samples 15:53:39 jd__: I was having separation of concerns in mind 15:53:45 jd__: collector is saving stuff to db so to say 15:54:03 jd__: and they could make use of multi-publisher 15:54:08 not sure it's worth it, especially considering collector might be bound to disapear 15:54:23 jd__: ah, right, gnocchi 15:54:27 well multi-publisher is kinda part of oslo.messaging 15:54:29 jd__: ok then 15:54:31 like you can provide several publishers 15:54:43 maybe you can't provide complex URL, but that should be changed in oslo.messaging I'd say 15:54:54 otherwise I'm afraid we're going to add too many layers at that point 15:55:33 is anyone an SQLAlchemy ORM expert by any chance? 15:55:36 I'm losing my mind. 15:55:36 jd__: ok. It just struck me as odd since dispatchers are used in both notification-agent and the collector, that's all 15:55:43 nsaje: you can draft up a quick spec if anything... in case there's other ideas you had 15:55:52 jd__: use core 15:55:57 gordc: haha, ok, I'll shut up now :-) 15:55:58 gordc: :( 15:56:10 nsaje: I would live events in a separate path 15:56:16 gordc: you'd advocate dropping the ORM entirely? 15:56:18 s/live/leave 15:56:19 jd__: I'm not an expert but I've certainly beat my head against it enough to have some experience 15:56:56 fabiog_: there's no actual path at the moment, they just get in and written to db right away. That's why I wanted to put them through a pipeline actually (but that's a whole load of work, not short-term) 15:57:00 gordc: speaking of SQL can you guys review this patch: https://review.openstack.org/116748 15:57:03 jd__: i don't know... it seems a lot better performance-wise... and unless you're using relationships i don't really know what orm offers over core 15:57:34 jd__: mayeb something as Bayer... i can take a look at orm stuff if you want an extra pair of eyes 15:57:42 nsaje: that is the most efficient way, they are already published by the services, why do you want to re-publish them? 15:57:44 nsaje: how would that affect the performance/bandwidht? 15:58:13 to ask Bayer* 15:58:17 nsaje: I asked because of what fabiog_ has just mentioned 15:58:31 gordc, ok, let us know the result 15:58:54 just an fyi, you guys will probably need to move discussion over to #openstack-ceilometre 15:59:10 any other items? 15:59:15 ok 15:59:21 https://bugs.launchpad.net/python-ceilometerclient/+bug/1351841 15:59:24 Launchpad bug 1351841 in python-ceilometerclient "python-ceilometerclient does not works without v3 keystone endpoint" [High,Triaged] 15:59:24 let's move current discussion to ceilomter 15:59:28 ok 15:59:37 gordc: ok 15:59:42 fabiog_: real quick. 15:59:44 I checked the bug and I discovered that port 35357 only advertize V3 15:59:50 ok SQLAlchemy expert, here's my problem http://paste.openstack.org/show/101806/ 15:59:53 that's why all the V2 stuff fails 15:59:54 you got 1 hour 16:00:04 jd__: i got nothing 16:00:06 wrong channel :D 16:00:15 but you got the idea 16:00:15 alright, ew're out of time. 16:00:18 ceilometer 16:00:22 #endmeeting