15:00:07 <eglynn> #startmeeting ceilometer 15:00:08 <openstack> Meeting started Thu Aug 7 15:00:07 2014 UTC and is due to finish in 60 minutes. The chair is eglynn. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:12 <openstack> The meeting name has been set to 'ceilometer' 15:00:20 <eglynn> hey y'all! 15:00:27 <tongli> hello. 15:00:29 <ildikov> o/ 15:00:33 <fabiog_> o/ 15:00:46 <cdent> hola 15:00:49 <llu-laptop> o/ 15:01:01 <_nadya_> hi! 15:01:16 <eglynn> #topic juno-3 planning 15:01:23 <eglynn> #link https://launchpad.net/ceilometer/+milestone/juno-3 15:01:32 <eglynn> let's focus in on the 2 blocked BPs 15:01:43 <eglynn> first the bigger-data-sql ... seems there's a new patchset pending, but converging on agreement? 15:01:50 <eglynn> #link https://review.openstack.org/#/c/101009 15:01:53 <DinaBelova> o/ 15:02:16 <eglynn> gordc, cdent: does it feel like convergence? 15:02:40 <sileht> o/ 15:02:56 <cdent> gordc just posted some new comment saying it seems pretty okay, but may have some issues with meter reads, so a bit more poking to come, but it is heading in the right direction 15:03:11 <cdent> (specially meter-list performance) 15:03:25 <eglynn> #link https://etherpad.openstack.org/p/ceilometer-big-data-pt2 15:03:26 <nealph> o. 15:03:37 <eglynn> that's the latest proposal basically ^^^ ? 15:03:52 <cdent> there's active code here: https://review.openstack.org/111313 15:04:25 <cdent> the decision was that there was no way to reach the ideal schema by thinking along, real code was required 15:04:41 <cdent> what's implemented there is very close to what's in the etherpad 15:04:49 <eglynn> cool, so we need to get the spec lined up the etherpad and the PoC code in order to get it landed, amiright? 15:05:06 <cdent> reckon so, yes 15:06:04 <eglynn> cdent: cool, I'll follow up with gordc after the meeting 15:06:13 <eglynn> next central-agent-partitioning ... we have too alternative proposals 15:06:24 <eglynn> we've done a preso/hangout on each this week 15:06:28 <eglynn> so it really is decision time now 15:06:34 * eglynn pastes links for each approach ... 15:06:39 <eglynn> (prepare for flood!) 15:06:49 <eglynn> #link https://review.openstack.org/101282 15:06:50 <eglynn> #link http://www.slideshare.net/FabioGiannetti/ceilometer-central-agent-activeactive-ha-proposal 15:06:52 <eglynn> #link https://review.openstack.org/111978 15:06:53 <eglynn> #link http://www.slideshare.net/EoghanGlynn/hash-based-central-agent-workload-partitioning-37760440 15:07:26 <fabiog_> eglynn: can we have some clarifications on the Tooz future support? 15:07:33 <eglynn> can everyone who cares about this feature (e.g. attended the hangouts or participated in the reviews already) please "nail their colours to the mast" on gerrit before EoD today? 15:07:57 <DinaBelova> eglynn, ok, will be done :) 15:08:14 <DinaBelova> eglynn, actually I wonder what will be the answer on the fabiog_ quesiton here 15:08:15 <ildikov> eglynn: I'm kinda waiting for the answer of that question that Fabio has just asked and also asked by you on gerrit 15:08:16 <eglynn> fabiog_: since you posed the question on gerrit already, I'll ask jd__ to respond there 15:08:23 <DinaBelova> ildikov, ++ 15:08:32 <DinaBelova> eglynn, thank you sir 15:08:33 <jd__> o/ 15:08:41 <DinaBelova> jd__, good evening :) 15:09:00 <eglynn> jd__: question on https://review.openstack.org/101282 about future maintainership of tooz 15:09:19 <eglynn> jd__: sorry wrong link! ... https://review.openstack.org/111978 15:10:11 <eglynn> let's complete that discussion on gerrit 15:10:12 <jd__> it has support for harlowj and it's going to be put under the Oslo maintenance 15:10:18 <jd__> s/for/from/ 15:10:45 <jd__> same as stevedore, etc… 15:11:09 <eglynn> jd__: a-ha, cool, that's good news about the oslo aspect 15:11:11 <DinaBelova> jd__, thanks for the clarification 15:11:26 <jd__> we're just waiting for some projects to use it before moving it under Oslo 15:11:29 <fabiog_> jd__: thanks, this helps 15:11:31 <jd__> and I'm working as we speak as using it with Nova 15:11:43 <jd__> (to replace their custome implementation of memcached/zookeeper) 15:11:46 <_nadya_> jd__: what projects? :) 15:12:03 <jd__> _nadya_: Nova and Neutron are first in sight last time I checked 15:12:04 <eglynn> _nadya_: nova 15:12:10 <DinaBelova> jd__, I remember kind of this mailing thread - am I right it was already started? 15:12:16 <jd__> there might be more I'm not aware of 15:12:29 <_nadya_> cool 15:12:42 <jd__> it's likely that a lot of project have crafted custom solutions that are not so robust and would have a better usage of tooz, but we'll see 15:12:55 <jd__> and Ceilometer obviously :) 15:13:32 <eglynn> jd__: and on the cards also, possibly new tooz drivers? (e.g. based on olso-messaging) 15:13:47 <jd__> sure thing, we just need someone to write them 15:13:56 <jd__> it's not on my priority list, though if I get bored, you never know… 15:14:01 <jd__> :) 15:14:31 <eglynn> fabiog_: it that sufficient information about tooz future status? 15:14:58 <fabiog_> eglynn: sure. I think it responds to my concern 15:15:06 <eglynn> fabiog_: excellent! 15:15:11 <eglynn> jd__: thank you sir! 15:15:29 <jd__> you're welcome ladies and gentlmen 15:15:58 <eglynn> ok, apart from those 2 blocked BPs, does anyone have any concerns about any other stuff target'd for juno-3? 15:16:14 <jd__> Gnocchi dispatcher? 15:16:20 * jd__ whispers 15:16:42 <ildikov> jd__: I have several excuses, none of them is good enough for mentioning it here... 15:16:53 <jd__> :-) 15:17:22 <eglynn> ildikov: are you still confident that it's do-able in the j-3 timeframe? 15:17:30 <ildikov> if anyone would like to jump in, I will not kill him/her, otherwise, I'm on it, just had some environmental issues to fix 15:17:42 <eglynn> ildikov: cool, understood! :) 15:18:03 <ildikov> eglynn: it has to be regardless my concerns... 15:18:17 <ildikov> eglynn: still 4 weeks, right? 15:18:28 <eglynn> ildikov: yep 4 weeks from today 15:18:34 <ildikov> eglynn: cool, tnx 15:18:46 <eglynn> OK, shall we move on? 15:18:55 <fabiog_> eglynn: I responded already in Gerrit ;-) 15:19:06 <eglynn> fabiog_: thank you sir! 15:19:10 <eglynn> #topic Ironic conductor ipmi -> ceilometer saga (punted from last week) 15:19:19 <eglynn> cdent: the floor is your's sir! 15:20:03 <cdent> I think we can punt this one further down, unless any is gasping for it. 15:20:13 <eglynn> cdent: k ... Haomeng's patch has now landed amiright? 15:20:19 <cdent> yes 15:20:21 <eglynn> #link https://blueprints.launchpad.net/ironic/+spec/send-data-to-ceilometer 15:20:30 <eglynn> cool, let's return to that so 15:20:39 <eglynn> #topic Tempest status 15:21:10 <eglynn> DinaBelova: your mongodb experimental job landed, nice work! 15:21:17 <DinaBelova> thank you sir :) 15:21:25 <DinaBelova> eglynn, it behaves quite nice :) 15:21:46 <DinaBelova> I encourage everyone who have ceilo changes to comment "check experimental" on your patches 15:21:52 <DinaBelova> and see what will happen 15:22:14 <DinaBelova> currently this job has experimental status and needs to be run manually 15:22:21 <eglynn> great! ... so maybe we could consider switching the main tempest job to mongodb if it continues to behave nicely for the next say week or so? 15:22:34 <DinaBelova> eglynn, I hope so, yes 15:22:55 <llu-laptop> DinaBelova: so the magic cast is 'check experimental', literally? 15:23:02 <DinaBelova> llu-laptop, yes 15:23:05 <llu-laptop> good 15:23:16 <eglynn> DinaBelova: so se could potentially swap the status of sql-a and mongodb in the gate? 15:23:35 <DinaBelova> eglynn, well, if TC and over community won't be agians - why not, actually 15:23:46 <DinaBelova> that will work faster and more stable I guess 15:23:51 <eglynn> i.e. mongodb becomes the main "blessed" job while sql-a becomes the second-tier 15:24:09 <eglynn> since that the reality in terms of what is recommended by distros for production 15:24:16 <DinaBelova> eglynn, yeah, I got your idea - that lgtm actually, but we need QA team blessing here :) 15:24:34 <eglynn> DinaBelova: cool :) ... I'll raise at the project meeting next Tuesday 15:24:57 <DinaBelova> eglynnm thank you sir - I'll try to attend - but it's late a little bit for me, so can't promise :) 15:24:59 <eglynn> #action eglynn raise blessing the mongodb-based tempest variant at PTL meeting 15:25:09 <eglynn> DinaBelova: np! 15:25:18 <DinaBelova> :) 15:25:27 <eglynn> cool, this feels like real progress :) 15:25:40 <eglynn> k, anything else on Tempest? 15:25:45 <DinaBelova> also I've raised this question yesterday, but not all of us was aware, so I'll say one more thing here 15:25:54 <eglynn> shoot! 15:26:08 <DinaBelova> we have notification tempest patches -2'ed - https://review.openstack.org/67164 and https://review.openstack.org/70998 15:26:10 <DinaBelova> that 15:26:23 <DinaBelova> that is because of unstable gate tests passing here 15:26:38 <_nadya_> nice :) 15:26:39 <DinaBelova> sometimes notifications are just not comint 15:26:42 <DinaBelova> coming* 15:26:47 <DinaBelova> _nadya_, hehe 15:27:06 <DinaBelova> eglynn - as I remember, you even disabled nova notifications now to keep gate more or less stable 15:27:11 <eglynn> DinaBelova: yes, I think these will have to be moved to the new in-tree functional tests 15:27:16 <_nadya_> not comming or not sent? 15:27:32 <DinaBelova> _nadya_, that actually is a good quesiton 15:27:45 <DinaBelova> vrovachev could not reprouce it locally 15:27:47 <eglynn> _nadya_: not arriving and persistent in the metering store in time I thought? 15:28:03 <DinaBelova> eglynn, yes, at least it looks so after the logs surfing 15:28:38 <DinaBelova> but we have no actual data to say - 'yes, it is so' - as it was not actually reproduced 15:28:43 <DinaBelova> locally 15:29:15 <eglynn> k, I think we're kind of blocked on that one unless we it's simply collector latency that would not be such an impact if the main tempest job were re-based on mongodb? 15:29:29 <DinaBelova> eglynn, yes, exactly 15:29:58 <ildikov> sounds more like an environmental or load related issue tho' 15:30:33 <gordc> DinaBelova: have you tried just bumping up the number of collector workers? 15:30:36 <eglynn> it's very hard to know without fully reproducing the Jenkins slave 15:31:03 <DinaBelova> gordc, don't remember actually - I thought Vadim had tried different combinations... 15:31:06 <_nadya_> ildikov: yep 15:31:09 <eglynn> gordc: that could be tried, tho' the tests would also have to unskipped to confirm 15:31:10 <ildikov> eglynn: I know that is why I said sounds like :( 15:31:13 <DinaBelova> ildikov, +1 15:31:17 <llu-laptop> in that case, should we postpone the patch https://review.openstack.org/#/c/80225/ until we're sure that the notification won't get lost, at least not change the default publisher until then 15:31:59 <gordc> eglynn: yeah, i'm not sure if its' helpful much either... might just be masking issue by incresaing workers. 15:32:02 <eglynn> llu-laptop: can't we verify that wil local tempest runs? 15:32:08 <gordc> would be good if there were logs for mq 15:32:23 <eglynn> gordc: k, worth experimenting with in any case I think 15:32:23 <DinaBelova> gordc, YES, for sure 15:32:40 <eglynn> k, let's move on with the agenda 15:32:45 <DinaBelova> eglynn, ok 15:32:46 <eglynn> #topic TSDaaS/gnocchi status 15:33:06 * eglynn passes the conch to jd__ ... 15:33:36 <jd__> nothing really knew on that side for this week 15:33:40 <jd__> s/knew/new/ 15:33:51 * DinaBelova whispers that initial python-opentsdbclient version was merged 15:33:57 <jd__> I've started to work on the archiving policy we discussed at mid-cycle 15:34:04 <jd__> cool DinaBelova 15:34:04 <eglynn> jd__: cool 15:34:16 <jd__> and I'm also working on tooz wrt the file driver we have in Gnocchi 15:34:20 <eglynn> DinaBelova: I also had a quick peek at the opentsdb driver 15:34:22 <DinaBelova> so I rewritten opentsdb driver change to fit it 15:34:28 <DinaBelova> eglynn, oh, cool 15:34:43 <jd__> finger crossed I'll have a first implementation of the archive policy patch for next week 15:34:52 <jd__> s/finger/fingers/ 15:34:59 <eglynn> jd__: excellent! 15:35:27 <eglynn> DinaBelova: so one thought that occurred is that we should think again out about what we consider the set of "standard aggregates" to be 15:35:41 <DinaBelova> eglynn, also I found one really interesting guy who used opentsdb much with ceilo - they have rewritten ceilo actually - so I'm trying to reach him for some experience here 15:35:42 <eglynn> DinaBelova: (given that opentsdb doesn't support median etc.) 15:35:52 <DinaBelova> eglynn, yeah, I found this moment too 15:35:53 <eglynn> cool 15:36:32 <eglynn> FYI on a more general point, I was asked to explain the background on gnocchi to the TC 15:36:38 <eglynn> ... this ML thread is a precursor to that conversation 15:36:46 <eglynn> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/042080.html 15:36:47 <DinaBelova> eglynn, yes, I saw the e-mail 15:37:24 <eglynn> that'll prolly be at the TC meeting next week, if there's space on the agenda 15:38:07 <eglynn> k, anything else to discuss on gnocchi folks? 15:38:45 <eglynn> cdent: shall we un-punt IPMI in that case? 15:39:02 <eglynn> #topic Ironic conductor ipmi -> ceilometer saga (punted from earlier in the meeting) 15:39:26 * eglynn passes conch to cdent ... 15:40:05 <cdent> The basic gist there is looking for advice on what to do when code on the ceilo side has to work with code on another project's side and that other side has insufficient testing such that simple runtime bugs are being shown up by my own manual testing. 15:40:18 <cdent> I wonder if people have good ways of "managing" that. 15:40:39 <cdent> (that's runtime bugs in the other side's code) 15:40:51 <eglynn> so I guess the obvious one is to draw attention to this testing deficit on gerrit 15:40:59 <eglynn> (which I think you've already done) 15:41:30 <eglynn> the less obvious approach is to jump in and start the ball rolling on the other project 15:41:47 <eglynn> i.e. by raising it on their meeting agenda 15:42:02 <eglynn> or even by authoring and proposing patches with tests 15:42:20 <eglynn> though that involves obviously a big investment of time 15:42:21 <jd__> send patches with unit tests + fixes to the other project, pointing how they suck? ;) 15:42:36 <cdent> you're so graceful jd__ :) 15:42:52 <jd__> at least reporting a bug with a basic test in it on how to reproduce is a good thing 15:43:00 <jd__> because sometimes you don't have the time or knowledge to fix it 15:43:22 <jd__> but most devs will feel the shame and will try to fix it since you provided a good way to reproduce it 15:43:29 <jd__> :) 15:44:03 <eglynn> so I think the telling them, and then telling them again, will get you so far ... 15:44:10 <eglynn> ... but jumping in and actually leading by example can be a more powerful motivator for the other project 15:44:21 <jd__> yes 15:44:23 <cdent> ✓ 15:44:32 * cdent worries about spread 15:44:33 <jd__> the good strategy is to give everything you can and terminate by "how can I help more?" 15:44:49 <jd__> that pushes the other dev to do something so you can help, and things move on :) 15:45:43 <eglynn> jd__: good point ... a helpful tone tends to be welcomed, whereas snark tends to be counter-productive 15:45:52 * jd__ nods 15:46:18 <eglynn> cdent: by "spread", you mean spreading cdent too thinly? 15:46:33 <eglynn> cdent: yeah, I hear ya ... it's a balancing act to be sure 15:46:48 <cdent> (yes, on spread) In the particular case the response I got was the it was basically impossible to unit test the particular path that was (multiple times) causing problems. 15:47:16 <cdent> Which suggests there's some kind of infrastructure problem that perhaps in-tree functional testing could help with, eventually. 15:47:23 <eglynn> cdent: you were unsatisified with that answer? ... i.e. felt it would/might be possible? 15:47:33 <eglynn> cdent: a-ha, yes, got it 15:48:29 * eglynn hopes we as a community are not starting to cargo-cult the new in-tree functional testing ... 15:48:36 * eglynn meant that half in jest :) 15:48:49 <cdent> it's a mythical savior, coming soon, probably around christmas 15:49:16 <DinaBelova> cdent, :D 15:49:32 <DinaBelova> yeah that looks quite mystical :) and mythical :) 15:49:34 <eglynn> cdent: let's codename the project "santa" :) 15:49:39 <DinaBelova> eglynn, lol 15:51:50 <eglynn> cdent: do you feel you've enough to go on with the above discussion? 15:52:14 <cdent> yup, just wanted to get a feel for people's approaches 15:52:27 <eglynn> cool 15:52:31 <eglynn> #topic open discussion 15:52:52 <ityaptin> Hi! One little update for performance tests. 15:52:58 <eglynn> ityaptin: cool 15:52:59 <ityaptin> I'm prepare doc for getting start tests with oslo messaging 15:53:04 <ityaptin> https://docs.google.com/document/d/1MXhZRXm8UoEN1kYt2NdZe7qyMb7pM9aW1kq5TuGz5lc/edit?usp=sharing 15:53:36 <ityaptin> In doc there is plan for prepare for test and repository url 15:53:57 <eglynn> ityaptin: a-ha, so this would it make it possible for anyone to run that load test? 15:54:05 <eglynn> (as requested by gordc last week) 15:54:16 <ityaptin> If someone wants to trying perf tests and have questions or run into issues - please comment or edit) 15:54:25 <ityaptin> threoreticall yes) 15:54:29 <eglynn> ityaptin: excellent, thanks! 15:54:43 <eglynn> it would be great if someone had time to try that 15:54:58 <eglynn> once it's proven out, maybe we could put the content up on the wiki? 15:56:03 <gordc> eglynn: i'm going to try to run it locally. of course it can't be compared to previous numbers since it's a diferent machine. 15:56:25 <gordc> ityaptin: i may ping you tomorrow. haven't tried to run it yet. 15:56:33 <ityaptin> yes, we can) Also we can add profiling graphs to wiki. They are very nice) 15:56:42 <ityaptin> gordc - ok) 15:56:48 <eglynn> gordc: excellent! :) 15:57:27 <eglynn> wow, we've 3 whole minutes left in our slot 15:57:31 <eglynn> ... that's unusual :) 15:57:34 <llu-laptop> ityaptin: in step2, which file to create? 15:58:13 <ityaptin> file with db urls, like hbase://localhost:9090 15:58:16 <ityaptin> one at line 15:58:47 <ityaptin> It's used for configuration backend runtime) 15:58:55 <llu-laptop> ityaptin: got that, thanks 15:59:43 <eglynn> ok, thanks folks as always for a productive meeting 15:59:49 <eglynn> ... let's call it a wrap! 15:59:54 <eglynn> #endmeeting