21:01:10 <eglynn-afk> #startmeeting ceilometer
21:01:11 <openstack> Meeting started Wed Dec  4 21:01:10 2013 UTC and is due to finish in 60 minutes.  The chair is eglynn-afk. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:14 <openstack> The meeting name has been set to 'ceilometer'
21:01:19 <dragondm> o/
21:01:26 <ildikov> o/
21:01:33 <eglynn-afk> I'm channelling jd__ again this week
21:01:34 <herndon_> o/
21:01:41 <terriyu> o/
21:01:50 <apmelton1> o/
21:01:56 <eglynn> anyone else lurking?
21:02:07 * dragondm imagines eglynn-afk with turban and crystal ball...
21:02:08 <lsmola> hello
21:02:12 <llu-laptop> o/
21:02:19 <eglynn> dragondm: LOL :)
21:02:36 <sandywalsh_> o/
21:02:40 <ildikov> dragondm: :)
21:03:02 <eglynn> OK, start your engines ladies & gentlemen ...
21:03:23 <eglynn> #topic Icehouse-1 status
21:03:35 <eglynn> well, it's been released
21:03:45 <eglynn> I'm y'all have seen the mail from ttx ...
21:03:55 <eglynn> #link http://tarballs.openstack.org/ceilometer/ceilometer-milestone-proposed.tar.gz
21:04:07 <eglynn> not a huge amount in it TBH
21:04:23 <eglynn> #link https://launchpad.net/ceilometer/+milestone/icehouse-1
21:05:11 <eglynn> well, I guess it's not quite released as yet
21:05:29 <gordc> jumped the gun.
21:05:32 <eglynn> milestone candidate for now
21:05:50 <eglynn> but I don't see anything else getting in at this stage
21:06:06 <eglynn> unless someone has something up their sleeve?
21:06:52 <eglynn> so lets consider icehouse-1 done & dusted, or?
21:07:10 <gordc> nothing from me.
21:07:43 <eglynn> thing is ...
21:07:47 <eglynn> a thin milestone-1 puts some pressure on us as the project team to deliver a chunky milestone-2
21:07:55 <eglynn> #justsayin'
21:08:04 <llu-laptop> how about https://review.openstack.org/#/c/58853/ for stable/havana?
21:08:35 <eglynn> llu-laptop: well the stable release cadence is independent of the main release cycle
21:08:39 <dragondm> Bunch of event stuff up for review. Presumably that'll land pretty quickly for i2, depending on reviews...
21:08:49 <eglynn> llu-laptop: but noted nonetheless
21:09:04 <eglynn> dragondm: yep, I'm thinking i2 for that
21:09:53 <eglynn> llu-laptop: yes, I'd love to have more eyes on that patch
21:10:15 <eglynn> llu-laptop: as I've got another patch proposed to stable that depends on it
21:10:34 <eglynn> (required for tempest to avoid a nasty intermittent integrity error ...)
21:10:52 <eglynn> so +1s (even non-binding are welcome!)
21:11:13 * dragondm will review after meeting...
21:11:24 <eglynn> dragondm: thanks!
21:11:27 <eglynn> anything else on icehouse-1?
21:11:30 <gordc> has anyone seen this https://bugs.launchpad.net/ceilometer/+bug/1257908 when testing the concurrency patch?
21:11:33 <uvirtbot> Launchpad bug 1257908 in ceilometer "DBDeadlock exception in sql backend" [Undecided,New]
21:11:46 <dragondm> Anything that will make tempest more stable is good...
21:12:21 <eglynn> gordc: I have not seen that
21:12:24 <gordc> ... or i should say with the concurrency patch... (i've no idea if they're related)
21:12:35 <gordc> eglynn: ok, cool. i'll keep debugging what i see.
21:13:03 <eglynn> gordc: smells like one of those Heisenbugs ;)
21:13:21 <eglynn> ok, moving on ...
21:13:28 <eglynn> #topic Tempest run errors
21:13:38 <eglynn> so I've looked into this
21:13:57 <eglynn> this is the issue raised by dkranz on the ML
21:14:17 <eglynn> basically cielo is spewing unexpected ERRORs into the logs
21:14:31 <eglynn> that causes Temptest CI runs to fail
21:14:45 <eglynn> and hence the specific log messages have to be whitelisted
21:14:50 <eglynn> whihc is badness
21:15:00 <eglynn> for obvious reasons ...
21:15:30 <eglynn> at least some of the issue appears to be caused by a race with suspended instances
21:15:33 <dkranz> eglynn: Hi. ping me again if you need me.
21:15:40 <eglynn> instance is suspended
21:16:07 <eglynn> then compute agent pollster fails with a libvirt error
21:16:51 <eglynn> so basically the compute agent needs to made tolerant of these instance-state-dependent errors
21:16:59 <eglynn> I've captured some ideas in ...
21:17:10 <eglynn> #link https://bugs.launchpad.net/ceilometer/+bug/1257302
21:17:11 <uvirtbot> Launchpad bug 1257302 in ceilometer "compute pollsters should tolerate libvirt errors reported when inspecting suspended instances" [Undecided,Triaged]
21:17:27 <eglynn> I'm gonna take that bug
21:17:48 <eglynn> but in general, we really need to be more selective about log.error
21:18:14 <eglynn> sdague sent a good mail about this to the ML yesterday
21:18:36 <eglynn> basically ERROR == "someone is woken up at 3am"
21:19:10 <eglynn> so lets start using ERROR severity a bit more sparingly. agreed?
21:19:25 <herndon_> ack!
21:19:33 <DanD> seems like it should only be a unrecoverable error
21:19:43 <eglynn> DanD: exactly!
21:19:44 <llu-laptop> agreed
21:19:45 <lsmola> +1
21:19:47 <dragondm> +1
21:19:52 <ildikov> +1
21:20:27 <eglynn> cool :)
21:20:29 <eglynn> ok, moving onwards ...
21:20:39 <eglynn> #topic Help needed for fixing MySQL errors in unit tests
21:20:51 <eglynn> so this is interesting
21:21:42 <eglynn> thing is ... testing against sqlite is a little, well, non-challenging for our sqlalchemy driver
21:21:51 <dragondm> Heh.
21:22:11 <eglynn> sqlite doesn't do a bunch of stuff like foreign key constraints, unique constraints etc
21:22:23 <eglynn> just looking at the migration scripts
21:22:40 <eglynn> you'll see a bunch of special casing for sqlite
21:23:07 <herndon_> Can we just ditch SQLite for good? no migrations, no support, nothing?
21:23:07 <eglynn> generally smoothening the happy path and leaving potential issues to lurk
21:23:14 <terriyu> do people actually use sqlite in real production environments?
21:23:20 <eglynn> herndon_: I hear ya!
21:23:22 <dragondm> I certainly hope not.
21:23:33 <eglynn> terriyu: no, yoyu'd be shot for doing that ;)
21:23:44 <terriyu> sorry for the newb question, guys :)
21:23:46 <dragondm> sqlite doesn't really do migrations altogether.
21:24:12 <eglynn> dragondm: yep, so much of the migration logic is skipped, it's barely worth talking about
21:24:54 <eglynn> problem is then nasty mysql-specific or postgres-specific bugs only rare their ugly heads in prod :(
21:25:00 <dragondm> And even what it does is really a hack in sqlalchemy migrate to simulate schema change, etc. We switch to alembic, and that will go away too.
21:25:07 <eglynn> (or in tempest if we're lucky!)
21:25:14 <herndon_> I thought the plan was to use mysql for unit tests, then we get rid of sqlite. Is that wrong?
21:25:27 <eglynn> herndon_: exactly!
21:25:36 <dragondm> eglynn: yah. Even mysql lets stuff slide that only postgres, or other bigger db's will catch.
21:25:45 <eglynn> herndon_: so jd__ has porposed a patch to do so ...
21:25:55 <dragondm> (i.e. like non-aggregate columns in an aggregate query)
21:26:02 <eglynn> but the feeling is that we'll have a deluge of bugs
21:26:18 <herndon_> I think I saw that, but it had a lot of test failures... anyway, I would love to help out in that effort.
21:26:44 <eglynn> so the motivation for raising this in the meeting agenda to elicit support for fix that inevitable deluge of bugs
21:26:54 <eglynn> herndon_: great!
21:26:56 <DanD> so we agreed that the gate would be on MySQL only and other qualifications would need to be manually run didn't we?
21:27:12 <eglynn> DanD: that we did
21:27:31 <eglynn> but gate == "unit tests" or gate == "tempest tests"
21:27:33 <dragondm> Yah, the bugs are in there already, we'll just be finding them. I'm all for that.
21:27:57 <herndon_> maybe we need a bug hunt day?
21:28:08 <eglynn> herndon_: a capital idea!
21:28:27 <eglynn> a DB bug squashing day
21:28:58 <eglynn> we have peeps in our community with a specific slat towards DB/migration logic
21:29:19 <DanD> so is the plan to remove sqllite dependencies and change default to MySQL, then fix the resulting fallout?
21:29:31 <eglynn> it would be great to leverage this in a formal bug squashing day
21:30:03 <eglynn> DanD: yep, maybe keeping sqlite as a real lightweight option
21:30:21 <eglynn> #action eglynn organize DB/migration bug squash day
21:30:32 <eglynn> (before i-2 goes out if poss)
21:31:05 <dragondm> sqlite would be usefull for tests above the db layer if we got the changes in to create the schema from the model layer, rather than migrations...
21:31:19 <dragondm> Other than that... Bleh.
21:31:26 <eglynn> dragondm: true that
21:31:43 <herndon_> yeah, what's the status of that by the way?
21:32:05 <dragondm> ^ indeed, how is that?
21:32:46 <eglynn> that I don't know, IIRC glance did it that way at one point
21:32:59 * eglynn will have to research that ...
21:33:16 <eglynn> may be some prior art we can steal
21:33:26 <eglynn> s/steal/pay homage to/ ;)
21:33:33 <eglynn> right-o, moving on with the agenda ...
21:33:44 <eglynn> #topic Release python-ceilometerclient?
21:33:48 <eglynn> no need
21:33:58 <eglynn> not only have I done it once
21:34:12 <eglynn> ... I did it twice this week
21:34:22 <eglynn> because of this regression ...
21:34:56 <eglynn> #link https://bugs.launchpad.net/oslo/+bug/1256883
21:34:59 <uvirtbot> Launchpad bug 1256883 in oslo "cliutils passing sortby=None breaks older versions of prettytable" [Medium,Fix committed]
21:35:12 <eglynn> so 1.0.8 is now out with the latest & greatest
21:35:14 <thomasem> o/
21:35:37 <eglynn> I suspect next release will be herndon_ 's event API support
21:35:59 <eglynn> and possibly group-by support too, perhaps?
21:36:13 <eglynn> ... anyway that's fairly forward-looking
21:36:42 <eglynn> k, let's move on ...
21:36:44 <eglynn> #topic Open discussion
21:37:22 <lsmola> llu-laptop, how is the hardware to central agent integration going?
21:37:55 <llu-laptop> https://blueprints.launchpad.net/ceilometer/+spec/support-resources-pipeline-item, still reviewing the patches
21:38:05 <sandy__> and how's the oslo.messaging integration going?
21:38:14 <sandy__> (I forget who was working on it)
21:38:24 <herndon_> i think it was silhet
21:38:32 <eglynn> sandy__: it's sileht
21:38:35 <eglynn> snap ;)
21:38:50 <terriyu> I think sileht is away today
21:38:52 <dragondm> Still wip, last I saw that...
21:39:08 <eglynn> sandy__: Mehdi pushed a WIP patch for review
21:39:11 <lsmola> llu-laptop, excellent
21:39:21 <llu-laptop> lsmola: after those patches landed, I'll rebase of https://blueprints.launchpad.net/ceilometer/+spec/monitoring-physical-devices on top of that
21:39:25 <sandy__> cool, thanks ... will look
21:39:29 <dragondm> sandy__: https://review.openstack.org/#/c/57457/
21:39:45 <lsmola> llu-laptop, great, looking forward to that
21:41:19 <eglynn> ... k, methinks we may be done here tonight
21:41:24 <eglynn> folks thank you for your time and attention!
21:41:47 * eglynn is on a hair-trigger with hash-tag endmeeting ;)
21:41:58 <lsmola> thanks guys, good night
21:42:01 <eglynn> ... anything else before we close?
21:42:11 <sandy__> night!
21:42:18 <thomasem> HAve a great one!
21:42:20 <sandy__> thanks eglynn
21:42:27 <terriyu> eglynn: nice of you to give a warning :)
21:42:32 <dragondm> heh.
21:42:33 <lsmola> :-)
21:42:39 <terriyu> nope, nothing from me
21:42:57 <eglynn> thanks folks, efficient & productive as always! :)
21:43:03 <ildikov> thanks guys and eglynn, good night
21:43:08 <ildikov> :)
21:43:08 <eglynn> see y'all on IRC tmrw :)
21:43:15 <eglynn> #endmeeting
21:43:17 <terriyu> on #openstack-ceilometer !
21:43:30 <ildikov> terriyu: +1
21:43:31 <eglynn> terriyu: well said!
21:43:42 <terriyu> thanks everyone
21:43:43 <dragondm> Yup, more pple need to switch channels
21:43:55 <eglynn> terriyu: that should have been an agenda item to be fair ...
21:44:04 <terriyu> eglynn: +1
21:44:47 <eglynn> let's all do the switch, as our illustrious PTL is only to be found on #openstack-ceilometer these days ...
21:45:10 <dragondm> eglynn: btw, I don't think the bot noticed your #endmeeting...
21:45:19 <llu-laptop> hello autobot, where's the log?
21:45:27 <eglynn> #endmeeting ceilometer
21:45:40 <dragondm> eglynn changed his nick.... the bot isn't recognizing him.
21:45:55 <eglynn-afk> #endmeeting