15:01:02 #startmeeting ceilometer 15:01:03 Meeting started Thu Feb 19 15:01:02 2015 UTC and is due to finish in 60 minutes. The chair is eglynn-office. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:05 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:07 The meeting name has been set to 'ceilometer' 15:01:20 o/ 15:01:24 o/ 15:01:40 o/ 15:01:41 <_elena_> o/ 15:01:42 o/ 15:02:05 o/ 15:02:31 #topic kilo-3 status 15:02:37 #link https://launchpad.net/ceilometer/+milestone/kilo-3 15:03:07 eglynn-office: we really need to move this forward, otherwise the API and Agents work will be late: https://review.openstack.org/#/c/142592/ 15:03:22 o/ 15:03:49 fabiog: as far as I can tell it is moving forward as fast as it can, given that it is far more complicated than initially imagined 15:04:04 fabiog: conf-datastore-agents is currently "Not Started" ... is that accurate? 15:04:05 eglynn-office: the disk meters patch is merged already 15:04:20 fabiog: does work on the BPs need to be strictly serialized? 15:04:34 eglynn-office: no it is started, it is only waiting for the API changes to hook into it 15:05:02 fabiog: OK, "slow progress" or "good progress" be more accurate? 15:05:02 eglynn-office: here it is https://review.openstack.org/#/c/150642/ 15:05:34 eglynn-office: I think is a good progress. Is it 90% done it only needs the v2 client stubs to connect into 15:05:54 fabiog: I'll jump in on both those reviews ... but as cdent said above, complex code can take a while to shephard over the line 15:06:04 fabiog: coolness, I'll update the status now 15:06:14 eglynn-office: thanks a lot 15:07:03 other than, I'm most concerned with instance-autorestart 15:07:14 since this was already bumped k1->k2->k3 15:07:33 I haven't heard from Einav, so I suspect at this stage it won't make it for kilo 15:08:29 just a quick reminder of the date ... kilo-3 is Mar 19th, feature proposal freeze is Mat 5th 15:08:33 eglynn-office: I'm not even sure we really need that one... 15:08:33 *Mar 5th 15:09:02 ildikov: yep, it seems to have been partially based on a mis-understanding of how things currebtly work 15:09:30 anything else on k3? 15:09:48 eglynn-office: and additionally it's an example for the non-metric meters that we try to get rid of with gordc now 15:10:00 just a note to say that some tests that don't work with HASHSEED random have leaked back into the test suite 15:10:05 eglynn-office: I think iops bp is implemented too 15:10:28 but there's as yet no progress on the wsmeext-based problem that is holding up that bug 15:10:50 ildikov: now marked as such, thanks 15:10:59 (I had it fail locally for me recently but then when I tried to trace it with instrumentation I could get a failure again with a few hundred tries) 15:11:04 couldn't 15:11:24 eglynn-office: just like disk meters 15:11:38 cdent: darned Heisenbug! :( 15:12:41 eglynn-office: tnx :) 15:13:31 ityaptin: are we expecting more regex support in different drivers for regexp-in-complex-query ? 15:13:42 ildikov: ... landed for sql-a and mongo IIRC, but not hbase? 15:13:52 ityaptin: ^^^ 15:14:00 ildikov: sorry, bad tab completion 15:14:06 eglynn-office: I remember those two for sure 15:14:29 eglynn: currently, only hbase is overboard . 15:14:48 k, let's leave that one open for now until Ilya can clarify if hbase will be included also 15:14:55 eglynn: db2 is supported via pymongo_base 15:15:33 ityaptin: overboard == will be done for kilo-3? 15:15:40 ityaptin: cool on db2 15:16:29 eglynn-office: this CR is blocker https://review.openstack.org/#/c/86683/ for complex query on hbase now 15:16:54 * cdent sighs 15:17:08 eglynn-office: because implementing mocked complex query for hbase, it is redundant work and waste of time 15:17:31 ityaptin: agreed 15:17:33 ityaptin: cool, got it, thanks! 15:17:53 k, moving on 15:18:00 eglynn-office: Or, for example, we can remove hbase mock and test hbase manually like postgres and others. 15:18:36 ityaptin: I just wanted to ask, what if would do this 15:18:39 eglynn-office while waiting for the patch will be merged 15:18:58 ityaptin: problem is can we rely on that ever being run manually? 15:19:19 eglynn-office: yep 15:19:59 eglynn-office: it's not the only backend we don't have on the gate 15:20:23 mysql/psql versus sqlite is one thing, but no automated CI coverage at all for hbase would be a concern 15:20:30 ildikov: are you thinking db2? 15:21:29 eglynn-office: yeap 15:22:21 ildikov, eglynn-office: Big part of db2 functional is covered by mongodb, but I have already looked patches which add uncovered functional 15:22:34 eglynn-office: and elasticsearch too IIRC 15:23:05 ityaptin: that's only partial coverage at best, amiright? 15:23:26 ityaptin: db2 is very similar to mongodb, but testing on mongo instead a real db2 is still just a workaround 15:23:41 instead *of* 15:24:10 ildikov: the intent was to include elasticsearch in the gate, but gordc ran into issues with ES not being in the blessed repos for trusty & fedora 15:24:31 eglynn-office: I remember the story 15:25:06 IMO ultimately we need all storage drivers either in the upstream gate or in 3rd party CI 15:25:06 eglynn-office: but all these things are not relevant for does it worth to have a mocked hbase in-tree, right? 15:25:20 ... or moved out-of-tree so no longer our problem 15:25:24 eglynn-office: that part I support 15:26:17 ildikov: the question is mocked habse versus someone taking it upon themselves to manually run the hbase tests periodically 15:26:29 * cdent perks ears at "moved out-of-tree" 15:26:37 ildikov: maybe the question should be ... mocked hbase versus third party CI 15:27:04 eglynn-office: third party CI 15:27:41 cdent: one idea would be to issue an ultimatum ... "who's interested in DB2 support? if you want it to stay in-tree, please provide 3rd party CI for it" 15:27:45 eglynn-office: TBH it does not make much sense to me to test against a mocked db, however all the against real db tests should go under a functional test folder 15:28:11 eglynn-office: but this would be a separate meeting topic 15:28:57 ildikov: yes and yes ... problem is that the "against real db tests" for hbase would have to skipped upstream currently 15:29:04 eglynn-office: so as a sum I support the third party CI and even the out-of-tree ideas, I don't see much point in maintaining big mocks 15:29:10 (IIRC hbase can't be installed on the upstream CI nodes) 15:29:25 ildikov: agreed, and I think that's ityaptin's feeling also 15:29:45 (on the not maintaining big mocks) 15:29:50 eglynn-office: sure, but we will not solve this problem right now, so let's see what we can do with the options that were mentioned here 15:30:24 eglynn-office: I mean starting it after the meeting :) 15:30:49 ityaptin: could Mirantis potentially provide 3rd party CI for hbase? 15:31:08 ityaptin: .... just one to think about, you don't need to answer yes/no right now 15:31:48 eglynn-office: I make consultation with Dina and others. 15:32:05 ityaptin: excellent, thank you! 15:32:07 right, let's move on to asyncio 15:32:14 #topic Use of ceilometer for PoC of asyncio/trollius adoption 15:32:23 #link https://review.openstack.org/153298 15:32:37 eglynn-office: what is the timeframe of this PoC? 15:32:51 ildikov: Liberty 15:33:08 eglynn-office: are we the only candidate for this? 15:33:13 for context, cast your minds back to the midcycle in Paris last July 15:33:35 this was discussed esp. with the olso-messaging guys on the fringes of that midcycle 15:34:07 ildikov: both ceilo and oslo-messaging would the initial candidates for PoC 15:34:48 eglynn-office: I was just wondering if it will be too much for Ceilo or not, but I will need to check the spec more 15:35:29 haypo has shown a lot of tenacity in keeping this discussion current for many months, despite oppostion 15:35:36 I would prefer to see a more narrow focus than the entire project 15:35:50 perhaps just the notification agent or something like that 15:36:12 and for sake of working around some of the debates, it should not involve any database access 15:36:17 eglynn-office: with the Gnocchi integration and all those feedback mails, I'm worried a bit TBH 15:36:30 ildikov++ 15:36:37 ++ 15:37:02 cdent: doesn't the notification agent directly access the DB also? 15:37:10 cdent: ... i.e. for record_event_data 15:37:19 eglynn-office: so personally I would prefer not to the candidates or with a really narrow scope that cdent mentioned 15:37:28 I thought record_event_data was rpc? 15:37:38 (with gordcs fixes) 15:37:45 eglynn-office: as much as I can remember Gordon fixed that 15:37:49 cdent: a-ha, yeah 15:38:10 so the collector and api are the database interactors 15:38:28 (until/if the pipeline stuff gets integrated) 15:39:33 cdent: k, so perhaps a smaller scope might be workable to avoid DB access ... but IIRC the notification agent now uses tooz for group coordination? 15:39:51 * eglynn-office is thinking tooz usage == redis driver in production 15:40:50 eglynn-office: until when should we decide this? 15:40:56 seems that would require a change to tooz to use the asyncio-aware redis client? 15:41:08 cdent: the pipeline stuff wiil use the API to store/read so it won't change 15:41:18 It's not so much about avoiding access to other services, but that Mike Bayer has demonstrated pretty consclusively that there's bigger fish to fry in the database handling world, and wants to fix it in oslo, so trying to leave that stuff out of scope. 15:41:33 cool, thanks fabiog 15:41:45 cdent: ok, fair point 15:42:18 ildikov: there isn't a hard deadline, as that spec is not likely to land in the very near future 15:42:41 asyncio + coordination doesn't really map well in my limited brain. 15:42:42 ildikov: ... however it would be good to reach a team consensus and express that in the gerrit review 15:43:15 * cdent will drop some comments on the review 15:43:25 cdent: thanks 15:43:33 eglynn-office: we do not really have many cores around now, so I would suggest to revisit this question 15:43:47 ildikov: yeap 15:44:18 eglynn-office: I'm still worried, maybe after checking the spec and book of review comments I will be less worried 15:44:58 sounds like folks on today are concerned by the risks/impact involved in a widely scoped changeover to asyncio 15:45:04 but I suspect more support would be forthcoming from a couple of the cores who not on the meeting today 15:45:05 ildikov: it's unfortunate but I think we need to keep a fair balance between making stuff work and generating hype and engagement from outside the project 15:45:24 and something like asyncio will generate some hype 15:45:32 which is lame, but life 15:45:39 cdent: true and true 15:46:26 cdent: but we need positive operator feedback mmuch more than happy fellow projects around IMHO, and being lab rats does not seem to give this to us 15:47:10 ildikov: I mostly agree, but on the other hand it seems most operators have given up on us 15:47:14 cdent: but you can count this fear on my female genes ;) 15:47:15 ildikov: ++ 15:47:32 cdent: we haven 15:47:34 t 15:47:49 I think most of the operators are still stuck on database issues related to performance 15:47:50 that's because you're a dear sweet friend linuxhermit ;) 15:47:52 at least we are 15:47:55 linuxhermit: that's good to hear 15:47:59 cdent: if that would be true then everyone would need to look for another project right this moment 15:48:35 mmester: it's a problem but we've got work arounds 15:49:04 linuxhermit: I'd love to hear more about your production setup 15:49:12 cdent: I'm just saying that we need to start making the customers happy in order to not starve to death, that's life too 15:49:30 (within the bounds of what can be publicly disclosed of course) 15:49:37 +1 15:49:52 cdent: but I don't want to argue about this, but someone has to list the concerns too in order to make a reasonable decision 15:50:08 let me find out how much I can share 15:50:19 linuxhermit: excellent, thank you! 15:50:26 but we = Cisco 15:50:38 :) 15:50:43 No need to argue about ildikov, I fundamentally agree with you. 15:50:44 linuxhermit: thanks in advance for the feedback even if that well be pretty short 15:50:45 coolness :) 15:50:51 so yeah... we have many different things between Cisco Cloud Services, Metacloud, and On premise 15:51:04 linuxhermit me = cisco too 15:51:16 cdent: cool :) 15:51:20 mmester: which part 15:51:27 OSS 15:51:35 CCS 15:52:26 #topic gnocchi status 15:52:31 I think most of the folks directly involved are not on today, so let's punt this one 15:52:41 #topic open-discussion 15:53:04 anyone got conference sessions they'd like to promote here? 15:53:15 * cdent is very happy with the very quick progress on improving the postgresql testing situation 15:53:15 vote solicitation is part of the game 15:53:21 me: 15:53:44 https://www.openstack.org/vote-vancouver/presentation/apis-matter 15:54:30 nice! 15:54:42 cdent: good one :) 15:55:09 https://www.openstack.org/vote-vancouver/presentation/stabilizing-the-jenga-tower-scaling-out-ceilometer 15:55:31 it's not me 15:55:36 https://www.openstack.org/vote-vancouver/presentation/the-anatomy-of-an-action 15:55:41 but it looks awesome and one of the cisco people is going it 15:55:54 linuxhermit: and me! :) 15:55:56 wow, gordc waring a suit and tie?!! 15:56:01 and you! 15:56:04 lol 15:56:07 ... say it ain't so! :) 15:56:07 https://www.openstack.org/vote-vancouver/presentation/ceilometer-and-the-oss-experience 15:56:07 brother's wedding. 15:56:16 i'm a suit 15:56:23 LOL :) 15:56:23 ha 15:57:03 coolness, I'd encourage everyone to vote early & often for these proposals 15:57:27 ... even if the voting isn't the only determinant of success 15:58:05 * gordc reading logs 15:58:10 (which I disagree with TBH, should be either a pure vote or a track chair decision, not a weird mixture of both) 15:58:10 i assume all is well 15:58:37 i pretty sure the vote button isn't connected to anything. 15:58:38 right, shotclock is against us folks 15:59:04 gordc: you cynic, you! 15:59:11 :) 15:59:16 lol 15:59:29 k, thanks for you time folks, let's call it a wrap 15:59:35 cya all thank you 15:59:36 laters! 15:59:38 #endmeeting ceilometer