Friday, 2014-03-28

*** shardy has quit IRC00:02
*** shardy has joined #openstack-ceilometer00:03
openstackgerritgordon chung proposed a change to openstack/ceilometer: DBDeadlock exception in sql backend  https://review.openstack.org/8046100:13
*** _nadya_ has joined #openstack-ceilometer00:17
*** _nadya_ has quit IRC00:22
*** matsuhashi has joined #openstack-ceilometer00:31
*** _cjones_ has quit IRC01:26
*** _cjones_ has joined #openstack-ceilometer01:26
*** _cjones_ has quit IRC01:30
*** nosnos has joined #openstack-ceilometer01:37
*** alexpilotti has joined #openstack-ceilometer01:42
openstackgerritgordon chung proposed a change to openstack/ceilometer: DBDeadlock exception in sql backend  https://review.openstack.org/8046101:48
*** LuanNH has joined #openstack-ceilometer02:45
*** matsuhashi has quit IRC03:04
*** flwang has quit IRC03:06
*** matsuhashi has joined #openstack-ceilometer03:10
*** nosnos has quit IRC03:17
*** changbl has joined #openstack-ceilometer03:20
openstackgerritgordon chung proposed a change to openstack/ceilometer: DBDeadlock exception in sql backend  https://review.openstack.org/8046103:20
*** matsuhashi has quit IRC03:31
*** flwang has joined #openstack-ceilometer03:39
*** alexpilotti has quit IRC03:46
*** matsuhashi has joined #openstack-ceilometer03:53
*** jaycromer has joined #openstack-ceilometer03:55
jaycromerhello all03:58
*** jaycromer has quit IRC04:02
*** nosnos has joined #openstack-ceilometer04:06
*** liusheng has quit IRC04:09
*** flwang has quit IRC04:30
*** flwang has joined #openstack-ceilometer04:31
openstackgerritgordon chung proposed a change to openstack/ceilometer: DBDeadlock exception in sql backend  https://review.openstack.org/8046104:38
openstackgerritgordon chung proposed a change to openstack/ceilometer: remove dump tables from previous migrations  https://review.openstack.org/8365704:41
*** _cjones_ has joined #openstack-ceilometer04:46
*** nati_ueno has quit IRC05:09
*** Ruetobas has quit IRC05:22
*** Ruetobas has joined #openstack-ceilometer05:28
*** Ruetobas has quit IRC05:33
*** Ruetobas has joined #openstack-ceilometer05:33
*** drjones has joined #openstack-ceilometer05:57
*** drjones has quit IRC05:57
*** drjones has joined #openstack-ceilometer05:58
*** _cjones_ has quit IRC06:01
*** drjones has quit IRC06:02
openstackgerritJenkins proposed a change to openstack/ceilometer: Imported Translations from Transifex  https://review.openstack.org/7676206:05
*** _nadya_ has joined #openstack-ceilometer06:07
*** _nadya_ has quit IRC06:17
*** _nadya_ has joined #openstack-ceilometer06:19
*** ildikov_ has quit IRC06:27
*** flwang has quit IRC06:35
*** flwang has joined #openstack-ceilometer06:37
*** saju_m has joined #openstack-ceilometer06:38
*** flwang has quit IRC06:41
*** flwang has joined #openstack-ceilometer06:43
*** liusheng has joined #openstack-ceilometer06:45
openstackgerritliusheng proposed a change to openstack/ceilometer: Fix the floatingip pollster  https://review.openstack.org/8367607:01
*** flwang has quit IRC07:02
*** _nadya_ has quit IRC07:03
*** mihgen has joined #openstack-ceilometer07:09
*** matsuhashi has quit IRC07:20
*** matsuhas_ has joined #openstack-ceilometer07:24
*** urulama has joined #openstack-ceilometer07:33
*** eglynn has joined #openstack-ceilometer07:34
*** ildikov_ has joined #openstack-ceilometer07:49
*** mihgen has quit IRC07:53
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: De-dupe selectable aggregate list in statistics API  https://review.openstack.org/8351108:27
*** eglynn has quit IRC08:27
*** mihgen has joined #openstack-ceilometer08:33
*** matsuhas_ has quit IRC08:53
*** matsuhas_ has joined #openstack-ceilometer08:57
*** inc0 has joined #openstack-ceilometer08:59
inc0good morning08:59
*** ildikov_ has quit IRC09:03
*** saju_m has quit IRC09:05
*** eglynn has joined #openstack-ceilometer09:12
*** nacim has joined #openstack-ceilometer09:12
*** yassine has joined #openstack-ceilometer09:13
*** vrovachev has joined #openstack-ceilometer09:17
eglynnjd__: hope it wasn't too cavalier of me, going ahead and targetting this late-breaking bug at RC1?09:19
eglynnhttps://bugs.launchpad.net/ceilometer/+bug/129851409:19
eglynn(... should be land-able today, depending on what o'clock ttx plans to cut RC1)09:19
*** saju_m has joined #openstack-ceilometer09:25
*** ildikov_ has joined #openstack-ceilometer09:33
*** bada has joined #openstack-ceilometer09:35
* jd__ enters review mode09:36
*** sayalilunkad has joined #openstack-ceilometer09:50
*** matsuhas_ has quit IRC10:00
nprivalovaeglynn, jd__, looks like we have working HBase with several threads :) but I need monkey patching for threading10:00
eglynnnprivalova: cool, monkey patching due to eventlet issues, or?10:01
jd__so that's not several threads then :)10:01
jd__that's greenthreads10:01
nprivalovaeglynn: yes10:01
eglynnnprivalova: ... so not true concurrency then as jd__ says10:02
nprivalovajd__: sorry, I used to avoid any threading all my life :D10:02
jd__nprivalova: which is a good idea10:02
jd__says the guy spending his days debugging the multithreading nightmare of kazoo in tooz10:03
jd__true story10:03
nprivalovajd__, eglynn, ok, I would say in another way. By default HBase doesn't work now with "simultaneous file read". I will upload new variant for review that works and you will fix my monkey-patching if needed, ok?10:05
jd__ok nprivalova10:05
jd__nprivalova: is that for rc1 or?10:05
nprivalovajd__: it would be perfect10:06
inc0eglynn, tell me please, do we plan to implement aggregates with more than one param?10:12
eglynninc0: I thought about that originally10:13
eglynninc0: ... e.g. aggregate.func=quantile&aggregate.param=0.25,0.5,0.7510:14
eglynninc0: ... but came to the conclusion that it would over complicate the API10:14
eglynninc0: ... and those cases could be handled instead via say:10:14
eglynnaggregate.func=quantile&aggregate.param=0.25&aggregate.func=quantile&aggregate.param=0.5&...10:14
eglynninc0: ... make sense?10:15
ildikov_eglynn, inc0: complex query for statistics is upcoming, we planned it for early Juno10:15
eglynnildikov_: will that impact on the aggregate selection?10:16
*** nosnos has quit IRC10:16
ildikov_eglynn, inc0: I think the structure of the request body will be capable of handling the multiple params situation10:17
eglynnildikov_: k, that's interesting10:17
eglynnildikov_: ... if we have a case that doesn't naturally decompose into multiple references to the same aggregate each with a single param10:18
eglynnildikov_: ... as does the quantile example above10:18
eglynnildikov_: ... or the multiple cardinalities we were discussing last night10:18
ildikov_eglynn: I think that it is the matter of design and needs how we handle the aggregates10:19
eglynnildikov_: yeah for the query-param version, prolly best to stick with single-param10:19
eglynnildikov_: ... but the JSON repr in the complex case obviously would give more flexibility10:20
ildikov_eglynn: I think quantile also could be handled, exactly because of the this flexibility10:20
*** alexpilotti has joined #openstack-ceilometer10:20
ildikov_eglynn: as I said, it should be the matter of design, but I think it is not a mission impossible case10:21
ildikov_eglynn: do you agree?10:21
eglynnildikov_: ... yep, if there's a definite usecase for multi-param10:21
*** sayali has joined #openstack-ceilometer10:22
openstackgerritNadya Privalova proposed a change to openstack/ceilometer: Use ConectionPool instead of one Connection in HBase  https://review.openstack.org/8343510:23
ildikov_eglynn: sure, we need to identify that what makes sense here, like you've mentioned the quantile or yesterday's case with the multiple cardinalities10:24
*** sayalilunkad has quit IRC10:25
openstackgerritNadya Privalova proposed a change to openstack/ceilometer: Use ConectionPool instead of one Connection in HBase  https://review.openstack.org/8343510:27
eglynnildikov_: ... yep, for those examples above10:28
eglynnildikov_: ... as an API user I'd be happy enough with either form10:29
eglynnildikov_: ... i.e. func=cardinality&param=resource_id&func=cardinality&param=project_id10:29
eglynnildikov_: ... or something like func=cardinality&param=resource_id,project_id10:29
ildikov_eglynn: sorry, I'm under mail mountains currently, so I just dropped here my first thoughts about this topic :)10:29
eglynnildikov_: ... but then again for somthing like say exponential smoothing10:29
eglynnildikov_: ... might make sense to have say: func=holt_trend&param=alpha,beta10:30
eglynnildikov_: ... so yeah, multi-param could definitely make more sense the deeper we get into this10:30
ildikov_eglynn: complex query started because of making the requests more flexible10:31
ildikov_eglynn: query *was* started10:31
eglynnildikov_: true that, and now that I think on it more, that flexibility could definitely be useful for https://blueprints.launchpad.net/ceilometer/+spec/period-spanning-statistics10:32
ildikov_eglynn: hmm, looks interesting :)10:34
ildikov_eglyyn: as for pure multiple params support, I think it makes sense, these things are usually come into the picture as time goes by and a given feature is used more and more10:36
ildikov_eglynn: flexibility is important I think10:36
eglynnildikov_: +1 :)10:37
ildikov_eglynn: I have to think about a little bit more on this bp you've just sent :)10:37
eglynnildikov_: ... btw I'm hoping that BP will be taken on by an OPW intern if her application is successful10:38
ildikov_eglynn: I said earlier, when we were implementing complex query that we should discuss about statistics too, because of the flexibility it provides could be used there too for multiple purposes10:38
eglynnsounds good :)10:38
ildikov_eglynn: so if it is ok with you, then we will get back to you with gibi, if we reach again the feature implementation phase and have some design like on the table10:39
eglynnildikov_: cool10:39
ildikov_eglynn: hm, nice bp for an intern :)10:40
ildikov_eglynn: ... I hope she will like it10:40
ildikov_eglynn: ... maybe I will let someone else reviewing that code, so that she will get a chance to get it landed once ;)10:41
eglynnildikov_: anamalagon (whom you know) is interested in the BP all right, but the OPW application process has to play out first10:41
eglynnildikov_: LOL ;)10:41
*** LuanNH has quit IRC10:43
ildikov_eglynn: what is the schedule of the OPW process?10:45
eglynnildikov_: summarized here ... https://wiki.openstack.org/wiki/OutreachProgramForWomen#Schedule10:46
ildikov_eglynn: cool, thanks10:48
nprivalovaeglynn: could you please take a look https://review.openstack.org/#/c/83435/3/ceilometer/cli.py ? is it acceptable?10:59
eglynnnprivalova: looking10:59
inc0sorry, I had ad-hoc meeting the moment I started topic;) Well, imho multi-param aggregates are just matter of time. We'll need it eventually. Maybe its worth creating a bp?11:08
inc0also I think constructions like aggregate.func=cardinality&aggregate.param.field=resource_id , so named params would also be worth thinking about11:11
ildikov_inc0: from complex query PoV, there is already a BP for statistics support, it only needs to be updated according to the new statistics functionality11:11
ildikov_inc0: if you plan to improve the current statistics endpoint, it's eglynn's area :)11:11
*** inc0_ has joined #openstack-ceilometer11:13
eglynninc0: ... yeah from the discussion above, I'm tending to agree that multi-param has a real use-case11:13
eglynninc0_: ... yeah from the discussion above, I'm tending to agree that multi-param has a real use-case11:13
eglynninc0, inc0_: will the real inc0 please stand up? ;)11:14
eglynn... and yeah a BP would be good if you're interested11:14
inc0_here:)11:15
inc0_network is flapping :(11:15
ildikov_eglynn, inc0_: I think in case of the current statistics functionality, it is important to keep the API somehow as simple as possible11:16
*** inc0 has quit IRC11:16
ildikov_eglynn, inc0_: I meant the named params here, but maybe I'm just a girl again and worrying too much as usual :)11:16
inc0_ildikov_, true, but again, in my opinion sooner or later someone would like to have multiparam aggregate11:17
inc0_ildikov_, its rather about "Explicit is better than implicit" from import this, but I tend to be zealotus;) for good or ill11:17
eglynninc0_: I guess the question is whether unamed multi-param would suffice, as opposed to kwarg-stylee?11:18
ildikov_inc0_: I know, that is why I mentioned the complex query related endpoints, where I have already a statistics bp, if it is only about having support somewhere11:18
eglynninc0_: ... anyway sounds like a detail that could be trashed out in the process of BP drafting and gerrit reviewing11:18
inc0_eglynn, possibly, but quite frankly I don't like that order in which you append GET params (func and params for aggregates) should matter, and now they do.11:19
ildikov_inc0_: if you plan to extend the current functionality, I'm not against the named params, I just said that it should be considered that which direction would be the best to not make the API too complex11:19
eglynninc0_: darned WSME! ;)11:19
ildikov_inc0_: WSME is not a nice "animal" in this "zoo" and that is all its fault...11:20
inc0_well, there was case with one particular giraffe in Denmark...11:21
*** julim has joined #openstack-ceilometer11:21
eglynn... poor ol' Marius11:22
inc0_the lion was happy thou11:22
*** julim has quit IRC11:22
eglynn... until some of his old friends got the bullet too11:23
ildikov_eglynn, inc0_: oh, Ive just found the wikipedia page about him :(11:23
inc0_I'm looking at your blueprint ildikov_, making queries using POST doesn't seem to be restful...maybe simply url-encoded json?11:25
inc0_in GET11:25
ildikov_eglynn, inc0_: I cannot clearly see the situation of our Marius, there were some initial missions to execute, but I cannot see the long term future in my crystal ball now11:26
ildikov_inc0_: I know, we had some bloody discussion about this earlier11:26
inc0_that approach would also help with aggregates = you could keep exactly the same datastructure in ceiloclient and api11:27
inc0_aggregates=json.dumps([{'func': 'cardinality', 'param': 'resource_id'}])11:27
ildikov_inc0_: finally this solution got landed as there were also discussions about that it should be an extension of the current simple query and that is this syntax even good or bad, etc11:28
ildikov_inc0_: it is not easy to have a common agreement in a large community like this, I'm not saying that it is a perfect solution, but you cannot suggest anything that will be pure black or white for everyone11:30
inc0_ildikov_, well, thats the problem of most projects I am or have been into;) with more than 2 people in it11:30
ildikov_inc0_: the statistics bp is the second part of the one for complex query support, so the design was given in this case11:30
inc0_especially in Poland.11:30
*** julim has joined #openstack-ceilometer11:31
ildikov_inc0_: I know the feeling exactly :)11:32
ildikov_inc0_: sorry, I need to run now to a short meeting, it's an extremely busy Friday :(11:33
inc0_ildikov_, have a nice meeting11:33
ildikov_inc0_: it will be enough for me, if it will not be ugly :)11:34
ildikov_inc0_: laters11:34
inc0_see ya11:35
*** saju_m has quit IRC11:50
openstackgerritIlya Tyaptin proposed a change to openstack/ceilometer: Add encoding argument to deserialising udp packets in collector  https://review.openstack.org/8372211:51
*** Alexei_987 has joined #openstack-ceilometer11:55
*** urulama has quit IRC11:56
nprivalovajd__, sileht, please take a look https://review.openstack.org/#/c/83435/3/ceilometer/cli.py12:03
*** gordc has joined #openstack-ceilometer12:16
openstackgerritgordon chung proposed a change to openstack/ceilometer: remove dump tables from previous migrations  https://review.openstack.org/8365712:18
openstackgerritNadya Privalova proposed a change to openstack/ceilometer: Use ConectionPool instead of one Connection in HBase  https://review.openstack.org/8343512:19
nprivalovagordc: hi! I really need your thought here https://review.openstack.org/#/c/83435/ . Finally I've made HBase work on real load, 1000 entries per minute. And we need this patch to say "Ceilometer supports HBase". If you have time please take a look12:23
gordcnprivalova: this is for juno right? (and backport)12:29
nprivalovagordc: I'm still dreaming about rc1 :)12:29
gordcnprivalova: ... i think we were going to cut that today...12:31
gordcnprivalova: i'll review it but i think you need to bump requirements as well. connectionpool is only available with >=0.5 but we're still accepting 0.412:32
nprivalovagordc: yep, I know... I want HBase in Icehouse. It's not really important how it will be there. backporting may be ok. I don't know this procedure good enough12:33
nprivalovagordc: I use connectionpool from happybase. I think it's different connection pool12:33
eglynngordc: re. RC1 timing, I was bugging ttx earlier for a definite deadline on this12:34
eglynn(... so that we know where we stand re. landing the last few patches)12:34
eglynn... if necessary he's willing to cut the ceilo tag on Monday to allow release-critical patches land12:35
eglynn... however we should still probably be aiming for EoD today if at all poss12:35
gordceglynn: cool cool. i like EoD deadline. give us sometime to make sure all the patches that did get in didn't break each other.12:37
gordcnprivalova: http://happybase.readthedocs.org/en/latest/api.html#happybase.ConnectionPool12:37
nprivalovagordc: don't get you. you are worried about happybase version?12:38
gordcnprivalova: yeah. iiuc, if someone uses happybase 0.4 (which requirements allow), your code will fail.12:39
*** claudiub has joined #openstack-ceilometer12:39
nprivalovagordc: ah, I see. We are changing that. We have a chance. Let me find an email12:41
nprivalovagordc: Dependency freeze exception for happybase (I would like version 0.8)12:41
nprivalovagordc: we may add 0.8 and remove 0.4 in one cr12:42
nprivalovagordc: actually I prefer  >=0.8... because all other are buggy12:43
gordcnprivalova: i see.. yeah, we'll need to depend on that.12:46
nprivalovagordc: writing email now12:46
openstackgerritSt├ęphane Albert proposed a change to openstack/python-ceilometerclient: Statistics groupby handling improvement  https://review.openstack.org/8025612:46
*** liusheng has quit IRC12:50
*** jdob has joined #openstack-ceilometer12:56
nprivalovaeglynn: could you please take a look https://review.openstack.org/#/c/78244/ once more? Do you have some more questions?13:00
eglynnnprivalova: ... I'm in meetings for the next 2 hours solid, but will look again at this promptly at 1500 UTC13:01
nprivalovaeglynn: ok, np13:01
*** zigo has quit IRC13:13
*** zigo has joined #openstack-ceilometer13:14
*** AMike has quit IRC13:15
openstackgerritA change was merged to openstack/ceilometer: improve performance of resource-list in sql  https://review.openstack.org/8034313:15
*** saju_m has joined #openstack-ceilometer13:15
*** thomasem has joined #openstack-ceilometer13:27
*** sayali has quit IRC13:27
openstackgerritgordon chung proposed a change to openstack/ceilometer: remove dump tables from previous migrations  https://review.openstack.org/8365713:38
*** alexpilotti has quit IRC13:42
*** flwang has joined #openstack-ceilometer13:42
*** prad has joined #openstack-ceilometer13:42
*** Wangpan has quit IRC13:42
*** Wangpan has joined #openstack-ceilometer13:43
*** zigo has quit IRC13:47
*** zigo has joined #openstack-ceilometer13:47
*** prad has quit IRC13:49
*** zigo has quit IRC13:51
*** prad has joined #openstack-ceilometer13:52
*** zigo has joined #openstack-ceilometer13:55
*** zigo has quit IRC13:59
*** zigo has joined #openstack-ceilometer13:59
*** nacim has quit IRC14:00
*** jmckind has joined #openstack-ceilometer14:00
*** nacim has joined #openstack-ceilometer14:01
*** Ruetobas has quit IRC14:04
*** Ruetobas has joined #openstack-ceilometer14:11
*** inc0_ has quit IRC14:16
*** Ruetobas has quit IRC14:16
*** openstack has joined #openstack-ceilometer14:20
*** Ruetobas has joined #openstack-ceilometer14:22
*** alexpilotti has joined #openstack-ceilometer14:27
openstackgerritA change was merged to openstack/ceilometer: Documenting hypervisor support for nova meters  https://review.openstack.org/7991014:29
openstackgerritgordon chung proposed a change to openstack/ceilometer: DBDeadlock exception in sql backend  https://review.openstack.org/8046114:33
openstackgerritNadya Privalova proposed a change to openstack/ceilometer: Use ConectionPool instead of one Connection in HBase  https://review.openstack.org/8343514:34
*** rwsu has quit IRC14:38
*** rwsu has joined #openstack-ceilometer14:41
eglynnnprivalova: see further question inline in https://review.openstack.org/#/c/78244/6/ceilometer/storage/impl_hbase.py15:06
nprivalovaeglynn: ok15:06
openstackgerritA change was merged to openstack/ceilometer: Remove escape character in string format  https://review.openstack.org/8333815:12
nprivalovaeglynn: I think that consistency problem still may be. But it's problem of data-model. I'm working on changing it.15:12
eglynnnprivalova: I'm not too concerned with the possible interleaving as I think this resolves naturally when the next sample is persisted15:15
eglynnnprivalova: however the TODO comment in the code and the reference to future changes is little vague15:16
nprivalovaeglynn: I think it should be a new bug instead of TODO comment :)15:17
eglynnnprivalova: ... how about just being super-specific in the comment, e.g.15:17
eglynn"# sample writes may be interleaved, but where slightly outdated resource metadata is persisted, this will naturally resolve when the next sample is encountered"15:18
eglynn... or words to that effect15:18
eglynnnprivalova: whereas "this method works ok only in single-thread environment" would tend to set alarm bells off ;)15:18
nprivalovaeglynn: yep, it should be changed somehow :)15:19
nprivalovagordc: are you here?15:19
openstackgerritgordon chung proposed a change to openstack/ceilometer: DO NOT MERGE  https://review.openstack.org/8378015:20
gordcnprivalova: yep15:21
nprivalovaeglynn:  it seems to me that Gordon doesn't agree with "resolves naturally when the next sample is persisted". I'm still trying to analyze it...15:21
nprivalovagordc: we are talking about https://review.openstack.org/#/c/78244/6/ceilometer/storage/impl_hbase.py15:21
jd__seems we like patches for rc115:21
jd__s/like/miss/15:21
eglynnjd__: ttx indicated willingness earlier to cut RC1 on Monday if neccessary to let release-critical patches land15:23
nprivalovajd__: https://review.openstack.org/#/c/83435/5 ? <cat from shrek>15:23
gordcjd__: we can probably skip https://bugs.launchpad.net/ceilometer/+bug/1259724. the patch is up but it's not important15:23
eglynnjd__: ... sounds we're gonna need that flexibility?15:23
gordcnprivalova: i think it may eventually resolve with future samples but it's not guaranteed to.15:24
*** kin has joined #openstack-ceilometer15:25
gordcnprivalova: personally i think it doesn't make sense to get it into rc1 if we know it's going to need a backport to fix it properly.15:25
jd__eglynn: well likely because it's getting late for ttx anyway15:25
jd__I don't think everything will be merged in the next couple of hours15:25
eglynngordc: not guaranteed to if another sample is never received for that resource?15:26
eglynnjd__: agreed15:26
nprivalovagordc: as I said to eglynn, I'm working on changing model15:26
jd__but Monday should be ok so hurry up! :)15:26
eglynnnprivalova: by changing the model, do you mean https://review.openstack.org/#/c/78244 is still a WIP? (i.e. not ready to land yet)15:27
gordceglynn: yeah, ie. if we lose meter and another sample for that meter never comes, it won't show in list of meters for that resource.15:27
gordceglynn: an edge case i guess.15:27
nprivalovaeglynn: no. new model is definitely a new change request15:28
eglynngordc: ... yeap fair point15:30
gordcnprivalova: you'll need a new model to get it working properly with multithread though right? the way i see it, the patches can't coexist together (without adding a disclaimer that it won't always work correctly)15:30
eglynngordc: ... though it might be seen as an acceptable edge case, seeing as the list of meters associated with a resources is kind of secondary info in a sense15:31
nprivalovagordc: 1. without Connection Pool Hbase doesn't work now15:31
eglynngordc: ... (as the same info can be reconstructed from the raw samples?)15:31
*** malini has joined #openstack-ceilometer15:32
eglynnnprivalova: "new model is definitely a new change request" => for RC1 or Juno?15:32
nprivalovaeglynn: Juno15:32
nprivalovaeglynn: now I just want to make it work :)15:32
eglynnnprivalova: got it15:33
gordceglynn: yep. that was my original suggestion to just use raw samples... right now it seems https://review.openstack.org/#/c/78244 will fix the bug and then connectionpool patch will cause a similar bug to happen again.15:33
eglynnnprivalova: so can we address gordc's concern that the two patches are mutually exclusive?15:33
eglynntwo patches ==  https://review.openstack.org/78244 & https://review.openstack.org/8343515:34
nprivalovaeglynn: let me try to explain again :)15:34
*** saju_m has quit IRC15:35
nprivalovaeglynn:  we need https://review.openstack.org/83435 in any case because HBase doesn't work without it15:35
eglynnnprivalova: so no 83435 => every falls apart with "Simultaneous file access"15:37
nprivalovaeglynn: yes15:37
*** ryanpetrello has left #openstack-ceilometer15:37
gordcnprivalova: just to clarify, you mean doesn't work with multiple collectors or doesn't work (full stop).15:37
nprivalovagordc: even if we start 1 collector but use rpc publisher record_metering_data starts to be "not_one_thread"15:38
gordcok. so doesn't work.15:38
nprivalovagordc: yep15:38
maliniHello!! I saw recent emails in the dev list about updating the Mongo version at the gate - but lost track of where it ended. Were you able to get Mongo updated?15:39
eglynnnprivalova: so it sounds like 83435 is absolutely crucial, without it we might as well say HBase isn't supported in Icehouse, correct?15:39
nprivalovaeglynn: yep15:40
eglynnnprivalova: ... in that case 83435 should be marked Critical?15:40
gordctbh, i'd rather have 83435 (after we get requirement bump)... and leave the other patch as a backport (since it'll need one anyways).15:40
nprivalovaeglynn: 78244 lives not very good with 83435. but it makes things better. without 78244 user will see more inconsistency15:41
gordcmalini: not sure what status is regarding mongo on gate... i would assume it's a dream for now.15:41
eglynnnprivalova: whereas the bug for https://review.openstack.org/78244 is far less serious15:42
eglynnnprivalova: ... so if we're gonna sacrifice one for the other?15:42
nprivalovaeglynn: yep, but I was afraid to mark bug as critical :)15:42
eglynn... https://review.openstack.org/78244 gets bumped15:42
malinigordc: do you have a workaround meanwhile? We have the same issue in Marconi & we are considering 3rd part testing15:42
eglynnnprivalova: and we mark https://bugs.launchpad.net/ceilometer/+bug/1298455 as critical and land that patch only?15:42
nprivalovaeglynn, gordc, I'm ok with delay for 7824415:43
nprivalovaeglynn: ok, agreed15:43
eglynnnprivalova: ... cool, lets do that so ... I'l retarget in LP now15:43
gordcnprivalova: eglynn: cool. let's fix the other bug properly as a single backport.15:43
nprivalovamalini: Mongo on gating seems unreachable :)15:44
eglynnnprivalova: ... https://bugs.launchpad.net/ceilometer/+bug/1288284 is now off the radar for RC115:44
eglynnnprivalova: ... https://bugs.launchpad.net/ceilometer/+bug/1298455 is on for RC1 and critical15:44
malininprivalova: :(15:45
nprivalovamalini: wrong smile, agreed15:45
eglynnnprivalova, gordc: and lets try to get https://review.openstack.org/83435 landed by EoD15:45
nprivalovaeglynn: and this https://review.openstack.org/#/c/82438/15:45
gordceglynn: i should bring up there is a clause with that patch. it requires a requiremnts bump15:45
eglynnnprivalova: ... a-ha, darn!15:46
nprivalovaeglynn: :D15:46
*** nealph has quit IRC15:46
eglynnnprivalova: ... so we're past dependency freeze, since last Tuesday IIRC15:46
nprivalovaeglynn: there was a discussion in mailing list15:47
eglynnnprivalova: so ttx is happy with "happybase>=0.4,!=0.6,!=0.7"15:48
nprivalovaeglynn: I'll talk to ttx now15:48
eglynnhttp://lists.openstack.org/pipermail/openstack-dev/2014-March/031278.html15:48
eglynnnprivalova: ... ^^^ is that sufficient for you?15:48
nprivalovaeglynn: he doesn't use Ceilometer every day :) Let me try to talk to him15:49
eglynnnprivalova: cool15:49
*** nati_ueno has joined #openstack-ceilometer15:54
openstackgerritA change was merged to openstack/ceilometer: De-dupe selectable aggregate list in statistics API  https://review.openstack.org/8351116:01
*** Ruetobas has quit IRC16:01
*** Ruetobas has joined #openstack-ceilometer16:03
*** mihgen has quit IRC16:03
*** giroro_ has joined #openstack-ceilometer16:06
*** Ruetobas has quit IRC16:08
*** changbl has quit IRC16:14
*** _cjones_ has joined #openstack-ceilometer16:20
*** claudiub has quit IRC16:25
*** nati_ueno has quit IRC16:27
eglynngordc: FYI nprivalova has decoupled https://review.openstack.org/83435 from https://review.openstack.org/8243816:29
eglynn(ttx is agreed that the "de facto" happybase version used is gonna be 0.8 by any sane deployer of hbase)16:30
gordceglynn: so we're keeping happybase>=0.4,<=0.6 in our requirements?16:33
eglynngordc: yes, with the understanding that in reality (in a deployment that actually uses hbase) these constraints will not be respected16:34
eglynngordc: ... i.e. we turn blind eye to stated requirements in this case16:35
gordceglynn: oh... doesn't it throw requirement failures if you have something outside the range?16:36
gordceglynn: or does it just give you warning?16:36
eglynngordc: do you mean in the gate or in production?16:36
eglynngordc: apparently we don't truely use hbase anywhere in the gate16:37
gordcgordc: i guess in production... i really just use devstack which i think stops you from continuing16:37
*** nealph has joined #openstack-ceilometer16:38
eglynngordc: TBH I've never used hbase in anger myself, but IIUC in production it would be the distro controlling the versioning, not the requirements we use in-tree16:39
gordceglynn: i see... not a fan of this but i guess since it's an optional path and hidden from gates we can let this in?16:41
eglynngordc: I don't like it either, but seems to me we're caught between a rock and a hard place16:41
gordceglynn: i blame nprivalova if this fails. lol16:42
eglynngordc: ... and this approach presents a pragmatic way out that's been endorsed by our illustrious release manager16:42
*** vrovachev has quit IRC16:43
*** sayalilunkad has joined #openstack-ceilometer16:43
*** jaypipes is now known as leakypipes16:43
gordceglynn: cool cool. something to revisit early in juno.16:43
eglynngordc: definitely!16:43
gordceglynn: did ttx say monday was last day? i guess we only have one rc1 bug left: https://bugs.launchpad.net/ceilometer/+bug/125972416:45
gordceglynn: and i'm pretty indifferent whether we get it in or not since it's low priority... makes me wonder if we should just push it and say after connectionpool patch we're done for rc116:45
*** eglynn_ has joined #openstack-ceilometer16:47
*** eglynn has quit IRC16:48
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Add note on aggregate duplication to API docco  https://review.openstack.org/8381716:48
openstackgerritgordon chung proposed a change to openstack/ceilometer: DBDeadlock exception in sql backend  https://review.openstack.org/8046116:51
*** inc0 has joined #openstack-ceilometer16:52
inc0good afternoon everyone16:53
eglynn_... folks I'm gonna delay cutting 1.0.10 until Monday, in line with RC116:53
eglynn_... also gives a chance for https://review.openstack.org/83514 to land16:53
*** jmckind has quit IRC17:03
*** promulo has joined #openstack-ceilometer17:10
*** nati_ueno has joined #openstack-ceilometer17:11
*** malini has left #openstack-ceilometer17:19
*** saju_m has joined #openstack-ceilometer17:20
*** sayalilunkad has quit IRC17:27
*** changbl has joined #openstack-ceilometer17:27
*** saju_m has quit IRC17:35
*** saju_m has joined #openstack-ceilometer17:36
*** promulo has quit IRC17:36
*** promulo has joined #openstack-ceilometer17:36
*** eglynn_ has quit IRC17:46
*** inc0 has quit IRC17:53
*** nacim has quit IRC17:54
openstackgerritgordon chung proposed a change to openstack/ceilometer: test do not merge  https://review.openstack.org/8384217:55
*** ildikov_ has quit IRC18:17
*** Alexei_987 has quit IRC18:27
*** shakayumi has joined #openstack-ceilometer18:44
*** shakayumi has quit IRC18:44
openstackgerritA change was merged to openstack/ceilometer: Use ConectionPool instead of one Connection in HBase  https://review.openstack.org/8343518:47
*** openstackgerrit has quit IRC18:48
*** openstackgerrit has joined #openstack-ceilometer18:48
*** shakayumi has joined #openstack-ceilometer18:58
*** shakayumi has quit IRC18:58
*** _nadya_ has joined #openstack-ceilometer18:58
*** _nadya_ has quit IRC18:59
*** _nadya_ has joined #openstack-ceilometer19:00
*** _nadya_ has quit IRC19:05
openstackgerritA change was merged to openstack/ceilometer: remove dump tables from previous migrations  https://review.openstack.org/8365719:06
*** _nadya_ has joined #openstack-ceilometer19:16
openstackgerritgordon chung proposed a change to openstack/ceilometer: test do not merge  https://review.openstack.org/8384219:26
*** Alexei_987 has joined #openstack-ceilometer19:31
*** ildikov_ has joined #openstack-ceilometer19:34
Alexei_987gordc: Hi you can ping me here to reduce latency between comments20:11
*** yassine has quit IRC20:16
*** _nadya_ has quit IRC20:25
*** julim has quit IRC20:27
gordcAlexei_987: whoops. didn't see your msg. my bad.20:30
Alexei_987gordc: we almost agreed anyway20:31
gordci added a comment... let me know what you think we can go from there. this isn't really targetting rc1 unless people find it useful as a stopgap between proper schema fix in Juno.20:32
Alexei_987gordc: what is the simplest way to reproduce this issue?20:32
Alexei_987gordc: running devstack with workers > 5 would be enough?20:32
gordcAlexei_987: ok sounds good. sorry for misleading title. was just quoting the error the database throws...20:33
gordcAlexei_987:  i've tried to reproduce it all today. it's being masked by maximum recursion error.20:33
*** julim has joined #openstack-ceilometer20:33
Alexei_987gordc: you mean you get a recursion on create_or_update?20:34
gordcAlexei_987: yes. so when i added logic to fix _create_or_update, i made it 'retry' by calling itself.... in previous code it'd do a requery.20:35
Alexei_987gordc: (facepalm)20:35
gordchttps://github.com/openstack/ceilometer/commit/21237b333ca2e6766a74dc28c5f788160349a9ce#diff-f0fc20a5e281dd5d212292afe53c667020:35
Alexei_987gordc: yeah I've seen this one20:35
Alexei_987gordc: it seems that our test sucks20:36
Alexei_987gordc: cause we are letting huge amount of bugs to slip through20:36
gordcwell the recursion works unless your writing too slowly...20:36
Alexei_987gordc: "works unless" is not good20:37
Alexei_987slow writes may happen for many reasons20:37
gordcAlexei_987: so it's technically pointing us to performance problem... i switch to recursion since previously it was just guessing (often wrong) that a obj would be found on query20:38
gordcAlexei_987: also, this code is not testable against sqlite...we need the real backend tests enabled20:38
Alexei_987gordc: I've proposed a session on summit to discuss performance issues and new data model20:38
Alexei_987we could make postgres tests work20:38
Alexei_987only review is needed :)20:39
gordcAlexei_987: cool, that'll be approved... performance was one of main concerns Julien and i had.20:39
gordcAlexei_987: we can't put it in post FFE! follow the rules! lol20:39
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Add note on aggregate duplication to API docco  https://review.openstack.org/8381720:40
Alexei_987gordc: I just don't like the fact that we are applying many hotfixes just to push release out20:40
Alexei_987I think that a cleaner way would be to postpone release and fix code properly20:40
gordcAlexei_987: you'll run into arguments what needs to be 'fixed' and the scope will drag the postponement on forever.20:41
Alexei_987gordc: I prefer having nightly builds and proper tests20:42
gordcAlexei_987: perfect world sure... this isn't. i see post FFE as fix small stuff and fix critical stuff.20:42
Alexei_987gordc: in such case any build is stable enough to be released20:42
gordcAlexei_987: +1000 proper tests20:42
gordcAlexei_987: when the real backend tests get in we'll be better. (hopefully)20:43
gordcAlexei_987: that said postgres is the db with issues. mysql is the one throwing most of the issues.20:43
Alexei_987gordc: I had both of them passing tests locally :)20:44
gordcAlexei_987: share your password so we can all huddle around and use it :)20:44
*** alexpilotti has quit IRC20:45
Alexei_987gordc:  it's running from this branch https://github.com/Alexei-Kornienko/ceilometer/tree/real-backends20:45
gordcAlexei_987: tempest should catch most of these issues... shame we have issues getting it to work.20:45
Alexei_987gordc: question about tempest - how can we test all backends on tempest?20:46
Alexei_987since it provides deployment with only 1 backend20:46
gordcAlexei_987: well i think there is a hope mongo will be available in future.... hbase is going to be a harder sell.20:47
Alexei_987gordc: I will also support hbase soon :)20:47
Alexei_987I will help nprivalova in supporting/developing it20:47
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Add note on aggregate duplication to API docco  https://review.openstack.org/8381720:48
gordcAlexei_987: i'm thinking we should really choose 2 or 3 backends and say 'we officially will work on these backends' if you want somehting else, you'll need to manage it yourself.20:48
gordcAlexei_987: yeah, hbase will be awesome once we get it going.20:48
Alexei_987gordc: +1 to choose blessed backend20:48
openstackgerritEoghan Glynn proposed a change to openstack/ceilometer: Add note on aggregate duplication to API docco  https://review.openstack.org/8381720:49
gordci think we should choose to really work hard on sql and hbase (or cassandra) and maybe mongo... we can't keep supporting every single db...we should leave the rest as 'use your db against this interface'20:49
gordcAlexei_987: i think we tried to choose one in HK. but no one was prepared to answer teh question.20:50
Alexei_987gordc: voting on mongodb :)20:51
gordcAlexei_987:  we should do a poll... i'm all for hbase and cassandra if they're easy to set up... sooo not hbase :)20:52
Alexei_987gordc: we should take many factors into account20:53
Alexei_987gordc: especially good horizontal scalability and reliability20:53
Alexei_987gordc: I don't think that easy setup is enough20:54
gordcAlexei_987: make an action item -- everyone come to summit with notes on what db they want... and we'll argue for a hour until we end up with 2 officiall databases.20:54
*** _nadya_ has joined #openstack-ceilometer20:55
gordcyou have a link to your performance session? just so i know what you want to cover and see if we need additional sessions.20:55
Alexei_987gordc: http://summit.openstack.org/cfp/details/16320:56
Alexei_987gordc: database backend can also be discussed during this session20:56
Alexei_987since it's coupled with data model20:56
Alexei_987at least on the level of SQL vs NoSQL20:56
gordcyeah. it'd be good to choose a backend or two and then agree on how to model each one.... right now all our dbs have the same model which make no sense as they're different dbs20:57
gordcAlexei_987: we might need two sessions for that but we'll figure it out.20:58
Alexei_987gordc: I'm just afraid that with many sessions we would not be able to attend all of them20:59
*** julim has quit IRC20:59
Alexei_987gordc: since I plan to visit not only ceilometer discussions20:59
gordcAlexei_987: who said you can work on other stuff outside ceilometer! :)21:00
gordcAlexei_987: we'll see what sessions are proposed. but yeah, data model is defiinitely a key topic.21:01
Alexei_987gordc: It's quite hard to have your stuff reviewed quickly so have to find other places to contribute to21:01
gordcAlexei_987: yeah, this session was a bit cluttered (a lot of people focused outside ceilometer). hoping we're a bit more focused in Juno especially before Juno-3. Icehouse-3 was just flooded here.21:03
gordcthis /session/cycle/21:03
Alexei_987gordc: let's discuss latest gossip :) who plans to be a PTL for the next cycle?21:05
gordclol pm me. j/k21:05
gordci guess anyone can be nomiated.... we'll see.21:05
gordci need to head out to run some errands. good hearing your ideas... enjoy your weekend.21:06
Alexei_987gordc: have a nice day :)21:06
*** gordc has quit IRC21:07
*** jergerber has joined #openstack-ceilometer21:08
*** jdob has quit IRC21:11
*** _nadya_ has quit IRC21:50
*** thomasem has quit IRC21:53
*** prad has quit IRC22:02
*** _nadya_ has joined #openstack-ceilometer22:18
*** _nadya_ has quit IRC22:33
*** _nadya_ has joined #openstack-ceilometer22:35
*** jergerber has quit IRC22:57
*** dhellmann is now known as dhellmann_23:13
*** giroro_ has quit IRC23:54
*** julim has joined #openstack-ceilometer23:54
*** leakypipes has quit IRC23:55
*** leakypipes has joined #openstack-ceilometer23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!