Thursday, 2013-02-07

*** sdake has quit IRC00:00
*** sarob has quit IRC00:01
*** garyTh has quit IRC00:03
*** bdpayne has joined #openstack-meeting00:06
*** fnaval has joined #openstack-meeting00:08
*** fnaval has quit IRC00:08
*** fnaval has joined #openstack-meeting00:09
*** dwcramer has joined #openstack-meeting00:11
*** lbragstad has quit IRC00:13
*** rnirmal has quit IRC00:15
*** ewindisch has quit IRC00:20
*** mrodden has quit IRC00:27
*** ewindisch has joined #openstack-meeting00:28
*** sacharya has joined #openstack-meeting00:33
*** colinmcnamara has quit IRC00:33
*** ewindisch has quit IRC00:37
*** woodspa has quit IRC00:40
*** Hg has quit IRC00:42
*** jrodom has quit IRC00:45
*** markwash has quit IRC00:46
*** henrynash_ has joined #openstack-meeting00:47
*** henrynash has quit IRC00:51
*** henrynash_ is now known as henrynash00:51
*** ryanpetrello has joined #openstack-meeting00:59
*** maoy has joined #openstack-meeting00:59
*** maoy has quit IRC01:00
*** Nachi has joined #openstack-meeting01:00
*** ryanpetrello has quit IRC01:01
*** ewindisch has joined #openstack-meeting01:02
*** nati_ueno has quit IRC01:02
*** stevebaker has joined #openstack-meeting01:03
*** adjohn has joined #openstack-meeting01:09
*** anteaya has quit IRC01:11
*** markvan has quit IRC01:14
*** kmartin is now known as kmartin_zz01:24
*** gyee has quit IRC01:33
*** lbragstad has joined #openstack-meeting01:33
*** yaguang has joined #openstack-meeting01:34
*** danwent has quit IRC01:34
*** ryanpetrello has joined #openstack-meeting01:35
*** bdpayne has quit IRC01:40
*** ewindisch has quit IRC01:59
*** danwent has joined #openstack-meeting02:01
*** AlanClark has joined #openstack-meeting02:07
*** Yada has quit IRC02:16
*** hemna is now known as hemnafk02:19
*** yaguang has quit IRC02:20
*** huats has quit IRC02:21
*** huats has joined #openstack-meeting02:21
*** huats has quit IRC02:21
*** huats has joined #openstack-meeting02:21
*** jog0 has quit IRC02:22
*** bencherian has quit IRC02:25
*** Mandell has quit IRC02:30
*** yaguang has joined #openstack-meeting02:32
*** adjohn has quit IRC02:34
*** ewindisch has joined #openstack-meeting02:40
*** Yada has joined #openstack-meeting02:45
*** vkmc has quit IRC02:50
*** salv-orlando has quit IRC02:53
*** ewindisch has quit IRC02:58
*** Nachi has quit IRC02:58
*** markmcclain has joined #openstack-meeting02:59
*** sarob has joined #openstack-meeting02:59
*** blamar_ has joined #openstack-meeting02:59
*** nati_ueno has joined #openstack-meeting02:59
*** blamar has quit IRC03:00
*** blamar_ is now known as blamar03:00
*** nati_ueno has quit IRC03:00
*** adjohn has joined #openstack-meeting03:04
*** blamar_ has joined #openstack-meeting03:07
*** esker has joined #openstack-meeting03:08
*** blamar has quit IRC03:10
*** blamar_ is now known as blamar03:10
*** ryanpetrello has quit IRC03:10
*** adjohn has quit IRC03:14
*** sarob has quit IRC03:16
*** Yada has quit IRC03:17
*** blamar has quit IRC03:19
*** blamar has joined #openstack-meeting03:21
*** anniec has quit IRC03:23
*** iben has quit IRC03:24
*** iben has joined #openstack-meeting03:25
*** iben has quit IRC03:29
*** danwent has quit IRC03:31
*** alexpilotti has quit IRC03:35
*** bencherian has joined #openstack-meeting03:39
*** adjohn has joined #openstack-meeting03:40
*** adjohn has quit IRC03:45
*** diogogmt has quit IRC03:54
*** diogogmt has joined #openstack-meeting03:59
*** markvan has joined #openstack-meeting04:05
*** ewindisch has joined #openstack-meeting04:06
*** markmcclain has quit IRC04:12
*** anniec has joined #openstack-meeting04:13
*** martine_ has joined #openstack-meeting04:15
*** maurosr has quit IRC04:18
*** diogogmt has quit IRC04:24
*** ogelbukh has quit IRC04:32
*** ogelbukh has joined #openstack-meeting04:33
*** novas0x2a|laptop has quit IRC04:34
*** esker has quit IRC04:35
*** adjohn has joined #openstack-meeting04:41
*** bdpayne has joined #openstack-meeting04:45
*** adjohn has quit IRC04:45
*** bencherian has quit IRC04:59
*** bdpayne has quit IRC05:00
*** bdpayne has joined #openstack-meeting05:05
*** danwent has joined #openstack-meeting05:05
*** Mandell has joined #openstack-meeting05:07
*** blamar_ has joined #openstack-meeting05:09
*** blamar has quit IRC05:11
*** blamar_ is now known as blamar05:11
*** sacharya has quit IRC05:17
*** otherwiseguy has quit IRC05:18
*** martine_ has quit IRC05:21
*** zul has quit IRC05:30
*** same5336_ has joined #openstack-meeting05:33
*** markvan has quit IRC05:36
*** same5336 has quit IRC05:37
*** same5336_ has quit IRC05:40
*** yaguang has quit IRC05:45
*** koolhead17 has joined #openstack-meeting05:48
*** davidha has quit IRC05:51
*** bdpayne has quit IRC05:54
*** koolhead17 has quit IRC05:56
*** bdpayne has joined #openstack-meeting06:00
*** bdpayne has quit IRC06:03
*** adjohn has joined #openstack-meeting06:05
*** mnewby has quit IRC06:07
*** jhenner has joined #openstack-meeting06:08
*** stevebaker has quit IRC06:22
*** asalkeld has quit IRC06:30
*** jhenner has quit IRC06:35
*** yaguang has joined #openstack-meeting06:39
*** novas0x2a|laptop has joined #openstack-meeting06:50
*** davidha has joined #openstack-meeting06:51
*** flaper87 has joined #openstack-meeting06:51
*** garyk has joined #openstack-meeting06:52
*** metral_ has joined #openstack-meeting07:08
*** metral_ has quit IRC07:09
*** metral has quit IRC07:12
*** mrunge has joined #openstack-meeting07:13
*** jhenner has joined #openstack-meeting07:24
*** henrynash has quit IRC07:31
*** ttrifonov_zZzz is now known as ttrifonov07:38
*** EmilienM has joined #openstack-meeting07:51
*** ketan985 has joined #openstack-meeting08:01
*** rafaduran has joined #openstack-meeting08:03
*** afazekas has joined #openstack-meeting08:28
*** spn has quit IRC08:33
*** spn has joined #openstack-meeting08:33
*** spn is now known as spn__08:34
*** spn has joined #openstack-meeting08:35
*** henrynash has joined #openstack-meeting08:40
*** spn has quit IRC08:45
*** dosaboy has joined #openstack-meeting08:45
*** egallen has joined #openstack-meeting08:52
*** novas0x2a|laptop has quit IRC08:54
*** davidha has quit IRC09:04
*** salv-orlando has joined #openstack-meeting09:09
*** darraghb has joined #openstack-meeting09:10
*** jhenner1 has joined #openstack-meeting09:16
*** novas0x2a|laptop has joined #openstack-meeting09:16
*** jhenner has quit IRC09:17
*** zbitter has quit IRC09:20
*** danwent has quit IRC09:21
*** Slower has quit IRC09:22
*** adjohn has quit IRC09:29
*** ketan985 has quit IRC09:30
*** Slower has joined #openstack-meeting09:46
*** bobba has joined #openstack-meeting09:49
*** Mandell has quit IRC09:51
*** novas0x2a|laptop has quit IRC09:54
*** novas0x2a|laptop has joined #openstack-meeting09:56
*** asalkeld has joined #openstack-meeting09:57
*** shang has joined #openstack-meeting10:00
*** giulivo has joined #openstack-meeting10:03
*** adjohn has joined #openstack-meeting10:07
*** mrunge has quit IRC10:09
*** psedlak has joined #openstack-meeting10:12
*** egallen_ has joined #openstack-meeting10:12
*** mrunge has joined #openstack-meeting10:14
*** egallen has quit IRC10:15
*** egallen_ is now known as egallen10:15
*** adjohn has quit IRC10:17
*** derekh has joined #openstack-meeting10:21
*** egallen_ has joined #openstack-meeting10:28
*** egallen has quit IRC10:31
*** egallen_ is now known as egallen10:31
*** bobba has quit IRC10:46
*** vkmc has joined #openstack-meeting10:53
*** bobba has joined #openstack-meeting10:58
*** yaguang has quit IRC10:58
*** yaguang has joined #openstack-meeting10:58
*** alexpilotti has joined #openstack-meeting11:08
*** dhellmann-afk has quit IRC11:09
*** maurosr has joined #openstack-meeting11:14
*** yaguang has quit IRC11:16
*** giulivo has quit IRC11:17
*** zul has joined #openstack-meeting11:29
*** zul has quit IRC11:30
*** zul has joined #openstack-meeting11:30
*** yaguang has joined #openstack-meeting11:32
*** yaguang has quit IRC11:38
*** dosaboy has quit IRC11:50
*** dosaboy has joined #openstack-meeting11:51
*** yaguang has joined #openstack-meeting11:54
*** yaguang has quit IRC12:07
*** ewindisch has quit IRC12:14
*** yaguang has joined #openstack-meeting12:20
*** mrunge has quit IRC12:20
*** martine has joined #openstack-meeting12:25
*** yaguang has quit IRC12:28
*** yaguang has joined #openstack-meeting12:29
*** yaguang has quit IRC12:37
*** adjohn has joined #openstack-meeting12:40
*** Hg_ has joined #openstack-meeting12:42
*** adjohn has quit IRC12:45
*** otherwiseguy has joined #openstack-meeting12:47
*** yaguang has joined #openstack-meeting12:50
*** jaypipes has quit IRC12:55
*** markvoelker has joined #openstack-meeting12:56
*** anniec has quit IRC12:57
*** yaguang has quit IRC13:08
*** otherwiseguy has quit IRC13:09
*** adjohn has joined #openstack-meeting13:11
*** jaypipes has joined #openstack-meeting13:12
*** adjohn has quit IRC13:16
*** Hg_ is now known as _Hg_13:16
*** _Hg_ is now known as HenryG13:18
*** HenryG has quit IRC13:21
*** HenryG has joined #openstack-meeting13:21
*** HenryG has quit IRC13:23
*** HenryG has joined #openstack-meeting13:23
*** HenryG has joined #openstack-meeting13:24
*** markvoelker1 has joined #openstack-meeting13:26
*** Yada has joined #openstack-meeting13:26
*** markvoelker has quit IRC13:27
*** markvoelker has joined #openstack-meeting13:28
*** HenryG has quit IRC13:28
*** HenryG has joined #openstack-meeting13:28
*** markvoelker1 has quit IRC13:31
*** HenryG has quit IRC13:31
*** yaguang has joined #openstack-meeting13:31
*** HenryG has joined #openstack-meeting13:32
*** HenryG has quit IRC13:32
*** HenryG has joined #openstack-meeting13:32
*** markvoelker has quit IRC13:33
*** dwcramer has quit IRC13:34
*** yaguang has quit IRC13:36
*** EmilienM has left #openstack-meeting13:37
*** EmilienM has joined #openstack-meeting13:37
*** markvoelker3 has joined #openstack-meeting13:38
*** sandywalsh has quit IRC13:41
*** adjohn has joined #openstack-meeting13:42
*** yaguang has joined #openstack-meeting13:44
*** yaguang has joined #openstack-meeting13:45
*** woodspa has joined #openstack-meeting13:45
*** woodspa has joined #openstack-meeting13:46
*** adjohn has quit IRC13:46
*** cdub has joined #openstack-meeting13:49
*** martine_ has joined #openstack-meeting13:49
*** martine has quit IRC13:49
*** yaguang has quit IRC13:50
*** yaguang has joined #openstack-meeting13:50
*** yaguang has joined #openstack-meeting13:50
*** noslzzp has joined #openstack-meeting13:52
*** ewindisch has joined #openstack-meeting13:54
*** dprince has joined #openstack-meeting13:58
*** lbragstad has quit IRC14:00
*** anteaya has joined #openstack-meeting14:02
*** annegentle has joined #openstack-meeting14:04
*** bobba has quit IRC14:12
*** adjohn has joined #openstack-meeting14:13
*** markvan has joined #openstack-meeting14:13
*** davidha has joined #openstack-meeting14:16
*** markvan has quit IRC14:17
*** adjohn has quit IRC14:17
*** iben has joined #openstack-meeting14:18
*** fnaval has quit IRC14:22
*** jd__ has joined #openstack-meeting14:24
*** dhellmann has joined #openstack-meeting14:25
*** yaguang has quit IRC14:26
*** b3nt_pin has joined #openstack-meeting14:26
*** joesavak has joined #openstack-meeting14:27
*** lbragstad has joined #openstack-meeting14:27
*** beagles has quit IRC14:27
*** bencherian has joined #openstack-meeting14:30
*** anniec has joined #openstack-meeting14:32
*** dprince has quit IRC14:36
*** b3nt_pin has quit IRC14:39
*** anniec has quit IRC14:39
*** anniec has joined #openstack-meeting14:39
*** b3nt_pin has joined #openstack-meeting14:42
*** b3nt_pin has quit IRC14:42
*** b3nt_pin has joined #openstack-meeting14:43
*** adjohn has joined #openstack-meeting14:43
*** salv-orlando has quit IRC14:46
*** fnaval has joined #openstack-meeting14:47
*** salv-orlando has joined #openstack-meeting14:48
*** adjohn has quit IRC14:48
*** dwcramer has joined #openstack-meeting14:48
*** sandywalsh has joined #openstack-meeting14:48
*** mtreinish has joined #openstack-meeting14:50
*** dprince has joined #openstack-meeting14:52
*** yjiang5_home has joined #openstack-meeting14:52
*** metral has joined #openstack-meeting14:54
*** jtran has joined #openstack-meeting14:55
*** markmcclain has joined #openstack-meeting14:55
*** llu-laptop has joined #openstack-meeting14:56
*** danspraggins has joined #openstack-meeting14:57
*** mrodden has joined #openstack-meeting14:58
*** iben has quit IRC14:58
*** jsavak has joined #openstack-meeting14:59
*** iben has joined #openstack-meeting14:59
*** markvan has joined #openstack-meeting14:59
*** sew has joined #openstack-meeting14:59
*** jaypipes has quit IRC14:59
*** eglynn_ has joined #openstack-meeting14:59
nijaba#startmeeting Ceilometer15:00
nijaba#meetingtopic Ceilometer15:00
nijaba#chair nijaba15:00
nijaba#link http://wiki.openstack.org/Meetings/MeteringAgenda15:00
nijabaATTENTION: please keep discussion focused on topic until we reach the open discussion topic15:00
openstackMeeting started Thu Feb  7 15:00:01 2013 UTC.  The chair is nijaba. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: Ceilometer)"15:00
openstackThe meeting name has been set to 'ceilometer'15:00
*** openstack changes topic to " (Meeting topic: Ceilometer)"15:00
openstackCurrent chairs: nijaba15:00
eglynn_o/15:00
spn__o/15:00
n0anoo/15:00
nijabao/15:00
danspragginso/15:00
sewo/15:00
jd__o/15:00
*** salv-orlando has quit IRC15:00
nijabanice to see all of you! let's start15:01
nijaba#topic actions from previous meeting15:01
*** openstack changes topic to "actions from previous meeting (Meeting topic: Ceilometer)"15:01
llu-laptopo/015:01
llu-laptopo/15:01
nijaba#topic jd contact Eric Windisch about nova's trusted-messaging blueprint status15:01
nijabajd__ any feedback on this?15:01
*** openstack changes topic to "jd contact Eric Windisch about nova's trusted-messaging blueprint status (Meeting topic: Ceilometer)"15:01
jd__yes, I had a conversation with Eric15:01
*** nealph has joined #openstack-meeting15:01
eglynn_it was mentioned at FOSDEM that the blueprint was "demoted"15:01
jd__I've updated the whiteboard at https://blueprints.launchpad.net/ceilometer/+spec/use-new-rpc-messsage with what we discussed15:01
sandywalsho/15:01
* eglynn_ is not sure what that means ...15:02
jd__Discussed with Eric 31st January: this is not targeted at Grizzly, but the code is likely to be ready for review around the 15th February, and hopefully will be merged by the end of February.15:02
dhellmanno/15:02
nijabaeglynn_: I meant not targetted for grizzly anymore15:02
eglynn_OK so demoted == punted to H15:02
*** esker has joined #openstack-meeting15:02
eglynn_yep, understood, thanks15:02
jd__eglynn_: yes, nijaba asked that to he HP guy after I told him what I discussed with Eric :)15:03
*** joesavak has quit IRC15:03
eglynn_jd__: cool, I was behind the curve that that15:03
*** iben has quit IRC15:03
eglynn_(thanks for the update ...)15:03
jd__so basically, that's wait'n see for us15:03
nijabaso the pb is that we won't have time to use it in G anyway15:03
*** yjiang5_home has quit IRC15:03
jd__I think so indeed15:04
*** apmelton has joined #openstack-meeting15:04
nijabaFeature freeze is Feb 19th, when we branch the release15:04
*** YehiaBeyh has joined #openstack-meeting15:04
eglynn_yep, so do we think that's a major issue for ceilometer's deployability?15:04
*** jaypipes has joined #openstack-meeting15:05
jd__I don't think so15:05
eglynn_(given that all the other RPC flows are also not secure)15:05
eglynn_+115:05
jd__that's not directly our problem, and *we* have some signing so far15:05
eglynn_true that15:05
dhellmannright, we do sign our messages15:05
jd__i.e. we're not the worst :)15:05
nijabaeglynn_: no, but it's a bummer15:05
*** spn__ is now known as spn15:05
dhellmannthe point was to be consistent, but if it's not in oslo then there's no issue with us not using it15:05
sandywalshif the nova rpc messaging isn't secure, the CM messaging is the least of our worries (imho)15:05
*** spn has quit IRC15:05
*** spn has joined #openstack-meeting15:05
jd__one thing I didnt' write on the whiteboard but I can mention to you15:05
eglynn_sandywalsh: agreed15:06
*** yjiang5_home has joined #openstack-meeting15:06
nijabaok, I think we know where we stand on this. let's move on15:06
nijaba#topic nijaba to specify draft policy on wiki for units15:06
nijaba#info done http://wiki.openstack.org/Ceilometer/Units15:06
nijabaI did make a call for comments on the ml.  Do you think we are ready to approve them or should we push a vote on the subject to next week?15:06
*** openstack changes topic to "nijaba to specify draft policy on wiki for units (Meeting topic: Ceilometer)"15:06
jd__RPC signin needs envelope on RPC messaging, and that's not activated by default for G15:06
eglynn_sandywalsh: although some might see the bar set higher for ceilo given that there may be $$$ involved at the end of the pipeline15:06
sandywalsheglynn_: I can still spoof all the notifications on the nova side to the same effect15:07
eglynn_sandywalsh: true that15:07
dhellmannnijaba: I haven't seen any objections. It seems we're ready to vote.15:07
nijabaok then, let's start the vote15:07
*** martine_ has quit IRC15:07
nijaba#startvote approve http://wiki.openstack.org/Ceilometer/Units ? yes, no, abstain15:08
openstackBegin voting on: approve http://wiki.openstack.org/Ceilometer/Units ? Valid vote options are yes, no, abstain.15:08
openstackVote using '#vote OPTION'. Only your last vote counts.15:08
nijaba#vote yes15:08
llu-laptop#vote yes15:08
dhellmann#vote yes15:08
sandywalsh#vote yes15:08
jtran#vote yes15:08
spn#vote yes15:08
eglynn_#vote yes15:08
n0ano#vote yes15:08
jd__#vote yes15:08
yjiang5_home#vote abstain15:08
nijabanot thrill on this one...15:08
danspraggins#vote yes15:08
sew#vote yes15:09
apmelton#vote yes15:09
nealph#vote yes15:09
nijaba#endvote15:09
openstackVoted on "approve http://wiki.openstack.org/Ceilometer/Units ?" Results are15:09
fnaval#vote yes15:09
openstackyes (13): sew, apmelton, n0ano, jtran, nealph, sandywalsh, jd__, nijaba, llu-laptop, dhellmann, spn, danspraggins, eglynn_15:09
openstackabstain (1): yjiang5_home15:09
nijabaok great!15:09
yjiang5_homehehe15:09
nijabaThat's it for last week's actions15:09
sandywalshout of curiousity yjiang5_home, did you have objections to the proposal?15:09
dhellmannwe should probably add some form of those rules to our docs15:09
sandywalsh(or just being a trouble maker ;)15:10
yjiang5_homesandywalsh: I just didn't think through this, so no idea yet.15:10
sandywalshgotcha ... thanks15:10
dhellmann#action dhellman to update documentation based on http://wiki.openstack.org/Ceilometer/Units15:10
nijabadhellmann: to the dev doc: yes.  or at least a pointer to the wiki15:10
nijabadhellmann: thanks15:10
nijaba#topic Preparing Ceilometer for end of incubation review15:10
*** openstack changes topic to "Preparing Ceilometer for end of incubation review (Meeting topic: Ceilometer)"15:11
dhellmannnijaba: I'll rephrase the rules in terms of what users can expect15:11
nealphdhellmann: at least a crosslink to the current api documentation15:11
nijabattx recently informed me that a TC meeting will happen on Tue Feb 12th at 20UTC to discuss if Heat and Ceilometer are ready to graduate out of incubation.  See "end of cycle graduation review" on the agenda15:11
nijaba#link http://wiki.openstack.org/Governance/TechnicalCommittee15:11
nijabaIt happens that the meeting is scheduled at the same time as the foundation board meeting. We are going to ask Alan to allow MarkMcLoughlin, Monty and I to join the TC meeting, but it that fails, ttx may have to reschedule the discussion.15:11
dhellmannI thought I would be able to attend that meeting, but there's a very good chance I won't. :-(15:11
nijabaWe need to prepare answers on:15:11
nijabaa) Why we think we're ready15:11
nijabab) be ready to explain why, even though discussion with healthnmon and sandywalsh have been deep and convoluted, we still beleive that they have not endengered the founding of ceilometer architecture.15:11
eglynn_so is this the first TC meeting in a sequence to consider graduation?15:11
nijabaeglynn_: this is what I understood15:12
eglynn_i.e. fact-finding, followed by a decision point at a subsequent meeting15:12
sandywalshconvoluted? :)15:12
nijabastatements in that directions on the wiki would be welcome, specially from core members, sandywalsh and healthnmon team.15:12
nijabaComments?15:12
jd__(I'll attend the meeting too)15:13
eglynn_IMO the main thing we need to prepared to defend is the stability of the core architecture15:13
nijabaeglynn_: agreed15:13
eglynn_(in particular the proposed changes around the nove interaction model)15:13
sandywalshassuming we can store the raw event data (somehow) ... I don't think there's any glaring objections from us15:13
nijabaI need some volunteers to start contributing to the wiki page to prepare this on a wiki page:15:13
nijaba#link http://wiki.openstack.org/Ceilometer/Graduation15:13
nijabaRaise your hand if you volunteer help on this15:13
eglynn_o/15:13
jd__o/15:13
sandywalsho/15:14
spno/15:14
*** adjohn has joined #openstack-meeting15:14
nijabaThis is great, thanks a lot for your help15:14
eglynn_IMO we need to provide comfort that ...15:15
eglynn_(a) we won't run off and do a keystone-lite rewrite during our first cycle as a core project15:15
eglynn_and:15:15
nijaba#action eglynn_, jd__, sandywalsh, spn, nijaba to help on http://wiki.openstack.org/Ceilometer/Graduation15:15
eglynn_(b) the architecture doesn't unduely constrain future usecases15:15
spnwill the number of bugs be one of the deciding factor?15:15
eglynn_spn: it shouldn't IMO15:15
jd__spn: that wouldn't be a smart metric15:16
llu-laptopeglynn_: what do you mean by saying "keystone-lite rewrite"?15:16
eglynn_spn: (shows activity, usage ...)15:16
* dhellmann has local distractions15:16
sandywalsheglynn_: we're commiting a lot of resources to CM, so there's very low risk we'll go back to stacktach. We can talk to ttx, et al about that if need be.15:16
dhellmanno/15:16
yjiang5_homenijaba: what's the criterial of ready for graduation? Anyone checked the meeting for Quantum or Cinder?15:16
jd__dhellmann: shoot! :)15:16
yjiang5_homesandywalsh: that's great.15:16
nijabayjiang5_home: confidence of the tc members.  thatś it15:16
eglynn_llu-laptop: during the essex cycle IIRC, keystone was rewritten and presented as fail accompli15:16
dhellmannnijaba: include me in that action15:16
sandywalsheglynn_: correct15:16
eglynn_llu-laptop: (the original keystone that is, not the one we all know and love...)15:17
nijaba#action eglynn_, jd__, sandywalsh, spn, nijaba, dhellmann to help on http://wiki.openstack.org/Ceilometer/Graduation15:17
eglynn_s/fail accompli/fait accompli/15:17
* eglynn_ had a Freudian slip ...15:17
nijabaeglynn_: a very nice one ;)15:17
jd__eglynn_: I was wondering what you meant, haha!15:17
nijabaok, so everyone clear on this?15:18
*** adjohn has quit IRC15:18
nijabaok, let's move on15:19
nijaba#topic Updating our "requests" dependency to 1.0 (dhellmann, zul, yolanda)15:19
*** openstack changes topic to "Updating our "requests" dependency to 1.0 (dhellmann, zul, yolanda) (Meeting topic: Ceilometer)"15:19
yjiang5_homeanything we need achieve before the review other than the wiki?15:19
zulhi15:19
flaper87nijaba: should'nt it be 1.1 ?15:19
eglynn_so there was some work on going to requests 1.1 for glance15:19
*** sacharya has joined #openstack-meeting15:19
flaper87this is the required version for glancelcient15:19
nijabayjiang5_home: write to your favorite tc member, telling him to vote yes ;)15:19
eglynn_(in order to address an issue with encoding IIRC)15:20
dhellmannthe clients use different versions at this point15:20
yjiang5_homenijaba: :)15:20
eglynn_but the glance change was slatted by everything else still being 1.015:20
eglynn_s/slatted/stalled/15:20
flaper87eglynn_: encoding + other improvements15:20
dhellmanneglynn_: actually <1.0, I thought15:20
*** rhochmuth has joined #openstack-meeting15:20
*** rnirmal has joined #openstack-meeting15:20
eglynn_flaper87: hey, thanks for the background info!15:20
flaper87eglynn_: :)15:21
eglynn_flaper87: are all the other clients on 1.0 or <1.0?15:21
sandywalsheglynn_: it's a mix.15:21
flaper87eglynn_: mixed15:21
eglynn_k15:21
zulyes its either using 1.0 or <1.015:21
flaper87but none of them on 1.115:21
sandywalshfoo.json vs. foo.json()15:21
eglynn_usual mess so, we need to cor-ord these deps better15:21
* eglynn_ states the obvious again ...15:22
dhellmannI perhaps got the version in the agenda item wrong15:22
dhellmannzul: do we want 1.1?15:22
flaper87dhellmann: if we don't use 1.1 we won't be able to migrate glanceclient15:22
dhellmannin any case, the topic to discuss was whether someone could volunteer to help with updating our client, and ceilometer, to work with the desired version15:22
eglynn_wouldn't that cuase version mismatch clash is say devstack if novaclient etc. installed first?15:22
zuldhellmann: i think 1.115:23
dhellmanncool15:23
* flaper87 volunteers15:23
dhellmannso, as with webob, I think what will have to happen is first a rollout of a rule that says >=0.8 (the minimum version being used now) -- to all projects15:23
eglynn_flaper87: thanks!15:23
dhellmannthat would have to include fixing them to work with 0.8 and 1.115:23
dhellmannthen, when all projects work like that, we can roll out >=1.115:24
eglynn_dhellmann: a-ha, yep relaxing the dep to >= first is the way to go15:24
* sandywalsh can help once he gets the HACKING branch finished :/15:24
jd__dhellmann: sounds like a good plan15:24
flaper87dhellmann: makes sense15:24
dhellmannbut if it isn't done in 2 steps, then there's no way to get the change past the smokestack gates15:24
eglynn_flaper87: you've confirmed that the distros are all covered packaging-wise for 1.1, right?15:25
dhellmannthat >=0.8 change will have to know about the json-as-a-property vs. json-as-a-method issue inside the requests library15:25
flaper87eglynn_: yeap, non of them use special features but simple requests calls. Some refactor has to be done for those calls but nothing big, AFAIR15:25
flaper87s/non/none15:26
dhellmannis there anything else to say about this?15:26
*** eharney has joined #openstack-meeting15:26
*** eharney has joined #openstack-meeting15:26
flaper87nope15:26
eglynn_sounds covered to me15:27
* sandywalsh does the blackjack dealers hand wipe 15:27
dhellmannwhat's up next, nijaba?15:27
nijaba#topic Coordinating speaker submissions for Summit (conference part)15:27
nijabaI have proposed to Doug to redo a duet presenting ceilometer project and architecture.15:27
nijabaI also would encourage anyone with a real life deployement of ceilometer to propose a talk about how they did it15:27
nijabaNote that the dealine for submitting talks is Feb 15th:15:27
nijaba#link http://t.co/PgvneZBO15:27
*** openstack changes topic to "Coordinating speaker submissions for Summit (conference part) (Meeting topic: Ceilometer)"15:27
nijabaComments?15:27
spnDesign topics has more time I guess15:28
llu-laptopnijaba: seems I can't access that URL15:28
sandywalshI think the place for our coordination is on the design topics ... and we'll update the bps as needed for that15:28
nijaba#link http://www.openstack.org/summit/portland-2013/call-for-speakers/15:28
nijaballu-laptop: I hope the full url is accessible for you15:29
sandywalshwe have some community talks proposed but mostly about metrics/sla's/etc in general and not CM specific15:29
*** sacharya has quit IRC15:29
sandywalsh(lessons learned with stacktach)15:29
eglynn_so we'll have our own design summit track, right?15:29
nijabasandywalsh: eglynn_: wait for next topic :)15:29
eglynn_k15:29
sandywalshwhoops, sorry :) haha15:29
nijabadhellmann: ready to redo the duet?15:30
* sandywalsh - premature articulation15:30
dhellmannnijaba: sure!15:30
nijaba\o/15:30
nijaba#topic Preparing blueprints for discussion in our dev room15:30
nijabaRAX has some talks planned around our metrics efforts with StackTach and moving that into CM.15:30
nijabaIt looks like we will have our own room for meeting and need to start planning how much time we will need15:30
nijabaI propose that we submit discussion topics as blueprints within the next few weeks so that we can get a good idea of how mich time will be needed on the basis of 1h per topic.15:30
nijabaComment?15:30
*** openstack changes topic to "Preparing blueprints for discussion in our dev room (Meeting topic: Ceilometer)"15:30
dhellmann+1 to blueprints15:31
sandywalsh#action RAX will comment on bps for summit15:31
* nijaba check if we can propose bp for the H summit15:31
llu-laptopdo we need some special prefix to the bp title for that purpose?15:31
* eglynn_ thinking the same, just use a naming convention for the BPs?15:32
dhellmannllu-laptop: I don't think so. We'll eventually use the blueprints for the actual work, right?15:32
spnbp's involving new ideas to be discussed  in #openstack-metering?15:32
nijabaI will try to see with ttx if we can that defined as a sprint so that we can propose the bp for the H sprint15:32
dhellmannunless you want to use "havana-" as the prefix?15:32
jd__it's better to create bp and set serie to havana and status to 'discussion' or something like that I think15:33
nijaba#action nijaba to write an email explaining how to propose a topic for the H summit15:33
sandywalshI don't think the naming really makes any difference does it? You just set the milestone to  be H?15:33
sandywalshjd__: +115:33
dhellmannjd__: sounds good15:33
sandywalsh(series, not milestone)15:33
eglynn_so the BPs are really placeholders for initial discusion etc.?15:33
nijabaplease hold on your bp until I check with ttx the preffered methoid, I'll send an email then15:33
eglynn_(.e. the actual design summit proposal is via separate topic mgmt system like before?)15:34
jd__eglynn_: yes15:34
eglynn_cool15:34
jd__nijaba: ack15:34
* jd__ sits on its bag of bp15:34
nijabaeglynn_: summit conference talks is what is separate, yes15:34
*** annegentle has quit IRC15:35
dhellmannnijaba: I think he means the official dev session proposal system used by the organizers15:35
eglynn_nijaba: well, I'm thinking of the design summit proposals ... that wasn't via launchpad BPs last time round, or?15:35
spncan we propose a bp and also can that be a part of summit topics for discussion?15:35
eglynn_dhellmann: yep, I was ...15:35
dhellmanneglynn_: this is something nijaba is asking *our group* to do15:35
nijabathat was up to the projects IIRC15:35
eglynn_a-ha, I see15:36
nijabathen the projects would report in a scheduling tool, separately15:36
dhellmanni.e., let's get some concrete blueprints put together to talk about, instead of more abstract topics15:36
eglynn_thanks for the clarification15:36
nijabadhellmann: +115:36
jd__sounds great15:36
sandywalshI think the foundation allocates slots based on the standing of the project15:36
jd__standing?15:36
sandywalsh(larger projects = more slots)15:36
nijabasandywalsh: I checked, and we should give an estimate of the time we need15:36
sandywalshdhellmann: +115:37
jd__will we have to wear tuxedos?15:37
sandywalshnijaba: cool, thanks15:37
* dhellmann will if jd__ will15:37
nijabajd__: of course.  as well as ice skates15:37
sandywalshpants?15:37
jd__sandywalsh: haha15:37
dhellmannYES EVERYONE HAS TO WEAR PANTS15:37
nijabasandywalsh: never!!!15:37
jd__lol15:37
dhellmann:-)15:37
*** Nithya has joined #openstack-meeting15:38
nijabaeveryone wears kilts15:38
* dhellmann shakes head15:38
nijabaI tink we are done on this very important topic, right?15:38
jd__for now I guess15:38
* eglynn_ was impressed by the Debian kilts at FOSDEM15:38
nijaba#topic Open discussion15:39
eglynn_(might have been a tad cold tho' ...)15:39
*** openstack changes topic to "Open discussion (Meeting topic: Ceilometer)"15:39
sandywalshdoes ceilometer have a logo?15:39
eglynn_not that I know of15:39
nijabasandywalsh: nope... no designers floating around...15:39
sandywalshwould be nice to define one before the summit, if not.15:39
dhellmannnijaba: do we have a deadline for submitting g3 changes?15:39
nijabadhellmann: feature freeze is Feb 19th15:39
dhellmannsandywalsh: +115:39
eglynn_sandywalsh: +115:39
dhellmannnijaba: ok15:39
nijabadhellmann: when we branch the release15:39
sandywalshlet's talk to our graphically minded friends, shall we?15:40
jd__have other projects a logo ?!15:40
dhellmannnijaba: I think what I have to do isn't so much a feature as cleanup15:40
eglynn_nijaba: so does the branching occur immediately once the g3 deadline passes?15:40
sandywalshjd__: some, but it makes us look grown up :)15:40
nijabasandywalsh: I don have any freinds (graphically minded)15:40
eglynn_nijaba: (or later, say when RC1 is cut)15:40
nijabaeglynn_: actually 2 days before the 21st15:41
eglynn_nijaba: so trunk is then theoretically open for H-oriented patches?15:41
* sandywalsh will ask around RAX for design services15:41
nijabaeglynn_: this is what I understood, yes, but this will be my first time15:41
jd__nijaba: are we ok towards our g3 goals btw?15:41
nijabasandywalsh: thanks15:41
nijabajd__: almost.  I sent a mail to tedy about is not started bp.15:42
yjiang5_homejd__: I have two patches pending for g3.15:42
jd__nijaba: okay15:42
nijabadhellmann: has one too15:42
nijabaon another topic, PTL elections are coming up at the end of the month15:43
* eglynn_ has qpid testing to do ...15:43
dhellmannnijaba: the notification listening code? I've started writing that up and discussing it with the oslo devs15:43
nijabadhellmann: perfect15:43
*** davidha has quit IRC15:43
jd__dhellmann: great15:43
dhellmann#link http://wiki.openstack.org/Ceilometer/blueprints/move-listener-framework-oslo15:43
yjiang5_homenijaba: should all BPs target grizily in g3?15:43
dhellmannthe names are subject to change15:43
nijabaSo that you know, I am considering not to present myself as PTL as I do not think it goes well with being a board member15:44
nijabayjiang5_home: not sure I understood your question15:44
*** davidha has joined #openstack-meeting15:44
*** adjohn has joined #openstack-meeting15:45
yjiang5_homenijaba: sorry. I mean "should all BPs target grizily be submiited/merged  in g3 milestone?15:45
nijabayjiang5_home: yep, or they will be punted to h15:45
nijabayjiang5_home: however, you can submit buggy code and fix it during the rc ;)15:45
sandywalshyep, the bp list going into g3 should pretty well be the final list15:45
yjiang5_homenijaba: got it, thanks.15:46
sandywalshnijaba: heh15:46
nijabasandywalsh: yes, I know this is bad, but don we all do it...15:46
sandywalshnijaba: I'm very guilty, don't worry15:46
nijabahehe15:46
*** dolphm has joined #openstack-meeting15:46
*** garyTh has joined #openstack-meeting15:48
nijabaso, anyone ready to play "who wants to be a PTL?" at the end of the month?15:48
eglynn_nijaba: it would be a good sign of a healthy project for the election to be contested, so I'd encourage folks to think about running ...15:48
nijabaeglynn_: +115:49
*** metral has quit IRC15:49
* eglynn_ was disappointed that so few PTL positions were contested in the last cycle ...15:49
sandywalshnijaba: would love to, but need to work on getting core first ;)15:49
*** adjohn has quit IRC15:49
nijabasandywalsh: ah, well... yes15:49
nealphninjaba: PTL = project team lead?15:49
* jd__ will run15:49
dhellmannnijaba: do we have a list of qualified candidates?15:49
nijabanealph: yes15:49
dhellmannjd__: +115:49
* eglynn_ also probably15:50
nijabadhellmann: any core members at this time15:50
sandywalshjd__: +115:50
jd__eglynn_: cool :)15:50
* sandywalsh wants to see Feats of Strength!15:51
yjiang5_homeI'd suggest vote among dhellmann jd__ eglynn_ , all of them are good candidate IMO. :-)15:51
jd__by the way did we mention the amazing talk we did last Sunday?15:51
nijabaanyway, this was just to give you guys some time to think about it.  ttx will be starting the election process at the end of the month, providing all the details15:51
nijabajd__: we did NOT!!!15:51
eglynn_cool, thanks for the early heads-up15:51
nijabait was a great trio!!!15:51
jd__so we did an amazing talk with eglynn_ and nijaba at FOSDEM  https://fosdem.org/2013/schedule/event/openstack_ceilometer/15:51
jd__(slides at the bottom)15:52
nijabaunfortunately, as it was the last talk of the day, it was NOT recorded15:52
llu-laptopwaiting for the video15:52
* dhellmann :-(15:52
llu-laptopoops :-(15:52
* flaper87 was there, great talk guys, honestly15:52
*** henrynash has quit IRC15:53
*** topol has joined #openstack-meeting15:53
jd__thanks flaper8715:53
*** tongli has joined #openstack-meeting15:53
jd__after SlapOS, it was easy anyway :p15:53
nijabaanything else?>15:53
llu-laptopI'll be on vacation from Feb 9th to Feb 15th for Chiese holidays, so I won't be able to attend next weekly meeting.15:53
jd__llu-laptop: enjoy!15:54
nijaballu-laptop: please mention this on the meeting page15:54
* eglynn_ has jury duty next week, will be partially off-grid ...15:54
nijaballu-laptop: enjoy +115:54
sandywalsheglynn_: Ms White in the Library with the Candlestick15:54
spnthe fosdem slides (http://goo.gl/JGidx) were great15:55
eglynn_LOL :)15:55
nijabaok.  Great meeting, as usual!15:55
jd__sandywalsh: :))15:55
nijabaSee you next week on Wed at 21UTC15:55
nijaba#endmeeting15:55
*** openstack changes topic to "OpenStack meetings || Development in #openstack-dev || Help in #openstack"15:55
openstackMeeting ended Thu Feb  7 15:55:34 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:55
dhellmannthanks, everyone!15:55
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ceilometer/2013/ceilometer.2013-02-07-15.00.html15:55
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ceilometer/2013/ceilometer.2013-02-07-15.00.txt15:55
nijaba#cluedo15:55
openstackLog:            http://eavesdrop.openstack.org/meetings/ceilometer/2013/ceilometer.2013-02-07-15.00.log.html15:55
sandywalshthanks all!15:55
eglynn_thanks folks!15:55
spnthanks ;-)15:55
*** danspraggins has quit IRC15:56
*** jtran has left #openstack-meeting15:56
*** metral has joined #openstack-meeting15:57
*** nealph has quit IRC15:57
*** lbragstad has quit IRC15:58
*** sacharya has joined #openstack-meeting15:59
*** henrynash has joined #openstack-meeting15:59
*** annegentle has joined #openstack-meeting16:01
*** john5223 has joined #openstack-meeting16:03
*** rhochmuth has quit IRC16:03
*** henrynash has quit IRC16:04
*** cp16net is now known as cp16net|away16:04
*** lbragstad has joined #openstack-meeting16:05
*** cp16net|away is now known as cp16net16:06
*** bdperkin is now known as bdperkin_gone16:06
*** mrodden has quit IRC16:06
*** bdperkin_gone has quit IRC16:09
*** anniec has quit IRC16:10
*** yjiang5_home has quit IRC16:11
*** hemna has joined #openstack-meeting16:12
*** sew has left #openstack-meeting16:12
*** mrodden has joined #openstack-meeting16:15
*** hemna_ has quit IRC16:15
*** adjohn has joined #openstack-meeting16:16
*** b3nt_pin is now known as beagles16:20
*** llu-laptop has left #openstack-meeting16:21
*** henrynash has joined #openstack-meeting16:22
*** henrynash_ has joined #openstack-meeting16:24
*** esker has quit IRC16:24
*** esker has joined #openstack-meeting16:25
*** bdpayne has joined #openstack-meeting16:27
*** dhellmann has quit IRC16:27
*** markmcclain has quit IRC16:28
*** henrynash has quit IRC16:28
*** henrynash_ is now known as henrynash16:28
*** danwent has joined #openstack-meeting16:31
*** maurosr has quit IRC16:34
*** psedlak has quit IRC16:36
*** sdake has joined #openstack-meeting16:39
*** same5336 has joined #openstack-meeting16:41
*** adjohn has quit IRC16:43
*** garyk has quit IRC16:45
*** koolhead17 has joined #openstack-meeting16:47
*** diogogmt has joined #openstack-meeting16:48
*** lbragstad has quit IRC16:51
*** salv-orlando has joined #openstack-meeting16:54
*** markvan has quit IRC16:58
*** mrodden1 has joined #openstack-meeting16:58
*** mlavalle has joined #openstack-meeting17:00
*** mrodden has quit IRC17:00
*** radez_g0n3 is now known as radez17:01
*** maurosr has joined #openstack-meeting17:01
*** noslzzp has quit IRC17:03
sdaguewho's here for tempest meeting?17:03
mtreinishme17:03
mlavalleme17:04
afrittolime17:04
Nithyame17:04
afazekasme17:04
* afazekas sick and slow today17:04
sdague#startmeeting openstack-qa17:04
openstackMeeting started Thu Feb  7 17:04:47 2013 UTC.  The chair is sdague. Information about MeetBot at http://wiki.debian.org/MeetBot.17:04
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:04
*** openstack changes topic to " (Meeting topic: openstack-qa)"17:04
openstackThe meeting name has been set to 'openstack_qa'17:04
*** danwent has quit IRC17:04
sdaguesweet, off we go17:04
*** ravikumar_hp has joined #openstack-meeting17:04
*** donaldngo_hp has joined #openstack-meeting17:05
sdagueok lets start with reviews17:05
sdague#topic outstanding reviews17:05
*** openstack changes topic to "outstanding reviews (Meeting topic: openstack-qa)"17:05
afazekashttps://review.openstack.org/#/c/20746/17:05
sdaguedoes anyone have particular reviews they are struggling with?17:05
sdague#link https://review.openstack.org/#/c/20746/17:06
sdagueafazekas: ok, where do we stand on that?17:06
afazekasNow we are skipping testing without a technical reason17:06
afazekas "SKIP: Need multiple users for this test." "SKIP: FlavorExtraData extension not enabled." appears in the log message before this patch17:07
sdagueok, so the issue being that our new skip decorator doesn't tell us why?17:07
afazekasThe generic_setup_package evaluated after the skip decision made17:08
sdagueok17:08
*** annegentle has quit IRC17:09
afazekaswhy it is not fixed as soon as possible ?17:09
sdagueok, lets get mordred and lifeless on that review to see if there is a testrepository fix to make this right17:09
sdaguewe'll also need jaypipes to drop his -217:10
afazekaswe have other solutions probably, but it can work now17:10
mordredsdague: aroo?17:10
sdaguemordred: if you could have a look at https://review.openstack.org/#/c/20746/17:10
mordred(looking)17:10
sdaguejay wanted your input17:10
davidkranzI seem to be confused by https://review.openstack.org/#/c/20681/ and hope another core reviewer can approve it or respond.17:11
*** markwash has joined #openstack-meeting17:11
mordredI will review that17:11
sdaguedavidkranz: I'll need to dive into that a bit deeper, haven't really looked at that one yet17:11
sdague#link https://review.openstack.org/#/c/20681/17:11
Nithyahttps://review.openstack.org/#/c/20901/, Where can we place the tests that are scenario based that are not a part of normal smoke / gating tests in tempest? I had submitted a blueprint for adding Nova VM lifecycle tests in tempests (modified version of tests/compute/servers/test_server_basic_ops.py). Reviewers have given a comment not to add it as a part of normal gating tests.17:12
davidkranzsdague: Thx17:12
sdague#link https://review.openstack.org/#/c/20901/17:12
davidkranzNithya: I think we need an attribute for non-gating tests.17:12
sdagueNithya: those should live in another directory, like the stress tests17:12
NithyaI will do the changes and submit a patch. Thank you17:12
afazekasNithya: we should create new folder for them17:13
sdagueI think that we need to have a Havana design session on exactly how to handle tests and attributes, as lifeless has added some attr support to testrepository now17:13
davidkranzsdague: ++, but we need some job to run these tests. Nightly?17:13
*** apmelton has left #openstack-meeting17:13
sdaguedavidkranz: I'm not convinced the lifecycle tests are something we want to run17:14
*** noslzzp has joined #openstack-meeting17:14
sdaguethey are way too stateful17:14
ravikumar_hpsdague: whoever wants to run , can run17:14
sdagueI'm ok with them being available, but I think in a real environment they are going to break a lot17:14
davidkranzsdague: If we don't want to run them then why are they in tempest?17:14
ravikumar_hpanyway it is not gated tests ,17:15
sdaguedavidkranz: we have stress/17:15
sdaguethose we don't run all the time17:15
sdagueor in any automated way17:15
afazekasdavidkranz: for home usage17:15
davidkranzsdague: Because we cant right now. But that was the goal.17:15
afrittoliin general I think it would be good to have a repo for non-gating tests17:15
mordredyes. I would like to eventually run stress tests in some sort of gating manner17:15
*** karinqe has joined #openstack-meeting17:15
*** noslzzp has quit IRC17:16
afrittolifor instance tests which are targeting multinode environments - e.g. scheduler tests17:16
afazekas\/nongation folder for these tests without any sub folder17:16
afrittolior race condition tests17:16
mordredmy wquestion would be what are the characteristics of the non-gating tests you are talking about that would make them non-gating17:16
sdaguemordred: too many false negatives17:16
mordredmultinode is something I intend to get working in the gate17:16
davidkranzIMO, nongating == "slow of flaky"17:16
davidkranz  ^^^ or17:16
sdagueyeh, basically17:16
mordredok. I'm TOTALLY fine with "slow or flaky" as the definition17:17
sdaguewe had to do a lot of work to make current tempest extremely deterministic17:17
afazekasI would like to create tests which has higher chance to cause flaky issues, with high performance, but still in python17:17
sdagueso things which we think aren't deterministic, we need to keep out of the gate17:17
mordred++17:17
afrittoliactually running those tests in a non-gating job would highlight whether those tests are slow and/or flacky]17:17
sdagueafazekas: yes, agreed17:17
*** Shree-HPCS has joined #openstack-meeting17:17
sdagueafrittoli: fair17:18
davidkranzsdague: We need to have a way to "incubate tests" in non-gating to be moved to gating when stable.17:18
afrittoli++17:18
Nithya++17:18
sdaguehowever, I'm going to table the philosophy for Havana summit if we could17:18
davidkranzsdague: Sure.17:18
sdagueI think we need a session on this, but there is a lot to discuss, especially details wise, so lets do it there :)17:18
afazekasI think they can live in the same git repository anyway17:18
sdagueafazekas: yes, agreed17:18
mordredsdague: ++17:19
sdagueok, so I wanted to chat about this - https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/speed-up-tempest,n,z17:19
sdague#link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/speed-up-tempest,n,z17:19
sdagueafazekas brings up the good point that we should make sure we don't loose the smoke functionality as that comes in before testr is ready17:20
sdagueeven though we aren't running it17:20
sdagueso I'm going to propose to the list (as soon as I get time) that chris redo those as a series, and wrap the decorator so that it does the right thing for nose and testr17:20
sdagueso it's all linked together17:20
sdagueafazekas that sound reasonable to you?17:21
*** dprince has quit IRC17:21
*** iben has joined #openstack-meeting17:21
afazekasok17:21
*** davidha has quit IRC17:21
sdagueok cool17:22
sdagueafazekas: so if you can rebase this https://review.openstack.org/#/c/20091/ - that can go in17:22
sdaguelooks like jenkins just failed to merge it17:22
*** noslzzp has joined #openstack-meeting17:23
sdagueotherwise people should get Jenkins to pass, I'm not looking at any Jenkins fails. :)17:23
sdagueand I think those are most of the outstanding reviews17:23
afazekassdague: I need to violate the T302 rule, since otherwise I got circular issue17:23
sdagueafazekas: do you have an example of where that happened?17:23
*** kmartin_zz is now known as kmartin17:24
afazekasI'll trace it , but it is because of the __init__ and clients cross references. (some part was function originally in another location, but I had to move them on a review)17:25
*** iben has quit IRC17:25
afazekasthe big T302 patch merges the clisnts.py to the __init__17:25
sdagueok, lets figure that one out, would be good to get to the bottom17:25
sdagueok, anything else on reviews?17:26
sdaguegoing once...17:26
sdaguegoing twice...17:26
donaldngo_hphttps://review.openstack.org/#/c/20681/17:26
donaldngo_hpI'd like to talk about that review17:26
sdagueok, davidkranz did bring it up before, but go for it17:27
donaldngo_hpCurrently the Keystone API does not return a token for /v3/token17:27
donaldngo_hpso currently the submitted review is to create a new v3restclient17:27
ravikumar_hpdonald_hp: right17:28
ravikumar_hpno changes for token in v317:28
ravikumar_hpit is same as v217:28
donaldngo_hpis /v3/token going to be implemented?17:28
*** EmilienM has left #openstack-meeting17:29
ravikumar_hpsince no tests exist for token, we are trying to submit tests . we need to check version . but nothing changed for V317:29
sdagueravikumar_hp: is devstack bringing up v3 in the gate? (I thought I saw dean doing something with that recently)17:29
*** topol has quit IRC17:30
sdaguedonaldngo_hp: ok, will take some time to look at it17:30
donaldngo_hpso for that review all tests depending on the /v3/ keystone api will need to use the new restclient api17:30
ravikumar_hpsdague: not sure17:30
donaldngo_hpsorry not api but implementation like /v3/domain17:30
sdaguealso, for people asking for reviews... if you could spend some time reviewing other patches in the queue, those of us with +2 would have more to go on17:30
sdagueok, lets move on from reviews17:31
sdague#topic coverage and additional tests17:31
*** openstack changes topic to "coverage and additional tests (Meeting topic: openstack-qa)"17:31
sdaguemtreinish, how about you talk about the coverage analysis and some of the additional test lists17:31
mtreinishsdague: ok17:31
*** Mandell has joined #openstack-meeting17:32
mtreinish#link https://etherpad.openstack.org/MissingTempestTests17:32
mtreinish#link https://etherpad.openstack.org/coverage-analysis17:32
mtreinishso using the results from the periodic coverage runs I've done some analysis of gaps in the tempest tests17:32
*** psedlak has joined #openstack-meeting17:32
mtreinishthen compiled a list of proposed tests to fill the gaps17:33
mtreinishthe list is pretty big so far, but it is still incomplete17:33
mtreinishtimello, got the first patch merged for this effort17:33
mtreinishso if people want to tackle one just mark your name next to a test name on the list17:34
sdagueawesome17:34
donaldngo_hpnice17:34
donaldngo_hpwhat does "test_multiple_create" do?17:34
sdagueserver create allows you to create more than one guest at a time17:35
mtreinishdonaldngo_hp: multiple create is an extension that does multiple server creates in one action17:35
afrittolijust to understand, this is collecting coverage from the API servers, but not from compute yet? I've seen a blueprint on nova side to enable a backport in compute for coverage report... is this in place and used already?17:35
mtreinishafrittoli: yeah this just based on the api server.17:36
afrittolifor instance coverage for the virt driver is zero, so I assume that's not collected17:36
afrittolimtreinish: ok thanks17:36
mtreinishthe coverage extension supports using backdoor ports for other services but its not turned on in the periodic run yet17:36
donaldngo_hpare we continuing to accept negative tests?17:37
donaldngo_hpmost of the needed tempest tests on the etherpad are for negative tests17:37
sdaguedonaldngo_hp: yes, negative testing is important17:37
mtreinishdonaldngo_hp: that is where the biggest gaps were17:37
ravikumar_hpsdague: some time ago hold was put for negative tests17:37
sdaguewe exposed some interesting nova bugs because of them17:37
donaldngo_hpin past we put a hold on negative tests17:38
donaldngo_hpand there was talk about fuzz testing17:38
mtreinishafrittoli: http://wiki.openstack.org/Nova/CoverageExtension gives some info on setting up the coverage extension17:38
ravikumar_hpwe have added lot of negative tests17:38
sdagueat this point no one's stepped up to do fuzz testing, so I'm all for negative tests17:38
sdagueespecially as they tend to be pretty cheap on execution17:38
ravikumar_hpsdague: ok17:39
afazekasnegative test are fast if we do not need a booted machine..17:39
sdague#topic open discussion17:40
*** openstack changes topic to "open discussion (Meeting topic: openstack-qa)"17:40
sdagueok, other topics of note, or anything else people want to chat about?17:40
mtreinishI'd like to bring up the glance client discussion17:41
afazekaspython -c "import this"17:41
afazekas"Flat is better than nested."17:41
sdague#topic glance client17:41
*** openstack changes topic to "glance client (Meeting topic: openstack-qa)"17:41
afazekasour code structure os more like java style than python now17:41
mtreinishso there is a ML started on this at: http://lists.openstack.org/pipermail/openstack-qa/2013-February/000199.html17:41
mtreinishthe big open question is whether using the http lib from python-glanceclient is different enough for writing a tempest glance client17:42
mtreinishor is it too similar to testing using python-glanceclient (like we do currently)17:42
sdaguewhat does http lib give us?17:42
sdagueis that the chunking implementation?17:42
davidkranzsdague: Back. The hold on negative tests was for the sort that were written the same as positive tests.17:43
mtreinishsdague: that is in it. which is the main motivation for wanting to use it17:43
sdagueif we can just take the chunking implementation, I'm ok with that17:43
mtreinishsdague: https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/http.py17:43
afazekasmtreinish:Do you get the merged response and the staus code ?17:43
davidkranzsdague: THe idea was to have negative tests be expressed more concisely and declaritively.17:43
davidkranzsdague: Like in fuzz testing.17:44
*** rellerreller has joined #openstack-meeting17:44
sdaguedavidkranz: ok, so that's probably a design summit session as well17:44
sdaguethe reality is, no one stepped up to do fuzz testing in grizzly17:44
mtreinishafazekas: I don't think so, we'd have to wrap around the response to convert it17:44
davidkranzsdague: daryl said they had something almost ready to submit.17:44
*** jog0 has joined #openstack-meeting17:45
sdagueso I'd say lets move forward with actually adding tests for release17:45
davidkranzsdague: We should ping him about that.17:45
ravikumar_hpsdague: i have a question on test submission for incubated projects like load balancer as service or database as service17:45
afazekasDo we want to test is the chunked encoding well formatted or just accepted by the lib, and working for some reason ?17:45
ravikumar_hpcan we submit ? or wait until those become core projects?17:45
sdagueravikumar_hp: those at least need to end up in a seperate directory17:46
ravikumar_hpsdague: that sounds good17:46
sdagueafazekas: that's a good question17:46
afrittolido we have a way to maintain test suites?17:46
mtreinishafazekas: it was more for functional testing of the glance api. I wasn't planning on verifying the chunk encoding formatting.17:46
sdaguemtreinish: is there another upstream python lib that does the chuncking?17:46
sdagueafrittoli: I'm not sure I understand the question17:47
afazekasif the answer is no, we do not need to reinvent the wheel.17:47
sdagueafazekas: I think we're mostly concerned with API testing at this point17:47
mtreinishsdague: yeah httplib works fine with chunking. (That's what the glanceclient lib uses)17:47
sdagueso lets solve the first issue that we don't test the glance api at all17:47
sdaguebecause we always use python-glanceclient17:48
donaldngo_hpafrittoli: attributes are a way to maintain test suites17:48
sdaguethen if we decide later we don't want to trust the chunking implementation, we can tackle that17:48
afrittoliattributes are going away with testtools17:48
mtreinishsdague: ok, sounds reasonable17:49
sdagueafrittoli: they will still be there, but different17:49
afrittolisdague: ok that's great17:49
afazekastesttools: not fully, They implemented something..17:49
afazekasafrittoli:  not fully, They implemented something..17:49
sdaguea big part of it is lifeless wants to see the use cases a little more to enhance the implementation17:49
sdaguethat will also be a portland conversation17:50
sdagueit's also something we could start to hash out on the mailing list17:50
afrittoliI thinkit would be good to have a list of tests (or folders) which are part of gating17:50
sdagueafrittoli: right, and right now that's tempest/tempest17:50
*** topol has joined #openstack-meeting17:51
sdagueas of last week we are gating on the full set17:51
*** anniec has joined #openstack-meeting17:51
afazekasafrittoli: just grep any jenkins log17:51
sdagueok, so as cyeoh, ivan, and lifeless are on the other side of the planet, and they are doing a lot of this, lets take this to the mailing list, as that's the right place to hash it out17:52
sdaguethis meeting is in the middle of the night for all of them17:52
sdagueok, mtreinish you have enough to move forward on glanceclient?17:52
mtreinishsdague: yeah I should17:52
sdaguecool17:52
sdague#topic open discussion17:52
*** openstack changes topic to "open discussion (Meeting topic: openstack-qa)"17:52
sdagueok, anything else from folks?17:52
afrittoliabout multinode tests17:53
afazekas?17:53
afrittolido we have plans to split  tests on multiple nodes, or to run them against a multinode environment17:53
*** rellerreller has quit IRC17:54
Shree-HPCS++ multinode gating test17:54
*** lglenden has joined #openstack-meeting17:54
afrittolithat could be a way of speeding-up on one side17:54
afazekasWe are using CPU time even before the tempest stating17:54
*** dprince has joined #openstack-meeting17:54
afrittoliand would allow us to run tests which make sense only on multinode environments17:55
sdagueI think on multinode someone just needs to start working on the approach with the CI team17:55
afazekasOne tempest instance can load a big environment, if it could run on multithread17:55
*** dhellmann has joined #openstack-meeting17:55
sdagueafrittoli: I encourage folks to work on that who are interested17:55
sdagueI think getting us to full gate in grizzly was a huge step forward17:56
sdagueand that would be another great step forward17:56
sdagueok, we're about at the end of our time17:56
sdagueanything else from folks?17:56
afrittolisdague: yes going full gate was a great step forward indeed17:56
*** hemna has quit IRC17:56
*** colinmcnamara has joined #openstack-meeting17:56
*** dprince has quit IRC17:57
afazekaspython way of code structuring17:57
afrittolilast question about pep8, there are a lot of strict requirements such as alphabetical order of imports17:57
sdagueafazekas: ok, go ahead17:57
afrittoliafazekas: sorry I got in the middle17:57
sdagueafrittoli: no worries17:57
afazekasthe python python -c "import this"  tells us the basic rules17:58
afazekas"Flat is better than nested."  means we should use less folder17:58
*** cjd_ has joined #openstack-meeting17:58
afazekaswe should not put a single class to single file17:58
*** markmcclain has joined #openstack-meeting17:58
sdagueafazekas: I don't really read it that way17:58
sdagueI read it as complex nesting in a file17:59
afazekasWe should have duplicated word in a an absolute path17:59
sdaguethe directory and file structure I think is the least of my concerns right now :)17:59
*** mtesauro has joined #openstack-meeting17:59
afazekasyes , it has multiple explanation :) , but generally we have too long absolute paths we repeated words17:59
*** rellerreller has joined #openstack-meeting18:00
sdagueafazekas: ok, how about we take it to the mailing list, as we're kind of out of time18:00
afazekasif we start applying the T302 style guide is will be more obvious18:00
afazekasok18:00
sdagueafazekas: ok18:00
sdagueok, I'm going to call it a meeting18:00
sdaguefollow on discussions, jump on #openstack-qa18:00
sdagueor the mailing list18:01
sdaguethanks everyone18:01
sdague#endmeeting18:01
*** openstack changes topic to "OpenStack meetings || Development in #openstack-dev || Help in #openstack"18:01
openstackMeeting ended Thu Feb  7 18:01:07 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_qa/2013/openstack_qa.2013-02-07-17.04.html18:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_qa/2013/openstack_qa.2013-02-07-17.04.txt18:01
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_qa/2013/openstack_qa.2013-02-07-17.04.log.html18:01
*** mlavalle has left #openstack-meeting18:01
*** bru has joined #openstack-meeting18:01
*** davidha has joined #openstack-meeting18:03
*** sarob has joined #openstack-meeting18:04
*** anniec has left #openstack-meeting18:04
*** anniec has joined #openstack-meeting18:04
*** nati_ueno has joined #openstack-meeting18:04
*** sarob has quit IRC18:04
*** sarob has joined #openstack-meeting18:05
*** danwent has joined #openstack-meeting18:05
*** estebang9 has joined #openstack-meeting18:06
mtesauroAnybody here for the OSSG meeting?18:06
cjd_yes18:07
rellerrellerI am18:07
estebang9yes18:07
lglendenyes18:07
mtesauroI guess lets give bdpayne a couple of minutes  - otherwise, I say lets hit the agenda18:08
mtesaurolink: http://wiki.openstack.org/Meetings/OpenStackSecurity18:08
bdpayneahh, sorry guys!18:08
bdpayneteaches me to ignore my calendar18:08
bdpayne#startmeeting OpenStack Security Group18:08
openstackMeeting started Thu Feb  7 18:08:48 2013 UTC.  The chair is bdpayne. Information about MeetBot at http://wiki.debian.org/MeetBot.18:08
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.18:08
*** openstack changes topic to " (Meeting topic: OpenStack Security Group)"18:08
openstackThe meeting name has been set to 'openstack_security_group'18:08
bdpayneOk, let's begin18:09
bdpayne#topic Storage Encryption Status18:09
*** openstack changes topic to "Storage Encryption Status (Meeting topic: OpenStack Security Group)"18:09
*** hemna has joined #openstack-meeting18:09
bdpayneCould the parties involved in the storage encryption efforts provide a status update?18:09
bdpayneIntel and APL, if you're here18:09
rellerrellerVolume encryption code has been submitted, but we are still awaiting reviews and acceptance18:10
lglenden#link https://review.openstack.org/#/c/21269/18:10
mtesauroBTW, I have a bunch of feedback - enough that its not right for IRC, any idea on the mail list?  That would be a good place for this.18:10
lglenden#link https://review.openstack.org/2126418:11
lglenden#link https://review.openstack.org/2126218:11
*** karinqe has left #openstack-meeting18:11
rellerrellerWhat do you mean by any idea on mail list?18:11
bdpayneok, I'm happy to look over the code18:11
mtesauroIn our first IRC meeting, it was mentioned that a mail list was being setup for this group.18:12
bdpaynebut I assume you need reviews from specific parties?18:12
bdpayneahh, for this let's just use the general dev list18:12
rellerrellerI'm not exactly sure of the approval process.  I need core reviewers.18:13
rellerrellerBut I appreciate feedback from everyone!18:13
bdpayneok, so here's what I will do:18:13
mtesauroI have ping'ed the Swift developers we have at Rack - I'm waiting a response.18:13
bdpayne#action Bryan to review the APL code18:13
mtesauroI am book with threat models but can walk over there on Friday18:13
bdpayne#action Bryan to figure out formal review process by talking with PTLs and help move that along18:13
*** garyk has joined #openstack-meeting18:13
*** sarob has quit IRC18:14
rellerrellerThanks!18:14
*** bdperkin has joined #openstack-meeting18:14
bdpaynenp18:14
*** derekh has quit IRC18:14
rellerrellerOur spec for volume encryption will help to give you an idea for our code design18:14
*** eglynn_ has quit IRC18:14
bdpaynesounds good18:15
bdpayneany other things we should know about the APL side?18:15
*** bdperkin has joined #openstack-meeting18:15
bdpayneif not, anyone from Intel here want to provide a status update?18:15
rellerrellerI can't think of anything off hand18:15
*** hyakuhei has joined #openstack-meeting18:16
lglendenI think it's also important to mention that, especially on the key management front, this is a first version to get people able to try out our code18:16
bdpayneok18:16
lglendenWe know that we haven't solved all the key management issues yet.18:16
bdpaynewith that said… are you wanting to get this into Grizzly or ?18:16
rellerrellerYes, grizzly is our goal18:16
bdpayneok18:16
bdpaynealright, I'm not hearing from Intel, so we can move on to the next topic18:17
bdpayne#topic Hardening Guide18:17
*** openstack changes topic to "Hardening Guide (Meeting topic: OpenStack Security Group)"18:17
bdpayneThe hardening guide hasn't move forward much since last week.  I've been a touch busy/18:17
hyakuheiDid you take a look at the small changes I made to the outline?18:17
bdpayneNo, I missed that18:17
hyakuheiI can spend some time putting the .tex files into a shape that reflects the outline over the weekend.18:18
bdpayneHrm, I need to get better notifications setup, aparently18:18
bdpaynesounds great18:18
hyakuheiJust a few small changes, don't recall exactly. Need more time to spend on these things.18:18
hyakuheiIn other news, HP Cloud should be adding some content in the near future.18:18
bdpayneexcellent18:18
bdpayneI am planning to work on this some… just slowly bubbling up the stack18:19
hyakuheiNeed to get it out of this bootstrap phase.18:19
bdpayneand I'm, of course, open to other's getting involved as well18:19
hyakuheiAbsolutely!18:19
bdpaynesomeone from Red Hat expressed interest last week18:19
bdpayneI'll resync with him as well18:19
hyakuheiCool. I spoke to a few people who were interested at the weekend while at FOSDEM.18:19
mtesauroBTW, the OWASP OpenStack Security project was announced yesterday to ~36,000+ community members18:20
bdpaynenice!18:20
mtesauroI've already gotten some requests to join the project.  We'll see what happens by the next meeting18:20
bdpayne#topic Open Discussion18:20
*** openstack changes topic to "Open Discussion (Meeting topic: OpenStack Security Group)"18:20
bdpaynemtesauro can you provide details on that effort?18:21
bdpayneshould this group and that group sync in some way or ?18:21
hyakuheihttps://www.owasp.org/index.php/OWASP_OpenStack_Security_Project18:21
hyakuheiSeems like mtesauro is the guy to talk to/sync with.18:21
bdpayne#link https://www.owasp.org/index.php/OWASP_OpenStack_Security_Project18:21
mtesauroYup that's the URL.  I need to add some real content and I got the mail list info this AM18:21
mtesauroIt is basically an attempt to draw people from OWASP into OpenStack to help with security testing, review etc.18:22
mtesauroI18:22
bdpayneyeah, very much needed18:22
hyakuheiSounds like a good idea.18:22
mtesauroI've been in both groups and there's a good opportunity for great interactions18:22
bdpaynesounds good… please do keep this group posted and let us know if there's anything we can do to help facilitate that work18:23
mtesauroNo problem.  I'm going to give it a couple of days for people to join then rally the troups.18:23
bdpayneok18:23
mtesauroSome of what happens will depend on the skill set/interest of the people from OWASP18:24
mtesauroI'm not worried about running out of work18:24
bdpayneAlso wanted to briefly mention the Security Note on LXC since hyakuhei is here18:24
bdpaynePer discussion at last week's meeting, I think that the note is ready to go, you want to push it out or ??18:24
bdpayneyou == hyakuhei :-)18:25
hyakuheiSure I'll do it tomorrow with any luck18:25
bdpaynesounds good18:25
bdpayneany other discussion?18:25
hyakuheiMailed to the -dev ML with a little boilerplate explaining what an OSN is etc.18:25
hyakuheiI've no other business other than apologies for being late.18:26
*** dprince has joined #openstack-meeting18:26
bdpayneok, sounds good18:27
bdpayneI was late too… it happens!18:27
bdpaynethanks everyone!18:28
mtesauroNo worries.18:28
bdpayne#endmeeting18:28
*** openstack changes topic to "OpenStack meetings || Development in #openstack-dev || Help in #openstack"18:28
openstackMeeting ended Thu Feb  7 18:28:11 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:28
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_security_group/2013/openstack_security_group.2013-02-07-18.08.html18:28
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_security_group/2013/openstack_security_group.2013-02-07-18.08.txt18:28
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_security_group/2013/openstack_security_group.2013-02-07-18.08.log.html18:28
*** hyakuhei has quit IRC18:29
*** cjd_ has quit IRC18:29
*** bdpayne has quit IRC18:30
*** joesavak has joined #openstack-meeting18:30
*** bru has quit IRC18:30
*** bdpayne has joined #openstack-meeting18:31
*** lglenden has left #openstack-meeting18:31
*** jsavak has quit IRC18:33
*** mtesauro has left #openstack-meeting18:33
*** zvm-vlam has joined #openstack-meeting18:35
*** eglynn_ has joined #openstack-meeting18:39
*** donaldngo_hp has quit IRC18:41
*** lbragstad has joined #openstack-meeting18:42
*** markmcclain has quit IRC18:47
*** ravikumar_hp has quit IRC18:48
*** eglynn_ has quit IRC18:52
*** martine has joined #openstack-meeting18:53
*** colinmcnamara has quit IRC18:53
*** dprince has quit IRC18:56
*** YehiaBeyh has quit IRC18:57
*** nati_ueno has quit IRC18:58
*** nati_ueno has joined #openstack-meeting18:58
*** mdomsch has joined #openstack-meeting18:59
*** Nachi has joined #openstack-meeting19:02
*** rellerreller has quit IRC19:02
*** nati_ueno has quit IRC19:05
*** pony has joined #openstack-meeting19:07
*** dprince has joined #openstack-meeting19:09
*** markmcclain has joined #openstack-meeting19:09
*** Shree has joined #openstack-meeting19:10
*** novas0x2a|laptop has quit IRC19:10
*** Shree-HPCS has quit IRC19:11
*** markvoelker3 has quit IRC19:11
*** markvoelker1 has joined #openstack-meeting19:11
*** cp16net is now known as cp16net|away19:19
*** cp16net|away is now known as cp16net19:21
*** otherwiseguy has joined #openstack-meeting19:26
*** vipul is now known as vipul|away19:27
*** darraghb has quit IRC19:29
*** Hg_ has joined #openstack-meeting19:31
*** Hg_ has quit IRC19:31
*** nati_ueno has joined #openstack-meeting19:31
*** annegentle has joined #openstack-meeting19:33
*** HenryG has quit IRC19:34
*** Nachi has quit IRC19:34
*** sarob has joined #openstack-meeting19:35
*** HenryG has joined #openstack-meeting19:35
*** dprince has quit IRC19:37
*** annegentle has quit IRC19:37
*** annegentle has joined #openstack-meeting19:38
*** vipul|away is now known as vipul19:40
*** nati_ueno has quit IRC19:41
*** rafaduran has quit IRC19:41
*** nati_ueno has joined #openstack-meeting19:42
*** Nithya has quit IRC19:44
*** koolhead17 has quit IRC19:45
*** Shree has quit IRC19:47
*** cp16net is now known as cp16net|away19:49
*** bdpayne has quit IRC19:50
*** stevebaker has joined #openstack-meeting19:50
*** bdpayne has joined #openstack-meeting19:50
*** dhellmann has quit IRC19:53
*** novas0x2a|laptop has joined #openstack-meeting19:55
*** dwcramer has quit IRC19:55
*** cp16net|away is now known as cp16net19:56
*** reed has quit IRC19:58
*** reed has joined #openstack-meeting19:58
*** dhellmann has joined #openstack-meeting20:01
*** dkehn has joined #openstack-meeting20:01
*** jsavak has joined #openstack-meeting20:01
*** maurosr has quit IRC20:02
*** joesavak has quit IRC20:04
*** markvoelker has joined #openstack-meeting20:07
*** reed has quit IRC20:07
*** afazekas has quit IRC20:09
*** Shree-HP has joined #openstack-meeting20:09
*** markvoelker1 has quit IRC20:10
*** esker has quit IRC20:10
*** esker has joined #openstack-meeting20:11
*** vipul is now known as vipul|away20:11
*** topol_ has joined #openstack-meeting20:12
*** bencherian has quit IRC20:14
*** topol has quit IRC20:16
*** topol_ is now known as topol20:16
*** henrynash has quit IRC20:19
*** sdake has quit IRC20:20
*** vipul|away is now known as vipul20:22
*** cp16net is now known as cp16net|away20:23
*** danwent has quit IRC20:24
*** cp16net|away is now known as cp16net20:24
*** lloydde has joined #openstack-meeting20:25
*** anniec has quit IRC20:26
*** markmcclain has quit IRC20:30
*** Nachi has joined #openstack-meeting20:31
*** nati_ueno has quit IRC20:32
*** dwcramer has joined #openstack-meeting20:32
*** vipul is now known as vipul|away20:37
*** gyee has joined #openstack-meeting20:37
*** dwcramer has quit IRC20:38
*** sdake has joined #openstack-meeting20:40
*** markmcclain has joined #openstack-meeting20:40
*** eglynn_ has joined #openstack-meeting20:49
*** danwent has joined #openstack-meeting20:52
*** ndipanov has joined #openstack-meeting20:55
*** sarob has quit IRC21:02
*** anniec has joined #openstack-meeting21:04
*** sandywalsh_ has joined #openstack-meeting21:04
comstudmeow21:04
devananda\o21:05
*** sarob has joined #openstack-meeting21:05
dansmithpoop21:05
*** sandywalsh has quit IRC21:05
dansmithvishy is always about 15 minutes late, so no need to panic yet :)21:06
*** noslzzp has quit IRC21:06
*** sarob_ has joined #openstack-meeting21:07
sdagueheh21:08
sdaguewell, who's here for the nova meeting?21:09
sdagueat least we can figure out how close to quorum we are21:09
dansmith#startmeeting21:09
openstackdansmith: Error: A meeting name is required, e.g., '#startmeeting Marketing Committee'21:09
dansmith#startmeeting nova21:09
openstackMeeting started Thu Feb  7 21:09:43 2013 UTC.  The chair is dansmith. Information about MeetBot at http://wiki.debian.org/MeetBot.21:09
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:09
*** openstack changes topic to " (Meeting topic: nova)"21:09
openstackThe meeting name has been set to 'nova'21:09
dansmithhe'll be here21:09
dansmithI'll #chair him when he shows up21:09
sdaguecool21:10
jog0o/21:10
sdagueo/21:10
dansmithalthough there's no agenda apparently21:10
*** sarob has quit IRC21:10
comstuddo not commit future versions of oslo21:11
dansmithheh21:11
dansmithcomstud: any chance rackspace could throw resources at the tempest gate problem?21:11
*** Mr_T has joined #openstack-meeting21:11
comstudnot sure21:11
dansmith1.5 hours to get a read on any patch is pretty brtual :(21:11
comstudwill ask21:11
dansmithand the way zuul works,21:11
dansmiththat can easily become six hours if you're unlucky and behind some bad patches21:12
sdaguedansmith: you can always see the zuul work in progress21:12
sdaguehttp://status.openstack.org/zuul21:12
dansmithsdague: ...yeah of course21:12
dansmithsdague: but that does nothing for getting a read on whether the full run is going to pass,21:12
dansmithwhich is always the last one21:12
westmaasdansmith: do we know where the problem is? spinning up envs? running the tests?21:13
westmaassorry, just catching up21:13
*** jerdfelt has joined #openstack-meeting21:13
comstuddansmith: look, i brought resources.21:13
dansmithwestmaas: I think sdague knows more, but I think it's mostly cores and memory, surprise :)21:13
sdagueit's just that we run a lot of tests now, and dansmith is impatient21:13
comstudlol21:13
westmaashaha21:14
sdaguetempest will get quicker once the testr conversion happens over the next couple weeks21:14
dansmithif I'm the only one that thinks it's frustrating,21:14
sdaguebut we'll also have to look at bumping devstack node sizes21:15
jog0dansmith: I agree with you21:15
dansmiththen that's fine, but I go days sometimes without being able to get something to a mergeable state because I'm always waiting for runs21:15
westmaassdague: do you know what size you use now?21:15
sdaguewhatever is the 4G nodes21:15
westmaasand the time is spent on test running, right?21:15
dansmithand it compounds with every change you have stacked on top of another, which is often where I'm at21:15
dansmithit's not as big of a deal for individual patches21:15
westmaasnot on bringing up the nodes21:15
sdaguewe get past devstack in about 9 minutes21:16
*** eglynn_ has quit IRC21:16
sdaguethere is detailed timing in all the logs21:16
dansmithsdague: devstack setup is not parallelized, right?21:17
sdaguenope21:17
dansmithI wonder if some of it could be?21:17
sdaguerealize, we spent a lot of time squeezing out time from those runs21:17
dansmithI'm just asking21:17
sdaguetotally happy for other people to look at it as well, but there's not much low hanging fruit in there :)21:18
dansmithsdague: if the instances are 2 or 4 cores right now,21:18
westmaasnode size could be the biggest thing unless its mosly about waiting idly for nodes to spin up21:18
dansmiththen the tempest testr change isn't going to help all that much,21:18
sdagueI think they are 2 cores21:18
dansmithwhich is why I was asking about larger nodes21:18
sdaguewestmaas: the tempest tests wait for guests to spin up21:18
sdaguethat's where a lot of time is21:18
sdaguewhich is why testr will help21:19
dansmithsdague: but chris was saying that the speedup is 50% of your core count, right?21:19
sdaguewe'll see what 2 cores does21:19
*** dims has joined #openstack-meeting21:20
westmaasguests might spin up a little faster on larger nodes.  if someone wants to reach out to me offline about using more RS resources we can do that, would just want to understand how much we are using first.21:20
dansmithokay, I thought he said that with 2 cores, four testr threads would actually go slower21:20
sdaguewith 2 cores there won't be 4 testr threads21:20
*** cp16net is now known as cp16net|away21:20
dansmithsdague: you can run it with as many threads as you want, of course21:20
sdaguedansmith: instead of speculating, why not wait for the actual testr review to hit tempest, then we'll have real data21:20
dansmithsdague: he was trying to see if there was really any idle time to gain by overcommitting21:21
*** cp16net|away is now known as cp16net21:21
dansmithyes, lets linearize the process!21:21
dansmithanyway, I guess vishy isn't going to show up after all21:21
sdagueI mean the easy answer is just turn back off a lot of tests, but the new tests already blocked breaks going in21:21
dansmithsdague: has anyone suggested that? I sure haven't :)21:22
sdague:)21:22
*** stack_learner has joined #openstack-meeting21:23
dansmithokay, so do we bail? No agenda, no rustlebee, no vishy21:23
dansmith(and no-db-compute, by the way)21:23
comstudhehe21:23
jog0one quick comment: I have my BP ready for review  https://blueprints.launchpad.net/nova/+spec/db-api-cleanup https://review.openstack.org/#q,topic:bp/db-api-cleanup,n,z21:24
dansmithjog0: yeah, I've been meaning to look at it21:24
dansmithjog0: I'd ask why so many of them are failing jenkins, but I know why :)21:24
jog0one of the patches is blocked on getting code into oslo.21:25
jog0and the new tempest tests have been very helpful21:25
jog0it seems nova/api/openstack/compute/contrib/security_groups.py has very little unit tests21:25
dansmithyes, they found the last item in quantum_api for me :)21:25
dansmithjog0: anything else?21:27
jog0dansmith:  nope21:27
dansmithanyone?21:27
dansmithbueller?21:27
comstudjust a comment maybe21:28
*** Yada has quit IRC21:28
comstuddb pooling becomes important when moving to conductor... yet we have something broken with our use of sqlalchemy that prevents db pooling from working..  seems that it's leaving transactions open.. thus innodb locks held.21:29
*** mdomsch has quit IRC21:29
comstudqueries end up timing out waiting on locks in mysql21:29
comstudworking on tracking it down.21:29
jog0devananda: ^21:30
dansmithcomstud: why is this not showing up in devstack? does it take a while to crop up?21:30
comstudwe were having a discussion yesterday or the day before21:30
sdaguecomstud: so how broken are things with the oslo sync that wasn't? I didn't even occur to me that markmc might have inbounded non committed oslo code :)21:30
comstuddoes devstack have db pooling on?21:30
sdaguecomstud: I don't think so21:30
comstudsdague: most of the things out of sync have merged now it appears21:30
comstuddansmith: it seems to only show when db pooling is on21:30
comstudhttps://review.openstack.org/#/c/21461/21:30
dansmithcomstud: is that something we should enable? or would it be too hard to track down issues?21:30
comstud^^ this is an improvement needed also to be committed to oslo and merged into nova21:31
uvirtbotcomstud: Error: "^" is not a valid command.21:31
comstudotherwise you get connect timeouts under load, etc21:31
*** dhellmann has quit IRC21:31
comstuddansmith: it's certainly not ready yet21:31
comstudi dunno if devstack provides enough parallelism and load to see the issue anyway21:31
dansmithokay21:32
comstudit's easy to reproduce in a test script that slams DB requests with pooling on21:32
dansmithand conductor makes the problem worse because it takes the db hit for everyone and just hits the problem faster?21:32
sdaguefrom a gate perspective, I don't think there is enough db traffic to have it make a difference21:32
jog0comstud: I wasn't able to see any difference in tempest or smoketests in devstack with db pool on21:32
sdagueyeh, I'm not surprised jog021:32
comstuddansmith: i have not run with nova-conductor yet, but I can tell you that yes it will.. because of that reason21:32
dansmithcomstud: okay21:32
comstuddansmith: I see it with global nova-cells which takes DB updates from all child cells21:33
comstudconductor and global nova-cells have nearly the same function21:33
dansmithI'm embarrassed to say that I don't really know what this means.. is this avoiding connection setup and teardown for every query?21:33
sdaguethe big timing issues in tempest are starting guests and cleaning them up, which we have to do carefully because it's easy to overrun memory in the gate21:33
comstudjog0: ya, I'm not surprised.. I don't think there's probably enough parallelism in devstack tests21:33
comstudso I think any perf issues you have are just something else21:34
comstudOh... one other thing to note :)21:34
comstudif you ditch sqlalchemy and do raw mysql queries with MySQLdb...21:34
comstudthe problems go away and things are SIGNIFICANTLY faster.21:34
dansmithheh21:34
comstudeven when doing serialized DB calls21:34
comstudwith no thread pooling.21:34
driptoncomstud: what about if you only use low-level sqlalchemy and not the top ORM layer?21:34
*** vipul|away is now known as vipul21:34
comstuddripton: I was going to try that next21:35
driptoncomstud: +121:35
dansmithwe're getting closer to not even needing the ORM layer for much, right?21:35
comstudalthough I'm focusing on just trying to find where sqlalchemy is holding transaction open21:35
dansmithwith jog0's patches21:35
sdaguewell you could at least write a non-sqlalchemy version of the api21:35
sdagueafter those21:35
comstudyea21:36
driptondansmith: we currently use the ORM layer all over, and I don't think jog0's patches change that21:36
comstuddansmith: what dripton said21:36
dansmithhrm..21:36
dansmithI don't get that21:36
jog0dripton: the patch prevents sqlalchemy  objects from being returned from nova.db.api functions21:36
*** adjohn has joined #openstack-meeting21:36
comstudIn a way it gets you closer, I guess21:37
jog0after that removing ORM should be easier21:37
driptonjog0: right but we still use ORM inside the api functions21:37
jog0dripton:  yeahhhh21:37
jog0for now21:37
comstudyou need to rewrite almost all of sqlalchmey/api.py :)21:37
driptonjog0: okay, we agree.  Not gone yet but maybe in H21:37
comstudstill21:37
sdaguedripton: yeh, but you now *could* write an alternate api implementation21:37
comstudit might as well be an alternate api impl21:37
comstudyes21:37
driptonsdague: agreed, we're going the right way21:37
driptonwe still need to prove that *low-level* sqlalchemy doesn't have the performance problem21:37
*** dwcramer has joined #openstack-meeting21:38
driptonhope not, because it buys us portability21:38
sdagueyeh, loosing portability would really suck21:38
comstuddepends on how low level we need to go21:38
comstudanyway, we'll see.  first thing is just trying to find what's broken21:38
comstudperfroamnce after21:38
sdaguecomstud: there are subtle typing things between even mysql and pg, so it's more than you might imagine21:38
comstudnod21:39
driptonmy db-archiving patch (not done yet) uses only low-level sqlalchemy.  I was worried someone might complain about that but it looks like we're headed that way anyway.21:39
sdaguebut maybe do custom things for just a few of the most expensive bits21:39
comstudI was considering starting a mysql only api implementation21:39
comstudwhich falls back to sqlalchemy for methods not implemented yet21:40
jog0comstud: +121:40
devananda++21:40
comstudwhich gives you a path to migrate if you're using myql21:40
comstudmysql21:40
driptoncomstud: before being that extreme please try with low-level sqlalchemy21:40
sdaguedripton: you mean just doing .insert().values().execute() kind of things?21:40
driptonsdague: yes21:40
comstuddripton: nod21:40
sdaguedripton: yeh, I was doing that in the data injection for migrations, because I can't use the models21:40
sdagueit's not too bad21:40
sdagueit would be interesting if that sped us up21:40
devanandaa chunk of the issues i've seen appear to come from the way sqlalchemy automagically handles creates transactions21:41
devanandaeven when there's no explicit "with session.begin", there is still a transaction21:41
devanandadoing the lower level calls directly might be circumventing that?21:42
driptondevananda: I *think* you can go transactionless with low-level sqlalchemy, but I need to test21:42
driptonmy code is using "with session.begin"21:43
comstuddevananda: yeah, that's part of the problem I'm positive21:43
comstudi changed a DB call to remove session.begin()21:43
comstudand I don't get any deadlocks anymore... but I still get lock wait timeouts.21:43
comstudbut means sqlalchemy is not committing somewhere21:43
comstudAFAIK21:43
devanandaright21:43
devanandacomstud: the real fix to the stuff we were discussing is, IMNSHO, to add UNIQUE indexes21:44
*** ayoung has quit IRC21:45
devanandausing a myslq-specific extension (INSERT .. ON DUP KEY UPDATE) is then possible21:45
devanandaand pgsql has a similar capability which they call upserts21:45
devanandait's just a bit trickier (have to use a trigger or such)21:45
devanandasqlite is screwed tho :)21:45
*** esker has quit IRC21:45
driptonI don't think anyone would really use sqlite in production21:45
devanandatrying to enforce uniqueness with locking outside of the database is prone to all kinds of problems (like these)21:46
comstuddevandanda: unfort there's no portable way to do it in sqlalchemy21:46
comstudthe ON DUP KEY UPDATE21:46
sdaguecan we extend sqlalchemy for it?21:47
comstudbut even without that deadlock problem.. I'm getting the lock wait timeouts21:47
sdagueat the end of the day it's just a meta compiler21:47
comstudsdague: there was a way to compile it onto the end of a query21:47
sdagueseems like we could have an .insert_update method and backends21:47
comstudbut you'd need to figure out how to do it for pg, mysql, sqlite, etc :)21:48
devanandacomstud: IIUC, the lock wait timeouts are coming from the same thing -- >1 thread trying to lock the same rows with an UPDATE statement, when it really just wants to INSERT.21:48
comstuddevananda: and innodb doesn't detect a deadlock?21:48
sdaguecomstud: yeh, but if we need to figure that out anyway for openstack code, might as well monkey patch sqlalchemy for it21:48
driptonI've done it with sqlalchemy the  manual way: inside a transaction, select, if found then update else insert.21:48
driptonugly but portable21:48
devanandacomstud: if neither thread has the lock and both want it (and can't get it), then it deadlocks.21:49
sdaguedripton: yeh, if we could encapsulate it nicely, at least the ugly could live off in a method somewhere21:49
devanandacomstud: if one thread has the lock and just sits on it, other threads will timeout. so yes, that problem is sqlalchemy not committing properly21:49
devanandabut it's holding the lock because it did an UPDATE on 0 rows21:49
comstudyeah, that's the issue it seems21:49
devananda(i think)21:49
comstudI'm trying to track down where it's sitting21:49
comstudand why21:49
devanandadripton: that "portable way" is precisely the cause of this problem :(21:50
driptonsigh21:50
devanandadripton: if the row doesn't exist, both trx will insert (and one should fail, but right now, won't because there's no UNIQUE key, leading to data inconsistency)21:50
driptonstackoverflow says sqlalchemy has session.saveorupdate, but that might be at the wrong level21:50
driptonCan we add unique keys wherever we need them?21:51
sdaguedripton: the sqlalchemy code isn't that bad, I'd say just read the code :)21:51
*** ndipanov has quit IRC21:51
devanandayes :)21:51
comstudi added them21:51
comstudin the method i was testing21:51
comstud(bw usage updates)21:51
comstudand it didn't help21:52
devanandaso uniques are possible now that soft_deletes set deleted=<id>21:52
devanandacomstud: it should help if you change the SQL21:52
*** sarob_ has quit IRC21:52
devanandacomstud: either use INSERT .. ON DUP KEY, or remove the SELECT and do this21:52
devanandatry:21:52
devananda  INSERT21:52
comstudyeah, ok, I had to... and I did get deadlocks to go away21:52
devanandaexcept Duplicate:21:53
devananda  UPDATE21:53
devanandawith no transaction around them21:53
comstudhowever, I was left with this lock wait timeout issue still.. which is a sep problem and I'm sure something in sqlalchemy21:53
devanandahmmm21:53
comstudthat only seems to happen when thread pooling21:53
devanandayea, that's odd then21:53
dansmithI think we've veered off nova meeting type stuff, anyone not in favor of ending?21:53
*** boris-42 has joined #openstack-meeting21:53
comstudya, sorry, was supposed to be a quick FYI ;)21:53
comstudgood discussion though21:54
*** dosaboy has quit IRC21:54
boris-42Hi all21:54
dansmithdon't stop discussing, I just want to end so I can run off at some point :)21:54
comstudnod21:54
dansmith#endmeeting21:54
*** openstack changes topic to "OpenStack meetings || Development in #openstack-dev || Help in #openstack"21:54
openstackMeeting ended Thu Feb  7 21:54:26 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:54
openstackMinutes:        http://eavesdrop.openstack.org/meetings/nova/2013/nova.2013-02-07-21.09.html21:54
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/nova/2013/nova.2013-02-07-21.09.txt21:54
openstackLog:            http://eavesdrop.openstack.org/meetings/nova/2013/nova.2013-02-07-21.09.log.html21:54
driptondb meeting in 6 minutes so we can resume comstud's discussion then21:54
*** jerdfelt has left #openstack-meeting21:54
comstudok21:54
comstudbio break then21:54
*** sarob has joined #openstack-meeting21:59
*** markvoelker has quit IRC22:00
devanandafolks back // want to continue the db discussion?22:02
comstudgo22:02
driptonhere22:02
devananda#startmeeting db22:02
openstackMeeting started Thu Feb  7 22:02:49 2013 UTC.  The chair is devananda. Information about MeetBot at http://wiki.debian.org/MeetBot.22:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.22:02
*** openstack changes topic to " (Meeting topic: db)"22:02
openstackThe meeting name has been set to 'db'22:02
*** ladquin has joined #openstack-meeting22:02
devananda(kinda lost my train of thought with the break, heh)22:03
comstudi have 2 separate issues22:03
comstudthough somewhat related22:03
comstudbut I dunno what the agenda here is..22:04
devanandaso i didn't have an agenda for today22:04
comstudok22:04
devanandalast meeting, we went over the status of all the db BP's, which was good, but i dont think we need to rehash that22:04
devanandathe problems you're seeing with pooling is pretty big IMO22:05
devanandaespecially how that will affect conductor22:05
comstudthey've uncovered at least one problem for sure, one with which you were already aware, I think22:05
comstudyeah22:05
comstudi'll recap22:05
comstudstarting with this first issue22:06
*** dolphm has quit IRC22:06
comstudwell, first...22:06
comstudWith the move to nova-conductor... we're now pushing DB queries to a single node (or set of nodes if you run multiple nova-conductors)...22:07
comstudbut in a large deployment, you have much less conductors than computes, likely.22:07
comstudYou don't want to have 1 per compute :)22:07
comstudThis means that DB pooling becomes important... as without it, DB calls block the whole python process.22:08
comstudThe RPC queue will back up because the service cannot keep up with all of the DB requests if it has to block on every single one without any sort of parallelism22:08
comstudGood so far?22:09
devanandayep22:09
comstudCools... so I've been doing testing with db pooling22:09
comstudWith the trunk code, under load, you can get some connect timeouts, etc.  This patch is needed:22:09
comstudhttps://review.openstack.org/#/c/21461/22:09
*** cdub has quit IRC22:09
comstudwhich changes how the thread pooling is done22:09
comstudAfter applying that...22:10
comstudissue #1) innodb reporting Deadlock22:10
boris-42Seems that DB pooling start working=)22:11
comstudwhen you get 2 bw_usage_update() requests in parallel for the same instance uuid (can be different mac addresses/networks), this happens22:11
comstudThis is due to doing UPDATE+INSERT in a single transaction22:11
devanandaiirc, it should also deadlock the first time any instance is provisioned for a new tenant, if that happens in parallel.22:11
devanandaand in a few places around fixed / floating IP allocation, if they happen fast enough in parallel22:12
comstudYeah, you had a list that reported a number of other spots where this could happen22:12
comstudbesides bw_usage_update()22:12
devanandaright. there are also some open bugs on LP about these now22:12
comstudIn any case... We've found that bw_usage_cache table should have a unique index on instance_uuid+mac+start_period, which it doesn't right now.22:13
comstudAnd that would allow us to do things like INSERT ON DUP KEY UPDATE22:13
comstudif we can find a portable way to do it with sqlalchemy :)22:13
driptonsounds like we should add the index regardless, while we hunt for the portable upsert22:14
comstudI tested a variation on this that removed the Deadlock22:14
comstudyes22:14
comstudI think we can solve that issue without *too* much trouble, however22:15
comstudissue #2:22:15
comstudEven after getting innodb Deadlocks to go away, I'm seeing queries getting stuck and timing out due to waiting on innodb locks.22:15
comstudThe only conclusion is that sqlalchemy is not committing a transaction... it's holding it somewhere22:16
devanandai dug through my notes and remembered that i found this issue a while back, too22:16
devanandahttps://bugs.launchpad.net/nova/+bug/1007038/comments/422:16
uvirtbotLaunchpad bug 1007038 in nova "Nova is issuing unnecessary ROLLBACK statements to MySQL" [Low,Confirmed]22:16
comstudbut I can only see this when thread pooling is on22:16
comstuddevananda: I was able to cause an extra rollback very easily..22:16
comstudkind of a side toopic22:17
comstudtopic22:17
devanandacomstud: it was happening in august last year. was thread pooling in at that time?22:17
comstudno22:17
comstudit seems to be our wrapping of DB exceptions22:17
devanandarather. it was happening when I disabled pool_reset_on_return22:17
comstudfrom what I could tell22:17
devanandahmmm22:17
comstudi had code such as this:22:17
comstudwith session.begin():22:17
comstud  try:22:18
comstud    do a query22:18
comstud   except DBDuplicateEntry:22:18
comstud   pass22:18
comstudand I noticed that 2 rollbacks would happen if there was an exception22:18
comstudand there'd be a error logged about it22:18
*** markmcclain has quit IRC22:18
devanandakinda makes sense22:18
devanandasqla is probably doing a rollback before the exception bubbles up to your code22:18
comstudit seems like a rollback would happen when the exception occurred22:19
comstudand then also when we exited the context manager.22:19
devanandaand then after you leave the with session context, another implicit rollback happens (because of pool_reset_on_return)22:19
*** jsavak has quit IRC22:19
devanandaright22:19
comstudyep!22:19
comstudexactly22:19
comstudthat's a side issue, I think22:19
comstudi'm concerned about where sqlalchemy seems to be holding a transaction22:19
devanandayea. me too22:20
comstudcausing these lock wait timeouts22:20
comstudI can cause the whole process to stall by slamming DB requests22:20
comstudafter meetings are done today, I was going to debug with eventlet backdoor22:20
comstudanyway, that's where I am with 'bugs' that show up with db pooling.22:21
comstudseparate issue, somewhat related:22:21
*** estebang9 has quit IRC22:21
comstudsqlalchemy is extremely slow compared to raw queries with MySQLdb.22:22
comstudAnd there are no lock wait timeouts, either22:22
comstud:)22:22
*** martine has quit IRC22:22
devananda:)22:22
driptoncomstud: I think that's only with the ORM.  Low-level sqla is pretty fast IMX22:22
comstudit very well might be, I just haven't tested it yet22:22
comstudbelliott and I have been working on this together22:22
devanandaby raw queries, you mean model.insert? or session.execute("INSERT ... INTO table")22:23
devanandajust want to be clear22:23
comstudmysqldb .cursor(....)22:23
driptonthe fast ones are session.execute, but it could be the Python insert() not necessaily the string insert22:23
driptoncomstud: please confirm for yourself, but I want to make sure we don't throw out the baby with the bathwater22:24
*** eharney has quit IRC22:24
comstudcursor.execute('UPDATE bw...')22:24
driptonmy experience is that low-level SQLAlchemy is an excellent portability shim, and high-level sqla ORM doesn't scale to big projects22:24
comstuddripton: I'm not proposing we do.  I'm just stating what I've seen so far22:24
driptonok22:24
devananda"doesn't scale" sums up my exp with all ORMs22:24
dripton+122:24
comstudit was just a quick hack to test something22:25
driptonbut I haven't tried all of them so I'm being nice22:25
comstudwanted to eliminate innodb just being stupid somehow22:25
driptonyep22:25
*** mtreinish has quit IRC22:25
driptonone layer at a time22:25
devanandawould it be viable to replace just the areas that are likely to deadlock with raw sql?22:26
driptonI think we'd end up replacing entire api calls that have problems22:26
comstudi was thinking about that yes22:26
devanandaie, anywhere that currently uses with_lockmode or select-then-update-or-insert22:26
devanandaand yea, that does mean several api calls get entirely rewritten to avoid locking issues22:26
comstudbut I think all of the calls potentially have this 'lock wait timeout' issue22:26
comstudbecause it's a sqlalchemy orm layer problem22:27
boris-42comstud +122:27
comstudwith db pooling anyway22:27
devanandawell22:27
driptonWe can start with a couple that are known bad then start attacking en masse if it works well22:27
devanandawhile i agree with that22:27
comstudbut in particular, I've seen it with 3 or 4 calls22:27
devanandaif we replaced all the ORM code that touches table X with raw sql, that didn't have locking problems22:27
comstudthey just happen to be the most common calls that cells uses.22:27
comstudso that doesn't eliminate any of the others22:27
devanandatrue. but lock waits and deadlocks are table specific22:28
devanandait's not like holding on lock on table_a causes a problem for table_b22:28
comstudsure sure22:28
comstudi know22:28
comstudbut i'd expect that it could happen on any table22:28
devanandagranted, there _is_ a larger problem with sqla here, which i'd love to see fixed :)22:28
comstudyea22:28
devanandaah. so I wouldn't22:28
*** dwcramer has quit IRC22:29
devanandaafaict it will only happen when the ORM is trying to pre-emptively lock rows22:29
devanandaie, with_lockmode or SELECT-then-INSERT22:29
devanandathere aren't that many places which do that22:29
comstudbut if there's somehow an implicit transaction for all of our DB api calls, does that not mean there's locking of some sort on every table?22:29
devanandawell, yes, but no :)22:30
comstudi was able to see this problem even after removing session.begin()22:30
comstudfor instance22:30
devanandaselect 1; begin; update table ... where ..; commit; rollback;22:30
comstud(the lock wait timeouts, not the Deadlock)22:30
devanandathat's the usual sqla query pattern for a single update22:30
driptonBut if we change an API call to not use the ORM then that should go away, right?22:30
driptonWe can put sqla in trace mode to see all the commands it sends to the DB, and verify that simple stuff doesn't do any extra transaction stuff.22:31
devanandaif sqla somehow forgets to do the final "commit; rollback;" then yea, the lock wait problem could happen anywhere22:31
comstudnod22:31
comstudatm, that's what I suspect somewhere.. but I don't know for sure22:32
devanandak22:32
comstudpossible i'm wrong and it's only happening in certain calls22:32
*** HenryG has quit IRC22:32
comstuddunno!22:32
*** mnewby has joined #openstack-meeting22:32
devanandai wonder whether it's related to eventlet?22:32
comstudbut I know we're approaching the limits here at RAX in global-cells22:33
comstudwith all of the DB calls it has to do22:33
comstudso I need Db pooling RSN22:33
driptondevananda: me too22:33
comstudI may have to resort to raw queries for things at least internally if I can't figure this out quickly22:33
comstudand I know this same problem will show up in conductor under load22:33
comstud(we're still using local conductor only)22:34
comstudi'm going to attempt to find the sqlalchemy issue first22:34
driptonsounds good22:34
comstudand if that takes too long, look at low level sqlalchemy calls22:34
devananda++22:34
devanandai wouldn't mind seeing low level sqla calls upstream, esp considering it's better for performance22:35
driptonI will have a work-in-progress patch containing low-level sqla up soon, if you need a reference.  It doesn't quite work yet.22:35
comstudi could def use a reference22:35
driptonI'll ping you when I upload it.22:36
comstudor.. it'd save some googling at least :)22:36
comstudhehe ty22:36
driptonHere's my fundamental problem with db-archiving: my current api call is to move *all* deleted rows to shadow tables in one call.  That could take a long time.  I don't know a clever way to subdivide the job.22:36
devanandacomstud: would it be possible to disable eventlet in conductor, while still enabling tpool?22:37
devanandaprobably a crazy idea ...22:37
devanandadripton: ORDER BY id LIMIT X;22:38
comstudtpool is implemented in eventlet22:38
comstudeventlet.tpool22:38
devanandahah. yep, crazy idea22:38
*** bencherian has joined #openstack-meeting22:38
*** anniec has quit IRC22:38
*** anniec_ has joined #openstack-meeting22:38
comstudwe could do our own threading, i suppose.. but you lose pseudo-parallelising pulling off the rabbit queue22:38
comstudmaybe it's a win in the end, though, i dunno22:39
comstudi suspect kinda not22:39
comstudbut i dunno!22:39
driptondevananda: yes, that works within one table.  I'm worried about cross-table issues due to foreign keys, but we don't have many so maybe it's okay.22:39
devanandadripton: there shouldn't be any FKs in production IMNSHO ....22:39
devanandadripton: but even so, i'm not sure how that would matter22:40
devanandadripton: loop on one table, in small chunks, until it's finished. then move to next table22:40
devanandamake sure each chunk is a separate transaction, too22:40
boris-42 devananda +122:40
*** stevebaker has quit IRC22:40
devanandaotherwise you can blow away the innodb undo space, and it will stall replication, etc...22:40
driptondevananda: sure, FKs would just impose an ordering on how to move things.  But, yeah, I'll do a bunch of little transactions and have a row limit per call to the api22:40
comstudi'm concerned about down time for large deployments22:41
comstudwith the changes to soft delete22:41
comstudthe deleted column22:41
comstudmaybe we just prune the tables first if we don't care about archiving22:42
driptoncomstud: you mean downtime when running the archiving operation?22:42
comstudDB migration22:42
comstudin general22:42
driptoncomstud: or when migrating the deleted column?22:42
driptonah22:42
comstudyeah that one22:42
comstudwe ran a test...22:42
comstudwe took the proposed migration and ran it against a copy of the DB22:43
comstudit took about 45 minutes IIRC22:43
comstud:)22:43
boris-42=)))22:43
devanandai would assume that deployuers are not going to just run all the migrations blindly, but we should include a note about the larger migrations22:43
dripton+122:43
devanandacomstud: 45min is nothing for a big ALTER TABLE ;)22:43
comstudyeah, but unfort due to use of sqlalchemy orm...22:43
boris-42alter table for all tables=)22:43
driptoncomstud: I was planning to include another API call to nuke deleted rows22:43
comstudit means shit is broken during the whole migration22:43
*** markmcclain has joined #openstack-meeting22:44
devanandasure. which is why you have two db's & HA22:44
devananda:)22:44
comstudi think we just prune our tables first22:44
*** asalkeld has quit IRC22:44
devanandaor that, hehe22:44
*** asalkeld_ has joined #openstack-meeting22:45
comstudi gotta drop off.. anything else for me?22:45
devanandaanyone have other topics to bring up? it's seeming like we're about done22:46
* comstud waits22:46
driptonI'm done22:46
devanandacool22:46
devanandathanks guys :)22:46
driptonthanks for the tip devananda22:46
devananda#endmeeting22:46
*** openstack changes topic to "OpenStack meetings || Development in #openstack-dev || Help in #openstack"22:46
openstackMeeting ended Thu Feb  7 22:46:34 2013 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)22:46
openstackMinutes:        http://eavesdrop.openstack.org/meetings/db/2013/db.2013-02-07-22.02.html22:46
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/db/2013/db.2013-02-07-22.02.txt22:46
openstackLog:            http://eavesdrop.openstack.org/meetings/db/2013/db.2013-02-07-22.02.log.html22:46
*** rnirmal has quit IRC22:48
*** stevebaker has joined #openstack-meeting22:54
*** annegentle has quit IRC22:54
*** Mandell_ has joined #openstack-meeting22:55
*** Mandell has quit IRC22:55
*** lloydde has quit IRC22:56
*** lloydde has joined #openstack-meeting22:56
*** _niko has quit IRC22:58
*** tongli has quit IRC22:58
*** fnaval has quit IRC22:59
*** reed has joined #openstack-meeting23:01
*** Mr_T has left #openstack-meeting23:03
*** topol has quit IRC23:08
*** john5223 has quit IRC23:09
*** anniec_ has quit IRC23:14
*** anniec has joined #openstack-meeting23:14
*** vipul is now known as vipul|away23:16
*** vipul|away is now known as vipul23:16
*** iben has joined #openstack-meeting23:16
*** dolphm has joined #openstack-meeting23:16
*** anniec_ has joined #openstack-meeting23:17
*** anniec has quit IRC23:17
*** anniec_ is now known as anniec23:17
*** fnaval has joined #openstack-meeting23:18
*** lbragstad has quit IRC23:19
*** colinmcnamara has joined #openstack-meeting23:20
*** radez is now known as radez_g0n323:21
*** dolphm has quit IRC23:21
*** sacharya has quit IRC23:28
*** alrs has quit IRC23:28
*** vipul is now known as vipul|away23:31
*** dkehn has left #openstack-meeting23:31
*** vipul|away is now known as vipul23:40
*** mnewby has quit IRC23:41
*** AlanClark has quit IRC23:42
*** danwent has quit IRC23:48
*** anniec has quit IRC23:54
*** anniec_ has joined #openstack-meeting23:54
*** anniec_ is now known as anniec23:54
*** diogogmt has quit IRC23:54
*** ijw has quit IRC23:54
*** ijw has joined #openstack-meeting23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!