Wednesday, 2014-08-27

*** Shrews_ is now known as Shrews00:05
Shrewsadam_g: hah. i'll adjust the review in the morning00:06
adam_gShrews, cool00:06
*** dmorita has joined #openstack-qa00:30
*** dims has joined #openstack-qa01:02
*** yamahata has joined #openstack-qa01:03
*** dims_ has joined #openstack-qa01:05
*** dims has quit IRC01:09
*** morganfainberg is now known as morganfainberg_Z01:13
*** aimon has joined #openstack-qa01:18
*** suneelb has quit IRC01:18
*** dims_ has quit IRC01:19
*** topshare has joined #openstack-qa01:29
*** mitz_ has joined #openstack-qa01:31
*** mitz has quit IRC01:33
*** morganfainberg_Z is now known as morganfainberg01:33
*** topshare has quit IRC01:33
*** Poornima has joined #openstack-qa01:35
*** weshay has quit IRC01:39
*** wuhg has joined #openstack-qa01:43
*** flwang has left #openstack-qa01:45
*** topshare has joined #openstack-qa01:50
*** amotoki has joined #openstack-qa01:50
*** nosnos has joined #openstack-qa01:56
*** pcrews has quit IRC01:57
*** topshare has quit IRC01:59
*** dims has joined #openstack-qa02:02
*** topshare has joined #openstack-qa02:03
*** topshare has quit IRC02:03
*** mriedem has joined #openstack-qa02:05
*** Guest61572 is now known as dansmith02:09
*** dims has quit IRC02:13
*** torandu has joined #openstack-qa02:15
*** torandu has quit IRC02:15
*** torandu has joined #openstack-qa02:16
*** rbak__ has joined #openstack-qa02:19
*** rbak_ has quit IRC02:21
*** mriedem has quit IRC02:28
*** ianw has quit IRC02:35
*** adalbas has joined #openstack-qa02:35
*** mrrrgn has joined #openstack-qa02:36
*** jamielennox|away has quit IRC02:37
*** jamielennox|away has joined #openstack-qa02:37
*** adalbas has quit IRC02:38
*** ianw has joined #openstack-qa02:38
*** ianw has quit IRC02:39
*** ianw has joined #openstack-qa02:40
*** ianw has quit IRC02:40
*** ianw has joined #openstack-qa02:41
*** jamielennox|away has quit IRC02:42
*** jamielennox|away has joined #openstack-qa02:42
*** harlowja is now known as harlowja_away02:44
*** ianw has quit IRC02:45
*** ianw has joined #openstack-qa02:46
*** dims has joined #openstack-qa02:54
* gmann is back (gone 17:00:02)02:57
*** jamielennox|away has quit IRC02:59
*** ianw has quit IRC02:59
*** ianw has joined #openstack-qa02:59
*** jamielennox|away has joined #openstack-qa03:00
*** mrrrgn has quit IRC03:00
*** ianw has quit IRC03:04
*** ianw has joined #openstack-qa03:05
*** dims has quit IRC03:16
*** nosnos has quit IRC03:23
*** nosnos has joined #openstack-qa03:24
*** nosnos has quit IRC03:28
*** harlowja_away has quit IRC03:29
*** catohornet has quit IRC03:35
*** ianw has quit IRC03:38
*** jamielennox|away has quit IRC03:38
*** ianw has joined #openstack-qa03:38
*** jamielennox|away has joined #openstack-qa03:39
*** nosnos has joined #openstack-qa03:53
*** cnesa7 has quit IRC03:54
*** Santosh_ has joined #openstack-qa03:56
Santosh_andreaf_: hi03:57
*** Puneet has joined #openstack-qa03:58
*** Santosh has quit IRC03:59
PuneetHI all04:00
PuneetCan anyone please review https://review.openstack.org/#/c/116870/04:01
*** oomichi has joined #openstack-qa04:01
*** chuck_ has quit IRC04:49
*** yfried_ has quit IRC04:53
*** ildikov_ has quit IRC04:57
*** yamahata has quit IRC04:58
*** yamahata has joined #openstack-qa04:58
*** torandu has quit IRC05:03
*** torandu has joined #openstack-qa05:03
*** neeti has joined #openstack-qa05:06
*** bkopilov has joined #openstack-qa05:06
*** rfolco has quit IRC05:11
*** torandu has quit IRC05:13
*** rfolco has joined #openstack-qa05:14
*** Puneet has quit IRC05:18
*** Puneet has joined #openstack-qa05:18
*** SridharG has joined #openstack-qa05:19
PuneetHi all05:19
*** Santosh_ has quit IRC05:23
*** Santosh has joined #openstack-qa05:24
PuneetCan anyone please review https://review.openstack.org/#/c/116870/05:25
PuneetCan anyone please review https://review.openstack.org/#/c/115621/05:27
*** yfried_ has joined #openstack-qa05:43
yfried_anyone knows how I can push something to gerrit on behalf of a colleague?05:45
yfried_Puneet: reviewing now05:47
kashyapyfried_, You might want to check on #openstack-infra05:50
*** k4n0 has joined #openstack-qa05:59
*** ildikov_ has joined #openstack-qa05:59
*** ianw has quit IRC06:08
*** ianw has joined #openstack-qa06:08
*** jamielennox|away has quit IRC06:10
*** jamielennox|away has joined #openstack-qa06:11
*** chandankumar has joined #openstack-qa06:14
*** flaper87|afk is now known as flaper8706:15
*** Longgeek has joined #openstack-qa06:20
*** nmagnezi__ has joined #openstack-qa06:27
yfried_kashyap: I did. no answer there...06:36
kashyapyfried_, Sometimes, I think responses are a bit erratic, IRC being IRC :-)06:37
*** mpaolino has joined #openstack-qa06:40
*** eezhova has quit IRC06:44
*** Puneet_ has joined #openstack-qa06:49
*** tnurlygayanov has joined #openstack-qa06:56
*** afazekas has joined #openstack-qa06:59
*** nosnos has quit IRC07:04
*** nosnos_ has joined #openstack-qa07:04
*** freyes has quit IRC07:09
*** freyes has joined #openstack-qa07:10
*** gema has quit IRC07:16
*** e0ne has joined #openstack-qa07:20
Puneet_Can you please suggest something on this https://review.openstack.org/#/c/116870/07:24
Puneet_i was reading router_name from conf file07:25
*** jordanP has joined #openstack-qa07:26
*** e0ne has quit IRC07:31
*** e0ne has joined #openstack-qa07:36
*** e0ne has quit IRC07:38
*** jamielennox|away has quit IRC07:38
*** ianw has quit IRC07:38
*** ianw has joined #openstack-qa07:38
*** jamielennox|away has joined #openstack-qa07:39
*** markus_z has joined #openstack-qa07:39
*** qba73 has joined #openstack-qa07:41
*** jlanoux has joined #openstack-qa07:42
*** e0ne has joined #openstack-qa07:42
*** tkammer has joined #openstack-qa07:45
*** fbo has joined #openstack-qa07:52
*** fbo has quit IRC07:55
*** e0ne has quit IRC07:55
*** andreaf_ has quit IRC07:56
*** e0ne has joined #openstack-qa07:59
*** jpich has joined #openstack-qa07:59
yfried_Puneet_: as I wrote - Tempest isolated_creds creates routers per test class. the module should also provide a method for fetching that router's name/id. I'm not too familiar with show this works or even IF said network resources are created for CLI tests. you will need to look into that your self or ask some more experienced members than me07:59
*** salv-orlando_ has joined #openstack-qa08:00
yfried_salv-orlando: ^ you worked on isolated_creds. am I right?08:00
*** jlibosva has joined #openstack-qa08:02
yfried_Puneet_: that's the module you are looking for: tempest/common/isolated_creds.py08:02
*** dmorita_ has joined #openstack-qa08:03
yfried_Puneet_: do "git blame" to find out the people who should know best how to work with its utils08:03
*** e0ne_ has joined #openstack-qa08:05
*** dhellmann_ has joined #openstack-qa08:08
*** SergeyLukjanov2 has joined #openstack-qa08:09
*** e0ne has quit IRC08:10
*** catohornet1 has quit IRC08:10
*** jgriffith has quit IRC08:10
*** dmorita has quit IRC08:10
*** salv-orlando has quit IRC08:10
*** dhellmann has quit IRC08:10
*** SergeyLukjanov has quit IRC08:10
*** SergeyLukjanov2 is now known as SergeyLukjanov08:10
*** salv-orlando_ is now known as salv-orlando08:10
*** dhellmann_ has quit IRC08:12
*** dhellmann has joined #openstack-qa08:12
*** jgriffith has joined #openstack-qa08:12
*** e0ne_ has quit IRC08:14
*** flaper87 is now known as flaper87|afk08:17
*** aix has joined #openstack-qa08:20
*** Longgeek has quit IRC08:20
*** Longgeek has joined #openstack-qa08:21
*** Longgeek has quit IRC08:25
*** Poornima has quit IRC08:42
*** yfried__ has joined #openstack-qa08:43
*** Longgeek has joined #openstack-qa08:43
*** yfried_ has quit IRC08:44
*** nosnos_ has quit IRC08:45
*** bdperkin has quit IRC08:46
*** nosnos has joined #openstack-qa08:46
*** markus_z has quit IRC08:47
*** bdperkin has joined #openstack-qa08:48
*** bdperkin has joined #openstack-qa08:48
*** markus_z has joined #openstack-qa08:48
*** markus_z has quit IRC08:48
*** nosnos has quit IRC08:50
*** markus_z has joined #openstack-qa08:53
*** e0ne has joined #openstack-qa09:04
*** yamahata has quit IRC09:06
*** oomichi has quit IRC09:09
*** nosnos has joined #openstack-qa09:11
*** luqas has joined #openstack-qa09:12
*** yamahata has joined #openstack-qa09:16
*** luqas has quit IRC09:17
*** luqas has joined #openstack-qa09:19
*** marun has joined #openstack-qa09:22
*** Puneet_ has quit IRC09:25
* gmann is away: 09:25
*** Poornima has joined #openstack-qa09:27
*** Poornima has quit IRC09:27
*** aix has quit IRC09:29
*** chandankumar has quit IRC09:32
*** krtaylor has quit IRC09:32
*** jamielenz has joined #openstack-qa09:33
*** bdperkin has quit IRC09:33
*** jamielennox|away has quit IRC09:36
*** ianw has quit IRC09:37
*** ianw has joined #openstack-qa09:37
*** yfried has joined #openstack-qa09:37
*** yfried__ has quit IRC09:39
*** chandankumar has joined #openstack-qa09:40
*** cnesa7 has joined #openstack-qa09:42
*** luqas has quit IRC09:43
*** bdperkin has joined #openstack-qa09:43
*** e0ne has quit IRC09:44
*** e0ne has joined #openstack-qa09:44
*** cnesa8 has joined #openstack-qa09:45
*** cnesa7 has quit IRC09:47
*** luqas has joined #openstack-qa09:48
*** tosky has joined #openstack-qa09:48
*** Longgeek has quit IRC09:50
*** xianghuihui has joined #openstack-qa10:01
*** luqas has quit IRC10:03
*** aix has joined #openstack-qa10:05
yfriedmasayukig: are you around?10:07
*** zz_johnthetubagu is now known as johnthetubaguy10:07
*** jamielennox|away has joined #openstack-qa10:07
*** ianw has quit IRC10:09
*** ianw has joined #openstack-qa10:09
*** jamielenz has quit IRC10:09
*** Longgeek has joined #openstack-qa10:16
*** marun is now known as marun_afk10:25
*** chandankumar has quit IRC10:26
*** chandankumar has joined #openstack-qa10:27
*** Longgeek has quit IRC10:32
*** zul has joined #openstack-qa10:43
*** marun_afk has quit IRC10:46
*** flaper87|afk is now known as flaper8710:47
*** mpaolino has quit IRC10:51
*** zul has quit IRC10:51
*** jhenner has joined #openstack-qa10:55
*** krtaylor has joined #openstack-qa10:57
*** Santosh has quit IRC10:59
*** Santosh has joined #openstack-qa10:59
*** Longgeek has joined #openstack-qa11:00
*** yamahata has quit IRC11:04
*** zul has joined #openstack-qa11:04
*** sergeysh has quit IRC11:09
*** dhellmann has quit IRC11:10
*** zul has quit IRC11:11
*** dhellmann has joined #openstack-qa11:13
*** flaper87 is now known as flaper87|afk11:14
*** nosnos has quit IRC11:20
*** nosnos has joined #openstack-qa11:20
*** cdent has joined #openstack-qa11:23
*** flaper87|afk is now known as flaper8711:23
cdentCould someone with the proper powers in tempest trigger a reverify on https://review.openstack.org/#/c/116885 please?11:24
*** sergeysh has joined #openstack-qa11:24
*** marun_afk has joined #openstack-qa11:25
*** nosnos has quit IRC11:25
*** dims has joined #openstack-qa11:36
*** dims has quit IRC11:40
*** dims has joined #openstack-qa11:41
*** dims has quit IRC11:41
*** dims has joined #openstack-qa11:42
*** flaper87 is now known as flaper87|afk11:42
*** dims_ has joined #openstack-qa11:44
*** flaper87|afk is now known as flaper8711:44
*** weshay has joined #openstack-qa11:45
flaper87sdague: chmouel hey guys, you around? Can I get your thoughts on this one? https://review.openstack.org/#/c/115265/ I'm sorry for all these pings I've been doing lately, I really need to push this glance.store thing forward11:45
*** dims has quit IRC11:47
*** dmorita_ has quit IRC11:49
*** dkranz has joined #openstack-qa11:52
*** gmann_ has joined #openstack-qa11:52
yfriedcdent: it's already in queue: http://status.openstack.org/zuul/11:56
sdagueflaper87: is glance.store actually a namespace package?11:57
yfriedcdent: and it's failing again: https://jenkins02.openstack.org/job/check-grenade-dsvm-icehouse/697/11:57
sdaguebecause if so, this is going to be an issue11:57
cdentmeh11:58
cdentyfried: How come the review hasn't been updated with a "I'm trying again"?11:58
cdentOr does it not do that now with the new interface?11:58
yfriedcdent: seems like jenkins comments (build start, build results) are gone. all you have now is latest build. no idea why. maybe old interface retains jenkins comments. but if you are looking for you patch in the pipeline - check zuul12:00
yfriedmtreinish: here?12:00
*** marun_afk is now known as marun12:00
cdentThanks.12:00
*** tosky_ has joined #openstack-qa12:00
yfriedmasayukig: you -1ed my patch without reasons. care to explain? or did you do it on behalf of andreaf?12:01
yfriedhttps://review.openstack.org/#/c/111719/12:01
*** tosky has quit IRC12:02
*** dims_ has quit IRC12:02
*** mwagner_lap has quit IRC12:02
*** dkranz has quit IRC12:03
*** jhenner has quit IRC12:03
*** jhenner has joined #openstack-qa12:03
*** mkollaro has joined #openstack-qa12:03
*** mkollaro1 has quit IRC12:06
*** flaper87 is now known as flaper87|afk12:14
*** dkranz has joined #openstack-qa12:16
*** flaper87|afk is now known as flaper8712:18
*** mpaolino has joined #openstack-qa12:18
*** aysyd has joined #openstack-qa12:20
*** chandankumar has quit IRC12:21
*** gmann_ has quit IRC12:25
*** cdent has quit IRC12:28
*** yamahata has joined #openstack-qa12:44
*** adalbas has joined #openstack-qa12:46
*** sballe has joined #openstack-qa12:47
*** mkollaro has quit IRC12:55
*** mkollaro has joined #openstack-qa12:55
*** topshare has joined #openstack-qa12:57
*** tosky_ has quit IRC13:00
*** mkollaro1 has joined #openstack-qa13:00
*** tosky_ has joined #openstack-qa13:00
mpaolinodtroyer, hi13:01
*** mkollaro has quit IRC13:02
*** marun has quit IRC13:03
*** marun has joined #openstack-qa13:03
mtreinishyfried: yes13:07
dtroyermpaolino: hi13:10
*** gordc has joined #openstack-qa13:11
*** cdent has joined #openstack-qa13:11
gordcsdague: when you have the time, can we get your opinion on: https://review.openstack.org/#/c/102354/ ...13:11
gordci think comments on patchset 17 are a good summary of discussion13:12
mpaolinodtroyer, can you have a look at https://review.openstack.org/#/c/108714/4 ?13:14
sdaguegordc: it's still not passing right?13:14
mpaolinoin particular, I received a comment related to the folder in which libvirt is downloaded13:15
*** mwagner_lap has joined #openstack-qa13:15
mpaolino(build_libvirt.sh, line 32)13:15
gordcsdague: cdent will split the patch in two... just wanted your opinion on where the 2nd patch should land.13:15
mpaolinoI would be interested to know what is the best solution from the qa point of view13:16
cdentI'm just getting started on that change now gordc13:16
gordcsdague: from what i've read, jogo mentioned it should be somewhere... just not sure if it should be in javelin13:16
gordccdent: awesome!13:16
sdaguegordc: so honestly.... I don't know at this point. I think our goal was to get basic checking in here. And honestly, that's probably something we want to talk about more in a summit context I think for more complicated upgrade testing.13:17
sdaguebecause there are implications for really deep inspection that I think we need to think about, and realistically from an overall OpenStack perspective things like compute and networks are probably more in need of the deep inspection at this point13:18
sdaguekind of like the heat functional test conversation in the project meeting, I'm happy the team wants to tackle those things, but it's a weird starting point to figure out a rhythm because it's so far up the layers in openstackness.13:19
gordcsdague: i see... so i guess for the deeper inspection piece we have, we should hold on to that for now?13:20
sdaguegordc: yeh, that's probably best13:20
*** mpaolino has quit IRC13:20
sdaguethe rest of it will be a real step forward13:20
gordcsdague: cool cool. we'll add some basic stuff for now... i'll let eglynn know when he gets back.13:21
sdagueand lets give it a chance to breath to figure out what deep inspection things make sense from openstack as a whole perspective13:21
cdentJust for reference, I went down the road of deeper inspection explicitly to avoid introducing yet more ambiguity in the test infrastructure.13:21
cdentBut I think there's probably a way to make the "part 1" useful without adding too much ambiguity.13:21
sdaguecdent: sure, that's fair13:21
*** e0ne is now known as e0ne_13:21
cdentI think with just part1 we'll be able make fewer conclusions about the health of the upgrade.13:22
cdentThat's okay, and certainly better than what we've got now.13:22
gordccdent: do you know what the original scope of bp was? should we open a second one to contain your deeper expansion work?>13:22
cdentThe scope of the bp changed a lot as it was being written, in large part because there wasn't clarity on how javelin was going to behave13:22
cdentFor example when it was written the create side of javelin only did create without any check13:23
cdentNow create is create + check and check is just check.13:23
cdentMuch of the fiddling that people have done to get stuff into javelin2 have impacted it's evolution13:23
cdentThere were a lot of questions during the spec writing about "what is it that we are really testing"13:24
cdentand it came out to do things:13:24
cdentdo metrics survive the upgrade13:24
cdentdoes metered resoures survive the upgrade in a sane way13:24
cdents/does/do/13:24
*** mriedem has joined #openstack-qa13:25
*** balles has quit IRC13:25
gordccdent:  gotcha... yeah it's something we should look into more... unfortunately javelin is one of those things where there's very few eyes on it right now.13:26
*** jecarey has joined #openstack-qa13:26
cdentI'm sure it will all come out fine in the end.13:26
yfriedmtreinish: rdekeil (my colleague) will be back tomorrow to push the tcp-ping patch13:26
mtreinishyfried: ok cool, thanks13:27
yfriedmtreinish: but I had a discussion arround the office about your patch13:27
yfriedmtreinish: we think it's better if you use the tcp-ping instead of regular ping in your patch to verify connectivity13:27
yfriedmtreinish: you will be checking tcp:22 connectivity (which is exactly what secgroup is ecpecting) without opening extra rules on secgroups13:28
mtreinishyfried: ok, that should be simple enough to change13:29
yfriedmtreinish: and the secgroup scenario won't be "violated" by messing about with its secgroup while its trying to test them13:29
mtreinishsdague: ^^^ ping before ssh was your idea, are you ok with that change?13:30
yfriedmtreinish: also, cirros images are listening to tcp:22 only (no udp or other ports) so I wonder if we could somehow change that without sshing to the VM13:30
yfriedmtreinish: ^ assuming we would like to check secgroup isolating ports/protocols (a test we plan to push)13:30
*** salv-orlando has quit IRC13:31
*** rbak__ has quit IRC13:31
sdaguemtreinish: sure, I'm fine with that13:32
mtreinishyfried: probably not, short of changing the image we're using to start another service13:32
yfriedmtreinish: whose incharge of cirros? could we ask for a change in future versions?13:33
yfriedwho's13:33
mtreinishsdague: ok, cool13:33
*** mpaolino has joined #openstack-qa13:34
mtreinishyfried: I really don't think we want to do that. It was just a hypothetical, because we could also just build the custom image as part of devstack.13:34
mtreinishyfried: but fyi, cirros dev is tracked here: https://launchpad.net/cirros13:34
*** mpaolino_ has joined #openstack-qa13:35
yfriedmtreinish: ok. I was expecting this, so we'll just check ssh first...13:35
yfriedmtreinish: and after ssh we can edit the image on runtime13:35
yfriedmtreinish: can I ask for your opinion as tempest core?13:36
yfriedmtreinish: about the scenario client migration13:36
mtreinishwhat about it?13:36
*** malini has joined #openstack-qa13:37
*** mpaolino has quit IRC13:38
yfriedmtreinish: moving to tempest client we are loosing the Object Oriented design that most native clients give. that's no big deal with api tests which are usually small and short op13:39
mtreinishyfried: so the oo model from the official clients is actually part of the problem. They abstract away too much, which makes debugging failures far more difficult13:40
yfriedmtreinish: but in scenarios it calls for more complicated/long/ugly code and sometimes we loose functionality (such as using a server object as dict key)13:40
mtreinishit does mean that the scenario tests will become larger and slightly more complex, but the tradeoff is worth IMO13:40
yfriedmtreinish: could you give an example?13:41
mtreinishyfried: look at any of the gate failures in the scenario tests, trying to figure out exactly what failed is more difficult13:41
*** mpaolino_ is now known as mpaolino13:41
mtreinishyfried: this effort comes directly from debugging when things fail13:42
yfriedmtreinish: I know the native clients are problematic, and I'm not trying to reverse that decision13:42
*** e0ne_ has quit IRC13:43
yfriedmtreinish: what I'm asking is, assuming I'm doing all the work, could I extended the OO wrap created by marun for neutron dicts (net_common.DeletableResources) to all/most other resources13:44
yfriedmtreinish: I have this as WIP and I would like to push this. it would still use the native clients, but would "tie" a client instance to a resource and allow better resource handling. we could also use this for a unified resource api/interface to use for cleanups/waits/searches13:45
mtreinishyfried: so there is no need for that if we're not using the client python apis. That was only added because the neturon client behaved differently to the other clients13:46
flaper87sdague: Do you have a min to talk about https://review.openstack.org/#/c/115265/ ?13:46
yfriedmtreinish: but I'm saying that there IS need/benefit in doing it13:46
yfriedit's much easier/cleaner to do server.reboot() or subnet.add_router()13:47
yfriedmtreinish: and you are still using tempest clients.13:47
mtreinishyfried: yes, but one of the big advantages of the tempest clients for testing is that it is very raw. While an resource model is easier to read it also abstracts away information13:48
mtreinishwhich for testing we don't want to necessarily ignore13:49
*** chandankumar has joined #openstack-qa13:50
*** e0ne has joined #openstack-qa13:50
yfriedmtreinish: I'm sorry. I fail to understand what information would be abstracted13:51
yfriedmtreinish: when you are using, for example server.reboot()13:52
*** pcrews has joined #openstack-qa13:52
yfriedmtreinish: instead of self.servers_client.reboot_server(server['id])13:52
dtroyermpaolino: I left some comments, hopefully clarifying what I think was meant.13:52
*** amotoki has quit IRC13:52
yfriedmtreinish: the same client would be use and you will get the same debug info you wanted13:53
mtreinishyfried: it's about being explicit in the test code. Functionally there isn't necessarily a difference, but look at the 2 lines, it says which client your using, where the id for the request is coming from, etc.13:54
mtreinishnow if this fails and someone looks at the traceback they see that line they know exactly how what to trace13:55
mpaolinodtroyer, perfect, thanks13:55
mtreinishwith server.reboot() it's not as straightforward13:55
yfriedmtreinish: check this out please: https://review.openstack.org/#/c/102700/13:56
*** Dafna is now known as Dafna_away13:57
yfriedmtreinish: other than the waiter func, this was a 20 lines patch. if I drop the OO wrapper for floating ips, it becomes a very big and complicated code13:57
yfriedmtreinish: I really think the added functionality would make it easier to right and read the code, as we are headed towards more complex scenarios in the future and we would start seeing a lot of duplicate code unless we manage scenario library better14:01
mtreinishyfried: I actually don't see that, we have plenty of waiters in code that don't need an object oriented abstraction to work.14:02
yfriedmtreinish: "we have plenty of waiters in code"14:03
mtreinishthere are lots of other methods that wait for status14:03
yfriedmtreinish: OO abstraction would make it so you only need 1 waiter14:03
*** BobBall has joined #openstack-qa14:04
yfriedmtreinish: would you prefer this patch https://review.openstack.org/#/c/111719/10 to delete OOwrapper and use dicts on ALL resources?14:05
*** topshare has quit IRC14:05
mtreinishyfried: no, it would just look that way in the test code. That's actually the crux of our difference in opinion14:06
*** rbak has joined #openstack-qa14:06
marunmtreinish: oo is still awesome :)14:06
marunmtreinish: having resources be bags of dicts really sucks14:06
mtreinishmarun: but it's that way by design, to couple it closer the raw http. It's definitely a pain to deal with14:07
BobBallHey - I'm seeing a large number of one-off failures with identical symptoms in the XenServer CI.  They all seem to occur when deleting tenants at the end of a test and get "Lock Timeout occurred for key, os-revoke-events"... Has anyone seen this before, or can provide suggestions on where to start debugging?14:07
mtreinishbut the explicitness is useful14:07
marunmtreinish: explicitness *can* be useful14:07
BobBallhttp://dd6b71949550285df7dc-dda4e480e005aaa13ec303551d2d8155.r49.cf1.rackcdn.com/78/95278/9/23521/results.html is an example failure14:07
marunmtreinish: but it has a cost14:07
marunmtreinish: if you think it's worth bearing, well, I guess we can differ on that point14:07
*** topshare has joined #openstack-qa14:07
mtreinishmarun: yeah I think a difference of opinion here is fine.14:08
marunmtreinish: so long as you don't want me writing code for tempest, that is true :)14:09
*** ildikov_ has quit IRC14:09
yfriedmtreinish: I fail to see the terrible cost of server.reboot(). I really do. I would understand if you demanded proper unitttests for the OOwrappers, but just to save 2 lines of stack trace while adding a lot more lines of code duplication seems wrong to me14:09
yfriedmtreinish: so to sum it up, you guys would -2 a patch/series of OOwrappers for resources. even if I provide proper unittest for it?14:11
mtreinishyfried: I'd probably just be -114:12
cdentsdague, gordc new version of https://review.openstack.org/#/c/102354 is up. Much smaller change. Could probably be even smaller but the current state gives a good level of confirmation that the data is mostly sane.14:12
mtreinishBobBall: that error msg is coming from keystone, the message there is the resp body of the tenant delete request14:12
yfriedmtreinish: AFAIK if you feel like this is a bad idea, than -2 is in order.14:13
mtreinishyfried: I can't speak for everyone, but that would probably be what I'd vote14:13
*** dpaterson has joined #openstack-qa14:13
yfriedmtreinish: thank you for your time. I might send this question to ML to get more opinions.14:15
BobBallmtreinish: I did check but there are no errors / traces in keystone at all... nor references to the lock timeout14:16
mtreinishBobBall: yeah I was looking at the keystone logs there, nothing stuck out at a quick glance14:16
*** markmcclain has joined #openstack-qa14:17
BobBallit seems to be failing around 30% of my tests at the moment - all of the individual test failures that I've looked at have the same failure14:17
mtreinishmorganfainberg: ^^^ have you seen a failure like that before, where keystone returns a 500(or 501) because of a timeout waiting for the os-revoke-events lock14:19
mtreinishBobBall: is there a way to get timestamps on the run_tests.log file?14:20
mtreinishI'm trying to see what keystone is logging around the time the failure is hit14:20
*** zz_jgrimm is now known as jgrimm14:22
*** yfried has quit IRC14:23
*** chandankumar has quit IRC14:24
*** Dafna_away is now known as Dafna14:27
*** dustins has joined #openstack-qa14:35
*** chandankumar has joined #openstack-qa14:41
cdentsdague, gordc, jogo: I preserved the undesirable javelin+telemetry functionality in a dependent patch: https://review.openstack.org/#/c/117259/14:42
*** wuhg has quit IRC14:42
*** jecarey has quit IRC14:43
*** afazekas has quit IRC14:43
*** malini|afk has joined #openstack-qa14:49
*** HenryG has joined #openstack-qa14:51
*** mlavalle has joined #openstack-qa14:51
*** SridharG has left #openstack-qa14:53
*** hemna_ is now known as hemna14:53
*** neeti has quit IRC14:53
*** openstackgerrit has joined #openstack-qa14:54
mtreinishdkranz: https://review.openstack.org/117049 should fix the coverage post jobs and would explain why it didn't work locally for some people14:54
*** neeti has joined #openstack-qa14:54
dkranzmtreinish: ok, thanks. I'll look14:55
*** xianghuihui has quit IRC14:55
*** flaper87 is now known as flaper87|afk14:56
*** flaper87|afk is now known as flaper8714:56
*** Longgeek has quit IRC14:56
BobBallmtreinish: good call; I've added timestamps now but clearly that'll take a while to come through on the test results14:58
mtreinishBobBall: cool, thanks.14:58
mtreinishBobBall: I do have to say looking through those logs it makes me realize how dependent I've become on os-loganalyze14:59
*** neeti has quit IRC15:00
BobBallI know that feeling!15:00
*** neeti has joined #openstack-qa15:01
BobBallthere are all sorts of niceities that the -infra jobs have that 3rd party jobs don't :/15:01
*** mikal has quit IRC15:05
*** topshare has quit IRC15:05
*** mikal has joined #openstack-qa15:06
*** topshare has joined #openstack-qa15:07
*** topshare has quit IRC15:07
*** tkammer has quit IRC15:08
*** topshare has joined #openstack-qa15:08
BobBallmtreinish: [Wed Aug 27 14:35:07.591400 2014] in http://dd6b71949550285df7dc-dda4e480e005aaa13ec303551d2d8155.r49.cf1.rackcdn.com/86/114286/12/23562/screen-key.txt.gz15:18
BobBallmtreinish: seems that we got the lock OK but didn't release it within the 6 seconds so it was released for us15:18
*** oomichi has joined #openstack-qa15:18
BobBallmtreinish: so keystone was trying to delete a tenant but for some reason couldn't delete it fast enough (which we already knew I guess)15:18
*** mpaolino has quit IRC15:22
*** dansmith has quit IRC15:22
*** ekhugen_alt has quit IRC15:22
*** tteggel has quit IRC15:22
*** akuznetsova has quit IRC15:22
*** jogo has quit IRC15:22
mtreinishBobBall: yeah, well it could have been something else holding the lock too long instead of the tenant delete15:22
mtreinishbut this confirms it was the tenant delete itself15:22
*** Longgeek has joined #openstack-qa15:22
BobBallfound it mtreinish15:22
BobBallour devstack VM is swapping15:22
*** mpaolino has joined #openstack-qa15:22
*** dansmith has joined #openstack-qa15:22
*** ekhugen_alt has joined #openstack-qa15:22
*** tteggel has joined #openstack-qa15:22
*** akuznetsova has joined #openstack-qa15:22
*** jogo has joined #openstack-qa15:22
*** Longgeek has quit IRC15:22
BobBalldevstack is now using > 6G of RAM15:23
BobBallso we'll blame ourselves for it :)15:23
mtreinishahh, yeah that would probably cause things to be slow15:23
mtreinishhow much ram do you give them?15:23
mtreinishBobBall: well, dhellmann found yesterday that on neutron jobs in the gate we are exceeding 8GB triggering oom-killer to blow away mysqld15:24
BobBall4G atm, but I can increase that a bit...15:24
*** markmcclain has quit IRC15:24
BobBalllooking at reducing the number of workers to ease up on the RAM requirement a bit15:24
mtreinishyeah, that should help a bit15:25
*** atiwari has joined #openstack-qa15:28
*** ildikov has joined #openstack-qa15:28
*** oomichi has quit IRC15:31
*** fungi has quit IRC15:34
*** openstack has joined #openstack-qa16:19
*** dkranz has joined #openstack-qa16:20
*** dustins has quit IRC16:21
*** yamahata has quit IRC16:28
*** ildikov_ has joined #openstack-qa16:30
*** dims has joined #openstack-qa16:30
*** ildikov has quit IRC16:33
*** jpich has quit IRC16:33
*** aimon has joined #openstack-qa16:33
BobBallAny way to restrict the number of nova-api workers in the gate?  I want to set osapi_compute_workers=$TEMPEST_CONCURRENCY but we seem to only have a localrc rather than a local.conf so I don't know how I can get that value through to nova from devstack?16:33
*** tosky_ has joined #openstack-qa16:38
*** bkopilov has quit IRC16:42
openstackgerritA change was merged to openstack/qa-specs: Add the qa spec for bp:more-selectable-swift-tests  https://review.openstack.org/10511516:46
*** krtaylor has quit IRC16:47
jogocdent: I wouldn't call it undesirable per se, but thanks16:52
cdentundesirable in that context?16:52
*** BobBall is now known as BobBall_Away16:54
jogocdent:  yeah in that context, not in general16:54
jogocdent: I will review the patch later today16:54
cdentthat's what I meant, perhaps should have been more explicit, but being too explicit is what got me down this rathole ;)16:54
jogocdent: we have all been there ^_^16:54
cdent:)16:54
jogocdent: anyway thank you for your patience on this one, and sorry for taking so long to respond I know this is a priority for your team16:55
*** bkopilov has joined #openstack-qa16:55
*** marun has quit IRC16:55
cdentIt would have only become a problem if we had pushed up closer to the juno3 milestone. I know everybody has a ton to do.16:55
cdentIt's been a very good learning experience for me.16:55
*** mpaolino has quit IRC16:56
*** markmcclain has joined #openstack-qa16:59
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-dev/devstack: Add oslo.concurrency and oslo.middleware  https://review.openstack.org/11730416:59
*** dpaterson has joined #openstack-qa17:01
*** suneelb has joined #openstack-qa17:01
*** amotoki has quit IRC17:02
*** torandu has quit IRC17:02
*** e0ne has quit IRC17:03
*** harlowja has joined #openstack-qa17:04
*** torandu has joined #openstack-qa17:09
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-infra/devstack-gate: Add oslo.utils and oslo.concurrency  https://review.openstack.org/11730717:12
*** sdmonov has joined #openstack-qa17:15
*** qba73 has quit IRC17:19
anteayaanyone working on classifying the failures for gate-tempest-dsvm-neutron-full? I can but don't want to replicate17:28
kashyapDoes this merit a new bug, I wonder, current gate failure from "gate-tempest-dsvm-full" job) -- http://logs.openstack.org/07/116707/1/gate/gate-tempest-dsvm-full/958f55e/logs/screen-n-cond.txt.gz17:31
* kashyap files one anyway for now17:33
kashyapAh, there's already one that's recently filed - https://bugs.launchpad.net/nova/+bug/135313117:35
*** Sukhdev has quit IRC17:36
*** aix has quit IRC17:37
anteayaso help me understand what is happening when the testr tests commands succeeded and the log checker identifies that various log files have errors17:39
openstackgerritRohan Kanade proposed a change to openstack/tempest: Add subnet tests for extra attributes  https://review.openstack.org/10632617:40
*** zz_dimtruck is now known as dimtruck17:44
*** hockeynut has joined #openstack-qa17:58
anteayathis fingerprint went in 13 hours ago: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/log/queries/1361924.yaml and should have caught this: http://logs.openstack.org/89/116889/1/gate/gate-tempest-dsvm-neutron-full/02de4a8/logs/devstacklog.txt.gz but the fingerprint is for console not devstacklog18:00
*** mikal has quit IRC18:00
anteayaI'll offer a patch to amend the fingerprint18:01
*** mkollaro1 has quit IRC18:04
*** mikal has joined #openstack-qa18:05
*** rbak has quit IRC18:07
*** mkollaro has joined #openstack-qa18:10
openstackgerritAnita Kuno proposed a change to openstack-infra/elastic-recheck: Expanded fingerprint 1361924  https://review.openstack.org/11732018:12
*** cnesa8 has quit IRC18:14
*** e0ne has joined #openstack-qa18:14
*** annegent- has joined #openstack-qa18:15
*** Sukhdev has joined #openstack-qa18:23
*** yfried has joined #openstack-qa18:24
*** rbak has joined #openstack-qa18:27
*** dims has quit IRC18:28
*** dims has joined #openstack-qa18:28
*** annegent- is now known as annegentle18:38
*** johnthetubaguy is now known as zz_johnthetubagu18:38
*** zigo has quit IRC18:43
openstackgerritAnita Kuno proposed a change to openstack-infra/elastic-recheck: Expanded fingerprint 1361924  https://review.openstack.org/11732018:43
*** harlowja has quit IRC18:46
*** zigo has joined #openstack-qa18:46
*** harlowja has joined #openstack-qa18:46
*** torandu_ has joined #openstack-qa18:48
*** tosky_ has quit IRC18:49
mtreinishdhellmann: did you open a bug for that failure yesteday? If not I can do it18:54
mkoderermtreinish: can I get a quick review on https://review.openstack.org/#/c/113899/318:58
*** atiwari has quit IRC18:59
*** torandu_ has quit IRC19:01
*** torandu has quit IRC19:01
*** torandu has joined #openstack-qa19:05
mtreinishmkoderer: LGTM, do you think it's worth updating the ones with args to use it too? (in a separate patch)19:05
mkoderermtreinish: yep I can do that.. decorators are sometimes tricky though ;)19:06
mkoderermtreinish: btw the schema unification is done... I just have a clean-up patch https://review.openstack.org/#/c/115918/19:07
mkoderermtreinish: and I guess I have to update the documentation ;)19:07
*** krtaylor has joined #openstack-qa19:07
openstackgerritYuriy Taraday proposed a change to openstack-dev/devstack: Add rootwrap daemon mode support for Neutron  https://review.openstack.org/10738619:14
*** cdent_ has joined #openstack-qa19:34
sdaguemrodden: have you looked into doing a bashateignore yet?19:34
*** cdent has quit IRC19:36
*** cdent_ is now known as cdent19:36
openstackgerritDean Troyer proposed a change to openstack-dev/devstack: Take a swing at processes without screen  https://review.openstack.org/11733919:36
dtroyersdague: (seeing the bashate comment) I was playing with doing a gitignore-stype thing with it, starting with using .gitignore itself…19:37
dtroyersdague: anyway, this review ^^^ is the current state of the process execution art19:37
sdaguedtroyer: ok, I'll take a look in a minute19:38
sdaguemrodden: https://review.openstack.org/#/c/113892/ probably could use landing, that would solve the biggest part of the ignore issue19:38
sdaguedtroyer: do you have +2 on bashate?19:39
dtroyersdague: I think so19:39
sdagueok, so I think that link above is probably going to solve most of the issues with integrating it in devstack19:39
sdagueit hides all the . dirs19:40
*** Santosh has quit IRC19:42
*** malini has quit IRC19:44
*** malini|afk is now known as malini19:44
*** cdent has quit IRC19:47
openstackgerritDavid Shrewsbury proposed a change to openstack/tempest: Fix network/secgroup scenario tests for ironic  https://review.openstack.org/11266019:47
openstackgerritDavid Shrewsbury proposed a change to openstack/tempest: Fix network/secgroup scenario tests for ironic  https://review.openstack.org/11266019:49
sdagueadam_g: so there was another patch in flight that did the db stuff that you had in save-state in your patch19:49
*** cdent has joined #openstack-qa19:49
*** cdent has quit IRC19:49
adam_gsdague, https://review.openstack.org/#/c/115001/ ?19:49
sdagueyeh19:49
adam_gsdague, so that will still fail if any of the databases MYSQL_SERVICES do not exist19:51
adam_gat least in grenade.sh (i think save-state runs without set -e?)19:51
*** cdent has joined #openstack-qa19:57
*** aimon has quit IRC19:57
*** e0ne has quit IRC19:59
openstackgerritAdam Gandelman proposed a change to openstack-dev/grenade: Fix check-sanity + save-state /w disabled services  https://review.openstack.org/11340520:00
adam_gsdague, ^ rebased on 11500120:01
sdagueadam_g: so one question.... this is because ironic is enabled but has no old db?20:03
sdagueI want to understand context for the existance check20:04
adam_gsdague, cinder does not currently get deployed /w ironic20:05
sdagueit feels like it would be better to just populate MYSQL_SERVICES correctly20:05
sdagueso populate it from actual enabled services20:05
sdagueinstead of a hardcoded list20:05
adam_gsdague, then we're in the business of mapping service names to assumed database names.  we're already stuck doing that with processes -> devstack services20:06
sdagueyes, but this has a failure mode of cinder db manage breaks and doesn't create a db, and we skip it as fine.20:07
sdaguemaybe it's fine20:08
sdagueseems odd20:08
sdagueI'll +A based on the fact that we'd probably catch the failure in a different mechanism20:08
adam_gagreed, tho i think we'd see more obvious failures in that case elsewhere20:08
*** suneelb has quit IRC20:10
adam_gsdague, thanks20:12
sdagueso I've been trying to hit all the right pieces related to the sideways upgrade, but I'm sure I missed chunks. So if there is stuff you need my eyes on please share urls here20:12
adam_gsdague, appreciated. we're down to just a handful: https://review.openstack.org/#/q/topic:ironic_tempest,n,z + https://review.openstack.org/#/q/topic:ironic_grenade,n,z20:14
adam_gim hoping the migration script stuff lands in ironic today or tomorrow and to have the experimental  check-grenade-dsvm-ironic-sideways job doing something useful by EOW20:15
*** qba73 has joined #openstack-qa20:17
*** samuelbercovici has joined #openstack-qa20:17
*** nsaje has quit IRC20:17
*** adalbas has quit IRC20:22
*** Sukhdev has quit IRC20:23
*** nmagnezi__ has joined #openstack-qa20:24
adam_ghmm. what am i missing? shouldn't this show an +A? https://review.openstack.org/#/c/112731/20:24
adam_git even went through gate checks20:24
*** aimon has joined #openstack-qa20:32
*** ildikov has joined #openstack-qa20:37
*** malini1 has joined #openstack-qa20:40
*** malini1 has quit IRC20:41
*** ildikov_ has quit IRC20:41
*** malini1 has joined #openstack-qa20:41
sdagueI think it was trivial rebase detection20:42
sdaguereadded the +A20:43
sdagueok, so we need a 2nd +2 on this - https://review.openstack.org/#/c/114087/ - jogo / dtroyer ?20:43
mtreinishsdague: +A20:47
sdagueoh, right, you have grenade +2?20:48
*** stevebaker has quit IRC20:49
*** stevebaker has joined #openstack-qa20:49
*** stevebaker has quit IRC20:49
dtroyerI did add an answer regarding the ordering question anyway…20:49
*** stevebaker has joined #openstack-qa20:49
*** stevebaker has quit IRC20:49
mtreinishyeah, I cleaned up the acls while you were out: https://review.openstack.org/#/c/114024/20:49
*** stevebaker has joined #openstack-qa20:49
*** stevebaker has quit IRC20:49
mtreinishas part of the devstack adoption20:49
*** stevebaker has joined #openstack-qa20:49
*** stevebaker has quit IRC20:49
*** stevebaker has joined #openstack-qa20:49
*** stevebaker has quit IRC20:50
*** stevebaker has joined #openstack-qa20:50
*** stevebaker has quit IRC20:50
*** stevebaker has joined #openstack-qa20:50
*** stevebaker has quit IRC20:50
*** stevebaker has joined #openstack-qa20:50
*** stevebaker has quit IRC20:50
*** stevebaker has joined #openstack-qa20:50
gordcrandom question, is there a reason tempest doesn't use import_opt import_group when using oslo.config options not defined in module?20:50
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for horizon not registered bug 1362325  https://review.openstack.org/11735020:51
mriedemhorizon is exploding ^20:51
*** dkranz has quit IRC20:52
openstackgerrityunhong-jiang proposed a change to openstack/tempest: Test boot from volume checking  https://review.openstack.org/11673920:52
openstackgerrityunhong-jiang proposed a change to openstack/tempest: Passing parameter to create image  https://review.openstack.org/11673820:52
mtreinishgordc: most likely because tempest is run within a test runner and not as it's own process. That puts some unique constraints on how we have to deal with the CONF object21:01
*** samuelbercovici has quit IRC21:01
mriedemmtreinish: is that why using CONF.foo as a default arg in a method blows up?21:02
mtreinishgordc: but, I don't think anyone has tried to use those methods in tempest as alternative to what's there21:02
gordcmtreinish: i see... wanted to ask before i started to pour through files to make changes..21:02
gordcmtreinish: hmm... maybe i'll give it a shot. see if it breaks anything... i assume it can't hurt.21:03
mtreinishmriedem: well that might have been a bug that was fixed by: http://git.openstack.org/cgit/openstack/tempest/commit/?id=a96ee210f20145372ce9430dceca6700c7ef2aa521:03
mriedemmtreinish: jogo: sdague: i'm gonna start going through this list of old -1'ed e-r changes and abandon some https://review.openstack.org/#/q/status:open+project:openstack-infra/elastic-recheck,n,z21:03
mriedemdidn't realize there were so many21:03
*** krtaylor has quit IRC21:04
mtreinishgordc: no it probably won't hurt, but it would mean a large refactor. So the bigger question is it worth it? :)21:05
gordcmtreinish: i feel like something is being hinted at...21:07
*** dpaterson has quit IRC21:07
sdaguemriedem: good call21:07
gordcmtreinish: i'll take a quick look. i'm pretty sure ADHD will kick in before i finish first few files21:07
*** catohornet has joined #openstack-qa21:08
*** stevebaker has quit IRC21:08
*** stevebaker has joined #openstack-qa21:08
mtreinishgordc: heh, no just words from my experience, I've been down this particular config object rabbit hole a couple of times before. :)21:08
gordcmtreinish: lol i assume it ended up with tables being flipped...21:09
gordci'll save it for another day... the grep was enough to put it off.21:10
mtreinishheh, yeah pretty much...21:10
*** markmcclain has quit IRC21:12
*** nmagnezi__ has quit IRC21:13
*** catohornet has quit IRC21:14
*** e0ne has joined #openstack-qa21:15
*** aysyd has quit IRC21:15
*** mwagner_lap has quit IRC21:15
*** markmcclain has joined #openstack-qa21:18
*** e0ne has quit IRC21:18
*** e0ne has joined #openstack-qa21:22
jogomriedem: yeah without auto abandon ...21:23
mroddensdague: was waiting for someone to test that patch out a bit more, which it looks like dean got to doing21:24
jogomriedem mtreinish sdague: re gate debugging21:24
jogoI was going to help pleia2 write a e-r query today and document how to do it a a first pass at some docs21:24
jogobut your schedules didn't work out, so I am looking for someone else to help21:25
jogoany ideas21:25
mriedemjogo: mtreinish has a blog post on the topic21:25
mriedemand the e-r readme is pretty comprehensive on writing the queries21:25
jogomriedem: he does? link21:26
jogomriedem: well I was thinking we may need more docs around what the logs look like21:26
jogobut not sure until a fresh pair of eyes look21:26
mtreinishjogo: it's still a draft21:27
anteayamriedem: I'm stumped then21:27
anteayamriedem: http://logs.openstack.org/89/116889/1/gate/gate-tempest-dsvm-neutron-full/02de4a8/logs/devstacklog.txt.gz21:27
sdaguesure, honestly I think often it's better to get the fresh eyes to write the docs, because it comes with what the new person feels is important vs. the things we think are (which might be important, but require too much context to understand)21:27
anteayamriedem: why is that log on the unclassified page?21:27
mtreinishjogo: I still need to do things like open the bug and push the e-r change before I publish it21:27
anteayajogo: I don't think the problem is not enough docs21:28
mriedemanteaya: i don't see devstacklog.txt.gz on the uncategorized bugs page21:28
anteayathe problem is people don't see the point21:28
anteayamriedem: under gate-tempest-dsvm-neutron-full : 13 Uncategorized Fails. 84.7% Classification Rate (85 Total Fails)21:28
anteaya2014-08-27T00:48: http://logs.openstack.org/89/116889/1/gate/gate-tempest-dsvm-neutron-full/02de4a8/21:29
anteayais that not there for you?21:29
jogoanteaya: maybe just not the write docs. Or maybe we have them but they are hard to find. I think a fresh pair of eyes would help21:29
mriedemanteaya: i see it21:29
jogoas I am too close to really say21:29
anteayajogo: I had a session for people at neutron mid-cycle (2 people showed up, one a collegue of mriedem)21:29
anteayaI couldn't convince the other person of the need to classify and submit fingerprints21:30
mriedemi work with people?!21:30
mriedemanteaya: i see what you mean, subunit2cvs is in devstacklog.txt.gz in there but not console.html21:30
anteayaso I'm told21:30
mriedemthat's weird, anyway, couldn't hurt to have your change, let me find it21:30
anteayamriedem: I'll make the fingerprint be what you want21:30
anteayanot sure what it should be though21:30
SergeyLukjanovmriedem, sdague, I think I know the reason for https://bugs.launchpad.net/horizon/+bug/136232521:30
SergeyLukjanovmriedem, sdague, we've deprecated today the sahara-dashboard repo's code21:31
anteayajogo: having pleia2's input on e-r docs is great21:31
SergeyLukjanovsdague, https://review.openstack.org/#/c/112000 and https://review.openstack.org/#/c/11200121:31
mriedemanteaya: +W21:31
SergeyLukjanovsdague, will fix the issue21:32
anteayabut I don't know if it will move the needle for getting more than the 8 already creating fingerprints to create fingerprints21:32
jogoanteaya: sadly it will have to wait till next week21:32
anteayajogo: kk21:32
jogoanteaya: for pleia2's thoughts21:32
anteayak21:32
anteayamriedem: not sure what +W means, hope that it is good for me21:32
mriedemSergeyLukjanov: those 2 changes aren't merged though21:32
*** tkammer has joined #openstack-qa21:32
mriedemSergeyLukjanov: that horizon failure is in the gate21:32
cdentHas there been any thought put into making a tool for presenting the logs that shows them in some kind of folded format?21:32
mriedemanteaya: workflow + 121:32
mriedem+Walrus21:33
anteayamriedem: ah thanks21:33
anteayaand +Walrus21:33
SergeyLukjanovmriedem, this two changes will fix the issue21:33
cdentAt the moment you look at the logs and kind of do a "ow"21:33
anteayako ko kachew21:33
*** e0ne has quit IRC21:33
mriedemSergeyLukjanov: ah ok21:33
mriedemi'll update the bug21:33
cdentIf the logs were folded that might make them a little more diggable21:33
mriedemcdent: folded how? you can filter the logs by log level for starters21:33
mriedemthat's what i do when looking for test failures, i filter by trace21:33
cdentthat's only some of the los21:34
cdentlogs21:34
mriedemsince trace is what i'm usually going to use for e-r queries21:34
mriedemcdent: what did you have in mind?21:34
cdentI'm not sure if my experience is common, but the grenade.sh log is where I usually find myself at a first pass21:35
mriedemcdent: the grenade log is the most fun thing ever when it fails21:35
cdentIt will lead me the right place once I find the problem in it21:35
SergeyLukjanovmriedem, sdague, the reason for the https://bugs.launchpad.net/horizon/+bug/1362325 is that we removed deprecated code from the sahara-dashboard repo that is still used in the devstack code21:35
mriedem<sarcasm/>21:35
cdentbut initially it's just a bunch of noise about install stuff21:35
openstackgerritA change was merged to openstack-infra/elastic-recheck: Expanded fingerprint 1361924  https://review.openstack.org/11732021:35
cdentSo I don't really have an idea, just wonder if there's an idea somewhere in there.21:36
cdentI don't generally like to say "maybe we just need better tools" but they might help some.21:36
cdentIt might be the case that the best way to learn how to do it is just do it, but that doesn't help the first few goes for someone new.21:37
cdentWhen I first looked at the logs I really thought someone was playing a joke on me.21:37
mriedemyeah idk, there are a few common failures in there, (1) setup/install of the services (old/new), (2) tempest smoke test failures (old/new), and (3) sanity checks at the end21:38
mriedemwith any test failure bug log dump you should start at the end and work up21:38
mriedemthe 'failed to start service' in grenade.sh.log has thrown me off when that's not the failure21:39
anteayaokay mriedem since you are here and after you are finished chatting with cdent21:39
anteayawhat do I do with testr tests that pass and log checker that finds errors in the log files? http://logs.openstack.org/51/105351/5/gate/gate-tempest-dsvm-neutron-full/d9b1be5/console.html21:40
mriedemjogo: found a problem with the 'open reviews' change on the e-r status page,21:40
mriedemjogo: it's showing open reviews in e-r itself :)21:40
mriedemi don't think that was the intent21:40
mriedemanteaya: depends on which log files are allowed to have failures, but for sure n-cond can't have errors in it21:40
mriedemanteaya: the summary for the fail is at the end http://logs.openstack.org/51/105351/5/gate/gate-tempest-dsvm-neutron-full/d9b1be5/console.html#_2014-08-26_16_08_29_34921:41
anteayaso what does it mean if n-cond has errors and testr has succeeded?21:41
mriedemcdent: ^21:41
* cdent is watching21:41
mriedemanteaya: so this is bug 1353131 where we have a retry on deadlock in the db api code,21:42
mriedemanteaya: so we can have the error on the first pass and then it works on the 2nd21:42
mriedemso tests don't fail, but the n-cond log has the error21:42
mriedemmaybe we need to whitelist that guy now21:42
mtreinishmriedem: heh, that's probably because it searches gerrit for the bug id in the message21:42
mtreinishwhich e-r patches will hit21:42
mriedemi've looked for ways to fingerprint that (so has dolphm) which filters out successful hits and so far nothing great21:42
mriedemmtreinish: yup21:43
mriedemso we just need to filter out openstack-infra/elastic-recheck projects21:43
anteayaick21:43
mriedemanteaya: e.g. https://review.openstack.org/#/c/114566/21:43
jogomriedem: yeah I noticed that ;)21:44
SergeyLukjanovchmouel, dtroyer, ianw, jeblair, sdague, folks, could you please take a look at  https://review.openstack.org/#/c/112000 and https://review.openstack.org/#/c/112001 ? We need to land them to stop installing sahara-dashboard in the devstack code (we removed the deprecated code from the sahara-dashboard today), it's needed to unbreak the gate - https://bugs.launchpad.net/horizon/+bug/136232521:44
mriedemanteaya: the weird thing is i've run queries against logstash for the log file error thing in console.html and it shows successful runs, i'm not sure why21:44
mriedemjogo: i'm opening a bug to track21:45
anteaya:(21:45
anteayathe fingerprint you linked me to of dolph's has a link to this bug in the commit message: https://bugs.launchpad.net/nova/+bug/129759021:46
mriedemanteaya: yeah, there are 2, arosen's was from icehouse i think?21:46
anteaya2014-03-2621:46
anteayaso yeah, we can say icehouse21:46
openstackgerritMatthew Treinish proposed a change to openstack-infra/elastic-recheck: Add query for bug 1362347  https://review.openstack.org/11735821:47
mtreinishjogo, mriedem: oom-killer fun ^^^21:47
openstackgerritChris Dent proposed a change to openstack/tempest: Add scenario test for swift-proxy ceilometer middleware  https://review.openstack.org/11597121:48
mriedemlooking21:48
anteayaokay so I'll be honest, right now everything you said makes sense to you mriedem and gives me the deer in the headlights feeling21:49
anteayaas in, I have no idea what to do with what you told me21:49
mriedemmuwahahaha21:49
mriedemanteaya: the short answer on that one is it's complicated21:49
anteayathat I did understand21:50
anteayaI can wander away quietly and try to find a safter unclassified failure21:50
anteayas/safter/safer21:50
mriedemso i'm searching on something like this:21:50
mriedemmessage:"Not Whitelisted" AND message:"ERROR nova.quota" AND message:"Failed to commit reservations" AND tags:"console"21:50
anteayagreat21:50
mriedemthat gives me 323 hits in 7 days, multiple queues but it's not 100% fail21:51
dtroyerSergeyLukjanov: both of those reviews appear to be failing on Xenserver multiple times, can you characterize what that is about?21:51
mriedemdtroyer: that's the neutron oom killer21:51
mriedemBobBall_Away: was talking about that in nova this morning21:52
mriedemdtroyer: xenserver ci has been blowing up due to that all week i think21:52
dtroyermriedem: ah, ok, thanks.  I hadn't looked at those jobs for a while21:52
mriedemanteaya: so getting back, i'd add that to the known bug fingerprint but the success rate on that query is too high21:53
mriedemanteaya: if i could figure out why it's successful in some runs we could filter those out, e.g. maybe there are jobs that aren't failing on errors in n-cond?21:53
mriedemanteaya: http://logs.openstack.org/40/111840/10/check/check-tempest-dsvm-full/34262e3/console.html#_2014-08-27_18_14_56_28421:54
SergeyLukjanovdtroyer, ++ for mriedem comment, I don't see any patch specific things in their logs, additionally there was +1 on the both patches from the xen ci21:54
chmouelSergeyLukjanov: look good to me but i haven't tested, i assume it works for you guys21:54
mriedemanteaya: if we could figure out when it's OK for log files to have errors (n-cond in this case), we can filter those out, but i don't have the answeratm21:55
mriedemi'm sure others in here would know21:55
*** dims has quit IRC21:55
mriedemjogo: https://bugs.launchpad.net/openstack-ci/+bug/1362348 i'll try to take a crack at that sometime when i find the time21:55
SergeyLukjanovchmouel, all sahara dashboard code has been merged to horizon and so we've started removing all old code related to the separated repo...21:56
dtroyerSergeyLukjanov: +2A21:56
chmouelok21:57
*** tkammer has quit IRC21:58
*** mriedem has quit IRC22:00
SergeyLukjanovdtroyer, thank you22:00
*** ianw has quit IRC22:04
*** malini1 has quit IRC22:04
*** ianw has joined #openstack-qa22:04
adam_gmtreinish, does the TODO added here sufficiently address your concerns? id like to improve that soonish but dont want to block the current ironic patch queue22:05
openstackgerritA change was merged to openstack-dev/grenade: Save base and target release state correctly  https://review.openstack.org/11500122:06
cdentgo me22:07
*** cdent has quit IRC22:08
*** gordc has quit IRC22:08
mtreinishadam_g: added where? Sorry it's just a context switch for me, I know we were talking about it yesterday...22:09
*** bnemec has quit IRC22:11
*** bnemec has joined #openstack-qa22:12
adam_gmtreinish, jeez, my bad https://review.openstack.org/#/c/113707/22:12
*** harlowja_ has joined #openstack-qa22:13
*** harlowja_ has quit IRC22:15
*** harlowja_ has joined #openstack-qa22:15
*** harlowja has quit IRC22:16
anteayaI missed him, I was on the phone22:26
*** mkollaro has quit IRC22:29
*** weshay has quit IRC22:34
*** sdmonov has quit IRC22:34
*** sballe_ has joined #openstack-qa22:35
*** bkopilov has quit IRC22:36
*** jgrimm is now known as jgrimm_zz22:37
*** sballe has quit IRC22:37
*** markmcclain has quit IRC22:38
*** krtaylor has joined #openstack-qa22:45
mtreinishadam_g: yeah, that was TODO covered it22:46
sdaguemtreinish: your -2 here can probably come off - https://review.openstack.org/#/c/96661/22:47
*** bkopilov has joined #openstack-qa22:48
*** ZackaryDoyle has joined #openstack-qa22:56
*** dimtruck is now known as zz_dimtruck22:57
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: clean up readme  https://review.openstack.org/11501123:00
jogosdague mtreinish: the fixed readme from dolph  ^23:00
jogofor e-r23:01
sdague+223:02
sdagueok, dinner time. Enjoy23:02
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for horizon not registered bug 1362325  https://review.openstack.org/11735023:08
*** jamielennox|away is now known as jamielennox23:10
salv-orlandomtreinish, sdague: is anybody already working on this? http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiTm90UmVnaXN0ZXJlZDogRGFzaGJvYXJkIHdpdGggc2x1ZyBcXFwic2FoYXJhXFxcIiBpcyBub3QgcmVnaXN0ZXJlZFwiIEFORCB0YWdzOlwiaG9yaXpvbl9lcnJvci50eHRcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiODY0MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDA5MTgxMDQxNjAxLCJtb2RlIjoiIiwiYW5hbHl6ZV9maWVsZCI6IiJ923:12
salv-orlandosdague, mtreinish: here it is, mriedem filed it https://bugs.launchpad.net/horizon/+bug/136232523:14
morganfainbergmtreinish, sdague, is there a way to log in devstack from within a function that's echo will be captured as the output?23:21
morganfainbergmtreinish, sdague, not seeing a clear way to do that23:21
morganfainbergi guess echo $msg 1>&2; would work.23:23
dtroyermorganfainberg: warn() is the closest thing… yeah…23:23
morganfainbergdtroyer, i was asked to log what apache version was detected in https://review.openstack.org/#/c/10242523:23
morganfainbergdtroyer, should i just explicitly echo, make a "info()" use "warn()"?23:24
dtroyermorganfainberg: ah, found it…echo_nolog() is defined in stack.sh.  that should be moved to functions-common, that's what you want if yo don't want it logged23:26
morganfainbergah. ok23:26
morganfainbergdtroyer, no uhm here look at Robbie's comment arounjd 109 https://review.openstack.org/#/c/102425/5/lib/apache23:27
morganfainbergerm23:27
morganfainbergdtroyer, line 70 that is23:27
morganfainbergdtroyer, is what i'm trying to address23:27
dtroyerso this is all run with xtrace on, right?  where does he want it logged?23:28
morganfainbergyeah i dunno23:29
morganfainbergi think i'm going to respond with 'but xtrace is on'23:29
morganfainbergnow that i'm re-looking at it23:29
dtroyerand if VERBOSE=False, you don't care…oh, that's why…write only to log and not stdout then23:29
morganfainbergso, do i need to do anything?23:30
dtroyerI don't think so?23:30
morganfainbergdtroyer, yeah that was my view.23:30
morganfainbergdtroyer, ok making sure i didn't miss something23:30
morganfainbergdtroyer, thanks!23:30
dtroyerHe needs to clarify if xtrace isn't enough23:30
dtroyernp23:31
openstackgerritMorgan Fainberg proposed a change to openstack-dev/devstack: Use the apache 2.4 ErrorLogFormat directive  https://review.openstack.org/10242523:33
*** oomichi has joined #openstack-qa23:39
* gmann is back (gone 14:16:04)23:41
morganfainberggmann, not sure about this channel but typically autoaway/autoreturn messages are frowned upon (you may want to disable that)23:42
openstackgerritMorgan Fainberg proposed a change to openstack-dev/devstack: Use the apache 2.4 ErrorLogFormat directive  https://review.openstack.org/10242523:51
openstackgerritMorgan Fainberg proposed a change to openstack-dev/devstack: Use the apache 2.4 ErrorLogFormat directive  https://review.openstack.org/10242523:53
*** bkopilov has quit IRC23:57
openstackgerritJames E. Blair proposed a change to openstack-dev/devstack: Move sahara client installation to lib/sahara  https://review.openstack.org/11200023:58
*** dims has joined #openstack-qa23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!