Thursday, 2014-09-04

*** mlavalle has joined #openstack-qa00:03
*** hockeynut has quit IRC00:04
*** hockeynut has joined #openstack-qa00:05
*** ZZelle_ has quit IRC00:06
openstackgerritSalvatore Orlando proposed a change to openstack/tempest: Test for helping root cause bug 1349617  https://review.openstack.org/11892500:08
*** flaper87 is now known as flaper87|afk00:08
*** nikhil__1 has joined #openstack-qa00:11
*** devananda has joined #openstack-qa00:11
*** stenchlarge has quit IRC00:12
*** nikhil___ has quit IRC00:12
*** dhellmann_ has quit IRC00:12
*** rfolco has quit IRC00:12
*** kmartin has quit IRC00:12
*** esmute has quit IRC00:12
*** devanand1 has quit IRC00:12
*** esmute has joined #openstack-qa00:12
*** dhellmann has joined #openstack-qa00:12
ianwdtroyer / sdague: https://review.openstack.org/#/c/118226/ (Move to epel 7-1 release) <- be great if you could sign-off on this00:14
*** melwitt has quit IRC00:19
*** melwitt has joined #openstack-qa00:19
*** dims has joined #openstack-qa00:21
*** rfolco has joined #openstack-qa00:21
*** sballe has quit IRC00:23
*** dims_ has joined #openstack-qa00:24
*** dims has quit IRC00:27
*** dmorita has joined #openstack-qa00:27
*** yamahata has joined #openstack-qa00:33
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-dev/devstack: Add oslo.concurrency and oslo.middleware  https://review.openstack.org/11730400:37
*** yamahata has quit IRC00:39
*** yamahata has joined #openstack-qa00:39
*** suneelb has quit IRC00:42
*** esmute has quit IRC00:50
*** esmute has joined #openstack-qa00:50
mlavallemtreinish: you around?00:53
*** mlavalle has quit IRC00:55
*** mriedem has joined #openstack-qa01:12
openstackgerritGhanshyam Mann proposed a change to openstack/tempest: Fix V2 hypervisor server schema attribute  https://review.openstack.org/11893601:17
dtroyerianw: +2.   can that wait until the check/gate queues thin a bit?01:17
*** jyuso has joined #openstack-qa01:18
ianwdtroyer: yep, just like to get my centos7/rhel7 jobs working :)01:21
*** liusheng has joined #openstack-qa01:21
mriedemsdague: mtreinish: jogo: i've never seen this worlddump in a grenade job before, not sure what is happening when javelin is called and the ping failures are happening01:30
mriedemhttp://logs.openstack.org/28/117628/14/check/check-grenade-dsvm/82a7c5c/logs/grenade.sh.txt.gz#_2014-09-03_06_08_33_44401:30
*** bnemec has quit IRC01:32
*** bnemec has joined #openstack-qa01:32
*** mlavalle has joined #openstack-qa01:33
jogoworlddump is running at he end01:38
jogoits not the failure01:38
mriedemright01:39
mriedemthe ping thing is01:39
jogomriedem: so that is odd there should be more logs01:40
mriedemjogo: well, it looks like it's the guys change that's blowing it up01:40
mriedemthat ping sendmsg op not permitted only shows up in his change01:40
*** ekhugen has left #openstack-qa01:40
mriedemhe's playing with firewall rules01:40
mriedemdepr01:40
mriedem*derp01:40
jogoyeah that is a good guess01:41
jogobut the logs missing is odd01:41
jogoI wonder if tempest config stuff changed underneath me01:41
jogomtreinish: ^01:41
jogomriedem: javelin should be emitting tempest logs01:42
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-dev/devstack: Add oslo.log  https://review.openstack.org/11833901:42
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Make output from check_logs less verbose  https://review.openstack.org/11874401:43
*** anvilmutant has joined #openstack-qa01:44
*** anvilmutant has quit IRC01:45
jogomriedem: yeah so there are some logs missing01:45
jogobut it looks like its his fault01:45
jogoon an unrelated note I found a fun novaclient bug01:45
jogoerr performance fail01:45
jogomriedem: oh I know what happened with the logs01:48
jogothey aren't being collected yet01:48
jogomriedem: https://review.openstack.org/#/c/113658/01:49
*** mlavalle has quit IRC01:49
*** HenryG has quit IRC01:51
openstackgerritJoe Gordon proposed a change to openstack/tempest: Move javelin2 over to use oslo logging  https://review.openstack.org/10760901:52
*** nosnos has joined #openstack-qa01:54
*** yjiang5 has quit IRC02:01
*** mriedem has quit IRC02:05
*** zz_dimtruck is now known as dimtruck02:16
*** HenryG has joined #openstack-qa02:40
*** ianw has quit IRC02:46
openstackgerritmelanie witt proposed a change to openstack/tempest: send paramiko logs to a file "paramiko.log"  https://review.openstack.org/11894602:46
*** ianw has joined #openstack-qa02:46
openstackgerritmelanie witt proposed a change to openstack-infra/devstack-gate: collect paramiko logs from tempest runs  https://review.openstack.org/11894702:50
*** yfried has joined #openstack-qa02:59
*** pcrews has quit IRC03:02
*** yfried has quit IRC03:07
*** chandankumar has joined #openstack-qa03:09
*** chandankumar has quit IRC03:14
openstackgerritHirofumi Ichihara proposed a change to openstack-dev/devstack: Fix keystone auth parameter to neutron conf  https://review.openstack.org/11851203:16
dtroyeradam_g: I finally got around to thinking through https://review.openstack.org/118740 and think there is a more consistent approach.  Sorry for the delay, connectivity was #%%^@#$!# for me today…03:19
*** Poornima has joined #openstack-qa03:26
*** shardy_z has quit IRC03:40
*** chandankumar has joined #openstack-qa03:58
*** chandankumar has quit IRC04:09
jogodtroyer: looks like grenade is one of the least reliable gate components http://jogo.github.io/gate/04:12
*** chandankumar has joined #openstack-qa04:13
*** dims_ has quit IRC04:21
*** dims has joined #openstack-qa04:21
*** dims has quit IRC04:26
*** morganfainberg_Z is now known as morganfainberg04:29
*** melwitt has quit IRC04:39
Santosh_gmann: hi04:39
Santosh_gmann: Request you to please review - https://review.openstack.org/#/c/116771/04:40
*** vaidy has joined #openstack-qa04:42
*** chandankumar has quit IRC04:45
*** vaidy-ril has joined #openstack-qa04:45
*** vaidy-ril has quit IRC04:46
clarkbjogo: it should get better once https://review.openstack.org/#/c/118753/ goes in04:47
clarkbI am addressing the pep8 fail. that error message ugh04:47
clarkbyes I should use an if statemtn there why not04:47
openstackgerritClark Boylan proposed a change to openstack/tempest: Properly detect grenade in check_logs script  https://review.openstack.org/11875304:48
*** vaidy has quit IRC04:49
*** Madan has joined #openstack-qa04:59
*** Ch00k has joined #openstack-qa05:22
Santosh_cyeoh: hi05:38
*** dimtruck is now known as zz_dimtruck05:40
*** rdekel has joined #openstack-qa05:41
*** chandankumar has joined #openstack-qa05:41
*** Longgeek has joined #openstack-qa05:44
*** neeti has joined #openstack-qa05:48
*** harlowja is now known as harlowja_away05:50
*** afazekas has joined #openstack-qa05:51
*** nmagnezi has joined #openstack-qa05:55
*** SridharG has joined #openstack-qa06:11
*** Longgeek has quit IRC06:12
*** flaper87|afk is now known as flaper8706:12
*** yfried has joined #openstack-qa06:12
*** Longgeek has joined #openstack-qa06:18
*** yfried_ has joined #openstack-qa06:19
*** yfried has quit IRC06:20
*** Longgeek_ has joined #openstack-qa06:21
*** Longgeek_ has quit IRC06:22
*** Longgeek_ has joined #openstack-qa06:23
*** akamyshnikova has quit IRC06:24
*** Longgeek has quit IRC06:24
*** Longgeek has joined #openstack-qa06:24
*** amotoki has joined #openstack-qa06:25
*** Longgeek_ has quit IRC06:26
gmannSantosh_: i will check that06:33
Santosh_gmann: Thanks .06:33
*** gszasz has joined #openstack-qa06:38
coolsvapgmann, hi06:40
gmanncoolsvap: hi06:41
coolsvapgmann, can you review https://review.openstack.org/115871?06:41
gmanncoolsvap: i will look.06:43
openstackgerritSwapnil Kulkarni proposed a change to openstack/tempest: Updated compute keypair clients for bp client-checks-success  https://review.openstack.org/11643506:44
coolsvapgmann, thx!06:44
*** marun_ has joined #openstack-qa06:49
*** stevebaker has quit IRC07:04
*** adam_g has quit IRC07:04
*** wendar has quit IRC07:04
*** wendar_ has joined #openstack-qa07:04
*** jpich has joined #openstack-qa07:05
*** stevebaker has joined #openstack-qa07:06
*** adam_g has joined #openstack-qa07:06
*** adam_g has quit IRC07:06
*** adam_g has joined #openstack-qa07:06
*** Madan has quit IRC07:06
*** luqas has joined #openstack-qa07:07
*** tkammer has joined #openstack-qa07:10
Santosh_yfried_: hi07:10
yfried_Santosh_: hi07:10
Santosh_yfried_: reuqest you to pls review - https://review.openstack.org/#/c/116771/07:10
*** evgenyf has joined #openstack-qa07:11
Santosh_marun_: hi07:13
*** sabeen has quit IRC07:14
*** sabeen has joined #openstack-qa07:14
yfried_Santosh_: quick question. in your patch "self.port" is the output of self.create_port() am I right?07:15
Santosh_yfried_: yes07:16
yfried_Santosh_: why would it give different output than show_port? they are both server outputs. not compared against some port dict created by the test07:17
*** neeti has quit IRC07:18
Santosh_yfried_: bcz ... there is a bug in neutron... that create port does not considers ... dhcp_opts option...07:18
Santosh_yfried_: however show port have that..thats why as per suggestion by andrea... i have used custom matchers..07:19
yfried_Santosh_: and?07:19
*** Madan has joined #openstack-qa07:19
yfried_Santosh_: so please add a comment with the bug num and "todo" to fix that once the bug is fixed07:20
Santosh_yfried_: This bug still has no number.... means i just observed... by using pdb....07:20
*** Longgeek has quit IRC07:20
*** neeti has joined #openstack-qa07:21
Santosh_yfried_:  my next target will be to raise a bug in neutron.... and fix there..07:21
yfried_Santosh_: so please submit it.07:21
Santosh_yfried_: sure will do that..07:21
*** Longgeek has joined #openstack-qa07:21
Santosh_yfried_: But for this atleast we can make check in ...which tests all other values ...except this one ..07:22
Santosh_yfried_: right ?07:22
yfried_Santosh_: I didn't understand your last question. sorry07:22
Santosh_yfried_: Means i would request that for now ..let this patch land in .... and we will work for fixing neutron bug in later patch..07:24
yfried_Santosh_: you can fix the bug (or not) when you have the time, but if you are doing some non-trivial stuff, you should add a comment inline to explain why07:25
*** Longgeek_ has joined #openstack-qa07:26
yfried_Santosh_: If I didn't have you online right now I'd never be able to understand this assertion.07:27
*** Longgeek_ has quit IRC07:27
*** Ch00k_ has joined #openstack-qa07:27
*** jlibosva has joined #openstack-qa07:27
*** Longgeek_ has joined #openstack-qa07:27
*** Ch00k has quit IRC07:27
*** Longgeek has quit IRC07:29
*** shardy has joined #openstack-qa07:30
*** Santosh_ has quit IRC07:30
*** jlanoux has joined #openstack-qa07:30
*** ildikov has joined #openstack-qa07:31
yfried_Shrews: adam_g: any of you guys around?07:32
*** Longgeek has joined #openstack-qa07:33
*** Longgeek_ has quit IRC07:36
*** Longgeek has quit IRC07:37
*** Longgeek has joined #openstack-qa07:38
*** mspreitz has quit IRC07:40
*** e0ne has joined #openstack-qa07:46
openstackgerritSantosh Kumar proposed a change to openstack/tempest: Check port's status also in show port tests  https://review.openstack.org/11677107:46
*** mpaolino has joined #openstack-qa07:48
*** tkammer has quit IRC07:57
openstackgerritSantosh Kumar proposed a change to openstack/tempest: Check port's status also in show port tests  https://review.openstack.org/11677108:02
*** qba73 has joined #openstack-qa08:04
openstackgerritBenny Kopilov proposed a change to openstack/tempest: Glance Negative - chanage image owner  https://review.openstack.org/11899508:11
*** sabeen has quit IRC08:11
*** safchain has joined #openstack-qa08:13
*** yfried_ has quit IRC08:16
*** ZZelle has quit IRC08:18
*** ZZelle has joined #openstack-qa08:19
*** rdekel has quit IRC08:44
*** lordd has joined #openstack-qa08:51
openstackgerritSantosh Kumar proposed a change to openstack/tempest: Check port's status also in show port tests  https://review.openstack.org/11677108:51
*** tosky has joined #openstack-qa08:53
*** Ch00k_ has quit IRC08:55
*** BobBall has joined #openstack-qa08:57
*** luqas has quit IRC09:03
BobBallsdague: I'd like to set the maximum number of API workers in devstack's default configuration for Nova (for metadata, API and ec2 API), for cinder and for glance to the same calculation as DEFAULT_CONCURRENCY in d-g (which ends up as TEMPEST_CONCURRENCY).  As we know that Tempest won't run with more threads than this, I think that means we know no more than this number of API nodes can be used at any one time?09:04
BobBallThe drive behind this is reducing memory usage as each worker can use ~100mb ram which is a notable chunk when multiplied up on an 8 processor system (4*5*100=2G more than necessary!)09:07
openstackgerritBenny Kopilov proposed a change to openstack/tempest:  test test  https://review.openstack.org/11900609:08
*** lordd has quit IRC09:10
*** mkollaro has joined #openstack-qa09:10
*** tkammer has joined #openstack-qa09:11
*** Longgeek_ has joined #openstack-qa09:13
*** Longgeek has quit IRC09:15
*** sridhar has joined #openstack-qa09:17
*** Longgeek_ has quit IRC09:17
*** Longgeek has joined #openstack-qa09:18
*** lordd has joined #openstack-qa09:21
*** luqas has joined #openstack-qa09:22
*** andreaf has joined #openstack-qa09:22
*** evgenyf has quit IRC09:24
*** evgenyf has joined #openstack-qa09:27
*** Ch00k has joined #openstack-qa09:30
*** Longgeek_ has joined #openstack-qa09:43
*** Longgeek has quit IRC09:46
openstackgerritSridhar Gaddam proposed a change to openstack/tempest: Improve IPV6 parity in Security Group testcases  https://review.openstack.org/9413010:13
*** nosnos has quit IRC10:14
*** nosnos has joined #openstack-qa10:15
*** yamahata has quit IRC10:16
*** nosnos_ has joined #openstack-qa10:18
*** nosnos has quit IRC10:19
*** cdent has joined #openstack-qa10:30
*** k4n0_ has quit IRC10:33
*** k4n0 has joined #openstack-qa10:35
*** dvinayku_ has joined #openstack-qa10:36
dvinayku_hello anyone online10:36
dvinayku_?10:37
*** jlanoux has quit IRC10:40
*** aix has joined #openstack-qa10:42
*** dmorita has quit IRC10:58
*** e0ne has quit IRC11:01
*** jyuso has quit IRC11:02
*** dims has joined #openstack-qa11:04
cmysterno11:05
*** Ch00k has quit IRC11:06
*** bkopilov has quit IRC11:12
*** mpaolino has quit IRC11:12
*** lordd has quit IRC11:13
*** lordd has joined #openstack-qa11:14
sdagueBobBall: we don't need Tempest processes to be < API workers. Think of API workers as pools.11:17
sdagueeventlet will dispatch multiple API requests at the same time even if we've got a single API worker, that's one of the reasons for using it11:18
*** dims has quit IRC11:19
sdaguewhat I meant in my comments is we should have a knob that changes the API workers, to handle the memory issue11:19
*** dims has joined #openstack-qa11:19
*** dims_ has joined #openstack-qa11:21
*** dims has quit IRC11:24
*** Poornima has quit IRC11:25
openstackgerritIan Wienand proposed a change to openstack-dev/devstack: Fix tools/xen/functiosn for bashate errors  https://review.openstack.org/11903711:26
*** dims_ has quit IRC11:26
*** dims has joined #openstack-qa11:27
*** jlanoux has joined #openstack-qa11:27
*** dvinayku_ has quit IRC11:28
BobBallsdague: see https://review.openstack.org/#/c/117517/ (specifically IanW's very valid concerns) - I was trying to think of a hard-coded value we can use that'd avoid mass proliferation of localrc variables11:28
*** bkopilov has joined #openstack-qa11:29
*** dims_ has joined #openstack-qa11:30
openstackgerritIan Wienand proposed a change to openstack-dev/devstack: Fix tools/xen/functions for bashate errors  https://review.openstack.org/11903711:30
*** eglynn__ is now known as eglynn-officeafk11:31
*** dims has quit IRC11:31
sdaguehonestly, we really need this global knob for other reasons imo11:32
BobBallok, thanks.11:38
*** Ch00k has joined #openstack-qa11:38
*** mwagner_lap has quit IRC11:40
*** gmann_ has joined #openstack-qa11:40
gmann_andreaf: hi11:41
*** dims_ has quit IRC11:42
*** dims has joined #openstack-qa11:43
*** luqas has quit IRC11:43
*** dims_ has joined #openstack-qa11:45
*** dims has quit IRC11:47
*** parstac_pete_ has joined #openstack-qa11:49
*** jroll|dupe has joined #openstack-qa11:50
*** dtroyer_zz has joined #openstack-qa11:50
*** jamielenz has joined #openstack-qa11:50
openstackgerritIan Wienand proposed a change to openstack-dev/devstack: Fix tools/xen/functions for bashate errors  https://review.openstack.org/11903711:50
*** bdperkin_ has joined #openstack-qa11:50
*** jroll has quit IRC11:50
*** bdperkin has quit IRC11:51
*** peterstac has quit IRC11:51
*** dtroyer has quit IRC11:51
*** jamielennox has quit IRC11:51
*** jamielenz is now known as jamielennox11:51
*** parstac_pete_ is now known as peterstac11:51
*** jroll|dupe is now known as jroll11:51
*** bdperkin_ is now known as bdperkin11:51
*** bdperkin has quit IRC11:51
*** bdperkin has joined #openstack-qa11:51
*** e0ne has joined #openstack-qa11:57
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Fix use of nonexistent class variable in accounts  https://review.openstack.org/11463611:58
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Add Credentials Provider factory  https://review.openstack.org/10768511:58
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Force use of Accounts for test purposes  https://review.openstack.org/11457811:58
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Add a NotLockingAccounts credentials provider  https://review.openstack.org/11863511:58
andreafgmann_ back from lunch and going to a meeting now - I'll be back in ~30min11:59
gmann_andreaf: sure11:59
*** mwagner_lap has joined #openstack-qa12:01
*** e0ne has quit IRC12:02
*** dims_ has quit IRC12:05
*** dims has joined #openstack-qa12:06
ianwBobBall / sdague : i still think the semantics of the flag are not that clear.  an example local.conf snippet that could be merged in before running would be better IMO -- that way it is exactly clear what is being set.  but also if i'm outvoted, np12:07
sdagueianw: that means you'd need to specify it for every service, I actually explicitly want a global flag12:08
sdague"this is a constrained environment, please don't use multi workers"12:08
*** e0ne has joined #openstack-qa12:08
ianwsdague: it already seems to miss some things like heat, ceilometer, neutron.  i think it's quite likely we forget about this when adding services too12:11
*** flaper87 is now known as echo12:11
*** echo is now known as flaper8712:11
sdaguewell, I think most of those still default to 1 worker12:12
sdagueglance and cinder only recently had defaults changed to go multi worker12:13
*** dpaterson has joined #openstack-qa12:14
*** Madan has quit IRC12:14
ianwalright, well i've nothing more to add.  it's only code, if there's a problem we can change it later :)12:14
*** gmann__ has joined #openstack-qa12:19
*** mpaolino has joined #openstack-qa12:20
*** gmann_ has quit IRC12:21
*** aysyd has joined #openstack-qa12:27
*** weshay has joined #openstack-qa12:27
*** adalbas has joined #openstack-qa12:28
psedlakandreaf: ping - hi, your change https://review.openstack.org/#/c/112115/10/tempest/common/custom_matchers.py introduced py26 incompat, would you mind me proposing change to do try/import/except/from unittest2 ... to fix it?12:33
*** mpavlase has joined #openstack-qa12:37
*** dosaboy_ is now known as dosaboy12:38
andreafpsedlak, I think there may be a change proposed for that already - let me see if I can find it12:38
andreafgmann: hi - what's up?12:39
openstackgerritA change was merged to openstack-dev/grenade: Extract database dumps to function  https://review.openstack.org/11776812:39
*** flaper87 is now known as flaper87|afk12:40
gmann__andreaf: regarding https://review.openstack.org/#/c/114636/12/tempest/common/accounts.py12:40
gmann__andreaf: thanks for explanation. I have 1 question for concurrency12:41
andreafpsedlak, uhm - I cannot find it - but we don't test python 2.6 - any reason you're concerned about it?12:41
andreafgmann__, yes?12:42
*** flaper87|afk is now known as flaper8712:42
andreafgmann__, I've added now https://review.openstack.org/#/c/114636/13/etc/accounts.yaml.sample,cm12:43
gmann__how user will get to know the exact number for concurrency, if that is less than number of tests running in parallel needed acounts at sme time, then tests going to fail12:43
andreafgmann__, concurrency is the number of python processes spawned by testr, it's either defined when starting the tests or testr defaults to the number of CPU cores available12:44
*** luqas has joined #openstack-qa12:44
andreafgmann__, in the gate we default to the number of CPU cores / 212:45
andreafWithin each python process tests are run serially, so in the worst case there will be  (# of processes) x (# of accounts per test)12:46
gmann__andreaf: yes.12:46
andreafthe number of accounts per test in <=2 - except for test which create additional test users - but they need admin credentials for that, so those tests would be skipped anyways if running without admin credentials configured12:47
andreafgmann__, which is the point of having pre-provisioned accounts12:47
gmann__andreaf: in case where account are not sufficient (yes that is invalid configuration) should we fail the test, IMO we should have some wait there to get some account free.12:48
andreafgmann__, if there are not enough accounts configured that will keep failing - it's better to run tests with less concurrency if more accounts cannot be configured12:50
gmann__andreaf: i mean tempest should be little bit flexible on this configuration, as this is parallel execution of tests not very hard requirement for running tests12:50
andreafs/configured/provisioned12:50
andreafgmann__, I don't get your point about being flexible - if you want to use parallel mode, and you don't allow tenant isolation to provision accounts on the fly - enough pre-provisioned accounts have to be configured12:52
andreafgmann__, I don't think tempest should try to work around incorrect configuration - that would give the false message that the configuration was fine and you'll end up wondering why your test run took double the time you expected it to take12:54
gmann__andreaf: agree with your point.12:54
andreafwe discussed this at the mid-year meetup and agreed that it was better to keep it simple12:54
gmann__andreaf: yes, that's what i thought now, wait will add extra time for tests12:55
*** ekhugen has joined #openstack-qa12:55
gmann__andreaf: that's cool.12:57
*** yamahata has joined #openstack-qa12:58
gmann__acdreaf: thanks for clarification :)12:58
*** dkranz has quit IRC12:58
openstackgerritA change was merged to openstack-infra/devstack-gate: Add oslo.log to devstack-vm-gate-wrap.sh  https://review.openstack.org/11699512:58
andreafgmann__, thank you for your reviews12:59
gmann__andreaf: i saw the comment in accounts.yaml.sample, that is helpfull. do you think mentioning about tools/create-test-accounts.py there will be more clear for users to configured the accounts?13:01
andreafgmann__, sure, perhaps that should be added in https://review.openstack.org/#/c/107758/113:02
gmann__andreaf: yes. that will be right patch.13:02
gmann__andreaf: thanks again. going to bed now. see u later :)13:03
andreafgmann__, good night13:04
*** gmann__ has quit IRC13:04
*** marun_ has quit IRC13:07
*** neeti has quit IRC13:11
*** eglynn-officeafk is now known as eglynn-office13:11
*** k4n0 has quit IRC13:12
*** dkranz has joined #openstack-qa13:14
*** tkammer has quit IRC13:14
mtreinishmlavalle: I am now, what's up13:14
mtreinishpsedlak, andreaf: that gets fixed in: https://review.openstack.org/#/c/116741/13:16
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-dev/devstack: Add oslo.concurrency and oslo.middleware  https://review.openstack.org/11730413:17
*** mestery has quit IRC13:17
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-dev/devstack: Add oslo.log  https://review.openstack.org/11833913:17
openstackgerritDavanum Srinivas (dims) proposed a change to openstack-dev/devstack: Add oslo.log  https://review.openstack.org/11833913:17
*** mriedem has joined #openstack-qa13:18
*** Ch00k has quit IRC13:19
andreafmtreinish: ok thank you13:21
*** mestery has joined #openstack-qa13:21
*** mestery has quit IRC13:21
*** mestery has joined #openstack-qa13:22
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Add Credentials Provider factory  https://review.openstack.org/10768513:25
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Backward compatibility to credentials in conf  https://review.openstack.org/11906013:25
*** marun_ has joined #openstack-qa13:25
andreafmtreinish, proposed solution for backward compatibility on credentials from tempest.conf ^^^13:26
andreafmtreinish, I also added unit tests for the NotLockingAccounts https://review.openstack.org/#/c/118635/213:26
mtreinishandreaf: hmm, that looks a lot like the test hack code :)13:27
mtreinishI was thinking we wouldn't write out a yaml, but instead just check if the yaml file is actually set it in config13:27
*** pcrews has joined #openstack-qa13:27
mtreinishand if it wasn't just return the creds from the other options in NotLockingAccounts13:27
*** dkranz has quit IRC13:28
*** Ch00k has joined #openstack-qa13:29
*** dkranz has joined #openstack-qa13:29
*** tkammer has joined #openstack-qa13:29
*** chandankumar has quit IRC13:34
*** amotoki_ has joined #openstack-qa13:35
*** dkranz has quit IRC13:36
andreafmtreinish, the NotLockingAccounts still uses the YAML13:36
andreafmtreinish, only it does not lock the accounts and always returns account #0 for primary and account #1 for alt13:36
andreafmtreinish, that way the new code only uses the YAML, the only exception being the bit of code which generates the YAML from tempest.conf13:38
mtreinishandreaf: I was thinking something like: http://paste.openstack.org/show/105794/13:40
andreafmtreinish: ok I see13:45
andreafmtreinish, it looks good13:46
mtreinishandreaf: ok, I'll push it up, should I steal your patch? Or push it as a separate one?13:47
andreafmtreinish: no there's a problem, we have a default value for the credentials file13:47
andreafmtreinish: so we need to check if the file is there, not only if it's configured13:48
andreafmtreinish: perhaps try except around self.get_cred() with fallback to auth.get_default_credential() is ok13:49
mtreinishoh, hmm ok I can check for the file first and set a class level flag of whethere it exists or not13:49
*** nosnos_ has quit IRC13:50
*** tkammer has quit IRC13:51
*** nosnos has joined #openstack-qa13:51
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for oslotest/infra py33 bug 1365512  https://review.openstack.org/11906313:51
andreafmtreinish, do you want to upload a new version of https://review.openstack.org/119060?13:52
*** toure has joined #openstack-qa13:52
*** weshay has quit IRC13:54
mtreinishandreaf: yeah I can13:55
*** sballe has joined #openstack-qa13:55
*** nosnos has quit IRC13:55
andreafmtreinish: ok great, thank13:55
*** cnesa10 has quit IRC13:55
andreafmtreinish: btw, I won't be able to join the meeting tonight - but I would like to ask for spec reviews of https://review.openstack.org/#/c/118352/ and https://review.openstack.org/9474113:56
andreafmtreinish: I think they should be both hi-prio bps for end of juno / kilo13:58
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Backward compatibility to credentials in conf  https://review.openstack.org/11906014:00
mtreinishandreaf: yeah, I agree. I'll put them on the agenda14:01
andreafmtreinish: thank you14:01
mtreinishandreaf: also I just pushed the patch ^^^14:02
*** yamahata has quit IRC14:02
mtreinishalthough I'm thinking I need to change the IOError to invalid configuration14:02
mtreinishbecause it's not actually trying IO14:02
*** yamahata has joined #openstack-qa14:02
jogosdague: http://jogo.github.io/gate/14:03
jogowee14:03
andreafmtreinish: well that should never happen even, or?14:04
*** tkammer has joined #openstack-qa14:04
sdagueyeh, well that's what tends to happen when the approve rush happens, and more gorp slips in14:05
jogosdague: yup, and then we have to 'stabilize'14:05
mtreinishandreaf: it's if you run with the parallel test-accounts provider class and specify a non-existent yaml file14:05
BobBallis gorp a technical term?14:05
mtreinishandreaf: which is probably an unlikely edge condition but definitely a configuration issue14:06
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Backward compatibility to credentials in conf  https://review.openstack.org/11906014:06
andreafmtreinish: ok14:07
andreafmtreinish, can I +2 a change that was initiated by me but not written by me at the end? I mean https://review.openstack.org/11906014:08
mtreinishandreaf: the only downside with this approach is that it delays raising an exception if a yaml file is needed until you try to use the cred provider14:08
mtreinishandreaf: sure I think that's fine14:08
*** rbak has joined #openstack-qa14:08
*** markmcclain has joined #openstack-qa14:10
*** Ch00k_ has joined #openstack-qa14:11
psedlakmtreinish: thx for the link to 'Make unit tests not depend on random hash seed', though i'm now wondering how sorted(dict(...)) will work14:12
psedlakmtreinish: as i'm getting `for k, v in sorted({'a':1}): pass` |==> ValueError: need more than 1 value to unpack14:13
*** mrodden_ has joined #openstack-qa14:14
*** lordd has quit IRC14:14
*** bdperkin_ has joined #openstack-qa14:14
mtreinishpsedlak: oh, I'm just sorting the keys, and pulling the values out inside the loop14:14
*** lordd has joined #openstack-qa14:14
BobBallpsedlak: indeed; sorting just returns the keys (try print sorted({'a':'b'}) and you get ['a'])14:14
mtreinishdict's don't have an order, you can either sort by the keys or the values14:15
psedlakmtreinish: BobBall: ah i see it now, thx you both :)14:15
mtreinishbut then they're treated as lists14:15
psedlakfor some reason i was reading it as loop over key and values14:15
*** yamahata has quit IRC14:17
*** Ch00k has quit IRC14:17
*** bdperkin has quit IRC14:17
*** mkollaro has quit IRC14:17
*** mrodden has quit IRC14:17
*** harlowja_away has quit IRC14:17
*** annegentle has quit IRC14:17
*** bdperkin_ is now known as bdperkin14:17
*** bdperkin has joined #openstack-qa14:17
*** mrodden_ is now known as mrodden14:17
*** lordd has quit IRC14:19
andreafmtreinish: I think the test-accounts patches are looking good now, hopefully we can merge up to https://review.openstack.org/#/c/107685/27 soon, so reviews from the rest of the core team would be welcome14:20
*** yamahata has joined #openstack-qa14:20
*** Ch00k_ has quit IRC14:20
mtreinishandreaf: yeah I'll bring it up during the meeting14:21
mtreinishalthought the factory patch needs a rebase now :)14:21
*** atiwari has joined #openstack-qa14:22
*** jecarey has joined #openstack-qa14:22
*** zz_jgrimm is now known as jgrimm14:25
*** mkollaro has joined #openstack-qa14:25
*** andreaf has quit IRC14:28
*** andreaf has joined #openstack-qa14:28
*** andreaf is now known as andreaf_14:28
*** 17SAA4MFR has joined #openstack-qa14:33
*** 17SAA4J4G has joined #openstack-qa14:33
*** annegentle has joined #openstack-qa14:33
*** annegentle_ has joined #openstack-qa14:33
*** 17SAA4MFR has quit IRC14:33
*** 17SAA4J4G has quit IRC14:33
*** annegentle has quit IRC14:34
*** annegentle_ is now known as Guest1862114:34
*** bkopilov has quit IRC14:34
*** jaypipes has joined #openstack-qa14:34
*** ChanServ sets mode: +o jaypipes14:34
*** lordd has joined #openstack-qa14:41
*** SridharG has left #openstack-qa14:44
jogoI just found a hypothetical bug in tempest14:45
mtreinishjogo: so it's not real? :)14:47
*** sridhar has quit IRC14:47
jogowell it may never happen14:48
jogomtreinish: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/compute/servers/test_list_server_filters.py#n22814:48
*** chandankumar has joined #openstack-qa14:48
jogosubstring matching14:48
jogos2_name and s3_name may have part_name in them14:48
jogopushing a patch now14:48
jogomtreinish: oh wait its the full random section14:50
jogonever mind14:50
mtreinishjogo: yeah that substring includes the complete random part14:51
jogois the random part always just numbers?14:51
jogoI want to add a regex name test14:51
*** ildikov has quit IRC14:51
mtreinishjogo: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/common/utils/data_utils.py#n2914:51
*** bkopilov has joined #openstack-qa14:51
*** Dafna has quit IRC14:52
jogomtreinish: thanks14:52
*** qba73_ has joined #openstack-qa14:53
*** qba73 has quit IRC14:54
*** dustins has joined #openstack-qa14:55
openstackgerritJoe Gordon proposed a change to openstack/tempest: Add test list_servers_filtered_by_name_regex  https://review.openstack.org/11908114:57
jogomriedem: ^14:57
openstackgerritJohn Davidge proposed a change to openstack-dev/devstack: Add IPv6 support for tenant data network  https://review.openstack.org/8798714:59
*** qba73 has joined #openstack-qa15:00
openstackgerritJoe Gordon proposed a change to openstack/tempest: Add test list_servers_filtered_by_name_regex  https://review.openstack.org/11908115:01
*** qba73_ has quit IRC15:01
*** zz_dimtruck is now known as dimtruck15:02
*** tkammer has quit IRC15:03
openstackgerritMartin Pavlásek proposed a change to openstack/tempest: Avoid to iterate over empty list in DictMismatch  https://review.openstack.org/11908715:09
*** dpaterson has quit IRC15:10
*** amotoki has quit IRC15:14
*** amotoki_ is now known as amotoki15:14
*** Longgeek has joined #openstack-qa15:17
*** Longgeek_ has quit IRC15:20
*** mlavalle has joined #openstack-qa15:36
*** evgenyf has quit IRC15:36
*** yjiang5 has joined #openstack-qa15:41
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for oslotest/infra py33 bug 1365512  https://review.openstack.org/11906315:43
*** dkranz has joined #openstack-qa15:49
*** chandankumar has quit IRC15:52
*** markmcclain has quit IRC15:53
*** jlibosva has quit IRC16:01
*** chandankumar has joined #openstack-qa16:02
*** weshay has joined #openstack-qa16:02
*** jlibosva has joined #openstack-qa16:03
*** dkranz has quit IRC16:03
*** gszasz has quit IRC16:03
*** yamahata has quit IRC16:06
*** marun_ is now known as marun_afk16:06
*** sballe has quit IRC16:12
*** afazekas has quit IRC16:13
*** adalbas has quit IRC16:21
openstackgerritMikhail S Medvedev proposed a change to openstack/tempest: Skip orchestration tests if Neutron is not available  https://review.openstack.org/11845416:22
*** kmartin has joined #openstack-qa16:23
*** mlavalle has quit IRC16:27
*** chandankumar has quit IRC16:27
*** tosky has quit IRC16:27
*** qba73 has quit IRC16:29
*** mlavalle has joined #openstack-qa16:29
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Update query for pip timeout bug 1270710  https://review.openstack.org/11912116:30
mriedemmrodden: have you seen these? http://logs.openstack.org/75/117475/3/gate/gate-devstack-bashate/5d1ef19/console.html16:31
mriedembug categorization rate for bashate in the gate is 10%16:31
mlavallemtreinish: you around?16:31
*** jlibosva has quit IRC16:31
mtreinishmlavalle: yeah16:32
*** jpich has quit IRC16:33
mlavallemtreinish: technical question: I have a patchset that depends on https://review.openstack.org/#/c/111719/13, that merged yesterday. how do I reflect that now with my patchset in my local git? First time I do this :-)16:33
mtreinishmlavalle: normally I just take the sha1 for the commit I need to rebase16:35
mtreinishthen on a branch with the commit it needs to be based on top of I cherry-pick that sha116:35
*** adalbas has joined #openstack-qa16:35
*** lordd has quit IRC16:35
mtreinishyou can also do it with the git rebase command, but I often make more mistakes with that method then cherry-picking16:35
mlavallemtreinish: thanks :-)16:36
mlavallemtreinish: so really no magic, just old git commands :-)16:37
mtreinishmlavalle: so something like: commit=`git rev-parse HEAD` && git fetch && git checkout origin/master && git cherry-pick $commit16:37
mtreinishof course assuming that the top commit is the one you need to rebase16:37
mlavallemtreinish: yeah, I need to rebase the top commit16:38
mtreinishoh, although thinking about it you could just do git rebase origin/master if it's commit that was already merged16:38
mlavallemtreinish: my commit is not already merger. it is the commit that it depends on in gerrit that merged16:39
mtreinishI normally only need to cherry pick if it's for rebasing ontop of an unmerged branch16:39
mtreinishmlavalle: yeah, so if you're working branch has a commit and you need to rebase it on a newly merged change do: git fetch && git rebase origin/master16:40
*** dpaterson has joined #openstack-qa16:42
mtreinishmlavalle: which patch do you need to rebase?16:42
mlavallemtreinish: yeah, that is what I was thinking last night. I just wanted an expert opinion :-)16:43
*** yfried_ has joined #openstack-qa16:43
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for gate-devstack-bashate xen functions fails  https://review.openstack.org/11912616:44
mtreinishheh, I'd hardly call myself a git expert16:44
mlavallemtreinish: so i need to rebase https://review.openstack.org/#/c/118232/, which is wip. Right now, in gerrit, it depends on https://review.openstack.org/#/c/111719/13, which merged yesterday16:45
*** sballe has joined #openstack-qa16:45
*** jlanoux has quit IRC16:47
*** aimon has quit IRC16:51
*** aimon has joined #openstack-qa16:53
mtreinishmlavalle: oh, you can also try the rebase button in gerrit, if it's a trivial rebase it'll take care of it16:53
*** suneelb has joined #openstack-qa16:53
mlavallemtreinish: thanks! :-).16:55
*** jlibosva has joined #openstack-qa17:08
*** johnthetubaguy is now known as zz_johnthetubagu17:09
*** tkammer has joined #openstack-qa17:10
*** melwitt has joined #openstack-qa17:12
mroddenmriedem: got a link to one of those bashate failures handy?17:12
mriedemmrodden: http://logs.openstack.org/75/117475/3/gate/gate-devstack-bashate/5d1ef19/console.html17:14
mroddensweet thanks17:14
*** chandankumar has joined #openstack-qa17:15
*** Longgeek_ has joined #openstack-qa17:16
*** e0ne has quit IRC17:17
*** jlibosva has quit IRC17:18
*** luqas has quit IRC17:18
*** Longgeek has quit IRC17:19
*** aix has quit IRC17:19
*** mpavlase has quit IRC17:19
*** harlowja has joined #openstack-qa17:20
*** luqas has joined #openstack-qa17:20
*** luqas has quit IRC17:21
mroddenhaven't changed that code ever in bashate so not sure why that would show up17:21
mroddencould be we never checked it before17:21
*** yjiang5 has quit IRC17:23
*** amotoki has quit IRC17:29
*** lordd has joined #openstack-qa17:30
*** mpaolino has quit IRC17:30
*** chandankumar has quit IRC17:30
*** yjiang5 has joined #openstack-qa17:31
*** ajo has joined #openstack-qa17:34
*** marun_afk has quit IRC17:36
*** lordd has quit IRC17:39
*** lordd has joined #openstack-qa17:42
*** mpaolino has joined #openstack-qa17:42
*** melwitt has quit IRC17:43
*** melwitt has joined #openstack-qa17:43
*** markmcclain has joined #openstack-qa17:48
mriedemmrodden: https://review.openstack.org/#/c/119037/17:48
*** e0ne has joined #openstack-qa17:50
*** lordd has quit IRC17:51
*** yjiang5 has quit IRC17:52
*** lordd has joined #openstack-qa17:52
*** lordd has quit IRC17:53
*** tkammer has quit IRC17:53
openstackgerritOpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements  https://review.openstack.org/11011017:55
*** tosky has joined #openstack-qa17:58
*** e0ne has quit IRC17:58
*** mpaolino has quit IRC17:59
openstackgerritAdam Gandelman proposed a change to openstack-dev/grenade: Set required DSG variables for Ironic DO NOT MERGE  https://review.openstack.org/11851518:05
openstackgerritAdam Gandelman proposed a change to openstack-dev/grenade: Add Ironic sideways migration testing  https://review.openstack.org/11185918:05
openstackgerritAdam Gandelman proposed a change to openstack-infra/devstack-gate: Set required devstack variables for Ironic+grenade  https://review.openstack.org/11676118:06
*** e0ne has joined #openstack-qa18:06
*** Ch00k has joined #openstack-qa18:10
*** dkranz has joined #openstack-qa18:10
*** ajo has quit IRC18:21
openstackgerritmelanie witt proposed a change to openstack/tempest: send paramiko logs to a file "paramiko.log"  https://review.openstack.org/11894618:29
*** e0ne has quit IRC18:29
openstackgerritBenny Kopilov proposed a change to openstack/tempest: glance add image with duplicate id  https://review.openstack.org/11916218:30
*** e0ne has joined #openstack-qa18:32
*** suneelb has quit IRC18:33
*** e0ne has quit IRC18:35
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for glanceclient/requests bug 1364893  https://review.openstack.org/11916418:35
*** mkollaro has quit IRC18:36
*** nikhil__1 is now known as nikhil18:38
*** nikhil is now known as nikhil_k18:38
*** e0ne has joined #openstack-qa18:39
*** nikhil_k is now known as nikhil__18:40
*** markmcclain has quit IRC18:41
*** yjiang5 has joined #openstack-qa18:41
*** markmcclain has joined #openstack-qa18:41
*** nikhil__ is now known as nikhilkomawar18:42
*** ianw has quit IRC18:42
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for test_volume_boot_pattern TypeError  https://review.openstack.org/11916818:42
*** nikhilkomawar is now known as nikhil_k18:48
*** nikhil_k is now known as nikhilk18:49
*** nikhilk is now known as nikhilk_18:50
*** nikhilk_ is now known as nikhil__18:50
*** nikhil__ is now known as nikhil___18:50
*** suneelb has joined #openstack-qa18:51
*** pcrews has quit IRC18:51
*** nikhil___ is now known as nikhil_k18:52
mtreinishmriedem: do you want to push a fix to ^^^? it should be a simple one liner...18:55
mtreinishI don't think I've seen one proposed yet18:56
*** dkranz has quit IRC18:56
mriedemmtreinish: running pep8 now18:57
openstackgerritMatt Riedemann proposed a change to openstack/tempest: Fix TypeError in TestVolumeBootPattern  https://review.openstack.org/11917318:58
mriedemmtreinish: there you go18:58
mtreinishmriedem: cool thanks19:00
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for gate-devstack-bashate xen functions fails  https://review.openstack.org/11912619:01
*** markmcclain has quit IRC19:04
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Only log console log if nova supports consoles  https://review.openstack.org/11917619:09
*** aysyd has quit IRC19:11
mtreinishmriedem: ^^^ it's just a baremetal thing I think19:14
openstackgerritDolph Mathews proposed a change to openstack-infra/elastic-recheck: add query for mysql server has gone away  https://review.openstack.org/11917719:17
*** nmagnezi has quit IRC19:18
*** Sukhdev has joined #openstack-qa19:22
*** ajo has joined #openstack-qa19:26
*** aysyd has joined #openstack-qa19:27
*** Longgeek_ has quit IRC19:29
*** markmcclain has joined #openstack-qa19:32
*** mpaolino has joined #openstack-qa19:32
*** mwagner_lap has quit IRC19:33
*** mpaolino has quit IRC19:34
*** StevenK has quit IRC19:38
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add some notes to the readme about queries that don't hit  https://review.openstack.org/11918419:39
mriedemmtreinish: jogo: sdague: clarkb: ^19:39
openstackgerritA change was merged to openstack-infra/elastic-recheck: Remove resolved fingerprints  https://review.openstack.org/11791619:40
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add some notes to the readme about queries that don't hit  https://review.openstack.org/11918419:41
*** StevenK has joined #openstack-qa19:43
*** pcrews has joined #openstack-qa19:46
mtreinishmriedem: +219:48
*** weshay has quit IRC19:50
*** suneelb1 has joined #openstack-qa19:52
*** suneelb has quit IRC19:52
*** arnaud has joined #openstack-qa19:52
*** suneelb1 has quit IRC19:52
*** suneelb has joined #openstack-qa19:52
*** ajo has quit IRC19:53
openstackgerritA change was merged to openstack/qa-specs: Add the qa spec for bp:add-icehouse-swift-tests  https://review.openstack.org/10512019:56
*** mriedem has quit IRC19:57
openstackgerritAdam Gandelman proposed a change to openstack-dev/devstack: Split keystone mod wsgi config from start/stop  https://review.openstack.org/11874019:57
openstackgerritAdam Gandelman proposed a change to openstack-dev/devstack: Split keystone mod wsgi config from start/stop  https://review.openstack.org/11874019:59
*** liusheng has quit IRC20:01
*** liusheng has joined #openstack-qa20:01
*** mpaolino has joined #openstack-qa20:01
*** liusheng has quit IRC20:02
*** aysyd has quit IRC20:03
*** suneelb has quit IRC20:12
*** suneelb has joined #openstack-qa20:12
*** suneelb has quit IRC20:12
*** Ch00k has quit IRC20:12
*** aysyd has joined #openstack-qa20:16
*** flaper87 is now known as flaper87|afk20:24
*** mpavlase has joined #openstack-qa20:26
*** mpaolino has quit IRC20:32
*** pcrews has quit IRC20:36
*** toure has quit IRC20:38
openstackgerritChris Dent proposed a change to openstack-dev/devstack: WIP explore issues with keystone discovery  https://review.openstack.org/11919920:40
*** yjiang5 is now known as yjiang5_away20:46
*** yjiang5_away has quit IRC20:47
sdaguelifeless: is this on your list for testtools hacking this week? https://bugs.launchpad.net/nova/+bug/119953620:50
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for glanceclient/requests bug 1364893  https://review.openstack.org/11916420:53
*** mlavalle has quit IRC20:54
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add query for test_volume_boot_pattern TypeError  https://review.openstack.org/11916820:54
*** mkollaro has joined #openstack-qa20:54
openstackgerritA change was merged to openstack-infra/elastic-recheck: Update query for pip timeout bug 1270710  https://review.openstack.org/11912120:54
*** mlavalle has joined #openstack-qa20:58
jogomtreinish: so http://logs.openstack.org/98/103798/9/check/check-tempest-dsvm-neutron-full/b059275/logs/ look at the footer20:58
*** suneelb has joined #openstack-qa20:59
*** suneelb has quit IRC21:00
*** arnaud has quit IRC21:00
*** mkollaro has quit IRC21:00
*** suneelb has joined #openstack-qa21:00
*** suneelb has quit IRC21:00
*** suneelb has joined #openstack-qa21:01
*** catohornet has quit IRC21:02
mtreinishjogo: cool21:04
jogoright!21:05
*** pcrews has joined #openstack-qa21:09
jogolol http://jogo.github.io/gate/21:11
*** aysyd has quit IRC21:13
mtreinishmelwitt: I was looking at: https://review.openstack.org/118946 and I'm wondering why we need a separate log file? Just changing the log level will set it to be picked up in the tempest log file21:14
*** marun_afk has joined #openstack-qa21:14
mtreinishalthough I think it makes things kinda verbose21:14
mtreinishthe way you have it there it will be logging in the tempest log file and create a new paramiko.log file21:15
jogomtreinish: https://review.openstack.org/#/c/118946 https://review.openstack.org/#/c/11892521:16
jogoboth should help dig into the second biggest bug21:16
jogoor 3rd21:16
*** cdent has quit IRC21:16
mtreinishjogo: heh, I was just asking about 118946 :)21:17
melwittmtreinish: I'm not sure I follow. change log level where? paramiko logging afaict doesn't go anywhere even though its loggers are there and set to debug when I ran locally. and I thought since it's so verbose that maybe it should be in its own file21:17
lifelesssdague: it wasn't, but I've got the tab open21:17
mtreinishmelwitt: it goes in the tempest log file right now21:18
mtreinishmelwitt: http://logs.openstack.org/46/118946/1/check/check-tempest-dsvm-full/60d2e18/logs/tempest.txt.gz#_2014-09-04_11_41_19_07021:18
jogomtreinish: hehe21:19
melwitthm... :( wondering how I didn't see that.21:21
mtreinishmelwitt: the paramiko.transport default is also explicitly set to info here: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/config.py#n115621:21
mtreinishI believe sdague did that a while back because the debug level wasn't providing any extra value and it just made the log file bigger21:22
*** Sukhdev has quit IRC21:22
mtreinishmelwitt: we can switch that back to debug if you think the extra logging is worth the filesize cost21:23
mtreinishclarkb: ^^^ assuming we can take the hit for short period to try and debug ssh failures21:23
clarkbmtreinish: we won't index debug level logs21:24
clarkbso it shouldn't matter21:24
clarkbunless the logs come out without being properly annotated21:24
sdagueclarkb: how's overall log size?21:24
sdaguelike our volume space21:24
mtreinishclarkb: oh good point, I forgot about that21:24
clarkbsdague: I dunno, found a bug yesterday that was bitbucketting a bunch of stuff so numbers are screwy21:24
sdagueheh21:24
sdagueyeh, realistically the logs are painfully short at this point21:25
sdagueI went looking for logs from a run in june, and they weren't there21:25
*** e0ne has quit IRC21:25
clarkbya that too21:26
melwittmtreinish: oh wow, okay. I wasn't sure if it would cost to much to keep them and thought to keep them separate if anything so as not to muddy the normal operation which is success. just when we hit these ssh related failures it would help21:26
mtreinishsdague: I've noticed some weirdness with the file extension on old links. Like a gz missing or something21:26
clarkbya we should still have june. we are 4 months of retention now so may should be falling off21:26
sdaguemtreinish: that's been a thing21:27
*** mmedvede has quit IRC21:27
sdaguewe probably need another redirect rule21:27
sdaguemelwitt: honestly, if you think it will help fix a bug, I'd say turn it back on21:28
mtreinishmelwitt: ok, well we can try to get more clever about it and switch the log level at runtime after an initial ssh failure. But, I think just switching that info in tempest.config to debug is fine21:28
clarkbyup agreed21:28
*** andreaf has joined #openstack-qa21:28
clarkb/dev/mapper/main-logs          12T   11T  1.5T  89% /srv/static/logs21:29
melwittsdague: it's sort of moot now that salv-orlando has reproduced the failure locally and we're chatting in #openstack with smoser about it. public key didn't make it to the VM authorized_keys21:29
sdaguemelwitt: ok, cool21:29
sdaguethat's a win to have figured that out21:29
melwittyeah. before this it was just a black box on where exactly the ssh public key auth process was failing out so I was looking for the paramiko logging21:31
mtreinishclarkb: heh, I have more storage than the logs volume in my closet :)21:33
clarkbmtreinish: we can only do 14TB total per node. 12TB were given to logs21:34
clarkbthis is why the swift logs are a thing but that is slowish and needs more testing21:34
*** suneelb has quit IRC21:36
*** suneelb has joined #openstack-qa21:36
mtreinishclarkb: yeah I remebered that, I just always assumed the upper bound was more than what I had, because I have ~22TB for my file server.21:36
*** suneelb has quit IRC21:37
*** mlavalle has quit IRC21:38
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1343313  https://review.openstack.org/11922221:44
*** markmcclain has quit IRC21:45
*** dustins has quit IRC21:50
*** mlavalle has joined #openstack-qa21:51
*** dims_ has joined #openstack-qa21:51
clarkbmtreinish: we could get a bunch of nodes together with 14TB each and create a distributed file system21:52
*** arnaud has joined #openstack-qa21:52
clarkbbut I imagine that won't be much better than swift and with swift someone else is dealing with it :)21:52
mtreinishclarkb: heh, yeah that's a fair point. Although I think mordred and jeblair would actually prefer using afs :)21:53
clarkbindeed21:53
*** dims has quit IRC21:54
*** tosky has quit IRC21:54
smosero/21:55
melwitt\o21:55
salv-orlandosmoser, melwitt: so the console log is saying “cirros-apply-local already run per instance"21:56
jeblairyeah, afs totally solves this and many other problems :)21:56
salv-orlandothis mean once cirros-init marker is calculated the process determines it did already run and quits.21:56
*** ianw has joined #openstack-qa21:57
melwittsalv-orlando: I was thinking about your idea and one of the tests we have seen this failure, it doesn't stop any instances before trying to ssh (https://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L141)21:57
melwittstop/start I mean21:58
smosersalv-orlando, so why is that a problem ?21:58
smoserif it did it what would un-do it ?21:58
*** dkranz has joined #openstack-qa21:59
*** mmedvede has joined #openstack-qa21:59
melwittonly if it somehow got marked as "did it" when it didn't yet reach the code where it copies the keys21:59
salv-orlandosmoser: not a problem.22:00
salv-orlandoper se22:00
*** dims_ has quit IRC22:00
salv-orlandoI think it might be a problem is somehow the instance was abruptly stopped while cirros-init was running and then restarded (like in a stop/start actions or resize)22:01
*** dims has joined #openstack-qa22:01
salv-orlandowhich is exactly what’s happening in the tests that are failing22:01
smoserhow do you stop it ?22:01
salv-orlandonova stop <instance_id>22:01
salv-orlandowell called through the tempest client22:01
melwittsalv-orlando: I don't think test_volume_boot_pattern does any stop/start does it?22:02
smosercirros-per definitely writes that file only after it runs the thing it supposed to run.22:02
salv-orlandosmoser: so that’s good and invalidates the theory.22:02
smoseryou can read that code22:02
salv-orlandosmoser: echo "[${_RET}]" "$@" > "${state_d}/${marker}"  is at the end of cirros-per22:03
openstackgerritJoe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1365738  https://review.openstack.org/11922722:03
smoserright22:03
smoserhow do you shut the instance down ?22:04
*** jgrimm is now known as zz_jgrimm22:04
*** suneelb has joined #openstack-qa22:04
* salv-orlando finding link to test code22:04
salv-orlandohttps://github.com/openstack/tempest/blob/master/tempest/scenario/test_network_advanced_server_ops.py#L9522:05
*** dims has quit IRC22:05
salv-orlandoeventually the call gets here: https://github.com/openstack/tempest/blob/master/tempest/services/compute/json/servers_client.py#L35422:05
salv-orlandoand it finally does this: https://github.com/openstack/tempest/blob/master/tempest/services/compute/json/servers_client.py#L22022:06
smoserso i guess to debug this i'd like to get the instance image on failure22:09
smoserso if it does a hard power off, then thats clearly kind of asking for trouble (althouh i wouldn't understand this specific trouble)22:10
smoserbut cirros should correctly respond to acpi22:10
*** dims has joined #openstack-qa22:14
salv-orlandosmoser: interestingly enough a few hours before the issue exploded a chance for doing soft shutdown merged22:15
smoserhm.22:16
salv-orlandosmoser: this -> http://git.openstack.org/cgit/openstack/nova/commit/?id=c07ed15415c0ec3c5862f437f440632eff1e94df22:16
smoseri absolutely agree with the change.22:17
smoseri'm surprised that we soft rebooted, but not soft poweroff22:17
smoserthat is odd22:17
mordredmtreinish: AFS FTW22:17
salv-orlandosmoser: indeed that change is ok for me as well. It’s just strange that this ssh issue started a few hours after it merged. Probably it’s just a coincidence. Let’s stay focused on facts.22:20
smoserit sounds like a fairly strong coincidence.22:21
smoser a.) change goes in that affects shutdown22:21
smoser b.) failures occur in a test that uses shutdown and expects certain behavior22:22
*** rockyg has joined #openstack-qa22:22
*** jecarey has quit IRC22:23
adam_gsalv-orlando, is there a specific scenario test that fails more than others in your local testing?22:24
*** sweston has quit IRC22:25
*** sweston has joined #openstack-qa22:25
openstackgerritAdam Gandelman proposed a change to openstack/tempest: Migrate baremetal_basic_ops to tempest clients  https://review.openstack.org/11767322:26
smoserhave to run. i'm sorry.22:27
*** andreaf has quit IRC22:29
salv-orlandosmoser: thanks for your help22:30
salv-orlandoadam_g: test_network_advanced_server_ops when neutron is enabled.22:30
*** spongewindbag has joined #openstack-qa22:38
*** dims has quit IRC22:49
*** dims has joined #openstack-qa22:50
adam_gof course when i have some time to help debug this, my local devstacks neutron is borked22:51
*** spongewindbag has quit IRC22:54
*** dims has quit IRC22:54
*** mmedvede has quit IRC22:55
*** amazingdeveloper has joined #openstack-qa22:55
*** mmedvede has joined #openstack-qa22:56
openstackgerritA change was merged to openstack-dev/devstack: Fix tools/xen/functions for bashate errors  https://review.openstack.org/11903722:59
openstackgerritA change was merged to openstack/tempest: Properly detect grenade in check_logs script  https://review.openstack.org/11875322:59
*** rbak has quit IRC23:01
mlavallemasayukig: do you want me to update https://etherpad.openstack.org/p/tempest-client-scenarios as I make progress with the load balancer scenario?23:01
*** rockyg has quit IRC23:01
*** amazingdeveloper has quit IRC23:01
stevebakermtreinish: once you've read my "Packaging of functional tests" reply let me know if you want to chat23:02
masayukigmlavalle: yes, of course :) if you have the time.23:03
mlavallemasayukig: will do.... :-)23:04
masayukigmlavalle: But I'll update it if I'll find your progress.23:05
mtreinishstevebaker: I'll probably look at it tomorrow morning it's getting late for me23:06
stevebakermtreinish: ok, no problem23:06
mlavallemasayukig: I'll also update the blueprint23:06
masayukigmlavalle: ok, thanks23:07
masayukigDoes anyone know about the conclusion of 'Lack of consistency in returning response from tempest clients'?23:08
masayukighttp://lists.openstack.org/pipermail/openstack-dev/2014-August/044489.html23:08
*** atiwari has quit IRC23:08
masayukigI'm interested in this topic. And I added this topic to the etherpad: https://etherpad.openstack.org/p/kilo-qa-summit-topics23:09
jrollianw: mind taking a look at https://review.openstack.org/#/c/114408/ ? :)23:11
jrollianw: scratch that, found a bug :P23:13
jrollsorry23:13
openstackgerritBrant Knudson proposed a change to openstack-dev/devstack: Keystone access log in key-access screen  https://review.openstack.org/11613423:14
ianwjroll: ok, let me know :)23:17
jrollheh, will do, thanks :)23:17
openstackgerritBrant Knudson proposed a change to openstack-infra/devstack-gate: Stash Apache httpd logs  https://review.openstack.org/11640423:19
*** mmedvede has quit IRC23:20
*** mpavlase has quit IRC23:22
*** mlavalle_ has joined #openstack-qa23:24
*** mlavalle has quit IRC23:25
openstackgerritJim Rollenhagen proposed a change to openstack-dev/devstack: Allow devstack to build ironic agent ramdisk  https://review.openstack.org/11440823:31
jrollianw: ^ there we go23:31
jrolldtroyer_zz: ^^23:31
openstackgerritA change was merged to openstack-dev/devstack: Use correct class path for token persistence backend  https://review.openstack.org/11787823:40
*** oomichi has joined #openstack-qa23:41
*** melwitt has quit IRC23:42
*** dimtruck is now known as zz_dimtruck23:58
openstackgerritJim Rollenhagen proposed a change to openstack-dev/devstack: Simplify ironic-python-agent download  https://review.openstack.org/11738423:58
jrollianw: there's also this one, if you don't mind: https://review.openstack.org/#/c/117384/23:59
* jroll can't wait to have this CI stuff running23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!