Thursday, 2013-09-19

*** hootjr has joined #openstack-qa00:01
*** SergeyLukjanov has quit IRC00:07
*** zzs has joined #openstack-qa00:20
*** zzs has left #openstack-qa00:24
*** jhenner has joined #openstack-qa00:31
*** matsuhashi has joined #openstack-qa00:32
*** nosnos has joined #openstack-qa00:40
openstackgerritminsel proposed a change to openstack/tempest: Handling network resources in tenant isolation  https://review.openstack.org/4557800:47
*** jhenner has quit IRC01:16
*** HenryG has quit IRC01:32
openstackgerritYuuichi Fujioka proposed a change to openstack-dev/devstack: Adds setting for allocation rates  https://review.openstack.org/4730502:11
*** dkranz has quit IRC02:17
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Do not merge test of neutron tenant isolation.  https://review.openstack.org/4633002:48
*** jamielennox is now known as jamielennox|away02:56
*** jamielennox|away is now known as jamielennox03:02
openstackgerritYuuichi Fujioka proposed a change to openstack-dev/devstack: Adds setting for allocation rates  https://review.openstack.org/4730503:14
*** jasondotstar has quit IRC03:43
*** mriedem has quit IRC03:55
*** thomasbiege1 has joined #openstack-qa04:04
*** thomasbiege1 has quit IRC04:06
*** sthaha has joined #openstack-qa04:07
*** sthaha has joined #openstack-qa04:07
*** SergeyLukjanov has joined #openstack-qa04:12
*** prekarat has joined #openstack-qa04:25
*** afazekas has joined #openstack-qa04:28
*** prekarat has quit IRC04:29
*** raies has joined #openstack-qa04:44
*** prekarat has joined #openstack-qa04:45
*** madan has joined #openstack-qa04:47
*** madan_ has joined #openstack-qa04:48
*** madan_ has quit IRC04:50
Anjuafazekas:  PING05:14
*** SergeyLukjanov has quit IRC05:22
*** ken1ohmichi has quit IRC06:14
afazekasAnju: pong06:34
Anjuhii afazekas : https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_rescue.py#L5606:34
AnjuL48 and L56 are same06:35
*** madan has quit IRC06:39
afazekas2. th probably  should be detach06:40
afazekasI am not sure the test class really needs 2 volume06:40
*** madan has joined #openstack-qa06:42
Anjuafazekas:  one is for attaching06:43
Anjuand the another is for detaching06:43
openstackgerritnayna-patel proposed a change to openstack/tempest: Test cases for V3 Project Actions  https://review.openstack.org/4134506:44
afazekasbut those volumes are not shared, they could be created in the test_ method , or if they are really shared the two test method could use the same one06:45
Anjuok afazekas  i am trying06:47
mkodererafazekas: regarding https://review.openstack.org/#/c/46087/06:49
mkodererafazekas: everything what I do is tested :P06:49
mkodererI wanted to discuss with sdauge since a wait doesn't solve the problem06:50
*** SergeyLukjanov has joined #openstack-qa06:52
afazekasmkoderer: I think he recommended to wait after kill -906:54
mkodererafazekas: yes that right otherwise we get zobmies06:54
mkodererI will call join after the kill06:55
mkodererafazekas: but a simple "wait" won't work06:55
mkodererafazekas: ok so it was a misunderstanding06:57
afazekashttp://docs.python.org/2/library/subprocess.html is the terminate() + wait () able to work  ?07:00
mkodererI think Popen.kill() sounds better07:03
mkoderersince process['process'].terminate() does the same07:03
mkodererafazekas: ok I need some time to test it.. not sure if I have this today07:04
mkodererafazekas: since we are going live with our environment...07:04
afazekasbest wishes07:06
raiesafazekas: when I tried "neutron vpn-service-update 0f62712d-d0af-4f51-ac6a-9c823c8f7ef0 --description "Updated " "  then following error07:08
raies"Invalid state PENDING_CREATE of vpnaas resource <built-in function id>"07:08
raiesplease help07:08
raiesme on this07:08
afazekasis the q-vpn service eneabled ?07:10
raiesafazekas: are you there ?07:10
raiesyes it is enabled07:10
raiesand I can create vpn service using neutron client but the state is showing "PENDING_CREATE"07:11
raieswhen status will change from "PENDING_CREATE" to "CREATED" or something like that07:15
raiesafazekas: ^^07:15
afazekasI guess when it is created07:18
afazekashttps://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L7107:18
afazekasI think q-vpn should be added there07:18
raiesI have added q-vpn in localrc07:20
afazekasYou can try to add to the devstack-gate repo before submitting test cases07:21
raieswhere can I find this repo ?07:22
afazekason the above url07:23
raiesbut when I will be creating vpn service then how will it affect ? And should I install above .sh file ?07:25
raiesif yes then how07:25
raiesI mean if I will checkout this .sh file then what I will do after making changes ?07:25
raiesafazekas: ^^07:27
afazekasyou do not need the above sh file, but the service should be added to that line in-order to be able to create gating tests07:27
afazekasvpn should use openswan/libreswan by default07:28
raiesthen how can I enable vpn there ?07:29
afazekasprobably you just need to add the q-vpn to that line07:29
raiesthat is fine what you say, but where on my machine I can get that file ?07:31
afazekasgit clone https://github.com/openstack-infra/devstack-gate.git  ; git review -s ..07:32
raiesI should clone it in devstack directory where localrc is ?07:33
raiesok I understood07:33
raiesI have to checkout and then , make changes and then commit ? right07:34
raies?07:34
afazekasyou do not need that script for local usage, but it is required state to have the vpn tests gating07:34
raiesI followed you steps07:37
raiesall are successful07:37
raiesbut still neutron vpn-service-list is showing "PENDING_CREATE"07:37
Anjuafazekas:   can we create volume once here07:38
Anjuand can use that voluem for attach abd detach07:38
Anju?07:38
afazekasprobably yes07:40
Anjuhmm07:42
*** fbo_away is now known as fbo07:46
Anjuafazekas:  https://bugs.launchpad.net/tempest/+bug/118736807:49
Anjui am facing the same problem07:50
afazekasyou can delete your instances07:50
*** jhenner has joined #openstack-qa07:57
*** jhenner has quit IRC07:58
*** giulivo has joined #openstack-qa08:01
*** yassine has joined #openstack-qa08:07
*** jhenner has joined #openstack-qa08:08
*** matsuhashi has quit IRC08:27
*** Madan1 has joined #openstack-qa08:27
*** matsuhashi has joined #openstack-qa08:27
openstackgerritzhu zhu  proposed a change to openstack/tempest: Add tempest tests for os-host/{host-name} api  https://review.openstack.org/4691408:29
openstackgerritCyril Roelandt proposed a change to openstack/tempest: Do not review this  https://review.openstack.org/4719408:29
openstackgerritMh Raies proposed a change to openstack-infra/devstack-gate: Enable q-vpn service  https://review.openstack.org/4733508:29
raiesafazekas ?08:33
raiesu there ?08:34
openstackgerritnayna-patel proposed a change to openstack/tempest: Test cases for V3 Project Actions  https://review.openstack.org/4134508:37
*** prekarat has quit IRC08:57
*** thomasbiege1 has joined #openstack-qa09:00
*** jasondotstar has joined #openstack-qa09:00
*** SergeyLukjanov has quit IRC09:09
openstackgerritMate Lakat proposed a change to openstack-dev/devstack: xenapi: Use C locale  https://review.openstack.org/4733809:10
*** bashok has joined #openstack-qa09:17
*** thomasbiege1 has quit IRC09:18
*** thomasbiege1 has joined #openstack-qa09:20
*** thomasbiege2 has joined #openstack-qa09:33
*** thomasbiege1 has quit IRC09:35
*** thomasbiege2 has quit IRC09:44
*** jhenner has quit IRC09:48
openstackgerritzhu zhu  proposed a change to openstack/tempest: Add tempest tests for os-host/{host-name} api  https://review.openstack.org/4691409:52
*** mkollaro has joined #openstack-qa09:55
Anjuafazekas : ping09:58
*** sungju has joined #openstack-qa10:03
openstackgerritHoisaleshwara Madan V S proposed a change to openstack/tempest: port test_flavors into v3 part1  https://review.openstack.org/4734310:06
*** annegentle has quit IRC10:15
*** annegentle has joined #openstack-qa10:17
*** fbo is now known as fbo_away10:24
*** giulivo has quit IRC10:27
*** giulivo has joined #openstack-qa10:38
*** bashok has quit IRC10:49
*** sungju has quit IRC10:51
*** jasondotstar has quit IRC10:52
*** hootjr has quit IRC11:14
*** matsuhashi has quit IRC11:25
*** nosnos has quit IRC11:38
*** nosnos has joined #openstack-qa11:39
*** nosnos has quit IRC11:43
*** HenryG has joined #openstack-qa11:44
openstackgerritMh Raies proposed a change to openstack/tempest: VPNaas IKE policies tests  https://review.openstack.org/4734811:46
*** fbo_away is now known as fbo11:48
*** mriedem has joined #openstack-qa11:51
openstackgerritMh Raies proposed a change to openstack/tempest: VPNaas vpnservice test cases  https://review.openstack.org/4733911:54
openstackgerritMh Raies proposed a change to openstack/tempest: VPNaas IKE policies tests  https://review.openstack.org/4734811:54
openstackgerritDennyZhang proposed a change to openstack/tempest: add __str__ function to RestClient class  https://review.openstack.org/4735011:55
*** raies has quit IRC11:56
*** hootjr has joined #openstack-qa12:00
*** adalbas has joined #openstack-qa12:03
openstackgerritDennyZhang proposed a change to openstack/tempest: add __str__ function to RestClient class  https://review.openstack.org/4735012:04
*** bashok has joined #openstack-qa12:08
*** jasondotstar has joined #openstack-qa12:11
*** adalbas has quit IRC12:11
*** Madan1 has quit IRC12:12
*** madan has quit IRC12:14
*** jhenner has joined #openstack-qa12:22
hootjrif anyone is a network guru (even basic understand of openstack networking), please let me know.  I'm basically just trying to understand if what I believe I've setup is indeed something that _should_ be working12:24
afazekasAnju: pong12:24
hootjrHere's my current setup: https://docs.google.com/file/d/0B4BfKJjYWlMKMTVtRThmZ0QwUk0/edit?usp=sharing   my vm cannot egress out.  And if I add a floating-ip, I can't ping or ssh into that vm either.12:25
afazekasraies: what is the question ?12:25
*** thomasbiege has joined #openstack-qa12:25
afazekashootjr: did you added a security group ?12:26
*** mkollaro has quit IRC12:26
hootjrI'm using RDO's havana release, which appears to have everything opened by a default packstack install12:26
afazekashootjr: did you configured your external interface ?12:26
hootjrmy phys eth0 is working properly.  I can easily ssh into that system from my workstation that is on the same network12:27
hootjrHere's the command that I used to create the subnet: neutron subnet-create --name public-subnet1 --allocation-pool start=192.168.100.129,end=192.168.100.190 --gateway 192.168.100.254 --enable_dhcp=False public 192.168.100.0/2412:27
afazekashttps://github.com/openstack-dev/devstack/blob/master/lib/neutron#L36612:32
afazekasYou can try to add the floating ip range to the br-ex interface12:35
hootjragh… looks like that may be the issue12:35
afazekassudo ip addr add 172.16.1.1/24 dev br-ex12:35
afazekassudo ip link set br-ex up12:35
hootjrcurrently (probably from the packstack defaults) br-ex is set to 172.24.4.255.  That IP does not fit with my current network that eth0 is attached to12:38
hootjrsorry.. 225, not 255.12:39
hootjrso I think I need to remove that and put on an IP at that correlates to my eth0 network, correct12:39
hootjr?12:39
hootjrthis is where I think I"m confusing things12:39
hootjrshould the "public" net actually be a different network than my net that is physically connected to the public --namely, eth0 (192.168.100.0/24)?12:40
afazekasif you just using one machine for trying it does not really matters12:44
hootjrok… so there is some network magic going on to route that "public" network over to my physical network then12:45
hootjryeah.. this is just a proof of concept in a little VMWare fusion environment12:45
hootjrWe're currently running a lot of Xen-based clusters12:45
hootjrwith trunks setup to them and probably about 70 VLAN's going across those trunks12:45
hootjrso it'll get more complex when I start testing it on real hardware in our sandbox environment12:46
*** dkranz has joined #openstack-qa12:46
hootjrbut for now I just want to play around12:46
afazekasIf you want to reach it externally it matters :)12:47
hootjrI do12:47
hootjrso if I'm on 192.168.100.0/24 network physically, which a 192.168.100.254 gateway12:48
afazekas#openstack-neutron  channel is more neutron specific :)12:48
hootjragh great.. I'll try that12:48
*** mkollaro has joined #openstack-qa12:50
*** rfolco has joined #openstack-qa12:53
*** thomasbiege has quit IRC12:59
*** weshay has joined #openstack-qa13:01
*** ekarlso has quit IRC13:06
*** prekarat has joined #openstack-qa13:17
*** prekarat has quit IRC13:19
*** prekarat has joined #openstack-qa13:19
openstackgerritRalf Haferkamp proposed a change to openstack-dev/devstack: Fix LDAP support for openSUSE  https://review.openstack.org/4626513:27
*** malini_afk is now known as malini13:39
*** dkranz has quit IRC13:52
*** dkranz has joined #openstack-qa14:06
*** klrmn has quit IRC14:17
*** malini has quit IRC14:17
*** klrmn has joined #openstack-qa14:18
*** yassine has quit IRC14:18
*** adalbas has joined #openstack-qa14:20
*** yassine has joined #openstack-qa14:20
*** ekarlso has joined #openstack-qa14:20
*** yassine_ has joined #openstack-qa14:22
*** malini has joined #openstack-qa14:23
*** yassine has quit IRC14:24
*** adalbas has quit IRC14:26
*** bashok has quit IRC14:27
*** adalbas has joined #openstack-qa14:41
*** mriedem has quit IRC14:59
*** mriedem1 has joined #openstack-qa15:00
*** RecheckWatchBot_ has joined #openstack-qa15:14
*** RecheckWatchBot has quit IRC15:15
*** RecheckWatchBot_ is now known as RecheckWatchBot15:15
*** dkranz has quit IRC15:19
*** HenryG has quit IRC15:22
*** HenryG has joined #openstack-qa15:22
openstackgerritCyril Roelandt proposed a change to openstack/tempest: Do not review this  https://review.openstack.org/4719415:31
*** dkranz has joined #openstack-qa15:32
*** malini1 has joined #openstack-qa15:32
*** malini1 has quit IRC15:33
*** mlavalle has joined #openstack-qa15:33
*** malini1 has joined #openstack-qa15:33
*** mlavalle has left #openstack-qa15:34
*** RecheckWatchBot has quit IRC15:34
*** malini has quit IRC15:35
*** RecheckWatchBot has joined #openstack-qa15:35
*** malini1 has left #openstack-qa15:36
*** malini1 has quit IRC15:36
*** RecheckWatchBot has quit IRC15:36
*** RecheckWatchBot has joined #openstack-qa15:38
*** HenryG has quit IRC15:39
*** HenryG has joined #openstack-qa15:40
*** RecheckWatchBot has quit IRC15:42
*** krtaylor has quit IRC15:43
*** thomasbiege has joined #openstack-qa15:44
*** RecheckWatchBot has joined #openstack-qa15:46
*** annegentle has quit IRC15:52
*** annegentle has joined #openstack-qa15:54
*** thomasbiege has quit IRC15:58
*** mriedem1 has quit IRC15:59
openstackgerritA change was merged to openstack/tempest: Dump basic network info  in the test_network_basic_ops  https://review.openstack.org/4043516:00
openstackgerritA change was merged to openstack/tempest: Add update-volume test  https://review.openstack.org/4710016:01
openstackgerritA change was merged to openstack/tempest: Use built-in print() instead of print statement  https://review.openstack.org/4708916:01
*** mriedem has joined #openstack-qa16:02
*** prekarat has quit IRC16:06
*** RecheckWatchBot has quit IRC16:09
*** RecheckWatchBot has joined #openstack-qa16:10
*** prekarat has joined #openstack-qa16:15
*** fbo is now known as fbo_away16:16
openstackgerritSalvatore Orlando proposed a change to openstack-dev/devstack: Nicira plugin: do not die if NVP gateway IP is missing  https://review.openstack.org/4741416:20
openstackgerritSalvatore Orlando proposed a change to openstack-dev/devstack: Neutron nicira plugin: set up connectivity with public network  https://review.openstack.org/4741516:20
*** RecheckWatchBot has quit IRC16:21
*** RecheckWatchBot has joined #openstack-qa16:22
openstackgerritSalvatore Orlando proposed a change to openstack-dev/devstack: Nicira plugin: do not die if NVP gateway IP is missing  https://review.openstack.org/4741416:25
openstackgerritSalvatore Orlando proposed a change to openstack-dev/devstack: Neutron nicira plugin: set up connectivity with public network  https://review.openstack.org/4741516:25
Anjuafazekas  there?16:29
*** mriedem has quit IRC16:29
afazekasAnju: hi16:30
Anjuafazekas : can u please review this16:30
Anjuhttps://review.openstack.org/#/c/47167/16:30
*** mriedem has joined #openstack-qa16:31
*** thomasbiege has joined #openstack-qa16:32
*** thomasbiege has quit IRC16:34
afazekasusually the '-' used in the rand name16:34
*** thomasbiege has joined #openstack-qa16:34
*** thomasbiege has quit IRC16:36
afazekas'invalid' : the names are 'non_existing', but in the 'valid' attribute range based on the used characters and length16:38
Anjuafazekas:  u are sying that i need to change the nam e?16:41
afazekasI would emphasize the not existing  fact, since the expected exception is a NotFound and the other test methods usually named in this way16:43
*** jhenner has quit IRC16:45
Anjuafazekas:  first i did this..but i did not find the way how to name a non existing tenant16:45
Anjuso i did this .afazekas  ..if u want i will change the patch16:46
dkranzsdague: Are we supposed to be approving the nova v3 stuff yet?16:46
Anjuafazekas  : yes. please tell me about the v3 api stuff also16:47
*** afazekas has quit IRC16:48
openstackgerritA change was merged to openstack/tempest: Add keystone user-update test  https://review.openstack.org/4647716:48
Anjudkranz:  v3 api  please tell how are going in this16:49
dkranzAnju: That is what I was asking16:49
Anjudkranz: https://review.openstack.org/#/c/39621/16:50
Anjudkranz:   i  see afazekas comments here16:50
Anjui am not understanding how to proceed in this16:51
jog0mtreinish: we should get a comment from elasticRecheck soon16:53
mtreinishjog0: yeah it already crashed I missed a hardcoded queries.json open16:54
jog0mtreinish: an unable to classify16:54
jog0:916:54
jog0:( *16:54
Anjudkranz:  ^^^16:56
*** RecheckWatchBot has quit IRC16:56
*** afazekas has joined #openstack-qa16:57
*** RecheckWatchBot has joined #openstack-qa16:57
*** mlavalle has joined #openstack-qa16:58
dkranzAnju: We decided to use the copy/modify approach for nova v316:58
Anjuohkkk16:58
Anjubut afazekas was not happy with this16:59
Anjuas he wrote in the comment16:59
mtreinishmeeting time17:00
openstackgerritSteve Baker proposed a change to openstack/tempest: Raise OS_TEST_TIMEOUT for heat slow test  https://review.openstack.org/4742217:02
RecheckWatchBotopenstack-dev/devstack change: https://review.openstack.org/47415 failed tempest with an unrecognized error17:02
mtreinishjog0: ^^^17:03
RecheckWatchBotopenstack/nova change: https://review.openstack.org/47162 failed tempest with an unrecognized error17:03
jog0woot!17:04
jog0saw the gerritbot too!17:04
jog0it may be worth tweaking that commit message17:04
jog0err comment message17:04
jog0to say we don't know what your error was17:04
jog0mtreinish: nice work!!17:05
mtreinishjog0: well the message is simple enough to change17:06
jog0maybe we should only comment when we know what the failure is and update the wiki that jenkins points to17:07
jog0mtreinish: otherwise its sorta redudant17:07
jog0but awesome17:07
*** yassine_ has quit IRC17:17
openstackgerritA change was merged to openstack/tempest: Add tempest tests for os-host/{host-name} api  https://review.openstack.org/4691417:20
*** jasondotstar has quit IRC17:25
*** thomasbiege has joined #openstack-qa17:29
RecheckWatchBotopenstack/glance change: https://review.openstack.org/47421 failed tempest with an unrecognized error17:29
*** rockyg has joined #openstack-qa17:30
*** yassine has joined #openstack-qa17:30
*** HenryG has quit IRC17:35
RecheckWatchBotopenstack/cinder change: https://review.openstack.org/47365 failed tempest because of: https://bugs.launchpad.net/bugs/121839117:39
*** krtaylor has joined #openstack-qa17:39
mtreinishjog0: ^^^ bug found message message17:40
*** mkollaro has quit IRC17:41
jog0mtreinish: amazing17:41
jog0clarkb: ^17:41
clarkb\o/17:43
*** jaypipes has quit IRC17:44
*** yassine has quit IRC17:49
*** sthaha has quit IRC17:51
*** yassine has joined #openstack-qa17:51
jog0afazekas: my vote for that bug is to turn it back on and force us to deal with the repercussions17:55
*** mriedem has quit IRC17:56
openstackgerritAnju Tiwari proposed a change to openstack/tempest: Added Negative tests for image member  https://review.openstack.org/4716717:56
*** mriedem has joined #openstack-qa17:57
stevebakerafazekas: hey, could you take a quick look at this? https://review.openstack.org/#/c/47422/17:58
*** jaypipes has joined #openstack-qa18:00
*** ChanServ sets mode: +o jaypipes18:00
psedlakmtreinish: yes, testname there is one thing which should be done18:01
mlavallemtreinish: ping18:01
mtreinishclarkb: https://github.com/openstack/tempest/blob/master/tempest/test.py18:01
mtreinishmlavalle: pong18:01
*** prekarat has quit IRC18:01
mtreinishclarkb: we already do some stuff with subunit but it's limited to stuff that is useful for debugging18:01
mlavallemtreinish: you mentioned you had a question18:01
psedlakmlavalle: as for example when cleanup for 'something' happens we were not sure which test was doing it (yes it was fixed-ips-xml using json clients :/)18:01
psedlakmtreinish: ^^ (sorry mlavalle)18:02
clarkbmlavalle: https://github.com/openstack/tempest/blob/master/tempest/test.py#L196 exists, does that not capture everything?18:02
clarkber mtreinish ^18:02
mtreinishmlavalle: yeah I was curious about ports cleanup18:02
mtreinishbecause the issue your hitting now is delete is async18:02
mtreinishso when you list ports things disappear from it because they're already in the process of being deleted18:02
mtreinishso if you delete a server the port get's cleanedup automatically?18:03
afazekasmtreinish: https://review.openstack.org/#/c/47422/18:03
mtreinishclarkb: yeah it get's most things (not from setupclass) but I think the complaint was in the log file (which we still output) there isn't anything to tie logs together18:03
mlavallemtreinish: I reproduced the error with the volume actions test that we hit last friday18:04
clarkbmtreinish: right, but the subunit log file has all of that metadata18:04
*** yfried has joined #openstack-qa18:04
clarkbmtreinish: but the setupclass stuff would be missed18:04
openstackgerritafazekas proposed a change to openstack/tempest: unskipping bug related to test_stamp_pattern.py  https://review.openstack.org/4743418:04
mtreinishclarkb: I'm not exactly sure what the complaint is either18:05
mlavallemtreinish: and it is leaving some ports alive. That is why, when I attempt to delete the subnet, I get an exception. Those ports stay there even when the test is over18:05
afazekasjoq0: https://review.openstack.org/4743418:05
jog0afazekas:  I saw18:05
mtreinishmlavalle: I'm just wondering if it's better to fix the tests so they don't leave ports around instead of trying to clean them all up at the end18:06
mtreinishI'm not sure either way18:06
mlavallemtreinish: my conclusion is that this particular that is doing something with the network and doesn't cleanup the ports it creates18:06
mlavallemtreinish: in principle, I agree with you. But the I thought that we may find the same thing with other tests18:06
mtreinishmlavalle: that's what I figured was going on which I why I thought it would be better to correct that instead18:06
mtreinishmlavalle: that's probably true things haven't really been run much with neutron18:07
mtreinishwe're going to uncover a lot of issues like this probably18:07
mlavallemtreinish: just look how many we have found in a week18:07
psedlakclarkb, mtreinish: yeah, it's that in tempest log there are all requests going out etc logged, but they can be mixed for parallel runs, and there is no info "why" it happened (1. logging of test names, not actually parallel related), but also as it's mixed together there so cleanup/other-tests things are there 'cross-logged' ... which could be solved by for example thread/pid in there18:08
mtreinishmlavalle: I'm fine with cheating and cleaning it up after the fact but I don't want the leaking tests to fall through the cracks either18:09
mtreinishso it never gets fixed18:09
afazekasmlavalle: I guess there is delay in freeing those ports deallocated by nova18:09
mtreinishwhich is what I'm afraid will happen with your patch18:09
mlavallemtreinish: yeah, but it that case, i didn't feel I knew enough cinder to fix the problem18:10
clarkbpsedlak: right. If we can somehow get all of that into the subunit log we will get the logs assocaited with each test separate from logs assocaited with other tests18:12
mtreinishmlavalle: well how about you open a bug about the leaking ports and leave a todo comment with your patch about fixing the port leaks around that cleanup function18:12
clarkbpsedlak: but it sounds like the setupClass stuff might be making that difficult18:12
mlavallemlavalle: mtreinish: great!!! I commit to keep track of that and remove the 'cheat' when the cinder test is fixed18:12
*** rockyg has quit IRC18:12
giulivomlavalle, what is the cinder test failing?18:13
mtreinishclarkb: it's just a matter of log level. The concern is on failure we don't want to pollute stdout with useless log messages18:13
afazekasclarkb: I wonder can we add decorator for log capturing in the setUpClasses18:13
mlavallegiulivo, mtreinish: I have to run to a meting. I'll get back to you later18:13
mtreinishok I need to run and grab something to eat myself18:14
afazekasmtreinish: can you approve this: https://review.openstack.org/#/c/47422/18:16
openstackgerritZhiQiang Fan proposed a change to openstack-dev/devstack: Replace OpenStack LLC with OpenStack Foundation  https://review.openstack.org/4743818:21
afazekasmkoderer, cyeoh, jaypipes: https://review.openstack.org/#/c/47422/18:24
*** afazekas has quit IRC18:25
*** yassine has quit IRC18:28
RecheckWatchBotopenstack/nova change: https://review.openstack.org/34825 failed tempest because of: https://bugs.launchpad.net/bugs/119196018:32
*** mlavalle has quit IRC18:33
*** yassine has joined #openstack-qa18:33
*** yassine has quit IRC18:33
*** thomasbiege has quit IRC18:35
*** anteaya has joined #openstack-qa18:40
*** roaet has left #openstack-qa18:50
*** rockyg has joined #openstack-qa18:51
openstackgerritCyril Roelandt proposed a change to openstack/tempest: Do not review this  https://review.openstack.org/4719418:55
mtreinishjog0: just to confirm I should remove the gerrit messages with an unknown tempest failure?18:58
openstackgerritA change was merged to openstack-dev/devstack: use method pip_install over sudo pip install so proxy settings work  https://review.openstack.org/4681619:02
*** mriedem has quit IRC19:03
rockygQuestion:I have a few test engineers exited to start contributing to Openstack and want them to focus on the api tests that are missing. How do we go about letting folks know we have targeted a text do we don't duplicate others work? File a bug for the test and assign it to ourselves? I'll be coaching them through the first ones.19:05
openstackgerritA change was merged to openstack/tempest: Raise OS_TEST_TIMEOUT for heat slow test  https://review.openstack.org/4742219:05
mtreinishrockyg: for a single test a bug is probably best. If it's a set of tests a blueprint would probably be better.19:06
rockygPardon errors: I'm on am S3 until I get the proxy through the firewall working19:07
mtreinishbut you'll probably hit some duplicated effort at some point it's unavoidable because I don't think many people look at bugs before they start working on adding new tests19:07
*** mriedem has joined #openstack-qa19:08
rockygI think it might be best to stay with singles then when we've got it down, go for the bp. Thanks!19:09
openstackgerritAnju Tiwari proposed a change to openstack/tempest: Added Negative tests for image member  https://review.openstack.org/4716719:09
*** rockyg has quit IRC19:14
*** thomasbiege has joined #openstack-qa19:28
*** woodspa has joined #openstack-qa19:28
mtreinishjog0: also on the readme on the todo items should I pull out: "Pull in list of queries from a more flexible source, so a commit isn't needed to update each time" because it's a file specified by config19:31
mtreinishor did you mean pull it from a remote source?19:31
*** woodspa has quit IRC19:31
dkranzjog0: I'm a little unclear about https://review.openstack.org/#/c/47434/ which you +1'ed. We started skipping at some point and this unskips it. What changed?19:35
mtreinishdkranz: I think it was to figure out how high a priority the bug should be.19:36
mtreinishbecause the bug is only at medium right now19:36
dkranzmtreinish: So if the gate starts failing a lot that would prove it should have higher priority?19:36
mtreinishyeah I think that was the intent19:37
dkranzmtreinish: OK :)19:37
openstackgerritCyril Roelandt proposed a change to openstack/tempest: Do not review this  https://review.openstack.org/4719419:37
*** thomasbiege has quit IRC19:38
*** mriedem1 has joined #openstack-qa19:39
*** adalbas has quit IRC19:40
*** mriedem has quit IRC19:41
mtreinishjog0: I found a bug in the retry until ES populated code you added yesterday19:57
mtreinishhttp://logs.openstack.org/02/44502/7/check/gate-tempest-devstack-vm-neutron/4f386e5/console.html devstack failed during setup19:57
mtreinishso the services logs in required files aren't ever generated19:57
mtreinishso it's stuck in an infinite loop19:57
mtreinishmaybe we should add a check if the files exist on the log server first19:58
jog0mtreinish: yes, remove from unknown failure and we can update the wiki.19:58
jog0mtreinish: and yes to pull in list of queries19:58
jog0we can pull in from a remote source with what we have now, just need to wire it all up19:59
jog0dkranz: nothing really, but ignoreing the test means we (nova) won't fix it20:00
mtreinishjog0: Ok I already pushed the change to remove the unknown failure message, I left the 'pull in list' line in todo20:00
mtreinishwe can just remove it on the next patch20:00
jog0and afazekas thinks its important and this way we can figure out how frequent it is20:00
jog0mtreinish: ohh hmm thats a good idea20:01
*** hootjr has quit IRC20:01
jog0we can hit the log server and check if the files are there20:02
jog0you want to do that or should I?20:02
mtreinishcan you take care of it?20:02
jog0sure20:03
dkranzjog0: So does setting the priority to Medium. That's why I asked for Russell's comment.20:04
jog0dkranz:  I am not saying we should make this high priority per-se but I am interested in seeing how frequent it is20:06
mtreinishjog0: is it on the recheck list?20:06
jog0if its frequent ... also its unclear if this a cinder bug or not20:06
jog0mtreinish: no since its been disabled20:06
openstackgerritCyril Roelandt proposed a change to openstack/tempest: Do not review this  https://review.openstack.org/4719420:10
jog0dkranz: if we want to keep that test off I fully inderstand20:10
dkranzjog0: Personally I would like to turn it on, with my "qa" hat on. But I understand why the nova team might not want to spend time trying to fix this flaky bug right now and having jobs fail just annoys people. There has to be a benefit to do that.20:12
dkranzjog0: Like people are going to use the failure info to actually fix it.20:13
jog0dkranz: well with elasticRecheck its much easier to figure out what went wrong so less burdon and we get failure rate numbers20:14
dkranzIf Russell doesn't heatedly object I'll give it my +220:15
* russellb perks up20:15
russellbneed me to look at something?20:15
*** krtaylor has quit IRC20:16
jog0russellb: the conversation above refers to https://review.openstack.org/#/c/47434/20:16
jog0I +1ed it because I like the idea but I have a hunch you should -1 it20:17
openstackgerritJohn Griffith proposed a change to openstack/tempest: Consider lc string for bootable True/False  https://review.openstack.org/4746020:17
mtreinishjog0: "query": "@message:\"The device apparently does not exist\" AND @fields.20:17
mtreinishfilename:\"console.html\" AND @fields.build_status:\"FAILURE\""20:17
mtreinishdo you think that's good enough for that bug if we turn it back on?20:17
jog0mtreinish: works for me20:18
*** yfried has quit IRC20:18
jog0mtreinish: that query showed nothing in past 7 days so looks specific enough20:19
mtreinishok I still won't push it unless we turn it back on though and once we hit a failure so I can confirm it works20:19
jog0*nod*20:19
russellbdkranz: jog0 i commented ... if the goal for turning it on is gathering more data on how often it happens, that seems fine20:30
dkranzrussellb: OK, great!20:31
russellbif it's turn it on because that might encourage people to fix it, that's probably just going to annoy people :-)20:31
dkranzrussellb: That's why I wanted your input.20:31
russellbok20:31
jog0russellb: thanks20:32
russellbnp20:32
*** anteaya has quit IRC20:44
*** RecheckWatchBot has quit IRC20:52
*** RecheckWatchBot has joined #openstack-qa20:52
openstackgerritZhiQiang Fan proposed a change to openstack/tempest: Replace OpenStack LLC with OpenStack Foundation  https://review.openstack.org/4747420:53
jog0mtreinish: do you have an example of a logs.openstack url that is missing files20:53
mtreinishjog0: http://logs.openstack.org/02/44502/7/check/gate-tempest-devstack-vm-neutron/4f386e5/20:54
mtreinishthat's the one that was stuck20:54
jog0thanks20:54
jog0perfect20:55
RecheckWatchBotopenstack/ceilometer change: https://review.openstack.org/46100 failed tempest with an unrecognized error21:02
jog0mtreinish: new patch coming soon21:15
*** morganfainberg is now known as CaptainMorgan21:20
RecheckWatchBotopenstack/heat change: https://review.openstack.org/44228 failed tempest with an unrecognized error21:21
RecheckWatchBotopenstack/heat change: https://review.openstack.org/45462 failed tempest with an unrecognized error21:21
jog0mtreinish: pushed21:24
*** dkranz has quit IRC21:28
*** mikal has joined #openstack-qa21:28
mikalSo, I cannot for the life of me see how I broke tempest with https://review.openstack.org/#/c/46867/21:29
mikalIs someone free enough to help me figure it out please?21:29
jog0mikal: I may know thisone21:32
mikaljog0: hurraj!21:32
mikalhurrah even21:32
mikalIt seems like one of the test instances isn't starting, but I really can't see why21:33
russellbjog0: is turning into a test failure nonja21:33
mikalIs that a french ninja?21:33
russellbheh, something like that21:33
jog0hehhttp://logs.openstack.org/67/46867/4/check/gate-tempest-devstack-vm-full/0def2e6/logs/screen-n-cpu.txt.gz#_2013-09-19_11_38_52_56021:35
jog0http://logs.openstack.org/67/46867/4/check/gate-tempest-devstack-vm-full/0def2e6/logs/screen-n-cpu.txt.gz#_2013-09-19_11_38_52_56021:35
jog0thats the first bug I think21:35
* mikal looks21:35
jog0the second failure postgres job appears to be different21:39
mikaljog0: so... that's close to the code I changed, but I didn't change that bit21:39
mikalUnless I grossly misunderstand the error21:39
jog0mikal: in this case a recheck no bug is a safe bet21:40
jog0If it is you then it will fail again21:40
mikaljog0: I've done that a few times with no joy21:41
mikalSo yeah, I agree its me21:41
mikalI just don't see _how_ its me21:41
mikalI shall stare some more I guess21:41
jog0mikal: looks like the bug in devstack-full occured elsewhere too21:42
jog0not jsut your job21:42
jog0so I don't think that one is you21:43
jog0http://logstash.openstack.org/#eyJzZWFyY2giOiJcInFlbXUtbmJkOiBGYWlsZWQgdG8gYmRydl9vcGVuXCIgIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiIxNDQwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzNzk2MjcwNDA0MjYsIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0=21:44
mikalHuh21:44
jog0logstash makes ugly URLs21:44
mikalBut that page is awesome21:45
jog0my guess is you are making a once transient bug  repeatable21:45
mikalHmmm, that failure gets hits going back at least a week21:46
jog0mikal: we only store logs going back one week in logstash21:46
jog0or maybe two21:46
jog0I forgot21:46
mikalOh bummer21:46
mikalI shall fedex clarkb a bigger hard disk21:46
jog0sorry if that doesn't help, I think what i am trying to say is you stepped on a landmine21:46
mikalYeah21:47
mikalI wonder how I get off it without it blowing up though...21:47
jog0good luck21:47
*** HenryG has joined #openstack-qa21:50
jog0mtreinish: we are starting the migration to infra21:51
RecheckWatchBotopenstack/horizon change: https://review.openstack.org/47445 failed tempest with an unrecognized error22:00
RecheckWatchBotopenstack/cinder change: https://review.openstack.org/47473 failed tempest with an unrecognized error22:01
RecheckWatchBotopenstack/swift change: https://review.openstack.org/45134 failed tempest with an unrecognized error22:01
RecheckWatchBotopenstack/horizon change: https://review.openstack.org/47445 failed tempest with an unrecognized error22:01
*** sungju has joined #openstack-qa22:02
*** jgriffith has joined #openstack-qa22:11
CaptainMorganmtreinish, is there another person that would be good to bug about the tempest change you looked at for me yesterday?  Just trying to pin things down so we can get the RC1 stuff taken care of for keystone.22:12
RecheckWatchBotopenstack/swift change: https://review.openstack.org/45134 failed tempest because of: https://bugs.launchpad.net/bugs/121191522:16
*** marun has quit IRC22:17
*** jerryz has joined #openstack-qa22:18
RecheckWatchBotopenstack/heat change: https://review.openstack.org/45462 failed tempest with an unrecognized error22:19
*** mriedem1 has quit IRC22:22
*** weshay has quit IRC22:29
openstackgerritDavanum Srinivas (dims) proposed a change to openstack/tempest: Ensure server is in ACTIVE state - possible fix for transient failures  https://review.openstack.org/4749422:32
RecheckWatchBotopenstack/nova change: https://review.openstack.org/47488 failed tempest because of: https://bugs.launchpad.net/bugs/119196022:42
mikaljog0: you still around?23:02
jog0mikal: yup23:02
mikaljog0: so... I just got back to digging into that problem I was having23:03
mikal2013-09-19 11:38:20.631 AUDIT nova.virt.block_device [req-e1f2e1fb-4bc6-4a8d-8f3a-598565886823 tempest.scenario.manager533713463-user tempest.scenario.manager823214836-tenant] [instance: 4cf4f756-6d05-42e7-936a-e0d351d77102] Booting with volume aa318c18-a7a4-434f-9786-309ed4be47ca at vda23:03
mikalSo...23:03
mikalThe disk file is missing because this instance was booted from volume23:03
jog0link to output23:03
mikalPlease hold23:04
mikalhttp://logs.openstack.org/67/46867/4/check/gate-tempest-devstack-vm-full/0def2e6/logs/screen-n-cpu.txt.gz#_2013-09-19_11_38_20_63123:04
*** giulivo has quit IRC23:08
RecheckWatchBotopenstack/keystone change: https://review.openstack.org/47404 failed tempest with an unrecognized error23:08
RecheckWatchBotopenstack/glance change: https://review.openstack.org/43899 failed tempest with an unrecognized error23:16
jog0mikal: so if boot from volume we shouldn't do this disk stuff?23:20
jog0wonder how that was transient before23:20
mikaljog0: the fix I proposed was to accurately detect injection failures for mandatory files. We previously erroneuosly ignored them23:21
mikalSo, I think I just turned an existing missed fail into a real fail23:21
mikalWhich is the correct behaviour here I suspect23:22
mikalI'm not sure how inject should work for volumes23:22
mikalBut its definitely a fail in this method23:22
mikalSo I don't think it was transcient. I think we were just always ignoring it.23:22
jog0hmm thats very possible23:26
jog0sounds like your having fun23:27
*** anteaya has joined #openstack-qa23:29
mikaljog0: is it possible that you've never experienced fun?23:30
mikaljog0: I read a wikipedia page which implied it wasn't much like this...23:30
jog0haha23:30
*** ken1ohmichi has joined #openstack-qa23:33
mtreinishjog0: sorry I had to leave early before. Cool I'll have to kill the bot when I go into the office tomorrow (I left my laptop there)23:34
*** anteaya has quit IRC23:36
mtreinishjog0: https://review.openstack.org/#/c/47491/1/modules/openstack_project/files/gerrit/acls/openstack-dev/elastic-recheck.config23:46
mtreinishyou're going to be a ptl?23:46
*** sungju has quit IRC23:49
jog0we can co-ptl if you wish23:58
jog0ptl is there mainly for pushing tags23:58
mtreinishjog0: I was just giving you a hard time :)23:58
mtreinishit's a funny title for something so small23:58
jog0yeah it is :)23:59
mtreinishjog0: I'm working on a query verification test that we can run when queries get's its own repo23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!