| mtreinish | it'll poll launchpad to make sure it's a valid bug | 00:00 |
|---|---|---|
| mtreinish | and do a sanity check on the logstash query (not sure how I'll do this yet) | 00:00 |
| *** matsuhashi has joined #openstack-qa | 00:00 | |
| jog0 | mtreinish: oh no Iam working on that too | 00:03 |
| jog0 | ohh good | 00:03 |
| jog0 | I am doing the sanity check part | 00:03 |
| mtreinish | jog0: haha ok I'll just handle the lp part (that's really simple) | 00:03 |
| jog0 | perfect | 00:04 |
| jog0 | check for if the bug is open and if its closed we want to leave it around for a week or so to be safe Ithink | 00:04 |
| mtreinish | ok | 00:04 |
| jog0 | actually I think we should just make sure the bug is a valid openstack bug | 00:07 |
| jog0 | and not the state | 00:07 |
| jog0 | because the logstash query part will detect closed bugs (if no hits in logstash for something that worked before, it can be closed | 00:07 |
| mtreinish | ok, is there a list of openstack lp projects somewhere? | 00:08 |
| jog0 | https://launchpad.net/openstack | 00:08 |
| mtreinish | yep that'll do it | 00:08 |
| jog0 | you can look up bugs on that I think | 00:08 |
| mtreinish | yeah there is a bug list there | 00:09 |
| mtreinish | of course that assumes launchpadlib lets you query that | 00:11 |
| jog0 | I played with the launchpadlib as part of recheckwatch | 00:12 |
| jog0 | its pretty nice | 00:12 |
| RecheckWatchBot | openstack/nova change: https://review.openstack.org/47504 failed tempest with an unrecognized error | 00:17 |
| jog0 | mtreinish: pushed a patch | 00:18 |
| mtreinish | jog0: ok I'll pull it | 00:19 |
| mtreinish | we've got similar naming tastes | 00:19 |
| * jog0 is looking forward to using gerrit instead of git | 00:19 | |
| jog0 | mtreinish: I figured | 00:19 |
| jog0 | you can just roll your code into that file | 00:19 |
| mtreinish | yeah that's what I was planning to do | 00:19 |
| mtreinish | it's funny how used to the gerrit workflow you get | 00:19 |
| jog0 | great minds think alike | 00:20 |
| jog0 | yeah | 00:20 |
| jog0 | so the queries test cought some bugs I had to fix | 00:20 |
| jog0 | so it works | 00:20 |
| *** yfried has joined #openstack-qa | 00:22 | |
| mtreinish | heh, nice | 00:22 |
| mtreinish | did you want to leave queries.json hardcoded? I had it read it in from config | 00:23 |
| *** nosnos has joined #openstack-qa | 00:23 | |
| jog0 | I did not | 00:28 |
| jog0 | just lazy | 00:29 |
| jog0 | can you add a TODO fix me for that | 00:29 |
| mtreinish | jog0: it's a 2 line fix I'll just include it in the lp check patch | 00:29 |
| jog0 | anyway heading out thanks again | 00:29 |
| jog0 | mtreinish: works for me | 00:29 |
| jog0 | until we are in gerrit I am playing loose with the patches | 00:30 |
| *** sthaha has joined #openstack-qa | 00:33 | |
| *** sthaha has joined #openstack-qa | 00:33 | |
| openstackgerrit | Zhi Kun Liu proposed a change to openstack/tempest: add volume list tests https://review.openstack.org/47508 | 01:04 |
| *** Anju has quit IRC | 01:09 | |
| RecheckWatchBot | openstack/nova change: https://review.openstack.org/36078 failed tempest with an unrecognized error | 01:18 |
| RecheckWatchBot | openstack/heat change: https://review.openstack.org/45462 failed tempest because of: https://bugs.launchpad.net/bugs/1191960 | 01:22 |
| RecheckWatchBot | openstack/nova change: https://review.openstack.org/36078 failed tempest with an unrecognized error | 01:26 |
| *** dkranz has joined #openstack-qa | 01:30 | |
| *** mriedem has joined #openstack-qa | 01:33 | |
| *** Anju has joined #openstack-qa | 02:58 | |
| *** Anju has quit IRC | 03:29 | |
| RecheckWatchBot | openstack/ceilometer change: https://review.openstack.org/46100 failed tempest because of: https://bugs.launchpad.net/bugs/1223469 | 03:39 |
| *** ken1ohmichi has quit IRC | 03:40 | |
| *** jerryz has quit IRC | 03:43 | |
| *** Anju has joined #openstack-qa | 03:43 | |
| *** prekarat has joined #openstack-qa | 04:14 | |
| *** prekarat has quit IRC | 04:15 | |
| *** prekarat has joined #openstack-qa | 04:15 | |
| *** madan has joined #openstack-qa | 04:21 | |
| *** mriedem has quit IRC | 04:28 | |
| *** raies has joined #openstack-qa | 04:36 | |
| raies | anyone from neutron ? | 04:37 |
| *** madan has quit IRC | 04:44 | |
| *** madan has joined #openstack-qa | 04:46 | |
| raies | afazekas: hi | 04:50 |
| *** krtaylor has joined #openstack-qa | 04:55 | |
| raies | mkoderer: hi | 04:55 |
| *** prekarat has quit IRC | 05:00 | |
| *** prekarat has joined #openstack-qa | 05:02 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: VPNaas vpnservice test cases https://review.openstack.org/47339 | 05:12 |
| *** yfried has quit IRC | 05:22 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: VPNaas IKE policies tests https://review.openstack.org/47348 | 05:24 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: VPNaas IKE policies tests https://review.openstack.org/47348 | 05:30 |
| raies | sdague: hi | 05:31 |
| raies | will yu please check https://review.openstack.org/#/c/47335/ | 05:32 |
| RecheckWatchBot | openstack/tempest change: https://review.openstack.org/47339 failed tempest with an unrecognized error | 05:42 |
| raies | RecheckWatchBot: As for I think it is failing due to not being implemented https://review.openstack.org/#/c/47335/ | 05:46 |
| raies | will someone from tempest, please check and approve https://blueprints.launchpad.net/tempest/+spec/vpnaas-api-test-cases | 05:51 |
| RecheckWatchBot | openstack/tempest change: https://review.openstack.org/47348 failed tempest with an unrecognized error | 05:59 |
| *** prekarat has quit IRC | 06:07 | |
| *** prekarat has joined #openstack-qa | 06:07 | |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: First cert attempt https://review.openstack.org/47076 | 06:20 |
| *** ken1ohmichi has joined #openstack-qa | 06:21 | |
| *** jlibosva has joined #openstack-qa | 06:23 | |
| openstackgerrit | Jamie Lennox proposed a change to openstack-dev/devstack: Allow deploying keystone with SSL certificates https://review.openstack.org/47076 | 06:27 |
| *** matsuhashi has quit IRC | 06:27 | |
| *** matsuhashi has joined #openstack-qa | 06:28 | |
| RecheckWatchBot | openstack/cinder change: https://review.openstack.org/46694 failed tempest with an unrecognized error | 06:42 |
| *** Madan1 has joined #openstack-qa | 06:51 | |
| *** thomasbiege has joined #openstack-qa | 07:08 | |
| *** thomasbiege has quit IRC | 07:08 | |
| *** CaptainMorgan is now known as morganfainberg | 07:19 | |
| *** giulivo has joined #openstack-qa | 07:37 | |
| *** yfried has joined #openstack-qa | 07:44 | |
| *** jamielennox is now known as jamielennox|away | 07:55 | |
| *** yfried has quit IRC | 07:55 | |
| *** yassine has joined #openstack-qa | 07:56 | |
| *** matsuhas_ has joined #openstack-qa | 08:03 | |
| *** matsuhashi has quit IRC | 08:03 | |
| *** arrsim` has quit IRC | 08:15 | |
| *** fbo_away is now known as fbo | 08:18 | |
| *** bashok has joined #openstack-qa | 08:26 | |
| *** matsuhas_ has quit IRC | 08:39 | |
| *** matsuhashi has joined #openstack-qa | 08:47 | |
| *** matsuhashi has quit IRC | 08:49 | |
| *** matsuhashi has joined #openstack-qa | 08:50 | |
| *** matsuhas_ has joined #openstack-qa | 08:51 | |
| *** matsuhashi has quit IRC | 08:55 | |
| *** mkollaro has joined #openstack-qa | 09:07 | |
| *** mkollaro has quit IRC | 09:12 | |
| *** mkollaro has joined #openstack-qa | 09:13 | |
| *** mkollaro has quit IRC | 09:18 | |
| *** mkollaro has joined #openstack-qa | 09:20 | |
| *** mkollaro has quit IRC | 09:28 | |
| *** mkollaro has joined #openstack-qa | 09:31 | |
| *** leizhang has joined #openstack-qa | 09:32 | |
| *** jhenner has joined #openstack-qa | 09:37 | |
| *** mkollaro1 has joined #openstack-qa | 09:37 | |
| *** mkollaro has quit IRC | 09:37 | |
| *** mkollaro1 has quit IRC | 09:42 | |
| *** matsuhas_ has quit IRC | 09:43 | |
| *** mkollaro1 has joined #openstack-qa | 09:43 | |
| *** mkollaro1 has quit IRC | 09:43 | |
| *** mkollaro has joined #openstack-qa | 09:45 | |
| *** mkollaro has quit IRC | 09:45 | |
| *** mkollaro has joined #openstack-qa | 09:47 | |
| *** leizhang has quit IRC | 09:56 | |
| *** leizhang has joined #openstack-qa | 09:56 | |
| *** masayukig has joined #openstack-qa | 10:14 | |
| *** nosnos has quit IRC | 10:17 | |
| *** fbo is now known as fbo_away | 10:24 | |
| *** fbo_away is now known as fbo | 10:24 | |
| openstackgerrit | Cyril Roelandt proposed a change to openstack/tempest: Do not review this https://review.openstack.org/47194 | 10:25 |
| *** ken1ohmichi has quit IRC | 10:32 | |
| openstackgerrit | Amala Basha proposed a change to openstack-dev/devstack: edit-glance-manage-command-for-recreate-db https://review.openstack.org/47552 | 10:59 |
| *** masayukig has left #openstack-qa | 11:15 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/tempest: Add a create_router utility function https://review.openstack.org/47553 | 11:22 |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/tempest: API tests for neutron router gateway https://review.openstack.org/47554 | 11:22 |
| *** Madan1 has quit IRC | 11:25 | |
| *** madan has quit IRC | 11:25 | |
| openstackgerrit | A change was merged to openstack/tempest: Skip test tokens https://review.openstack.org/46341 | 12:14 |
| *** weshay has joined #openstack-qa | 12:27 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/tempest: Enable tenant connectivity check over unreachable nets https://review.openstack.org/46900 | 12:27 |
| openstackgerrit | A change was merged to openstack/tempest: Remove wait_for_image_resp_code https://review.openstack.org/38995 | 12:30 |
| openstackgerrit | A change was merged to openstack/tempest: Enable the quota test cases https://review.openstack.org/33152 | 12:30 |
| openstackgerrit | Anju Tiwari proposed a change to openstack/tempest: Added Negative tests for image member https://review.openstack.org/47167 | 12:33 |
| *** sthaha has quit IRC | 12:48 | |
| *** adalbas has joined #openstack-qa | 12:53 | |
| *** hootjr has joined #openstack-qa | 12:55 | |
| openstackgerrit | Anju Tiwari proposed a change to openstack/tempest: Added Negative tests for image member https://review.openstack.org/47167 | 13:06 |
| *** mriedem has joined #openstack-qa | 13:13 | |
| *** jlibosva has quit IRC | 13:16 | |
| *** fbo has quit IRC | 13:18 | |
| *** fbo has joined #openstack-qa | 13:18 | |
| *** jasondotstar has joined #openstack-qa | 13:28 | |
| openstackgerrit | A change was merged to openstack/tempest: Fix misused assertTrue in unit tests https://review.openstack.org/46853 | 13:41 |
| *** russellb is now known as rustlebee | 13:48 | |
| *** prekarat has quit IRC | 14:11 | |
| openstackgerrit | A change was merged to openstack/tempest: Adds tests covering Swift's container quotas middleware https://review.openstack.org/46232 | 14:12 |
| *** thomasbiege1 has joined #openstack-qa | 14:23 | |
| *** thomasbiege1 has quit IRC | 14:24 | |
| *** malini has joined #openstack-qa | 14:29 | |
| *** giulivo has quit IRC | 14:30 | |
| *** giulivo has joined #openstack-qa | 14:31 | |
| *** gfidente has joined #openstack-qa | 14:36 | |
| *** giulivo has quit IRC | 14:38 | |
| openstackgerrit | A change was merged to openstack/tempest: Remove claim that scenario tests need 2 services https://review.openstack.org/47067 | 14:55 |
| *** RecheckWatchBot has quit IRC | 15:06 | |
| *** wramthun has quit IRC | 15:17 | |
| *** wramthun has joined #openstack-qa | 15:22 | |
| *** RecheckWatchBot has joined #openstack-qa | 15:26 | |
| *** mkollaro has quit IRC | 15:30 | |
| RecheckWatchBot | openstack/horizon change: https://review.openstack.org/47584 failed tempest with an unrecognized error | 15:31 |
| *** openstackgerrit has quit IRC | 15:33 | |
| *** openstackgerrit has joined #openstack-qa | 15:33 | |
| *** yassine has quit IRC | 15:35 | |
| RecheckWatchBot | openstack/keystone change: https://review.openstack.org/46613 failed tempest because of: https://bugs.launchpad.net/bugs/1211915 | 15:42 |
| *** mkollaro has joined #openstack-qa | 15:49 | |
| *** mkollaro has quit IRC | 15:49 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add locking to test_list_hosts_with_zone https://review.openstack.org/47599 | 15:51 |
| mtreinish | jog0: ^^ a new bug caught by elasticRecheck :) | 15:52 |
| *** jeblair is now known as corvus | 15:53 | |
| *** corvus is now known as jeblair | 16:00 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Sync install_venv_common.py from oslo-incubator https://review.openstack.org/47602 | 16:01 |
| RecheckWatchBot | openstack/nova change: https://review.openstack.org/47114 failed tempest because of: https://bugs.launchpad.net/bugs/1191960 | 16:08 |
| *** dims has joined #openstack-qa | 16:09 | |
| dims | mtreinish, Hi, Got a few mins for a tempest review? Please let me know. Taking a stab at something that is triggering a lot of rechecks - https://review.openstack.org/#/c/47494/ | 16:10 |
| *** thomasbiege1 has joined #openstack-qa | 16:13 | |
| RecheckWatchBot | openstack/horizon change: https://review.openstack.org/47584 failed tempest with an unrecognized error | 16:13 |
| dims | annegentle, i have a review for the FontTriplet/Jenkins issue too (https://review.openstack.org/#/c/47509/) - not sure who i have to ask for review and on which channel | 16:13 |
| *** thomasbiege1 has quit IRC | 16:16 | |
| *** bashok has quit IRC | 16:18 | |
| *** comstud is now known as bearhands | 16:24 | |
| openstackgerrit | Anju Tiwari proposed a change to openstack/tempest: Added Negative tests for image member https://review.openstack.org/47167 | 16:57 |
| *** hemnafk is now known as hemna | 16:57 | |
| *** bashok has joined #openstack-qa | 16:57 | |
| *** fbo is now known as fbo_away | 17:04 | |
| mtreinish | dims: sure give me a min though I just got back from lunch | 17:12 |
| dims | mtreinish, thx! | 17:13 |
| mtreinish | dims: reviewed | 17:38 |
| dims | mtreinish, thx will fix the commit message | 17:41 |
| mtreinish | dims: ok, but I'm still not comfortable pushing it through until we better characterize that bug. | 17:42 |
| *** jasondotstar has quit IRC | 17:44 | |
| annegentle | dims: huh, do we ignore fop errors at our own peril though? | 17:44 |
| dims | annegentle, we are just making sure the cache is empty, that's all | 17:45 |
| dims | the Fop 1.0 is 3-4 years old | 17:45 |
| annegentle | dims: oh ok, and I see the pool for multiprocessing too | 17:45 |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack/tempest: Ensure server is in ACTIVE state, Fix for transient failures https://review.openstack.org/47494 | 17:46 |
| mtreinish | dims: ^^ my objection on the title wasn't the '-possible' but the whole fix for transient failures part :) | 17:47 |
| mtreinish | Ensure server is in ACTIVE state is enough for the commit subject | 17:48 |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack/tempest: Ensure server is in ACTIVE state https://review.openstack.org/47494 | 17:49 |
| dims | thx again | 17:49 |
| *** dkranz has quit IRC | 18:00 | |
| mtreinish | jog0: one thing I've been noticing with elasticRecheck is that it spends a lot of time waiting for ES and I'm pretty sure that it drops other failures while it's waiting | 18:16 |
| mtreinish | I'm thinking we might want to break out each classify() call into it's own thread or process | 18:17 |
| mtreinish | (a thread probably makes more sense) | 18:18 |
| *** thomasbiege has joined #openstack-qa | 18:20 | |
| mtreinish | jog0: something like this: http://paste.openstack.org/show/47336/ (and an equivelent change for bot.py) | 18:33 |
| *** thomasbiege has quit IRC | 18:36 | |
| *** jasondotstar has joined #openstack-qa | 18:40 | |
| *** leizhang has quit IRC | 18:55 | |
| clarkb | mtreinish: or a queue at least | 19:03 |
| mtreinish | clarkb: yeah that would work too, but I still think there is a bug in the waiting code, because it's still waiting on: https://review.openstack.org/#/c/47387/2 | 19:12 |
| mtreinish | which failed quite some time ago | 19:12 |
| clarkb | gotcha | 19:13 |
| *** RecheckWatchBot has quit IRC | 19:13 | |
| *** RecheckWatchBot has joined #openstack-qa | 19:14 | |
| *** weshay has quit IRC | 19:32 | |
| *** malini is now known as malini_afk | 19:33 | |
| openstackgerrit | Clint Byrum proposed a change to openstack/tempest: Test that Heat max_template_size is applied https://review.openstack.org/45732 | 19:34 |
| *** weshay has joined #openstack-qa | 19:48 | |
| morganfainberg | clarkb, once a change has gone into tempest (merged) is there a delay (or are devstack gates take the latest tempest each run?) before it takes effect? | 20:03 |
| morganfainberg | clarkb, if you know. | 20:03 |
| clarkb | morganfainberg: no delay | 20:04 |
| clarkb | morganfainberg: in fact the change behind it in the queue is tested with the change (no need to wait for merging) | 20:04 |
| morganfainberg | clarkb, awesome, thanks. | 20:04 |
| morganfainberg | clarkb, helps me move towards getting RC1 bugs worked out :) | 20:05 |
| RecheckWatchBot | openstack/keystone change: https://review.openstack.org/46207 failed tempest with an unrecognized error | 20:08 |
| RecheckWatchBot | openstack/keystone change: https://review.openstack.org/47514 failed tempest with an unrecognized error | 20:11 |
| jog0 | mtreinish: looking | 20:12 |
| jog0 | mtreinish: I was under the assumption that the gerritlib queued changes | 20:13 |
| jog0 | I think using a queue is better then adding threading in because its less complex | 20:14 |
| mtreinish | jog0: ok that's what I figured (and I agree too) This was just kind of a cheap fix to get around when we get stuck waiting. | 20:16 |
| jog0 | mtreinish: and I do think there is something wrong with teh wait logic | 20:17 |
| jog0 | not sure what yet though | 20:17 |
| jog0 | looking now | 20:19 |
| *** dims has quit IRC | 20:21 | |
| *** dims has joined #openstack-qa | 20:25 | |
| jog0 | mtreinish: about to board a flight so will have to pick this up in a few hours | 20:27 |
| mtreinish | jog0: sure no problem I'll take a quick look too before I head home in <1hr | 20:28 |
| jog0 | oh right east coast, o/ | 20:29 |
| RecheckWatchBot | openstack/neutron change: https://review.openstack.org/47355 failed tempest because of: https://bugs.launchpad.net/bugs/1211915 | 20:31 |
| *** bashok is now known as ashok2988 | 20:31 | |
| mikal | jog0: so... I've now cycled back to that tempest failure from yesterday | 20:36 |
| mikal | jog0: what's the standard way forward where tempest is revealing a genuine bug in nova? Disable the tempest test and file a bug? | 20:36 |
| RecheckWatchBot | openstack/swift change: https://review.openstack.org/45134 failed tempest with an unrecognized error | 20:41 |
| *** hootjr has quit IRC | 20:45 | |
| *** ashok2988 has quit IRC | 20:47 | |
| *** bdperkin is now known as bdperkin_gone | 20:53 | |
| *** weshay has quit IRC | 21:00 | |
| *** malini_afk is now known as malini | 21:04 | |
| *** rfolco has quit IRC | 21:16 | |
| *** EmilienM has quit IRC | 21:22 | |
| openstackgerrit | Eoghan Glynn proposed a change to openstack-dev/devstack: Modified ceilometer alarm evaluator console script https://review.openstack.org/47637 | 21:23 |
| *** EmilienM has joined #openstack-qa | 21:23 | |
| *** mriedem has quit IRC | 21:33 | |
| sdague | mikal: is this an always fail, or a sometimes fail? | 21:34 |
| mikal | sdague: well, its a little complicated | 21:34 |
| mikal | I just fixed a bug which was ignoring failures to inject mandatory data into images | 21:35 |
| mikal | Which uncovered a persistant failure in tempest, which previously wasn't fatal to the test but should have been | 21:35 |
| mikal | (i.e. an instance is trying to inject data into a non-existant image because its a boot from volume instance) | 21:35 |
| sdague | mikal: the problem with disabling tests in tempest... is they tend to stay that way for 6 months | 21:35 |
| mikal | So, my patch didn't break this test | 21:35 |
| mikal | But it did reveal a previously masked bug | 21:36 |
| sdague | so it would be much better if we could get the main issue fixed instead | 21:36 |
| mikal | Which is sort-of-related | 21:36 |
| mikal | I'm not keen on blocking my current bug fix because it revealed another bug though | 21:36 |
| mikal | I agree the other bug should be fixed, but I'd like to see that done in parallel | 21:36 |
| mikal | Partially because I think we need a mailing list thread here? | 21:37 |
| mikal | Should file inject work on boot from volume? I'm undecided. | 21:37 |
| sdague | sure, why don't we take it there, and agree on path forward, including the tempest side | 21:37 |
| mikal | Well, if we filed a bug for the newly revealed failure, and added a skip for this tempest test | 21:37 |
| mikal | Then I would assign that bug to myself immediate and start the tread | 21:37 |
| mikal | s/immediate/immediately/ | 21:38 |
| mikal | s/tread/thread/ | 21:38 |
| mikal | sigh | 21:38 |
| mikal | I think its a pretty big deal that file injection currently silently fails | 21:38 |
| sdague | agreed | 21:38 |
| sdague | push all the reviews, get a thread going. I'll jump on it monday ... also why you working sat morning :) | 21:39 |
| mikal | Because I'm a loser | 21:39 |
| mikal | Well, the review for the inject fix is out there now, but fails tempest so will never land | 21:40 |
| mikal | https://review.openstack.org/#/c/46867/ | 21:40 |
| sdague | hmmm... so if I look at that fail it seems to indication that boot from volume no longer works after that patch. is that correct? | 21:43 |
| mikal | Boot from volume where file injection is configured will no longer work. | 21:43 |
| mikal | But it wasn't working already | 21:43 |
| mikal | i.e. an instance without its injected files is kind of useless | 21:43 |
| mikal | (no root password, no network config, etc etc) | 21:44 |
| sdague | not if it is cloud initing | 21:44 |
| mikal | I'd be comfortable with a world in which we said to use config drive or metadata server for this instance type | 21:44 |
| mikal | That's true, but then why is injection turned on? | 21:44 |
| mikal | Ok, I have a draft email I will send now | 21:47 |
| mikal | You're cc'ed | 21:47 |
| sdague | ok | 21:48 |
| *** adalbas has quit IRC | 21:54 | |
| *** notmyname has joined #openstack-qa | 21:57 | |
| notmyname | I have a tempest question | 21:57 |
| notmyname | or maybe just a request for more eyes | 21:57 |
| notmyname | anyone here who can assist with looking at logs on a failing test run? | 21:58 |
| sdague | notmyname: fire away, though it's friday evening, so I'm going to depart soon for a drink | 21:58 |
| notmyname | sdague: don't worry, this will make you want to drink more ;-) | 21:58 |
| sdague | heh | 21:58 |
| notmyname | sdague: http://logs.openstack.org/34/45134/8/check/gate-tempest-devstack-vm-postgres-full/4006575/console.html <-- failed test | 21:59 |
| notmyname | looks like a swift bug, right? | 21:59 |
| notmyname | ok, so look into the swift proxy logs: http://logs.openstack.org/34/45134/8/check/gate-tempest-devstack-vm-postgres-full/4006575/logs/screen-s-proxy.txt.gz (search for TestContainer189244130) | 21:59 |
| notmyname | and you'll see that swift is returning an error | 21:59 |
| notmyname | ie trying to create a container, but getting a 404 | 21:59 |
| notmyname | this happens when the account doesn't exist | 22:00 |
| sdague | ok | 22:00 |
| notmyname | but if you look at the line(s) right above, you'll see that the account was created | 22:00 |
| notmyname | and for further wierdness, http://logs.openstack.org/34/45134/8/check/gate-tempest-devstack-vm-neutron/f3e1ad1/console.html passed the same test on the same patch | 22:00 |
| notmyname | (search for http://logs.openstack.org/34/45134/8/check/gate-tempest-devstack-vm-neutron/f3e1ad1/console.html) | 22:00 |
| notmyname | err... | 22:00 |
| notmyname | search for tempest.api.object_storage.test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds | 22:01 |
| notmyname | so it passes on one, fails on the other (for the same swift patch) | 22:01 |
| notmyname | and the swift logs say that it's doing the right thing (other than the final 404) | 22:01 |
| sdague | so those tests may very well be executing at exactly the same time | 22:02 |
| notmyname | looking at the transaction id (tx3ff8dd5a2b1b4f4dbfc70-00523cbd62) for the failing test in the other logs (ie account logs), you'll see that it all looks fine | 22:02 |
| sdague | could there be an issolation issue | 22:02 |
| notmyname | perhaps (and perhaps likely), let me look at one more thing | 22:03 |
| sdague | especially if cleanup wasn't issolated correctly | 22:04 |
| sdague | which would be my wild guess | 22:04 |
| notmyname | there is no delete for the account logged, so I don't think it's that | 22:04 |
| sdague | ok | 22:04 |
| notmyname | FWIW, AUTH_f334ca5f79a94192966fe3833c947d85 is the account string to search for | 22:04 |
| sdague | notmyname: ok, in which logs | 22:05 |
| notmyname | any of the swift ones | 22:05 |
| sdague | container or object | 22:05 |
| sdague | ok | 22:05 |
| notmyname | ok, the account server PUT request for that account (ie when it's created) has the same transaction id as the failing test (tx3ff8dd5a2b1b4f4dbfc70-00523cbd62) | 22:06 |
| notmyname | sdague: anything else you can tell me about how these tests are run? any diferences between the one that passes and the one that fails? | 22:10 |
| sdague | actually, they are in the same class, so they aren't running in parallel | 22:11 |
| *** gfidente has quit IRC | 22:11 | |
| notmyname | ah, cool good to know | 22:12 |
| sdague | we are running parallel by class in tempest, on 4 cores. But testr distributes them somewhat arbitrarily | 22:13 |
| morganfainberg | any tempest folks around that have a few to step me through a couple things? I'm struggling to understand some of the identity setup stuff (e.g. create_user) and chase down possibly the mishmash of tenantId, project_id, default_project_id that seems to be referencing items on the user that may be incorrect. | 22:13 |
| sdague | so there may be other object tests running at the same time | 22:13 |
| sdague | so all tests in a class will be in on testr process, but different test classes could be in different processes | 22:14 |
| morganfainberg | specifically, is http://git.openstack.org/cgit/openstack/tempest/tree/tempest/services/identity/v3/json/identity_client.py#n50 supposed to be the default project? | 22:15 |
| *** dims has quit IRC | 22:16 | |
| sdague | anyway, time for me to run off to dinner, and I think most folks here are east coast or .eu, so they are probably at the bar or asleep atm | 22:20 |
| notmyname | sdague: thanks | 22:25 |
| notmyname | sdague: we're still looking over in -swift | 22:25 |
| *** dims has joined #openstack-qa | 22:30 | |
| *** openstack has quit IRC | 22:35 | |
| *** openstack has joined #openstack-qa | 22:35 | |
| *** malini is now known as malini_afk | 22:36 | |
| *** adalbas has joined #openstack-qa | 23:15 | |
| *** hemna is now known as hemnafk | 23:27 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-dev/devstack: Nicira plugin: do not die if NVP gateway IP is missing https://review.openstack.org/47414 | 23:31 |
| *** dims has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!