| oomichi | catherineD: hi, we have changed tempest structure of Cinder API tests so much based on http://lists.openstack.org/pipermail/openstack-dev/2017-March/114507.html | 00:11 |
|---|---|---|
| *** markvoelker has joined #openstack-interop | 00:13 | |
| oomichi | catherineD: the change is done now, so we'd like to apply changes to defcore side. do you have any tool to check the existence of tempest tests on defcore side ? | 00:13 |
| oomichi | catherineD: eg: the tool checks next.json by compairing the latest tempest | 00:15 |
| *** markvoelker has quit IRC | 04:11 | |
| *** markvoelker has joined #openstack-interop | 05:12 | |
| *** markvoelker has quit IRC | 05:17 | |
| *** gcb has quit IRC | 05:26 | |
| *** gcb has joined #openstack-interop | 05:39 | |
| *** openstackgerrit has joined #openstack-interop | 06:06 | |
| openstackgerrit | zhufl proposed openstack/defcore master: Remove duplicated testcase test_get_private_image https://review.openstack.org/454489 | 06:24 |
| *** pcaruana has joined #openstack-interop | 06:34 | |
| *** gcb has quit IRC | 06:38 | |
| *** gcb has joined #openstack-interop | 06:39 | |
| *** gcb has quit IRC | 06:46 | |
| *** gcb has joined #openstack-interop | 06:51 | |
| openstackgerrit | zhufl proposed openstack/defcore master: Remove duplicated testcase test_get_private_image https://review.openstack.org/454489 | 07:02 |
| *** markvoelker has joined #openstack-interop | 07:13 | |
| *** markvoelker has quit IRC | 07:17 | |
| *** MarkBaker has joined #openstack-interop | 08:01 | |
| *** gcb has quit IRC | 08:09 | |
| *** gcb has joined #openstack-interop | 08:10 | |
| *** cdent has joined #openstack-interop | 08:42 | |
| *** MarkBaker has quit IRC | 10:03 | |
| *** MarkBaker has joined #openstack-interop | 10:16 | |
| *** cdent has quit IRC | 10:38 | |
| *** markvoelker has joined #openstack-interop | 11:14 | |
| *** openstackgerrit has quit IRC | 11:18 | |
| *** markvoelker has quit IRC | 11:19 | |
| *** pcaruana has quit IRC | 11:22 | |
| *** cdent has joined #openstack-interop | 11:50 | |
| *** markvoelker has joined #openstack-interop | 12:14 | |
| *** gcb has quit IRC | 12:15 | |
| *** gcb has joined #openstack-interop | 12:29 | |
| *** gcb has quit IRC | 12:46 | |
| *** MarkBaker has quit IRC | 13:08 | |
| *** MarkBaker has joined #openstack-interop | 13:12 | |
| *** MarkBaker has quit IRC | 13:17 | |
| *** MarkBaker has joined #openstack-interop | 13:30 | |
| *** cdent has quit IRC | 13:43 | |
| *** cdent has joined #openstack-interop | 13:43 | |
| *** cdent has quit IRC | 14:29 | |
| *** cdent has joined #openstack-interop | 14:52 | |
| *** cdent has quit IRC | 15:17 | |
| *** cdent has joined #openstack-interop | 15:17 | |
| *** cdent has quit IRC | 15:59 | |
| *** cdent has joined #openstack-interop | 17:05 | |
| catherineD | oomichi: Hi The way I usually do is running RefStack tests with the lastest Tempest version and see which tests fail. Let me do one run and send you the test list that get affected. | 17:13 |
| oomichi | catherineD: cool, that could be helpful. I am creating some script to check the existence of Tempest tests on my side | 17:14 |
| oomichi | catherineD: The script detects 127 tests don't exit on Tempest from next.json | 17:14 |
| catherineD | oomichi: great! Will let you know | 17:15 |
| catherineD | wow that is a lot of tests !!! | 17:15 |
| oomichi | catherineD: most part is due to Cinder api test restructure, but it detects 2 neutron tests don't exist on Tempest | 17:16 |
| catherineD | oh is this only Image tests? I low that we have about 45 tests that does not exist but already taking care of by adding aliases | 17:16 |
| catherineD | ok | 17:16 |
| oomichi | catherineD: ah, I didn't care about aliases TBH, I guessed aliases should be for old Tempest, right? That means non-aliases should represent the latest tempest, right? | 17:17 |
| catherineD | yea the guidelines would include some tests that no longer exists in Tempest.... take a look at networks-l2-CRUD on https://refstack.openstack.org/#/guidelines | 17:19 |
| catherineD | oomichi: currently, we should have around 44 known tests that are aliases and do not exist in latest Tempest. .. | 17:21 |
| oomichi | catherineD: cool, OK. Let me double-check on my side | 17:22 |
| catherineD | oomichi: so there may be 127 - 44 = 83 new aliaes ... let me do some testing and will send you the list .... | 17:22 |
| catherineD | aliases | 17:22 |
| catherineD | then we can compare our findings | 17:22 |
| *** openstackgerrit has joined #openstack-interop | 17:26 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Fix test module path of networks-subnet-pools-CRUD https://review.openstack.org/454813 | 17:26 |
| oomichi | catherineD: yeah, I will dig more | 17:26 |
| oomichi | catherineD: ^^^ is the first step for matching both sides | 17:26 |
| oomichi | catherineD: do I need to fix 2017.01.json also? | 17:27 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Fix test module path of networks-subnet-pools-CRUD https://review.openstack.org/454813 | 17:28 |
| catherineD | oomichi: yes the first step is to identified the tests which got rename and being used by DefCore for either required or advisory | 17:31 |
| catherineD | the fixes need to be done on 2016.08, 2017.01 and next | 17:31 |
| oomichi | catherineD: I see. The above test is included in 2917.01 and next only without 2016.08 | 17:32 |
| catherineD | that is good | 17:32 |
| oomichi | catherineD: the latest patch https://review.openstack.org/454813 could work good, I guess now | 17:32 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Fix test module path of networks-subnet-pools-CRUD https://review.openstack.org/454813 | 17:33 |
| oomichi | catherineD: I did break JSON format on PS2, the above PS3 could be good, sorry about that | 17:34 |
| catherineD | oomichi: for 2017.01 and 2017.08 we need to add aliases ( not just fix the name) .... because they are approved guildelines | 17:34 |
| catherineD | for next it is depending on whether it is a test that the users have a chance to test it before... most of the time need to add aliases | 17:35 |
| oomichi | catherineD: the test never exist on Tempest side as the first commit of the test, do we need aliase for this case? | 17:35 |
| catherineD | one sec ... let me check | 17:37 |
| oomichi | catherineD: https://review.openstack.org/#/c/227119/ has added the test on Tempest at the first commit. | 17:37 |
| oomichi | the test module name is different from defcore side | 17:38 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Add aliase for test_update_extra_route https://review.openstack.org/454825 | 17:48 |
| oomichi | catherineD: ^^^ is one more for network test, that was due to test name change on Tempest side | 17:51 |
| catherineD | I am running the entire API test ... so it would take about 1.5 to 2 hours ... will let you know .. | 17:52 |
| oomichi | catherineD: oh, thanks for running all API tests. Yeah, that should take time | 17:55 |
| catherineD | yea I will get back to you | 17:55 |
| oomichi | catherineD: btw how about running the Tempest test existence check on defcore(or refstack) gate as non-voting job as referrence? | 17:56 |
| oomichi | that could be helpful for us | 17:56 |
| mnaser | does this channel cover/talk about some of the work under the interop challenge or is that another unrelated thing | 18:13 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Add aliases for test_volumes_list https://review.openstack.org/454844 | 18:24 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Add aliases for test_volumes_list https://review.openstack.org/454844 | 18:28 |
| *** krtaylor has quit IRC | 19:18 | |
| *** openstackgerrit has quit IRC | 19:19 | |
| *** cdent has quit IRC | 19:52 | |
| *** tongli has joined #openstack-interop | 22:35 | |
| oomichi | catherineD: thanks for your review, I have one question about next.json. Do we need to keep old Tempest test paths as aliases on next.json ? I guessed next.json could be for newer tempest, is that wrong? | 22:36 |
| catherineD | The reason we need to keep the old one in the next because at any instant DefCore will have to support 2 version + next guideline ... | 22:38 |
| catherineD | for today it is 2016/08 and 2017.01 and next (which will be 2017.08 ?) | 22:38 |
| catherineD | so the tests may be still available in the previous guideline | 22:39 |
| catherineD | aslo DefCore has not removed any aliases so far.. | 22:39 |
| catherineD | I think this is a good discussion item for the next IRC meeting | 22:40 |
| oomichi | catherineD: ok, thanks. Maybe I just wanted to avoid maintaining old temepst test list on defcore because I'm lazy | 22:41 |
| catherineD | The volume V2 name change will be messy because it affects a lot of tests | 22:42 |
| catherineD | yea we can put that in the meeting agenda for discussion | 22:42 |
| catherineD | Not really lazy .. it is a lot of work and messy ... I feel like it will affect about 60 tests !!! | 22:43 |
| catherineD | That is a lot of tests to add aliases .. | 22:43 |
| *** tongli has quit IRC | 22:45 | |
| oomichi | catherineD: yeah, that is my concern due to messy json | 23:27 |
| *** openstackgerrit has joined #openstack-interop | 23:33 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Fix test module path of networks-subnet-pools-CRUD https://review.openstack.org/454813 | 23:33 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/defcore master: Add aliase for test_update_extra_route https://review.openstack.org/454825 | 23:37 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!