| *** primeministerp1 has quit IRC | 00:00 | |
| *** ohnoimdead has quit IRC | 00:08 | |
| *** hisaharu has quit IRC | 00:09 | |
| * Daviey raises apologies to ttx for missing the meeting | 00:11 | |
| *** dendrobates is now known as dendro-afk | 00:18 | |
| *** dendro-afk is now known as dendrobates | 00:22 | |
| *** nati2 has quit IRC | 00:24 | |
| *** donald650 has quit IRC | 00:27 | |
| *** Kiall has quit IRC | 00:30 | |
| rmk | oops | 00:36 |
|---|---|---|
| *** rmk has left #openstack-meeting | 00:36 | |
| *** nati2 has joined #openstack-meeting | 00:38 | |
| *** Kiall has joined #openstack-meeting | 00:38 | |
| *** somik has quit IRC | 00:46 | |
| *** martine has joined #openstack-meeting | 00:46 | |
| *** ziyadb has quit IRC | 00:48 | |
| *** adjohn has quit IRC | 00:53 | |
| *** dendrobates is now known as dendro-afk | 01:20 | |
| *** dragondm has quit IRC | 01:35 | |
| *** rkukura has joined #openstack-meeting | 01:51 | |
| *** zns has joined #openstack-meeting | 01:56 | |
| *** medberry is now known as med_out | 01:57 | |
| *** chrisg1 has left #openstack-meeting | 01:58 | |
| *** novas0x2a|laptop has quit IRC | 02:13 | |
| *** martine has quit IRC | 02:31 | |
| *** shang has quit IRC | 02:32 | |
| *** shang has joined #openstack-meeting | 02:34 | |
| *** nati2_ has joined #openstack-meeting | 02:45 | |
| *** nati2 has quit IRC | 02:46 | |
| *** anotherjesse has joined #openstack-meeting | 02:47 | |
| *** nati2_ has quit IRC | 02:57 | |
| *** zul has joined #openstack-meeting | 03:13 | |
| *** shang_ has joined #openstack-meeting | 03:21 | |
| *** anotherjesse has quit IRC | 03:22 | |
| *** rkukura has quit IRC | 03:32 | |
| *** anotherjesse has joined #openstack-meeting | 03:35 | |
| *** anotherjesse has quit IRC | 03:37 | |
| *** rkukura has joined #openstack-meeting | 03:44 | |
| *** toluene has joined #openstack-meeting | 03:49 | |
| *** rkukura has quit IRC | 03:51 | |
| *** zul has quit IRC | 03:53 | |
| *** clayg has left #openstack-meeting | 03:59 | |
| *** adjohn has joined #openstack-meeting | 04:03 | |
| *** hggdh has quit IRC | 04:05 | |
| *** hggdh has joined #openstack-meeting | 04:08 | |
| *** reed has quit IRC | 04:23 | |
| *** danwent has joined #openstack-meeting | 04:28 | |
| *** littleidea has quit IRC | 04:59 | |
| *** nati2 has joined #openstack-meeting | 05:21 | |
| *** danwent has left #openstack-meeting | 05:25 | |
| *** shang has quit IRC | 06:03 | |
| *** shang_ has quit IRC | 06:03 | |
| *** shang has joined #openstack-meeting | 06:04 | |
| *** adjohn has quit IRC | 06:04 | |
| *** zns has quit IRC | 06:25 | |
| *** toluene has quit IRC | 06:30 | |
| *** joonwon has joined #openstack-meeting | 07:50 | |
| *** joonwon has quit IRC | 07:53 | |
| *** nati2 has quit IRC | 08:14 | |
| *** nati2 has joined #openstack-meeting | 08:14 | |
| *** nati2 has quit IRC | 08:38 | |
| *** darraghb has joined #openstack-meeting | 08:56 | |
| *** tsuzuki_ has joined #openstack-meeting | 08:57 | |
| *** ovidwu has quit IRC | 09:25 | |
| *** primeministerp1 has joined #openstack-meeting | 11:03 | |
| *** tsuzuki_ has quit IRC | 11:27 | |
| *** primeministerp1 has quit IRC | 11:53 | |
| *** zul has joined #openstack-meeting | 12:02 | |
| *** rkukura has joined #openstack-meeting | 12:03 | |
| *** dendro-afk is now known as dendrobates | 13:11 | |
| *** littleidea has joined #openstack-meeting | 13:12 | |
| *** jaypipes has joined #openstack-meeting | 13:13 | |
| *** jsavak has joined #openstack-meeting | 13:15 | |
| *** zns has joined #openstack-meeting | 13:25 | |
| *** rkukura has quit IRC | 13:28 | |
| *** martine has joined #openstack-meeting | 13:49 | |
| *** mdomsch has joined #openstack-meeting | 13:54 | |
| *** Gordonz has joined #openstack-meeting | 14:00 | |
| *** zns has quit IRC | 14:00 | |
| *** jsavak has quit IRC | 14:01 | |
| *** jsavak has joined #openstack-meeting | 14:01 | |
| *** wwkeyboard has joined #openstack-meeting | 14:10 | |
| *** rkukura has joined #openstack-meeting | 14:10 | |
| *** adrian17od has joined #openstack-meeting | 14:12 | |
| *** reed_ has joined #openstack-meeting | 14:32 | |
| *** zns has joined #openstack-meeting | 14:49 | |
| *** danwent has joined #openstack-meeting | 14:55 | |
| *** zns has quit IRC | 14:57 | |
| *** dragondm has joined #openstack-meeting | 14:59 | |
| *** adjohn has joined #openstack-meeting | 15:03 | |
| *** zns has joined #openstack-meeting | 15:03 | |
| *** adrian17od has quit IRC | 15:13 | |
| *** adrian17od has joined #openstack-meeting | 15:20 | |
| *** med_out is now known as medberry | 15:21 | |
| *** rnirmal has joined #openstack-meeting | 15:26 | |
| *** adrian17od has quit IRC | 15:32 | |
| *** deshantm_laptop has joined #openstack-meeting | 15:41 | |
| *** corrigac has joined #openstack-meeting | 15:47 | |
| *** ColinLyons has joined #openstack-meeting | 15:49 | |
| *** danwent has left #openstack-meeting | 15:49 | |
| *** nati2 has joined #openstack-meeting | 15:53 | |
| *** mahmoh has joined #openstack-meeting | 15:54 | |
| *** dwalleck has joined #openstack-meeting | 16:02 | |
| jaypipes | are we ready to start the QA meeting? | 16:04 |
| dwalleck | Sure | 16:04 |
| jaypipes | nati2: ? | 16:04 |
| *** bcwaldon has joined #openstack-meeting | 16:05 | |
| nati2 | oh we held QA meeting here? | 16:05 |
| jaypipes | nati2: yes | 16:05 |
| *** mattray has joined #openstack-meeting | 16:05 | |
| nati2 | Ohw sorry! | 16:05 |
| jaypipes | #startmeeting | 16:05 |
| openstack | Meeting started Wed Oct 12 16:05:27 2011 UTC. The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:05 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic. | 16:05 |
| jaypipes | #info if you are not in the QA team and want to be, please join https://launchpad.net/~openstack-qa-team | 16:05 |
| jaypipes | #topic Status on unit test analysis and bug feedback | 16:06 |
| *** openstack changes topic to "Status on unit test analysis and bug feedback" | 16:06 | |
| *** gigig has joined #openstack-meeting | 16:06 | |
| *** venkater_ has joined #openstack-meeting | 16:06 | |
| jaypipes | nati2: care to give a status? I notice a bunch of bugs from you and your teammates :) | 16:06 |
| *** patelna_ has joined #openstack-meeting | 16:06 | |
| *** rkukura has left #openstack-meeting | 16:07 | |
| jaypipes | patelna_: hi nayna | 16:07 |
| patelna_ | hey Jay | 16:07 |
| jaypipes | patelna_: just started the meeting... current topic is status of unit test stuff. nati2 to provide status | 16:07 |
| patelna_ | great...Nachi joining | 16:07 |
| nati2 | Hi Jay , hold on please, I thought the meeting was on #openstack-qa http://etherpad.openstack.org/openstack-qa. I'm redirecting guys who joined to #openstack-qa | 16:07 |
| patelna_ | we're in the #openstack-QA channel | 16:07 |
| jaypipes | patelna_: ah, no, this channel :) | 16:08 |
| jaypipes | patelna_: this channel has the meetingbot and stuff... | 16:08 |
| patelna_ | Got it!!!! thanks | 16:08 |
| nati2 | yes, let's use this channel | 16:08 |
| *** KumarKR has joined #openstack-meeting | 16:08 | |
| gigig | Hello Nayna and Jay | 16:08 |
| jaypipes | gigig: morning/afternoon :) | 16:09 |
| *** markmc has joined #openstack-meeting | 16:09 | |
| nati2 | Hi gigi! | 16:09 |
| patelna_ | Hey Gigigi | 16:09 |
| gigig | hi nachi :) | 16:09 |
| jaypipes | ALL: we just started the meeting... current topic is status of unit test stuff. nati2 to provide status | 16:09 |
| nati2 | OK | 16:09 |
| nati2 | Nayna are you joining here? | 16:09 |
| jaypipes | nati2: patelna_ is nayna | 16:10 |
| KumarKR | hello folks | 16:10 |
| patelna_ | I am | 16:10 |
| mtaylor | hey gigig. hey patelna_ | 16:10 |
| patelna_ | Hey Monty | 16:10 |
| nati2 | hey monty | 16:10 |
| gigig | hey monty | 16:10 |
| mtaylor | hey nati2 | 16:10 |
| patelna_ | Nachi = Patelna = nayna | 16:10 |
| nati2 | gotcha! | 16:10 |
| patelna_ | let's go... | 16:10 |
| patelna_ | please provide an update on Unit Test Coverage | 16:10 |
| nati2 | OK My team start analyzing code and adding unit test on it | 16:11 |
| nati2 | you can see coverage at https://jenkins.openstack.org/ | 16:11 |
| nati2 | But our stuff is not merged yet. | 16:11 |
| nati2 | And also we added some blueprints and bugs | 16:12 |
| nati2 | https://blueprints.launchpad.net/openstack-qa/+spec/nova-exception-policy | 16:12 |
| nati2 | This buleprint is for exception and log handling | 16:12 |
| nati2 | Based on this policy, We added some bugs Related bugs | 16:13 |
| nati2 | sorry typo | 16:13 |
| jaypipes | nati2: where are the branches that are proposed to nova for the QA bug fixes and test cases? I don't see any here: https://review.openstack.org/#q,status:open+project:openstack/nova,n,z | 16:13 |
| venkater_ | hey nati2 - I see results for Nova .. Is swift unit tests also run? | 16:13 |
| nati2 | I think it is not created yet. | 16:13 |
| nati2 | venkater_: > Monty Do you know about swift unit tests status? | 16:14 |
| nati2 | See related bugs on https://blueprints.launchpad.net/openstack-qa/+spec/nova-exception-policy. | 16:14 |
| mtaylor | swift unit tests are run as part of gating the swift trunk | 16:14 |
| *** dendrobates is now known as dendro-afk | 16:14 | |
| nati2 | We also add input value validation blueprints. https://blueprints.launchpad.net/openstack-qa/+spec/nova-input-value-validation-policy | 16:15 |
| mtaylor | and coverage job for swift is: https://jenkins.openstack.org/view/Swift/job/swift-coverage/ | 16:15 |
| nati2 | And manager api policy https://blueprints.launchpad.net/openstack-qa/+spec/nova-manager-api-policy | 16:15 |
| nati2 | thanks mtaylor | 16:15 |
| nati2 | There are more 10 bugs reported in my lab (but it is Japanese now, I'll translate it today :) ) | 16:16 |
| gigig | haha | 16:16 |
| nati2 | Would you please comment and question about blueprints and bugs ? | 16:16 |
| jaypipes | nati2: ok, good. yes, I'd like to see those blueprints containing more details... | 16:17 |
| nati2 | It it all from me. | 16:17 |
| nati2 | gotcha. I'll add examples and more details | 16:17 |
| dwalleck | Hi, Daryl from Rack QE. The blueprint for API input validation sounds more like addition of input validation, not the testing of it. Is that correct? | 16:17 |
| jaypipes | nati2: well, not the exception blueprint :) that one is great! | 16:17 |
| *** patelna has joined #openstack-meeting | 16:17 | |
| nati2 | gotcha : jaypipes | 16:17 |
| jaypipes | nati2: see dwalleck's ? above | 16:18 |
| nati2 | dwalleck: I think it is in scope of QA | 16:18 |
| dwalleck | nati2: In scope for us to add it? Or test it? | 16:18 |
| nati2 | Because the code must support irregular case. | 16:18 |
| nati2 | I suppose the scope of QA is not only test | 16:18 |
| dwalleck | Testing it certainly. I didn't know it would be our blueprints to actually add the validation | 16:18 |
| nati2 | We should specify unclear specs also. | 16:19 |
| patelna | QA scope is to consolidate and add tests as well | 16:19 |
| gigig | agreed | 16:19 |
| wwkeyboard | I would think the quality assurance team would tackle anything that helps assure the quality of the project | 16:19 |
| nati2 | wwkeyboard: ++ | 16:19 |
| wwkeyboard | input validation and exception handling would be part of that. | 16:19 |
| dwalleck | Okay, just making sure | 16:19 |
| *** KenWhite-RAX has joined #openstack-meeting | 16:20 | |
| wwkeyboard | As well as code quality and database design. | 16:20 |
| patelna | for Diablo-4 QA is catching up with the coverage | 16:20 |
| *** vladimir3p has joined #openstack-meeting | 16:20 | |
| nati2 | Yes I agreee | 16:20 |
| nati2 | Statement coverage is one of mesurement | 16:20 |
| nati2 | Most important thing is to solve unclear specs | 16:21 |
| *** markvoelker has joined #openstack-meeting | 16:21 | |
| gigig | how are we measuring test coverage or rather tracking it | 16:21 |
| patelna | agree | 16:21 |
| nati2 | and specs are based on coding policies | 16:21 |
| wwkeyboard | nati2: I agree, thats why I fear coverage as a metric. | 16:21 |
| dwalleck | Right, I just didn't understand that it was the responsibility of the group was not just to test validation, but add to it and improve code quality | 16:21 |
| nati2 | wwkeyboard: agreed | 16:21 |
| nati2 | dwalleck: gotcha! thanks for your point. It make clear of the mission of this team :) | 16:22 |
| mtaylor | gigig: https://jenkins.openstack.org/view/Nova/job/nova-coverage/? | 16:22 |
| mtaylor | #link https://jenkins.openstack.org/view/Nova/job/nova-coverage/? | 16:22 |
| jaypipes | I would agree with above... checking the consistency of the spec vs. the actual behaviour is certainly in QA's purview | 16:22 |
| gigig | mtaylor: thanks :) | 16:22 |
| mtaylor | #info coverage is tracked per-build in jenkins | 16:22 |
| wwkeyboard | When filing these bugs are you making sure to inspect the supporting tests as well? The missing coverage may signal other insufficient tests. | 16:22 |
| mtaylor | ooh! why is the nova coverage job failing... | 16:23 |
| nati2 | There are some coding policies (pep8 etc), however I think coding policies should support irregular cases such as exceptions or irregular input value. | 16:23 |
| nati2 | mtaylor: tanks | 16:23 |
| *** rnirmal has quit IRC | 16:23 | |
| nati2 | wwkeyboard: Yes absolutly. Coverage is also important. | 16:23 |
| wwkeyboard | nati2: good, thank you | 16:24 |
| nati2 | see we already added for that. https://blueprints.launchpad.net/openstack-qa/+spec/nova-unit-compute | 16:24 |
| *** darraghb has quit IRC | 16:24 | |
| nati2 | Low coverage is one of kind of bugs | 16:24 |
| KumarKR | Agree, we need to verify a) are we building the right product and b) is the product built right? Wondering, if we have product owners to particpate in checking consistency of specs. | 16:24 |
| nati2 | KumarKR++ | 16:24 |
| KumarKR | :-) | 16:25 |
| nati2 | At first, I think we should share what's our thought | 16:25 |
| nati2 | So would you please share your teams plan as a blueprints and bugs | 16:25 |
| nati2 | Blueprint is spec such as exception policies | 16:25 |
| nati2 | BugReport is instance of it. | 16:26 |
| jaypipes | KumarKR: well, first things first, we need to identify places where the spec doesn't even exist ;) | 16:26 |
| gigig | jaypipes: applause | 16:26 |
| nati2 | jaypipes++ | 16:26 |
| KumarKR | agree. i need to dive into this sooner :-) | 16:27 |
| patelna | totally agree...we need dev to help us as well | 16:27 |
| patelna | we should start with API coverage first | 16:27 |
| jaypipes | a good example of that is the Glance "spec" for the 1.0 API. It, well, leaves a lot to be desired... | 16:27 |
| wwkeyboard | What are we considering the 'spec'? The api spec? | 16:27 |
| *** ColinLyons has quit IRC | 16:27 | |
| wwkeyboard | Or the unit tests? | 16:27 |
| jaypipes | wwkeyboard: yes. anything here: http://docs.openstack.org/api/ | 16:28 |
| gigig | jaypipes: can we talk about the api framework we will consolidate to since we have 7? | 16:28 |
| nati2 | gigig: 7 ? | 16:28 |
| jaypipes | wwkeyboard: so, there are "approved" specs, like the 1.1 Compute API spec, "in progress" specs (like the Glance v1 API), and "proposed" specs, like the 2.0 Images API spec being developed. | 16:29 |
| gigig | jaypipes: there are a few out there nachi :) | 16:29 |
| westmaas | nati2: think she's talking about the tests in openstack-integration-tests | 16:29 |
| wwkeyboard | jaypipes: OK, do we want to try and include executable examples within those specs? | 16:29 |
| jaypipes | gigig: you're talking about the 7 *test* frameworks :) | 16:29 |
| nati2 | westmaas: gotcha | 16:29 |
| patelna | I think there are 2 separate tasks we need to start with (a) API full coverage/add Test Cases for gaps (b) Agree on API framework | 16:29 |
| gigig | jaypipes: Yes | 16:29 |
| gigig | westmaas: ty gabe | 16:29 |
| nati2 | patelna: agreed | 16:29 |
| jaypipes | gigig: yep, we'll get to that in a sec | 16:29 |
| gigig | jaypipes: cool | 16:29 |
| jaypipes | first things first though, I'd like to get some resolution on the following: | 16:30 |
| venkater_ | idea: we can have blueprint review session . Either by email notification or phone call | 16:30 |
| jaypipes | If nati2's team is doing these traceability matrices, is nati2's team also responsible for submitting bug reports from those and developing new tests cases or improving poor-=quality test cases? | 16:30 |
| nati2 | Oh, I wanna make sure this. My team's primary focus is Diablo maintenance (backport). How about yours? | 16:30 |
| jaypipes | nati2: and Nova-only for your team... we should be specific. | 16:31 |
| nati2 | jaypipes: Thanks yes | 16:31 |
| jaypipes | alright, then I propose the following: | 16:31 |
| jaypipes | #vote nati2 and team continue to do the unit test analysis, submitting of bug reports and additional unit tests. | 16:32 |
| nati2 | jaypipes: Yes our team is responsible to add unit test (Sorry It may not all of modules) | 16:32 |
| jaypipes | #vote nati2 and team continue to do the unit test analysis, submitting of bug reports and additional unit tests for NOVA | 16:32 |
| jaypipes | type #agreed if that is good with you. | 16:32 |
| gigig | #agreed | 16:32 |
| westmaas | #agreed | 16:32 |
| nati2 | #agreed | 16:32 |
| dwalleck | #agreed | 16:32 |
| wwkeyboard | #agreed | 16:32 |
| patelna | We should help Nachi | 16:32 |
| patelna | he is the owner...but this is a big task | 16:33 |
| venkater_ | Yes. | 16:33 |
| jaypipes | patelna: ok. can I count on you to arrange resources from HP where nati2 needs assistance? | 16:33 |
| patelna | Yes | 16:33 |
| nati2 | Thanks. We may not support all modules (such as LDAP or something) | 16:33 |
| jaypipes | patelna: or perhaps around Glance and Keystone, since nati2's team is focued on Nova? | 16:33 |
| jaypipes | more help the better IMHO | 16:33 |
| nati2 | It sounds great | 16:33 |
| gigig | jaypipes: we can help here too | 16:34 |
| patelna | Ravi - can you take Glance | 16:34 |
| nati2 | Or Middium or large tests | 16:34 |
| patelna | Keystone - we can ask Kim to lead this | 16:34 |
| venkater_ | Yes . Nayna | 16:34 |
| jaypipes | k... | 16:34 |
| jaypipes | #action patelna to find resources to help small/unit test analysis and coverage for Keystone and Glance | 16:34 |
| jaypipes | #action jaypipes to assist patelna in getting resources familiar with Gerrit/Git reviewing and Launchpad | 16:35 |
| jaypipes | ok. now, I think we can move on to discuss integration tests. OK with everyone? | 16:35 |
| *** reed_ is now known as reed | 16:35 | |
| nati2 | Cool Jay+ | 16:35 |
| patelna | #agree | 16:35 |
| nati2 | #agreed | 16:35 |
| jaypipes | OK | 16:35 |
| gigig | patelna: let me know if you need help on the last item | 16:35 |
| gigig | and yes lets move on | 16:35 |
| patelna | Yes ...will do | 16:35 |
| jaypipes | #topic Action items around unified integration test suite | 16:35 |
| *** openstack changes topic to "Action items around unified integration test suite" | 16:35 | |
| KumarKR | #agree | 16:35 |
| westmaas | wwkeyboard and I would like to propose a unified way to run the tests in openstack-integration-tests, I think we can get a bp in by tomorrow if that is acceptable. | 16:36 |
| *** dprince has joined #openstack-meeting | 16:36 | |
| jaypipes | So, right now, the only test suite currently included in https://github.com/openstack/openstack-integration-tests is Kong | 16:36 |
| nati2 | Rohit is starting about analize 7 test frameworks. (Sorry he can not come here today) | 16:36 |
| jaypipes | westmaas: yep, getting to that... one sec. | 16:36 |
| * westmaas shushes | 16:36 | |
| gigig | haha | 16:37 |
| nati2 | It sounds great to have a blueprint for that | 16:37 |
| jaypipes | westmaas and I have been compiling info on the various suites: http://wiki.openstack.org/openstack-integration-test-suites | 16:37 |
| nati2 | westmaas++ jaypipes++ | 16:37 |
| jaypipes | clearly we have a lot of work ahead of us in bringing the "good stuff" from the non-Kong test suites into the main project | 16:37 |
| dwalleck | Is there a plan to come up with a universal design and architecture for testing? Or just to merge the tests together so they can run? | 16:37 |
| venkater_ | Is there a notification if new blueprint is added to the docs/wiki? | 16:38 |
| wwkeyboard | dwalleck: We hope to come up with one. | 16:38 |
| patelna | we need to decide on one framework | 16:38 |
| jaypipes | westmaas: I'd like to propose that we have 1 person take the lead on developing the "unified test runner" and 1+ people taking the lead for each non-Kong test suite to bring it into the main project | 16:38 |
| mtaylor | dwalleck: I would like universal design/architecture | 16:38 |
| nati2 | I think test scenario is also important | 16:38 |
| westmaas | dwalleck: a plan for a plan :) think we need a tiny bit more analysis first. | 16:38 |
| gigig | patelna: ++ | 16:38 |
| westmaas | jaypipes: sounds good. | 16:38 |
| jaypipes | venkater_: no, and that is really annoyinhgk I know. | 16:38 |
| dwalleck | But before we start going forward with testing? It seems like coming with up with a design before going forward with development would be best | 16:38 |
| wwkeyboard | jaypipes: sounds good | 16:38 |
| wwkeyboard | I would add to that that we have someone in charge of looking for duplication & missing tests. | 16:39 |
| jaypipes | OK, now, is there anyone that would like to volunteer to be the LEAD for the unified test *runner*? | 16:39 |
| mtaylor | jaypipes: as long as when we say "test runner" we don't mean "re-write nose" | 16:39 |
| dwalleck | I just want to make sure whatever we built is maintainable and scalable | 16:39 |
| westmaas | I'm happy to lead that, if someone else wants it, no worries. | 16:39 |
| dwalleck | Is there a reason we can't just use nose? | 16:39 |
| jaypipes | mtaylor: no, we mean "take the best pieces of the existing things and make it a single way to run tests" | 16:39 |
| dwalleck | I would be also | 16:39 |
| mtaylor | jaypipes: just being clear | 16:39 |
| nati2 | westmaas++ Rohit will help you :) | 16:40 |
| jaypipes | dwalleck: the big one is it is not multi-threaded/processed | 16:40 |
| wwkeyboard | +1 for westmaas | 16:40 |
| jaypipes | dwalleck: whereas DTest is. | 16:40 |
| patelna | +1 for westmass | 16:40 |
| dwalleck | jaypipes: Actually there is a multithreaded plugin for nose | 16:40 |
| jaypipes | dwalleck: hmm, great to know. :) Perhaps you can work with westmaas to come up with the best runner? | 16:41 |
| westmaas | dwalleck: I will work closely with you | 16:41 |
| dwalleck | Sure, sounds good | 16:41 |
| jaypipes | westmaas: seems like you are the lead for the test runner piece. | 16:41 |
| westmaas | alright | 16:41 |
| jaypipes | #action westmaas will lead the effort to create a singular best-practice test runner | 16:42 |
| nati2 | cool | 16:42 |
| gigig | shall we put a timeline on this since we need to get consolidated sooner than later? | 16:43 |
| patelna | awesome!!!! | 16:43 |
| jaypipes | Alright, now I believe that one of the first things we need to do (other than start work on the test runner) is to actually GET the other test suites into the openstack-integrated-tests project... | 16:43 |
| westmaas | will ping the list with a bp that specifies requirements to look for input on those requirements within a day. | 16:43 |
| gigig | westmaas: great | 16:43 |
| jaypipes | I can volunteer to do that drudge work... | 16:43 |
| patelna | I add ask Donald Ngo from my team to be included in this group as well | 16:43 |
| wwkeyboard | jaypipes: the importing of the other tests? | 16:43 |
| jaypipes | drudge work == copy-pasting the code from Backfire/Stacktester/Zodiac/Novasmoketests/etc into the openstack-integrated-tests project | 16:43 |
| jaypipes | wwkeyboard: yep | 16:43 |
| gigig | jaypipes: applause | 16:44 |
| wwkeyboard | if you do that I'll start looking for duplicates & missing tests | 16:44 |
| jaypipes | HOWEVER... doing so will be pointless if work continues in those other projects... | 16:44 |
| dwalleck | But all these tests are so different in design. How is that going to work? | 16:44 |
| wwkeyboard | westmaas suggested we really need to talk about directory structure as well. | 16:44 |
| jaypipes | dwalleck: some are different, but MANY are virtually identical :) | 16:44 |
| wwkeyboard | And I think we need to decide on a client(s) to access OS with | 16:44 |
| westmaas | wwkeyboard: I think rohit is also analyzing and looking for dupes | 16:45 |
| *** reed has quit IRC | 16:45 | |
| patelna | the dir structure should follow the same code path/release/versioning | 16:45 |
| dwalleck | Well, it's more than directory structure. Some use novaclient, some directly call httplib, some have intermediate interfaces | 16:45 |
| wwkeyboard | westmaas: I will ping him | 16:45 |
| jaypipes | wwkeyboard: the directory structure I proposed at the summit unconference was a directory named for each of the original test suites, so we can just bring them all into the project, then start the process of merging them into a /tests directory and a /runner directory, | 16:45 |
| dwalleck | We really need to think about sustainability when doing this | 16:45 |
| nati2 | wwkeyboard: yes rohit will help | 16:45 |
| jaypipes | patelna: could you elaborate on that suggestion? | 16:45 |
| nati2 | dwalleck: sustainability? | 16:46 |
| jaypipes | dwalleck: we agreed at the summit that there is value in running BOTH httlib2 AND novaclient-based client tests | 16:46 |
| patelna | so for diablo unit tests it should be checked in the same branch | 16:46 |
| dwalleck | If we have several different architectures within the same suite, is will be painful to maintain | 16:46 |
| jaypipes | patelna: ah, I see now | 16:46 |
| gigig | agree with Patelna | 16:46 |
| nati2 | dwalleck: I got it | 16:47 |
| dwalleck | But novaclient tests should really be a different suite or at least sub-part, but seperate from other tests | 16:47 |
| patelna | for essex we should have a place holder for test dir structure... | 16:47 |
| wwkeyboard | dwalleck: what do you mean by architectures? | 16:47 |
| westmaas | dwalleck: agreed. at the conference we agreed there is value to each approach we just need a sane way to approach them and organize them. | 16:47 |
| westmaas | dwalleck: I'm slow, what you just said. | 16:47 |
| dwalleck | If we continue just calling httplib2 directly from tests, that won't scale like we need it to | 16:47 |
| gigig | all i have to step away for a phone screen - *waves* | 16:48 |
| dwalleck | wwkeyboard: As in a design for the framework. For example, I made an intermediate layer between requests and the tests so that if the structure of requests changes, there's easy places to fix issues like that | 16:48 |
| wwkeyboard | My fear with using a single client like novaclient is that a bug in the spec that is written into the API can be duplicated in the client. | 16:48 |
| *** gigig has left #openstack-meeting | 16:48 | |
| patelna | yes...even architecture framework + tests needs to be check'd into Git rightly | 16:48 |
| wwkeyboard | dwalleck: thats what I was calling the client, I agree with you | 16:48 |
| jaypipes | Hey all, what do you all think of this? http://paste.openstack.org/show/2689/ | 16:49 |
| jaypipes | where the top directory structure is what we do initially, just to get stuff in there, and the bottom directory structure is the endgoal... | 16:49 |
| nati2 | Ah,, looks good to me. Test must support multiple clients | 16:49 |
| wwkeyboard | I would rather the version be at the top level, it would make it easier to remove when we depreciate something. | 16:49 |
| nati2 | So there are client directory | 16:49 |
| jaypipes | wwkeyboard: sure, that makes sense too. | 16:50 |
| patelna | that is a gd idea | 16:50 |
| nati2 | yes it is good idea | 16:50 |
| jaypipes | patelna: version at top level instead of under component? | 16:50 |
| jaypipes | westmaas: thoughts? | 16:50 |
| jaypipes | dwalleck: thoughts? | 16:50 |
| patelna | yes...at top level | 16:51 |
| venkater_ | it looks good . client -> type of client. How about version? when we support multiple version? | 16:51 |
| wwkeyboard | Then when we move versions all we have to do is 'cp' the old version' | 16:51 |
| wwkeyboard | and a diff will tell you about what has changed. | 16:51 |
| patelna | yes... | 16:51 |
| westmaas | jaypipes: lets start there, and then see where we go. | 16:51 |
| wwkeyboard | But that might be to primitive, idk | 16:51 |
| nati2 | :wwkeyboard it sounds cool | 16:51 |
| dwalleck | Well, my only concern is that I still see it broken down by original test project. Would it make more sense to break it down by something like novaclient tests and non-novaclient tests? | 16:51 |
| westmaas | dwalleck: that's the before | 16:52 |
| jaypipes | venkater_: diffferenltly versioned clients could just be different python modules under /client/httplib2/ ... | 16:52 |
| dwalleck | Ahh, I see now. Nevermind! :-) | 16:52 |
| westmaas | we will get rid of those as they get merged in | 16:52 |
| jaypipes | dwalleck: :) | 16:52 |
| jaypipes | well, as was mentioned, we can always change it later... | 16:52 |
| jaypipes | alrighty, then.. | 16:52 |
| *** ohnoimdead has joined #openstack-meeting | 16:52 | |
| jaypipes | #action jaypipes to grab all other test suites code and put into openstack-integrated-tests project, each in its own subdirectory. | 16:53 |
| nati2 | jaypipes++ | 16:53 |
| *** littleidea has quit IRC | 16:53 | |
| patelna | jaypipes ++ | 16:53 |
| jaypipes | OK, those were the two big status things I wanted to chat about and get agreement on. Does anyone have any feedback from the summit or issues you'd like to discuss? | 16:54 |
| westmaas | jaypipes: stacktester is already merged in with kong in the repo, just fyi | 16:54 |
| jaypipes | westmaas: ah, good to know. thx! | 16:54 |
| jaypipes | #topic Open Discussion | 16:54 |
| *** openstack changes topic to "Open Discussion" | 16:54 | |
| nati2 | I wanna discuss about diablo branches | 16:54 |
| patelna | me 2 | 16:54 |
| nati2 | Which branch I should request merge? | 16:54 |
| jaypipes | nati2: we need to create them first :) | 16:54 |
| nati2 | And also I know about status of fixes team | 16:55 |
| jaypipes | nati2: I'll send an email to the PTLs about it. | 16:55 |
| KenWhite-RAX | Sorry, I'm a little slow here, but basically all the different frameworks ( kong, backfire, etc) get checked in to one location then merged. THEN duplicate tests are removed and then we check code coverage? | 16:55 |
| wwkeyboard | KenWhite-RAX: The duplicates should be removed as they are merged. | 16:55 |
| jaypipes | KenWhite-RAX: code coverage is more unit tests... these are functional integration tests... | 16:55 |
| patelna | Daviey -- what is his role for the maintenace branch? | 16:55 |
| nati2 | jaypipes: Cool. However I suppose it is responsibility of fixes team. | 16:55 |
| jaypipes | nati2: well, that is true enough. | 16:56 |
| jaypipes | nati2: do we have that fixes team even created on Launchpad yet? | 16:56 |
| KenWhite-RAX | Right right sorry, mixing my apples and oranges | 16:56 |
| nati2 | Muu I think fixes and qa team must be merged | 16:56 |
| dwalleck | Yeah, I was curious about that. So why do we call it an integration suite instead of a functional suite? Just me being picky :) | 16:56 |
| nati2 | mtaylor: How do you think about this? | 16:56 |
| jaypipes | nati2: no, I thought a decision was made that QA and fixes team are separate | 16:57 |
| nati2 | jaypipes: gotcha | 16:57 |
| nati2 | jaypipes: Then fixes team should have their launchpad | 16:57 |
| jaypipes | dwalleck: because it tests functional components of more components than just Nova... tests the integration of Keystone, Glance, Nova, etc | 16:57 |
| mtaylor | yes. jeblair is working on the fixes team right now | 16:57 |
| jaypipes | dwalleck: but the distinction is a grey area to be sure :) | 16:57 |
| venkater_ | Is Daviey in fixes team and maintains brach Diablo? | 16:58 |
| nati2 | mtaylor: cool! Fixes have meetings? | 16:58 |
| jaypipes | mtaylor: I believe it was decided that the fixes team would be just a few people... basically some interested parties from the distros, the PTLs who vote on which bug fixes/backports to apply, and a couple others? | 16:58 |
| mtaylor | nati2: not yet | 16:58 |
| mtaylor | jaypipes: yes. I actually think we were going to add Daviey, markmc and zul and let them take adding more people from there | 16:59 |
| patelna | who will review the code for the fixes team ...so we don't have regressions? | 16:59 |
| jaypipes | patelna: good question... | 17:00 |
| jaypipes | patelna: it may depend on the project. | 17:00 |
| jaypipes | patelna: for Glance, I think a few interested glance-core contribs will do reviews at least. | 17:00 |
| patelna | we should formalize this...maybe the previous PL? | 17:00 |
| zul | markmc already started a write up | 17:00 |
| patelna | thanks ...Zul | 17:00 |
| jaypipes | patelna: well, i think it would be up to the project ;) | 17:01 |
| mtaylor | actually, no | 17:01 |
| nati2 | Hi did you read mail about [Openstack] [RFC] Stable branch? It sounds several guys do the same thing at different place. | 17:01 |
| mtaylor | what we talked about at ODS is that these teams would not really be as associated with the projects, as the projects are focused on forward dev | 17:01 |
| patelna | we should involve the core developer | 17:01 |
| mtaylor | markmc just sent a proposed policy ... but the thought was that this was a place for distros and integrators to collect their work and collaborate | 17:02 |
| *** dprince has quit IRC | 17:02 | |
| mtaylor | one of the reasons for the formation of the team was exactly that the core devs/ptls did _not_ want to maintain old releases moving forward, whereas the distros and integrators do | 17:02 |
| westmaas | mtaylor: who has core status on openstack-integration-tests? | 17:02 |
| nati2 | mtaylor: Aha This is from your team. I got it. | 17:02 |
| westmaas | sorry if this is what is being discussed having a hard time following | 17:02 |
| mtaylor | westmaas: it's not - we moved on to stable branch update process it seems | 17:03 |
| mtaylor | westmaas: I'm not sure about who is core on openstack-integration-tests - lemme check | 17:03 |
| mtaylor | jeblair: ^^ | 17:04 |
| mtaylor | jeblair: "<westmaas> mtaylor: who has core status on openstack-integration-tests?" | 17:05 |
| jaypipes | OK, well perhaps we'll shelf that... :) | 17:07 |
| westmaas | haha | 17:07 |
| westmaas | apparently | 17:07 |
| jaypipes | #action jeblair to email ML about membership for integration tests | 17:07 |
| jeblair | hi | 17:08 |
| jaypipes | alrighty, anything else ayone wants to bring up? | 17:08 |
| jeblair | any chance we colud get this meeting listed on the openstack meetings ics calendar? :) | 17:08 |
| jaypipes | #action jaypipes to get this meeting listed on the openstack meetings ics calendar? :) | 17:08 |
| patelna | that is a gd ask | 17:08 |
| nati2 | It is useful | 17:09 |
| jaypipes | OK, I'll adjourn the meeting for this week, then. See you all on the mailing list and next week on IRC ;) | 17:09 |
| jaypipes | #endmeeting | 17:09 |
| *** openstack changes topic to "Openstack Meetings: http://wiki.openstack.org/Meetings | Minutes: http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/" | 17:09 | |
| openstack | Meeting ended Wed Oct 12 17:09:54 2011 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:09 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-10-12-16.05.html | 17:09 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-10-12-16.05.txt | 17:09 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-10-12-16.05.log.html | 17:09 |
| nati2 | Thanks Jay! | 17:10 |
| KumarKR | Thanks everyone! | 17:10 |
| westmaas | thx | 17:10 |
| patelna | Thanks | 17:10 |
| venkater_ | Thanks . Bye | 17:10 |
| *** venkater_ has quit IRC | 17:11 | |
| *** dwalleck has left #openstack-meeting | 17:11 | |
| KumarKR | fyi, i will be OOO for next two weeks and back in Nov 1st week. | 17:11 |
| KumarKR | touch base with you all then. Bye | 17:11 |
| nati2 | Enjoy OOO :) | 17:11 |
| *** patelna has quit IRC | 17:11 | |
| KumarKR | thanks. | 17:11 |
| *** KumarKR has left #openstack-meeting | 17:12 | |
| *** novas0x2a|laptop has joined #openstack-meeting | 17:13 | |
| *** mattray has quit IRC | 17:13 | |
| *** KenWhite-RAX has quit IRC | 17:14 | |
| *** KenWhite-RAX has joined #openstack-meeting | 17:16 | |
| *** mattray has joined #openstack-meeting | 17:19 | |
| *** patelna_ has quit IRC | 17:21 | |
| *** littleidea has joined #openstack-meeting | 17:27 | |
| *** reed has joined #openstack-meeting | 17:28 | |
| *** deshantm_laptop has quit IRC | 17:29 | |
| *** deshantm_laptop has joined #openstack-meeting | 17:30 | |
| *** mahmoh has left #openstack-meeting | 17:30 | |
| *** novas0x2a|laptop has quit IRC | 17:42 | |
| *** Gordonz has quit IRC | 17:43 | |
| *** novas0x2a|laptop has joined #openstack-meeting | 17:43 | |
| *** bengrue has quit IRC | 17:45 | |
| *** Gordonz has joined #openstack-meeting | 17:54 | |
| *** bengrue has joined #openstack-meeting | 17:55 | |
| *** markmc has left #openstack-meeting | 18:11 | |
| *** medberry is now known as med_out | 18:13 | |
| *** joesavak has joined #openstack-meeting | 18:14 | |
| *** jsavak has quit IRC | 18:16 | |
| *** jakedahn has quit IRC | 18:29 | |
| *** jaypipes has quit IRC | 18:31 | |
| *** deshantm_laptop has quit IRC | 18:31 | |
| *** joesavak has quit IRC | 18:41 | |
| *** joesavak has joined #openstack-meeting | 18:41 | |
| *** hggdh has quit IRC | 18:43 | |
| *** hggdh has joined #openstack-meeting | 18:45 | |
| *** littleidea has quit IRC | 18:47 | |
| *** rnirmal has joined #openstack-meeting | 18:55 | |
| *** dwalleck has joined #openstack-meeting | 18:58 | |
| *** reed has quit IRC | 19:00 | |
| *** dwalleck has quit IRC | 19:03 | |
| *** mdomsch has quit IRC | 19:05 | |
| *** dwalleck has joined #openstack-meeting | 19:06 | |
| *** dendro-afk is now known as dendrobates | 19:19 | |
| *** dwalleck has quit IRC | 19:30 | |
| *** littleidea has joined #openstack-meeting | 19:42 | |
| *** med_out is now known as medberry | 20:02 | |
| *** bcwaldon has left #openstack-meeting | 20:06 | |
| *** HowardRoark has joined #openstack-meeting | 20:18 | |
| *** primeministerp1 has joined #openstack-meeting | 20:31 | |
| *** reed_ has joined #openstack-meeting | 20:50 | |
| *** markvoelker has quit IRC | 20:53 | |
| *** reed_ is now known as reed | 20:56 | |
| *** zns has quit IRC | 20:57 | |
| *** zns has joined #openstack-meeting | 21:06 | |
| *** martine has quit IRC | 21:11 | |
| *** zns has quit IRC | 21:18 | |
| *** zns has joined #openstack-meeting | 21:18 | |
| *** joesavak has quit IRC | 21:22 | |
| *** vladimir3p has quit IRC | 21:25 | |
| *** primeministerp1 has quit IRC | 21:30 | |
| *** dendrobates is now known as dendro-afk | 21:32 | |
| *** zns1 has joined #openstack-meeting | 21:50 | |
| *** zns has quit IRC | 21:51 | |
| *** dwalleck has joined #openstack-meeting | 21:58 | |
| *** vladimir3p has joined #openstack-meeting | 21:59 | |
| *** dwalleck has quit IRC | 22:00 | |
| *** vladimir3p_ has joined #openstack-meeting | 22:05 | |
| *** vladimir3p has quit IRC | 22:08 | |
| *** dendro-afk is now known as dendrobates | 22:15 | |
| *** donaldngo_hp has joined #openstack-meeting | 22:20 | |
| *** donaldngo_hp has quit IRC | 22:22 | |
| *** donaldngo_hp has joined #openstack-meeting | 22:22 | |
| *** ohnoimdead has quit IRC | 22:28 | |
| *** littleidea has quit IRC | 22:33 | |
| *** zns1 has quit IRC | 22:34 | |
| *** HowardRoark has quit IRC | 22:45 | |
| *** mattray has quit IRC | 22:47 | |
| *** dendrobates is now known as dendro-afk | 22:48 | |
| *** dendro-afk is now known as dendrobates | 22:51 | |
| *** blamar has quit IRC | 22:55 | |
| *** littleidea has joined #openstack-meeting | 23:07 | |
| *** medberry is now known as med_out | 23:16 | |
| *** HowardRoark has joined #openstack-meeting | 23:22 | |
| *** rnirmal has quit IRC | 23:23 | |
| *** dwalleck has joined #openstack-meeting | 23:30 | |
| *** jaypipes has joined #openstack-meeting | 23:40 | |
| *** wwkeyboard has left #openstack-meeting | 23:41 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!