Thursday, 2015-07-02

*** zz_dimtruck is now known as dimtruck00:16
murphyslawbbshey guys00:51
murphyslawbbsso i've been dumped into the testing world recently, given the task of finding out how to do tests against one of our apis. Should i reuse tempest?00:52
*** jasonsb has quit IRC01:31
*** dimtruck is now known as zz_dimtruck01:51
*** zz_dimtruck is now known as dimtruck03:11
*** jasonsb has joined #openstack-tailgate04:00
*** dimtruck is now known as zz_dimtruck06:18
*** mwagner_lap has quit IRC11:13
gemamurphyslawbbs: it depends on the API, I guess11:22
gemaand what kind of testing you are after11:22
*** mwagner_lap has joined #openstack-tailgate11:53
murphyslawbbsthe api is a suspend/resume API for Juno12:05
murphyslawbbsgema: right now we just want a proof of concept so that we have a stencil that we can use to create more tests12:05
murphyslawbbsand tests for more apis12:06
murphyslawbbsi was thinking of using something like tempestlib or testr12:06
gemayou could use that, I believe, but I am not sure the tempest team would take the tests on12:08
gemaif you wanted to contribute them12:08
gemaif that is not a problem for you, then tempest should be ok12:08
murphyslawbbsgema: but tempest would require an existing running openstack right?12:10
murphyslawbbswith a ketstone url, etc12:10
gemamurphyslawbbs: yep12:10
murphyslawbbsok thx12:10
gemamurphyslawbbs: how many components are involved in the suspend/resume?12:10
gemamurphyslawbbs: if I were you I'd make sure I can run tempest on my test environment before moving on with the implementation12:12
gemathe right version of tempest, that is12:12
gemaI believe if you do git checkout 312:12
gemayou'd have juno12:12
gema(on the tempest git cloned repo)12:14
*** malini has joined #openstack-tailgate12:59
murphyslawbbsgema: there are two components, the api calls on a component called PSC to perform the actual commands13:03
murphyslawbbsthanks for the checkout tip13:03
murphyslawbbsso what time is the meeting, you said 17:00, is that GMT?13:03
*** malini has quit IRC13:04
gemamurphyslawbbs: UTC13:06
gemamurphyslawbbs: I hope that is the same time as before13:06
gemaIdid the calculation a bit roughly13:06
gemamurphyslawbbs: can you change the invitation to be weekly and at the same time and read as happening on IRC?13:08
gemamurphyslawbbs: that invite was quite handy :D13:08
murphyslawbbshehe ok will do13:08
gemathx!13:08
*** malini has joined #openstack-tailgate13:15
*** malini has quit IRC13:15
*** malini has joined #openstack-tailgate13:16
*** malini has quit IRC13:46
*** malini has joined #openstack-tailgate13:48
*** xwwt has joined #openstack-tailgate13:54
*** localloop127 has joined #openstack-tailgate14:02
*** zz_dimtruck is now known as dimtruck14:04
*** hogepodge has quit IRC14:28
*** malini1 has joined #openstack-tailgate14:43
*** malini has quit IRC14:43
*** hogepodge has joined #openstack-tailgate15:01
*** malini1 has quit IRC15:05
*** malini has joined #openstack-tailgate15:06
*** malini has quit IRC15:20
*** malini has joined #openstack-tailgate15:24
*** murphyslawbbs has quit IRC15:25
*** jasonsb has quit IRC15:29
*** jasonsb has joined #openstack-tailgate15:30
*** clee has joined #openstack-tailgate15:32
*** jasonsb has quit IRC15:34
maliniI might miss today's meeting15:44
*** sabeen has quit IRC15:47
*** malini has quit IRC15:49
*** mwagner_lap has quit IRC15:51
*** jasonsb has joined #openstack-tailgate16:26
*** sabeen has joined #openstack-tailgate16:28
*** ChanServ sets mode: +o localloop12716:47
gemahey folks16:52
gemadon't worry malini16:52
gemauhmm, she's gone :)16:52
*** sabeen has quit IRC16:56
jasonsbhi gema16:58
gema#startmeeting Tailgate17:00
openstackMeeting started Thu Jul  2 17:00:04 2015 UTC and is due to finish in 60 minutes.  The chair is gema. Information about MeetBot at http://wiki.debian.org/MeetBot.17:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:00
openstackThe meeting name has been set to 'tailgate'17:00
gemahi everyone, can anyone here for today's meeting say something, plz :)17:00
gemao/17:00
jasonsbmorning gema17:01
gemaevening :D17:01
jasonsbyour a trooper17:01
gemaexciting times :D17:01
jasonsbthank you for setting up the meeting bot17:01
gemayou are welcome, it was fun17:01
gemashall we wait a couple of mins to give people time to show up if they are going to?17:02
*** sabeen1 has joined #openstack-tailgate17:02
jasonsbcertainly.  i was going to ask about hogepodge17:02
gemahe said he'd be here17:02
jasonsbi didn't fully grasp your conversation from few days back17:03
hogepodgehi17:03
hogepodgeo/17:03
gemahi hogepodge17:03
gemaclee: you around?17:03
jasonsbhogepodge: greetings17:03
localloop127i'm here. was not paying attention when things started17:03
gemagoodio, let's get started then17:03
gemaI have a sort of agenda17:04
gema#topic Progress on last week's actions17:04
gemaI did the ones assigned to me, bot, sending email to the mailing about the meeting and adding kloudbuster to the test doc17:04
gema#link test strategy doc: https://docs.google.com/document/d/19N4lZJ5mYkQbkXrtBGntf7C1bWvwt1RKoSU8NerAphQ/edit#17:04
gemaspyderdyne is not around17:05
gemaso we'll have to postopone asking him about rally17:05
cleegema: kind of!17:05
gemaclee: go ahead :D17:05
cleegema: I'm in a dreamhost meeting17:05
gemaah, ok, so half here17:05
cleegema: I emailed dkranz but have yet to receive a reply17:05
gemalet's start with the defcore topic next then17:05
gemaclee: we'll give you time to wrap up17:06
gema#topic DefCore17:06
cleegema: the DH meeting will probably last longer than this one, but I'll try to pay attention here when I can17:06
*** sabeen3 has joined #openstack-tailgate17:06
gemaok17:06
gemaI had a conversation with hogepodge the other day and I will let him introduce himself and the defcore project17:06
gemahogepodge: thanks for coming17:06
*** sabeen1 has quit IRC17:07
hogepodgeHi,17:07
hogepodgeDefcore is a board-approved process that requires OpenStack clouds to pass a series of API tests as part of our trademark program.17:07
hogepodgeIt originally came out of clause in the bylaws that stated that OpenStack branded products needed to pass a "faithful implementation of testing standards" (or some name like that), FITS17:08
hogepodgeWe starting testing clouds against a test suite based on tempest a few months ago. They are non-admin API calls that check basic functionality of Nova, Swift, Cinder, and Keystone (and implicitly Glance and Neutron)17:09
hogepodgeWe use a tool called Refstack to semi-anonymously collect test results (companies can voluntarily self-identify their results publicly for the Defcore process)17:10
hogepodgeThe goal of defcore is to promote interoperability across public, hosted, and distributions.17:11
gemahogepodge: could you define interoperability in this context, what do you mean by that?17:11
hogepodgeInteroperability would be a stable set of APIs that work consistently across clouds. So an application written for one cloud can work on a second cloud.17:12
gemaok17:12
gemaand you also aim at keeping backwards compatibility in the future?17:12
gemaor is that out of scope?17:12
hogepodgeWe still have a ways to go to get true interoperability, but the recently approved process is a good start, and our aim is to spend the next year really nailing down APIs and holding vendor's and devs accountable to making clouds work together.17:12
hogepodgeWe try to have a lagging process, and are looking for APIs that will be around for a while or have a clear deprecation strategy17:13
gemaack17:13
hogepodgeWe also allow validation against the latest two approved standards to give vendors wiggle room.17:13
gemajasonsb: do you still have questions?17:15
hogepodgeWe meet weekly for general committee work, and also are starting up a new capabilities meeting to introduce new tests, with more attention on this cycle being given to images store and networking17:15
gemahogepodge: sound like the capabilities meeting is a good one for us to attend also?17:15
hogepodgeWe would love for as many members of this group to participate as possible. Defcore is meant to be a community driven process, and the tailgate team seems to have a number of overlapping concerns. The advantage to participating is you get the backing of the board and foundation in the approved test suite, and everyone who sells an OpenStack branded cloud or distro needs to meet that standard.17:16
gemabased on our conversation the other day, you said that tempest will keep maintaining any tests that defcore uses for certification, correct?17:17
gema(instead of devolving them to the different projects)17:17
jasonsbgema: good question17:18
hogepodgegema yes, that's what the ptl of the qa team wants.17:18
hogepodgegema we do have a procedure for introducing different test suites if necessary, though17:18
gemahogepodge: do you have a link to that?17:18
hogepodgegema in general, every document is located here: https://git.openstack.org/cgit/openstack/defcore/tree/17:19
gemaack17:19
gema#link https://git.openstack.org/cgit/openstack/defcore/tree/17:19
gema#link http://eavesdrop.openstack.org/#DefCore_Committee_Meeting17:19
hogepodgeOur new schema includes a field for test repositories. https://git.openstack.org/cgit/openstack/defcore/tree/2015.next.json#n4317:20
gemahogepodge: sounds like your test suites are the right place for a good number of the test cases we were aiming to be adding17:20
jasonsbhogepodge: can you speak to the relationship of your testing vs big tent?17:20
jasonsblooks like a take-it-slow approach, but i was wondering what the overall strategy is17:21
hogepodgejasonsb: We're focusing on what would be controversially called the "core" right now. Compute, block storage, object storage, image repository, and networking.17:21
hogepodgejasonsb: Nothing would preclude new OpenStack Powered programs from branching out to other projects. For example, a hypothetical "OpenStack Powered Database" program might have tests for Trove.17:22
hogepodgejasonsb: but all changes to the projects in the program must be committee proposed and board approved17:22
jasonsbok, the core has a good home in tempest currently17:22
hogepodgejasonsb: we want to be tied to tempest only for simplicity, but also are open to other test suites if we can't meet our goals with tempest only.17:23
gemahogepodge: so if we wanted to get involved and start contributing , what would that look like?17:24
hogepodgegema: the immediate spot we need help in right now is in determining capabilities for the upcoming proposed standards17:24
hogepodgegema: we also need people to write non-admin api tests to help us fill the gaps in our testing, to add as capabilities17:24
gemahogepodge: for someone like me, that is mainly a "black box tester" of clouds, what is a capability17:25
hogepodgegema: If you have a cloud, we would also love for more people to run refstack-client against it and report on all of the passing tests (this is automated with a client-server model)17:25
gemahogepodge: I can run your tests in a couple of my test clouds, or is it only production clouds that you are after17:26
hogepodgegema: a capability falls into a category of behavior, like "compute instances", and is a specific test that checks for a capability, like "boot image with hostname"17:26
hogepodgegema as many clouds as possible, test or production17:26
gemahogepodge: ok17:26
gema#action gema to submit a few test runs of refstack and get familiar with it17:27
hogepodgeWe will use the data set (which will be admittedly biased) to give us guidance on what capabilities are commonly in use17:27
jasonsbthis sounds like another possible application for our introspection thingy17:27
gemahogepodge: ok, our test clouds are representative of what we deploy for customers anyway17:27
jasonsbso the test results can be supplied along with a description17:28
hogepodgegema: if you have a valid tempest configuration refstack should be easy. It only runs tempest.api.* tests17:28
gemahogepodge: I have that for icehouse17:28
gemahogepodge: I am struggling for kilo, I was going to discuss that later17:28
hogepodgejasonsb: No, it just sends an anonymous cloud id.17:28
jasonsbhogepodge: ok17:29
hogepodgejasonsb: along with a list of all tests that passed (skip and fail results are not reported). It's just a json data structure with no other output from the subunit output or logs17:29
hogepodgejasonsb: we will have a user auth system in place soon, though, to help with self-identification17:30
jasonsbhogepodge: understood.17:30
gemahogepodge: sounds like we should be collaborating and see where we go from there, I cannot talk for the rest of the folks in our group but I will probably start a thread on the mailing17:31
gemahogepodge: are you receiving emails?17:31
jasonsbgema: agreed.17:31
hogepodgegema yes17:32
gemahogepodge: good, we'll do that then, see if we can get some traction with the others as well17:32
gemahogepodge: thanks for the information17:32
gemamoving to the next topic then17:33
jasonsbhogepodge: one more question17:33
jasonsbif its ok17:33
gemasure go for it17:33
hogepodgethat's great, thanks. Your group is an important part of the userbase we want in helping us figure out how to best test clouds. We have a mailing list, defcore-committee, that's had lots of active discussion #link http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee17:33
jasonsbif you have these test results in place, how do you envision it being used in practice?17:33
jasonsbby somebody who owns an application17:33
hogepodgejasonsb: we working up a way to pair API calls with tests17:34
jasonsbthey probably need to understand if their application fits inside defcore i would imagine17:34
hogepodgejasonsb: so if an app dev wants to know if their app is interoperable, they should be able to check the test calls to see if they are supported by a cloud.17:34
hogepodgewhether they are on the defcore list or not17:34
jasonsbi see17:35
jasonsbthat sounds quite interesting17:35
hogepodgejasonsb: we're actively pairing tests to api right now #link https://review.openstack.org/#/c/194737/17:35
hogepodgeone tricky bit is api calls can change depending on cloud config17:36
hogepodgenova-net vs the n-permutations of neutron, glance v1/v2, etc17:36
jasonsbyes indeed17:36
jasonsband some clients are very slow to update to new api's17:37
jasonsb(horizon)17:37
gemahogepodge: that's why coverage doesn't really get you that far in terms of interop and system wide testing17:37
hogepodgehorizon is working on glance v2 right now17:37
hogepodgegema: it's a hard problem, indeed. :-(17:37
gemahogepodge: it's a fascinating problem17:37
jasonsbi wonder if we can work together to fix that17:38
gemaspecially at the scale of openstack :D17:38
hogepodgeDown the line we may update refstack client to report a test: api dictionary17:38
jasonsbaudit somethign something17:38
gemahogepodge: do you guys have a say in the cyclomatic complexity of calls? i.e. how many levels are too many?17:39
hogepodgegema: no, we tend to use whatever the devs have provided17:39
gemaok17:39
jasonsbinteresting stuff.17:40
jasonsbgema:  you had next topic?17:40
gemayep17:40
hogepodgegema: We have a criteria for admitting a test as required: #link https://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst17:40
hogepodgethanks everyone for you time17:40
gemahogepodge: cool, I will be reading through all of it and asking questions17:40
*** malini has joined #openstack-tailgate17:41
gemahogepodge: thank you for  your time, this has been very educating :D17:41
gema#topic Inspector Gadget (automatic service discovery)17:41
gemaclee: you around? you were going to talk to dkranz about whether redhat were able to contribute17:41
*** malini has quit IRC17:41
gemaclee: did you?17:41
*** malini has joined #openstack-tailgate17:42
gemajasonsb: he seems busy, do you have anything on this topic?17:42
malinio/17:42
gemamalini: :D17:42
malinibetter late than never :D17:43
jasonsbi dont have anything at moment17:43
gemadoes anyone have any update on the discovery services?17:43
gemacoolio, that was quick :D17:43
gema#topic Open Discussion17:43
gemado you guys have any other topic to discuss?17:44
malinidid we already figure out the chair for next week?17:44
gemamalini: not there yet17:44
gemaI was going to bring up the issue of kilo not having a stable version of tempest17:44
gema#link http://osdir.com/ml/openstack-dev/2015-05/msg01073.html17:45
cleegema: I emailed him but got no reply17:45
gemaclee: ok17:45
cleeand he *was* idling in here, but it looks like he's gone now17:45
gemaclee: ok, I will keep the action there for next week then17:45
* clee nods17:45
gema#action  clee ask dkranz if redhat is able to contribute to the inspector gadget concept17:45
gemaso am I the only one with issues testing kilo with a stable version of tempest?17:46
maliniwe had somebody else from redhat here17:46
malinimwagner ?17:46
malinigema: probably because nobody else ran tht?17:46
jasonsbi am curious about the tooling redhat uses17:46
jasonsband what they might be willing to contribute to tailgate17:46
*** mwagner_lap has joined #openstack-tailgate17:47
gemamalini: indeed17:47
gemajasonsb: you can add it to next week's agenda and bring it up17:47
gemasee if they attend the meeting17:47
jasonsbgema: ok17:47
malinimwagner_lap: will you have any info on tht?17:47
gemamalini: he's not present17:48
gemahasn't said anything yet :D17:48
malinimaybe _lap means something17:48
gemaalright, any other topic?17:48
malinigema: I am curious abt the kilo tempest17:48
gemamalini: what do you want to know?17:49
maliniis it something we can start to help fix?17:49
gemamalini: I wanted to talk to matthew and ask him17:49
gemamalini: because what he said in that email is that there is not supposed to be a stable version of tempest17:49
malinihmm…I don't understand that17:49
gemamalini: but if we are going to use tempest to validate stable versions of clouds, we kind of need a stable version of the test suite as well17:49
maliniexactly17:50
gemamalini: they tag old version of tempest on a day17:50
gemaand they don't really maintain them17:50
gemathey move on with the development17:50
gemathat's how I understand it anyway17:50
gemaand that's why I wanted to talk to him17:50
gemawe'll have to talk to him and then decide17:51
malinithx for the info gema17:51
gemamaybe if they don't want to maintain stable versions we should do that17:51
gemabut that is a big commitment17:51
maliniOpenstack supports the past two releases -rt?17:52
gemaand by maintain I don't mean adding new tests, just keeping the ones there running should be enough17:52
gemamalini: I think so17:52
jasonsbthis sounds liek a challenge for defcore17:52
gemayet in his email he says: Tempest does not have a stable branch like other openstack projects. For more17:52
gemainfo about that see:17:52
gema#link http://specs.openstack.org/openstack/qa-specs/specs/implemented/branchless-tempest.html17:52
jasonsbhow to give a soft-landing to people who want to test clouds but don't do it often enough to keep up with temptest17:53
gemajasonsb: it's not about often17:53
gemajasonsb: the tests move on with the development17:53
jasonsbtrue true17:53
gemaif you don't have a stable version that goes with a particular release17:53
gemayou are not running the kilo tests17:53
gemabut something else17:53
gemaliberty tests against kilo17:53
gemasort of17:53
gema#action gema to talk to matthew about tempest kilo and report back17:54
gemathat's all I had17:54
gemalet's change topic if you guys have nothing else17:54
gema#topic Next Week Meeting17:54
jasonsbi have one question17:54
gemajasonsb: ok17:55
jasonsbbut its going to sound pretty noob17:55
gemayou have 1 min17:55
gemamake it quick17:55
jasonsbi was wondering if somebody could give a walkthrough of the rackspace testing17:55
malinisure we can17:55
jasonsbin general i was thinking we should start to focus on where to start for a core set of tests17:55
gemamalini: wanna do that next meeting?17:55
jasonsband i thought maybe a walkthrough of rackspace or canonical would be good place to start?17:56
malinisure - I also want sam to be here for tht17:56
gema#action malini to organise for a walkthrough of the rackspace testing17:56
maliniI will confirm with sam before committing to the next meeting17:56
gemamalini: do it for whenever sam can make it17:56
jasonsbyay17:56
gemawe can have that topic on that week17:56
jasonsbthnx17:56
*** murphyslawbbs has joined #openstack-tailgate17:57
malinitht is infact a good idea17:57
gemanow, I won't be here next week17:57
maliniI guess each of us can walk thru how we test17:57
gemaany volunteers for chair?17:57
gemamalini: yep, sounds good17:57
maliniI am tentative next week - but will try to be here17:57
gemajasonsb: will you be here for sure?17:57
murphyslawbbssorry im so late :). I will resend invite for next week17:58
jasonsbyes17:58
gemajasonsb: wanna chair?17:58
jasonsbweek after i'm out though17:58
jasonsbok17:58
maliniI can do week after tht17:58
gema#action jasonb to chair on July 9th17:58
gema#action malini to chair on July 1617:58
gemasounds like we are done17:59
gema#endmeeting17:59
jasonsbgood day all17:59
openstackMeeting ended Thu Jul  2 17:59:36 2015 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:59
openstackMinutes:        http://eavesdrop.openstack.org/meetings/tailgate/2015/tailgate.2015-07-02-17.00.html17:59
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/tailgate/2015/tailgate.2015-07-02-17.00.txt17:59
openstackLog:            http://eavesdrop.openstack.org/meetings/tailgate/2015/tailgate.2015-07-02-17.00.log.html17:59
gemathank you everyone18:00
gemait was a good meeting18:00
gemaI will send the link to the logs tomorrow18:00
gemato the mailing18:00
jasonsbsnds gd18:00
jasonsbthank you for your efforts :)18:00
gemanp!18:00
gemaEOD for me now18:00
gemanight!18:00
malinigood night gema :)18:01
*** malini has quit IRC18:06
*** jasonsb has quit IRC18:11
*** malini has joined #openstack-tailgate18:22
*** sabeen3 has left #openstack-tailgate18:55
*** sabeen1 has joined #openstack-tailgate18:56
*** malini has quit IRC20:04
*** hogepodge has quit IRC20:17
*** hogepodge has joined #openstack-tailgate20:18
*** localloop127 has quit IRC20:19
*** jasonsb has joined #openstack-tailgate20:24
*** jasonsb has quit IRC20:24
*** jasonsb has joined #openstack-tailgate20:25
*** dimtruck is now known as zz_dimtruck22:06
*** xwwt has quit IRC22:19
*** sabeen1 has quit IRC22:32
*** jasonsb has quit IRC23:07
*** sabeen1 has joined #openstack-tailgate23:07
*** jasonsb has joined #openstack-tailgate23:08
*** jasonsb has quit IRC23:12
*** jasonsb has joined #openstack-tailgate23:13

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!