17:00:42 <gema> #startmeeting tailgate 17:00:42 <openstack> Meeting started Thu Aug 6 17:00:42 2015 UTC and is due to finish in 60 minutes. The chair is gema. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:46 <openstack> The meeting name has been set to 'tailgate' 17:00:48 <jasonsb> hi 17:00:49 <gema> #topic rollcall 17:00:50 <jose-idar> present! 17:00:54 <jasonsb> hi 17:00:58 <gema> o/ 17:01:47 <malini> o - I am half here 17:01:52 <gema> ok 17:02:11 <gema> I am going to confess I haven't had time to prepare this meeting, so we are going to go by ear 17:02:21 <jasonsb> np 17:02:22 <malini> we just had me & jasonsb in the last meeting 17:02:33 <malini> So we conveniently just chatted :) 17:02:33 <gema> were there any actions? 17:02:50 <malini> gema: it'll be good to follow up from the actions on the meeting before 17:02:51 <jasonsb> dont think so 17:02:59 <gema> malini, jasonsb ok 17:03:04 <jasonsb> i reported about the kolla midcycle meetup 17:03:29 <gema> jasonsb: ok, I will read it, I will add the link to it to my summary tomorrow morning so that people have the reference 17:03:42 <gema> #topic actions from previous weeks 17:03:59 <gema> #action spyderdyne to give more updates on HA testing 17:04:12 <gema> I think we are carrying this one forward since he is not around 17:04:31 <gema> did the review of the questionaire start? 17:04:45 <jasonsb> i wanted to ask about the questionaire 17:04:55 <gema> #action gema to sort out through the logs and get the review about the questionaire started 17:04:57 <jasonsb> is it fine to use 17:05:00 <jasonsb> or does it need tweeks 17:05:08 <gema> jasonsb: it needs tweaks imo 17:05:20 <jasonsb> ok, i'll follow up with you 17:05:25 <gema> yep 17:05:44 <gema> there are a bunch of links on the July log 17:06:02 <gema> #link http://eavesdrop.openstack.org/meetings/tailgate/2015/tailgate.2015-07-23-17.00.html (see for questionaire links) 17:06:28 <gema> I am taking the review initiation from spyderdyne and will start it myself with all the links he gave us 17:06:45 <gema> maybe people are on holidays this month 17:06:56 <gema> so I am not sure what would be a good deadline for this review 17:07:07 <gema> first week of September? 17:07:11 <jasonsb> what is the action? 17:07:24 <gema> I will send an email with all the information and instructions on where to put comments 17:07:26 <jasonsb> review the form to finalize? 17:07:34 <gema> yep 17:07:44 <gema> we'll all send comments, then ammend the form, then send it around 17:07:53 <jasonsb> i'll take it 17:07:57 <gema> you'll take what? 17:08:13 <jasonsb> solicit comments and ammend 17:08:24 <gema> coolio, I gave it to myself but I will change that right away :D 17:09:04 <gema> #action jasonb to initiate the review of the questionaire on the ML and handle review comments, review due to end early Sept (in case people are on holidays) 17:09:32 <gema> jasonsb: I will still send you an email with the bunch of links that we gathered the week you were out 17:09:37 <gema> and try to explain what they were for 17:09:38 <gema> :D 17:09:43 <jasonsb> sounds good 17:09:43 <gema> will do that tomorrow morning 17:10:18 * gema malini to get jose-idar on the mailing 17:10:22 <gema> was this done? 17:10:24 <jose-idar> :) 17:10:24 <malini> done 17:10:26 <gema> cool 17:10:40 <gema> #action gema to submit a few test runs of refstack and get familiar with it 17:10:51 <gema> #action gema to talk to matthew about stable tempest kilo and report back 17:11:03 <gema> #action gema to send out the defcore details on actionable gaps 17:11:32 <gema> the two first ones I haven't had time to do (my work has been erratic from one thing to another) but I still want to get to them 17:11:49 <gema> the third one I have been trying to get to the defcore meeting but they haven't had it for the past two weeks 17:11:52 <jasonsb> i still need to attend the defcore meetings 17:11:58 <gema> not sure if it was due to their midcycle 17:12:08 <gema> or if it is because they changed channel and I missed that 17:12:14 <gema> I will get holld of hogepodge and ask 17:12:44 <gema> #action gema to figure out where the defcore meetings are and send a note around if they have changed 17:13:02 * gema jasonb to give an update on inspector gadget efforts 17:13:15 <jasonsb> http://paste.openstack.org/show/411202/ 17:13:28 <jasonsb> i was thinking of extending the guru report 17:13:38 <gema> #link http://paste.openstack.org/show/411202/ 17:13:46 <gema> what is the guru report? 17:13:59 <jasonsb> maybe this could be added to api's so that our tools could report it 17:14:08 <jasonsb> #link https://wiki.openstack.org/wiki/GuruMeditationReport 17:14:26 <jasonsb> its already in the code base, and seems like a good start 17:14:36 <hogepodge> gema: meetings are Wednesdays at 1500. We skipped this week, but will be returning next week 17:14:41 <jasonsb> if you send a SIGUSR1 to a api it will dump out that report 17:14:51 <gema> hogepodge: excellent, thanks, I thought I was in the wrong channel or something 17:15:26 <gema> jasonsb: that is awesome, I hadn't heard of it :D 17:15:44 <jasonsb> yes, some of the basic work is alerady there 17:15:59 <jasonsb> we just need to figure out how we are going to use it and incorporate it 17:16:17 <gema> jasonsb: ok, so what is the action this time?> 17:16:49 <gema> jasonsb: as far as I remember, inspector gadget is supposed to tell us what services are available in the cloud so we can run the appropriate tests 17:16:55 <jasonsb> do a demo of it added to nova-api? 17:17:06 <gema> demo of what? 17:17:30 <jasonsb> demo of keystone query to find out what endpoints are registered 17:17:37 <gema> jasonsb: sounds good 17:17:38 <jasonsb> then query the api for the endpoint and dump out the config 17:17:59 <jasonsb> maybe its a so-so idea, but why not 17:18:24 <gema> #action jasonb to figure out the mechanics of getting endpoints and gurumeditationreport and maybe create a simple script that does this 17:18:28 <gema> sounds good? 17:18:47 <jasonsb> sounds gd 17:18:50 <gema> ok 17:19:20 <gema> that's all there is about previous actions 17:19:30 <gema> do you guys have topics you want to discuss? 17:20:07 <gema> jose-idar: you are quiet 17:20:12 <gema> what have you been up to :) 17:20:26 <jose-idar> mostly defcore, tempest and the like 17:20:33 <jose-idar> just a bunch of meetings 17:20:41 <gema> jose-idar: were you in the midcycle? 17:20:44 <gema> for defcore? 17:20:48 <jose-idar> no 17:20:50 <gema> ah, ok 17:21:17 <jasonsb> gema i would encourage you to read the log of last meeting 17:21:27 <gema> #action gema to read last week's meeting 17:21:30 <jasonsb> i think there is some interesting possibilities with kolla 17:21:34 <gema> ok, will do 17:21:48 <jasonsb> they are like-minded people 17:21:59 <gema> are they an o penstack group? 17:22:06 <jasonsb> yes 17:22:12 <gema> ok,will read with great interest 17:22:17 <jasonsb> deploy openstack services in docker 17:22:30 <jasonsb> if you deploy it, does it work? 17:22:44 <gema> but we are interested in the does it work 17:22:44 <jasonsb> what tools should you use to test the production cloud you just deployed? 17:22:50 <gema> ok 17:22:52 <jasonsb> all the same things 17:23:07 <gema> do they have an official project? 17:23:14 <jasonsb> just accepted to big tent 17:23:18 <gema> awesome 17:23:49 <jasonsb> they would like tests 17:23:59 <jasonsb> but dont' want to develop or mananage them 17:24:00 <gema> jasonsb: everybody would like tests, that's the point 17:24:04 <jasonsb> yup 17:24:19 <gema> reliable, repeatable, non-invasive, awesome tests 17:24:25 <jasonsb> yes 17:24:29 <jasonsb> more of that please 17:24:43 <gema> tests that answer: does my cloud work? with a reliable 80% or 100% 17:24:50 <jasonsb> if you have any suggestions of where to start 17:25:02 <jasonsb> :) 17:25:11 <gema> I do, I have been derailed from my strategy and never finished putting it together 17:25:36 <jasonsb> oh ok. i'll followup with you 17:25:39 <gema> #action gema to continue with the strategy doc 17:25:52 <gema> you can follow up now if you want 17:25:55 <gema> we still have 35 mins 17:26:27 <jasonsb> the people in kolla meetup were more operations oriented than dev oriented 17:26:34 <jasonsb> (at least from experience point of view) 17:26:51 <gema> so they are users 17:26:54 <gema> and sys admins 17:26:57 <jasonsb> so i heard all of the same feedback about tempest and rally that we have discused in past 17:27:10 <jasonsb> dev + heavy admin 17:27:16 <gema> devops, ok 17:27:22 <jasonsb> sure 17:27:50 <jasonsb> so need a tool which can start small and light 17:28:09 <gema> we don't need a tool, jasonsb , we need test cases 17:28:10 <jasonsb> which (i'll cut and paste) reliable, repeatable, non-invasive, awesome tests 17:28:15 <gema> we have plenty of tools 17:28:54 <jasonsb> ok, so lets make some test cases 17:29:01 <jasonsb> you tell me the framework 17:29:05 <gema> tempest 17:29:08 <jasonsb> and we can contribute to kolla 17:29:36 <gema> tempest uses unittest, I believe, which is good enough for us to start with 17:29:45 <gema> and we can always move those scripts somewhere else 17:29:53 <gema> or we could even do standalone unittests 17:30:01 <jasonsb> ok, but we would need to solve some of the operational problems with tempest 17:30:12 <jasonsb> needs to work with multiple versions of openstack 17:30:14 <jasonsb> needs to clean up 17:30:16 <jasonsb> blah blah 17:30:22 <gema> jasonsb: all tests need to clean up 17:30:59 <gema> and tempest offers us a few apis that we can use and common operations 17:31:13 <gema> but it takes longer to ramp up and write tests than unittest 17:31:27 <jasonsb> ok 17:31:31 <malini> sorry..I am only half here 17:31:32 <gema> jasonsb: or we could start by making tempest clean up 17:31:37 <gema> contributing some changes to that 17:31:38 <gema> end 17:31:41 <malini> But is the plan to use tempest-lib ? 17:31:49 <gema> malini: there is no plan 17:31:54 <malini> :D 17:31:57 <gema> malini: we can use whatever we want 17:32:13 <gema> malini: or we could say: our tests are self contained scripts 17:32:20 <gema> if the script returns 0 you are good 17:32:21 <rockyg> o/ 17:32:26 <malini> hello rockyg!! 17:32:27 <gema> if it returns something else , not so much 17:32:30 <gema> rockyg: o/ 17:32:40 <jasonsb> ok 17:32:47 <malini> gema: hmm…but we shud have better test results than a 0 or 1 17:32:53 <jasonsb> this sounds like defcore effort too 17:33:06 <gema> malini: that's a fail or a pass 17:33:09 <jasonsb> (two birds is good) 17:33:10 <gema> what else do you want? :D 17:33:23 <malini> why/where did it fail? 17:33:24 <gema> malini: we surely can count 0s and 1s 17:33:26 <rockyg> I'm on Defcore. I can help 17:33:32 <gema> rockyg: go ahead please 17:33:42 <gema> rockyg: trying to decide framework here 17:34:00 <gema> malini: you achieve that by making your tests spit comments 17:34:03 <rockyg> Hey. Just heard about you guys. This is *great* I want to help, but I've got too many irons in the fire. 17:34:17 <gema> rockyg: we are all more or less the same 17:34:18 <rockyg> But, I can do some influencing and advising 17:34:36 <rockyg> I saw Gema 17:34:49 <rockyg> 's post on the Summit session proposals 17:34:55 <gema> rockyg: so if we were to create tests that you guys can use, should we use tempest? 17:35:07 <gema> or tempest-lib 17:35:10 <rockyg> I was gonna try to get a birds of a feather going in Tokyo. 17:35:37 <rockyg> So, it should work *with* tempest_lib, but doesn't have to be in the tempest repo 17:36:04 <gema> so you want the test runner to be tempest lib 17:36:11 <rockyg> Go out to Refstack repo (I think it's under openstack-infra now or will be shortly) 17:36:21 <rockyg> Look at the readme for tempest-client 17:36:28 <gema> rockyg: ok 17:37:03 <rockyg> that's what DefCore is requiring to be the test runner/results collection 17:37:28 <gema> and do you have an idea of where are the biggest gaps in coverage? 17:37:45 <gema> for defcore's purposes 17:37:56 <rockyg> Other thing is that we're entertaining proposals for other test sets within the Refstack server that validate interop. 17:38:14 <rockyg> Oh, yeah, to some extent. 17:38:39 <rockyg> But what we're finding is that there are actual code gaps that make interop hard. 17:38:50 <gema> yeah, testability is an issue for us too 17:39:21 <rockyg> And really, right now DefCore uses api tests for interop where really, we should have tests for user use cases 17:39:40 <gema> yeah, I have a problem with calling refstack tests "interop" 17:39:46 <gema> but I don't want to be picky yet :D 17:39:50 <rockyg> Like give me a vm with external access and a private management network 17:40:02 <jose-idar> gema: +1! 17:40:38 <rockyg> Monty Taylor and John Garbutt have come out loudly about the fact that there are like three ways to create a vm and none of them are good. 17:41:33 <gema> what's wrong with nova boot? 17:41:40 <rockyg> Sorry for any time gaps, I'm managing the diversity meeting, too right now. 17:41:57 <gema> rockyg: don't worry, thanks for coming, this is very enlightening 17:42:10 <rockyg> I think QA/E should be considered a minority for OpenStack Foundation's sake ;-) 17:42:27 <gema> :) 17:42:32 <rockyg> I just talked to Sam Danes (rackspace) and he' 17:42:41 <rockyg> s really interested in this group, too. 17:42:54 <gema> sam was with us in vancouver 17:43:02 <gema> at the middle of our early conversations, right malini ? 17:43:02 <rockyg> I have a possible strategy for acceptance -- FMEA test framework 17:43:23 <rockyg> It's not part of tempest charter. No ERROR messages allowed in tests for passing 17:43:53 <malini> gema: yes 17:43:57 <rockyg> And if we get something like this happening, I think we'd be the first OpenSource project collection with *real* QA 17:44:07 <gema> rockyg: that's the idea 17:44:15 <gema> we were all trying to do it separately 17:44:20 <gema> so we decided to join forces 17:44:24 <gema> it's too big of a task 17:44:30 <rockyg> Yup. Sam was there. That's how I heard about you after I saw the proposals 17:45:08 <rockyg> Yes. Much too big, but once a framework and rules for writing the tests are in place, lots of ops guys will provide tests 17:45:17 <gema> yep, that's my hope 17:45:28 <gema> that's what happened in ubuntu when we started QA few years ago :D 17:45:42 <gema> and that's why I was talking about any binary and using the return code 17:45:43 <rockyg> So, the focus should be on getting some possible frameworks and some sample tests. 17:45:51 <gema> because people like to write tests however they like them 17:45:59 <gema> and if they are good, they can be integrated 17:46:06 <rockyg> Yeah. Exactly. 17:46:17 <gema> if we are too prescriptive, we won't get far 17:47:09 <gema> ok, I have the feeling I should put an action from our conversation with rockyg , but I don't know where to start 17:47:29 <malini> Identify one scenario from rockyg 17:47:34 <rockyg> So, is this a weekly meeting? Or just everyone happens to be here? 17:47:45 <malini> 'try' to write a test with tempest-lib 17:47:50 <gema> rockyg: it's a weekly, sometimes there's more people than other times :D 17:48:12 <gema> malini: who should attempt such task? 17:48:14 <rockyg> So, 10am PDT Thursdays? I'll put it on my calendar 17:48:26 <malini> gema: any of us can. I wouldn't mind doing it 17:48:39 <gema> malini: ok, have a go and let us know, then 17:48:41 <malini> I have always been planning to try out tempest-lib 17:48:46 <gema> yep 17:48:57 <malini> rockyg: is there a scenario for us to start with? 17:48:58 <rockyg> What might also be useful is to get a test or a representative collection of tests that folks are running at their sites 17:49:04 <malini> just as a POC 17:49:11 <rockyg> I know Sam has a number. 17:49:14 <gema> #action malini to try to write a test with tempest-lib that spins a VM and destroys is, as a POC 17:49:33 <rockyg> I think we need a smallish test to start. give me a vm is too big 17:49:46 <gema> ok, list VMs ? 17:49:49 <gema> nova list 17:49:53 <gema> but from the api? 17:50:17 <gema> and if it doesn't error it is a pass 17:50:36 <rockyg> Ooh, but if you can spin a vm, gather the info from it a user needs (networks, ssh, etc), then delete it, that would be very good. 17:50:52 <gema> rockyg: we were trying to start small, right? 17:51:04 <rockyg> Yeah. But I can hope? 17:51:10 <gema> that's why I thought spin vm and destroy vm was a good start 17:51:29 <gema> even if it takes a while to get glance serving images and keystone authenticating 17:51:30 <rockyg> Actually, one that is not doable now is to list available images in a cloud 17:51:42 <gema> there's no api for that? 17:51:52 <rockyg> All DefCore tests are user space, no admin. 17:52:01 <gema> that's a good point 17:52:08 <rockyg> There is currently a test that does that, but it requires admin. 17:52:28 <malini> if its admin only, it doesnt make sense to be in defcore -rt? 17:52:31 <jose-idar> rockyg: do you know if the call requires admin, or if the test is requiring that erroneously? 17:52:34 <rockyg> And since glance v2 is not in nova yet, there is no user facing api. 17:53:03 <rockyg> That's why it's not in Defcore as a requirement. Test is flagge ;-) 17:53:08 <gema> why, I can list the images available to me on my env without admin 17:53:11 <gema> glance image-list 17:53:22 <gema> (on kilo) 17:53:31 <malini> hinking abt it, it shud be user accessible 17:53:38 <rockyg> Is that v1 or v2, and is it an internal api that's exposed, or external 17:53:51 <gema> I use the glance client 17:53:59 <gema> not sure what version of the api it uses 17:54:01 <gema> how do I check that? 17:54:19 <rockyg> Logs, I think, and we all know they suck 17:54:26 <gema> glance logs? 17:54:49 <jasonsb> yuo can specify which api with glance client 17:54:52 <gema> #action gema to look at the glance logs of her test cloud and check what api is being used (we can list images without admin) 17:54:57 <gema> ok, will check and report back 17:55:05 <rockyg> Thanks! 17:55:08 <gema> np 17:55:43 <gema> ok, we are about to hit the end of the meeting, malini are you comfortable with the POC? 17:55:48 <gema> feel free to change the test case 17:55:50 <rockyg> One of the next areas Defcore is expanding into is Neutron for 2016.01 17:55:52 <gema> to whatever is achievable 17:56:12 <malini> "malini to try to write a test with tempest-lib that spins a VM and destroys is, as a POC" is this still the POC test? 17:56:17 <rockyg> They will need user based tests. What are my networks and which is which 17:56:30 <gema> malini: that's the action, just do whatever yoou think is achievable 17:56:32 <gema> if that is not 17:56:39 <malini> sounds good 17:56:53 <rockyg> I'll let you know that rackspace and helion require different paths to make it happen. 17:57:27 <malini> hmmm..so my test shud theoretically run on any OS cloud 17:57:31 <gema> yep 17:57:33 <rockyg> Monty Taylor also might have a bunch of code checked in under shade that show you how he does it 17:57:39 <gema> malini: but start with yours 17:57:56 <malini> yes..I dont like anybody else's code anyways :D 17:58:02 <gema> no, I mean your cloud xD 17:58:03 <rockyg> lol 17:58:12 <gema> then we can run your script on all the other ones and see xD 17:58:17 <jose-idar> the novaclient manages it via auth plugins, as long as identity is out of scope, it should work on most clouds. 17:58:48 <gema> alright , sounds like we are almost at the end of the day 17:58:58 <gema> any volunteers for chairing next week? 17:59:03 <rockyg> Defcore wants Keystone tests, too, but so far, they are all admin. 17:59:37 <gema> #action gema to check what parts of keystone are accesible without admin on ubuntu's distro 17:59:44 <jose-idar> I'm just advocating that the scopes should be separate. the nova tests shouldn't fail because of auth if they don't have to, for example. 18:00:02 <gema> jose-idar: it depends on whether the test is end to end or not 18:00:10 <gema> if it is end to end it should fail 18:00:10 <rockyg> +100 18:00:16 <gema> if it is not, it shouldn't 18:00:24 <gema> and for interop reasons, you want end to end 18:00:28 <jose-idar> sure, but there should also be one that isn't end to end 18:00:37 <gema> we need to make auth compatible between clouds 18:00:42 <gema> it shoudln't be that difficult 18:00:48 <jose-idar> because it's just as important to know if the components work as it is to know if the cloud works together. 18:01:15 <gema> jose-idar: true, but the scope of the tests is another long conversation to have in the future I think :D 18:01:16 <rockyg> So far, Defcore is only working on the first part. 18:01:17 <jose-idar> cross-cloud auth compatibility should probably be #1 priority honestly :D 18:01:46 <rockyg> But would accept tests for the second. 18:01:52 <jose-idar> the problem with putting it off is that defcore is going to set the prescedent for end-to-end tests when nothing works together yet. 18:02:12 <rockyg> yeeup 18:02:14 <gema> jose-idar: a failing test is a great starting point, something to fix 18:02:17 <jose-idar> they should have started on per-component first, but that would require them writing actual specs instead of relying on the already exant temptest tests. 18:02:44 <rockyg> jose-idar, welcome to my world :-/ 18:02:55 <gema> rockyg: maybe we can help you change that 18:03:06 <rockyg> Yes, please! 18:03:06 <jose-idar> a component test that fails is much clearer than an end to end test that fails, especially considering a user that runs the tests on a cloud who's logs they can't acess 18:03:09 <gema> jose-idar: can we attempt to write one of those specs as an example? 18:03:33 <jose-idar> probably! 18:03:34 <gema> jose-idar: I mean you sound like you know what you want such spec to look like 18:03:43 <gema> can you start one so we can contribute to it 18:03:46 <gema> say .. for nova? 18:03:48 <rockyg> Also, users should have access at least to their vm logs. 18:04:13 <rockyg> jose-idar, please? 18:05:23 <rockyg> The key is, the real test is whether you can have a vm image with your app on it and recreate it or move it to another cloud. That's interop. 18:05:30 <beisner> o/ hi, apologize for the late pop-in, just reading backscroll 18:05:33 <rockyg> Oh, and manage on both 18:05:50 <rockyg> We ask for so little ;-) 18:05:59 <gema> #action jose-idar to start a spec for component tests for nova (user side, no admin), so we can all contribute cases and maybe implement them at some point , but at least start to get organised 18:06:06 * gema looks shyly at jose-idar 18:06:25 <malini> should be 'stares at jose-idar' ;) 18:06:30 <gema> haha 18:06:32 <jose-idar> sorry, got pulled away 18:06:42 <jose-idar> sure!@ 18:06:48 <gema> jose-idar: \o/ thanks! 18:06:55 <rockyg> \o/ 18:07:10 <gema> ok, I am going to call it a meeting before I start giving actions to non present people :D 18:07:13 <jose-idar> i'll sync up w/ dary walleck, because I think he's working on a oparallel task 18:07:18 <jose-idar> thansk everyone!@ 18:07:24 <gema> #endmeeting