17:00:11 <gema> #startmeeting tailgate
17:00:11 <openstack> Meeting started Thu Jul 23 17:00:11 2015 UTC and is due to finish in 60 minutes.  The chair is gema. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:15 <openstack> The meeting name has been set to 'tailgate'
17:00:24 <gema> #topic rollcall
17:00:27 <gema> o/
17:00:48 <gema> anyone around for the meeting today?
17:00:52 <jose-idar> I am here
17:00:54 <jose-idar> :)
17:01:12 <gema> hello jose-idar, can you introduce yourself, plz
17:01:16 <gema> I don't think I know you
17:01:36 <gema> (whilst the others arrive ;)
17:01:48 <jose-idar> Hi!  I'm QE for CloudBlockStorage at The Castle (for Rackspace).
17:02:10 <malini> welcome jose-idar :)
17:02:17 <gema> coolio, another rackspacer :D welcome!
17:02:18 <jose-idar> You can blame me for things like Opencafe and data driven testing :)
17:02:39 <gema> good, will do :D
17:02:42 <gema> malini: o/
17:02:47 <spyderdyne> i think htey call themselves rackers :)
17:02:53 <gema> ups :D
17:02:58 <jose-idar> hi malini!
17:03:02 <malini> some of us do :)
17:03:34 <malini> its great to have you here jose-idar
17:03:57 <gema> indeed
17:04:11 <gema> anyone else around today? ( I think we'll move into topics at 5 past)
17:04:12 <jose-idar> super interested in what ya'll have going on.  specifically the mission statement.  I love the idea of a community growing around the idea that Openstack needs far more than just gate tests.
17:04:39 <gema> jose-idar: indeed, you are going to have to give us an intro to data driven testing
17:04:44 <gema> we'll embrace it, I think :D
17:04:54 <jose-idar> !  definetly a goal :)
17:04:55 <openstack> jose-idar: Error: "definetly" is not a valid command.
17:05:02 <gema> lol
17:05:10 <gema> openstack: we are good, thanks
17:05:21 <jose-idar> ha!
17:05:35 <gema> alright, moving on
17:05:44 <gema> #topic Actions from last week
17:05:47 <malini> I am trying to find our 'master' etherpad link
17:05:56 <malini> anybody has it handy?
17:06:06 <gema> #link https://etherpad.openstack.org/p/openstack-tailgaters
17:06:15 <spyderdyne> posted my email ocnvo with Boris on etherpad: https://etherpad.openstack.org/p/openstack-tailgate-rallyCleanupDiscussion
17:06:20 <spyderdyne> convo
17:06:25 <malini> thanks gema..
17:06:29 <gema> np
17:06:32 <gema> spyderdyne: context?
17:06:43 <malini> jose-idar: the link has some stuff we have been talking over the last 2-3 months
17:06:50 <gema> spyderdyne: oh, your first action
17:06:53 <jose-idar> reading now, many thanks
17:06:59 * gema : spyderdyne composing a spec to present to rally on topic of rally cleanup
17:07:06 <spyderdyne> one of my action items was to contact Boris and discuss ways to provide better cleanup after rally tasks in the event of some type of failure
17:07:13 <gema> #link https://etherpad.openstack.org/p/openstack-tailgate-rallyCleanupDiscussion
17:07:29 <gema> spyderdyne: what was the outcome of the conversation?
17:08:02 <spyderdyne> they are working on the cleanup feature and it will be similar to the bash example i provided in the thread
17:08:17 <spyderdyne> (stolen from KloudBuster)
17:08:23 <gema> spyderdyne: so you'll be collaborating with them on that or let them do their thing and then review?
17:08:47 <spyderdyne> i will be monitoring and assist if they want help with it
17:08:54 <gema> spyderdyne: ack, keep us posted
17:08:58 <spyderdyne> but it is already WIP
17:09:00 <spyderdyne> will do
17:09:17 * gema : spyderdyne to give more updates on HA testing
17:09:31 <gema> any news?
17:10:08 <spyderdyne> HA test plans are created. working with another Cisco internal team on them.  Unable to run due to PTO and our target environment not being ready yet
17:10:34 <gema> spyderdyne: do you want me to keep this action on the list or do y ou want me to drop it?
17:10:37 <spyderdyne> i would rather not share them until we know they work.  will update as i get more to share
17:10:51 <gema> maybe a better way of doing this is you schedule a presentation whenever you are ready to share
17:10:53 <spyderdyne> keep it
17:10:55 <gema> ok
17:11:01 <spyderdyne> definitely
17:11:07 <gema> #ACTION spyderdyne to give more updates on HA testing
17:11:12 <spyderdyne> roger roger
17:11:18 <gema> not sure if the bot works with capitalized actions
17:11:23 <gema> I guess we'll find out
17:11:37 * gema spyderdyne, jasonsb to send out the google survey link on per project testing strategies
17:11:51 <gema> jason is not around today
17:11:55 <gema> so another one on you spyderdyne
17:12:07 <spyderdyne> i completed the survey questions and turned them over for merciless editing
17:12:39 <gema> spyderdyne: where are we editing them?
17:12:40 <malini> & we were merciless enough to not edit it :-$
17:12:42 <spyderdyne> i would like everyone to take a few minutes to review them and give notes or approve before we try sending them out
17:12:47 <spyderdyne> lol
17:12:54 <malini> will do spyderdyne
17:12:58 <gema> spyderdyne: can you send an email requesting a review and we can do this over email?
17:13:07 <gema> maybe send a google doc where we can add comments?
17:13:17 <gema> instead of the one where you are supposed to answer the questions
17:13:18 <spyderdyne> i tried digging everyone's email addresses out but failed to do so
17:13:25 <gema> spyderdyne: send it to the mailing
17:13:33 <malini> spyderdyne: are u in the tailgate ML ?
17:13:39 <spyderdyne> currently the questionaire is open to editing by anyone who has the link that i provided last week
17:13:40 <gema> openstack-tailgaters@external.cisco.com
17:13:54 <spyderdyne> we will tighten that down once we are happy with it
17:13:54 <malini> jose-idar: we need to get you added to tht
17:14:06 <jose-idar> ah. thanks!
17:14:12 <gema> spyderdyne: I get a weblike page where I can answer the questions, not a document I can edit
17:14:13 <spyderdyne> will send the invite out to that addr then
17:14:19 <gema> https://docs.google.com/a/canonical.com/forms/d/1fbY5MpMgnQmYT1l2KSjfUkKFqO_xnvgWWeTW4tt93ro/viewform
17:14:33 <malini> same here
17:14:39 <spyderdyne> b/c it is unlocked for everyone right now
17:14:59 <spyderdyne> that will change once we are ready to push it out
17:15:09 <malini> spyderdyne: all I can do with tht is answer the questions
17:16:50 <spyderdyne> live form:  https://docs.google.com/forms/d/1fbY5MpMgnQmYT1l2KSjfUkKFqO_xnvgWWeTW4tt93ro/viewform
17:17:00 <spyderdyne> #link https://docs.google.com/forms/d/1fbY5MpMgnQmYT1l2KSjfUkKFqO_xnvgWWeTW4tt93ro/viewform
17:17:06 <gema> spyderdyne: I have created a public doc, can you copy the questions there and then ask the mailing to comment?
17:17:10 <gema> https://docs.google.com/forms/d/1fbY5MpMgnQmYT1l2KSjfUkKFqO_xnvgWWeTW4tt93ro/viewform
17:17:13 <gema> no, sorry
17:17:18 <spyderdyne> responses #link https://docs.google.com/spreadsheets/d/1U63d9y4rhz03TpqK9SpSGMNT7IzAv6m_gMSrjn4S3wA/edit#gid=2011115021
17:17:18 <gema> https://docs.google.com/document/d/1Gt_HuP8dJvDZHdjjGZVcnUydpzjhijMj3c-Ehu_cw4U/edit
17:17:38 <spyderdyne> edit form #link https://docs.google.com/forms/d/1fbY5MpMgnQmYT1l2KSjfUkKFqO_xnvgWWeTW4tt93ro/edit
17:18:00 <gema> spyderdyne: that still doesn't allow us to have a conversation, just edit
17:18:05 <spyderdyne> not messy at all right? ;)
17:18:10 <gema> spyderdyne: totally x)
17:18:22 <spyderdyne> will create a new etherpad for forms discussion.
17:19:13 <gema> #action spyderdyne to start a questionaire review on the ML
17:19:23 <gema> #action malini to get jose-idar on the mailing
17:19:46 <malini> let me do tht right away - I have very short term memory
17:19:48 <spyderdyne> etherpad discusison #link https://etherpad.openstack.org/p/openstack-tailgate-forms-discussion
17:20:06 <gema> spyderdyne: any #link command preceeded by words, won't be picked by the bot
17:20:16 <spyderdyne> ok
17:20:17 <gema> I was wondering how to make all those links appear in the log
17:20:19 <gema> x)
17:20:49 <gema> spyderdyne: just for the future
17:21:08 <gema> spyderdyne: in any case, put them in your email, together with the one we can all review and collaborate on, that way everybody has them handy
17:22:21 <gema> ok, that settles the questionaire actions, cos our mailing review substitutes the action about finalizing details
17:22:29 <gema> next actions are mine
17:22:49 <gema> #action gema to submit a few test runs of refstack and get familiar with it
17:22:52 <gema> #action gema to talk to matthew about stable tempest kilo and report back
17:22:55 <gema> #action gema to send out the defcore details on actionable gaps
17:23:13 <gema> I am sorry folks, we were flooded on the night from thursday to friday and my week went downhill from there
17:23:33 <gema> hopefully this week will be more productive workwise :)
17:23:48 <jose-idar> malini: jose.idar@rackspace.com
17:23:50 <malini> gema: oops! I hope everything is in a recoverable state
17:24:05 <malini> jose-idar: I already emailed the ML owner & added you in cc
17:24:15 <gema> malini: yep, all sort of ok, plenty of furniture in the wrong place, that's all
17:24:17 <spyderdyne> #link https://docs.google.com/spreadsheets/d/1U63d9y4rhz03TpqK9SpSGMNT7IzAv6m_gMSrjn4S3wA/edit#gid=2011115021
17:24:18 <spyderdyne> <gema> https://docs.google.com/document/d/1Gt_HuP8dJvDZHdjjGZVcnUydpzjhijMj3c-Ehu_cw4U/edit
17:24:28 <spyderdyne> #link https://docs.google.com/forms/d/1fbY5MpMgnQmYT1l2KSjfUkKFqO_xnvgWWeTW4tt93ro/edit
17:24:58 <spyderdyne> they shoudl be captured by the bot now
17:25:06 <gema> spyderdyne: perfect, thanks
17:25:22 <gema> moving on to next topic then
17:25:31 <gema> unless you guys think I am missing anything?
17:25:34 <gema> 3
17:25:36 <gema> 2
17:25:39 <gema> 1
17:25:44 <malini> 0
17:25:47 <gema> #topic inspector gadget
17:25:56 <gema> anyone present and able to talk about this topic?
17:26:00 <spyderdyne> @boomGoesTheDynamite
17:26:38 <gema> spyderdyne: does that mean yes?
17:26:47 <gema> or shall I add an action for jasonb on this one?
17:27:03 <gema> not sure if we are allowed to give actions to non present people :D
17:27:03 <spyderdyne> i wasnt involved in that work
17:27:11 <spyderdyne> we are
17:27:20 <spyderdyne> they can decline them if they need to
17:27:25 <gema> #action jasonb to give an update on inspector gadget efforts
17:27:37 <gema> coolio
17:27:55 <gema> I am not going to make questionaire a topic again cos we kind of discussed next steps already
17:28:06 <gema> #topic AOB
17:28:08 <malini> sounds good
17:28:16 <malini> what is AOB ?
17:28:20 <gema> any other business?
17:28:25 <malini> :D
17:28:46 <malini> anybody made it to the defcore meeting?
17:29:03 <gema> I didn't , was dealing with  house stuff :/
17:29:09 <malini> I keep forgetting :/
17:29:23 <gema> malini: don't worry, I am still traumatized, it'll fade :D
17:29:33 <malini> :D
17:29:50 <malini> I meant, I keep forgetting to attend the defcore meeting
17:29:54 <gema> ahhh x)
17:29:57 <malini> trauma is my permanent state
17:29:59 <gema> put it in your calendar
17:30:07 <gema> the machine will annoy you to no end
17:30:14 <gema> so you remember
17:30:15 <malini> good point
17:30:47 <malini> for anybody else who need te info 'Weekly on Wednesday at http://www.timeanddate.com/worldclock/fixedtime.html?hour=15&min=00&sec=0 in ircs://irc.freenode.net:6697/openstack-meeting-4'
17:31:01 <gema> yep
17:31:31 <gema> I think this week was their midcycle
17:31:33 <gema> I may be wrong
17:31:39 <gema> maybe they didn't even do the meeting yesterday :?
17:32:34 <gema> anyway, the only other topic is next week's chair
17:32:38 <gema> any volunteers?
17:33:21 <malini> I can take tht
17:33:25 <gema> thanks :D
17:33:35 <malini> yw :)
17:33:40 <gema> we have some time, jose-idar do you want to tell us a bit about data driven testing?
17:33:47 <gema> I am quite interested in what you have to say
17:34:17 <jose-idar> Only that through much experimentation, I've found it to be a huge leap forward from the way Tempest approaches testing in general.
17:34:29 <gema> what is it exactly?
17:35:24 <jose-idar> In practice, (implementation through, say, unittest), it's as easy as test methods that take parameters
17:35:50 <gema> yep, but what is it?
17:35:53 <jose-idar> There's a lib called ddt for python that adds this to unittest, and we use a similar library internally.
17:36:13 <jose-idar> the gist is that you write a test that makes a single assumption, and drive that assumption over many sets of data.
17:36:26 <gema> jose-idar: can you give me an example of what one of these test cases would look like writen in english
17:36:31 <gema> rather than implementation details
17:36:36 <jose-idar> creating a server over combinations of imaegs and flvors for instance.
17:36:41 <gema> ahh
17:37:06 <malini> not to hijack from jose-idar, but here is an example testscenarios claims to do something similar & is openstack blessed
17:37:09 <malini> blah
17:37:12 <malini> nvm
17:37:12 <jose-idar> the idea is the make the tests simple and non-enviornment dependent.
17:37:15 <malini> let me try again
17:37:19 <malini> https://github.com/stackforge/poppy/blob/master/tests/api/services/test_services.py#L211
17:37:40 <jose-idar> malini: Awesome!  I couldn't remember what that was called, but that's where I first saw ddt
17:37:43 <jose-idar> the lib
17:38:00 <malini> testscenarios is the openstack approved lib to do something similar - But I haven't had any luck understanding it & the docs are sparse
17:38:20 <gema> malini: that's because everybody talks implementation
17:38:31 <gema> I wanted to understand how you conceptualize the tests, so if I got it right
17:38:36 <jose-idar> If we could get tempest to allow this kind of test writing, it would make testing on things that are not devstack much easier and more useful.
17:38:52 <gema> jose-idar, you'd take the outcome of a combinatorial tree and implement your test cases from there
17:39:03 <gema> trying to cover as many combinations as it makes sense, etc
17:39:04 <gema> right?
17:39:18 <jose-idar> for a complete scenario sure, but filtering those datasets down is just as important
17:39:34 <gema> jose-idar: have you heard of classification tree method?
17:39:45 <jose-idar> It's proven very useful for security to write fuzzing tests this way, iterating over known sets of sql injection strings for example.
17:39:46 <malini> hmmm….I believe combinatorial is where testscenarios comes in
17:40:02 <gema> jose-idar: https://en.wikipedia.org/wiki/Classification_Tree_Method
17:40:08 <gema> this is what I would use to trim down :D
17:40:26 <jose-idar> in practice though, the datasets are largely driven by buisness logic
17:40:46 <jose-idar> you may not want users using certain combos of images and flavors, for example
17:41:05 <gema> yep
17:41:24 <spyderdyne> arent you already telling images what flavors they can use?
17:42:01 <spyderdyne> the image meta should return a list of valid flavors right?
17:42:09 <jose-idar> sure, but relying on the mechanisms that nova relies on is only testing half of the problem.  You could also check the combinations they dont' allow, for example.
17:42:23 <gema> as a negative scenario
17:42:27 <spyderdyne> gotcha
17:42:47 <spyderdyne> so run though all flavors and record the result, marking a fail as a pass for a disallowed flavor
17:42:48 <jose-idar> and for the most part, making the tests generate the datasets on the fly via api calls at run time solves a lot of problems
17:43:18 <malini> AFAIK negative tests were not in the scope of old tempest
17:43:19 <gema> jose-idar: but you need to verify that those apis are returning the right thing
17:43:23 <jose-idar> grabbing that list and inverting the expectation is an instant, complete set of negative tests.
17:43:31 <spyderdyne> right
17:43:36 <gema> malini: there are some negative tests, I believe
17:43:38 <gema> but not many
17:43:47 <gema> there were, I don't know where they are now
17:43:48 <jose-idar> gema: yep! We've even had tests that take their expectation as a parameter, for example.
17:44:16 <gema> jose-idar: that is all very interesting
17:44:28 <gema> we'll need to keep chatting about this as we start to generate tests
17:44:43 <jose-idar> It'd be an easy way to decrease coverage gaps and increase visibility of edge case bugs
17:44:48 <gema> to make sure we make the most of the development effort, we don't have a lot of manpower
17:44:52 <gema> but we do have a lot of gaps
17:44:54 <jose-idar> it's also great for stress/perforamance testing.
17:45:46 <gema> yep, I will probably grab your help afterI send the email with the gaps/tests from defcore tot he list
17:45:57 <gema> so that you put me on the right track to start adding coverage
17:46:30 <jose-idar> sounds like a plan!  Like I said, i'm just really excited about the prospect of making things better all around :D
17:46:38 <gema> jose-idar: likewise
17:46:40 <gema> we all are :D
17:47:03 <gema> coolio, anymore questions/topics/concerns/rants?
17:47:48 <gema> then I will give you guys 10+ mins back on your day :D
17:47:55 <gema> #endmeeting