| *** zz_dimtruck is now known as dimtruck | 00:48 | |
| *** dimtruck is now known as zz_dimtruck | 01:34 | |
| *** zz_dimtruck is now known as dimtruck | 01:38 | |
| *** mwagner_afk has joined #openstack-tailgate | 02:43 | |
| *** dimtruck is now known as zz_dimtruck | 04:26 | |
| gema | jasonsb: backwards compatibility would determine whether a change is compatible with previous versions of the other systems | 05:12 |
|---|---|---|
| gema | jasonsb: so say you make a change in nova-compute and want to make sure it is going to play well with old versions of glance, nova-cloud-controller, keystone, etc | 05:13 |
| gema | jasonsb: I have used backwards compatibility testing in the past to determine whether a change required recompiling of the whole system or if it could be updated easily in isolation | 05:14 |
| gema | jasonsb: but that's more applicable when talking about a single operating system than about the cloud, I think | 05:15 |
| gema | jasonsb: uhmm, too early for me it seems, | 05:19 |
| gema | jasonsb: if the user of your API has to recompile or change their app, then you've broken backwards compatibility | 05:20 |
| gema | :) | 05:20 |
| gema | jasonsb: you can break it either by changing the api's signature or by changing the behaviour | 05:21 |
| gema | (third party apps rely on previous behaviour) | 05:21 |
| jasonsb | i got it | 06:10 |
| jasonsb | there is talk on ml about rolling upgrades and other things | 06:10 |
| jasonsb | i wonder if some formal testing in this area might help alot of folks | 06:11 |
| jasonsb | i'll give some thought | 06:11 |
| jasonsb | cheers | 06:11 |
| gema | jasonsb: ack, give it some thought and we can add it | 09:31 |
| gema | jasonsb: I think we need functional testing before we get to the fancy stuff, though | 09:32 |
| gema | functional clear and concise testing that can answer the question: "does my cloud work?" | 09:32 |
| gema | yes, 100%, or you are just 97% there, etc | 09:32 |
| *** hogepodge has quit IRC | 10:58 | |
| *** hogepodge has joined #openstack-tailgate | 11:00 | |
| *** hogepodge has quit IRC | 11:56 | |
| *** hogepodge has joined #openstack-tailgate | 12:02 | |
| *** hogepodge has quit IRC | 13:00 | |
| *** hogepodge has joined #openstack-tailgate | 13:02 | |
| *** hogepodge has quit IRC | 13:11 | |
| *** hogepodge has joined #openstack-tailgate | 13:12 | |
| *** xwwt has joined #openstack-tailgate | 13:29 | |
| *** hogepodge has quit IRC | 13:39 | |
| *** hogepodge has joined #openstack-tailgate | 13:41 | |
| *** pglass has joined #openstack-tailgate | 14:07 | |
| *** zz_dimtruck is now known as dimtruck | 14:12 | |
| *** localloop127 has joined #openstack-tailgate | 14:24 | |
| hogepodge | jasonsb: gema Seems like there's some overlap with what defcore is trying to accomplish | 14:46 |
| hogepodge | jasonsb: gema: we have testing in place, but really want to make it better, as interoperability is one of our main goals | 14:46 |
| *** malini has joined #openstack-tailgate | 14:59 | |
| *** localloop127 has quit IRC | 15:04 | |
| *** jasonsb has quit IRC | 15:09 | |
| *** malini has quit IRC | 15:12 | |
| *** malini has joined #openstack-tailgate | 15:19 | |
| *** localloop127 has joined #openstack-tailgate | 16:03 | |
| gema | hogepodge: what kind of testing do you have atm? | 16:08 |
| gema | hogepodge: we definitely don't want to overlap, just contribute to where the gaps are :D | 16:08 |
| hogepodge | gema: we have a subset of non-admin API tests that tries to establish a baseline of functionality | 16:09 |
| gema | this is a separate test suite from tempest? | 16:09 |
| gema | and from rally? | 16:09 |
| hogepodge | gema: we draw from tempest | 16:09 |
| gema | hogepodge: what does that mean, it is a fork?> | 16:09 |
| hogepodge | gema: if a product built on openstack wants to use the logo, it needs to pass this set of tests: http://git.openstack.org/cgit/openstack/defcore/tree/2015.05/2015.05.required.txt | 16:10 |
| hogepodge | gema: so they need to run tempest | 16:10 |
| gema | hogepodge: ok, that is a subset of tempest | 16:11 |
| gema | do you have any tests that are not in tempest at all? | 16:11 |
| hogepodge | It has some weaknesses right now that we're tying to address. Like, how do you manage images (glance v1/v2 is painful). How do you manage networks (nova-net, the millions of permutations you can run neutron in) | 16:11 |
| hogepodge | gema no | 16:11 |
| gema | hogepodge: ack | 16:11 |
| hogepodge | gema my feeling is that if new tests are being written, there's a tremendous amount of value in putting them into tempest, especially if they impact interoperability | 16:12 |
| gema | hogepodge: I would like to have a proper chat with you about this, because what you guys are doing is very valuable but we feel tempest is not comprehensive enough to determine whether a cloud is good or not | 16:12 |
| gema | hogepodge: maybe the solution is contributing to tempest , but since the tests in tempest are being put in the projects they belong to | 16:13 |
| gema | we feel that any cross project tests are not going to have a proper "home" there | 16:13 |
| gema | hogepodge: according to the discussion during the summit | 16:13 |
| hogepodge | gema I've spoken with the mtreinish quite a bit about this. His feeling is that any test that defcore wants for interoperability belongs in tempest and is granted an exception | 16:14 |
| *** eglute has joined #openstack-tailgate | 16:14 | |
| gema | hogepodge: we are interested in crossproject testing specially, those that do not necessarily belong to a particular project | 16:14 |
| gema | hogepodge: oh, that is awesome news | 16:14 |
| hogepodge | gema: we are very interested in the same thing | 16:14 |
| gema | so we could contribute them there | 16:14 |
| gema | \o/ excellent news | 16:14 |
| hogepodge | gema We have weekly defcore meetings on wednesdays over IRC, and it would be great if you and other reps from this group attended. | 16:14 |
| gema | hogepodge: sounds good, what time are your meetings? | 16:15 |
| hogepodge | gema defcore is a board back process that impacts logo usage, so it carries real teeth. | 16:15 |
| gema | hogepodge: one of our main objectives was to create tests that were of real value to you guys | 16:15 |
| gema | because at the end of the day all the integrators want to validate that the clouds they install (we install) are compliant | 16:16 |
| gema | and are openstack | 16:16 |
| gema | if that makes sense | 16:16 |
| hogepodge | gema: right now we tick tock between morning and afternoon. | 16:16 |
| gema | and we had no proper set of tests that would determine that | 16:16 |
| hogepodge | gema that's what we want too | 16:16 |
| gema | hogepodge: is there a calendar somewhere? | 16:16 |
| hogepodge | gema really want | 16:16 |
| hogepodge | gema I"m looking for the link | 16:16 |
| gema | hogepodge: ack | 16:17 |
| *** malini has quit IRC | 16:18 | |
| gema | hogepodge: the other thing we are working on is an automated way to configure the test suite, so that we can discover all the services available | 16:18 |
| hogepodge | gema: mailing list link for our previous meeting http://lists.openstack.org/pipermail/defcore-committee/2015-June/000871.html | 16:18 |
| gema | rather than having to configure it manually | 16:18 |
| hogepodge | gema: +1 | 16:18 |
| gema | hogepodge: the downside is that it'll require admin as things stand today | 16:18 |
| hogepodge | gema actually, you might want to look at the entire archive for the last month | 16:18 |
| gema | unless we convince someone to enable an api | 16:18 |
| gema | hogepodge: indeed, will do | 16:18 |
| *** markvoelker has joined #openstack-tailgate | 16:19 | |
| hogepodge | gema: we specifically only test non-admin. It limits things, and makes assumptions about available information and resources | 16:19 |
| gema | hogepodge: yep, what is the reason to focus on non-admin? | 16:19 |
| gema | I think I should join your mailing, indeed | 16:20 |
| gema | you guys are talking about the samethings | 16:20 |
| hogepodge | gema: we want an arbitrary cloud user to point the test suite at a cloud and independently evaluate if that cloud is actually compliant with the defcore standard | 16:20 |
| gema | hogepodge: ok, what about the admin we hand over the cloud to after installing it | 16:20 |
| gema | hogepodge: those people also want to validate it | 16:20 |
| hogepodge | gema: if they have admin privileges, we have a mechanism for testing all of the apis and reporting on them. | 16:21 |
| gema | hogepodge: ack | 16:21 |
| hogepodge | gema: so two levels. 1) Am I defcore compliant. 2) What api tests do I actually pass. The first is required for logo, the second is useful for comparison | 16:22 |
| gema | for comparison to what? | 16:22 |
| *** malini has joined #openstack-tailgate | 16:22 | |
| gema | oh, you mean that 1) is answered with yes/no, and 2) with a list of tests? | 16:23 |
| hogepodge | gema: yes | 16:23 |
| gema | ok | 16:23 |
| gema | hogepodge: sounds like getting in the loop with what you guys are doing is important and we can contribute | 16:24 |
| gema | hogepodge: but this is like a first step | 16:24 |
| hogepodge | gema: really, we just want everyone to run testr with tempest.api and report back on every test that passes. If we can collect that data across clouds we can see what is widely working and what isn't | 16:24 |
| gema | after that we also care about other non-functional attributes of clouds | 16:24 |
| gema | that are operationally important | 16:25 |
| hogepodge | gema: We very much want community input on defcore. In terms of determining what to test, but also in writing the tests. | 16:25 |
| *** jasonsb has joined #openstack-tailgate | 16:25 | |
| hogepodge | gema: your scope seems larger from that point of view | 16:25 |
| gema | hogepodge: yep, but we have to start somewhere | 16:25 |
| gema | and defcore seemed like the right place | 16:25 |
| hogepodge | gema: my dream is to have a secure service where you can point it at a cloud with your user creds and it tells you everything you can do with it. | 16:26 |
| gema | hogepodge: indeed, but the testability of openstack is not there yet | 16:26 |
| gema | maybe if we gain a bit of momentum and credibility with defcore tests we can start influencing the testability as well | 16:26 |
| hogepodge | gema the meeting link I was looking for http://eavesdrop.openstack.org/#DefCore_Committee_Meeting | 16:26 |
| gema | hogepodge: ack, thanks | 16:27 |
| gema | hogepodge: 1UTC is not going to happen for me, I think | 16:27 |
| hogepodge | gema I think it's further along than you may think, but it definitely needs work no question. | 16:27 |
| gema | 1500 sounds more appropriate | 16:27 |
| hogepodge | I have a feeling that we're going to fall back to 1500 | 16:28 |
| gema | hogepodge: ok, I will refer to tailgaters our conversation on Thursday (our meetings are here on Thurs at 1700 UTC, I think | 16:28 |
| gema | hogepodge: well, I am in the uk, I don't think my partner would appreciate me sneaking out of bed in the middle of the night to attend a meeting | 16:29 |
| gema | :) | 16:29 |
| gema | hogepodge: where are you based? | 16:29 |
| hogepodge | gema: portland, or. | 16:30 |
| gema | hogepodge: ack | 16:30 |
| gema | hogepodge: I appreciate you stopping by and giving me the overview | 16:30 |
| hogepodge | gema: I'll start attending your meetings | 16:30 |
| gema | hogepodge: sounds good | 16:30 |
| hogepodge | gema: sure, I was shocked when I learned about the group. "what? these are all the people we want involved in defcore!" :-D | 16:30 |
| gema | hogepodge: we had you in mind, have you seen the etherpad? | 16:31 |
| gema | https://etherpad.openstack.org/p/testing2.0 | 16:31 |
| gema | this is the log from our session on Friday at ODS | 16:31 |
| gema | defcore validation was at the top of the list | 16:32 |
| gema | :D | 16:32 |
| hogepodge | I started looking at it. | 16:32 |
| hogepodge | gema where are you based? | 16:32 |
| gema | hogepodge: UK | 16:32 |
| gema | and I work for canonical | 16:32 |
| gema | you have people from several integrators in our group | 16:33 |
| gema | kind of a strange bunch, I thought when we met :D | 16:33 |
| gema | hogepodge: and we are all QA folk | 16:33 |
| gema | not so much developers | 16:33 |
| gema | but test automation is at the core of what we do | 16:34 |
| hogepodge | gema to me the line is always blurry. What's dev without qa? | 16:34 |
| gema | hogepodge: depends on who you ask, I guess :) | 16:34 |
| gema | hogepodge: oh, so you work for the openstack foundation | 16:35 |
| gema | cool | 16:35 |
| gema | (sorry, couldn't help it :D) | 16:36 |
| hogepodge | gema: are you involved with the orange box at all? maas needs log rotation implemented :-D | 16:36 |
| gema | hogepodge: nah, orange box is another part of the company | 16:36 |
| gema | but maas definitely needs that :D | 16:36 |
| hogepodge | gema yeah, I'm focused on interoperability, which is tightly linked to testing | 16:36 |
| hogepodge | gema it likes to eat up my hard drive. ;-) | 16:36 |
| gema | hogepodge: https://bugs.launchpad.net/maas/+bug/1381603 | 16:36 |
| openstack | Launchpad bug 1381603 in MAAS "Log rotation for maas.log seems broken or non existent" [Undecided,Fix committed] | 16:37 |
| gema | good news :) | 16:37 |
| gema | hogepodge: I am glad to hear you are using maas :D | 16:37 |
| hogepodge | gema: heh :-) It came preinstalled. I was actually thinking of moving to ironic, because openstack | 16:39 |
| gema | hogepodge: is tomorrow a 3pm meeting? | 16:39 |
| *** malini has quit IRC | 16:39 | |
| hogepodge | gema: yes | 16:40 |
| gema | hogepodge: ok, I think I have at least 30 mins I can use to attend | 16:40 |
| gema | will be better organised next time | 16:40 |
| hogepodge | thanks, we can add an agenda item for introductions | 16:40 |
| gema | ok, I will send an email to all the tailgaters, just in case some of them can also make it | 16:41 |
| hogepodge | gema excellent. Thanks. You can add me to that list too if you'd like. | 16:43 |
| gema | I will ask the owner, right now it is hosted at cisco | 16:43 |
| gema | if you tell me your email I will ask him | 16:43 |
| hogepodge | chris@openstack.org | 16:43 |
| gema | ack | 16:43 |
| *** malini has joined #openstack-tailgate | 16:45 | |
| malini | "What's dev without qa?" —>it is very blurry - rt? | 16:49 |
| gema | malini: I am about to send an email | 16:49 |
| gema | malini: hogepodge is from DefCore | 16:49 |
| malini | tht is awesome | 16:49 |
| gema | malini: do you remember who owns the mailing? | 16:50 |
| malini | gema: marten | 16:50 |
| malini | I dont know his nick | 16:50 |
| gema | malini: don't worry | 16:50 |
| malini | & I am sure I spelled him wrong | 16:50 |
| gema | Maarten ;) | 16:51 |
| malini | gema: mthibaut@cisco.com | 16:51 |
| gema | malini: yep, he's on the mailing, so he'll get my message | 16:51 |
| gema | thanks | 16:51 |
| gema | malini: email sent | 16:53 |
| gema | I am goin to EOD now, duty calls in the kitchen! | 16:53 |
| malini | stomach is always the biggest priority :) | 16:53 |
| malini | Enjoy your dinner | 16:54 |
| gema | thanks :D | 16:54 |
| *** jasonsb has quit IRC | 16:59 | |
| *** pglass has quit IRC | 17:03 | |
| *** mwagner_afk has quit IRC | 17:06 | |
| *** malini has quit IRC | 17:11 | |
| *** jasonsb has joined #openstack-tailgate | 17:24 | |
| *** hogepodge has quit IRC | 17:26 | |
| *** hogepodge has joined #openstack-tailgate | 17:27 | |
| *** malini has joined #openstack-tailgate | 17:51 | |
| *** mwagner_afk has joined #openstack-tailgate | 18:02 | |
| *** pglass has joined #openstack-tailgate | 18:13 | |
| *** jasonsb has quit IRC | 18:33 | |
| *** malini has quit IRC | 18:35 | |
| *** malini has joined #openstack-tailgate | 18:48 | |
| *** markvoelker has quit IRC | 18:52 | |
| *** vtapia_ has joined #openstack-tailgate | 18:59 | |
| *** vtapia has quit IRC | 18:59 | |
| *** hogepodge has quit IRC | 19:02 | |
| *** hogepodge has joined #openstack-tailgate | 19:03 | |
| *** dwalleck has joined #openstack-tailgate | 19:32 | |
| *** dwalleck has quit IRC | 19:32 | |
| *** mwagner_afk has quit IRC | 19:39 | |
| *** dwalleck has joined #openstack-tailgate | 19:42 | |
| *** dwalleck has quit IRC | 19:44 | |
| *** malini has quit IRC | 20:01 | |
| *** malini has joined #openstack-tailgate | 20:04 | |
| *** dwalleck has joined #openstack-tailgate | 20:05 | |
| *** dwalleck has quit IRC | 20:10 | |
| *** dwalleck has joined #openstack-tailgate | 20:11 | |
| *** malini has left #openstack-tailgate | 20:13 | |
| *** jasonsb has joined #openstack-tailgate | 20:18 | |
| *** jasonsb has quit IRC | 20:30 | |
| *** jasonsb has joined #openstack-tailgate | 21:09 | |
| *** dwalleck has quit IRC | 21:11 | |
| *** dimtruck is now known as zz_dimtruck | 21:14 | |
| *** dwalleck has joined #openstack-tailgate | 21:22 | |
| *** dwalleck has quit IRC | 21:23 | |
| *** mwagner_lap has joined #openstack-tailgate | 21:26 | |
| *** zz_dimtruck is now known as dimtruck | 21:29 | |
| *** localloop127 has quit IRC | 21:55 | |
| *** pglass has quit IRC | 22:03 | |
| *** dimtruck is now known as zz_dimtruck | 22:11 | |
| *** dwalleck has joined #openstack-tailgate | 22:17 | |
| *** dwalleck has quit IRC | 22:23 | |
| *** dwalleck has joined #openstack-tailgate | 22:24 | |
| *** mwagner_lap has quit IRC | 22:32 | |
| *** jasonsb_ has joined #openstack-tailgate | 23:45 | |
| *** jasonsb has quit IRC | 23:45 | |
| *** zz_dimtruck is now known as dimtruck | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!