17:02:22 <ruhe> #startmeeting murano
17:02:23 <openstack> Meeting started Tue Jun 17 17:02:22 2014 UTC and is due to finish in 60 minutes.  The chair is ruhe. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:27 <openstack> The meeting name has been set to 'murano'
17:02:55 <ruhe> murano team, are you here?
17:03:03 <slagun> o/
17:03:03 <tsufiev> ruhe, affirmitive
17:03:04 <sjmc7> hello
17:03:05 <akuznetsova_> hi
17:03:08 <dteselkin> Hi
17:03:25 <ruhe> #link https://wiki.openstack.org/wiki/Meetings/MuranoAgenda#Agenda
17:03:26 <btully> hi
17:03:36 <serg_melikyan> o/
17:03:44 <ruhe> #topic action items review
17:03:56 <ruhe> i don't see any action items from the previous meeting
17:04:09 <ruhe> maybe we missed to record them?
17:04:52 <ruhe> ok. let's move to the next and the most important topic for today
17:04:56 <ruhe> #topic Discuss proposal to impose stricter testing requirements
17:05:02 <ruhe> #link https://blueprints.launchpad.net/murano/+spec/improve-testing
17:05:09 <ruhe> #link https://etherpad.openstack.org/p/murano-testing
17:05:31 <ruhe> did everyone get familiar with this ^ document?
17:06:13 <btully> sadly, i did not. looking at it now
17:06:38 <ativelkov> o/
17:06:54 <ruhe> so, the idea is to block all the feature development and focus and stabilistation, test coverage, CI stability. that'll require several steps
17:07:22 <ruhe> we can discuss them one by one
17:07:33 <ruhe> #topic murano-ci
17:07:42 <sjmc7> just to interject - i think new features are ok as long as they're supported by tests
17:08:06 <ruhe> right
17:08:08 <gokrokve> sjmc7: +1
17:08:27 <btully> sjmc7: +1
17:08:38 <ruhe> first step for murano-ci is to add real deployment tests and make it voting
17:08:52 <ativelkov> agree, but it worth investing time to cover the existing features rather then add new ones
17:09:03 <ruhe> sergey murashov is working on these tests
17:10:06 <ruhe> dteselkin and IgorYozhikov are responsible for stability of underlying murano-ci infrastructure
17:10:40 <sjmc7> what is missing from the CI tests now?
17:11:03 <ruhe> sjmc7: passing deployment test. deployment of a simple service
17:11:22 <ruhe> the question is: should we allow any major changes (even big bug-fixes) before murano-ci stuff is done?
17:11:29 <sjmc7> ruhe - yes, i think we have to
17:11:53 <sjmc7> unless the CI tests will be ready very soon
17:12:22 <ruhe> but, with a strong requirement to have sufficient UT coverage. right?
17:12:29 <sjmc7> yes
17:12:35 <sjmc7> i don't want to stall development
17:12:36 <katyafervent2> hi
17:12:55 <sjmc7> i want to improve quality as we move forward
17:13:00 <ruhe> any objections on this plan?
17:13:34 <tsufiev> ruhe, still no idea what to do with dashboard
17:13:48 <ruhe> tsufiev: it deserves another topic
17:13:54 <tsufiev> ok
17:14:11 <ruhe> #agreed make murano-ci stable. while we're working on that we'll allow changes, but with a strong requirement to have sufficient UT coverage
17:14:48 <ruhe> also i wanted to discuss our plan to improve UT coverage of existing code
17:14:54 <ruhe> #topic murano UT coverage
17:15:05 <ruhe> please take a look at line 26
17:15:20 <sjmc7> of?
17:15:33 <tsufiev> sjmc7, i suppose https://etherpad.openstack.org/p/murano-testing
17:15:35 <ruhe> of https://etherpad.openstack.org/p/murano-testing
17:15:45 <ruhe> it's 28 now :0
17:16:00 <ruhe> Improve UT coverage. We can split components
17:16:17 <sjmc7> ok. i'm happy to take the API section
17:16:31 <sjmc7> but in my experience it's much better to add tests for bug fixes/changes than to retroactively write them
17:16:49 <ruhe> sjmc7: great. please add your name to the corresponding section
17:17:01 <ruhe> sjmc7: right, be we still need a base to add new tests
17:17:12 <sjmc7> yup, agreed
17:17:23 <katyafervent2> i can take the client
17:17:28 <ruhe> you've added base for API tests, but we miss engine,pl and db stuff
17:17:33 <sjmc7> yep
17:17:49 <ruhe> katyafervent2: great, please add your name to the corresponding section
17:18:26 <ruhe> are there any other areas of stackforge/murano we didn't cover yet?
17:18:36 <sjmc7> nope
17:18:49 <ruhe> #topic testing of murano-agent
17:18:50 <slagun> dashboard?
17:19:03 <sjmc7> dashboard's further down
17:19:34 <ruhe> apparently we don't have jenkins automation for murano-agent
17:20:05 <ruhe> i'll take an AI to add a patch to infra to enable py26,27,pep8,etc jobs for the agent
17:20:46 <ruhe> also, i'd like to propose an idea to build an image with the agent and test it with our deployemnt tests on each patch-set in the agent
17:21:32 <ruhe> dteselkin, IgorYozhikov, tnurlygayanov: would it be feasible?
17:22:24 <ruhe> that's more of a long-term item for the testing plan
17:22:28 <IgorYozhikov> I believe yes, for example, we could have predefined simple EP for agent tests
17:22:42 <IgorYozhikov> EP-execution plan
17:22:43 <dteselkin> well, it's possible to prepare a base image and install only agent - it looks doable
17:23:17 <ruhe> ok. i've added you both to this action item
17:23:22 <dteselkin> ok
17:23:29 <ruhe> anythin else on murano-agent?
17:23:59 <ativelkov> when do we expect to have cloud-init installable agent?
17:24:44 <ruhe> ativelkov: not in j2 definitely
17:24:53 <sjmc7> i've done it, but only from a build i put in swift
17:25:12 <ativelkov> what about j in general?
17:25:21 <IgorYozhikov> at 1st we should decide which distros we will support or any another way for install
17:25:35 <ruhe> ativelkov: first step is to figure out how this should be done
17:26:07 <ruhe> i mean, we need to have a good blueprint for this
17:26:10 <sjmc7> this seems like a longer-term goal then
17:26:17 <ativelkov> Yep
17:26:59 <ativelkov> I  just want to make sure that we will not have to rewite the UTs when we make the agent installable
17:27:46 <ruhe> ativelkov: i don't think we'll need to rewrite a lot of things. cloud-installable agent is a different feature, it deserves it's own set of tests
17:28:12 <ativelkov> ok
17:28:20 <ruhe> #topic python-muranoclient  testing
17:28:53 <ruhe> katyafervent2 already signed up to setup a base for UT for the client
17:30:02 <ruhe> also, sergey murashov is writing the deployment tests using the client, which will give us a good coverage of the client too
17:30:32 <ruhe> basically, we will have the same job to test changes in murano and in client
17:30:48 <katyafervent2> i have a review has first tests in the client
17:31:08 <ruhe> katyafervent2: which means that your patch has a good chance to pass the review :)
17:31:22 <akuznetsova_> I prepared a suite of functional tests for python-muranoclient
17:32:27 <katyafervent2> I guess I'll take unit testing and it shiuld be different, right
17:32:56 <akuznetsova_> yes
17:33:10 <ruhe> akuznetsova_: those tests might not be needed if we find a way to run deployment tests on changes in python-muranoclient. let's discuss that in more details later with sergey murashov and tnurlygayanov
17:33:11 <katyafervent2> ok lets move on
17:33:29 <ruhe> #topic murano-dashboard
17:33:50 <akuznetsova_> ruhe: what about python-muranoclient + dashboard ?
17:34:26 <sjmc7> i'm not sure how much coverage we can reasonable do with unit tests for the dashboard
17:34:35 <sjmc7> there are some things that can realistically be unit tested
17:34:48 <ruhe> akuznetsova_: good question. maybe we can run selenium tests against changes in python-muranoclient
17:35:17 <sjmc7> yeah - i think those tests will be much more useful
17:35:40 <tsufiev> ruhe, perhaps there had to be an AI for me to investigate how unit testing is done in Django
17:35:50 <sjmc7> we should see what we can add. even horizon seems to have pretty poor test coverage
17:36:32 <tsufiev> i guess, Django developers should have invented some approach to unit tests
17:36:32 <sjmc7> tsufiev - drupalmonkey has been looking at this a little bit, you could discuss with him
17:36:40 <sjmc7> yes, django has a good test framework
17:37:00 <ruhe> the question i had for everyone, especially to people who work on the dashboard - what should we do with the dashboard/selenium tests? that's reflected in the etherpad
17:37:05 <tsufiev> sjmc7, is he here?
17:37:21 <sjmc7> yes tsufiev. i can be part of that conversaton too
17:37:37 <sjmc7> ruhe - i'd like ot make those reliable and voting
17:37:47 <tsufiev> ruhe, selenium tests do their job now
17:38:12 <tsufiev> sjmc7, +1
17:38:17 <akuznetsova_> ruhe: i think we should leave them, but i will try to make easier as possible
17:39:00 <tsufiev> sjmc7, then perhaps we could start with writing some tests using that Django testing framework?
17:39:04 <sjmc7> yup
17:39:23 <sjmc7> but again, i don't know how useful it will be because horizon's such a thin layer on top of the service APIs
17:39:24 <ruhe> so, we keep the tests, we make this job voting (or at least treat it as voting). person sending a patch to the dashboard is responsible to make the tests pass. is that correct?
17:39:41 <sjmc7> yes ruhe, that makes sense to me
17:40:06 <sjmc7> i think 'treat as voting' is sensible
17:40:12 <tsufiev> sjmc7, what's true for horizon, not entirely true for muranodashboard
17:40:35 <tsufiev> sjmc7, i mean, in some places it's thicker than horizon
17:40:41 <ruhe> tsufiev: btully: did you have a chance to run/debug selenium tests locally?
17:41:05 <ruhe> do we need to have some dev docs to explain how to run these tests?
17:41:09 <tsufiev> ruhe, no, never asked myself that question :)
17:41:18 <btully> i did not, no. if there's documentation i'd be happy to start doing that
17:41:24 <tsufiev> but definitely such docs would be very helpful
17:41:57 <akuznetsova_> we have one https://wiki.openstack.org/wiki/Murano/TestsDocumentation but it is outdated
17:42:03 <sjmc7> it's not trivial running those tests locally
17:42:06 <akuznetsova_> I will update it
17:42:10 <ruhe> #action akuznetsova_ create a document explaining how to run, debug and fix selenium tests
17:42:52 <ruhe> perhaps the best place to place these docs would be developer documentation we keep under project 'murano' and publish to murano.readthedocs.org
17:43:06 <akuznetsova_> ok
17:43:27 <ruhe> #agreed we keep selenium tests, make them voting eventually. person sending a patch to the dashboard is responsible to make the tests pass
17:43:49 <ruhe> now to the general testing approach
17:44:02 <ruhe> #topic general testing policy
17:44:11 <ruhe> that' t the top part of the document
17:44:13 <btully> i've used selenium in the past along with phpunit for automated testing. once i get up to speed on how murano is using it (along with the python syntax) i'd be happy to help with that side of things
17:45:12 <ruhe> we've said several times that a patch should have sufficient test coverage, but different people might understand this differently
17:45:34 <ruhe> sjmc7 put these requirements on paper
17:45:59 <ruhe> that's line #8 of https://etherpad.openstack.org/p/murano-testing
17:46:20 <ruhe> Tests verifying correct behavior
17:46:21 <ruhe> Tests verifying exceptional behavior
17:46:21 <ruhe> Sufficient assertions to ensure that tests are accurately assessing functionality
17:46:45 <ruhe> any additions or comments?
17:47:55 <ruhe> ok
17:47:59 <tsufiev> ruhe, tests veryfying correct behaviour should be mandatory
17:48:24 <tsufiev> but there are 1000+ ways of making an error
17:48:57 <sjmc7> so test the ones you can think of, and when one you didn't pops up in production, test it and then fix it
17:49:39 <ruhe> sjmc7: that's a good motto
17:50:09 <ruhe> any other ideas/suggestion for this big topic?
17:50:37 <ruhe> #topic Other plans for juno-2
17:50:58 <ruhe> any other plans, blueprints, ideas for juno-2?
17:51:11 <ruhe> #link https://launchpad.net/murano/+milestone/juno-2
17:51:11 <sjmc7> we may have some for next week, not right now
17:51:55 <btully> from a UI perspective, on the environments page when there are multiple envs
17:52:15 <btully> there are checboxes added to each row
17:52:33 <btully> but you can't act on multiple envs at the same time
17:52:43 <btully> so checkboxes should be removed
17:52:52 <btully> (not sure if that's been captured previously)
17:53:10 <ativelkov> I would prefer to return a way of deleting multiple envs
17:53:10 <tsufiev> btully, sometimes they help to delete multiple environments at once
17:53:20 <akuznetsova_> btully: it was, we had multiple deletion
17:53:29 <ativelkov> tsufiev: it does now work now. We had it before and need to return
17:53:33 <btully> right but you can't delete multiple
17:53:37 <btully> ahh ok
17:53:44 <tsufiev> ativelkov, there was a discussion with stanlagun about that
17:54:37 <btully> so either add back ability to delete multiple or remove checkmarks :)
17:54:41 <tsufiev> he said that should not be available, because environment removal can be very complex operation, thus possibly making impossible to toss them into batch
17:54:41 <slagun> I was against
17:55:48 <ativelkov> This needs a separate discussion
17:55:54 <tsufiev> we should make a separate etherpad for that :)
17:56:03 <slagun> It is very expensive operations in terms on how many things you can accidentally loose. Especially data on database servers etc
17:56:23 <ativelkov> slagun: well, heat is able to delete stacks in a batch
17:56:55 <ruhe> so, who's going to create an etherpad? where should we discuss next step? irc, ML?
17:56:58 <slagun> heat is a low level tool. We are building user-friendly tool
17:57:35 <ativelkov> Environments are suppoaed to be isolated
17:57:37 <tsufiev> slagun, that makes sense if you're going to ask a lot of questions during environment removal
17:57:48 <tsufiev> is it really the case?
17:57:48 <ativelkov> so, I don't see any problems if we delete more then one at once
17:57:51 <sjmc7> separate discussion! :)
17:58:50 <ativelkov> agree :)
17:58:57 <tsufiev> +1
17:59:01 <ativelkov> we are almost out of time anyway
17:59:03 <ruhe> btully: please bring this topic to discussion in an etherpad or ML thread
17:59:36 <ruhe> let's end this meeting before we run out of time. thanks everyone!
17:59:43 <ativelkov> thanks!
17:59:45 <ruhe> #endmeeting