15:00:16 #startmeeting manila 15:00:17 Meeting started Thu Apr 9 15:00:16 2015 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:18 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:20 The meeting name has been set to 'manila' 15:00:35 hello all 15:00:35 hi 15:00:38 hi 15:00:38 Hi 15:00:40 hi 15:00:40 Hi 15:00:40 hi 15:00:41 hi 15:00:45 #agenda https://wiki.openstack.org/wiki/Manila/Meetings 15:00:46 hi 15:01:15 #topic Kilo-RC1 status (bugs) 15:01:35 so first of all, I'm happy that we've found and closed so many bugs since K-3 15:01:40 #link https://launchpad.net/manila/+milestone/kilo-rc1 15:02:01 however I think it's past time for a RC1, and we've got a few left to close 15:02:14 hi 15:02:14 so I'm going to cover them now 15:03:00 I'm looking retarget bugs out of Kilo soon 15:03:38 so please be responsive to reviewer feedback if you're a submitter, and if you're a reviewer, please focus on the 5 remaining bugs 15:03:49 Failed to allow-access for native_glusterfs 15:03:52 https://bugs.launchpad.net/bugs/1417352 15:03:54 Launchpad bug 1417352 in Manila "Failed to allow-access for native_glusterfs" [Medium,In progress] - Assigned to Csaba Henk (chenk) 15:03:56 https://review.openstack.org/#/c/171302/ 15:04:10 this one had some serious issues discovered on code review 15:04:18 at least one problem still remains 15:04:33 bswartz: yes I see my bad left in there by mistake 15:04:49 (probably forgot to save the edited file) 15:04:56 I'll correct in a minute 15:05:20 unit tests should have caught the issues before it got to code review -- so a reminder to everyone: cover your exception code paths with unit tests! 15:05:39 and of course, run the unit tests before pushing upstream 15:05:43 it only takes like 2 minutes 15:06:03 bswartz: don't want to nitpick but this one would not be caught by a test :) 15:06:48 arguably 15:07:10 csaba: what kind of test do we need to discover it? 15:07:37 3rd party vendor tests? 15:07:44 "%(message)" % {"message":"foo"} is syntactically OK... hacking could detect probably 15:07:50 csaba: simple substitution verification is enough 15:07:53 the tests need to exercise the exception code path and validate that the correct exception is thrown 15:08:15 it's true that this bug would only result in a bad message getting logged, which typically isn't validated by unit tests 15:08:24 I think you'll have to write test to check the output of every message to detect this one 15:08:31 vponomaryov: why is it not done? 15:08:33 I like the idea of a hacking check 15:08:43 I don't think that is covered by anyone 15:09:07 csaba: ? why is it not done by your tests? 15:09:11 adding a hacking rule sound good 15:09:17 anyways, I think that bugfix will be okay -- it's just a case study for why unit tests are valuable 15:09:38 can I go to the next one? 15:09:52 share server can not be deleted if hung in transitional state 15:09:52 vponomaryov: AFAIK we don't test message text, it's a job for hacking IMHO, which would work regardless of unit test coverage 15:09:56 https://bugs.launchpad.net/bugs/1434563 15:09:58 Launchpad bug 1434563 in Manila "share server can not be deleted if hung in transitional state" [Medium,In progress] - Assigned to Igor Malinovskiy (imalinovskiy) 15:10:02 https://review.openstack.org/#/c/170528/ 15:10:32 valeriy have you read u_glide's responses? 15:10:35 I agree that this should be a hacking check, no one has unit test for every message 15:10:52 this one seems almost ready 15:10:55 bswartz: not yet 15:10:58 okay 15:11:05 I think it's just waiting on you 15:11:11 no need to discuss here 15:11:15 ok 15:11:16 ok 15:11:18 NetApp cDOT driver is too strict in delete workflows 15:11:22 https://bugs.launchpad.net/bugs/1438893 15:11:23 Launchpad bug 1438893 in Manila "NetApp cDOT driver is too strict in delete workflows" [Medium,In progress] - Assigned to Clinton Knight (clintonk) 15:11:25 https://review.openstack.org/#/c/171789/ 15:11:31 this one was recently pushed up 15:11:53 reviewers please take a look at it 15:12:00 I have a couple questions from Valeriy that I just started looking at. Further reviews are welcome. More details are in the Launchpad bug. 15:12:30 Response 404 NotFound for listing share types request 15:12:33 https://bugs.launchpad.net/bugs/1441602 15:12:34 Launchpad bug 1441602 in Manila "Response 404 NotFound for listing share types request" [Medium,In progress] - Assigned to Valeriy Ponomaryov (vponomaryov) 15:12:46 so I'd like to talk about this one 15:12:58 there is an issue which cases tempest to fail transiently 15:13:04 I reproduced it on lab and found out that it not only returns bad response but also hurts DB records 15:13:09 we've needed to do too many rechecks in the last week or 2 15:13:30 vponomaryov is investigating and trying to fix 15:13:36 if anyone wants to help, that would be very welcome 15:13:52 so, even if we do not fit Kilo with it, it will be backport fix definitely 15:14:14 because it leads to data loss 15:14:19 vponomaryov: we should hold RC1 for a DB corruption issue 15:14:20 vponomaryov recently pushed 4 identical patchsets to try and test a fix with a large number of job runs -- in case anyone wondered about those 15:14:55 yes I think this is an RC blocker, so I'd like to ask everyone who can to help out 15:15:02 DB affected with deletion of all private share types 15:15:36 vponomaryov: do you know for sure which commit introduced the issue? 15:15:38 when concurrent requests for type-list and list of some type access projects are performed in parallel 15:15:58 xyang1: +1 -- indeed, in Cinder we've had pushback on reviews where we did asserts matching message content (on the grounds the test becomes too fragile) 15:15:59 bswartz: appearance of share type access functionality 15:16:07 bswartz: I believe Cinder has this bug too 15:16:15 bswartz: it is not tested there at all 15:16:25 bswartz: by tempest, as we had for some time 15:16:36 bswartz: before addon of tests 15:16:40 okay so it's not a problem caused by the new tests -- the problem already existed and the new tests just revealed it? 15:17:16 yes 15:17:20 tbarron: I got questions on one patch because I added a unit test for a message:). But usually I try to unit test message for exceptions 15:17:22 If we're sure of that then we have to hold up the RC 15:18:02 bswartz: would be great to have it till monday 15:18:10 "hold up" 15:18:25 for a data corruption bug we'll hold the RC as long as we need to 15:18:45 the risk is that the later the RC is cut, the less testing we can expect before the actual release 15:18:57 bswartz: how long can we hold it up? 15:18:58 because there are always some who don't start testing until there is an RC 15:19:25 xyang1: technically, April 30 15:19:43 bswartz: ok, GA date 15:19:43 however that would be terrible 15:20:05 bswartz, right, we want to cut it early so that people will do more testing 15:20:05 from my perspective the sooner we get RC1 the better 15:20:52 okay we don't need to hash this one out any more 15:20:59 #topic XML API removal 15:21:02 bswartz: here is DB error that leads to data loss - #link http://paste.openstack.org/show/201360/ 15:21:43 vponomaryov: would be nice to collect theses infos in the bug report 15:22:16 yeah, let's compile as much as we can in the LP bug, so if others want to help they can come up to speed on what's been discovered so far 15:22:28 Remove Limited XML API Support from Manila 15:22:33 https://bugs.launchpad.net/bugs/1440782 15:22:34 Launchpad bug 1440782 in Manila "Remove Limited XML API Support from Manila" [Low,In progress] - Assigned to Igor Malinovskiy (imalinovskiy) 15:22:36 https://review.openstack.org/#/c/170904/ 15:22:58 so many of you know that XML hasn't worked in the REST API for a very long time 15:23:11 we've wanted to remove it, but we never got around to it until recently 15:23:28 I think u_glide's patch is pretty low risk, and it's better to get it in Kilo rather than Liberty 15:23:42 but I wanted to ask the rest of you if you have any concerns about a change like this 15:24:38 +1 for removal in Kilo 15:24:49 +1 from my side as well 15:24:50 u_glide: are you addressing the few reamining issue with this to make it mergeable? 15:24:55 bswartz: I'm OK with it. It's a sizable change, but given our test coverage, I think the risk is low. 15:25:05 bswartz: I've added my comments in the patch that this seems to be a very big change at this stage, but if most people are okay, I'm fine 15:25:16 okay 15:25:23 There wasn't any tempest tests for xml?... +1 for removal 15:25:29 i think since rc1 has not been released yet, it would be ok. 15:25:29 bswartz: yes, I will upload new patch in next 15 minutes 15:25:38 okay this seems clear then 15:25:41 mkoderer: they were, long-long time ago 15:25:47 we'll keep it in 15:25:47 mkoderer: in far-far galaxy =) 15:25:53 vponomaryov: :) 15:26:06 u_glide1 will fix any bugs if this patch resulted in any:) 15:26:07 but keep reviewing to make sure it's good before merging 15:26:14 +1 15:26:21 lol 15:26:55 okay that's it for Kilo-related stuff 15:27:33 I think we'll be okay with everything except for https://bugs.launchpad.net/bugs/1441602 15:27:34 Launchpad bug 1441602 in Manila "Response 404 NotFound for listing share types request" [Medium,In progress] - Assigned to Valeriy Ponomaryov (vponomaryov) 15:27:57 so vponomaryov please share information as you learn it and don't hesitate to ask for help if you need it 15:28:33 on to liberty stuff 15:28:39 #topic Specs repo 15:29:01 so many other openstack projects have been experimenting with specs repos for the last 2 releases 15:29:29 I think that people generally like them, but they are not without problems 15:29:52 It's because of the problems that I've held off on introducing them for Manila 15:30:04 but more people have been asking so I'd like to open up the discussion 15:30:50 the main value of specs repos is that they create a formalized way to give feedback on design proposals, and they create a template to follow that forces people to consider things they might otherwise ignore when proposing a new feature 15:30:50 bswartz: I personally like to have more details for a feature 15:31:20 so the manila bp that I saw wasn't giving me all the information I would need 15:31:28 have specs reviews would solve that 15:31:37 also they create a formal record of what designs were approved, and a process for those approvals 15:31:48 it would be good to have it for when we need it for big features but most patches wouldn't use it 15:31:52 I have 2 concerns 15:32:02 bswartz: I think it is good for Manila to use the same process that other projects are using 15:32:08 bswartz: would this supersede lp bps or would these be used together? 15:32:32 csaba: lp bps and specs are usually used together 15:32:37 but it's not a must 15:32:50 the first is that a spec only allows you to discuss a feature at the hypothetical level -- it's common for unexpected issues to come up during implementation that may require changes to the design 15:33:02 csaba: you have a have a bp for a new feature, but spec is not required for every bp item 15:33:18 and also some people have a hard time reading a spec and understanding what it means -- whereas a code submission is unambiguous -- you can download and run it 15:33:23 csaba: so you can't have a spec without a bp 15:33:40 mkoderer, xyang1: thanks 15:34:02 bswartz: reading a spec is much easier for ops people for instance 15:34:15 mkoderer: +1 15:34:21 the other issue with specs is that they get out of date quickly -- it's common for other teams to approve specs, and then implement something different from the spec, and to never go back and fix the spec to match the implementation 15:34:32 bswartz: so yes.. code is easier to read for developers.. but if you want to get feedback from "customers" specs are really helpful 15:34:43 other concern is that often design specs are not read and the design gets clobbered later anyway 15:34:56 So it is a tool to get agreement on design, but a faulty one. 15:35:12 Still worthwhile for big feature collab 15:35:30 so IF we do decide as a team to start using a spec repo, we will need to create a policy on when/if approved specs should be updated to match the implementation 15:35:39 mkoderer: that's true if the specs are up-to-date. if the information is wrong, it's sometimes even worse than no information. 15:36:15 toabctl: yeah.. I agree.. outdated specs are an issue.. 15:36:48 Yes, spec should be updated after the code is merged 15:36:53 there is a related problem for people trying to use a specs repo as a source-of-truth which is that specs are a bunch of standalone documents, which can be hard to read and understand as a whole 15:37:11 toabctl: +1 15:37:19 I would argue that a well written architecture guide would be more valuable to and end user than a bunch of specs 15:37:33 or a deployer 15:37:39 bswartz: +1 15:37:47 bswartz: +1 15:37:57 specs are better used to agree on a design for future work than as doc for existing work 15:38:07 so we have to understand what problem we're trying to solve by introducing specs 15:38:08 bswartz: but the guide will be written after implementation? 15:38:27 mkoderer: unfortunately that's the only choice 15:38:32 chicken egg problem 15:38:50 it would be weird to edit the architecture document before there is working code 15:39:30 ideally there's a spec to discuss, then an implementation and then (or in parallel) documentation which should make the spec for OPS obsolete 15:39:31 it depends how much content is necessary to understand what the new feature/bp is going to do 15:40:08 bswartz: there's a risk that we will be stuck in spec:) 15:40:14 i have found one of the best times to propose new features is at a design summit or a hackathon, for a Q&A session 15:40:14 toabctl, +1 15:40:15 :) 15:40:53 toabctl: +1 15:41:11 my feeling is that if we do specs, we either have to keep them up to date forever, or there has to be an understand that their immediately obsolete when they're agreed to, in which case we'd need to diligent about updating the architecture guide after a feature merges 15:41:24 so if we link a spec to the documentation later on everyhing shouel be fine 15:41:30 understanding( 15:41:43 up-to-date-forever: -1 15:42:02 bswartz: we should allow code to be submitted and reviewed before a spec is approved, otherwise, we could be stuck:) 15:42:02 mkoderer: +1 15:42:17 specs are still good for hashing out ideas in a international-multi-timezone community like we have 15:42:31 xyang1, +1 If reviewers don't need a spec. Great. 15:42:45 xyang1: oh I'm not in favor of blocking implementation until the spec is agreed to at all 15:42:47 +1 for luis ;-) 15:43:00 I love the idea of spec and code in parallel 15:43:01 xyang1: so other teams do not accept code without a spec 15:43:18 Sometimes a feature deserves a spec rat hole because it is controversial. 15:43:19 xyang1: but for me this is too much 15:44:09 so do we agree that specs are helpful? :) 15:44:17 developers just have to understand that they're risking wasting their time implementing something before the spec is agreed to because the spec could get -2'd or the significantly changed after discussion 15:44:41 bswartz: I mean that a usual risk for every commit 15:44:48 but I think it's extremely valuable to propose a new feature and have a working prototype to demonstrate the idea 15:44:48 bswartz: +1, but if a spec is needed for a new feature, the final code shouldn't merge if it doesn't match the spec. 15:45:10 cknight: that will not work:) 15:45:17 cknight: it is the other way around 15:45:28 your spec has to match your final code 15:45:51 do we really have issue that will be fixed with appearance of specs repo? 15:45:51 yeah I agree with xyang1: the code review is the decider of how the feature turns out 15:45:52 I've found out from my own experience there is no way to completely match an approved spec 15:45:52 test -- can you guys see this? 15:46:04 xyang1: OK, either way. :-) 15:46:08 people keep come up with new ideas 15:46:11 lpabon, test passed 15:46:16 cook, thanks 15:46:17 lpabon: yes 15:46:19 cool 15:47:02 my $0.02 is that specs are.... ok.. and most of the time they talk about something that the code never does 15:47:19 So reviewer can -1 to say 'fix your spec' before merge -- but after that expect the spec to age and get obsolete 15:47:19 specially in open source projects 15:47:26 vponomaryov: the issue is that large new features are hard to discuss without some kind of document -- in the last we've used wikis 15:47:31 in the past* 15:47:42 markstur_: I don't think that is efficient 15:47:54 people will come up with new ideas after you fix the spec 15:47:58 it is never ending 15:48:06 we have to allow the code to be merged first 15:48:11 big +1 for neverending =) 15:48:14 and update spec as soon as possible 15:48:37 yeah -- I feel strongly that unless we commit to updating the specs forever, people have to accept that they're instantly obsolete and the code becomes the source of truth 15:48:53 wiki page has history as well 15:48:54 bswartz, i do not think that is going to work in a community based environment 15:49:10 vponomaryov: a wiki page does not have a "review" process 15:49:44 wikis actually do have a "discussion" function but I don't think it works too well 15:49:49 a spec should be enough of the architecture to understand how to read the code (commented code that is) 15:50:15 I don't think wiki is good, it is too easy for anyone to make changes 15:50:34 bswartz: +1. If the code can merge first, there is much less incentive to update the spec later. Of course the spec becomes obsolete, but if there was never a point in time the spec matched the code, the spec has little value after the initial discussion. 15:50:36 xyang1: do not truct people ? =) 15:50:46 s/truct/trust/ 15:51:06 vponomaryov: trust but verify :-) 15:51:13 cknight: I think the value is mostly that initial discussion -- after that happens the spec loses its value rapidly 15:51:14 vponomaryov: we have code review because we never trust people :) 15:51:18 vponomaryov: if it is people in this group, that is fine, but it could be modified by anyone in the world:) 15:51:37 vponomaryov: I had some trouble keep correcting one particular wiki:) 15:51:54 xyang1: same about gerrit 15:52:02 xyang1: anyone can add patch-sets 15:52:08 vponomaryov: there is a review process 15:52:09 bswartz, that is why i think bringing up new features to design summits, irc meetings, or hackathons are the best method... and not necessarily a long spec 15:52:10 I wrote several "specs" about the manila driver modes and network plugins 6 months ago -- they're all very wrong now 15:52:24 at least we can see before it is changed:) 15:52:33 bswartz, go fix'em 15:53:01 markstur_: fix them tomorrow, and every day =) 15:53:02 lpabon: problem is things usually change after the design summit:) 15:53:16 Seems like we'll never agree to have perfect specs 15:53:22 markstur_: I should, but it probably makes more sense to write the current truth into an architecture document rather than a bunch of disconnected specs 15:53:28 xyang1, The developer is just bringing up an idea.. which gets discussed with others 15:53:36 I don't see why we cant try them for additional reviewer info 15:54:08 lpabon: right, but it could be a complete re-design, the benefit of spec is it keeps a history 15:54:08 okay well we clearly have a bunch of different opinions on this topic, which is good 15:55:00 xyang1, a complete re-design is a great thing..because it is the community working together, and not a single person (or company) working in a cave then coming out with a spec 15:55:05 bswartz next topic? 15:55:24 bswartz, +1 on docs instead of specs for implemented things 15:55:26 well I think we need to figure out how to decide this one 15:55:28 lpabon: in a cave - =) 15:55:29 perhaps a vote next week? 15:55:33 vponomaryov, :-) 15:55:38 bswartz: +1 15:55:45 bswartz: +1 15:55:57 it probably deserves an ML post too 15:55:59 bswartz: +1 15:56:00 I'll write that up 15:56:10 in the last few minutes, 1 more topic 15:56:16 #topic Core team members from the same company 15:56:17 anyways, my message is -- make it a community effort by bringing ideas up as early as possible 15:56:42 we now have a core team including 2 mirantis developers, 2 netapp developers, and 1 emc developer 15:57:03 some have raised concerns about the concentration of power in 2 companies 15:57:20 there is a diversity concern too -- but we can solve that by continuing to grow the core team 15:57:21 lpabon: "as early as possible" may not happen:). ideas will keep coming till GA 15:57:23 (and i worked for two of them in my previous life) 15:58:01 I'd like to propose, that in the interest of fairness, we create a rule that the two +2s before a +A on any patch cannot come from a single company 15:58:11 bswartz: contentration of power in two companies because other not active? 15:58:15 +1mil 15:58:20 bswartz: +1, I've been adhering to this already. But I think there should be some leeway for things like emergency gate fixes. 15:58:20 bswartz: having some more cores from distributors would be quite helpful 15:58:32 bswartz: at the end someone need to package Manila and give support :) 15:58:46 cknight: that leeway already exists -- any of us and break the rules and +2A anything, but we REALLY avoid that 15:58:56 cknight, +1 15:59:01 in fact I don't think I've needed to use that power in more than 6 months 15:59:05 bswartz: ok, thanks 15:59:52 vponomaryov: it's true that developers have to be doing the hard work of doing reviews to be considered for the core team 16:00:04 so I think it is good to have diversity, but we also should not make a hard rule saying no more cores from the 3 companies 16:00:07 and up until recently, the set of people doing that was small 16:00:11 bswartz: so it is problem not of these two companies 16:00:21 bswartz: that are "active" 16:00:30 but I'm happy to see many new people doing reviews, so we can continue to grow the core team 16:00:55 vponomaryov: the concern is that netapp could propose something nobody likes and shove it thorugh with a +2 from both me and cknight 16:01:04 or mirantis could do the same 16:01:16 the diversity concern is entirely different and we don't have time to get into that today 16:01:40 it's important to me that we strive for diversity and fairness 16:02:00 bswartz: sure. I think it is good to require another core from a different company to +2 as much as possible 16:02:14 bswartz: are you looking to implement this measure based on some prior actual problem? Or the possible future occurrence of such? 16:02:18 \leave 16:02:18 this rule will be unofficial unfortunately, because I don't think we have an official set of bylaws anywhere 16:02:51 okay we're over our time 16:02:52 thanks all 16:02:56 #endmeeting