17:02:13 <hogepodge> #startmeeting refstack
17:02:14 <openstack> Meeting started Tue Nov 28 17:02:13 2017 UTC and is due to finish in 60 minutes.  The chair is hogepodge. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:18 <openstack> The meeting name has been set to 'refstack'
17:02:29 <hogepodge> #link https://etherpad.openstack.org/p/refstack-meeting-17-11-28 Agenda
17:03:22 <guinm> o/
17:05:15 <hogepodge> Pretty light attendance today
17:05:27 <hogepodge> Guess we'll get started.
17:05:46 <hogepodge> #topic RefStack work
17:06:01 <hogepodge> #link https://review.openstack.org/#/c/523323/1 Apply pep8 to wsgi
17:06:14 <hogepodge> This is more of a meta issue.
17:06:55 <hogepodge> A few months ago there were concerns raised about patch automation.
17:07:53 <hogepodge> My policy for RefStack, unless there are major objections to it, are that automated patches will be rejected unless they have been approved by the TC.
17:08:23 * guinm nods
17:08:42 <hogepodge> If an active team member finds value in the patch and wants to resubmit it, that's a way it can be landed with the original submitter as a co-author on it.
17:09:47 <hogepodge> I want to discourage what it regarded as bad behavior by community members, but don't want to discourage contributions that have merit.
17:11:01 <hogepodge> #link https://review.openstack.org/#/c/507695/ Disable Anonymous Upload
17:12:16 <hogepodge> This patch is still in flight. Once it lands we can do the database update for marking test results that have been used in validation.
17:13:14 <hogepodge> #link https://review.openstack.org/#/c/506826/ Subunit API Spec
17:13:29 <hogepodge> This landed, thanks for mguiney for her work on it and all of the great reviews.
17:13:53 <hogepodge> Next steps are implementation, so looking forward to that.
17:14:17 * guinm is looking forward to getting started
17:14:53 <mguiney> (whoops forgot to wear the correct hat)
17:16:26 <hogepodge> #link https://review.openstack.org/#/c/521343/ RefStack API Documentation
17:16:33 <hogepodge> workd being done as time permits.
17:16:58 <hogepodge> mguiney: if we're the only people here I think we can just post the agenda links, since we covered all of this in person yesterday
17:17:23 <hogepodge> mguiney: sound good to you? I'd rather we get back 40 minutes of our day rather than rehash everything
17:18:00 <hogepodge> #topic refstack-client
17:18:07 <mguiney> sounds good
17:18:09 <hogepodge> #link https://review.openstack.org/#/c/489421/ Tempest Config Generation
17:18:15 <hogepodge> #link https://review.openstack.org/#/c/515494/ Sample Tempest Configuration
17:18:26 <hogepodge> #topic Python TempestConf
17:18:34 <tosky> just a quick update on python-tempestconf: we have a first job (courtesy of tripleo)
17:18:34 <hogepodge> https://review.openstack.org/#/c/522679/ Add zuul layout to run a TripleO job
17:18:45 <hogepodge> hi tosky !
17:18:52 <hogepodge> Excellent, I was wondering about that job
17:18:57 <tosky> (sorry, I joined and got sidetracked)
17:19:18 <tosky> Martin Kopec is still working on devstack and packstack jobs, but he hit the same issue related to authentication
17:19:35 <tosky> he sent an email to openstack-dev, hopefully zuul people will answer (if not, we will ping them)
17:19:43 <tosky> and that's it for now
17:19:48 <hogepodge> What's the authentication issue?
17:20:21 <tosky> this is his email with a detailed description: http://lists.openstack.org/pipermail/openstack-dev/2017-November/124906.html
17:22:29 <hogepodge> Thank you
17:23:07 <hogepodge> If I get a change I'll try to dig deeper too.
17:23:28 <hogepodge> This is all related to this patch.
17:23:31 <hogepodge> #link https://review.openstack.org/#/c/513330/ Add initial .zuul file, devstack and packstack job
17:23:54 <hogepodge> There's also this patch
17:23:57 <hogepodge> #link https://review.openstack.org/#/c/519714/ Add EC2-API support
17:24:02 <hogepodge> Any comments on that?
17:24:30 <hogepodge> It's a WIP
17:25:37 <tosky> yes, and I guess that the last comment from the author still applies
17:25:57 <tosky> added a comment
17:26:09 <tosky> I can try to recheck, it should trigger the tripleo job too now
17:26:39 <hogepodge> ok
17:27:04 <hogepodge> Is there anything else we should know about the project? Upcoming goals you have for it?
17:27:21 <hogepodge> Also, plans for the integration work with refstack-client?
17:29:06 <tosky> as mentioned last time, we will have a meeting to plan the internal refactoring to make it more extensible
17:29:15 <tosky> but I guess that it slipped with Chandan on vacation :)
17:29:20 <tosky> it will probably be resumed next week
17:29:22 <hogepodge> :-)
17:29:40 <tosky> even more about the integration with refstack-client, as he was working directly on that
17:29:41 <hogepodge> great
17:31:29 <hogepodge> Moving on
17:31:32 <hogepodge> #topic Governance
17:31:58 <hogepodge> Active discussion here about modifying governance for interop programs
17:32:00 <hogepodge> #link https://review.openstack.org/#/c/521602/ Clarify testing for interop programs
17:32:28 <mugsie> if there is any feedback for ^, please ping me whenever suits
17:32:40 <hogepodge> Patch was updated this morning, and this impacts our future work.
17:32:45 <hogepodge> hi musgie!
17:32:57 <mugsie> My major problem is that tempest can, and does break the plugin API
17:33:24 <mugsie> and I do not think that is something we can allow for a interop test senario
17:34:17 <mugsie> I think this is one that might drag on a little bit
17:34:18 <hogepodge> mugsie: can you elaborate on how it breaks the plugin api?
17:34:51 <hogepodge> I think it's important to resolve these issues, because from a testing standpoint there has to be a reliable and usable way to run interop tests
17:34:55 <mugsie> they have removed parts of the tempest code base that were marked as "stable" which caused our plugin to stack trace when we tried to load it
17:35:15 <hogepodge> This impacts refstack and interopwg, plus all of the projects currently in and being considered for interop
17:35:20 <mugsie> and I do not discount something like that happening again
17:35:46 <mugsie> of the options in the proposal, #3 is my personal favorite, but #1 is a close second
17:36:17 <hogepodge> this is a big deal, since we're relying on tempest plugins for the additional projects
17:36:58 <mugsie> I don't think we should rely on a plugin interface for some tests and not others - if all projects relyed on it as a plugin, it won't be broken, but if it is only smaller "less important" projects, I think it will be
17:37:35 <hogepodge> I haven't kept up on the mailing list as well as I should, is this being discussed there or in the qa meetings too?
17:37:48 <mugsie> not as far as I know
17:37:49 <hogepodge> We'll be sure to add it as an agenda item for the interopwg meeting
17:38:03 <mugsie> I think it is just on  the patch, and some talk in #openstack-tc
17:38:04 <mguiney> nothing in the last qa mtg, iirc
17:38:10 <hogepodge> ok, just wanted to make sure we weren't missing any information on it
17:39:09 * luzc2 waive hello
17:39:16 <hogepodge> Hi luzc2!
17:39:38 * mguiney waves back
17:39:50 <hogepodge> I'm not in openstack-tc, but I'll join and follow
17:40:26 <hogepodge> I've been a proponent of the plugin model, but it's it breaking for projects then it needs to be a addressed.
17:40:44 <mugsie> it hasn't been discussed in a few days, but it will probably cycle back around
17:40:58 <tosky> the removal of a stable API from tempest is really rare and should follow the deprecation policies, could this be a real problem in the long term?
17:41:42 <hogepodge> We're more involved on the implementation side for refstack, so the interopwg meeting tomorrow will generate much more policy discussion that markvoelker and eglute and the rest of the team can weigh in on.
17:42:06 <tosky> also, the stabilization of tempest is still ongoing (even if almost done), so this may not be a problem anymore in the future
17:42:10 <tosky> what was the change that was previously stable and then broke?
17:42:49 * markvoelker perks up with ears burning
17:43:04 <mugsie> tosky: let me dig it up
17:43:19 <hogepodge> "A wild markvoelker appears!"
17:43:24 <mugsie> it was also the attitute we got when trying to fix it makes me worried.
17:43:55 * markvoelker hasn't had coffee since this morning...so, perhaps "a mild markvoelker appears" is more appropos
17:44:01 <mugsie> https://review.openstack.org/#/c/434304/
17:44:35 <tosky> mugsie: but that was 9 months ago
17:44:43 <tosky> the plan about "what is considered stable" improved over time
17:45:09 <mugsie> sure - but the fact it is happened, and I do not trust it won't agian
17:45:43 <tosky> I do trust it won't again
17:45:54 <tosky> oh, I even +1 the revert commit - I forgot about that :) https://review.openstack.org/#/c/436612/
17:46:14 <mugsie> there is still no gates to ensure that plugins work, and that is just the most recent issue we have had
17:46:31 <tosky> well, there is a test that plugins can be loaded at least
17:46:35 <tosky> thanks to Chandan
17:46:56 <mugsie> yeah, our tests loaded fine, just barfed when we went to run them
17:47:35 <tosky> also, looking at the revert commit, the 3 core who added +2 are still around
17:47:41 <tosky> not sure about the person who disagreed
17:48:22 <mugsie> he was quite adamant about not building a tool for non "core" projects
17:48:39 <tosky> and I disagreed with him in more than one time
17:48:45 <tosky> that view is not shared by other cores
17:49:14 <hogepodge> I think the discussion might be going a bit sideways from the scope of refstack.
17:49:17 <tosky> I understand the concerns, but I consider that event more as one shot, and if tempest want to follow the proper policies for deprecation of stable APIs, something like that can't happen
17:49:31 <tosky> uh, sorry
17:49:32 <mugsie> I also would prefer the increased review quality for our interop tests - we do not have experts on the team to review them
17:49:51 <hogepodge> This is an important issue because there needs to be a policy decision about how the plugin interface is handled, and refstack depends on it.
17:50:01 <mugsie> we have people who know enough to write a test, but nessisarly understand the impact of a change to an existing test
17:50:29 <mugsie> (all the reasons we pushed to have the other refstack tests in tempest)
17:51:01 <mugsie> hogepodge: what caused your preference for plugin?
17:51:13 <tosky> not all tests can go in tempest
17:51:16 * mugsie may be missing other viewpoints
17:51:27 <hogepodge> mugsie: greater freedom for projects and less work for maintaining tests across multiple repositories
17:51:58 <tosky> even few core projects have tempest plugins (neutron, keystone)
17:52:18 <mugsie> yeah - I get that. personally I think it *should* be hard for someone to change a refstack test
17:52:54 <tosky> if you change that with "for someone to break a refstack test", I would be in agreement
17:53:16 <mguiney> ++ that
17:53:21 <hogepodge> mugsie: that's the biggest concern, is that maintaining stability of plugin tests is more difficult
17:53:25 <mugsie> well, if you need to change something in the test, the chances are high you are breaking it :)
17:53:58 <mguiney> not super sure about that one, tbh
17:54:05 <hogepodge> sometimes fixing it to be correct for the domain space
17:54:13 <mguiney> ^
17:54:29 <mugsie> hogepodge: I will say it now, in plugin tests, with no oversight from people who have indepth knowledge of tempest will find it very hard to be stable
17:54:44 <hogepodge> (that was a lot of the initial defcore work... modifying tempest tests to not assume admin and devstackisms)
17:54:45 <mugsie> what is the issue with a separate repo with all refstack tests?
17:54:57 <tosky> the point is that if more than the core projects are covered by interoperability tests, and even for core projects, there is no way to not use plugins
17:55:09 <tosky> so the question is how to prevent breakages, hence my previous statement
17:55:16 <tosky> not saying that it's easy
17:55:25 <luzc2> tosky main issue is maintainance
17:55:30 <hogepodge> mugsie: there is an assumption of maturity for projects and commitment to stability, but your point is taken
17:55:36 <luzc2> I guess the idea is also to increase project awareness of interop, so the reviewers of the projects oppose to change the behavior unless needed
17:56:07 <mugsie> tosky: well, there is a way to avoid plugins - go to #1 option in the proposal
17:56:08 <tosky> luzc2: and a separate repository (a separate plugin), as mugsie proposed, is exactly increase maintainance, because there is a potential of duplicated test
17:56:09 <luzc2> and I will expect as experts on their own project they can judge if a behavior is correctly tested or not
17:56:12 <tosky> and not enough reviewers
17:56:18 <hogepodge> interopwg (like many other projects) is already understaffed, so anything to make it easier for current and future participants to maintain is welcome
17:56:29 <tosky> mugsie: also for non-core projects? That's in conflict with the current focus of tempest
17:56:44 <mugsie> tosky: what is a core project btw?
17:56:54 <mugsie> (like the list of projects)
17:57:18 <hogepodge> Adding a trademark program for a project is a recognition that it's more than just a "non core" project, and that it's important to the community.
17:57:22 <tosky> mugsie: the (sinister?) six: neutron, nova, keystone, glance, cinder, swift
17:57:31 <mugsie> ah, not heat?
17:57:34 <tosky> no
17:57:37 <hogepodge> Three minutes left in the meeting, just as a time check.
17:57:53 <mugsie> hogepodge: ++
17:57:58 <tosky> all the tests for all other projects are outside tempest (and even some tests for the six)
17:58:20 <hogepodge> This was the last agenda item, but is there anything else related that folks wanted to bring up?
17:59:37 <hogepodge> To be continued at InteropWG meeting tomorrow, I believe at 1600 UTC
17:59:51 <hogepodge> #openstack-meeting-3
17:59:55 <hogepodge> Thanks everybody!
17:59:56 * mugsie notes the time
17:59:58 * mguiney nods
18:00:08 <mguiney> have good day yall
18:00:11 <hogepodge> #endmeeting