15:01:09 <krtaylor> #startmeeting third-party
15:01:10 <openstack> Meeting started Wed Apr 15 15:01:09 2015 UTC and is due to finish in 60 minutes.  The chair is krtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:14 <openstack> The meeting name has been set to 'third_party'
15:01:23 <sweston> o/
15:01:28 <asselin> o/
15:01:52 <krtaylor> who is here for third party CI working group meeting?
15:02:02 <krtaylor> hi sweston, asselin
15:02:11 <patrickeast> hey guys
15:02:11 <asselin> hi
15:02:14 <sweston> hi
15:02:19 <krtaylor> hey patrickeast
15:02:19 <mmedvede> o/
15:02:27 <krtaylor> hi mmedvede
15:03:25 <krtaylor> welcome everyone! let's get started, full agenda today
15:03:31 <krtaylor> #link https://wiki.openstack.org/wiki/Meetings/ThirdParty#4.2F15.2F15_1500_UTC
15:03:56 <rhe00> hi
15:04:20 <krtaylor> a quick announcement: Gerrit 2.9 upgrade Saturday May 9, 2015
15:04:26 <krtaylor> hi rhe00
15:04:51 <krtaylor> there is also another potential outage this weekend, Saturday I believe
15:05:09 * krtaylor looks for specifics
15:05:35 <krtaylor> that is for the project rename
15:06:53 <krtaylor> nope Friday
15:07:10 <krtaylor> "Friday, April 17 at 22:00 UTC Gerrit will be unavailable for about 2 hours while we rename some projects and perform some database maintenance."
15:07:40 <krtaylor> ok, any other quick announcements?
15:07:44 <zz_ja> krtaylor, what is expected impact of upgrade to third party cis?
15:08:33 <asselin> I figure shouldn't be "worse" than impact to infra's ci system
15:08:48 <krtaylor> zz_ja, just gerrit being down, connections to gerrit would be impacted
15:08:50 <krtaylor> right
15:09:24 <asselin> you might need to restart some services, e.g. zuul
15:09:58 <zz_ja> fair nuf
15:10:26 <krtaylor> #topic Topics for discussion at Liberty summit in Vancouver
15:10:56 <krtaylor> so,we are proposed as a cross-project session
15:11:04 <krtaylor> here is the topics list:
15:11:09 <krtaylor> #link https://etherpad.openstack.org/p/liberty-third-party-ci-working-group
15:11:24 <krtaylor> please add topics and put your nick next to it
15:11:41 <krtaylor> that needs to be finished in the next coming week
15:11:53 <krtaylor> so we can finalize the agenda if we are selected
15:12:06 <krtaylor> get those topics in there asap
15:12:51 <krtaylor> any questions about our proposed design session topics?
15:14:26 <krtaylor> #topic In-tree 3rd party ci (downstream-puppet)
15:14:36 <krtaylor> anything you'd like to mention asselin ?
15:14:51 <krtaylor> the effort is making good progress
15:15:10 <asselin> just need reviews & more ppl to refactor the other sections
15:15:23 <krtaylor> there is a ton of new patches
15:15:32 <krtaylor> #link https://review.openstack.org/#/q/topic:downstream-puppet,n,z
15:15:39 <asselin> please focus on the one's I submitted :)
15:15:43 <krtaylor> hehheh
15:15:46 <krtaylor> will do
15:16:05 <zz_ja> how do we expect this work to proceed (faster,slower) through the next few weeks with the release cutoff?
15:16:50 <asselin> zz_ja, this is the best time to do it IMHO, b/c of the release cutoff...work is slower in other projects
15:17:02 <asselin> infra doesn't have a cutoff / branchless
15:17:08 <krtaylor> shouldn't be much impact, there should be lots of progress at summit too
15:17:18 <zz_ja> ok good was hoping that was the case asselin
15:17:51 <zz_ja> krtaylor, only so many of us get travel approval; emily will carry our ci torch
15:17:56 <krtaylor> I expect the Friday sprint in Vancouver would be a great opportunity to finish up a bunch of this, with everyone in the same room
15:18:18 <krtaylor> zz_ja, yeah, unfortunate
15:18:24 <zz_ja> I *finally* got a pile of VMs delivered for CI, so now we have something to install on for real
15:18:53 <asselin> #link https://storyboard.openstack.org/#!/story/2000101
15:18:59 <asselin> sign up here ^^ :)
15:19:24 <krtaylor> yes, its a great place to get involved, if you have the extra bandwidth
15:19:28 <asselin> btw, you need to login and change preferences to see all the stories
15:19:54 <asselin> profile --> page size --> 100
15:20:02 <krtaylor> asselin, good to know
15:20:31 <krtaylor> any questions on downstream-puppet work?
15:20:43 <asselin> although seems to not work today. will ask in -infra later
15:21:16 <krtaylor> hm, I'm not seeing a problem
15:21:26 <krtaylor> anyway, lets move on
15:21:32 <krtaylor> #topic Repo for third party tools
15:22:02 <krtaylor> I have not started all the creation needed for this, I was hoping to see more involvement first
15:22:31 <krtaylor> eventhough we (IBM) will need the stackforge repo in order to contribute our tools
15:23:24 <krtaylor> any thoughts on this? do we feel like this will be a create-it-and-they-will-come kind of thing?
15:23:55 <asselin> yes
15:24:50 <sweston> what is the expectation for the tools which would go into this repo?
15:24:52 <krtaylor> we haven't seen that with changing the meeting times, it has not increased involvement despite everyone at summit insisting that was the barrier
15:25:33 <krtaylor> sweston, basically anything that helps CI operators that falls outside of infra services
15:26:40 <krtaylor> monitoring, deployment, logging tools, configurations and plugins to monitoring frameworks
15:26:52 <sweston> nagios?
15:27:27 <krtaylor> sweston, exactly
15:27:41 <krtaylor> sweston, I was just starting to look into that for our environment
15:28:13 <sweston> krtaylor: it's a great tool ... maybe we can put together a puppet manifest for it, to start with
15:28:32 <krtaylor> any thoughts on the repo? would you contribute to it?
15:28:35 <patrickeast> ^ i have something on my backlog to set that up too, if we have a shared set of configs for it for zuul, jenkins, etc that would be pretty awesome
15:28:53 <sweston> yes
15:28:53 <asselin> I think infra uses a different too for that use-case
15:29:13 <krtaylor> the nice thing is that it would allow for gerrit reviews/contributions to these tools and plugins
15:29:49 <sweston> where would it be hosted?
15:30:02 <krtaylor> stackforge
15:30:07 <sweston> ok
15:30:55 <krtaylor> something like stackforge/third-party-ci-tools
15:31:21 <sweston> any other suggestions for other tools?
15:31:56 <krtaylor> sweston, https://wiki.openstack.org/wiki/ThirdPartyCIWorkingGroup#Third_Party_CI_System_Tools_Index
15:31:59 <krtaylor> oops
15:32:03 <krtaylor> #link https://wiki.openstack.org/wiki/ThirdPartyCIWorkingGroup#Third_Party_CI_System_Tools_Index
15:32:18 <krtaylor> links to some existing tools
15:32:33 <asselin> #link http://ci.openstack.org/cacti.html
15:32:43 <krtaylor> it would require us to review and support these tools
15:32:45 <asselin> is that similar to nagios? ^^
15:33:03 <krtaylor> I'd really like for us to agree and commit to using it
15:33:27 <patrickeast> asselin: yea looks like the same kind of thing
15:33:43 <asselin> krtaylor, +1 I think having a place to share would reduce barriers
15:33:43 <patrickeast> i'm in for using the repo
15:34:03 <asselin> and having reviews is a good thing imho
15:34:09 <krtaylor> anyone not willing to support this?
15:34:22 <krtaylor> asselin, agreed
15:34:26 <sweston> I think it's a great idea
15:35:15 <krtaylor> #agreed krtaylor will create  stackforge/third-party-ci-tools
15:35:23 <krtaylor> any objections?
15:35:48 <krtaylor> do we need a vote?  :)
15:36:12 <sweston> might make some of us feel important ;-)
15:36:14 <patrickeast> +1 frp, ,e
15:36:17 <patrickeast> from me*
15:36:29 <sweston> +1
15:36:31 * patrickeast feels important
15:36:34 <krtaylor> #startvote create  stackforge/third-party-ci-tools?
15:36:34 <openstack> Begin voting on: create  stackforge/third-party-ci-tools? Valid vote options are Yes, No.
15:36:35 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
15:36:47 <sweston> #vote Yes
15:36:49 <asselin> #vote yes
15:36:50 <krtaylor> #vote yes
15:36:56 <patrickeast> #vote Yes
15:37:21 <krtaylor> rhe00, ? mmedvede ?
15:37:31 <krtaylor> anyone else?
15:37:50 <krtaylor> last call
15:38:07 <krtaylor> #endvote
15:38:08 <openstack> Voted on "create  stackforge/third-party-ci-tools?" Results are
15:38:33 <krtaylor> ok, that was cool, I've never used that tool before :)
15:38:51 * asselin doesn't see the results?
15:38:57 <sweston> where are the results?  bot is slow
15:38:58 <krtaylor> hm, no results
15:39:25 <krtaylor> well, log is sufficient
15:39:47 <krtaylor> #topic monitoring dashboard
15:40:19 <krtaylor> sweston, can you work with jhesketh to resolve his comments and move that forward?
15:40:42 <sweston> #link https://review.openstack.org/#/c/135170/
15:40:56 <sweston> posted a response already this morning
15:41:08 <krtaylor> yes, I see that! I just refreshed
15:41:18 <krtaylor> excellent, thank you
15:41:29 <sweston> I feel like nobody has read the stackforge radar code
15:41:39 <krtaylor> reviews please everyone
15:42:15 <krtaylor> ok, lets move on to get all the agenda in, we can come back if time
15:42:22 <krtaylor> #topic Highlighting Third-Party CI Service
15:42:35 <sweston> #link https://github.com/stackforge/radar
15:42:43 <sweston> krtaylor: sorry, moving on
15:42:48 <krtaylor> today's highlighted service is HP Storage CI
15:42:59 <krtaylor> asselin, you have the floor
15:43:08 <asselin> krtaylor, thanks
15:43:09 <krtaylor> tell us about what you are testing
15:43:38 <asselin> so I've been working on HP Storage CI to test the hp 3par/storeserv and lefthand/storevirtual cinder drivers
15:43:58 <asselin> I started about a year ago, working on jaypipe's repo.
15:44:14 <asselin> I forked it and fixed it up, adding nodepool support:
15:44:20 <asselin> #link https://github.com/rasselin/os-ext-testing
15:45:18 <asselin> I started out with this basic template for testing cinder drivers
15:45:20 <asselin> #link https://github.com/rasselin/os-ext-testing-data/blob/master/etc/jenkins_jobs/config/dsvm-cinder-driver.yaml.sample
15:45:47 <krtaylor> and I'll add that lots of operators have benefited from your fixed up repo
15:46:04 <asselin> which takes advantage of the devstack-gate hooks to configure the localrc/local.conf to the particular backend
15:46:23 <asselin> krtaylor, yes, but no idea how many...
15:46:37 <asselin> we test iscsi & fc transports
15:47:10 <asselin> for iscsi, originally I had setup nova to create 2 nics on the nodepool instances, one for public, one for iscsi
15:47:45 <asselin> but after I couldn't reproduce it, decided to defer that to the future. still an open issue.
15:48:11 <asselin> so for ci testing we use the same nic for both regular traffic and iscsi traffic
15:48:14 <patrickeast> i might be able to help some with that, we recently switched our nodes over to one management and two iscsi nic's
15:48:23 <patrickeast> seems to be working
15:48:28 <asselin> patrickeast, that would be awesome, thanks
15:49:08 <asselin> for FC, we use pci passthrough.
15:49:09 <krtaylor> asselin, how did you get around it?
15:49:27 <asselin> krtaylor, use a single nic for all ip & iscsi traffic.
15:49:44 <zz_ja> asselin, what's the relationship between your fork and the on-going puppet work?
15:50:26 <asselin> zz_ja, so my fork is basically a copy-paste of the important parts of system-config openstack module
15:51:00 <asselin> the plan is to refactor parts out of system-config, then system-config & my fork can use the same common puppet script
15:51:23 <asselin> eventually I'd like to get rid of my fork and have everything upstream
15:52:22 <sweston> asselin: +1
15:52:33 <zz_ja> asselin, figured that as ultimate goal just wasn't sure where you were today.  we will be building a new from-scratch ci system starting probably next week.
15:52:36 <krtaylor> ++
15:53:04 <krtaylor> exciting times for external CI
15:53:17 <asselin> so the big change I've done compared to what's in github on the -data repo is to create a new devstack-gate-hp project hosted internally
15:53:56 <asselin> it basically mimics devstack-gate in that any changes to the scripts can be run internally vis ci to ensure they pass
15:54:22 <asselin> that's been a huge help to keep the system agile & stable at the same time
15:54:42 <asselin> for example, cinder requires all 'volume' tests to be run
15:54:55 <asselin> we had some issues, so the regex included some exclusions.
15:55:44 <asselin> as I worked on resolving those issues in the devstack-gate-hp branch, I let the internal ci system run the tests to "verify' it before merging it and going live on the upstream ci system
15:56:25 <krtaylor> asselin, how is your environment hosted? do you also have a production and development environments?
15:56:38 <patrickeast> thats pretty cool, we usually just kind of manually try stuff out... having it automated sounds super helpful
15:57:02 <asselin> we have a few systems setup using the same scripts
15:57:24 <asselin> we have a gerrit server hosted internally at hp that I added the devstack-gate-hp project to
15:57:49 <asselin> so just configured the scripts to point to that gerrit event stream instead of openstack's
15:58:10 <krtaylor> right, ok
15:58:13 <asselin> there's a single devstack-gate-hp project that both environments checkout on each job
15:58:25 <asselin> (same as how the jobs checkout a fresh devstack-gate)
15:58:39 <asselin> so keeping as much in common as possible
15:59:09 <asselin> any questions for last minute?
15:59:12 <krtaylor> very interesting
15:59:34 <patrickeast> asselin: for your FC testing did you look into virtual hba's instead of passthrough?
15:59:59 <patrickeast> or i guess in addition to
16:00:00 <asselin> patrickeast, no, not yet
16:00:15 <krtaylor> asselin, thanks for sharing your environment, very good
16:00:26 <sweston> asselin: yes, thanks!
16:00:26 <krtaylor> we should call it quits for today
16:00:32 <krtaylor> thanks everyone!
16:00:33 <patrickeast> asselin: thanks for sharing!
16:00:33 <sweston> thanks everyone
16:00:55 <krtaylor> #endmeeting