17:02:27 <asselin__> #startmeeting third-party
17:02:27 <openstack> Meeting started Tue Oct 13 17:02:27 2015 UTC and is due to finish in 60 minutes.  The chair is asselin__. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:30 <openstack> The meeting name has been set to 'third_party'
17:02:46 <asselin__> hi, who's here from 3rd party working meeting?
17:02:59 <mmedvede> hey asselin__ :)
17:03:06 <skylerberg> Here
17:03:28 <sweston> \o
17:03:39 <asselin__> Hi, sorry I'm late, dealing with some internal issues
17:04:40 <asselin__> #topic announcements
17:04:50 <asselin__> anyone have any announcements?
17:05:03 <mmedvede> just a reminder on stackforge deprecation
17:05:11 <skylerberg> The governance patch merged this morning for CI Watch.
17:05:17 <mmedvede> \o/
17:05:31 <asselin__> congrats :)
17:05:41 <mmedvede> I wonder what would be the next step for to add ciwatch cores
17:06:18 <asselin__> good question, fungi was going to nominate, we should ask again in -infra or in the meeting today.
17:06:36 <fungi> nominate for...?
17:06:48 <fungi> oh
17:06:48 <asselin__> ci-watch core
17:07:06 <fungi> i'll add them, just let me know who should be reviewing/approving
17:07:36 <asselin__> I'll nominate skylerberg and mmedvede for your nomination :)
17:07:56 <mmedvede> asselin__ is already core there, I believe
17:08:09 <mmedvede> asselin__: I accept the nomination :)
17:08:29 <asselin__> really? I thought I was there for 2 minutes only. :)
17:08:35 <mmedvede> oh, maybe
17:08:41 <mmedvede> I did not check again
17:08:47 <fungi> ahh, yeah need to run our script to fix up those groups first. doing that now
17:10:04 <asselin__> thanks fungi
17:10:07 <mmedvede> thank you fungi
17:10:56 <asselin__> another announcement: single-node ci puppet class landed last week https://review.openstack.org/#/c/200330/
17:10:56 <fungi> skylerberg: what's your gerrit name/id/something?
17:11:09 <skylerberg> fungi: sberg
17:11:48 <fungi> mmedvede: and yours?
17:12:12 <mmedvede> fungi: should be msmedved
17:12:26 <mmedvede> or mmedvede, email mmedvede@us.ibm.com
17:12:41 <fungi> asselin__: skylerberg: mmedvede: welcome to ciwatch-core
17:12:51 <skylerberg> fungi: thanks :)
17:12:52 <asselin__> thanks :)
17:12:53 <mmedvede> fungi: \o/
17:13:05 <asselin__> let me fun begin
17:13:09 <mmedvede> yay, can do some merging there, finally
17:13:09 <fungi> thank you all!
17:13:49 <asselin__> what a great annoucement :)
17:14:04 <asselin__> #topic open discussion
17:14:22 <mmedvede> ok, so I did more work on puppet-ciwatch module
17:14:28 <asselin__> so there's no agenda today (my bad)
17:14:34 <asselin__> #topic puppet-ciwatch
17:14:39 <mmedvede> and deployed ciwatch internally
17:14:58 <mmedvede> #link WIP puppet-ciwatch https://review.openstack.org/226089
17:15:49 <mmedvede> plus there are two patches to get official home for puppet module
17:15:55 <mmedvede> #link project config patch https://review.openstack.org/#/c/231699/
17:16:09 <mmedvede> #link governance https://review.openstack.org/#/c/231698/
17:16:46 <mmedvede> the module does deploy with mysql
17:17:18 <mmedvede> I wondered if we should use postgres instead, skylerberg mentioned that is what was used for http://ci-watch.tintri.com
17:17:19 <asselin__> cool!
17:17:47 <mmedvede> I need to run is longer. E.g. today for no apparent reason mysqld stopped on my server
17:18:11 <skylerberg> mmedvede: I think mysql should be fine.
17:20:00 <mmedvede> skylerberg: would you have time to do any work on ciwatch? I was thinking to maybe to transition to cookiecutter -generated project structure
17:20:28 <mmedvede> #link https://github.com/openstack-dev/cookiecutter
17:20:46 <skylerberg> mmedvede: Yes, I will have some time, but it will be limited and unpredictable.
17:20:47 <mmedvede> basically, it creates all we need to enable infra tests, and more
17:20:57 <asselin_> Hi sorry dropped
17:21:18 <skylerberg> mmedvede: Sounds like a good idea.
17:21:37 <mmedvede> asselin_: I was suggesting to use openstack-dev/cookiecutter template on ciwatch
17:21:46 <asselin_> +1
17:21:58 <skylerberg> Let's use https://launchpad.net/ciwatch to start tracking changes we want to make.
17:22:08 <mmedvede> skylerberg: good point
17:22:23 <mmedvede> should we also use bugs to track features we want to implement?
17:22:24 <skylerberg> Then I can assign tasks to myself when I do have time.
17:22:39 <skylerberg> mmedvede: That is what I was thinking.
17:23:15 <skylerberg> I am open to other suggestions though.
17:23:23 <asselin_> skylerberg, I think you can add myself and mmedvede to the maintainers
17:23:40 <asselin_> https://launchpad.net/~ciwatch-drivers/+members
17:23:41 <mmedvede> works for me to use launchpad for task tracking/ division
17:24:25 <skylerberg> asselin_, mmedvede: What are your launchpad ids?
17:24:51 <mmedvede> skylerberg: https://launchpad.net/~msmedved
17:24:54 <asselin_> ramy-asselin
17:25:54 <skylerberg> done
17:26:19 <asselin_> thanks
17:26:57 <asselin_> mmedvede, let us know when we should start reviewing the patches
17:27:47 <mmedvede> asselin_: I think it is safe to merge the ciwatch patches that are just history transfer from third-party-ci-tools
17:28:28 <asselin_> +1, I meant the puppet-ciwatch ones, or should we wait until they're proposed to the new puppet-ciwatch project
17:28:49 <mmedvede> asselin_: as I would work on puppet-ciwatch, I'll find more things to fix in ciwatch itself
17:28:55 <mmedvede> asselin_: oh, that one we should not merge
17:28:56 <mmedvede> :)
17:30:24 <asselin_> ok, so let's merge the historical changes, apply cookiecutter on ciwatch, and review the puppet-ciwatch project-config patch so we can get that started.
17:31:28 <mmedvede> asselin_: +1, that would get us moving. puppet-ciwatch can also start using the new ciwatch source
17:31:40 <asselin_> +1
17:31:55 <asselin_> and start to use launchpad to track issues/features
17:32:17 <asselin_> anything else?
17:33:26 <mmedvede> on ciwatch, I had question to skylerberg
17:33:43 <mmedvede> but can talk later, if we have something else to discuss now
17:34:22 <asselin_> I wanted to discuss summit planning a bit. mmedvede will you be there?
17:34:23 <skylerberg> mmedvede: Go ahead
17:35:03 <mmedvede> skylerberg: is there any reason that gerrit events are first stored into test log, and then can be loaded to db separately.
17:35:45 <mmedvede> asselin_: no, unfortunately would not make it to summit. Next one definitely
17:36:40 <skylerberg> mmedvede: That is mostly for testing during development. I wanted to be able to quickly populate the database with real data after making changes to the database schema.
17:37:28 <mmedvede> skylerberg: ok. I figured it was for historical reason. Was not apparent that it is not necessary to manually run db populate
17:38:26 <skylerberg> mmedvede: Correct. It was useful during development because I could load in a log from the event stream even if I didn't have ciwatch running.
17:38:59 <mmedvede> skylerberg: it is a good thing to have. Just need to update the doc later
17:39:23 <mmedvede> asselin_: I have nothing else, please proceed
17:39:24 <asselin_> I also think it's useful
17:39:45 <asselin_> for development & testing
17:40:07 <mmedvede> on a tangent, I was thinking it might be possible to seed the db from zuul logs
17:40:49 <asselin_> that's true, the debug logs capture the gerrit event stream
17:40:52 <skylerberg> mmedvede: That could be nice. It would be great to have a data source that sticks around. Right now if there is a networking problem, data is missed and can't be found.
17:41:35 <mmedvede> also, it seems like it does not add duplicates, so you can seed it from different sources, if I am correct, and fill the gaps
17:41:51 <asselin_> this is one of the disadvantages of reading the event stream. perhaps it would be good to also be able to queury the gerrit database
17:42:41 <skylerberg> mmedvede: That is correct. However, the new sources should be more current than old source or entries could be changed to not reflect a recheck.
17:42:54 <mmedvede> asselin_: real-time stream is nice, because it is real-time. But could add a check to validate if any events are missing, and use gerrit api to fill it
17:43:10 <skylerberg> mmedvede: +1
17:43:14 <asselin_> mmedvede, +1
17:43:25 <mmedvede> I would not worry about it now though, there are more important things to be done first :)
17:43:34 <asselin_> +1
17:43:44 <skylerberg> +1
17:43:52 <mmedvede> thank you :)
17:43:55 <asselin_> yeah that can be added after the initial version is deployed
17:44:20 <asselin_> another issue I've noticed but didn't get a chance to confirm is that rechecks don't populate in ci-watch
17:44:31 <mmedvede> skylerberg: did you see any problems with db getting big after awhile?
17:44:48 <skylerberg> asselin_: I should test it more, but they should update it.
17:44:58 <mmedvede> asselin_: yes, we also do not have general -1 (e.g. -1 that are not from failed job, but from failed merge)
17:45:19 <skylerberg> mmedvede: I didn't see that, but I didn't haveit running that long and it was growing slowly.
17:45:53 <skylerberg> mmedvede: We could ask apoorvad to check how large the db is now.
17:46:51 <skylerberg> We should periodically purge old entries from the db.
17:47:14 <apoorvad> skylerberg: I will let you know in a few
17:47:15 <mmedvede> skylerberg: maybe purge largest items sooner? e.g. comments
17:47:58 <skylerberg> apoorvad: Thanks!
17:48:01 <mmedvede> the older data is useful to have locally, e.g. simple fail/success should not take much space, but should allow to do stats
17:48:33 <skylerberg> mmedvede: Yes, that would work.
17:48:57 <asselin_> Yeah I don't think comments are needed at all, right?
17:48:57 <mmedvede> asselin_: we got into feature discussion, sorry, we can get it done later, and open launchpad bugs for it
17:49:24 <asselin_> mmedvede, no problem, I don't think there are any other topics, and its' just the 3 of us
17:49:31 <asselin_> and this is an important topic ;)
17:49:47 <mmedvede> asselin_: you wanted to talk about summit planning
17:50:01 <mmedvede> asselin_: yes, I like talking about features :)
17:50:03 <asselin_> mmedvede, well seems to be just me here going :)
17:51:06 * mmedvede is envious
17:51:17 * skylerberg is too
17:51:37 <asselin_> we don't have any official slots, but we can setup some unofficial talks. I'll reach out to the mailing list
17:52:01 <mmedvede> asselin_: advertise the working group there, more people here is better
17:52:09 <asselin_> mmedvede, +1
17:52:10 <mmedvede> more third-party CIs
17:52:34 <asselin_> yeah, we can try to recruit people to work on  ciwatch as well
17:52:59 <mmedvede> +1
17:53:05 <skylerberg> +1 :)
17:53:29 <asselin_> anything else you'd like to see happen?
17:53:54 <mmedvede> on ciwatch?
17:54:01 <asselin_> in general for 3rd party ci
17:54:19 <mmedvede> during the summit?
17:54:33 <asselin_> for the next 6 months
17:54:57 <mmedvede> well, need to continue downstream-puppet effort
17:55:00 <mmedvede> it helps a lot
17:55:20 <asselin_> ci-watch, migrate people over to common-ci/downstream-puppet
17:55:23 <mmedvede> asselin_: you did a lot of good work there already
17:55:46 <apoorvad> http://paste.openstack.org/show/476170/
17:56:05 <asselin_> trying to see what's the next big thing needed, but perhaps just getting those two solid would be great
17:57:32 <mmedvede> asselin_: yes
17:57:50 <mmedvede> apoorvad: so it is 140 MB?
17:58:11 <mmedvede> 140 Kb does not seem sane
17:58:32 <skylerberg> apoorvad: the database should be postgres. I am not sure why mariadb would be on that system.
17:58:37 <apoorvad> Hmmm. That seems too small  to me
17:58:39 <apoorvad> too
17:58:43 <asselin_> ok, if there's nothing else I'd like to end the meeting. we can continue on openstack-third-party-ci
17:58:49 <asselin_> #endmeeting
17:58:59 <mmedvede> apoorvad: thank you for looking it up
17:59:04 <asselin__> #endmeeting