20:07:14 <zigo> #startmeeting deb_packaging
20:07:15 <openstack> Meeting started Mon Sep 26 20:07:14 2016 UTC and is due to finish in 60 minutes.  The chair is zigo. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:07:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:07:18 <openstack> The meeting name has been set to 'deb_packaging'
20:07:26 <zigo> Sorry for being late.
20:07:42 <tlbr> o/
20:07:51 <zigo> #chair onovy zigo tlbr
20:07:51 <openstack> Current chairs: onovy tlbr zigo
20:08:07 <zigo> #topic project status
20:08:19 <zigo> so, we have everything up to rc1, but Horizon.
20:08:36 <zigo> libjs-tv4 has been accepted by Debian FTP masters, so we can finally start the Horizon packaging.
20:08:43 <tlbr> great
20:09:04 <zigo> Ironic & Ironic-client also needs update to 6.2.0 and 4.2.0 respectively.
20:09:28 <zigo> Then we need to figure out how to run tempest stuff...
20:10:05 <zigo> fungi: clarkb: pabelanger: mordred: Any of you here? We'd need some advice.
20:10:40 <zigo> Nop, never mind.
20:10:58 <zigo> onovy: tlbr: Any idea how we shall start running tempest in infra?
20:11:07 <zigo> Maybe a periodic job?
20:11:32 <tlbr> is it hard to setup on OS infra ?
20:11:43 <zigo> I'm not sure yet.
20:11:57 <onovy> periodic job sounds good
20:11:59 <tlbr> ok, so we need help of infra-guys
20:12:24 <zigo> My idea was that, once we got the tempest stuff in, and Newton is finally released and uploaded to Sid, we should run it as a check job for any packaging change.
20:12:41 <zigo> Not yet right now though, as it may be too long to run.
20:12:42 <fungi> i'm around, just busy
20:13:01 <onovy> yep
20:13:10 <zigo> fungi: We were wondering how we could setup a tempest validation script for packaging.
20:13:26 <zigo> fungi: We have that already written, we just need to figure out how to set it up on infra.
20:13:35 <fungi> the obvious choice is to run tempest against each proposed patch to your packaging repos
20:14:00 <onovy> which will took ages to finish
20:14:10 <onovy> for example i send ~ 200 patches today
20:14:19 <zigo> fungi: Yeah, but maybe do that only once Newton is finally released, so we kind of "lock" things, without too much disturbance when we work on the release.
20:14:22 <onovy> i don;t want tempest for each one :]
20:14:39 <zigo> onovy: But once we have everything released, that's fine for you, no?
20:14:51 <onovy> zigo: not sure. tempest takes long time to finish
20:14:52 <fungi> i'm assuming that would have to: 1. build a package with the prospective change checked out and applied, 2. install that package along with the already official packages, 3. start tempest and report the results
20:15:20 <zigo> fungi: Yes, something like that.
20:15:27 <fungi> also depends on whether you run all tempest tests or just a subset (smoke for example)
20:15:28 <onovy> it will slow down release process of package
20:15:38 <onovy> but it will higher quality :)
20:15:51 <zigo> fungi: One of my fear is the mac filter in the nodes.
20:16:15 <zigo> fungi: My current setup uses a bridge, and then we get the MAC of that bridge instead of the eth0.
20:16:20 <zigo> fungi: Will it be a problem?
20:16:25 <fungi> i mean, our ci is designed to test proposed changes and find bugs before you merge them. it has less efficient solutions for finding bugs after merging and reporting those
20:16:52 <onovy> so let's do it same
20:17:14 <onovy> but we have some lock-in problems
20:17:15 <zigo> I had the MAC filter issue on the Mirantis OpenStack deployment we have (accessed through the Mirantis VPN, very private...).
20:17:24 <onovy> example: we can't update lib and python wrapper around it at same time
20:17:26 <onovy> but we need to
20:17:27 <fungi> zigo: i don't know exactly what bridge issues you're describing, but we strongly suggest jobs don't rely on the system-provided network interfaces directly and instead use an abstraction you can control
20:17:48 <zigo> fungi: What do you mean?
20:18:13 <fungi> don't rely on being able to do things that communicate outside the test node
20:18:36 <fungi> the job should be entirely encapsulated within the node (or multinode group) on which the job is initiated
20:18:50 <onovy> we are running single node tempest i think
20:19:22 <fungi> and if it needs to test network connections to things, it should do them over a loopback or similar local connection to another local process
20:19:23 <zigo> fungi: Ok, got you, though our script used to be controled over ssh, so we'll have to fix that.
20:19:56 <zigo> It shouldn't be so hard, and yeah, I didn't think about that bit that we don't really care outside connectivity in the case of infra.
20:20:04 <fungi> right, the job launcher will ssh in and run a script that initiates your job, and collect results/artifacts over rsync+ssh
20:20:14 <zigo> fungi: How can I start a test stuff then?
20:20:24 <zigo> Oh...
20:20:29 <fungi> but everything else needs to try to stay local to the machine
20:20:33 <zigo> But what if ssh / rsync fails with no connectivity?
20:20:49 <zigo> So, same issue then.
20:21:01 <zigo> Anyway, I guess best is to just try.
20:21:07 <fungi> it streams stdout of teh main job process continuously, so you'll at least hopefully get a console log with output up to the point where the job node became unreachable
20:21:41 <zigo> fungi: All the sources are currently located within "openstack-meta-packages", maybe I can start it as a non-voting check for this package?
20:21:50 <fungi> seems reasonable
20:21:51 <zigo> (then slowly fix things there...)
20:22:31 <zigo> fungi: Ok, thanks for your help, moving on then.
20:22:42 <fungi> i don't know how feasible it is, but if you can use devstack to deploy a system with your packages then that might be the easiest solution
20:23:05 <fungi> anyway, yes doesn't need to be designed in your meeting. find us in the infra channel
20:23:05 <zigo> fungi: I have designed everything already, nothing much to rewrite.
20:23:13 <fungi> great
20:23:27 <zigo> fungi: The only thing that needs to rewrite is how to be smart when calling the scripts.
20:23:36 <zigo> fungi: It used to run on a bare metal node...
20:23:41 <zigo> fungi: Oh, that's another question !
20:24:01 <zigo> fungi: 8GB of RAM may be a bit short for installing everything and running tempest...
20:24:09 <zigo> fungi: Or is that enough?!?
20:24:23 <zigo> I used to run with 32 GB on a real bare metal node.
20:25:03 <fungi> depends on what "everything" is
20:25:23 <fungi> i mean, we do limit the default service list we run for most devstack+tempest jobs
20:25:24 <zigo> fungi: Nova, Neutron, Cinder, Glance, Swift, Keystone, Horizon ...
20:25:34 <zigo> (at least)
20:25:54 <zigo> Ceilometer too.
20:26:00 <fungi> but use existing devstack+tempest jobs for a guideline as to what we're able to run within 8gb, at least
20:27:17 <zigo> fungi: I think I'll just try, and see how it goes. If I make sure to reduce the amount of thread per servers, that should be fine, normally.
20:27:35 <fungi> yep, i encourage the scientific method there
20:27:55 <zigo> :)
20:28:22 <zigo> #action Zigo to add a non-voting job to deb-openstack-meta-packages to start tempest tests.
20:28:45 <zigo> #topic unit test re-enabling
20:28:53 <zigo> Has anyone worked on that?
20:28:56 <zigo> tlbr: ?
20:29:14 <tlbr> nope
20:29:14 <zigo> There's not so many remaining.
20:29:19 <onovy> yep
20:29:22 <onovy> i done in swiftclient
20:29:32 <zigo> testtools, traceback2, reno are the only one remaining.
20:29:45 <onovy> btw: i created section in https://etherpad.openstack.org/p/openstack-deb-packaging which is progress of my packages. for anyone interested :]
20:29:59 <zigo> onovy: Do you want to do them?
20:30:17 <onovy> testtools,...? no :)
20:30:29 <onovy> really i just pushed ~200 patches and many of them failed
20:30:44 <zigo> Failed, how?
20:30:48 <zigo> With failed downloads?
20:30:56 <onovy> missing deps, etc.
20:31:02 <zigo> Oh...
20:31:06 <zigo> Got to fix that then.
20:32:08 <zigo> #topic onovy mass commits
20:32:24 <zigo> onovy: Another thing is that I don't think it's very nice to push so many patches at just once...
20:32:34 <zigo> Then it becomes very hard to push patches.
20:32:46 <onovy> why very hard?
20:32:49 <zigo> onovy: Could you imagine something that would push in a slower pace?
20:33:13 <zigo> onovy: Let's say I add libjs-tv4 to the auto-backports, then it will take forever to do its POST job.
20:33:40 <onovy> right. so it's not about pushing, but reviewing
20:34:15 <zigo> Well, yeah ...
20:34:29 <zigo> onovy: Anyway, it's a *very* good thing to do such a mass rebuild of everything.
20:34:54 <onovy> we should have better solution to rebuild everything - without commit/review ;]
20:34:55 <zigo> onovy: It will prevent "Luca Neusbaum" kind of bugs getting in the BTS ! :)
20:35:12 <onovy> and we should rebuild periodicaly
20:35:55 <zigo> onovy: How would you get the feedback then?
20:36:08 <onovy> you can get feedback from periodic job
20:36:12 <zigo> How?
20:36:17 <onovy> email?
20:36:22 <zigo> No idea ! :)
20:36:25 <zigo> I'm asking ... :P
20:36:32 <onovy> email, autobug create
20:36:57 <zigo> ok
20:37:15 * zigo recieves a fair amount of robot spam already ...
20:37:17 <zigo> :P
20:37:55 <zigo> #topic open discussion
20:37:58 <zigo> Anything else you guys wana talk about?
20:38:13 <onovy> i think everyone should go throu review list
20:38:15 <onovy> and fix -1
20:38:19 <onovy> or abadon
20:38:23 <onovy> we should clean it up
20:38:27 <zigo> Yeah, I'll do that.
20:38:31 <onovy> https://review.openstack.org/#/q/projects:openstack/deb-+AND+is:open
20:38:33 <zigo> Some are obvious.
20:38:35 <onovy> this link will help
20:38:43 <zigo> Like the glance-store one (ie: add enum34 as build-depends)
20:38:51 <onovy> anyone else? :)
20:39:02 <zigo> I'll do some of them for sure.
20:39:16 <zigo> I just wonder how it could have been built the first time though.
20:39:45 <onovy> maybe i wasn't build first time
20:39:47 <onovy> it
20:40:01 <zigo> Some didn't indeed.
20:40:06 <zigo> Like the Horizon plugins.
20:40:14 <zigo> Some others did, like glance-store.
20:40:21 <onovy> there was package which didn't have any reviews at all
20:40:21 <zigo> I didn't look deep enough yet to be able to tell.
20:40:27 <onovy> so nobody tried to build it
20:40:30 <zigo> Right.
20:40:50 <zigo> There's that as well indeed.
20:41:10 <zigo> #action zigo & onovy to check on the mass commit FTBFS
20:41:19 <onovy> and tlbr ? :]
20:42:23 <zigo> He's watching a series ? :P
20:42:36 <zigo> It's too late for Moscow, I guess.
20:42:43 <zigo> Maybe we should put the meeting time earlier.
20:42:44 <tlbr> i'm here
20:42:47 <zigo> Oh! :)
20:43:05 <zigo> tlbr: Will you try to fix some failed to build in the mass-commit of onovy?
20:43:18 <zigo> tlbr: Do you have time for that, or do you prefer working on the Horizon packaging maybe?
20:43:21 <tlbr> zigo, ok, i can do that tomorrow if that is ok ?
20:43:34 <onovy> cool, thanks
20:43:34 <zigo> tlbr: I don't think we'll be done in a single day.
20:43:41 <tlbr> yes, I've just finished working on tenacity
20:43:52 <zigo> #action tlbr to also check on the mass commit FTBFS
20:43:52 <tlbr> zigo, ok then
20:44:13 <zigo> tlbr: onovy: Anything else you guys wana talk about?
20:44:23 <onovy> i need to remove python-eventlet from backports
20:44:27 <zigo> Oh.
20:44:34 <zigo> onovy: Ask pabelanger about it.
20:44:40 <zigo> onovy: In #openstack-infra
20:44:55 <onovy> zigo: let's talk about it outside of meeting
20:45:00 <zigo> Ok.
20:45:04 <zigo> We're done then.
20:45:12 <zigo> #endmeeting