20:01:00 <ianw> #startmeeting diskimage-builder
20:01:01 <openstack> Meeting started Thu Jun  1 20:01:00 2017 UTC and is due to finish in 60 minutes.  The chair is ianw. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:01:04 <openstack> The meeting name has been set to 'diskimage_builder'
20:01:21 <ianw> hello
20:01:32 <andreas-f> Hello Ian!
20:01:58 <ianw> #link https://wiki.openstack.org/wiki/Meetings/diskimage-builder
20:02:30 <ianw> #topic Annoucements
20:03:30 <ianw> ahh, nothing
20:03:47 <ianw> #topic Actions from last meeting
20:03:59 <ianw> I think we can skip this
20:04:20 <andreas-f> Yep - Its quite a while...
20:04:58 <ianw> #topic Testing
20:05:08 <ianw> sorry, forgot to add this to the agenda
20:05:15 <andreas-f> np
20:05:24 <ianw> bit of movement on suse out there as it goes towards the gate
20:05:55 <ianw> i think moving the nodepool jobs to voting is a good idea
20:06:09 <andreas-f> So we'll get a voting suse?
20:06:17 <ianw> #link https://review.openstack.org/#/c/469538/ nodepool jobs to voting
20:06:18 <patchbot> patch 469538 - openstack-infra/project-config - diskimage-builder/glean: Upgrade nodepool-src-* jo...
20:07:09 <ianw> a voting job that boots it, at least.  seeing as we broke it, also not the first time we've ignored the nodepool failures
20:07:54 <andreas-f> Yes good point.
20:08:02 <andreas-f> +1
20:08:06 <ianw> it does mean we're co-gating a bit on glean, to bring the vm up in devstack, but i think that's ok for the benefit of knowing it boots
20:09:30 <ianw> #agreed dib devstack/nodepool jobs to voting
20:09:47 <ianw> any more immediate testing issues?
20:10:04 <ianw> i've just woken up, but i'm presuming setuptools has unbroken itself
20:10:27 <andreas-f> I don't know of any testing issues.
20:10:58 <ianw> #topic Review review
20:11:21 <ianw> I don't think there's anything out there particularly stuck at the moment?
20:11:35 <andreas-f> except the UEFI ;-)
20:11:56 <ianw> oh
20:11:58 <ianw> #link https://review.openstack.org/#/c/468186/
20:11:59 <patchbot> patch 468186 - diskimage-builder - Drop support for Ubuntu precise
20:12:31 <ianw> I'm actually thinking we can think about dropping trusty
20:12:59 <andreas-f> trusty might be in use.
20:13:06 <andreas-f> I think it is not EOL.
20:13:30 <andreas-f> EOL: April 2019
20:14:07 <ianw> sorry, i meant as a build platform, really part of the previous topic, if nobody is using it in the wild.  it would only be 3rd party ci i think now, as infra has moved builders to xenial
20:14:07 <andreas-f> But on the other side, we also dropped support for RHEL 6 / Centos 6...
20:14:32 <andreas-f> Ah, ok.
20:14:48 <andreas-f> Does this mean some simplification / unification?
20:15:53 <ianw> trusty?  i don't know, maybe.  but anyway, unless there's objections i'm ok with precise removal
20:16:42 <andreas-f> Yes - let's remove precise now.
20:16:54 <ianw> #agreed merge 468186
20:17:09 <andreas-f> It looks that there are some dependencies for trusty: hpdsa element (never saw before).
20:18:48 <ianw> my only thought really was that we could remove the -trusty- jobs from the gate, and leave python2.7 testing up to centos
20:19:39 <ianw> we can probably wait until there is a problem, or infra starts to wind-down trusty node support
20:19:55 <ianw> any more pressing review requests?
20:20:13 <andreas-f> Ok - maybe I think I'm missing something: how are these (somewhat) special elements (like hpdas element) tested?
20:20:32 <andreas-f> For me it looks that this works only under trusty.
20:20:46 <andreas-f> So it we drop trusty, we should remove this element.
20:20:57 <ianw> they are probably not, and have probably bit-rotted
20:21:09 <ianw> see that dracut kernel one doing the patching
20:22:17 <andreas-f> So maybe a more general question: what do to with those elements?
20:22:42 <ianw> i think, these days, we would probably suggest any new like that don't belong in dib core but as elements as part of your project
20:23:01 <fungi> officially dropping support for precise is probably wise anyway since we'll (soon) cease to have servers we can test that on
20:23:19 <ianw> as we find stuff broken and unused, remove it
20:23:41 <andreas-f> It's hard to say if it is unused.
20:23:48 <andreas-f> Maybe some project uses it?
20:25:19 <ianw> we just have to use our best judgement, do some code searches/archaeology and respond if it turns out we did the wrong thing
20:25:29 <fungi> if it's a project in the openstack ecosystem, you can look on http://codesearch.openstack.org/
20:26:04 <fungi> though i think we don't index stable branches, recent stable branches should be using constraints anyway
20:26:42 <andreas-f> Ack.
20:26:44 <andreas-f> Currenly we have 99 elements - should check how many of them are tested or used.
20:27:02 <andreas-f> (long term thingy...)
20:29:16 <ianw> i'm happy to consider things on a case-by-case basis, like when it comes time to remove trusty looking at elements that only work there
20:30:24 <ianw> i don't feel like going through everything is the best use of time; a broken but unused element doesn't really affect the core mission of getting a usable .qcow out
20:30:36 <ianw> it just ... is
20:31:02 <ianw> anyway, moving on
20:31:22 <ianw> I guess everything else falls under
20:31:29 <ianw> #topic Open Discussion
20:31:56 <andreas-f> Yes.
20:32:03 <ianw> andreas-f: what is your priority there?
20:32:19 <andreas-f> UEFI boot: because it's open since 14 month.
20:33:00 <andreas-f> and it is an element which is IMHO a step forward.
20:33:12 <ianw> ok, do we have a consumer that needs it?
20:33:38 <ianw> #link https://review.openstack.org/#/c/287784/
20:33:39 <patchbot> patch 287784 - diskimage-builder - Add support for building images capable of UEFI
20:34:04 <andreas-f> I personally don't know of one.
20:34:23 <andreas-f> I know that HP uses it in one of the products.
20:34:38 <andreas-f> And I don't know if Yonalda needs this for her security.
20:34:54 <ianw> HP ... HP ... that's a company that used to exist, right? <joking>
20:35:37 <ianw> so 287784 is not based on dib-block-device at all?
20:35:53 <andreas-f> Yes, that the major problem.
20:36:02 <andreas-f> And there are another two:
20:36:07 <andreas-f> 1) Missing GPT support
20:36:22 <andreas-f> 2) old bootloader element
20:37:04 <ianw> so 1) ... writing our own mbr stuff is already super complex
20:37:17 <ianw> do we need to actually implement gpt writing?
20:37:35 <ianw> i did not understand why we did not just write out a parted/sfdisk script and get that to do all that
20:37:37 <andreas-f> I know: do you know a usable library here?
20:38:12 <ianw> os.system(parted)
20:38:16 <ianw> not sure it's a library :)
20:38:36 <andreas-f> parted *is* a library; but GPL.
20:39:15 <andreas-f> Here are some more arguments:
20:39:17 <andreas-f> https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/block_device/level1/mbr.py#L27
20:39:20 <ianw> right, i kind of got that looking back in the history why we didn't link against it, but we can call it?
20:39:57 <andreas-f> Yes; I used it in the first versions of block_device.
20:40:07 <andreas-f> It does not do what you want...
20:40:18 <andreas-f> ... but what it thinks you want ;-(
20:41:45 <andreas-f> These tools want to 'optimize' things; and I did not find a way to switch this off.
20:42:28 <ianw> ok, well i just think finding someone to write low-level, like byte-level GPT partitioning code is tough, and then to also have two people give it decent review, and maintain it is ... unlikely
20:44:25 <ianw> how does guestfish do it's partitioning in the back end?
20:44:28 <ianw> could we just use that?
20:46:10 <andreas-f> I'm just looking....
20:47:21 <andreas-f> too big... will do this offline.
20:47:29 <ianw> daemon/parted.c seems to be it
20:48:03 <andreas-f> it's also GPL.
20:48:04 <ianw> it is essentially system() calling out to parted, AFAICS
20:48:48 <ianw> well, not copy it, but it suggests to me that probably getting an external tool to do it is going to be a better way forward than writing it from scratch
20:48:49 <andreas-f> Yes - system()
20:50:08 <andreas-f> Did not check parted for GPT until now. Maybe we could use it at least for GPT.
20:50:12 <ianw> seems to have bits for parted/sfdisk/sgdisk all together
20:50:27 <ianw> maybe it requires some combo
20:51:04 <andreas-f> I checked all of them for MBR - all of them using some /sys-fs access to get some buffer infos.
20:51:16 <andreas-f> ... which they will use for 'optimizations'.
20:52:35 <andreas-f> So calling 'parted' might be a solution.
20:53:10 <andreas-f> 2) bootloader element refactoring is needed
20:53:51 <andreas-f> IMHO we should split the bootloader into a grub and extlinux.
20:54:18 <ianw> is this another case of who's using extlinux?
20:54:50 <andreas-f> Yes ;-)
20:54:53 <andreas-f> Then there is the need to get all the needed info from the block device layer and push it into the bootloader element.
20:55:41 <andreas-f> Hmmmm....
20:55:44 <andreas-f> So thinking about this:
20:55:46 <andreas-f> A lot of work - and maybe nobody really needs / uses it.
20:56:05 <ianw> right, that was my next point, i presume you're wanting to drive all this?
20:56:11 <andreas-f> Maybe we should ask Yolanda if this is needed for some kind of 'secure boot'?
20:56:28 <ianw> yes, let's sync with her on this, and it's priority to LVM
20:56:45 <andreas-f> Ack.
20:57:07 <ianw> i think starting at the bottom would work best.  there is quite a bit of unit testing templating going on in dib-block-device now
20:57:39 <andreas-f> Yes - saw the patches (which currently all fail...)
20:57:40 <ianw> i think any gpt implementation in there can be put in and relatively well tested
20:58:06 <ianw> the current failures were setuptools related, that should be fixed now
20:58:28 <ianw> then we can start building the elements ontop of it
20:58:34 <andreas-f> Ok - will check tomorrow.
20:58:52 <ianw> so, to summarise on this
20:59:02 <ianw> 1. sync with yolanda on relative importance
20:59:32 <ianw> 2. thoroughly investigate external tools for writing partitions, with reference to things like guestfish
20:59:46 <andreas-f> Ack.
21:00:09 <ianw> 3. start at implementation within dib-block-device at least covered by unit testing
21:00:27 <ianw> alright, well that feels like some progress
21:00:47 <ianw> we've hit 1 hour
21:00:54 <andreas-f> Yes saw this.
21:01:25 <andreas-f> Ok - then: have a nice day.
21:01:28 <andreas-f> Talk / mail / review in some hours.
21:01:50 <andreas-f> (Here it is 23:00 and I need some sleep now....)
21:02:19 <ianw> ok, thanks.  7am and i need some breakfast :) ttyl
21:02:31 <ianw> #endmeeting