20:00:58 #startmeeting diskimage_builder 20:00:58 Meeting started Thu Aug 18 20:00:58 2016 UTC and is due to finish in 60 minutes. The chair is cinerama. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:00 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:02 The meeting name has been set to 'diskimage_builder' 20:01:09 hi folks 20:01:22 if we don't get anyone this is going to be pretty short :) 20:02:59 ohai 20:03:04 sorry, was just in a cal 20:03:06 haha 20:03:10 hey! 20:03:22 yea, I had meetings lined up back-to-back, ofc they run over 20:03:27 ugh. 20:03:46 anyway, got anything for the agenda today? it's been fairly quiet 20:03:57 Yea, I think we have dib on PTO 20:04:32 the big thing is sort of a selfish patch that I really really want us to review because it has caught so many regressions in CI but we catch them after we release since we havent merged it yet 20:04:34 * greghaynes looks 20:04:35 for me: i want to revisit https://review.openstack.org/#/c/334785 and friends 20:05:03 https://review.openstack.org/#/c/181162/ 20:05:19 and andreas' refactoring changes 20:05:20 which I think needs ianw's eyes because I believe its failing on a bug we have for centos 20:05:30 hey 20:05:40 hello there :) 20:05:46 hi! 20:05:52 cinerama: I'm with you on https://review.openstack.org/#/c/334785 20:06:08 ianw: I'd like to get ^ merged, left you a comment on there 20:06:22 an annoying one, sorry 20:06:22 how are we doing release management? if we have stuff we want to do for 2.0 should we just start landing it? 20:06:42 ianw: does it sound fine to do that in 2.0? 20:07:13 yea. so thats probably a good thing to get a solid plan for - how are we going to 2.0... 20:07:37 i feel like maybe a feature branch might be a good idea 20:07:49 I think so too 20:07:51 that is an option 20:07:53 because we kind of need to be able to release quickly if something is messed up for infra 20:08:00 or tripelo 20:08:01 yup 20:08:17 that might give us some more scope to land some of these big changes 20:08:27 The other big side is to make sure features we put in that are breaking we put in corresponding deprecation warnings in to dib master 20:09:15 true, although nobody reads warnings :) unless we have a -Werror type thing 20:09:36 Yea, they are more of a CYA thing it seems like, which is :( 20:09:49 I'd support a -werror for sure 20:10:03 sounds good 20:10:21 there's the "organisational" stuff i'd like to look at too, like moving disk-image-create to an entry-point function, and moving the elements under the python pkg 20:10:21 I'm thinking though, if we can detect the case for the element overriding behavior change (for example) we can deprecate and fail with werror 20:10:55 so do we have a rough consensus on 1) cutting a 2.0 feature branch & landing big stuff there 2) add -werror 3) add deprecation warnings? 20:11:01 yep, same, if we can spit out "this is going away" ASAP that gives us more flexibility to remove it sooner 20:11:31 I really like the idea of moving to all entrypoints btw :) 20:11:43 cinerama: I think so 20:11:57 #agreed cut 2.0 feature branch 20:12:05 It'd be great if the plan could get written up. I know ianw has a feature list, so now the process would be good just so we can get feedback 20:12:09 #agreed add werror 20:12:24 #agreed add deprecation warnings (using werror) 20:12:35 woo irc features 20:12:53 do we have a link to the feature list? 20:12:56 what I am wondering actually - if we feature branch are we going to still deprecate? 20:13:18 like, if we add the entrypoint change in the feature branch, how can we properly deprecate that in master 20:13:24 since master wont have the alternative 20:13:35 well, i don't think there's really anything to deprecate in that case, really 20:13:40 it should be transparent 20:13:54 ianw: I think removal of the bin/foo needs some deprecation, theres folks who run them directly 20:14:04 unless we want to leave them forever 20:14:28 well there's still a script there, it's just a #!/bin/python script instead of a #!/bin/bash 20:14:56 ah, ok. And then it just calls the same thing as the entrypoint? 20:15:02 and for those who invoke via 'sh bin/disk-image-create'? 20:15:21 bkero: Yea, I think ianw is saying we leave the scripts but make them python 20:15:47 Dunno if we want to do this -- but what other projects have done is just leave the script in place, but gut it and make it call a separately named python script 20:16:05 I don't think dib has a big enough legacy to need something like that though 20:16:12 bkero: that wouldn't be that safe ... /bin/sh can be dash on ubuntu and would just completely fail with the bash in dib 20:16:40 ianw: it's dash in sh-compatibility mode though 20:16:56 but i mean, in general, there's always that grey area of "undefined behaviour" 20:17:01 the element override is similar 20:17:14 So how about this, stuff which we dont need to deprecate or we can deprecate we dont need ot feature branch - we can just put it in master with a deprecation warning? 20:17:28 that sounds good 20:17:31 then we flag the things in the 2.0 etherpad which are breaking and feature branch them 20:18:01 well, yeah, but so far we've pretty much failed at that 20:18:22 these much "bigger" things 20:18:56 because we've always had in our mind "if infra breaks tomorrow, we've got to push a release on this" 20:18:57 Yea, sorry,my tl;dr is I have a big internal project wrapping up nowish and I'm arguing to take a break to do some large dib work once its closed 20:19:19 right, theres that fear too - I really worry about that with anything backwards incompat 20:19:40 so for that - I think we could actually get a CI job for some of these larger and more fragile images 20:19:56 yeah, if we have the branch, we could do a little more sanity checking 20:20:05 with the pressure off 20:20:19 and really https://review.openstack.org/#/c/181162/ and friends, if those merge that would give us a ton more confidence 20:20:32 https://review.openstack.org/#/c/204639/ being the other one 20:20:42 they keep getting hung up because we land bugs the tests would have caught 20:21:01 heh yeah 20:21:10 sure, also i got the nodepool dsvm bit working 20:21:17 test i mean 20:21:24 sounds like once we have those in we should be able to start landing refactoring &c 20:22:22 ianw: yea, thats actually a really good point about the feature branch - we can have folks test with it 20:22:28 greghaynes: what's needed to land those? Just a jenkins recheck and rebase? 20:22:38 bkero: I think they are hung up on actual dib bugs 20:22:54 oh 20:22:55 I tihnk the centos locale bug is the first one 20:23:22 ianw: so maybe we can convince infra to help a bit with that - add an image using dib 2.0? 20:23:39 yep, i feel like if we get things into a branch, our next meeting might be more "what extra testing should we do on dib 2.0 branch" 20:23:49 SGTM 20:23:59 nice 20:24:02 greghaynes: yeah, but even still, i can easily be doing offline testing of infra builds with a 2.0 branch 20:24:09 yep, true 20:24:44 ok, so someone needs to ask infra for a 2.0 branch I think 20:24:49 alright, i'll take it as an action item to do that 20:24:54 awesome 20:25:02 and i'll email out when done 20:25:12 thanks 20:25:31 sounds good 20:25:47 ianw: I also need to pick your brain on https://review.openstack.org/#/c/181162/ 20:26:04 ianw: its the mmap / bus error thing 20:26:07 i guess the only thing is, what's the end point for the branch? will we just never think it's finished? 20:26:40 i guess we need to agree on what features should be part of it and then go when they are complete 20:26:44 ianw: maybe we timebox it? 20:26:50 heh 20:26:52 that's another option 20:26:56 maybe smarter 20:27:06 this is sounding like a openstack dev cycle TBH 20:27:49 yeah i was imagining spec approvals and such 20:28:16 greghaynes: ahh, that :( i think for fedora 24, that problem goes away, they've moved to sensible locale packages. centos might be the hangup 20:28:17 it's a model we could follow more informally 20:28:26 ianw: yep, it is 20:28:48 ianw: so does that mean locale workarounds in the centos element? 20:29:06 so maybe we have have a time limit where we try and cut the release by? 20:29:14 when? 20:29:32 I actually need to reread the feature list before I can answer that 20:29:46 yeah, maybe everyone add what they want into https://etherpad.openstack.org/p/dib2.0 20:29:53 but i'm thinking ~6moish, maybe we just try and line it up with the openstack release 20:30:36 So cut a release when the rest of the projects do? 20:30:53 Isn't there an org that arranges all that? Maybe add dib to that list? 20:31:10 yep, thats kind of what I was getting at, its starting to just sound like the openstack release process 20:31:14 bkero: unfortunately, centos 7 doesn't really have a nice/documented way to clear out unnecessary locales 20:31:17 which isnt a bad thing 20:31:53 ianw: If I recall, can't we supply a /etc/locale.gen file in the element and tell it to rebuild? 20:32:08 Maybe this is off-topic for the meeting and should be discussed in #openstack-dib 20:32:32 ok. So this is super unsatisfying to say but I think first we need to nail down the feature list and the requirements for what is going to break backwards compat. I've kind of failed at putting enough effort in to it so maybe we can set a deadline on that for next meeting? 20:32:35 bkero: *shrug* we have time 20:32:56 the only thing i'd say about that is that it's not yet clear if this is just because we've hit a bit of a log-jam of big features, or if it's going to be like for this forever 20:33:08 ianw: yea, I agree with that 20:33:17 we have a limited amount of core bandwidth 20:33:37 ianw: I really hope this is a log jam 20:34:11 yeah, if we clear out some of this big stuff, maybe we go back to humming along with point releases happily for a long time 20:34:19 i think so 20:34:30 if it's not happening, then maybe we look at more formal release cycles, etc 20:34:40 but anyhow, if we can get that solidified for next meeting and have a thing on the agenda for discussing how we want those specific features to merge that should move us forward 20:35:05 rather than a general plan we can basically just walk through the feature list and say for each feature how do we want to land it 20:35:42 and I suspect that is some of them go in to feature branch and hopefully with a clear end in sight? 20:35:54 Sounds doable 20:36:14 You can also mark them as 2.0 even if they're in feature branch so you can know when to expect to land them (and an order) 20:36:17 ok, well for mine, https://etherpad.openstack.org/p/dib2.0 is pretty much the list of things 20:36:28 not all of it is strictly backwards incompat 20:36:39 but it's changing a lot of undefined behvaiour 20:36:43 which is maybe the same thing 20:36:54 true 20:37:04 yea, we have some license there depending on how big of a surface area it is 20:37:21 technically it is but we can be reasonable if its super corner case or whatnot 20:37:31 things like disk-image-create moving from #!/bin/bash to #!/bin/python 20:38:47 Yea, tahts fine I think 20:38:54 as long as the actual interface is the same 20:39:16 speaking from a bifrost perspective as long as the flags etc are the same should be fine 20:40:15 does that sound good as an action item though - go through the list and formulate a plan for each feature whether we can implement it backwards compat or if not then flag it as 2.0 feature branch. Try to get a list of 2.0 feature branch things for next meeting and then get consensus if its something we can see merging withing a reasonable timeframe? 20:40:30 sgtm 20:40:50 well i keep saying it :) -> https://etherpad.openstack.org/p/dib2.0 20:41:26 ianw: yep, those are some backwards compat doable and some not so its just go through that list? 20:41:49 or add missing things to it 20:41:50 and also probably through andreas' patches to see if any of them need to be feature branchey 20:41:53 Let's go through the list and categorize things 20:42:07 i have a huge blind spot for the uefi stuff for example 20:42:26 Are there UEFI reviews? 20:42:32 i just have zero experience building stuff for that environment 20:42:45 Oh. I have a bit of experience if you can point me at some reviews. 20:42:49 There were 2 think, I think one merged 20:42:57 oh yes 20:43:17 Being able to make ESP partitions, installing programs, and updating EFI variables for boot order would be a good feature 20:43:20 https://review.openstack.org/#/c/287784/ is still pending 20:43:31 had a +2 from greghaynes recently 20:43:49 bkero you might want to have a look 20:43:54 sure 20:44:12 We can also add testing using tianocore's kvm biosfile 20:44:46 #action everyone to read through https://etherpad.openstack.org/p/dib2.0 - categorize features as things which need to be part of the 2.0 feature branch or we can implement in a backwards compat manner. Also add any changes which may need to be similarly categorized and are missing (review andreas' refactoring code for example) 20:45:01 how do we feel about uefi as a 2.0 thing vs a regular thing 20:45:23 cinerama: its just a matter of if its backwards compat doable, I dont remember seeing anything which wasnt 20:45:34 k 20:45:38 It sounds like we need a....spreadsheet. <3 20:46:08 heh, I think we can just label on the etherpad, it isnt that large of a list I hope 20:46:14 I really hope 20:46:27 maybe compile it out in to something 20:46:45 cinerama: The EFI support looks pretty contained to the vm and bootloader elements. I need to read the patch to figure out how invasive it is inside the elements though. 20:46:46 we could add something as a spec i guess 20:46:53 but really first I think we all need to put the review time in on those features so we can get consensus 20:47:07 bkero: feel free to throw your thoughts in on the review 20:47:15 cinerama: I will as soon as the meeting is over :) 20:48:45 anyone have anything else to discuss? 20:49:00 oh, yes 20:49:06 I need feedback on a cores thing 20:49:14 actually, two things 20:49:27 shoot 20:49:30 so first I didnt get much feedback on us becoming a project team, I hope thats a good and not a bad thing 20:49:44 as in, are there any objections to it... 20:49:50 right 20:50:01 because if not then we need to do the whole propose a project team, elect a ptl dance 20:50:15 with various stuff like the meeting, channel, etc going through i think we're more convincing 20:51:07 as an independent entity 20:51:13 right, IMO it makes sense but hopefully theres consensus 20:51:19 ianw: ^ thoughts? 20:52:11 the second thing is - right now we have tripleo-core as a subset of dib-core which I'd like to remove but keep on folks who have done 2 or more reviews this past cycle by adding them to dib-core if thats ok? 20:52:35 ahh, i dunno? :) i'm not sure i have anything much to say 20:52:38 both things kind of need consensus from the core team but I didnt get ML feedback 20:53:26 ok. As long as its not an objection :) mostly just trying to not run off changing the project without core consent :p 20:54:01 that sounds fine with me, we can grandfather folks in who are interested 20:54:16 good deal 20:54:33 no, no objections 20:54:43 i'm glad someone is thinking about it, so i don't have to :) 20:54:54 haha, fair enough 20:55:15 ok, thats all I have... 20:55:24 me too. 20:55:53 O/ 20:56:07 #link https://etherpad.openstack.org/p/dib2.0 20:56:11 (before i forget) 20:56:15 So next meeting we go over the dib 2.0 etherpad and discuss how to land the items listed there, and how to define them as done? 20:56:29 bkero: Yep 20:56:29 sounds good 20:56:38 anyone have anything else they'd like to add? 20:57:19 o/ good work team 20:57:40 thanks everyone 20:57:45 cinerama: ty 20:57:52 #endmeeting