14:01:02 <dprince> #startmeeting tripleo
14:01:02 <openstack> Meeting started Tue Dec 22 14:01:02 2015 UTC and is due to finish in 60 minutes.  The chair is dprince. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:06 <openstack> The meeting name has been set to 'tripleo'
14:01:29 <derekh> hi
14:01:32 <marios> o/
14:01:34 <shardy> o/
14:01:38 <shadower> hey
14:01:46 <dprince> hi everyone
14:01:48 <dtantsur> o/
14:01:53 <florianf> o/
14:02:04 <d0ugal> Hello!
14:02:38 <dprince> #topic agenda
14:02:38 <dprince> * bugs
14:02:38 <dprince> * Projects releases or stable backports
14:02:38 <dprince> * CI
14:02:38 <dprince> * Specs
14:02:41 <dprince> * one off agenda items
14:02:43 <dprince> * open discussion
14:03:09 <dprince> There are no one-off agenda items this week. So anything extra we can just do in open discussion I guess
14:03:58 <d0ugal> shadower: Didn't you have the validations on the agenda?
14:04:02 <d0ugal> validations API*
14:04:05 <shardy> dprince: I wanted to get feedback re the removal of tripleoclient and tripleo-common stable/liberty branches
14:04:10 <shardy> ref my ML thread
14:04:17 <shardy> (we can cover it in open discussion)
14:04:27 <shadower> d0ugal: yeah but that belongs under "Specs" I think
14:04:27 <dprince> shardy: yep, lets just do it there
14:04:32 <dprince> shardy: thanks for mentioning
14:04:36 <d0ugal> shadower: Ah, makes sense.
14:04:51 <dprince> okay, lets get started
14:04:55 <dprince> #topic bugs
14:05:26 <dprince> one puppet bug worth mentioning is
14:05:32 <dprince> #link https://bugs.launchpad.net/puppet-keystone/+bug/1528308
14:05:32 <openstack> Launchpad bug 1528308 in puppet-keystone "Keystone_endpoint warning everywhere by default" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)
14:05:43 <EmilienM> o/
14:05:55 <EmilienM> we're going to land the fix today
14:05:57 <dprince> This is something getting fixed in puppet which may momentarily break us (while all the patches land)
14:06:21 <EmilienM> if we land all patches together, we will break CI ~20 min
14:06:56 <dprince> EmilienM: would like to check and see if there is anything we could do to make this more upgrade friendly. I guess is is really just a dependency thing between the puppet modules though
14:07:11 <EmilienM> yeah, it's very rare
14:07:31 <EmilienM> we can do a retrospective after we fix the bug
14:07:39 <EmilienM> and try to fix a sane solution
14:08:31 <dprince> any other bugs to mention this week?
14:10:51 <dprince> okay, moving on
14:10:56 <dprince> #topic  Projects releases or stable backports
14:12:41 <dprince> shardy: any stablish updates this week
14:12:41 <marios> no real update, but i'm trying to cherry pick as many reviews as i can to stable/liberty so we can cut a release there. grateful for the reviews so far
14:12:47 <dprince> marios: ah, cool
14:12:58 <dprince> marios: thanks for the update
14:13:21 <shardy> dprince: Nothing to update, all seems to be going OK with stable CI and folks are starting to review more, so thanks! :)
14:13:41 <marios> dprince: np (to clarify, for tripleo heat templates specifically)
14:13:49 <shardy> Also, my patch for stable dashboard for gerrit-dash-creator landed, I find it useful so perhaps others may too:
14:13:55 <shardy> https://review.openstack.org/#/c/256379/
14:14:14 <shardy> it can probably be improved, e.g if anyone can figure out how to exclude patches not passing CI
14:15:37 <dprince> shardy: nice, we should like the dashboard into our TripleO wiki's where appropriate too perhaps
14:16:15 <shardy> dprince: Yeah, we could - I like having it a separate dashboard, because stable reviews require a different approach than normal patches to master
14:16:32 <dprince> shardy: yes, agree
14:16:56 <shardy> I also posted https://review.openstack.org/#/c/260144/ which shows how to build a stable/liberty overcloud on a master undercloud
14:17:13 <shardy> that's overlapping with my backwards compat topic tho ;)
14:17:26 <shardy> tl;dr - it works :)
14:18:08 <dprince> #topic CI
14:18:56 <dprince> one infrastructure thing to mention here is derekh resolved an issue on one of the compute hosts in our cloud
14:19:06 <derekh> rh1 back to normal, I rebooted comput nodes, rebuilts Test envs and a couple of other things last week some time,
14:19:07 <dprince> so we get more reliable nodepool instances again
14:19:24 <derekh> yup, ci instances not failing any more
14:19:26 <dprince> derekh: nice, thanks for the updates
14:19:27 <shardy> nice
14:19:33 <derekh> jobs seems stable enough,
14:19:48 <derekh> I was on PTO yesterday, was there an outage?
14:19:48 <shardy> I had a question re the stable/liberty overcloud on a master undercloud and CI
14:20:00 <derekh> looks like there might have been
14:20:21 <shardy> if we used that approach as a basis for an upgrade job, would it make sense to have a new "upgrades-liberty-master" job, or wire it in, e.g to the exising HA job?
14:20:54 <shardy> I'm wary of making the existing jobs take a lot longer, but also another job has significant overlap
14:21:20 <derekh> shardy: I thought the plan was to use the stable job, then do an upgrade at the end
14:21:42 <derekh> shardy: ya it will make things longer, suppse we could try it and see how long it gets
14:21:45 <shardy> derekh: ack, we can do that, but that doesn't provide coverage of the requirment for a backwards compatible undercloud
14:22:00 <shardy> unless we update the undercloud as part of the job I guess
14:22:06 <dprince> shardy: yeah, lets have a different job for the backwards compat undercloud case
14:22:31 <shardy> my point was, for upgrades, we always expect folks to upgrade the undercloud first
14:22:51 <derekh> shardy: dprince were probably close to the point where extra jobs are going to add to zuul queues at peak times every day
14:23:00 <shardy> so, if we test master-undercloud deploying a liberty overcloud, then the next step could be doing the upgrade of the overcloud
14:23:19 <derekh> shardy: dprince thats a hunch but I can take a look and see for sure, we may be able to squeeze one more in
14:23:26 <shardy> derekh: is there any way we can e.g reuse images between jobs or something, to reduce the overall load?
14:24:00 <derekh> shardy: yes get the periodic job working and then use the artifacts it generates
14:24:09 <dprince> shardy: like perhaps have upgrade jobs not use DIB at all?
14:24:23 <derekh> shardy: for that we need to merge these 2 patches
14:24:28 <derekh> https://review.openstack.org/#/c/244519/
14:24:29 <shardy> dprince: Yeah, something like that, just consume an existing stable-liberty image, then upgrade it to master
14:24:34 <derekh> https://review.openstack.org/#/c/244526/
14:25:02 * d0ugal can't access anything on gerrit
14:25:04 <shardy> derekh: Ok, thanks, I'll check those out and we can chat more about this later
14:25:06 <derekh> My plan was to have the periodic job save aritfacts that couldbe used in other jobs
14:25:10 <marios> d0ugal: same
14:25:25 <shadower> d0ugal: marios: same but it worked after ~5th reload
14:25:28 <derekh> by that stalled as those patches are still waiting
14:25:31 <shardy> derekh: cool, that sounds like it may work then
14:25:36 <marios> shadower: has been weird since ~ top of the hour
14:25:41 <shadower> yea
14:25:50 <derekh> shardy: yup
14:25:55 <shadower> just saying that if you keep trying you may get lucky
14:26:02 <d0ugal> shardy: k, we can collectively DDoS :)
14:26:08 <d0ugal> I meant shadower
14:26:11 <marios> shadower: thanks
14:27:49 <dprince> any other CI things then?
14:27:58 <derekh> dprince: nothing from me
14:28:10 <dprince> #topic Specs
14:28:57 <shadower> so, mandre and I wrote a spec + PoC for an API that runns validations on various stages of the deployment
14:29:03 <shadower> to catch bad configs, hw issues etc. early
14:29:10 <shadower> https://review.openstack.org/#/c/255792/
14:29:39 <shadower> we'd like to build it on top of the TripleO API and just add the new functionality
14:29:41 <shadower> would appreciate reviews
14:29:49 <shardy> shadower: nice - I noticed yesterday that our existing validations don't actually stop the deployment when an error is detected
14:30:02 <d0ugal> shardy: On the CLI?
14:30:04 <shadower> shardy: the ones in tht?
14:30:15 <shardy> so that's probably a tripleoclient patch to at least prompt "are you sure" (unless there's one already posted)
14:30:27 <shardy> d0ugal: yeah, the CLI
14:30:42 <d0ugal> heh, I am sure they did stop it at one point :)
14:30:55 <shardy> minor thing, just mentioning we need to prompt the user and/or have a --force mode where we always stop by default
14:31:14 <d0ugal> shardy: Maybe you need this: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L887-L894
14:31:24 <d0ugal> Anyway - that's a different topic.
14:31:25 <shardy> d0ugal: hehe, OK I'll check it out, maybe a regression (we should probably test with a known-bad config)
14:32:20 <shardy> d0ugal: Yeah, I guess I'm saying that should default to True
14:32:29 <d0ugal> So on the point of shadower's spec, it would be good to see more reviews on the API spec and patches.
14:32:32 <d0ugal> shardy: I agree
14:32:39 <shadower> yes please
14:32:51 * d0ugal is having trouble getting links
14:33:03 <shadower> it's rather tiny because it just lists the extensions to the tripleo api
14:33:35 <shadower> so we'd piggyback for things like auth, db, config etc.
14:33:47 <marios> thanks shadower will try and revisit tomorrow morning reviews time (gerrit still down here can't even peak)
14:33:51 <akrivoka> shadower: the addition of validations to the api will require adding a database to the tripleo api, correct?
14:34:19 <shardy> I thought we expected tripleo-api to be stateless?
14:34:23 <shadower> akrivoka: yeah, pretty much
14:34:46 <shadower> I mean, we'd need to store the validation results someplace and be able to query them
14:35:00 <d0ugal> shardy: Ideally we wanted that, but I don't think swift will work for us
14:35:06 <d0ugal> for the API in general
14:36:40 <shardy> d0ugal: Ok, I guess now isn't really the time to discuss that, we can chat more later
14:37:08 <d0ugal> shardy: Sure, I need to send an email to the list about it - I had it as a topic in ast weeks meeting
14:37:37 <shardy> d0ugal: ack, I missed last weeks meeting unfortunately, a ML thread would be good
14:37:42 <dprince> d0ugal: okay, will look for your email
14:37:47 <shadower> shardy, d0ugal: the validations will def want to be able to store state. I guess we could use swift instead of a real db but then we'd have to write a db (at least indexes + querying) on top of it
14:37:54 <dprince> and perhaps we can add to the next meeting agenda if needed as well
14:38:22 <d0ugal> dprince: Maybe not next week thoug :)
14:38:30 <shadower> haha
14:38:44 <dprince> d0ugal: yeah, I gotcha
14:38:44 <shadower> isn't that how unpopular laws get passed in politics?
14:38:45 <d0ugal> oh, I just noticed I miss-read. nvm
14:38:58 <shardy> shadower: Ok, thanks, it'd be interesting to discuss more on the ML about that
14:39:18 <shadower> shardy: yeah, prolly makes more sense there
14:39:37 <dprince> any other specs things then?
14:40:02 <dprince> I would add that I'll post an updated version of the composable roles spec soon with slight modifications
14:42:21 <dprince> #open discussion
14:42:33 <shardy> #link http://lists.openstack.org/pipermail/openstack-dev/2015-December/082794.html
14:42:46 <shardy> So, I started a ML thread about backwards compatibility
14:42:50 <shardy> feedback would be welcome
14:43:24 <shardy> the question is, if we get CI coverage of e.g deploying a liberty overcloud with master tripleoclient and tripleo-common, do we still need the stable branches for those repos
14:43:57 <slagle> i replied, but i'd rather see backwards compatibility there instead of stable branches
14:44:06 <shardy> Or, do we expect coupling (especially with tripleo-common) with the master/mitaka services which would prevent removing them?
14:44:35 <shardy> slagle: thanks - I guess I'm still a little unsure if we need both backwards compat and stable branches (especially for -common)
14:44:56 <dprince> I feel like many projects bring this up. And ultimately the cleanest solutions is stable branches
14:45:01 <shardy> e.g we may need tripleo-common to be able to deploy a liberty overcloud, but it might rely on mitaka undercloud features?
14:45:18 <dprince> maintaining the logic in master to support all cases just gets confusing sometimes
14:45:41 <slagle> all the clients are trending towards strict backwards compatibility afaict
14:45:47 <slagle> not stable branches
14:45:55 <shardy> dprince: for clients, lifeless is proposing to move away from that model in a cross-project spec
14:46:22 <shardy> I'm leaning towards leaving the stable branches for now, and implementing a CI job which proves we maintain backwards compat on master
14:46:36 <shardy> then, if it becomes clear that e.g we don't need the tripleoclient branch, we can remove it?
14:46:40 <dprince> shardy: sure, I guess the assumption is there that you've got something you like to start with
14:47:08 <dprince> There are remnants of tripleoclient I think we'd like to see evolve before we commit to them long term
14:47:10 <shardy> Also, I wanted feedback from trown re the RDO implications of removing a branch for an existing release
14:47:18 <shardy> vs just not cutting another one for stable/mitaka
14:47:52 <shardy> Ok, so it sounds like the consensus is yes to the CI coverage, but leave the branches in place for now?
14:48:44 <dprince> maybe a rough consensus. I don't see any harm in adding CI coverage though
14:49:02 <shardy> Ok, well I'll do that first then and we can revisit the branches discussion after
14:49:05 <shardy> thanks
14:49:16 <shardy> feel free to add further thoughts to the ML thread
14:50:31 <dprince> do we want to have the meeting next week?
14:50:43 * d0ugal wont be here
14:50:49 * shardy won't be here
14:50:49 <dprince> I know many people may be off the week between Christmas and New Years
14:50:49 * derekh no here either
14:51:26 <shardy> I'd say cancel it
14:51:44 <marios> bah humbug!
14:51:47 <dprince> okay, I will send a mail to the list to cancel next week.
14:52:04 <dprince> Next TripleO meeting w/ be Jan. 5th then
14:54:14 <dprince> any other topcis then for this week?
14:54:16 <dtantsur> folks, wanted to draw your attention to the profile matching patch https://review.openstack.org/#/c/250405 which proved to be much bigger than anyone could expect..
14:54:56 <shardy> dtantsur: thanks, will check it out
14:54:59 <dtantsur> (I'll submit a couple more tiny changes as soon as gerrit gets up)
14:56:09 <shardy> dtantsur: can that consume the profiles from the nodes json file we use for registering the nodes?
14:56:18 <dtantsur> associated docs patch: https://review.openstack.org/#/c/257867/ (if it makes things clear)
14:56:34 <dtantsur> shardy, it will reuse whatever will be provided via 'capabilities' in the nodes json
14:56:50 <shardy> dtantsur: Ok, thanks, so manual tagging is still possible
14:57:04 <dtantsur> shardy, yes, and direct ironic update will work too
14:57:11 <shardy> ++ sounds good
14:57:18 <dtantsur> shardy, check out the docs patch first, it explains the expected flow
14:57:23 <shardy> ack, thanks
14:57:30 <dtantsur> thanks
14:57:31 <dprince> dtantsur: yeah, so long as manual tagging is still possible via the undercloud json file it sounds fine
14:59:12 <dprince> thanks everyone
14:59:15 <dprince> have a nice break
14:59:30 <dtantsur> happy holidays
14:59:32 <d0ugal> Thanks! See you next year!
15:00:01 <dprince> #endmeeting tripleo