14:00:22 <EmilienM> #startmeeting tripleo
14:00:27 <openstack> Meeting started Tue Dec 13 14:00:22 2016 UTC and is due to finish in 60 minutes.  The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:31 <openstack> The meeting name has been set to 'tripleo'
14:00:33 <EmilienM> #topic agenda
14:00:35 <EmilienM> * one off agenda items
14:00:37 <EmilienM> * bugs
14:00:39 <EmilienM> * Projects releases or stable backports
14:00:41 <EmilienM> * CI
14:00:43 <EmilienM> * Specs
14:00:45 <EmilienM> * open discussion
14:00:47 <EmilienM> who is around today?
14:00:50 <mwhahaha> o/
14:00:50 <jpich> o/
14:00:51 <Ng> .o/
14:00:53 <chem> o/
14:00:53 <shadower> hey
14:00:53 <jtomasek> o/
14:00:54 <shardy> Hi!
14:00:55 <marios> \o
14:01:02 <beagles> o/
14:01:07 <akrivoka> hiya \o
14:01:21 <hrybacki> o/
14:01:28 <rbrady> o/
14:01:38 <ccamacho> o/
14:01:42 <EmilienM> what a nice gang here
14:01:43 <gfidente> o/
14:01:59 <gfidente> oh you said that before I o/
14:02:22 <bandini> o/
14:02:43 <EmilienM> we had some actions
14:02:52 <EmilienM> team to review tripleo spec reviews policy https://review.openstack.org/#/c/406264/
14:02:55 <EmilienM> jpich to breakdown https://review.openstack.org/#/c/406264/
14:02:55 <trown> o/
14:02:57 <EmilienM> beagles to abandon old reviews and propose a policy to document it
14:02:59 <EmilienM> EmilienM to test tripleo with centos7.3 and see how it goes
14:03:01 <EmilienM> EmilienM to send update on ML about releases ocata-2 and newton
14:03:24 <weshay> o/
14:03:33 <pradk> o/
14:03:33 <sshnaidm> o/
14:03:45 <EmilienM> I need to postpone my actions
14:03:48 <beagles> I did a cleanup but got sidetracked off of policy edit.. .will get to that this week it seems
14:03:48 <thrash> o/
14:03:56 <jpich> Policy broken down, the straight up migration from the wiki has merged, the new tidbit is at https://review.openstack.org/#/c/407533/
14:04:02 <EmilienM> #action EmilienM to test tripleo with centos7.3 and see how it goes
14:04:10 <EmilienM> #action EmilienM to send update on ML about releases ocata-2 and newton
14:04:21 <matbu> o/
14:04:30 <EmilienM> beagles: thanks, are you doing all tripleo repos?
14:05:06 <EmilienM> jpich: approved
14:05:17 <beagles> EmilienM: almost all, I skipped diskimage-builder - some unfamiliar names on there
14:05:24 <shardy> EmilienM: aren't we doing a yum update as part of the image build?
14:05:30 <jpich> EmilienM: Cool, thanks
14:05:31 <beagles> EmilienM: but will revisit all the same
14:05:51 <shardy> all my local boxes (including undercloud) are yum updated and therefore running centos7.3
14:05:59 <beagles> EmilienM: I'll reach out to those that seem still active if they are still around before abandoning them
14:06:24 <EmilienM> shardy: right, that's why I wanted to see how it works, I've heard some issues with nova and qemu
14:06:25 <trown> EmilienM: wrt 7.3 testing... we are doing it now because we run yum update :)
14:06:26 <beagles> #action beagles propose policy edit for periodically abandoning old reviews
14:06:31 <EmilienM> jpena sent an email about it
14:06:42 <EmilienM> trown: ok nice
14:06:48 <shardy> EmilienM: Ok, I guess I assumed we did a yum update in CI too
14:06:51 <trown> EmilienM: so everywhere that is using centos and runs yum update is using 7.3
14:06:54 <EmilienM> beagles: you can skip dib, it's fine
14:07:00 <beagles> EmilienM: okay
14:07:18 <sshnaidm> shardy, we do
14:07:22 <trown> EmilienM: overcloud images might be the only place that is not tested yet in tripleo-ci, because I don't thing we run update there
14:07:28 <EmilienM> and we don't have any trouble with the qemu/nova thing?
14:07:35 <shardy> Ok, so we're already testing 7.3 then, which is what I thought
14:07:50 <sshnaidm> trown, https://github.com/openstack-infra/tripleo-ci/blob/3bd847ccd63d209c13091dd4f5d8f3de3102aadf/scripts/common_functions.sh#L106-L106
14:08:21 <EmilienM> ok let's start the agenda
14:08:28 <trown> sshnaidm: nice
14:08:30 <EmilienM> #topic off agenda items
14:08:34 <EmilienM> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:08:41 <EmilienM> ccamacho: shoot
14:09:03 <shardy> EmilienM: I've had several problems with nova, but none related to centos 7.3 AFAIK, maybe there are issues I've not hit yet tho
14:09:03 <ccamacho> we are going to send some TripleO cheatsheets to FOSDEM and devconf
14:09:22 <ccamacho> feedback/reviews are welcomed there https://github.com/ccamacho/tripleo-cheatsheet
14:09:28 <weshay> nice
14:09:49 <weshay> that cheatsheet made it to reddit :)
14:09:53 <EmilienM> shardy: are they reported somewhere?
14:10:07 <pradk> ccamacho, thats awesome
14:10:17 <EmilienM> ccamacho: nice work here :)
14:10:25 <ccamacho> but they are WIP
14:10:29 <EmilienM> ccamacho: do you need us to take some actions? review them, etc?
14:10:41 <ccamacho> mostly give feedback for adding content there
14:10:44 <shardy> EmilienM: yes https://bugs.launchpad.net/tripleo/+bug/1649341
14:10:44 <openstack> Launchpad bug 1649341 in puppet-nova "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz)
14:10:44 <marios> ccamacho: looks great weshay noway :)
14:11:06 <d0ugal> o/
14:11:08 <ccamacho> becasue we should send them for printing at the end of the year
14:11:20 <shardy> EmilienM: I then updated everything to delorean-current and nova then has RPC problems, so trunk may be broken, but I don't think it's because of centos
14:11:21 <EmilienM> shardy: oh this one, mwhahaha is on it iirc
14:11:37 <shardy> EmilienM: Yeah we worked through it yesterday
14:11:40 <EmilienM> ccamacho: do you want to use ML to share infos here and get wide feedback?
14:11:52 <ccamacho> awesome, Ill send then an ML
14:12:03 <EmilienM> shardy: thanks. I'll try to help Alex also, we might have found some chicken&egg problem here
14:12:08 <EmilienM> ccamacho: thanks
14:12:10 <ccamacho> openstack-dev [TripleO] as usual
14:12:11 <shardy> ccamacho: will these be associated with some talk?
14:12:13 <EmilienM> bandini: go ahead
14:12:16 <bandini> thanks
14:12:44 <ccamacho> shardy I dont think so, they are general cheatsheets for people jumping into TripleO
14:12:52 <bandini> so yeah I'd like some high-level feedback on https://review.openstack.org/#/c/408726/ which fixes https://bugs.launchpad.net/tripleo/+bug/1615983
14:12:53 <openstack> Launchpad bug 1615983 in tripleo "if $::hostname == boostrap_node checks are fragile" [High,In progress] - Assigned to Michele Baldessari (michele)
14:13:06 <bandini> I am not entirely happy with the approach I have taken as it is a lot of code churn
14:13:11 <bandini> so am looking for other ideas
14:13:21 <bandini> maybe shardy or dprince have some thoughts there?
14:13:27 <marios> bandini: is this for ocata?
14:13:30 <EmilienM> so the idea is to build an array of nodes for each service
14:13:34 <EmilienM> and take the first one for the bootstrap?
14:13:37 <marios> bandini: i mean rather, it isn't meant to be for newton? (me checks bug)
14:13:48 <EmilienM> marios: ocata man... we don't backport features ;-)
14:14:01 <shardy> ccamacho: Ok, I'll provide feedback on the ML - I wonder if targetting quickstart might be easier as the deploying cheat-sheet is pretty complex for an unassisted new user
14:14:02 <marios> bandini: fwiw will have a look added myself to reviews, sorry completely missed that one
14:14:07 <EmilienM> marios: this is for Composable HA
14:14:09 <bandini> well this is a bug, but it is extremely unlikely to hit newton
14:14:13 <marios> EmilienM: thanks
14:14:24 <bandini> we hit it 100% with composable ha though as soon as we put a pacemaker resource in two different roles
14:14:24 <EmilienM> bandini: well, not sure we can backpor that one
14:14:34 <EmilienM> it changes the interface quite a bit
14:14:34 <ccamacho> shardy awesome we can include also a OOOQ cheatsheet
14:14:36 <bandini> EmilienM: yeah I don't think we need to backport anything to newton
14:14:40 <shardy> (and, we're trying to move away from instack-virt-setup)
14:14:57 <bandini> EmilienM: plus I'd first like to get consensus on the approach, maybe there are cleaner alternatives I have not thought of
14:15:38 <bandini> anyway, if anyone can throw some cluebones my way on that review, that would be awesome ;)
14:15:51 <EmilienM> bandini: I had a quick look and it looks good to me
14:16:03 <EmilienM> bandini: but i'll spend more time. This thing looks critical
14:16:31 <EmilienM> #action HA & puppet experts to look https://review.openstack.org/#/c/408726/
14:16:33 <bandini> yeah the more composable we get the more we will need to fix that one way or the other
14:17:05 <EmilienM> bandini: thanks!
14:17:12 <EmilienM> anything else before we go to regular agenda?
14:17:18 <dprince> bandini: I think you may be onto the right track
14:17:18 <bandini> I am good, thanks
14:17:33 <EmilienM> yes I agree. Though we haven't test it yet
14:17:33 <ccamacho> mwhahaha quick question, testing upstream composable upgrades, it hangs ... :(   and I only found an connection error on heat as we are not setting up the transport_url ( https://github.com/openstack/puppet-heat/commit/b865dffaa68602f520d3b6fe6bfbe8f6f8a30cf1#diff-0e9e2411cdc9fbc9f7df5a11085ba2a9 ) do you think this can break the deployment?  Currently testing https://review.openstack.org/#/c/410171/ but hitting an
14:17:44 <dprince> bandini: one suggestion would be could we keep the mechanism in t-h-t such that we can use a similar approach for containers
14:17:50 <EmilienM> and I would like to see a CI job that test it, since slagle did the 3-nodes CI job working
14:18:24 <bandini> dprince: oh I see, I will look into that
14:18:30 <dprince> bandini: with containers we aren't using Puppet for the db_sync, rather we would execute the db_sync by using docker-cmd to run a quick db_sync using the container itself
14:18:34 <EmilienM> ccamacho: a bit out of topic now, can we stick on agenda?
14:18:45 <ccamacho> EmilienM sure
14:18:53 <bandini> oh I see
14:19:25 <shardy> dprince, bandini: yeah but if we move this back into tht, we should avoid reintroducing a hard-coded 0 reference again
14:19:59 <shardy> perhaps we can look into using yaql first() or something, as the lookup on the ResourceGroup ends up being by name, not by index
14:20:17 <bandini> shardy: ack, noted
14:20:18 <shardy> we fixed that by moving the lookup into puppet-tripleo, where it is by index
14:20:34 <EmilienM> and export some hiera for the first() occurence?
14:20:37 <dprince> shardy: would be nice, but I'd give the generic mechanism (works across containers and baremetal) priority. But ++ on trying first() w/ yaql
14:20:50 <shardy> EmilienM: Yeah something like that
14:21:11 <dprince> EmilienM: yes, I'd say keep the puppet modules dumber. Just assume they are told whether to sync or not to sync by heat
14:21:14 <shardy> dprince: ack, I'm totally fine with moving it back to tht, but I'd rather avoid the hard-coded "0" issue we had before
14:21:18 <mwhahaha> ccamacho: I'll take a look today and get back to you
14:21:35 <EmilienM> dprince: ++
14:21:42 <ccamacho> mwhahaha thanks man
14:22:11 <EmilienM> are we done with $topic?
14:22:45 <EmilienM> #topic bugs
14:22:50 <EmilienM> #link https://launchpad.net/tripleo/+milestone/ocata-2
14:23:17 <EmilienM> we have a huge number of bugs and I would like to drop a note about that
14:23:26 <EmilienM> 16 Confirmed, 50 Triaged, 41 In Progress
14:24:04 <EmilienM> by the end of ocata-3, we'll freeze features in TripleO and will move efforts into fixing bugs and making things more stable until we release the final ocata
14:24:43 <EmilienM> the only exception will be composable upgrades I guess
14:24:50 <EmilienM> any feedback on that ^ is welcome
14:25:07 <marios> EmilienM: end of january for O3 right? yes i fully expect we'll be backporting to stable/ocata for at least the last bits
14:25:19 <EmilienM> marios: yes
14:25:39 <EmilienM> marios: don't think about branching now, we'll wait a bit more than last time
14:25:44 <EmilienM> so we reduce the # of backports
14:25:53 <marios> EmilienM: shardy matbu chem ccamacho and anyone else that is interested we were thinking of having an upgrades squad syncup this week to talk about how to proceed with that
14:26:06 <EmilienM> marios: that's excellent
14:26:09 <ccamacho> marios++
14:26:28 <EmilienM> marios: feel free to send an email when it's done, with a summary of your progress
14:26:30 <matbu> marios: +1
14:26:36 <marios> EmilienM: ack
14:26:40 <chem> marios: cool
14:26:42 <EmilienM> something on openstack-dev [tripleo] [upgrade] blabla
14:26:45 <EmilienM> marios: thanks man
14:26:47 <matbu> marios: irc mtg ?
14:26:58 <tikitavi> Hello, can anybody help me? I'm trying to publish installation guide for our service to docs.openstack.org
14:27:07 <marios> matbu: possibly... whatever works. personally i think a call may be better.
14:27:22 <EmilienM> tikitavi: we're in the middle of a meeting, please ask on #openstack-dev or #openstack-doc
14:27:27 <matbu> marios: ack, up to you
14:27:36 <EmilienM> do we have outstanding bugs this week?
14:27:47 <EmilienM> one I see is https://bugs.launchpad.net/bugs/1649341
14:27:50 <openstack> Launchpad bug 1649341 in puppet-nova "Undercloud upgrade fails with "Cell mappings are not created, but required for Ocata"" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz)
14:27:54 <EmilienM> but shardy / mwhahaha are already working on it
14:28:02 <EmilienM> is there anything we have missed?
14:29:06 <EmilienM> #topic Projects releases or stable backports
14:29:18 <EmilienM> so I missed it last week but I need to send an email
14:29:35 <EmilienM> this week, we'll release ocata-2
14:29:46 <EmilienM> i'll take care of moving bugs/blueprints to ocata-3
14:30:14 <EmilienM> also, if a blueprint has zero or slow progress, I'll contact the assignee and propose a postpone to pike-1
14:30:40 <EmilienM> I plan to release a new Newton version by next week also, when ocata-2 will be done
14:31:05 <EmilienM> #link https://releases.openstack.org/ocata/schedule.html
14:31:17 <EmilienM> bookmark this link if you need any info about release ^
14:31:49 <EmilienM> but after January 27th, we'll stop implementing features in TripleO
14:32:12 <EmilienM> and work more on bug fixing and stabilization
14:32:29 <EmilienM> I think that's the plan. Any feedback or questioN?
14:33:20 <trown> seems reasonable
14:33:27 <shardy> +1 sounds good to me
14:33:30 <EmilienM> it means, after ocata-3, we have 5 weeks to stabilize things
14:33:41 <EmilienM> (if we use the trailing cycle)
14:34:10 <EmilienM> marios: for composable upgrades, we'll still accept patches, as it's the most critical thing in this cycle.
14:34:18 <bandini> +1
14:34:24 <marios> EmilienM: ack thanks for the clarification
14:34:41 <EmilienM> #action EmilienM to start postponing bugs / blueprints to ocata-3 or pike-1
14:34:58 <EmilienM> #topic CI
14:35:50 <EmilienM> I would like CI folks to review https://review.openstack.org/#/q/topic:3-node please
14:36:21 <EmilienM> slagle: would it be possible to have a 3-node CI job that test composable HA? or do we need more nodes?
14:37:16 <EmilienM> also, I have an update about ovb jobs and recent problems
14:37:16 <weshay> nice
14:37:41 <EmilienM> derekh, sshnaidm and panda worked on rh1 issues and it seems things are working again
14:37:50 <EmilienM> we need to merge this patch once it passes CI https://review.openstack.org/#/c/410131/
14:38:08 <EmilienM> #action CI folks to review https://review.openstack.org/#/q/topic:3-node and https://review.openstack.org/#/c/410131/
14:38:36 <sshnaidm> EmilienM, I see ha job passed, so should be ok
14:38:39 <panda> ping test on ha job for the ovb fix just passed
14:38:44 <EmilienM> bandini: could we run a 2-nodes overcloud and test composable HA?
14:38:51 <EmilienM> or do we need more nodes?
14:39:31 <bandini> EmilienM: I think we can, I need to have a think about it
14:39:53 <EmilienM> bandini: ok thx, please let me know post meeting, it's really important that we have coverage for that
14:40:02 <bandini> EmilienM: agreed
14:40:09 <EmilienM> thx
14:40:20 <EmilienM> any other info about CI?
14:41:07 <EmilienM> #topic Specs
14:41:36 <EmilienM> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:42:39 <EmilienM> I would like to remind that specs not approved by ocata-2 (this week) won't be implemented in ocata and postponed to pike-1
14:43:14 <EmilienM> the GUI and validations specs look in good shape, I'll review them this week
14:43:26 <shadower> thanks :-)
14:44:23 <shardy> #link https://bugs.launchpad.net/tripleo/+milestone/pike-1
14:44:38 <EmilienM> yeah, we already creating the milestones :D
14:44:44 <shardy> Note folks can target things to pike now, so we track them on the roadmap even if they aren't expected to land for Ocata
14:45:10 <EmilienM> do we need to discuss about a spec in specific?
14:45:53 <EmilienM> let's open the discussion
14:45:57 <EmilienM> #topic open discussion
14:46:01 <shardy> There has been some ML discussion about fully composable networks
14:46:03 <EmilienM> feel free to bring any topic or question
14:46:04 <shardy> http://lists.openstack.org/pipermail/openstack-dev/2016-December/108779.html
14:46:28 <shardy> Interested to get feedback on the ideas in that thread - I think we might consider it as a feature for pike
14:46:34 <EmilienM> matbu: what is the progress on upgrade job?
14:46:51 <EmilienM> shardy: composable everything! :-)
14:47:05 <shardy> EmilienM: hehe :)
14:47:26 <shardy> EmilienM: I think the upgrades job patch is getting pretty close but there are still some issues to be worked out
14:47:44 <shardy> I'm trying to get it working locally, so we can iterate faster with the testing
14:47:50 <EmilienM> shardy: nice
14:48:15 <shardy> will share notes when I have it functional
14:48:21 <EmilienM> shardy: it would be nice to start a specs for the composable networks, maybe trozet can start it
14:48:26 <matbu> EmilienM: i think we are realyl close, i'll make an update of the current review (i'm testing the change atm)
14:48:37 <shardy> EmilienM: agreed - it's been discussed a few times but I don't think there is a spec
14:48:38 <EmilienM> he's not online here but I'll check with him if he's interested
14:49:00 <matbu> EmilienM: the job was green yesterday twice
14:49:09 <shardy> EmilienM: as mentioned in the ML thread, we may end up partially implementing it as a side-effect of those two bugs related to composable roles and network isolation
14:50:20 <EmilienM> matbu: how can we make a pingtest?
14:50:39 <shardy> EmilienM: we can run pingtest when we've added the services pingtest requires
14:50:40 <EmilienM> shardy: can we create a basic pingtest that create keystone resources?
14:50:42 <matbu> EmilienM: hm actually we need a specific pingtest
14:50:56 <shardy> EmilienM: but for now I was planning to add a basic sanity smoke test which uses e.g openstackclient
14:51:09 <EmilienM> shardy: sounds good enough
14:51:31 <EmilienM> shardy: as we only deploy keystone for now
14:51:41 <EmilienM> matbu, shardy: how many time takes the job?
14:51:48 <EmilienM> are we hitting timeout?
14:51:51 <matbu> EmilienM: it took 59
14:51:54 <matbu> minutes
14:51:58 <shardy> yup, then we'll add e.g glance and create/delete an image, eventually moving over to the pingtest stack instead
14:52:00 <EmilienM> in total?
14:52:05 <matbu> yep
14:52:10 <EmilienM> wow, that's good
14:52:13 <matbu> this is really fast
14:52:16 <EmilienM> yeah, next step is glance :D
14:52:23 * EmilienM dancing
14:52:28 <matbu> well i wrong i think it's 49 minutes
14:52:40 <shardy> I think that's not actually testing the full upgrade yet, as it's deploying master tht, but yeah it looks pretty good
14:53:11 <shardy> and I think we can combine the upgrade with converge steps (just for CI) when we land https://review.openstack.org/#/c/409857/
14:53:17 <EmilienM> shardy: oh, you mean we don't upgrade everything else?
14:53:27 <EmilienM> shardy: we don't run yum update I mean?
14:53:30 <matbu> shardy: it does the converge now
14:54:17 <shardy> EmilienM: No it does run the yum update, it's just not using the right tht yet
14:55:14 <EmilienM> ok
14:55:26 <EmilienM> folks, 5 min left, do we have anything else to discuss this week?
14:56:28 <EmilienM> if I don't freeze, I'll see you next week! thanks for your time and have fun
14:56:33 <EmilienM> #endmeeting