14:00:47 <mwhahaha> #startmeeting tripleo
14:00:47 <mwhahaha> #topic agenda
14:00:47 <mwhahaha> * Review past action items
14:00:47 <mwhahaha> * One off agenda items
14:00:47 <mwhahaha> * Squad status
14:00:47 <mwhahaha> * Bugs & Blueprints
14:00:47 <openstack> Meeting started Tue Oct 10 14:00:47 2017 UTC and is due to finish in 60 minutes.  The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:48 <mwhahaha> * Projects releases or stable backports
14:00:48 <mwhahaha> * Specs
14:00:48 <mwhahaha> * open discussion
14:00:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:49 <mwhahaha> Anyone can use the #link, #action and #info commands, not just the moderatorǃ
14:00:50 <mwhahaha> Hi everyone! who is around today?
14:00:51 <lvdombrkr> folks, where i can see pacemaker detailed logs, for example reason why node not become as master etc
14:00:51 <openstack> The meeting name has been set to 'tripleo'
14:00:53 <beagles> o/
14:00:54 <dprince> hi
14:00:55 <jtomasek> o/
14:01:00 <Camacho|phone> Hola!
14:01:01 <abishop> o/
14:01:05 <jaosorior> o/
14:01:07 <marios> o/
14:01:08 <cdearborn> 0/
14:01:08 <jfrancoa|lunch> o/
14:01:09 <d0ugal> Hey
14:01:11 <gfidente> o/
14:01:17 <jistr> hi
14:01:21 <thrash> o/
14:01:22 <fultonj> o/
14:01:25 <chem> o/
14:01:32 <EmilienM> o/
14:01:56 <larsks> o/
14:02:32 <jrist> o/
14:02:47 <mwhahaha> hey everyone, let's go
14:02:51 <mwhahaha> #topic review past action items
14:02:52 <mwhahaha> mwhahaha to follow up with shardy about services and ovb jobs
14:03:10 <mwhahaha> so I think shardy piped in last week after i made this item, not sure we have much to do on this at the moment
14:03:36 <mwhahaha> i think there's also efforts to move ovb to rdo cloud so it's probably not the best time to make changes. so we can circle back around to this later in the cycle
14:04:05 <mwhahaha> the next item was ceph-upgrade ci
14:04:21 <mwhahaha> fultonj, gfidente: any update on ceph-upgrade ci?
14:04:27 <fultonj> that's in the integration squad pad
14:04:35 <mwhahaha> k we'll get to that later
14:04:48 <fultonj> ack
14:04:50 <mwhahaha> mwhahaha to move bugs <= medium to queens-2 and review > medium for validity - DONE
14:04:57 <mwhahaha> that's in ont he past action items from last week
14:05:15 <trown> o/ sort of ci squad is also having retro
14:05:32 <mwhahaha> #topic one off agenda items
14:05:32 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:05:33 <EmilienM> trown: now?
14:05:37 <trown> EmilienM: ya
14:05:43 <EmilienM> trown: ouch, bad timeing :/
14:05:43 <mwhahaha> (mwhahaha) Newton EOL & CI review
14:05:57 <mwhahaha> fyi, i've asked for a stay of EOL for tripleo projects
14:05:58 <mwhahaha> http://lists.openstack.org/pipermail/openstack-dev/2017-October/123221.html
14:06:05 <mwhahaha> but it's been asked to review with the stable team what that means
14:06:16 <EmilienM> have we got feedback on that?
14:06:16 <mwhahaha> so i'm looking for some volunteers to help with that effort
14:06:26 <mwhahaha> yea tonyb want's to chat about the impact
14:06:34 <EmilienM> mwhahaha: count me in
14:06:43 <mwhahaha> he said it was probably ok but wants to understand what that means
14:06:57 <EmilienM> well, see if we can keep branches and stable jobs
14:07:04 <mwhahaha> #action EmilienM to sync with tonyb about stable/newton retirement CI
14:07:07 <EmilienM> we're still doing backports now
14:07:23 <rook> shardy: tagged you in a doc
14:07:38 <mwhahaha> so please also take some time to review the email thread around that and chime in if you have input
14:07:41 <EmilienM> we can expect things going down in future for newton but for now we still have backports
14:07:49 <EmilienM> ok I will
14:07:55 <mwhahaha> moving on
14:07:56 <mwhahaha> (jaosoroir) Blueprint: Add option for containers to log to stdout
14:07:58 * mwhahaha pokes jaosorior
14:08:10 <jaosorior> hey!
14:08:21 <jaosorior> So, just giving a heads up to people about that blueprint
14:08:32 <jaosorior> Here's the link to the spec https://review.openstack.org/#/c/510001/
14:09:06 <jaosorior> But in a nutshell, the plan is to get all of the containers to log to stdout (as one would usually expect with containers)
14:09:23 <jaosorior> and let the docker logging driver handle the logs
14:09:39 <jaosorior> getting some extra metadata with that
14:09:57 <fultonj> jaosorior: so would you not log to /var/log/containers on the container host?
14:10:00 <shardy> jaosorior: can we do that and still log to file on the host?
14:10:08 <jaosorior> larsks: ^^
14:10:21 <larsks> shardy: sure, we can double-log if somebody wants that.
14:10:36 <larsks> But the two big advantages here are:
14:10:46 <jaosorior> fultonj, shardy: though, the plan right now is to make it pluggable. So we would still log to /var/log/containers by default
14:10:50 <larsks> (a) lots of useful metadata, including k8s metadata about the pod in which a container is running, and
14:11:04 <larsks> (b) we support most services automatically, rather than requiring some sort of per-service log configuration.
14:11:35 <mwhahaha> sounds like some good coversation for the spec review :D
14:11:41 <jistr> with k8s logging to /var/log local bindmount essentially becomes an antipattern
14:11:52 <larsks> jistr: agreed!
14:11:56 <shardy> larsks: ack, and this will just work with the existing fluentd integration?
14:11:58 <mwhahaha> #action team to review logging to std spec https://review.openstack.org/#/c/510001/
14:12:06 <jistr> (but until then it's fine/useful imo :) )
14:12:24 <gfidente> jaosorior fwiw, this is what the ceph containers currently do
14:12:27 <larsks> shardy: I think it would be an "instead of" rather than "works with".
14:12:30 <fultonj> https://bugs.launchpad.net/tripleo/+bug/1721841
14:12:32 <openstack> Launchpad bug 1721841 in tripleo "ceph docker containers do not log to /var/log/containers" [High,Triaged] - Assigned to John Fulton (jfulton-org)
14:12:46 <jaosorior> shardy: unfortunately it won't "just work", either we rework the fluentd configuration to take input for journald, or we introduce another forwarder (was planning to use rsyslog)
14:12:48 <shardy> larsks: ack
14:12:52 <gfidente> jaosorior jistr they log to stdout and rely on the dockerd journal socket
14:13:05 <jaosorior> gfidente: actually, we took the approach based on input from folks working on Common Logging, and Ceph :D
14:13:06 <gfidente> (d was in the wrong place)
14:13:07 <shardy> jaosorior: Ok, well we'll need to figure out the operator impact of that
14:13:26 <shardy> I think the local file logging will be needed at least for this pre-k8s iteration of TripleO + containers
14:13:27 <jaosorior> shardy: well, if they deploy the default, everything will stay the same
14:13:40 <fultonj> diff directions but coming together (whereever it is) is good
14:13:47 <jaosorior> shardy: one would need to manually choose to deploy to stdout (with an env file)
14:13:59 <shardy> jaosorior: ++ sounds good then
14:14:42 <openstackgerrit> Numan Siddique proposed openstack/tripleo-heat-templates master: ovn: Remove setting of DockerNeutronApiImage param  https://review.openstack.org/510900
14:14:59 <mwhahaha> jaosorior thanks, any other things you want to bring up around this?
14:15:24 <jaosorior> so, folks, please check out the spec, and the reviews https://review.openstack.org/#/q/(topic:+bp/logging-stdout-rsyslog+OR+topic:tripleo-logging)+status:open
14:15:39 <jaosorior> that was all :D thanks everyone
14:15:43 <mwhahaha> cool thanks
14:15:44 <mwhahaha> (d0ugal/fultonj) Improve user experience for when ansible fails
14:15:48 * mwhahaha pokes fultonj and d0ugal
14:15:54 <fultonj> ack
14:16:10 <fultonj> there are two examples in the pad showing what the tripleo ansible errors look like
14:16:12 <d0ugal> ack also
14:16:39 <fultonj> e.g. http://paste.openstack.org/show/623232
14:16:56 <fultonj> .e.g users may not know to ssh in and find the playbook and run it
14:17:08 <fultonj> same for the heat > mistral > ansible cycle example
14:17:10 <fultonj> we can document
14:17:12 <fultonj> but....
14:17:30 <fultonj> if we can drop them into an ansible shell with an inventory so they can do ad hoc commands, that would be nice IMO
14:17:44 <fultonj> do we solve w/ http://lists.openstack.org/pipermail/openstack-dev/2017-September/122234.html
14:17:56 <shardy> yeah or at least print a message showing how to re-run the step via config download & ansible-playbook
14:18:03 <jistr> i think situation with cryptic failures in Ansible will get better w/ config-download
14:18:04 <jistr> yea +1
14:18:18 <mwhahaha> there's also just the terrible error messages we generate
14:18:33 <mwhahaha> or even lack there of sometimes
14:18:34 <d0ugal> Can ansible output anything other than it's own formatted text?
14:18:39 <openstackgerrit> Oliver Walsh proposed openstack/puppet-tripleo stable/newton: Handle duplicate/invalid entries in migration SSH inbound addresses  https://review.openstack.org/510799
14:18:46 <d0ugal> or, in other words, can we get json or something we can make sense of?
14:18:50 <larsks> d0ugal: well, you can write out data in whatever format you want...
14:19:00 <jtomasek> d0ugal: I believe that shadower did some formating for ansible output for tripleo-validations
14:19:06 <larsks> Or have a plugin collect execution info and dump that for you.
14:19:15 <d0ugal> jtomasek: ah, I thought that looked much neater, I'll need to take a look
14:19:16 <openstackgerrit> Oliver Walsh proposed openstack/puppet-tripleo stable/newton: Handle duplicate/invalid entries in migration SSH inbound addresses  https://review.openstack.org/510799
14:19:21 <d0ugal> larsks: right, that would be useful.
14:19:30 <mwhahaha> sdoran do you know if ansible can output better error messages or there's a way we can improve that?
14:19:40 <fultonj> even without nicer ansible output the user needs to get to it
14:19:42 <rook> oh logging in Pike, how i loath thee
14:19:52 <sdoran> That's a fun topic.
14:19:56 <fultonj> at the moment they have the examples in the pad
14:19:58 <shadower> d0ugal: there may be a json formatter already in ansible that you should just enable (but I've forgotten and never really tried it and may in fact be hallucinating)
14:20:04 <shardy> it would probably help if we output the stack failures list with --long by default when a deploy fails
14:20:08 <sdoran> Do you mean output to standard out?
14:20:23 <d0ugal> shadower: :) I did a quick google before, I'll look around again
14:20:24 <sdoran> Usually -vvvv gives you the best debugging output.
14:20:28 <mwhahaha> sdoran: i think currently we only output the stderr
14:20:33 <rook> I did have some cool dashboard with kibana built with the rsyslog -> elastic work we have in browbeat...
14:20:41 <sdoran> But the preferred method today is to enable logging.
14:20:48 <shadower> d0ugal: if there isn't, this is what tripleo-validations does: https://github.com/openstack/tripleo-validations/blob/master/validations/callback_plugins/validation_output.py
14:20:50 <sdoran> The log file gets a lot of good info if you enable it.
14:21:04 <shadower> d0ugal: as larsks said, you can make it output anything you want
14:21:15 <mwhahaha> d0ugal, shardy: so can we enable the logging and capture that as part of the execution workflow?
14:21:45 <shadower> d0ugal: I actually planned to do a json output and have the tripleo UI consume that, but got called away before I got the chance
14:21:45 <shardy> mwhahaha: yes we could, or perhaps modify the heat-config-ansible hook to capture and send back better data
14:21:51 <mwhahaha> allowing for additional debugging with some id? ie go download the log
14:22:01 <shardy> mwhahaha: this will obviously get easier if/when we move to driving ansible via mistral outside of heat
14:22:31 <d0ugal> We need to make sure we can display -vvvv to the user in a sensible way, since the output could be huge.
14:22:42 <fultonj> "_send back_" is the important part IMO
14:22:45 <mwhahaha> ok so it sounds like we have some possibilities but probably need to PoC and investigate our logging options for the ansible execution
14:22:54 <d0ugal> yup, sounds good
14:23:01 <mwhahaha> who can spend some time investigating?
14:23:09 * mwhahaha prepares the action item
14:23:18 <shardy> fultonj: yeah, well we do have a transport for that already, it's just perhaps not capturing/filtering all the right data
14:23:28 <d0ugal> mwhahaha: I'll take a look
14:23:28 <sdoran> I would say the best way to do logging for later analysis would be to enable logging and ship it to an analysis tool like an elastic stack.
14:23:30 <gfidente> mwhahaha shardy I think execution could be going away in queens
14:23:35 <gfidente> jistr ^
14:23:39 <sdoran> You could have it just forward the log.
14:23:42 <mwhahaha> #action d0ugal investigate ansible logging options around execution to hopefully improve debugging/UX
14:23:46 <jtomasek> shardy: in case we plan to let user re-run certain step manually, it would be very good to provide that functionality via API too
14:23:57 <rook> fultonj Ben England recommends using ANSIBLE_STDOUT_CALLBACK=debug for ansible logging additional to -vvv
14:24:09 <shardy> jtomasek: ack, yes it should be possible via a similar workflow to that being implemented for minor updates
14:24:14 <mwhahaha> gfidente: that's fine but we could also have a wrapper around the raw playbook execution that includes log capture
14:24:16 <shardy> e.g config download then run ansible via mistral
14:24:18 <sdoran> You could also use a callback plugin to capture job data, but I think you'd get more bang for your buck by just grabbing the log file.
14:24:21 <sdoran> export ANSIBLE_LOG_PATH=~/ansible.log
14:24:25 <sdoran> export ANSIBLE_DEBUG=True
14:24:30 <matbu> d0ugal: mwhahaha shardy i was distract by the RH openstack meeting
14:24:32 <fultonj> rook: you can -v all you want today
14:24:38 <fultonj> call back has a wip
14:24:41 <rook> sure fultonj
14:24:52 <jtomasek> shardy: sounds good
14:24:53 <matbu> d0ugal: mwhahaha shardy but in the upgrade dfg we are currently facing the ansible output issue
14:24:53 <fultonj> rook: i mean it's in the THT already if you want it
14:24:55 <jistr> gfidente, shardy: right re execution going away possibly. We want to rewrite service_workflow_tasks into ansible essentially
14:25:05 <rook> fultonj the additional env var.
14:25:07 <jistr> (following the earlier disussion on ML)
14:25:10 <matbu> so we are thinking on improving it quickly
14:25:10 <shardy> gfidente: yeah this gets easier if we do simplify the layers driving ansible
14:25:20 <gfidente> mwhahaha shardy so it's in the mistral ansible action that we'd need to parse output, not from execution anymore
14:25:23 <fultonj> rook: i'll show you after
14:25:37 <jistr> shardy: FYI WIP is here https://review.openstack.org/#/c/510122 https://review.openstack.org/#/c/510781/
14:26:27 <jistr> shardy: already PoC tested with bogus tasks, now we need to try and rewrite the useful tasks for k8s, ceph-ansible etc.
14:26:28 <mwhahaha> matbu: facing how so? figuring out failures or do you need to parse output
14:26:42 <matbu> mwhahaha: facing failure
14:26:44 <fultonj> with slagle's work and ansible driving more do we get this there? but pike has the issue?
14:26:46 <shardy> jistr: ack thanks will check those out
14:26:53 <marios> jistr: i saw the tripleo-common one this morning and meant to ask you if we already have those defined somewhere (the undercloud_deploy_tasks)
14:26:56 <mwhahaha> matbu: ok d0ugal is going to investigate some options so maybe you can work with him
14:27:11 <matbu> mwhahaha: yep we have some ideas too
14:27:14 <marios> jistr: but i see your comment above not yet
14:27:15 <matbu> so sound good
14:27:26 <mwhahaha> ok yea work with him and provide some ideas
14:27:32 <jistr> marios: yea not yet, all is to come. I hope to play with Kubespray first, and we could hopefully use some patterns out of there for the rest.
14:27:49 <mwhahaha> ok any other important comments on the ansible logging or can we move on?
14:27:53 <marios> jistr: thanks
14:28:04 <shardy> fultonj: yeah there's probably a short term and (different) long term answer but there's probably a common piece around filtering the ansible output to surface the error
14:28:12 <mwhahaha> fultonj, d0ugal thanks for bringing this up. sounds like we have a bunch of folks running into the same problems
14:28:29 <openstackgerrit> Merged openstack/tripleo-heat-templates stable/pike: Add IronicPxe to the default controller  https://review.openstack.org/507981
14:28:53 <fultonj> rook: https://github.com/openstack/tripleo-common/blob/master/workbooks/ceph-ansible.yaml#L10
14:29:06 <mwhahaha> ok moving on to squad status
14:29:07 <mwhahaha> #topic Squad status
14:29:16 <mwhahaha> ci
14:29:16 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ci-squad-scrum
14:29:28 <mwhahaha> upgrade
14:29:28 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status
14:29:40 <mwhahaha> containers
14:29:40 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-containers-squad-status
14:29:52 <mwhahaha> dprince, mandre -^ missing today's status
14:30:13 <openstackgerrit> Marios Andreou proposed openstack/tripleo-heat-templates master: EARLY WIP: Convert tags to when statements for Q major upgrade workflow  https://review.openstack.org/510902
14:30:14 <mwhahaha> integration
14:30:15 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status
14:30:26 <jtomasek> #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status
14:30:44 <mwhahaha> jtomasek: thanks! i'll add it in for next week
14:31:09 <jtomasek> @mwhahaha: thanks
14:31:15 <mwhahaha> validations is still missing a status etherpad
14:31:24 <mwhahaha> networking is also missing a status etherpad
14:31:48 <beagles> mwhahaha, I sent an email out this morning "[tripleo][networking] Organizing the networking squad "
14:31:58 * mwhahaha hasn't gotten to his email
14:32:01 <mwhahaha> sounds good
14:32:03 <mwhahaha> i'll keep an eye on it
14:32:13 <beagles> mwhahaha, basically the networking squad "wasn't" but with everything that's been going on, I think it's time to get it together
14:32:29 <beagles> (or past time really ... )
14:32:29 <mwhahaha> #link http://lists.openstack.org/pipermail/openstack-dev/2017-October/123356.html - networking squad
14:32:42 <mwhahaha> beagles: yea sounds good, thanks for starting the effort
14:32:50 <gfidente> beagles now how can the networking squad not be together
14:32:56 <gfidente> am I the only one seeing the irony?
14:33:00 <beagles> :)
14:33:07 <EmilienM> nice summaries, I found it very useful to have async status
14:33:14 <mwhahaha> d0ugal/thrash: do you guys have a status update for workflows?
14:33:36 * gfidente yes you were
14:33:42 <thrash> mwhahaha: making good progress on bp's to support ui.
14:34:03 <mwhahaha> thrash: thanks, let us know if you need anything specific. Also make sure to keep the bps up to date
14:34:07 <thrash> mwhahaha: should have most with patches at least posted by M1
14:34:11 * mwhahaha will be mentioning bps next
14:34:39 <mwhahaha> there's also a python3 squd listed but i'm not sure anyone is actually doing any python3 specific things
14:34:51 <mwhahaha> s/squd/squad
14:35:20 <mwhahaha> so that's all on the status, thanks everyone for providing them
14:35:35 <EmilienM> no we can probably remove that one
14:35:55 <mwhahaha> ok
14:36:05 <mwhahaha> #action mwhahaha to remove python3 squad from the list
14:36:12 <mwhahaha> #topic bugs & blueprints
14:36:12 <mwhahaha> #link https://launchpad.net/tripleo/+milestone/queens-1
14:36:12 <mwhahaha> For Queens we currently have 69 (+7) blueprints and about 444 (-32) open bugs. 193 queens-1, 254 queens-2 and 3 queens-3. Please take some time to review your blueprint status and make sure it is properly up to date.  queens-1 is next week!
14:36:55 <mwhahaha> So yea queens-1 next week. please update bugs/blueprints if they won't merge
14:37:09 <marios> mwhahaha: for this one https://blueprints.launchpad.net/tripleo/+spec/major-upgrade-workflow i repurposed it this morning after chat with matbu, so we can track upgrades for Q... i'll retarget to Q3
14:37:22 <mwhahaha> marios: ok thanks i was going to ask about that one
14:37:28 <EmilienM> q3
14:37:32 <marios> thanks
14:37:43 <EmilienM> I thought the workflow would be for m2
14:37:49 <EmilienM> so m3 we spend time on stabilization
14:37:51 <mwhahaha> marios: yea m3 might be a bit late, anyway to get that in m2?
14:37:58 <marios> mwhahaha: ack
14:38:19 <EmilienM> marios: do you think it's realistic to have it for m2? like the basic workflow?
14:38:45 <marios> EmilienM: we can try it *might* be possible... the workflow in theory isn't radically different to what we have in some parts already
14:38:52 <marios> (ansible etc)
14:38:55 <EmilienM> yeah
14:38:57 <marios> EmilienM: but T&C apply :)
14:39:28 <marios> EmilienM: but today we are still landing things for P so...
14:39:38 <EmilienM> indeed
14:39:42 <mwhahaha> yea we have to catch up or it'll keep slipping
14:40:39 <mwhahaha> marios: if you have things that others can assist with it would be good to publicise some of the plans
14:40:55 <mwhahaha> maybe we can get additional assitance to land either pike or queens efforts
14:41:08 <marios> mwhahaha: ack we have a spec already (for example) linked in the upgrades squad status for interested folks
14:41:12 <marios> mwhahaha: as starter at least
14:41:16 <EmilienM> marios: is there work on the CI space we need to prioritize to help you in the testing? or all is already under control?
14:41:25 <matbu> EmilienM: marios (in a mtg) but Yes its realistic
14:41:47 <marios> EmilienM: it is on our radar that we have a basic keystone only job in place for m2
14:42:09 <marios> EmilienM: and yeah we will reach out to ci squad
14:42:23 <mwhahaha> sounds like there's some movement on it, so let's revisit it in ~2 weeks or so
14:42:47 <EmilienM> marios: cool
14:43:18 <mwhahaha> #action marios and matbuto provide an upgrade workflow status in 2 weeks (reminder)
14:43:30 <mwhahaha> so the other blueprint i wanted to ask about was https://blueprints.launchpad.net/tripleo/+spec/ovs-container-support
14:43:37 <trozet> EmilienM: who can I talk to about the oooqs experimental gate job?  I would like to know where tempest runs from and how that external vxlan tunnel is being used
14:43:48 <mwhahaha> dprince, Slower, beagles: do we have any additional info on the ovs containerization?
14:44:29 <EmilienM> trozet: we can talk about that after the meeting if you don't mind
14:44:30 <matbu> marios: EmilienM actually we have an *almost* noop workflow in reviews
14:44:42 <beagles> mwhahaha, I don't have any at the moment - its the next thing I'm working on after finishing off some octavia related stuff
14:44:53 <trozet> EmilienM: ok thanks
14:44:56 <beagles> mwhahaha, others might have some updates though...
14:45:00 <mwhahaha> beagles: ok so we need to get that figured out asap
14:45:05 <marios> matbu: cool add it to the squad status etherpad?
14:45:12 <beagles> mwhahaha, yep
14:45:46 <mwhahaha> i'll keep bringing it up as a reminder
14:45:50 <mwhahaha> moving on
14:45:51 <mwhahaha> #topic projects releases or stable backports
14:46:02 <matbu> marios: yep i'll
14:46:04 <mwhahaha> EmilienM: did the newton/ocata/pike releases go out?
14:46:07 <EmilienM> this week
14:46:10 <mwhahaha> k
14:46:12 <EmilienM> i'm still on it
14:46:15 <EmilienM> probably today
14:46:22 <mwhahaha> any stable backports folks want attention on?
14:46:37 <mwhahaha> i've also asked to switch the scenario upgrade jobs to non-voting for ocata
14:46:44 <mwhahaha> to free up stable/ocata reviews
14:46:55 <jaosorior> well, this one is merging in stable/newton https://review.openstack.org/#/c/510738/  hope the release includes it :D
14:47:01 <mwhahaha> #link https://review.openstack.org/#/q/status:open+topic:tripleo-upgrade-jobs-nv
14:47:49 <EmilienM> jaosorior: we release every 2 weeks btw
14:48:03 <mwhahaha> or at least will try to :D
14:48:09 <mwhahaha> gates willing
14:48:15 <itlinux> hello all. does anyone have a multi-region config with OOO?
14:48:17 <itlinux> thanks
14:48:36 <mwhahaha> moving on to specs
14:48:41 <mwhahaha> #topic specs
14:48:41 <mwhahaha> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:48:49 <mwhahaha> reminder queens-1 next week, so we should be merging specs
14:48:56 <mwhahaha> please take some time to review the open specs
14:49:16 <mwhahaha> anyone have a spec (other than the already mentioned logging one) that they'd like to bring attention to?
14:49:17 <EmilienM> what happens in queens-2? do we postpone specs to rocky?
14:49:28 <mwhahaha> yes i think so
14:49:42 <EmilienM> we might want make this information more visible then
14:49:42 <shardy> we've never managed to do that before due to lack of velocity in the specs repo
14:50:03 <shardy> but worth a try I guess, particularly given the stabilization theme this cycle
14:50:30 <EmilienM> given that we had many CI issues during queens-1 and have lost some time, we could defer the deadline to November 3
14:50:31 <mwhahaha> I'll send a note to the ML about it
14:50:33 <mwhahaha> and we can discuss it
14:50:50 <EmilienM> so we have 3 weeks
14:51:00 <EmilienM> after that, no more specs approved for queens
14:51:04 <EmilienM> how does it sound?
14:51:18 <gfidente> EmilienM +1
14:51:28 <mwhahaha> well i wasn't going to move the specs until the end of queens-2
14:51:33 <mwhahaha> to at least give some time for review
14:51:39 <mwhahaha> but if you want to do it earlier that works for me
14:52:12 <EmilienM> I think we wanted us to be a bit more conservative for queens
14:52:27 <EmilienM> freezing queens specs earlier is imho part of this effort
14:52:30 <mwhahaha> k let's look at the schedule and i'll send an ML note
14:52:47 <EmilienM> during Pike, folks sent Pike specs after m3 :D
14:52:51 <mwhahaha> https://releases.openstack.org/queens/schedule.html
14:52:52 <EmilienM> we might want to fix that at least
14:53:40 <mwhahaha> so i think queens-2 is a decent spot to freeze the specs
14:54:16 <mwhahaha> it's not like we've been super hard on requiring specs as well
14:54:27 <EmilienM> if we respect it, it's already a good progress comparing to newton & pike cycles
14:54:36 <mwhahaha> k i'll send an ML note
14:54:56 <mwhahaha> #action mwhahaha to send a note about spec reviews and spec freeze at queens-2
14:55:17 <mwhahaha> busy meeting today, so let's allow for some open discussion
14:55:18 <mwhahaha> #topic open discussion
14:55:24 <mwhahaha> anyone have anything else they want to talk about?
14:56:57 <mwhahaha> i'll take that as a no
14:57:02 <mwhahaha> thanks everyone
14:57:03 <mwhahaha> #endmeeting