14:00:35 <weshay|ruck> #startmeeting tripleo
14:00:35 <weshay|ruck> #topic agenda
14:00:35 <openstack> Meeting started Tue Mar  3 14:00:35 2020 UTC and is due to finish in 60 minutes.  The chair is weshay|ruck. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:35 <weshay|ruck> * Review past action items
14:00:35 <weshay|ruck> * One off agenda items
14:00:35 <weshay|ruck> * Squad status
14:00:36 <weshay|ruck> * Bugs & Blueprints
14:00:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:38 <weshay|ruck> * Projects releases or stable backports
14:00:40 <weshay|ruck> * Specs
14:00:41 <openstack> The meeting name has been set to 'tripleo'
14:00:42 <weshay|ruck> * open discussion
14:00:46 <weshay|ruck> Anyone can use the #link, #action and #info commands, not just the moderatorǃ
14:00:48 <weshay|ruck> Hey folks! who's around?
14:00:50 <weshay|ruck> 0/ howdy folks
14:00:51 <fultonj> o/
14:00:52 <EmilienM> o/
14:00:56 <pkopec> o/
14:00:58 <beagles> o/
14:00:59 <ratailor> o/
14:01:01 <ntt> I can do "docker container stop $(docker container ls -aq)" and "docker container rm $(docker container ls -aq)"
14:01:11 <jbadiapa> o/
14:01:13 <tkajinam> o/
14:01:16 <chandankumar> \o/
14:01:17 <marios|ruck> o/
14:01:20 <ekultails> o/
14:02:10 <ykarel> o/
14:02:18 <gfidente> o/
14:02:38 <rlandy> o/
14:02:41 <weshay|ruck> #topic review past action items
14:02:41 <weshay|ruck> http://eavesdrop.openstack.org/meetings/tripleo/2020/
14:02:47 <cloudnull> o/
14:02:55 <weshay|ruck> anything from last week anyone needs to follow up on?
14:03:26 <weshay|ruck> #topic one off agenda items
14:03:27 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:03:42 <weshay|ruck> ok.. quite a bit in the agenda today... mainly around centos-8
14:04:09 <cloudnull> ++
14:04:48 <weshay|ruck> ok. so last week we reviewed the pinned projects etc...
14:04:51 <weshay|ruck> * Prep work for the unpinning
14:05:17 <weshay|ruck> unpinning has the potential to introduce a number of layered bugs that we would all have to dig out of..
14:06:16 <weshay|ruck> proposed criteria for unpinning ussuri in centos 8 would be to have the following passing prior to unpinning
14:06:17 <weshay|ruck> Required coverage prior unpinning Ussuri:
14:06:17 <weshay|ruck> Undercloud jobs
14:06:18 <weshay|ruck> Standalone , Standalone scenario 1-4, 7, 10
14:06:18 <weshay|ruck> Containers-Multinode
14:06:18 <weshay|ruck> OVB featureset01
14:06:40 <weshay|ruck> does anyone have comments or want to add / remove from that criteria?
14:06:50 <chandankumar> weshay|ruck, list looks good
14:06:52 <weshay|ruck> https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#Unpinning-Ussuri
14:07:21 <marios|ruck> weshay|ruck: chandankumar: how about a full-tempest job or *-api/*-scenario pair
14:07:50 <chandankumar> marios|ruck, yes, let;s add that also
14:07:55 <rlandy> on the tempest component
14:07:57 <weshay|ruck> We will be using a set of pre-commit tests.. and the component pipeline https://hackmd.io/5uYRmLaOTI2raTbHWsaiSQ to limit tripleo's exposure to the 2-3 months of backlogged openstack packages
14:08:39 <weshay|ruck> ok.. so that's the current plan.. if you have any questions or comments let us know
14:08:54 <weshay|ruck> while the pin is in place our ability to backport to 16.x is limited
14:09:07 <weshay|ruck> ykarel, is next.. same topic
14:09:08 <weshay|ruck> * [ykarel] WIP patch for unpin on RDO https://review.rdoproject.org/r/#/c/25612/, need c8 jobs passing and feedback from TripleO to get this merged.
14:09:26 <openstackgerrit> Emilien Macchi proposed openstack/tripleo-docs master: Document how an undercloud can be cleaned-up  https://review.opendev.org/711018
14:09:30 <ykarel> yes i added what can be expected with unpinning
14:09:33 <weshay|ruck> ykarel, any additional comments
14:09:37 <ykarel> and more about clients and libraries
14:09:54 <weshay|ruck> * Clients/libraries are also outdated as u-c https://review.rdoproject.org/r/#/c/24796/
14:09:55 <weshay|ruck> * As soon as that patch merges what can be expected
14:09:55 <weshay|ruck> * CentOS8 DLRN will build all missing commits due to unpin
14:09:55 <weshay|ruck> * All unpinned projects will fail in CentOS7 DLRN as these projects do not work with py2 thus making CentOS7 repo not consistent
14:09:55 <weshay|ruck> * This change will directly impact only promotion pipeline, as there will be new packages to test
14:10:07 <ykarel> weshay|ruck, yes i think good to think of an approach where things don't blocked for long
14:10:24 <ykarel> as added unpinning will not hit upstream, it will hit testing
14:10:32 <ykarel> so it's just fixing things at different layer
14:10:38 <ykarel> not blocking all together
14:11:00 <ykarel> so the jobs i added in WIP patch is  not an hard list
14:11:10 <ykarel> i started with that as that was the only ready ones
14:11:23 <weshay|ruck> It will be key to not unleash all the changes from the last 2-3 months together in a single build
14:11:32 <weshay|ruck> for debugging efforts
14:11:43 <ykarel> as said yes we can split in as required
14:11:47 <weshay|ruck> :)
14:11:50 <ykarel> and go in phases
14:11:54 <weshay|ruck> +1
14:12:06 <ykarel> debugging needs to be done in both cases
14:12:13 <weshay|ruck> aye
14:12:22 <ykarel> the only problem i see it will diverge centos7 and centos8
14:12:25 <weshay|ruck> any questions for ykarel and the RDO team?
14:13:11 <weshay|ruck> ykarel, I don't think we plan on releasing ussuri on centos-7.. so that is a non-issue afaict
14:13:31 <ykarel> weshay|ruck, but currently both jobs are running in check and promotion pipeline
14:13:42 <ykarel> so as soon as we merge any of those patches
14:13:52 <ykarel> packages will diverge
14:13:55 <weshay|ruck> ykarel, centos-8 replaces any centos-7 job.. we will not have both
14:14:05 <ykarel> weshay|ruck, i mean currently both are running
14:14:15 <weshay|ruck> understood..
14:14:31 <weshay|ruck> I think we can move on
14:14:40 <weshay|ruck> additional centos-8 notes:
14:14:41 <weshay|ruck> * Upstream zuul changes
14:14:41 <weshay|ruck> * standalone and standalone scenario jobs https://review.opendev.org/#/q/topic:centos-8-jobs+(status:open+OR+status:merged)
14:14:41 <weshay|ruck> * general centos-8
14:14:41 <weshay|ruck> * The status is changing on a daily basis
14:14:42 <weshay|ruck> * blocking bugs https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#Known-Blocking-Bugs
14:14:46 <weshay|ruck> * General Status: https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#CentOS-8-jobs
14:15:05 <weshay|ruck> centos-8 jobs should be running in all tripleo projects at this time.. please check https://review.opendev.org/#/q/topic:centos-8-jobs+(status:open+OR+status:merged)
14:15:29 <ykarel> with this i recall a miss, /me checks
14:15:36 <weshay|ruck> There are two known centos-8 bugs
14:15:40 <weshay|ruck> centos-8 multinode, undercloud, ovb jobs are hanging on the undercloud install Edit
14:15:40 <weshay|ruck> https://bugs.launchpad.net/tripleo/+bug/1865574
14:15:41 <weshay|ruck> tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument
14:15:41 <weshay|ruck> https://bugs.launchpad.net/tripleo/+bug/1865754
14:15:42 <openstack> Launchpad bug 1865574 in tripleo "centos-8 multinode and undercloud jobs are hanging on the undercloud install " [Critical,Triaged]
14:15:43 <openstack> Launchpad bug 1865754 in tripleo "tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument" [Critical,Triaged]
14:15:52 <weshay|ruck> thanks to the folks picking those up and debugging
14:16:00 <weshay|ruck> OK...
14:16:07 <weshay|ruck> That's it for the agenda
14:16:19 <weshay|ruck> pausing for any questions before we move on
14:16:48 <weshay|ruck> #topic Active Squad status
14:16:48 <weshay|ruck> ci
14:16:48 <weshay|ruck> #link https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw?both
14:16:48 <weshay|ruck> validations
14:16:48 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status
14:16:48 <weshay|ruck> ceph-integration
14:16:50 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status
14:16:52 <weshay|ruck> transformation
14:16:54 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-ansible-agenda
14:16:56 <weshay|ruck> mistral-to-ansible
14:16:58 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-mistral-to-ansible
14:17:25 <weshay|ruck> LOOKING FOR GERRIT CHANGES THAT NEED REVIEW
14:17:52 <weshay|ruck> please post any reviews that need more exposure
14:18:05 <cloudnull> https://review.opendev.org/#/q/starredby:cloudnull+status:open,n,z
14:18:05 <ykarel> weshay|ruck, https://review.opendev.org/#/c/710091/2/zuul.d/build-containers.yaml
14:18:13 <ykarel> not sure if it was missed or intentional
14:18:32 <cloudnull> If folks can take a look at those starred reviews it'd be great.
14:18:47 <weshay|ruck> I see https://review.opendev.org/#/q/topic:THT2TA+(status:open) for tripleo-ansible
14:19:17 <weshay|ruck> for ceph integration I see
14:19:17 <weshay|ruck> https://review.opendev.org/#/c/709083
14:19:17 <weshay|ruck> https://review.opendev.org/#/c/709297
14:19:53 <openstackgerrit> Gael Chamoulaud proposed openstack/tripleo-validations stable/train: Adds Search Path Override  https://review.opendev.org/706859
14:20:05 <weshay|ruck> Validations looks good re: reviews
14:20:05 <weshay|ruck> https://review.opendev.org/#/dashboard/?title=Validation+Reviews&foreach=(project:openstack/tripleo-validations+AND+(owner:%22Gael+Chamoulaud+%3Cgchamoul@redhat.com%3E%22+OR+owner:%22C%C3%A9dric+Jeanneret+(Tengu)+%3Ccjeanner@redhat.com%3E%22+OR+owner:%22Mathieu+Bultel+%3Cmbultel@redhat.com%3E%22))+is:open&You+haven't+voted+in+the+current+revision=NOT+label:Code-Review%3C=2,self+reviewer:self+NOT+owner:self+limit:10&Waiting+For+Second++2
14:20:05 <weshay|ruck> =NOT+label:Workflow%3C=-1+NOT+label:Verified%3C=-1+NOT+label:Code-Review%3C=-2+label:Code-Review%3E=2&Waiting+For+Reviews=NOT+label:Workflow%3C=-1+NOT+label:Verified%3C=-1+NOT+label:Code-Review%3E=2+NOT+label:Code-Review%3C=-1&Negative+Review=label:Code-Review%3C=-1&Failed+CI=label:Verified%3C=-1+NOT+label:Workflow%3C=-1&Work+In+Progress=label:Workflow%3C=-1&Latest+Reviews=limit:20
14:20:08 <weshay|ruck> dang
14:20:10 <weshay|ruck> sorry
14:20:29 <weshay|ruck> anything else from active squads?
14:20:51 <weshay|ruck> #topic Moderately Active Squads
14:20:51 <weshay|ruck> Ironic-integration
14:20:52 <weshay|ruck> https://etherpad.openstack.org/p/tripleo-ironic-integration-squad-status
14:20:52 <weshay|ruck> upgrade
14:20:52 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status
14:20:52 <weshay|ruck> edge
14:20:54 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-edge-squad-status
14:20:56 <weshay|ruck> networking
14:20:58 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status
14:21:00 <weshay|ruck> Any squad wanting to bring up their status, or a topic for the general public?
14:21:10 <weshay|ruck> nada
14:21:12 <weshay|ruck> #link https://launchpad.net/tripleo/+milestone/ussuri-3
14:21:24 <weshay|ruck> Blueprints:
14:21:24 <weshay|ruck> 1 Good progress
14:21:24 <weshay|ruck> Bugs:
14:21:24 <weshay|ruck> 5 New, 20 Incomplete, 3 Invalid, 2 Won't Fix, 6 Confirmed, 368 Triaged, 120 In Progress, 2 Fix Committed, 78 Fix Released
14:22:22 <weshay|ruck> ramishra, and ekultails are your tripleo ruck/rovers please ping them w/ any bugs or questions
14:22:34 <weshay|ruck> instead of hammering the same old people
14:22:42 <weshay|ruck> ratailor, ekultails :)
14:23:00 <weshay|ruck> iStoryboard bugs.
14:23:01 <weshay|ruck> #link https://storyboard.openstack.org/#!/project_group/76
14:23:22 <weshay|ruck> #topic projects releases or stable backports
14:23:22 <weshay|ruck> #topic specs
14:23:22 <weshay|ruck> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:23:42 <weshay|ruck> any updates or comments on open blueprints / specs
14:24:20 <weshay|ruck> moving on
14:24:22 <weshay|ruck> #topic open discussion
14:24:22 <weshay|ruck> Anything else that folks want to bring up to the meeting?
14:24:25 <weshay|ruck> ade_lee, ^
14:24:39 <ade_lee> weshay|ruck, hey there
14:24:50 <ade_lee> yeah -- I'd like to talk about https://review.opendev.org/691906
14:25:15 <ade_lee> which creates a new ansible role that I think should live in tripleo-anisble
14:25:22 <ade_lee> let me set some context ..
14:26:04 <ade_lee> so the way we do tls everywhere right now, is we create a bunch of host entries and services in IPA by using a service called novajoin
14:26:10 <ratailor> weshay|ruck, yea, should this go as a standalone ansible role https://review.opendev.org/#/c/709112/
14:26:15 <ratailor> pkopec, ^^
14:26:38 <ade_lee> novajoin is a nova metadata service - meaning nova talks to it
14:26:58 <pkopec> right, I wanted to ask similar question re https://bugzilla.redhat.com/show_bug.cgi?id=179182
14:27:00 <openstack> bugzilla.redhat.com bug 179182 in eclipse "Null Pointer when building feature in update-site" [Medium,Closed: currentrelease] - Assigned to overholt
14:27:02 <ade_lee> it works well enough , but it desn;'t work when we don't have nova
14:27:33 <ade_lee> and so we needed an ansible role that would be driven from tripleo to do what novajoin used to do
14:27:37 <pkopec> bad paste, sorry https://bugzilla.redhat.com/show_bug.cgi?id=1791824
14:27:41 <openstack> pkopec: Error: Error getting bugzilla.redhat.com bug #1791824: NotPermitted
14:27:48 <ade_lee> thats what the new tripleo-ipa service does
14:27:50 <openstackgerrit> Merged openstack/tripleo-heat-templates master: Sync neutron-ml2-ansible.yaml files  https://review.opendev.org/710131
14:27:53 <weshay|ruck> so.. I think dumping anything into tripleo-ansible is not ideal
14:28:25 <ratailor> weshay|ruck, I wanted that for https://bugzilla.redhat.com/show_bug.cgi?id=1690281
14:28:26 <openstack> ratailor: Error: Error getting bugzilla.redhat.com bug #1690281: NotPermitted
14:28:30 <EmilienM> while I understand that ipa needs to be integrated into tripleo, I also see that role pretty standard and usable outside of tripleo
14:28:30 <weshay|ruck> however using tripleo-ansible as a model for new efforts that have a fuzzy relationship to tripleo re: goverence is a better way
14:29:03 <weshay|ruck> It's fairly easy to create a new project in opendev.. assign the required cores etc..
14:29:09 <ratailor> weshay|ruck, Where should it go then ? should it be standalone ansible-role like https://github.com/openstack/tripleo-ipsec
14:29:16 <weshay|ruck> the CI jobs can run with tripleo for sure
14:29:42 <EmilienM> note that tripleo-ipsec was created long before tripleo-ansible
14:29:44 <weshay|ruck> ratailor, pkopec ade_lee give me a few days.. to speak with a few folks
14:29:59 <ratailor> weshay|ruck, ack. Thanks!
14:30:01 <weshay|ruck> and we'll model this for you
14:30:07 <pkopec> weshay|ruck, ack, thanks
14:30:08 <weshay|ruck> so the path is more clear
14:30:31 <EmilienM> we need to make a good job at making a role that is deployment agnostic
14:30:40 <EmilienM> and with some variables/params, can be used in tripleo
14:30:55 <weshay|ruck> the project and goverence I think is the question here.. I want to reassure everyone that we can integrate these new projects in CI alongside tripleo
14:30:56 <EmilienM> IPsec, IPA and all these things need to have roles that are used by tripleo
14:30:59 <EmilienM> not living in tripleo
14:31:22 <weshay|ruck> cloudnull, I'll probably be pulling you aside to consult a bit
14:31:36 <cloudnull> pull away :D
14:31:37 <weshay|ruck> let's follow up on these items in the next #tripleo mtg
14:32:29 <ade_lee> EmilienM, the role right now does things that I think are pretty  tripleo specific -- its hard to see its use outside of tripleo -- but if we want it outside of triple-ansible , then fine
14:32:44 <weshay|ruck> #action develop a model for projects that integrate with tripleo but are not directly governed by tripleo core
14:32:45 <ade_lee> as long as we have the right ci -- and its pulled in ok
14:32:56 <weshay|ruck> hrm.. why doesn't that work
14:32:59 <weshay|ruck> lolz
14:33:10 <weshay|ruck> ade_lee, aye.. yes..
14:33:37 <weshay|ruck> let's follow up next week and see where are then
14:33:53 <weshay|ruck> any other open discussion?
14:34:01 <ade_lee> weshay|ruck, it does make getting that standalone tls job working more important coz thats what we'd use for ci ..
14:34:12 <chandankumar> I want to discuss about this review
14:34:14 <chandankumar> https://review.opendev.org/#/c/710980/
14:34:15 <weshay|ruck> ade_lee, agree
14:34:38 <chandankumar> for c8 scenario 7 job, it expects /usr/bin/python which is not there
14:34:39 * weshay|ruck notes that getting centos-8 and unpinning has a direct impact on our ability to ship to customers
14:34:49 <weshay|ruck> and unfortunately that is consuming the ci team atm
14:35:29 <chandankumar> I donot want to carry additional var PythonInterpreter in TQE to set that param
14:35:32 <weshay|ruck> chandankumar, aye.. there were some other bugs opened on a similar issue
14:35:42 <chandankumar> is there any better way to handle that
14:36:09 <chandankumar> PythonInterpreter is used in two places in tht and another in ipa
14:36:21 <weshay|ruck> chandankumar, so all the projects should be moved to python3 by the end of ussuri
14:36:22 <marios|ruck> chandankumar: as commented there, it breaks the interface, we should set that parameter in an environment file and pass it into the job(s)
14:36:34 <weshay|ruck> so master / ussuri should assume python3
14:36:36 <weshay|ruck> right?
14:36:40 <marios|ruck> chandankumar: there i mean https://review.opendev.org/#/c/710980/
14:36:52 <chandankumar> marios|ruck, but the same env file will be used for c7 and c8 currently
14:36:58 <weshay|ruck> if a centos-7 job is failing.. it can now be ignored until there is a c8 job
14:37:05 <chandankumar> it will break the job making chicken egg problem
14:37:11 <weshay|ruck> chandankumar, ya.. we don't care about c7 in ussuri at this point
14:37:16 <marios|ruck> chandankumar: right now, if someone sets the PythonInterpreter it is ignored by your review, so it breaks the interface
14:37:17 <weshay|ruck> 3rd party
14:37:42 <weshay|ruck> we will actively moving c7 jobs to non-voting  or removed
14:37:53 <weshay|ruck> do not try and support BOTH at the same time
14:37:58 <weshay|ruck> this is an OR operation
14:38:03 <chandankumar> weshay|ruck, marios|ruck ok marios|ruck , I will change it to py3 and make c7 job voting
14:38:04 <weshay|ruck> clear?
14:38:08 <chandankumar> *nv
14:38:12 <weshay|ruck> :)
14:38:18 <chandankumar> and get it landed, it will help our case
14:38:21 <chandankumar> marios|ruck, weshay|ruck thanks!
14:38:22 <weshay|ruck> ++
14:38:38 <cloudnull> chandankumar couldn't we get rid of that var and just run something like this in the script `$(command -v python3 || command -v python) ...`
14:38:43 <weshay|ruck> ok.. folks.. last chance for new topics
14:39:04 <chandankumar> cloudnull, I think I can do that
14:39:07 <cloudnull> that would simplify things and detect the python version on the fly
14:39:09 <marios|ruck> cloudnull: chandankumar: sure we could/should get rid of it if we no longer need it but we have to deprecate it at least in case someone else is relying on that parameter
14:39:10 <weshay|ruck> #action why doesn't action work
14:39:14 <weshay|ruck> lolz
14:39:25 <weshay|ruck> #endmeeting