14:00:35 <weshay|ruck> #startmeeting tripleo 14:00:35 <weshay|ruck> #topic agenda 14:00:35 <openstack> Meeting started Tue Mar 3 14:00:35 2020 UTC and is due to finish in 60 minutes. The chair is weshay|ruck. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:35 <weshay|ruck> * Review past action items 14:00:35 <weshay|ruck> * One off agenda items 14:00:35 <weshay|ruck> * Squad status 14:00:36 <weshay|ruck> * Bugs & Blueprints 14:00:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:38 <weshay|ruck> * Projects releases or stable backports 14:00:40 <weshay|ruck> * Specs 14:00:41 <openstack> The meeting name has been set to 'tripleo' 14:00:42 <weshay|ruck> * open discussion 14:00:46 <weshay|ruck> Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:00:48 <weshay|ruck> Hey folks! who's around? 14:00:50 <weshay|ruck> 0/ howdy folks 14:00:51 <fultonj> o/ 14:00:52 <EmilienM> o/ 14:00:56 <pkopec> o/ 14:00:58 <beagles> o/ 14:00:59 <ratailor> o/ 14:01:01 <ntt> I can do "docker container stop $(docker container ls -aq)" and "docker container rm $(docker container ls -aq)" 14:01:11 <jbadiapa> o/ 14:01:13 <tkajinam> o/ 14:01:16 <chandankumar> \o/ 14:01:17 <marios|ruck> o/ 14:01:20 <ekultails> o/ 14:02:10 <ykarel> o/ 14:02:18 <gfidente> o/ 14:02:38 <rlandy> o/ 14:02:41 <weshay|ruck> #topic review past action items 14:02:41 <weshay|ruck> http://eavesdrop.openstack.org/meetings/tripleo/2020/ 14:02:47 <cloudnull> o/ 14:02:55 <weshay|ruck> anything from last week anyone needs to follow up on? 14:03:26 <weshay|ruck> #topic one off agenda items 14:03:27 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:03:42 <weshay|ruck> ok.. quite a bit in the agenda today... mainly around centos-8 14:04:09 <cloudnull> ++ 14:04:48 <weshay|ruck> ok. so last week we reviewed the pinned projects etc... 14:04:51 <weshay|ruck> * Prep work for the unpinning 14:05:17 <weshay|ruck> unpinning has the potential to introduce a number of layered bugs that we would all have to dig out of.. 14:06:16 <weshay|ruck> proposed criteria for unpinning ussuri in centos 8 would be to have the following passing prior to unpinning 14:06:17 <weshay|ruck> Required coverage prior unpinning Ussuri: 14:06:17 <weshay|ruck> Undercloud jobs 14:06:18 <weshay|ruck> Standalone , Standalone scenario 1-4, 7, 10 14:06:18 <weshay|ruck> Containers-Multinode 14:06:18 <weshay|ruck> OVB featureset01 14:06:40 <weshay|ruck> does anyone have comments or want to add / remove from that criteria? 14:06:50 <chandankumar> weshay|ruck, list looks good 14:06:52 <weshay|ruck> https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#Unpinning-Ussuri 14:07:21 <marios|ruck> weshay|ruck: chandankumar: how about a full-tempest job or *-api/*-scenario pair 14:07:50 <chandankumar> marios|ruck, yes, let;s add that also 14:07:55 <rlandy> on the tempest component 14:07:57 <weshay|ruck> We will be using a set of pre-commit tests.. and the component pipeline https://hackmd.io/5uYRmLaOTI2raTbHWsaiSQ to limit tripleo's exposure to the 2-3 months of backlogged openstack packages 14:08:39 <weshay|ruck> ok.. so that's the current plan.. if you have any questions or comments let us know 14:08:54 <weshay|ruck> while the pin is in place our ability to backport to 16.x is limited 14:09:07 <weshay|ruck> ykarel, is next.. same topic 14:09:08 <weshay|ruck> * [ykarel] WIP patch for unpin on RDO https://review.rdoproject.org/r/#/c/25612/, need c8 jobs passing and feedback from TripleO to get this merged. 14:09:26 <openstackgerrit> Emilien Macchi proposed openstack/tripleo-docs master: Document how an undercloud can be cleaned-up https://review.opendev.org/711018 14:09:30 <ykarel> yes i added what can be expected with unpinning 14:09:33 <weshay|ruck> ykarel, any additional comments 14:09:37 <ykarel> and more about clients and libraries 14:09:54 <weshay|ruck> * Clients/libraries are also outdated as u-c https://review.rdoproject.org/r/#/c/24796/ 14:09:55 <weshay|ruck> * As soon as that patch merges what can be expected 14:09:55 <weshay|ruck> * CentOS8 DLRN will build all missing commits due to unpin 14:09:55 <weshay|ruck> * All unpinned projects will fail in CentOS7 DLRN as these projects do not work with py2 thus making CentOS7 repo not consistent 14:09:55 <weshay|ruck> * This change will directly impact only promotion pipeline, as there will be new packages to test 14:10:07 <ykarel> weshay|ruck, yes i think good to think of an approach where things don't blocked for long 14:10:24 <ykarel> as added unpinning will not hit upstream, it will hit testing 14:10:32 <ykarel> so it's just fixing things at different layer 14:10:38 <ykarel> not blocking all together 14:11:00 <ykarel> so the jobs i added in WIP patch is not an hard list 14:11:10 <ykarel> i started with that as that was the only ready ones 14:11:23 <weshay|ruck> It will be key to not unleash all the changes from the last 2-3 months together in a single build 14:11:32 <weshay|ruck> for debugging efforts 14:11:43 <ykarel> as said yes we can split in as required 14:11:47 <weshay|ruck> :) 14:11:50 <ykarel> and go in phases 14:11:54 <weshay|ruck> +1 14:12:06 <ykarel> debugging needs to be done in both cases 14:12:13 <weshay|ruck> aye 14:12:22 <ykarel> the only problem i see it will diverge centos7 and centos8 14:12:25 <weshay|ruck> any questions for ykarel and the RDO team? 14:13:11 <weshay|ruck> ykarel, I don't think we plan on releasing ussuri on centos-7.. so that is a non-issue afaict 14:13:31 <ykarel> weshay|ruck, but currently both jobs are running in check and promotion pipeline 14:13:42 <ykarel> so as soon as we merge any of those patches 14:13:52 <ykarel> packages will diverge 14:13:55 <weshay|ruck> ykarel, centos-8 replaces any centos-7 job.. we will not have both 14:14:05 <ykarel> weshay|ruck, i mean currently both are running 14:14:15 <weshay|ruck> understood.. 14:14:31 <weshay|ruck> I think we can move on 14:14:40 <weshay|ruck> additional centos-8 notes: 14:14:41 <weshay|ruck> * Upstream zuul changes 14:14:41 <weshay|ruck> * standalone and standalone scenario jobs https://review.opendev.org/#/q/topic:centos-8-jobs+(status:open+OR+status:merged) 14:14:41 <weshay|ruck> * general centos-8 14:14:41 <weshay|ruck> * The status is changing on a daily basis 14:14:42 <weshay|ruck> * blocking bugs https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#Known-Blocking-Bugs 14:14:46 <weshay|ruck> * General Status: https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#CentOS-8-jobs 14:15:05 <weshay|ruck> centos-8 jobs should be running in all tripleo projects at this time.. please check https://review.opendev.org/#/q/topic:centos-8-jobs+(status:open+OR+status:merged) 14:15:29 <ykarel> with this i recall a miss, /me checks 14:15:36 <weshay|ruck> There are two known centos-8 bugs 14:15:40 <weshay|ruck> centos-8 multinode, undercloud, ovb jobs are hanging on the undercloud install Edit 14:15:40 <weshay|ruck> https://bugs.launchpad.net/tripleo/+bug/1865574 14:15:41 <weshay|ruck> tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument 14:15:41 <weshay|ruck> https://bugs.launchpad.net/tripleo/+bug/1865754 14:15:42 <openstack> Launchpad bug 1865574 in tripleo "centos-8 multinode and undercloud jobs are hanging on the undercloud install " [Critical,Triaged] 14:15:43 <openstack> Launchpad bug 1865754 in tripleo "tripleo-ci-centos-8-scenario001-standalone tempest-conf fails 500 PUT http://192.168.24.1:9292/v2/images/ RADOS invalid argument" [Critical,Triaged] 14:15:52 <weshay|ruck> thanks to the folks picking those up and debugging 14:16:00 <weshay|ruck> OK... 14:16:07 <weshay|ruck> That's it for the agenda 14:16:19 <weshay|ruck> pausing for any questions before we move on 14:16:48 <weshay|ruck> #topic Active Squad status 14:16:48 <weshay|ruck> ci 14:16:48 <weshay|ruck> #link https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw?both 14:16:48 <weshay|ruck> validations 14:16:48 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:16:48 <weshay|ruck> ceph-integration 14:16:50 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:16:52 <weshay|ruck> transformation 14:16:54 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-ansible-agenda 14:16:56 <weshay|ruck> mistral-to-ansible 14:16:58 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-mistral-to-ansible 14:17:25 <weshay|ruck> LOOKING FOR GERRIT CHANGES THAT NEED REVIEW 14:17:52 <weshay|ruck> please post any reviews that need more exposure 14:18:05 <cloudnull> https://review.opendev.org/#/q/starredby:cloudnull+status:open,n,z 14:18:05 <ykarel> weshay|ruck, https://review.opendev.org/#/c/710091/2/zuul.d/build-containers.yaml 14:18:13 <ykarel> not sure if it was missed or intentional 14:18:32 <cloudnull> If folks can take a look at those starred reviews it'd be great. 14:18:47 <weshay|ruck> I see https://review.opendev.org/#/q/topic:THT2TA+(status:open) for tripleo-ansible 14:19:17 <weshay|ruck> for ceph integration I see 14:19:17 <weshay|ruck> https://review.opendev.org/#/c/709083 14:19:17 <weshay|ruck> https://review.opendev.org/#/c/709297 14:19:53 <openstackgerrit> Gael Chamoulaud proposed openstack/tripleo-validations stable/train: Adds Search Path Override https://review.opendev.org/706859 14:20:05 <weshay|ruck> Validations looks good re: reviews 14:20:05 <weshay|ruck> https://review.opendev.org/#/dashboard/?title=Validation+Reviews&foreach=(project:openstack/tripleo-validations+AND+(owner:%22Gael+Chamoulaud+%3Cgchamoul@redhat.com%3E%22+OR+owner:%22C%C3%A9dric+Jeanneret+(Tengu)+%3Ccjeanner@redhat.com%3E%22+OR+owner:%22Mathieu+Bultel+%3Cmbultel@redhat.com%3E%22))+is:open&You+haven't+voted+in+the+current+revision=NOT+label:Code-Review%3C=2,self+reviewer:self+NOT+owner:self+limit:10&Waiting+For+Second++2 14:20:05 <weshay|ruck> =NOT+label:Workflow%3C=-1+NOT+label:Verified%3C=-1+NOT+label:Code-Review%3C=-2+label:Code-Review%3E=2&Waiting+For+Reviews=NOT+label:Workflow%3C=-1+NOT+label:Verified%3C=-1+NOT+label:Code-Review%3E=2+NOT+label:Code-Review%3C=-1&Negative+Review=label:Code-Review%3C=-1&Failed+CI=label:Verified%3C=-1+NOT+label:Workflow%3C=-1&Work+In+Progress=label:Workflow%3C=-1&Latest+Reviews=limit:20 14:20:08 <weshay|ruck> dang 14:20:10 <weshay|ruck> sorry 14:20:29 <weshay|ruck> anything else from active squads? 14:20:51 <weshay|ruck> #topic Moderately Active Squads 14:20:51 <weshay|ruck> Ironic-integration 14:20:52 <weshay|ruck> https://etherpad.openstack.org/p/tripleo-ironic-integration-squad-status 14:20:52 <weshay|ruck> upgrade 14:20:52 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:20:52 <weshay|ruck> edge 14:20:54 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-edge-squad-status 14:20:56 <weshay|ruck> networking 14:20:58 <weshay|ruck> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:21:00 <weshay|ruck> Any squad wanting to bring up their status, or a topic for the general public? 14:21:10 <weshay|ruck> nada 14:21:12 <weshay|ruck> #link https://launchpad.net/tripleo/+milestone/ussuri-3 14:21:24 <weshay|ruck> Blueprints: 14:21:24 <weshay|ruck> 1 Good progress 14:21:24 <weshay|ruck> Bugs: 14:21:24 <weshay|ruck> 5 New, 20 Incomplete, 3 Invalid, 2 Won't Fix, 6 Confirmed, 368 Triaged, 120 In Progress, 2 Fix Committed, 78 Fix Released 14:22:22 <weshay|ruck> ramishra, and ekultails are your tripleo ruck/rovers please ping them w/ any bugs or questions 14:22:34 <weshay|ruck> instead of hammering the same old people 14:22:42 <weshay|ruck> ratailor, ekultails :) 14:23:00 <weshay|ruck> iStoryboard bugs. 14:23:01 <weshay|ruck> #link https://storyboard.openstack.org/#!/project_group/76 14:23:22 <weshay|ruck> #topic projects releases or stable backports 14:23:22 <weshay|ruck> #topic specs 14:23:22 <weshay|ruck> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:23:42 <weshay|ruck> any updates or comments on open blueprints / specs 14:24:20 <weshay|ruck> moving on 14:24:22 <weshay|ruck> #topic open discussion 14:24:22 <weshay|ruck> Anything else that folks want to bring up to the meeting? 14:24:25 <weshay|ruck> ade_lee, ^ 14:24:39 <ade_lee> weshay|ruck, hey there 14:24:50 <ade_lee> yeah -- I'd like to talk about https://review.opendev.org/691906 14:25:15 <ade_lee> which creates a new ansible role that I think should live in tripleo-anisble 14:25:22 <ade_lee> let me set some context .. 14:26:04 <ade_lee> so the way we do tls everywhere right now, is we create a bunch of host entries and services in IPA by using a service called novajoin 14:26:10 <ratailor> weshay|ruck, yea, should this go as a standalone ansible role https://review.opendev.org/#/c/709112/ 14:26:15 <ratailor> pkopec, ^^ 14:26:38 <ade_lee> novajoin is a nova metadata service - meaning nova talks to it 14:26:58 <pkopec> right, I wanted to ask similar question re https://bugzilla.redhat.com/show_bug.cgi?id=179182 14:27:00 <openstack> bugzilla.redhat.com bug 179182 in eclipse "Null Pointer when building feature in update-site" [Medium,Closed: currentrelease] - Assigned to overholt 14:27:02 <ade_lee> it works well enough , but it desn;'t work when we don't have nova 14:27:33 <ade_lee> and so we needed an ansible role that would be driven from tripleo to do what novajoin used to do 14:27:37 <pkopec> bad paste, sorry https://bugzilla.redhat.com/show_bug.cgi?id=1791824 14:27:41 <openstack> pkopec: Error: Error getting bugzilla.redhat.com bug #1791824: NotPermitted 14:27:48 <ade_lee> thats what the new tripleo-ipa service does 14:27:50 <openstackgerrit> Merged openstack/tripleo-heat-templates master: Sync neutron-ml2-ansible.yaml files https://review.opendev.org/710131 14:27:53 <weshay|ruck> so.. I think dumping anything into tripleo-ansible is not ideal 14:28:25 <ratailor> weshay|ruck, I wanted that for https://bugzilla.redhat.com/show_bug.cgi?id=1690281 14:28:26 <openstack> ratailor: Error: Error getting bugzilla.redhat.com bug #1690281: NotPermitted 14:28:30 <EmilienM> while I understand that ipa needs to be integrated into tripleo, I also see that role pretty standard and usable outside of tripleo 14:28:30 <weshay|ruck> however using tripleo-ansible as a model for new efforts that have a fuzzy relationship to tripleo re: goverence is a better way 14:29:03 <weshay|ruck> It's fairly easy to create a new project in opendev.. assign the required cores etc.. 14:29:09 <ratailor> weshay|ruck, Where should it go then ? should it be standalone ansible-role like https://github.com/openstack/tripleo-ipsec 14:29:16 <weshay|ruck> the CI jobs can run with tripleo for sure 14:29:42 <EmilienM> note that tripleo-ipsec was created long before tripleo-ansible 14:29:44 <weshay|ruck> ratailor, pkopec ade_lee give me a few days.. to speak with a few folks 14:29:59 <ratailor> weshay|ruck, ack. Thanks! 14:30:01 <weshay|ruck> and we'll model this for you 14:30:07 <pkopec> weshay|ruck, ack, thanks 14:30:08 <weshay|ruck> so the path is more clear 14:30:31 <EmilienM> we need to make a good job at making a role that is deployment agnostic 14:30:40 <EmilienM> and with some variables/params, can be used in tripleo 14:30:55 <weshay|ruck> the project and goverence I think is the question here.. I want to reassure everyone that we can integrate these new projects in CI alongside tripleo 14:30:56 <EmilienM> IPsec, IPA and all these things need to have roles that are used by tripleo 14:30:59 <EmilienM> not living in tripleo 14:31:22 <weshay|ruck> cloudnull, I'll probably be pulling you aside to consult a bit 14:31:36 <cloudnull> pull away :D 14:31:37 <weshay|ruck> let's follow up on these items in the next #tripleo mtg 14:32:29 <ade_lee> EmilienM, the role right now does things that I think are pretty tripleo specific -- its hard to see its use outside of tripleo -- but if we want it outside of triple-ansible , then fine 14:32:44 <weshay|ruck> #action develop a model for projects that integrate with tripleo but are not directly governed by tripleo core 14:32:45 <ade_lee> as long as we have the right ci -- and its pulled in ok 14:32:56 <weshay|ruck> hrm.. why doesn't that work 14:32:59 <weshay|ruck> lolz 14:33:10 <weshay|ruck> ade_lee, aye.. yes.. 14:33:37 <weshay|ruck> let's follow up next week and see where are then 14:33:53 <weshay|ruck> any other open discussion? 14:34:01 <ade_lee> weshay|ruck, it does make getting that standalone tls job working more important coz thats what we'd use for ci .. 14:34:12 <chandankumar> I want to discuss about this review 14:34:14 <chandankumar> https://review.opendev.org/#/c/710980/ 14:34:15 <weshay|ruck> ade_lee, agree 14:34:38 <chandankumar> for c8 scenario 7 job, it expects /usr/bin/python which is not there 14:34:39 * weshay|ruck notes that getting centos-8 and unpinning has a direct impact on our ability to ship to customers 14:34:49 <weshay|ruck> and unfortunately that is consuming the ci team atm 14:35:29 <chandankumar> I donot want to carry additional var PythonInterpreter in TQE to set that param 14:35:32 <weshay|ruck> chandankumar, aye.. there were some other bugs opened on a similar issue 14:35:42 <chandankumar> is there any better way to handle that 14:36:09 <chandankumar> PythonInterpreter is used in two places in tht and another in ipa 14:36:21 <weshay|ruck> chandankumar, so all the projects should be moved to python3 by the end of ussuri 14:36:22 <marios|ruck> chandankumar: as commented there, it breaks the interface, we should set that parameter in an environment file and pass it into the job(s) 14:36:34 <weshay|ruck> so master / ussuri should assume python3 14:36:36 <weshay|ruck> right? 14:36:40 <marios|ruck> chandankumar: there i mean https://review.opendev.org/#/c/710980/ 14:36:52 <chandankumar> marios|ruck, but the same env file will be used for c7 and c8 currently 14:36:58 <weshay|ruck> if a centos-7 job is failing.. it can now be ignored until there is a c8 job 14:37:05 <chandankumar> it will break the job making chicken egg problem 14:37:11 <weshay|ruck> chandankumar, ya.. we don't care about c7 in ussuri at this point 14:37:16 <marios|ruck> chandankumar: right now, if someone sets the PythonInterpreter it is ignored by your review, so it breaks the interface 14:37:17 <weshay|ruck> 3rd party 14:37:42 <weshay|ruck> we will actively moving c7 jobs to non-voting or removed 14:37:53 <weshay|ruck> do not try and support BOTH at the same time 14:37:58 <weshay|ruck> this is an OR operation 14:38:03 <chandankumar> weshay|ruck, marios|ruck ok marios|ruck , I will change it to py3 and make c7 job voting 14:38:04 <weshay|ruck> clear? 14:38:08 <chandankumar> *nv 14:38:12 <weshay|ruck> :) 14:38:18 <chandankumar> and get it landed, it will help our case 14:38:21 <chandankumar> marios|ruck, weshay|ruck thanks! 14:38:22 <weshay|ruck> ++ 14:38:38 <cloudnull> chandankumar couldn't we get rid of that var and just run something like this in the script `$(command -v python3 || command -v python) ...` 14:38:43 <weshay|ruck> ok.. folks.. last chance for new topics 14:39:04 <chandankumar> cloudnull, I think I can do that 14:39:07 <cloudnull> that would simplify things and detect the python version on the fly 14:39:09 <marios|ruck> cloudnull: chandankumar: sure we could/should get rid of it if we no longer need it but we have to deprecate it at least in case someone else is relying on that parameter 14:39:10 <weshay|ruck> #action why doesn't action work 14:39:14 <weshay|ruck> lolz 14:39:25 <weshay|ruck> #endmeeting