14:02:15 <mwhahaha> #startmeeting tripleo 14:02:16 <mwhahaha> #topic agenda 14:02:16 <mwhahaha> * Review past action items 14:02:16 <mwhahaha> * One off agenda items 14:02:16 <mwhahaha> * Squad status 14:02:16 <mwhahaha> * Bugs & Blueprints 14:02:16 <openstack> Meeting started Tue Aug 14 14:02:15 2018 UTC and is due to finish in 60 minutes. The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:02:16 <mwhahaha> * Projects releases or stable backports 14:02:17 <mwhahaha> * Specs 14:02:17 <mwhahaha> * open discussion 14:02:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:02:18 <mwhahaha> Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:02:18 <mwhahaha> Hi everyone! who is around today? 14:02:20 <Tengu> althoug 14:02:20 <weshay> bah 14:02:20 <openstack> The meeting name has been set to 'tripleo' 14:02:21 <gfidente> o/ 14:02:25 <Tengu> hmm. 14:02:25 <weshay> 0/ 14:02:25 <mwhahaha> i see no one actually pays attention to when i start the meeting :D 14:02:38 <Tengu> why was this "although" there.... well. anyway. 14:02:38 <EmilienM> o/ 14:02:43 <panda|ruck> oo 14:02:45 <ccamacho> o/ 14:02:49 <marios> o/ 14:02:51 <ccamacho> heello! 14:02:52 <rfolco|rover> o/ 14:03:07 <chandankumar> ykarel: because if we run the stuff in container, there are lots of tempest plugin installed and earlier we are using * as a wild card which runned all the tests which leads to faiulure that'w hy i have listed the tests 14:03:10 <mandre> o/ 14:03:12 <d0ugal> o/ 14:03:15 <chandankumar> \o/ 14:03:49 <fultonj> ha 14:03:54 <slagle> hi 14:04:22 <ksambor> \o 14:05:02 <mwhahaha> Alright let's do this 14:05:12 <mwhahaha> #topic review past action items 14:05:12 <mwhahaha> None! 14:05:25 <mwhahaha> thanks jaosorior for running the meeting last week 14:06:16 <mwhahaha> #topic one off agenda items 14:06:16 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:06:26 <mwhahaha> Outstanding blueprints (mwhahaha) 14:06:39 <mwhahaha> so we have 3 outstanding blueprints that had an FFE 14:06:50 <mwhahaha> We need to get them wrapped up like this week as we approach rc1 time 14:06:58 <mwhahaha> #link https://blueprints.launchpad.net/tripleo/+spec/container-prepare-workflow 14:07:09 <mwhahaha> stevebaker: -^ we need to make sure this is wrapped up 14:07:16 <jrist> o/ 14:07:20 <mwhahaha> i think we landed most of it, but i know there was an outstanding thing or two 14:07:28 <mwhahaha> #link https://blueprints.launchpad.net/tripleo/+spec/tripleo-routed-networks-templates 14:07:43 <mwhahaha> hjensas, dsneddon, bfournie -^ we need to get this landed ASAP 14:08:00 <mwhahaha> i've reviewed some of it this morning, but there's a patch or two with some -1s that we need to address if it is to make Rocky 14:08:09 <mwhahaha> #link https://blueprints.launchpad.net/tripleo/+spec/tripleo-vgpu 14:08:11 <EmilienM> (otherwise moving to stein-1 and backporting work if needed) 14:08:40 <mwhahaha> owalsh: -^ not sure the status of the vgpu stuff, i landed the revert of the image patch for it. please make sure we're in a good spot 14:09:00 <openstackgerrit> wes hayutin proposed openstack/tripleo-quickstart master: fix if statement for standalone https://review.openstack.org/591678 14:10:10 <mwhahaha> (Tengu) Validation Framework (again): extend it for "in-deploy" validations - comments/proposal wanted 14:10:13 <ooolpbot> URGENT TRIPLEO TASKS NEED ATTENTION 14:10:14 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1786764 14:10:14 <openstack> Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) 14:10:24 <Tengu> :) 14:10:44 <mwhahaha> #link https://review.openstack.org/589169 14:10:56 <Tengu> so, basically, we need to extend a bit the scope of the framework in order to let teams add validations during the deploy itself. 14:11:05 <Tengu> i.e. validate a service is running before continuing. 14:11:25 <Tengu> might need to reformat a bit the current spec, and find a way. Some ideas are already in comment. 14:11:46 <Tengu> but on that specific part, I'm a bit lost. I lack some knowledge, so anyone with points is welcome. 14:11:46 <myoung> o/ 14:12:51 <Tengu> (done for me, you can continue mwhahaha :)) 14:12:56 <mwhahaha> k thanks 14:13:03 <mwhahaha> (jrist) - Might need help with TripleO-UI undercloud container problems/failures 14:13:06 <jrist> aye 14:13:11 <jrist> a few bugs listed but general issues 14:13:23 <jrist> one of the main issues is the config file isn't properly written 14:13:38 <owalsh> mwhahaha: ack, looking... 14:13:39 <jrist> weshay and I were drilling down Thursday but it's a bit of a rabbit hole 14:13:40 <EmilienM> can we list the bugs? 14:13:43 <EmilienM> I'm happy to help 14:13:44 <jrist> might be an order of operation thing 14:13:51 <jrist> EmilienM: it's in the meeting items 14:14:09 <mwhahaha> #link https://bugs.launchpad.net/tripleo/+bug/1782588 14:14:09 <openstack> Launchpad bug 1782588 in tripleo "OVB configs not set up for containerized undercloud" [High,Fix released] 14:14:15 <mwhahaha> #link https://bugs.launchpad.net/tripleo/+bug/1782438 14:14:15 <openstack> Launchpad bug 1782438 in tripleo "tripleo ui endpoints misconfigured in containerized undercloud" [High,Triaged] 14:14:16 <weshay> aye 14:14:26 <jrist> oh didn't know fix released for first one. thanks for whoever did that 14:14:36 <jrist> the endpoints misconfigured is the nasty one 14:14:42 <jrist> it's affecting plan upload 14:14:47 <jrist> and the plans don't exist to begin with 14:14:49 <openstackgerrit> Dougal Matthews proposed openstack/tripleo-common master: Clean up node registration output. https://review.openstack.org/569811 14:14:51 <jrist> as well as validations missing entirely 14:15:06 <jrist> would appreciate some eyes on the problems 14:15:20 <jrist> might be a simple thing if you're container-knowledgeable 14:15:42 <openstackgerrit> Ronelle Landy proposed openstack-infra/tripleo-ci master: Add browbeat-minimal.yml to ovb playbook list https://review.openstack.org/590090 14:15:58 <jrist> that's it from me, thanks mwhahaha 14:16:22 <weshay> jrist, you have a patch on better validation too right? 14:16:22 <mwhahaha> thanks 14:16:31 * weshay doesn't see it 14:16:32 <mwhahaha> (myoung) CI Community Meeting starts immediately upon this meeting closing @ https://bluejeans.com/7050859455. This is our team's weekly "open office hours." All are welcome! Ask/discuss anything, we don't bite. Agenda (add items freely) --> https://etherpad.openstack.org/p/tripleo-ci-squad-meeting. 14:16:33 <jrist> weshay: aye 14:16:39 <jrist> weshay: coming 14:17:30 <EmilienM> honza, jrist: I'll take a look at that bug (the first one) 14:18:00 <jrist> thanks EmilienM 14:18:46 <mwhahaha> Any other items for the agenda? 14:19:36 <openstackgerrit> Ronelle Landy proposed openstack-infra/tripleo-ci master: Add Browbeat env settings to rdocloud https://review.openstack.org/583576 14:20:05 <jistr> panda|ruck: hi, plese see my last comment at https://bugs.launchpad.net/tripleo/+bug/1786764 14:20:05 <openstack> Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) 14:20:18 <mwhahaha> sounds like nope, moving on 14:20:24 <jistr> oops sorry 14:20:26 <gfidente> mwhahaha I had a question 14:20:26 <mwhahaha> #topic Squad status 14:20:26 <mwhahaha> ci 14:20:26 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting 14:20:26 <mwhahaha> upgrade 14:20:27 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:20:27 <mwhahaha> containers 14:20:27 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-containers-squad-status 14:20:27 <mwhahaha> config-download 14:20:27 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-config-download-squad-status 14:20:28 <mwhahaha> integration 14:20:28 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:20:29 <mwhahaha> ui/cli 14:20:29 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status 14:20:30 <mwhahaha> validations 14:20:30 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:20:31 <mwhahaha> networking 14:20:31 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:20:32 <mwhahaha> workflows 14:20:32 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status 14:20:33 <mwhahaha> security 14:20:33 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-security-squad 14:20:44 <gfidente> mwhahaha rather open and generic question I forgot to add into etherpad 14:20:58 <mwhahaha> gfidente: let's wait until the end then 14:21:01 <EmilienM> the container squad is looking at replacing docker/docker-distribution by podman/buildah ( points of contact are myself and stevebaker at this point ) 14:21:02 <gfidente> mwhahaha ack 14:21:50 <openstackgerrit> Daniel Alvarez proposed openstack/tripleo-heat-templates master: Decoupling number of API and RPC workers for neutron-api https://review.openstack.org/591679 14:22:06 <mwhahaha> any other status related items? 14:22:14 <openstackgerrit> Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Enable collectd to connect to metrics QDR https://review.openstack.org/576057 14:22:25 <EmilienM> question for CI squad but I guess we can wait for release status 14:22:38 <weshay> the answer is -1 14:22:43 <EmilienM> lol as usual :P 14:22:45 <EmilienM> I want to ask what's the progress on getting things ready for stable/rocky 14:22:58 <fultonj> EmilienM: will s/docker/podman/ land in Rocky? 14:22:59 <mwhahaha> let's chat about that in the release part 14:23:00 <weshay> reviews are up, waiting on review 14:23:02 <EmilienM> fultonj: nope 14:23:07 <EmilienM> fultonj: target is stein 14:23:09 <fultonj> ack 14:23:12 <EmilienM> weshay: #link ? 14:23:25 <panda|ruck> EmilienM: weshay most of the reviews are merged. only missing the promoter cofiguration and phase 1 jobs 14:23:29 <EmilienM> fultonj: amount of work is big :D 14:23:35 <weshay> ah.. merged 14:23:35 <weshay> https://review.rdoproject.org/r/#/c/15374/ 14:23:42 <panda|ruck> EmilienM: https://review.rdoproject.org/r/15444 14:23:44 <weshay> https://review.rdoproject.org/r/#/c/15376/ 14:23:46 <panda|ruck> this is the only one left 14:24:07 <EmilienM> ack 14:24:16 <EmilienM> I'll review it today 14:24:23 <EmilienM> can we say, let's branch tripleoclient end of this week? 14:24:24 <openstackgerrit> Jose Luis Franco proposed openstack/tripleo-heat-templates master: Fix small nit in enable_validations parameter. https://review.openstack.org/591680 14:24:43 <EmilienM> like Thursday or Friday (if release team is ok for the latest) 14:25:18 <EmilienM> (don't be scared to say no :D) 14:25:49 <mwhahaha> would be nice if we were in the right spat to be discussing all of this 14:25:52 * mwhahaha looks at EmilienM 14:25:59 <EmilienM> well I'll propose it by Thursday and sync with CI team to see how jobs are doing once we have the branch created on tripleoclient 14:26:15 * EmilienM hides 14:26:17 <weshay> that guy never listens 14:26:23 <weshay> just +2's all day long 14:26:28 <panda|ruck> jistr: so the updates gets ir broken until we get a promotion ? 14:26:33 <mwhahaha> #topic bugs & blueprints 14:26:33 <mwhahaha> #link https://launchpad.net/tripleo/+milestone/rocky-rc1 14:26:33 <mwhahaha> For Rocky we currently have 42 (+0) blueprints and about 295 (+1) open Launchpad bugs. 295 rocky-rc1, 432 stein-1. 109 (+1) open Storyboard bugs. 14:26:33 <mwhahaha> #link https://storyboard.openstack.org/#!/project_group/76 14:26:45 <panda|ruck> jistr: gates are* 14:26:52 <jistr> panda|ruck: yea i think it's a symptom of some deeper problem 14:26:53 * mwhahaha wonders if the openstack bot is still alive 14:27:02 <jistr> on review, it was all protected by depends-on 14:27:21 <mwhahaha> so i'm goign to move all non-critical bugs to stein 14:27:23 <jistr> but it seems like the container content is not following the depends-on from the patches 14:27:43 <mwhahaha> as previously mentioned, rc1 is coming up and we need to start branching things 14:28:14 <mwhahaha> #action mwhahaha to move all non-critical bugs to Stein 14:28:24 <openstackgerrit> Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove md5 checksum output from CA injection https://review.openstack.org/591681 14:28:38 <mwhahaha> any questions about bugs/blueprints? 14:28:43 <jistr> panda|ruck: not sure how could that happen -- if we land patches in correct order, we should get RPMs in correct order, and container builds should be fine. But it doesn't seem to be like that. So i anticipate we'll keep getting bugs like this until we find out what's the root cause. 14:29:20 <EmilienM> the bot is down, nothing was saved 14:29:25 <EmilienM> let's do it again 14:29:31 <panda|ruck> o/ 14:29:38 <weshay> lolz 14:29:41 <jrist> :( 14:29:45 <weshay> panda|ruck++ 14:30:00 * mwhahaha gives up 14:30:17 <mwhahaha> #topic projects releases or stable backports 14:30:29 <mwhahaha> ok now we can talk about branching 14:30:41 <mwhahaha> so rc1... 14:31:00 <EmilienM> tripleoclient: end of this week if nothing bad comes up in the middle? 14:31:14 <mwhahaha> sounds reasonable 14:31:19 <EmilienM> last time we dropped the branch it was really because CI wasn't ready 14:31:25 <mwhahaha> do we want to target next week for the rest? 14:31:28 <EmilienM> have we patched tripleo-ci repo as well? 14:31:40 <EmilienM> I would have waited a bit 14:31:51 <EmilienM> we usually wait to have stable/X jobs working for tripleoclient 14:32:00 <EmilienM> once they work and actually deploy stable/rocky, then branch others 14:32:27 <jistr> we need this for TOCI when branching happens, but not sooner :) https://review.openstack.org/#/c/590774/ 14:32:28 <EmilienM> I would target week of Aug. 27th 14:32:39 <mwhahaha> k 14:32:42 <openstackgerrit> Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove md5 checksum output from CA injection https://review.openstack.org/591681 14:32:43 <EmilienM> jistr: yeah that, thx 14:33:05 <EmilienM> #action EmilienM to release tripleoclient and branch stable/rocky this week 14:33:24 <EmilienM> #action EmilienM to release other tripleo projects rc1 week of Aug 27th 14:34:15 <mwhahaha> anything else release related? 14:34:28 <EmilienM> nothing much, still doing stable releases every 2 weeks 14:34:43 <EmilienM> mwhahaha: could we create stein-2 and stein-3 please? 14:34:45 <EmilienM> for blueprints 14:34:51 <mwhahaha> yea 14:35:02 <mwhahaha> i'll do that this week 14:35:12 <mwhahaha> #action mwhahaha to create additional stein milestones 14:35:25 <mwhahaha> alright let's move on 14:35:29 <mwhahaha> #topic specs 14:35:29 <mwhahaha> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:35:36 <EmilienM> we can use https://releases.openstack.org/stein/schedule.html 14:36:05 <mwhahaha> specs are up for review and we should be proposing items for discussion at teh PTG 14:36:15 <mwhahaha> please take some time to look at specs 14:37:30 <mwhahaha> #topic open discussion 14:37:35 <mwhahaha> gfidente: what did you want to talk about? 14:38:25 <gfidente> I was wonder if anybody tried to assess how much work is needed to be able to test multinode jobs with centos8 ? 14:38:36 <EmilienM> centos what? :D 14:38:51 <panda|ruck> gfidente: you don't want to know 14:38:53 <gfidente> if we even think about upgrading the infra node or just deploy newer container images on top of centos7? 14:39:11 <gfidente> panda|ruck yeah I figured this is going to be complicated :D 14:39:30 <mwhahaha> gfidente: so there are plans to try and get tripleo running on fedora28 to deploy centos7 containers 14:39:34 <mwhahaha> gfidente: and we'll go from there 14:39:58 <mwhahaha> beyond that, there are folks discussing it and if you want to be included let us know 14:40:26 <gfidente> mwhahaha I was worried about the ceph bits 14:40:38 <openstackgerrit> Ronelle Landy proposed openstack-infra/tripleo-ci master: Test work for using zuulV3 native in RDO Cloud https://review.openstack.org/587228 14:40:49 <gfidente> mwhahaha I'd be happy to get into that conversation to listen :D 14:41:12 <mwhahaha> gfidente: so yea you'll need to make sure the ceph bits can deploy using python3 and what the timeline is to get ceph containers on a newer OS 14:41:21 <mwhahaha> gfidente: are the moment ceph is not our primary concern 14:41:35 <mwhahaha> cause you know, python3, podman, buildah, etc :D 14:41:35 <gfidente> mwhahaha yes this is a big issue to me :D 14:42:03 <mwhahaha> gfidente: so you may want to start raising that issue with ceph-ansible as docker may be unavailable 14:42:05 <gfidente> mwhahaha I don't want to go through the process of disabling ceph everywhere until basic stuff stabilizes :D 14:42:30 <mwhahaha> gfidente: so if ceph-ansible can deploy on fedora28, it'll be ok for the short term 14:42:33 <gfidente> mwhahaha sure we're working on that, was trying to get some understanding of that process to help smooth the transition 14:42:59 <mwhahaha> gfidente: for us, since there is no newer centos, we're targeting a fedora28 node to provide python3 and the other bits 14:43:15 <mwhahaha> gfidente: that'll serve as the transition platform to allow us some testing until a newer centos becomes available 14:43:43 <gfidente> mwhahaha and that is a good start indeed 14:43:50 <mwhahaha> but there's a bunch that is still in the process of being worked on 14:44:00 <mwhahaha> i'm currently trying to get tripleoclient packages for python3 14:44:09 <mwhahaha> but we need to wait until we cut a release so i can drop the instack-undercloud dependencies 14:44:15 <mwhahaha> so it'll be a few weeks before we have something 14:44:27 <mwhahaha> EmilienM: -^ partially why i wanted to cut a rocky release sooner rather than later 14:44:37 <gfidente> mwhahaha sure, again I am trying to help with that process and not cause more issues 14:44:59 <EmilienM> mwhahaha: we can cut it once tripleoclient stable/rocky CI is working 14:45:11 <mwhahaha> so in the short term, i would reach out to the ceph team to understand their support of python3 stuff and plans to around docker/buildah 14:45:12 <EmilienM> it can happen next week if things go well (TM) 14:46:33 <mwhahaha> annnnything else? 14:47:03 <gfidente> mwhahaha yeah this is in-the-work already 14:48:23 <mwhahaha> cool, thanks everyone 14:48:25 <mwhahaha> #endmeeting