14:02:15 #startmeeting tripleo 14:02:16 #topic agenda 14:02:16 * Review past action items 14:02:16 * One off agenda items 14:02:16 * Squad status 14:02:16 * Bugs & Blueprints 14:02:16 Meeting started Tue Aug 14 14:02:15 2018 UTC and is due to finish in 60 minutes. The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:02:16 * Projects releases or stable backports 14:02:17 * Specs 14:02:17 * open discussion 14:02:17 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:02:18 Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:02:18 Hi everyone! who is around today? 14:02:20 althoug 14:02:20 bah 14:02:20 The meeting name has been set to 'tripleo' 14:02:21 o/ 14:02:25 hmm. 14:02:25 0/ 14:02:25 i see no one actually pays attention to when i start the meeting :D 14:02:38 why was this "although" there.... well. anyway. 14:02:38 o/ 14:02:43 oo 14:02:45 o/ 14:02:49 o/ 14:02:51 heello! 14:02:52 o/ 14:03:07 ykarel: because if we run the stuff in container, there are lots of tempest plugin installed and earlier we are using * as a wild card which runned all the tests which leads to faiulure that'w hy i have listed the tests 14:03:10 o/ 14:03:12 o/ 14:03:15 \o/ 14:03:49 ha 14:03:54 hi 14:04:22 \o 14:05:02 Alright let's do this 14:05:12 #topic review past action items 14:05:12 None! 14:05:25 thanks jaosorior for running the meeting last week 14:06:16 #topic one off agenda items 14:06:16 #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:06:26 Outstanding blueprints (mwhahaha) 14:06:39 so we have 3 outstanding blueprints that had an FFE 14:06:50 We need to get them wrapped up like this week as we approach rc1 time 14:06:58 #link https://blueprints.launchpad.net/tripleo/+spec/container-prepare-workflow 14:07:09 stevebaker: -^ we need to make sure this is wrapped up 14:07:16 o/ 14:07:20 i think we landed most of it, but i know there was an outstanding thing or two 14:07:28 #link https://blueprints.launchpad.net/tripleo/+spec/tripleo-routed-networks-templates 14:07:43 hjensas, dsneddon, bfournie -^ we need to get this landed ASAP 14:08:00 i've reviewed some of it this morning, but there's a patch or two with some -1s that we need to address if it is to make Rocky 14:08:09 #link https://blueprints.launchpad.net/tripleo/+spec/tripleo-vgpu 14:08:11 (otherwise moving to stein-1 and backporting work if needed) 14:08:40 owalsh: -^ not sure the status of the vgpu stuff, i landed the revert of the image patch for it. please make sure we're in a good spot 14:09:00 wes hayutin proposed openstack/tripleo-quickstart master: fix if statement for standalone https://review.openstack.org/591678 14:10:10 (Tengu) Validation Framework (again): extend it for "in-deploy" validations - comments/proposal wanted 14:10:13 URGENT TRIPLEO TASKS NEED ATTENTION 14:10:14 https://bugs.launchpad.net/tripleo/+bug/1786764 14:10:14 Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) 14:10:24 :) 14:10:44 #link https://review.openstack.org/589169 14:10:56 so, basically, we need to extend a bit the scope of the framework in order to let teams add validations during the deploy itself. 14:11:05 i.e. validate a service is running before continuing. 14:11:25 might need to reformat a bit the current spec, and find a way. Some ideas are already in comment. 14:11:46 but on that specific part, I'm a bit lost. I lack some knowledge, so anyone with points is welcome. 14:11:46 o/ 14:12:51 (done for me, you can continue mwhahaha :)) 14:12:56 k thanks 14:13:03 (jrist) - Might need help with TripleO-UI undercloud container problems/failures 14:13:06 aye 14:13:11 a few bugs listed but general issues 14:13:23 one of the main issues is the config file isn't properly written 14:13:38 mwhahaha: ack, looking... 14:13:39 weshay and I were drilling down Thursday but it's a bit of a rabbit hole 14:13:40 can we list the bugs? 14:13:43 I'm happy to help 14:13:44 might be an order of operation thing 14:13:51 EmilienM: it's in the meeting items 14:14:09 #link https://bugs.launchpad.net/tripleo/+bug/1782588 14:14:09 Launchpad bug 1782588 in tripleo "OVB configs not set up for containerized undercloud" [High,Fix released] 14:14:15 #link https://bugs.launchpad.net/tripleo/+bug/1782438 14:14:15 Launchpad bug 1782438 in tripleo "tripleo ui endpoints misconfigured in containerized undercloud" [High,Triaged] 14:14:16 aye 14:14:26 oh didn't know fix released for first one. thanks for whoever did that 14:14:36 the endpoints misconfigured is the nasty one 14:14:42 it's affecting plan upload 14:14:47 and the plans don't exist to begin with 14:14:49 Dougal Matthews proposed openstack/tripleo-common master: Clean up node registration output. https://review.openstack.org/569811 14:14:51 as well as validations missing entirely 14:15:06 would appreciate some eyes on the problems 14:15:20 might be a simple thing if you're container-knowledgeable 14:15:42 Ronelle Landy proposed openstack-infra/tripleo-ci master: Add browbeat-minimal.yml to ovb playbook list https://review.openstack.org/590090 14:15:58 that's it from me, thanks mwhahaha 14:16:22 jrist, you have a patch on better validation too right? 14:16:22 thanks 14:16:31 * weshay doesn't see it 14:16:32 (myoung) CI Community Meeting starts immediately upon this meeting closing @ https://bluejeans.com/7050859455. This is our team's weekly "open office hours." All are welcome! Ask/discuss anything, we don't bite. Agenda (add items freely) --> https://etherpad.openstack.org/p/tripleo-ci-squad-meeting. 14:16:33 weshay: aye 14:16:39 weshay: coming 14:17:30 honza, jrist: I'll take a look at that bug (the first one) 14:18:00 thanks EmilienM 14:18:46 Any other items for the agenda? 14:19:36 Ronelle Landy proposed openstack-infra/tripleo-ci master: Add Browbeat env settings to rdocloud https://review.openstack.org/583576 14:20:05 panda|ruck: hi, plese see my last comment at https://bugs.launchpad.net/tripleo/+bug/1786764 14:20:05 Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) 14:20:18 sounds like nope, moving on 14:20:24 oops sorry 14:20:26 mwhahaha I had a question 14:20:26 #topic Squad status 14:20:26 ci 14:20:26 #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting 14:20:26 upgrade 14:20:27 #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:20:27 containers 14:20:27 #link https://etherpad.openstack.org/p/tripleo-containers-squad-status 14:20:27 config-download 14:20:27 #link https://etherpad.openstack.org/p/tripleo-config-download-squad-status 14:20:28 integration 14:20:28 #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:20:29 ui/cli 14:20:29 #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status 14:20:30 validations 14:20:30 #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:20:31 networking 14:20:31 #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:20:32 workflows 14:20:32 #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status 14:20:33 security 14:20:33 #link https://etherpad.openstack.org/p/tripleo-security-squad 14:20:44 mwhahaha rather open and generic question I forgot to add into etherpad 14:20:58 gfidente: let's wait until the end then 14:21:01 the container squad is looking at replacing docker/docker-distribution by podman/buildah ( points of contact are myself and stevebaker at this point ) 14:21:02 mwhahaha ack 14:21:50 Daniel Alvarez proposed openstack/tripleo-heat-templates master: Decoupling number of API and RPC workers for neutron-api https://review.openstack.org/591679 14:22:06 any other status related items? 14:22:14 Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Enable collectd to connect to metrics QDR https://review.openstack.org/576057 14:22:25 question for CI squad but I guess we can wait for release status 14:22:38 the answer is -1 14:22:43 lol as usual :P 14:22:45 I want to ask what's the progress on getting things ready for stable/rocky 14:22:58 EmilienM: will s/docker/podman/ land in Rocky? 14:22:59 let's chat about that in the release part 14:23:00 reviews are up, waiting on review 14:23:02 fultonj: nope 14:23:07 fultonj: target is stein 14:23:09 ack 14:23:12 weshay: #link ? 14:23:25 EmilienM: weshay most of the reviews are merged. only missing the promoter cofiguration and phase 1 jobs 14:23:29 fultonj: amount of work is big :D 14:23:35 ah.. merged 14:23:35 https://review.rdoproject.org/r/#/c/15374/ 14:23:42 EmilienM: https://review.rdoproject.org/r/15444 14:23:44 https://review.rdoproject.org/r/#/c/15376/ 14:23:46 this is the only one left 14:24:07 ack 14:24:16 I'll review it today 14:24:23 can we say, let's branch tripleoclient end of this week? 14:24:24 Jose Luis Franco proposed openstack/tripleo-heat-templates master: Fix small nit in enable_validations parameter. https://review.openstack.org/591680 14:24:43 like Thursday or Friday (if release team is ok for the latest) 14:25:18 (don't be scared to say no :D) 14:25:49 would be nice if we were in the right spat to be discussing all of this 14:25:52 * mwhahaha looks at EmilienM 14:25:59 well I'll propose it by Thursday and sync with CI team to see how jobs are doing once we have the branch created on tripleoclient 14:26:15 * EmilienM hides 14:26:17 that guy never listens 14:26:23 just +2's all day long 14:26:28 jistr: so the updates gets ir broken until we get a promotion ? 14:26:33 #topic bugs & blueprints 14:26:33 #link https://launchpad.net/tripleo/+milestone/rocky-rc1 14:26:33 For Rocky we currently have 42 (+0) blueprints and about 295 (+1) open Launchpad bugs. 295 rocky-rc1, 432 stein-1. 109 (+1) open Storyboard bugs. 14:26:33 #link https://storyboard.openstack.org/#!/project_group/76 14:26:45 jistr: gates are* 14:26:52 panda|ruck: yea i think it's a symptom of some deeper problem 14:26:53 * mwhahaha wonders if the openstack bot is still alive 14:27:02 on review, it was all protected by depends-on 14:27:21 so i'm goign to move all non-critical bugs to stein 14:27:23 but it seems like the container content is not following the depends-on from the patches 14:27:43 as previously mentioned, rc1 is coming up and we need to start branching things 14:28:14 #action mwhahaha to move all non-critical bugs to Stein 14:28:24 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove md5 checksum output from CA injection https://review.openstack.org/591681 14:28:38 any questions about bugs/blueprints? 14:28:43 panda|ruck: not sure how could that happen -- if we land patches in correct order, we should get RPMs in correct order, and container builds should be fine. But it doesn't seem to be like that. So i anticipate we'll keep getting bugs like this until we find out what's the root cause. 14:29:20 the bot is down, nothing was saved 14:29:25 let's do it again 14:29:31 o/ 14:29:38 lolz 14:29:41 :( 14:29:45 panda|ruck++ 14:30:00 * mwhahaha gives up 14:30:17 #topic projects releases or stable backports 14:30:29 ok now we can talk about branching 14:30:41 so rc1... 14:31:00 tripleoclient: end of this week if nothing bad comes up in the middle? 14:31:14 sounds reasonable 14:31:19 last time we dropped the branch it was really because CI wasn't ready 14:31:25 do we want to target next week for the rest? 14:31:28 have we patched tripleo-ci repo as well? 14:31:40 I would have waited a bit 14:31:51 we usually wait to have stable/X jobs working for tripleoclient 14:32:00 once they work and actually deploy stable/rocky, then branch others 14:32:27 we need this for TOCI when branching happens, but not sooner :) https://review.openstack.org/#/c/590774/ 14:32:28 I would target week of Aug. 27th 14:32:39 k 14:32:42 Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove md5 checksum output from CA injection https://review.openstack.org/591681 14:32:43 jistr: yeah that, thx 14:33:05 #action EmilienM to release tripleoclient and branch stable/rocky this week 14:33:24 #action EmilienM to release other tripleo projects rc1 week of Aug 27th 14:34:15 anything else release related? 14:34:28 nothing much, still doing stable releases every 2 weeks 14:34:43 mwhahaha: could we create stein-2 and stein-3 please? 14:34:45 for blueprints 14:34:51 yea 14:35:02 i'll do that this week 14:35:12 #action mwhahaha to create additional stein milestones 14:35:25 alright let's move on 14:35:29 #topic specs 14:35:29 #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:35:36 we can use https://releases.openstack.org/stein/schedule.html 14:36:05 specs are up for review and we should be proposing items for discussion at teh PTG 14:36:15 please take some time to look at specs 14:37:30 #topic open discussion 14:37:35 gfidente: what did you want to talk about? 14:38:25 I was wonder if anybody tried to assess how much work is needed to be able to test multinode jobs with centos8 ? 14:38:36 centos what? :D 14:38:51 gfidente: you don't want to know 14:38:53 if we even think about upgrading the infra node or just deploy newer container images on top of centos7? 14:39:11 panda|ruck yeah I figured this is going to be complicated :D 14:39:30 gfidente: so there are plans to try and get tripleo running on fedora28 to deploy centos7 containers 14:39:34 gfidente: and we'll go from there 14:39:58 beyond that, there are folks discussing it and if you want to be included let us know 14:40:26 mwhahaha I was worried about the ceph bits 14:40:38 Ronelle Landy proposed openstack-infra/tripleo-ci master: Test work for using zuulV3 native in RDO Cloud https://review.openstack.org/587228 14:40:49 mwhahaha I'd be happy to get into that conversation to listen :D 14:41:12 gfidente: so yea you'll need to make sure the ceph bits can deploy using python3 and what the timeline is to get ceph containers on a newer OS 14:41:21 gfidente: are the moment ceph is not our primary concern 14:41:35 cause you know, python3, podman, buildah, etc :D 14:41:35 mwhahaha yes this is a big issue to me :D 14:42:03 gfidente: so you may want to start raising that issue with ceph-ansible as docker may be unavailable 14:42:05 mwhahaha I don't want to go through the process of disabling ceph everywhere until basic stuff stabilizes :D 14:42:30 gfidente: so if ceph-ansible can deploy on fedora28, it'll be ok for the short term 14:42:33 mwhahaha sure we're working on that, was trying to get some understanding of that process to help smooth the transition 14:42:59 gfidente: for us, since there is no newer centos, we're targeting a fedora28 node to provide python3 and the other bits 14:43:15 gfidente: that'll serve as the transition platform to allow us some testing until a newer centos becomes available 14:43:43 mwhahaha and that is a good start indeed 14:43:50 but there's a bunch that is still in the process of being worked on 14:44:00 i'm currently trying to get tripleoclient packages for python3 14:44:09 but we need to wait until we cut a release so i can drop the instack-undercloud dependencies 14:44:15 so it'll be a few weeks before we have something 14:44:27 EmilienM: -^ partially why i wanted to cut a rocky release sooner rather than later 14:44:37 mwhahaha sure, again I am trying to help with that process and not cause more issues 14:44:59 mwhahaha: we can cut it once tripleoclient stable/rocky CI is working 14:45:11 so in the short term, i would reach out to the ceph team to understand their support of python3 stuff and plans to around docker/buildah 14:45:12 it can happen next week if things go well (TM) 14:46:33 annnnything else? 14:47:03 mwhahaha yeah this is in-the-work already 14:48:23 cool, thanks everyone 14:48:25 #endmeeting