14:00:20 <jaosorior> #startmeeting tripleo 14:00:21 <openstack> Meeting started Tue Nov 20 14:00:20 2018 UTC and is due to finish in 60 minutes. The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:24 <openstack> The meeting name has been set to 'tripleo' 14:00:33 <jaosorior> #topic agenda 14:00:35 <jaosorior> * Review past action items 14:00:37 <jaosorior> * One off agenda items 14:00:39 <jaosorior> * Squad status 14:00:41 <jaosorior> * Bugs & Blueprints 14:00:43 <jaosorior> * Projects releases or stable backports 14:00:45 <jaosorior> * Specs 14:00:47 <jaosorior> * open discussion 14:00:49 <jaosorior> Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:00:51 <jaosorior> Hey folks! who's around? 14:00:53 <Tengu> «o/ 14:00:56 <rfolco> o/ 14:01:09 <ksambor> o/ 14:01:20 <fultonj> o/ 14:01:32 <marios> o/ hello 14:01:42 <shardy> o/ 14:01:55 <panda|rover> o/ 14:02:02 <dpeacock> o/ 14:02:32 <jtomasek> o/ 14:02:34 <openstackgerrit> Merged openstack/tripleo-common master: Exit when container is missing or empty when downloading logs https://review.openstack.org/618572 14:02:35 <openstackgerrit> Merged openstack/tripleo-common stable/queens: Set SSH server keep alive options https://review.openstack.org/618118 14:02:36 <openstackgerrit> Merged openstack/tripleo-common stable/rocky: Omit grep process in nova_api healthcheck. https://review.openstack.org/618524 14:02:36 <openstackgerrit> Merged openstack/tripleo-common stable/rocky: Fix nova-api healthcheck https://review.openstack.org/618526 14:02:37 <openstackgerrit> Merged openstack/puppet-tripleo stable/rocky: Include ::cinder::nova in cinder's configuration https://review.openstack.org/618316 14:02:39 <openstackgerrit> Merged openstack/tripleo-heat-templates master: ceph: allow curl tasks to run in dry run mode https://review.openstack.org/618753 14:04:12 <jaosorior> #topic review past action items 14:04:31 <jaosorior> None. 14:04:40 <jaosorior> #topic one off agenda items 14:04:42 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:04:53 <quiquell> o/ 14:04:59 <jaosorior> The first one: (rfolco) Remove newton and ocata jobs 14:05:16 <rfolco> I think it's time to remove newton and ocata jobs, so I'd like to ask here first if any objections 14:05:41 <rfolco> #link https://review.openstack.org/#/c/617999/ 14:05:58 <EmilienM> o/ 14:06:15 <mwhahaha> o/ 14:06:45 <rfolco> the challenge is that non-containerized scenario jobs are being "used" in many projects... so zuul syntax check complains if you just remove the job definition... 14:06:52 <rfolco> #link http://codesearch.openstack.org/?q=-%20tripleo-ci-centos-7-scenario00%5B0-9%5D-multinode-oooq(%3A.*)%3F%24&i=nope&files=&repos= 14:07:11 <beagles> o/ 14:07:51 <ccamacho> o/ 14:09:14 <chem> o/ 14:09:47 <jrist> o/ 14:09:56 <rfolco> zuul also checks config for all branches on "untrusted-projects"... which makes me remove these old jobs from all branches on each project :( 14:10:17 <ooolpbot> URGENT TRIPLEO TASKS NEED ATTENTION 14:10:17 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1803547 14:10:19 <openstack> Launchpad bug 1803547 in tripleo "tempest run failed with 'The specified regex doesn't match with anything ' on tripleo-ci-centos-7-scenario001-multinode-oooq-container" [Critical,In progress] - Assigned to chandan kumar (chkumar246) 14:11:19 <jaosorior> rfolco: so, just to get it clear. Other projects are running uncontainerized scenarios, right? 14:11:45 <jaosorior> and that currently is a blocker for removing the ocata and newton jobs. Why is that the case? I didn't quite get the link 14:11:50 <rfolco> jaosorior, other projects have these jobs configured in their zuul yaml 14:12:02 <jaosorior> rfolco: the newton and ocata jobs? 14:12:27 <rfolco> yes, scenario non-containerized are ocata and newton *only* 14:12:44 <rfolco> #link https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/multinode-jobs.yaml#L164 14:12:49 <mwhahaha> I thought we ran them in pike 14:13:16 <rfolco> we should 14:13:33 <mwhahaha> In the meantime you can move the definitons to a deprecated zuul layout to get them out of the main definitons and the propose the removal from the stable branches 14:13:39 <mwhahaha> It's likely puppet 14:14:09 <rfolco> ok, its a good approach 14:14:54 <mwhahaha> This is a side effect of not branching tripleo-ci 14:15:10 <holser_> o/ 14:16:16 <jaosorior> The next topic is: FYI centos minor updates coming to the "continuous release" repo for testing soon 14:16:18 <rfolco> mwhahaha, true. I'll move to a separate file, propose removal on all these projects and its branches, then propose removal of tripleo-ci. Is that ok ? 14:16:49 <mwhahaha> Yes 14:17:04 <rfolco> thanks mwhahaha 14:17:14 <jaosorior> great! 14:17:30 <jaosorior> So, the next topic is: FYI centos minor updates coming to the "continuous release" repo for testing soon 14:17:34 <jaosorior> Not sure who proposed that ^^ 14:17:39 <weshay_pto> panda|rover, ^ 14:17:56 <weshay_pto> jaosorior, it's just a FYI.. 14:17:58 <weshay_pto> really 14:18:20 <panda|rover> weshay_pto: jaosorior yes so we are setting up jobs that add another repo to the centos base 14:18:20 <jaosorior> got it 14:18:34 <panda|rover> weshay_pto: jaosorior so we know id advance of any problem that may come up 14:18:35 <jaosorior> notes and prep are here: 14:18:42 <jaosorior> #link https://tree.taiga.io/project/tripleo-ci-board/issue/309?kanban-status=2027733 14:19:20 <jaosorior> panda|rover: that's gonna be quite useful. We already got bit by some issues with some base packages in 7.6 14:19:21 <jaosorior> thanks! 14:19:37 <panda|rover> still some things left to decide, like when to run those jobs, and other minor details. Jobs are running for now, standalone passed yesterday, testing nultinode and ovb today 14:19:56 <panda|rover> i all branches 14:20:00 <panda|rover> in 14:20:41 <jaosorior> panda|rover: how often are those packages updated? daily? 14:21:40 <ykarel> panda|rover, weshay_pto, so cr repo not helping here? i checked there are new packages already released, no? 14:21:52 <ykarel> fs021 running with that repo 14:21:56 <openstackgerrit> Brent Eagles proposed openstack/tripleo-heat-templates stable/rocky: Fix ansible conditional for ovs upgrade. https://review.openstack.org/619005 14:22:04 <panda|rover> jaosorior: no, last update right now was 12th 14:22:06 <weshay_pto> ykarel, ya.. look at the taiga task, they are coming soon, but not there yet 14:22:37 <ykarel> weshay_pto, okk will check, what i last checked fs021 had some failures with recent packages 14:22:38 <Tengu> panda|rover, weshay_pto: would it be possible to push some 3rd party package like podman ? 14:22:40 <jaosorior> ah, forgot the "coming soon" bit. 14:23:13 <weshay_pto> Tengu, that is up to the ykarel, apevec and co. It's a good idea 14:23:21 <Tengu> EmilienM: -^^ :) 14:23:37 <weshay_pto> there is some work being discussed around further gating centos packages that can break the gate 14:23:47 <Tengu> weshay_pto: well, of course - we already were bitten hard by podman and its friends. Being able to test them BEFORE breaking gate is a must. 14:25:14 <ykarel> weshay_pto, ack clear from the taiga, it's tracking fs021 for updates, Thanks 14:26:20 <ykarel> Tengu, push podman where? in cr repo? 14:26:30 * jaosorior reading the taiga page 14:26:36 <Tengu> ykarel: in the job just described by weshay_pto and panda|rover 14:26:44 <Tengu> so in fs021 14:26:52 <openstackgerrit> Jaganathan Palanisamy proposed openstack/tripleo-common master: Derives NUMA aware vSwitches parameters https://review.openstack.org/617158 14:27:32 <weshay_pto> Tengu, ya.. the rdo guys are putting together a proposal around gating tripleo deps now 14:27:37 <weshay_pto> Tengu++ 14:27:47 <Tengu> :) 14:27:54 <weshay_pto> good that you are thinking about it, wish more folks were 14:28:08 <openstackgerrit> Merged openstack/tripleo-heat-templates master: Add bootparams service for all roles https://review.openstack.org/617938 14:28:09 <openstackgerrit> Merged openstack/tripleo-quickstart master: Use a shorter default domain name https://review.openstack.org/618773 14:28:10 <openstackgerrit> Merged openstack/instack-undercloud stable/queens: Fix curly brackets for ntp::servers: to prevent HTML escaping https://review.openstack.org/618845 14:28:21 <Tengu> weshay_pto: well, we had enough issues with podman updates -.-' 14:28:36 <Tengu> kind of think you can't forget. bad podman. 14:29:09 <weshay_pto> srsly 14:29:23 <ykarel> Tengu, ack need to check how that works, what i know currently it's coming from virt-container repo(https://buildlogs.centos.org/centos/7/virt/x86-64/container/) so those are already in fs021 but that's withoug podman currently 14:29:37 <openstackgerrit> Jaganathan Palanisamy proposed openstack/tripleo-common master: Derives NUMA aware vSwitches parameters https://review.openstack.org/617158 14:29:45 <ykarel> i mean fs021 not have podman enabled 14:30:11 <Tengu> ykarel: ok! ping me if you need infos (or EmilienM - but he's more or less afk right now) 14:30:20 <ykarel> Tengu, ack 14:30:28 <ykarel> will catch you 14:30:53 <jaosorior> alright 14:31:08 <jaosorior> thanks weshay_pto and panda|rover for the info 14:31:17 <jaosorior> The next topic is on me: 14:31:20 <jaosorior> TripleO operator feedback 14:31:26 * ykarel|away out it's late here 14:31:34 <jaosorior> There was a session in the summit about feedback for deployment projects (in general) 14:31:34 * ykarel|away Thanks all 14:31:44 <jaosorior> and there was plenty of feedback on tripleo: 14:31:51 <jaosorior> #link https://etherpad.openstack.org/p/BER-deployment-tools-feedback 14:32:02 <jaosorior> the gist of it was related to folks having a very hard time with its cmoplexity 14:32:30 <jaosorior> some of the issues there could be addressed by config-download. So, seems we're moving in the right direction there. 14:32:41 <jaosorior> However, some of them were due to lack of documentation 14:32:48 <jaosorior> So... we gotta get better at that. 14:33:23 <marios> jaosorior: our entire docs need a big overhaul 14:33:29 <marios> jaosorior: thats a good point we can fix 14:34:14 <jaosorior> stuff from: * How do we make tripleo deploy custom options that are not directly managed by a heat parameter 14:34:20 <openstackgerrit> Quique Llorente proposed openstack-infra/tripleo-ci master: Merge featureset custom env files and override https://review.openstack.org/619010 14:34:35 <marios> jaosorior: was any one deployment tool deemed much better/simpler in that regard ? 14:34:39 <marios> jaosorior: compared to ooo 14:34:59 <mwhahaha> Probably most of them because they don't have the same level of configurations 14:35:20 <jaosorior> marios: there wasn't much of a comparison going on, mostly feedback 14:35:28 <marios> mwhahaha: (right its tricky to compare usually, when a given tool is assuming many things about what it is deploying) 14:35:32 <marios> jaosorior: ack 14:35:36 <mwhahaha> Being really flexible has lead to some over complexity in many areas 14:35:57 <marios> mwhahaha: but that doesn't stop project foo from being perceived as 'easier' assuming it fulfills your use case 14:35:58 <mwhahaha> Next cycle I hope we can focus on simplifying some areas 14:35:58 <jaosorior> mwhahaha: well, it's also related to our architectural decisions. Folks were really confused by containers and by the change in log files for instance 14:36:16 <jaosorior> at some point it would be great to have a docs squad 14:36:27 <jaosorior> so we can pin-point what's missing in our docs, and try to address it. 14:36:55 <mwhahaha> Yes we've also not done a good job at keeping stable interface done around containers 14:37:26 <marios> so some of it is stale but we do have a lot of good content too. a lot of the issue/one of the first issues is about organising that imo right now its hard to find stuff 14:38:06 <mwhahaha> Yes 14:38:41 <openstackgerrit> Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test zuul-in-a-box https://review.openstack.org/617930 14:38:48 <jaosorior> So... if we would create a "squad" or working group... or whatever. To try to address our docs issue. Who would join? :D 14:39:06 <openstackgerrit> Quique Llorente proposed openstack/tripleo-heat-templates master: Add env file for standalone scenario002 https://review.openstack.org/618537 14:39:16 <marios> jaosorior: maybe we can make it one of our goals for T? though that's a way off. maybe too late for S though.. 14:39:21 <marios> i mean 'improve docs' 14:39:47 <marios> jaosorior: i would join 14:39:47 <jaosorior> marios: well... ideally each release we keep up-to-date docs 14:40:00 <jaosorior> so, that should be part of the definition of done of each feature. 14:40:11 <marios> jaosorior: i can at least help to review things for starters 14:40:35 <jaosorior> even if we don't dive super-deep into it, we could start some small cleanup work already 14:40:36 <marios> jaosorior: but don't know how much timei can dedicate to re-organising docs. if no one else does i'll give it a go and see what happens :) 14:40:49 <marios> jaosorior: so maybe its me and you in this squad ;) 14:40:54 <jaosorior> marios: lets sync up tomorrow and throw up some ideas. 14:41:00 <marios> jaosorior: sounds good 14:41:08 <jaosorior> cool! 14:41:26 <jaosorior> Anyway, the link is there in case folks want to see the feedback. Lets move on for now to the next topic. 14:41:38 <weshay_pto> jaosorior, I can help 14:41:38 <jaosorior> #topic Squad status 14:41:39 <jaosorior> ci 14:41:41 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting 14:41:43 <jaosorior> upgrade 14:41:45 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:41:47 <jaosorior> containers 14:41:49 <jaosorior> #link https://trello.com/b/S8TmOU0u/tripleo-podman 14:41:51 <jaosorior> integration 14:41:53 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:41:55 <jaosorior> ui/cli 14:41:57 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status 14:41:59 <jaosorior> validations 14:42:01 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:42:03 <jaosorior> networking 14:42:05 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:42:07 <jaosorior> security 14:42:09 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-security-squad 14:42:11 <jaosorior> edge 14:42:13 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-edge-squad-status 14:42:15 <jaosorior> Any squad wanting to bring up their status, or a topic for the general public? 14:42:17 <jaosorior> weshay_pto: thanks! we're 3 now then :D 14:44:51 <jaosorior> #topic bugs & blueprints 14:44:53 <jaosorior> #link https://launchpad.net/tripleo/+milestone/stein-2 14:44:55 <jaosorior> For Stein we currently have 29 (for stein-2) and 3 (for stein-3) blueprints open in Launchpad. 14:44:57 <jaosorior> Bugs: 765 (+8) stein-2, 3 (+3) stein-3. 101 (-1) open Storyboard bugs. 14:44:58 <openstack> bug 765 in Launchpad itself "Librarian client should timeout faster" [Low,Triaged] https://launchpad.net/bugs/765 14:44:59 <jaosorior> #link https://storyboard.openstack.org/#!/project_group/76 14:45:28 <jaosorior> oho 14:45:55 <Tengu> uhuhu 14:46:16 <jaosorior> #topic projects releases or stable backports 14:46:25 <jaosorior> Nothing from me on this side. 14:47:38 <jaosorior> #topic specs 14:47:40 <jaosorior> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:47:46 <jaosorior> Any specs that folks want to bring up here? 14:48:07 <arxcruz> weshay: hey boss, welcome back 14:48:23 <weshay> hey arxcruz 14:49:52 <jaosorior> #topic open discussion 14:49:53 <jaosorior> Anything else that folks want to bring up to the meeting? 14:50:00 <rfolco> CI Community Meeting starts now at https://bluejeans.com/4113567798 14:50:28 <jaosorior> ack, thanks rfolco 14:55:04 <jaosorior> Alright! 14:55:06 <jaosorior> thanks for joining folks! 14:55:09 <jaosorior> #endmeeting