14:00:13 <mwhahaha> #startmeeting tripleo 14:00:13 <mwhahaha> #topic agenda 14:00:13 <mwhahaha> * Review past action items 14:00:14 <mwhahaha> * One off agenda items 14:00:14 <mwhahaha> * Squad status 14:00:14 <mwhahaha> * Bugs & Blueprints 14:00:14 <mwhahaha> * Projects releases or stable backports 14:00:14 <mwhahaha> * Specs 14:00:14 <openstack> Meeting started Tue Oct 23 14:00:13 2018 UTC and is due to finish in 60 minutes. The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:15 <mwhahaha> * open discussion 14:00:15 <mwhahaha> Anyone can use the #link, #action and #info commands, not just the moderatorǃ 14:00:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:18 <openstack> The meeting name has been set to 'tripleo' 14:00:23 <mwhahaha> Hi everyone! who is around today? 14:00:24 <Tengu> «o/ 14:00:26 <fultonj> o/ 14:00:29 <shardy> o/ 14:00:31 <ksambor> \o 14:00:32 <d0ugal> o/ 14:00:44 <florianf> o/ 14:00:45 <rlandy> o/ 14:00:51 <dpeacock> o7 14:01:00 <ccamacho> hi! 14:01:01 <slagle> hi 14:01:15 <chem> >o/ 14:01:16 <matbu> o/ 14:01:17 <marios> o/ 14:01:28 <jfrancoa> o/ 14:02:04 <jrist> o/~ 14:02:29 <beagles> o/ 14:02:47 <rfolco|rucker> o/ 14:03:04 <redrobot> o/ 14:03:23 <mwhahaha> alright decent turnout today, let's go 14:03:29 <mwhahaha> #topic review past action items 14:03:34 <mwhahaha> rlandy, weshay, dtantsur to look into switching to TCP for IMPI and possibly tuning retries 14:03:38 <holser_> o/ 14:03:56 <dtantsur> etingof looking into redfish stuff. I did not look into retries, sorry. 14:03:58 <mwhahaha> rlandy: any update on the OVB ipmi issues? (other than rdo cloud failing us on a regular basis) 14:04:20 <panda|lunch> o/ 14:04:28 <rlandy> mwhahaha: I have been working on baremetal to track introspection 14:04:34 <mwhahaha> k sounds like we need to continue to leave that on the action item list for next week 14:04:49 <rlandy> OVB is failing for numerous reasons before we get to introspection :( 14:04:51 <mwhahaha> #action rlandy, weshay, dtantsur to look into switching to TCP for IMPI and possibly tuning retries - postponed until next time 14:04:57 <mwhahaha> k 14:04:59 <marios> mwhahaha: i'm trying to find weshay debug patches not sure they landed yet 14:05:23 <mwhahaha> marios: ok let me know if you need assistance on anything 14:05:28 <mwhahaha> etingof to take a look at OVB+slushy-tools 14:05:37 <mwhahaha> as dtantsur sounds like that is currently in progress 14:06:00 <dtantsur> s/slushy/sushy/ (/me has bad associations with "slushy") 14:06:09 * etingof adds sushy-tools into the OVB -- https://github.com/etingof/openstack-virtual-baremetal/blob/add-sushy-emulator/bin/install_openstackbmc.sh 14:06:11 <mwhahaha> :o 14:06:48 <marios> mwhahaha: reviews still needed here https://review.openstack.org/#/c/610072/ https://review.openstack.org/#/c/610078/ https://review.openstack.org/#/c/610087/ 14:07:09 <mwhahaha> marios: k i'll take a look 14:07:40 <etingof> so redfish BMC works in OVB, but it is apparently quite slow 14:07:59 * etingof is still working on it 14:08:00 <dtantsur> how slow? seconds, minutes? 14:08:45 <chandankumar> \o/ 14:09:38 <dtantsur> mwhahaha: etingof and I are on another meeting, so feel free to not block on our responses 14:09:51 <mwhahaha> dtantsur: got it. 14:10:14 <mwhahaha> sounds like there's at least some progress, etingof do keep us up to date. thanks for your efforts 14:10:17 <ooolpbot> URGENT TRIPLEO TASKS NEED ATTENTION 14:10:18 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1797918 14:10:18 <openstack> Launchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) 14:10:19 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1798195 14:10:20 <openstack> Launchpad bug 1798195 in tripleo "rdo-cloud yum repos unavailable during container updates and failing the undercloud install " [Critical,Triaged] 14:10:22 <mwhahaha> moving on 14:10:25 <mwhahaha> #topic one off agenda items 14:10:25 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-meeting-items 14:10:34 <mwhahaha> (mwhahaha) deployment THT template reorg 14:11:00 <mwhahaha> I sent a note to the ML about moving the service templates around and cleaning up the puppet/docker templates 14:11:02 <mwhahaha> #link http://lists.openstack.org/pipermail/openstack-dev/2018-October/135810.html 14:11:25 <mwhahaha> Please take a minute to review this email and provide feedback. We'd like to get started on this if it's OK 14:11:33 <mwhahaha> #link https://review.rdoproject.org/r/#/c/16994/ 14:11:37 <mwhahaha> for the packaging update for it 14:11:44 <shardy> a while back (in Dublin) we discussed the idea of flattening the puppet/docker templates, I wonder if this could be an opportunity to do that 14:11:56 <mwhahaha> shardy: that's exactly what i was proposing 14:12:12 <mwhahaha> shardy: in the email i took a stab at flattening the aodh templates for review 14:12:41 <mwhahaha> #link https://review.openstack.org/#/c/611188/ 14:12:49 <shardy> mwhahaha: ah nice, I only saw dprince's initial comment about the directory structure 14:13:06 <shardy> FWIW my suggestion to do that was shot down in Dublin as dprince was keen to maintain baremetal puppet support 14:13:22 <shardy> but since we no longer test it, this is probably a good time to revisit the discussion 14:13:45 <mwhahaha> yea, it was mentioned in the mail thread but that is something i'd like to get consensus around 14:13:50 <dprince> if I'm the last man standing up for baremetal support I suppose it isn't worth keeping then 14:14:02 <mwhahaha> since we don't test it anymore I think it's time to reduce the complexity. 14:14:11 <dprince> mwhahaha: regarding your patch (I haven't replied yet) does it help identify what we are using for config though? 14:14:28 <mwhahaha> dprince: there are some options on that, yes 14:14:32 <dprince> mwhahaha: like that was my initial issue... I couldn't easily tell that we no longer used Puppet 14:14:50 <mwhahaha> dprince: so if it's not containers it listed the language (puppet/ansible) 14:14:52 <dprince> mwhahaha: this is why ansible/services made sense to me. 14:15:04 <mwhahaha> dprince: though not sure i'm not sure the best way to represent that in containers 14:15:09 <dprince> mwhahaha: especially for things that will probably always run on baremetal 14:15:17 <dprince> mwhahaha: chrony 14:15:23 <mwhahaha> dprince: https://review.openstack.org/#/c/588111/ 14:15:26 <shardy> dprince: in an ideal world we'd support both, but the CI and template overhead is pretty high - I'd be more convinced about keeping it if we still tested it somewhere 14:15:28 <dprince> mwhahaha: for containers the other directory is fine I think though 14:15:38 <mwhahaha> so i used deployment/timesync/chrony-ansible.yaml 14:15:47 <mwhahaha> which would indicate it's not containerized and uses ansible 14:15:59 <mwhahaha> where as for containers, we would have something like deployment/aodh/aodh-api-containers.yaml 14:16:14 <dprince> mwhahaha: so long as we aren't moving changes environments/ files I think I'm okay with whatever helps at this point 14:16:33 <mwhahaha> dprince: yes environment files should not be moved around (updated yes, moved no) 14:16:41 <dprince> last time I touched things I renamed all the env files (because we agreed to do it). But that is so bad as its the only thing users consume 14:16:44 <mwhahaha> it's just puppet/* docker/* combined -> deployment/ 14:17:04 <dprince> we can rename the other templates as much as we'd like I think but the environment/ files are the ones we should be more careful about 14:17:04 <shardy> the whole nested stack inheritance thing is a pretty big overhead, I suspect the standalone install will run much faster if we reduce/remove that 14:17:29 <mwhahaha> yea that's my hope to get some perf improvements out of this as well 14:17:48 <mwhahaha> reducing complexity and hopefully some heat processing speedup 14:17:55 <dprince> shardy: the killer for the standalone is actually the services too. Having a shim that no-ops out the ones we have set to OS::Heat::None would help us the most I think. 14:18:11 <dprince> shardy: if I create my own roles file it runs as fast as always... 14:18:35 * dprince has been wanting to do this in tripleoclient 14:18:48 <mwhahaha> yea we need to reduce the default service definitions in the overcloud-resource-registry 14:18:50 <shardy> dprince: yeah that's why I was trying to get to use server side merging, then we could remove all the OS::Heat::None stuff and just build the list of *Services 14:19:07 <shardy> https://review.openstack.org/#/c/448209/ needs to be revived to do that, it's on my todo list 14:19:55 <dprince> shardy: I actually want to do it client side though. 14:20:12 <dprince> shardy: does merge server side fix something I'm not aware of? 14:20:32 <shardy> dprince: if we merge server side then you can do -e enable_foo.yaml and have it append to e.g ControllerServices 14:20:47 <shardy> so you just remove all the disabled by default services from the role definition 14:20:52 <shardy> and the -e options add them to the list 14:21:12 <shardy> we could hack around it client side, but my idea was to leverage the heat merging feature instead 14:21:47 <dprince> shardy: I would rather see us refactor the swift plan storage first (Ian's patch) 14:21:47 <shardy> not been a super high priority tho tbh, that patch is pretty stale 14:22:03 <shardy> dprince: yeah this can be done after that 14:22:43 <mwhahaha> so yea sounds like there's some improvements to be had in a few places 14:22:44 <dprince> process_templates.py is so much easier to understand that the Swift processing code in tripleo-common 14:22:55 <mwhahaha> dprince: where is ian's patch? is it in a decent state still? 14:23:40 <dprince> mwhahaha: https://review.openstack.org/#/c/581153/ is also stale 14:23:41 <dprince> :/ 14:23:53 <dprince> I was trying to pair with him this week to fix it up again! 14:24:05 <mwhahaha> dprince: ok let me know if you need some help 14:24:10 <dprince> thrash: you around to help us with this too? 14:24:19 <mwhahaha> anyway that's it on the agenda bits. Please review my ML post and the associated reviews 14:24:21 <dprince> thrash: pairing on https://review.openstack.org/#/c/581153/ 14:24:27 <mwhahaha> i'd like to get some movement on that this week 14:25:00 <ksambor> shardy, dprince I can help with this one https://review.openstack.org/#/c/448209/ 14:25:02 <naturalblue> Hey everyone. Can anyone help with a controller deployment issue. 14:25:03 <mwhahaha> specifically if we can get https://review.rdoproject.org/r/#/c/16994/ then that'll help us test the other patches 14:25:14 <mwhahaha> naturalblue: give us a bit, we're in the middle of a meeting 14:25:22 <naturalblue> sorry 14:25:29 <naturalblue> mwhahaha: sorry 14:25:31 <mwhahaha> alright moving on to status 14:25:36 <mwhahaha> #topic Squad status 14:25:36 <mwhahaha> ci 14:25:36 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting 14:25:36 <mwhahaha> upgrade 14:25:36 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status 14:25:37 <mwhahaha> containers 14:25:37 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-containers-squad-status 14:25:37 <mwhahaha> edge 14:25:37 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-edge-squad-status 14:25:38 <mwhahaha> integration 14:25:38 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status 14:25:39 <mwhahaha> ui/cli 14:25:39 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status 14:25:40 <mwhahaha> validations 14:25:40 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status 14:25:41 <mwhahaha> networking 14:25:41 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status 14:25:42 <mwhahaha> workflows 14:25:43 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status 14:25:43 <mwhahaha> security 14:25:43 <mwhahaha> #link https://etherpad.openstack.org/p/tripleo-security-squad 14:25:53 <mwhahaha> any status items folks would like to raise for visibility? 14:26:04 <etingof> followup: a redfish power status call currently takes ~15secs on RDOcloud. the difference wrt ipmi is that you can't ask redfish for just one thing (e.g. power status), you effectively ask many things at once. with the redfish emulator backed by openstacksdk that translates into multiple cloud API calls... 14:26:15 <openstackgerrit> Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Added support for installing tempest plugins from git https://review.openstack.org/612377 14:27:30 <pvc_> anyone know how to remove a pci_device? 14:27:38 <pvc_> Device 0003:01:05.1 not found: could not access /sys/bus/pci/devices/0003:01:05.1/config: No such file or directory 14:27:44 <pvc_> nova_libvirt logs 14:27:47 <openstackgerrit> Chandan Kumar proposed openstack/tripleo-quickstart master: [DNM] testing tempest plugin installation from git https://review.openstack.org/612386 14:28:22 <openstackgerrit> Chandan Kumar proposed openstack/tripleo-quickstart master: [DNM] testing tempest plugin installation from git https://review.openstack.org/612386 14:28:22 <mwhahaha> sounds like no on status items 14:28:25 <mwhahaha> #topic bugs & blueprints 14:28:25 <mwhahaha> #link https://launchpad.net/tripleo/+milestone/stein-1 14:28:25 <mwhahaha> For Stein we currently have 28 blueprints and about 740 (-3) open Launchpad bugs. 736 stein-1, 4 stein-2. 102 open Storyboard bugs. 14:28:25 <mwhahaha> #link https://storyboard.openstack.org/#!/project_group/76 14:28:39 <mwhahaha> so just to highlight, technically this week is stein m1 14:28:52 <mwhahaha> soooooo yea we're already at milestone 1 14:29:09 <mwhahaha> if your blueprint is targeted to stein-1, you should probably update that 14:29:26 <mwhahaha> any bug related issues? 14:29:48 <shardy> I have one 14:29:56 <shardy> we did a revert for bug #1798525 14:29:56 <openstack> bug 1798525 in tripleo "The undercloud-upgrades job is failing during the upgrade with "error was: 'ironic_api_short_bootstrap_node_name' is undefined" " [Undecided,Triaged] https://launchpad.net/bugs/1798525 - Assigned to Marios Andreou (marios-b) 14:30:11 <marios> shardy: fyi i filed it here https://tree.taiga.io/project/tripleo-ci-board/issue/219?kanban-status=2027733 so you don't have to explain? 14:30:11 <shardy> but my revert-revert passes CI ref https://review.openstack.org/#/c/611800/ 14:30:14 <marios> (as much) 14:30:19 <openstackgerrit> Chandan Kumar proposed openstack/tripleo-quickstart master: [DNM] testing tempest plugin installation from git https://review.openstack.org/612386 14:30:34 <shardy> marios: Ok thanks, I just wanted to ask for help as I still don't understand why the revert-revert works 14:30:37 <shardy> or how to proceed 14:30:38 <marios> shardy: rlandy and i have had a look and we still don't know why, rlandy suspects the depends-on 14:30:44 <marios> shardy: current plan 14:30:56 <openstackgerrit> Chandan Kumar proposed openstack/tempest-tripleo-ui master: [DNM] testing tripleo ui temepst plugin https://review.openstack.org/612689 14:31:02 <marios> shardy: unless we get a better one is to make it non voting and move it from gate for now, merge the revert 14:31:02 <shardy> that series fixes some real issues with bootstrapping and custom roles, so it'd be good to get some eyes on it and agree the way forward 14:31:07 <marios> shardy: but the RCA is still ongoing/ 14:31:21 <shardy> marios: Ok thanks for the update 14:31:47 <florianf> There are 2 patches related to bug #1798590 (broken default plan creation). If anyone could review them it would be awesome. The second one will also need a backport when merged: https://review.openstack.org/#/c/611011/ and https://review.openstack.org/#/c/611305/ 14:31:47 <openstack> bug 1798590 in tripleo "No default plan is created during the deployment of a containerized undercloud" [Undecided,In progress] https://launchpad.net/bugs/1798590 - Assigned to Florian Fuchs (flo-fuchs) 14:34:39 <mwhahaha> ok please take a look at the various patches 14:34:41 <mwhahaha> moving on 14:34:42 <mwhahaha> #topic projects releases or stable backports 14:35:03 <mwhahaha> i don't think we have anything note worthy for this. We'll be doing monthly stable releases. 14:35:13 <mwhahaha> like I said technically M1 is this week 14:35:23 <mwhahaha> so i'll poke EmilienM and we'll get a release together 14:35:40 <mwhahaha> #topic specs 14:35:40 <mwhahaha> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open 14:35:44 <mwhahaha> any spec related items? 14:38:24 <mwhahaha> sounds like no. please take a moment to review the open specs 14:38:26 <mwhahaha> #topic open discussion 14:38:28 <mwhahaha> any other items? 14:38:55 <marios> TripleO CI Community Meeting next @ https://bluejeans.com/5878458097 14:39:00 <rfolco|rucker> I Community Meeting starts now at https://bluejeans.com/5878458097 - agenda: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting ~L32 14:39:46 <mwhahaha> alright thanks everyone 14:39:48 <mwhahaha> #endmeeting