14:00:15 <jaosorior> #startmeeting tripleo
14:00:16 <openstack> Meeting started Tue Sep  4 14:00:15 2018 UTC and is due to finish in 60 minutes.  The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:17 <jaosorior> #topic agenda
14:00:19 <jaosorior> * Review past action items
14:00:20 <openstack> The meeting name has been set to 'tripleo'
14:00:21 <jaosorior> * One off agenda items
14:00:23 <jaosorior> * Squad status
14:00:25 <jaosorior> * Bugs & Blueprints
14:00:27 <jaosorior> * Projects releases or stable backports
14:00:29 <jaosorior> * Specs
14:00:31 <jaosorior> * open discussion
14:00:33 <jaosorior> Anyone can use the #link, #action and #info commands, not just the moderatorǃ
14:00:35 <jaosorior> Hey folks! who's around?
14:00:38 <mwhahaha> ahoy
14:00:48 <jaosorior> \o
14:00:53 <fultonj> o/
14:00:54 <jbadiapa> o/
14:00:54 <ksambor> o/
14:00:58 <jrist> o/
14:01:01 <dpeacock> o/
14:01:02 <marios> o/
14:01:13 <owalsh> o/
14:01:20 <Tengu> \o/
14:01:36 <slagle> hi
14:01:47 <florianf> o/
14:01:52 <holser_> o/
14:02:00 <chem> o/
14:02:00 <shardy> o/
14:02:13 <beagles> o/
14:03:14 <jaosorior> #topic review past action items
14:03:23 <jaosorior> there was one: jaosorior to update the wiki and tripleo docs to refer to the squads and their respective etherpads
14:03:41 <jaosorior> So... I updated the wiki to mention the squads
14:03:51 <jaosorior> #link https://wiki.openstack.org/wiki/Meetings/TripleO
14:04:07 <jaosorior> And added the squads to your docs too:
14:04:09 <jaosorior> #link http://tripleo.org/contributor/index.html#squads
14:04:15 <EmilienM> o/
14:04:49 <matbu> o/
14:04:53 <jaosorior> And that's all on that action.
14:05:04 <jaosorior> #topic one off agenda items
14:05:09 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-meeting-items
14:05:15 <ccamacho> hey folks! o/
14:05:34 <rascasoft> o/
14:05:35 <jaosorior> So, due to a lot of folks being in the PTG, there will be no meeting next week
14:06:01 <jaosorior> I'll send a reminder of this to the mailing list
14:06:15 <jaosorior> #topic PTG agenda
14:06:23 <jaosorior> There is a proposed schedule for the PTG:
14:06:29 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-ptg-stein
14:06:42 <jaosorior> but there is still time for folks to request changes and topics. So, please do.
14:07:09 <EmilienM> thanks jaosorior for bringing up an agenda !
14:07:51 <Tengu> jaosorior: one note: if we're to do a presentation about the validation framework, I'll be remote from EMEA
14:08:15 <shardy> thanks jaosorior this is looking good
14:08:44 <jaosorior> Tengu: we can make sure to enable a bluejeans session so you can participate
14:09:05 <Tengu> jaosorior: securing that would be great - if possible at a nice emea-compatible time :)
14:09:08 <jaosorior> I believe such a bridge was available last year. EmilienM was it just your bridge that was put up for folks to join?
14:09:21 <EmilienM> yeah
14:09:27 <EmilienM> I'll take care of that no worries
14:09:35 <jaosorior> thanks EmilienM
14:09:53 <jbadiapa> If anyone can take a conference call micro it would be great
14:09:59 <Tengu> ah, it's the UX/UI part. just found the schedule.
14:10:14 <jaosorior> Tengu: I believe your topic is schedules for thursday in the evening, so it would be morning for oyu.
14:10:17 <ooolpbot> URGENT TRIPLEO TASKS NEED ATTENTION
14:10:19 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1786764
14:10:20 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1787910
14:10:21 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1789898
14:10:21 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1790199
14:10:21 <openstack> Launchpad bug 1786764 in tripleo "Wrong versions of tripleo-common in container images updated in CI" [Critical,Triaged] - Assigned to Sorin Sbarnea (ssbarnea)
14:10:22 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1790489
14:10:22 <ooolpbot> https://bugs.launchpad.net/tripleo/+bug/1790490
14:10:23 <openstack> Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)
14:10:24 <openstack> Launchpad bug 1789898 in tripleo "check failing due to missing python-tripleoclient" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami)
14:10:25 <openstack> Launchpad bug 1790199 in tripleo "fs035: openstack overcloud deploy returns error code without any signs or error" [Critical,Triaged]
14:10:27 <openstack> Launchpad bug 1790489 in tripleo "ERROR heat.engine.resource ResourceFailure: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6" [Critical,New]
14:10:28 <openstack> Launchpad bug 1790490 in tripleo "failure with collect-logs : Verify Sphinx build" [Critical,Triaged]
14:10:29 <Tengu> jaosorior: more or less yep.
14:10:54 <jaosorior> #topic Logging
14:10:59 <jaosorior> jbadiapa this one is yours
14:11:24 <jbadiapa> jaosorior, thanks
14:11:37 <jbadiapa> tripleo logging status is not in a very good status
14:11:46 <florianf> mwhahaha: thanks for the quick review!
14:12:01 <jbadiapa> there are several services that are not properly configured
14:12:33 <jbadiapa> so, I would like to enable fluentd on all the scenarios if that is possible
14:12:53 <mwhahaha> jbadiapa: So I'm not sure we should enable it on all scenarios
14:13:06 <mwhahaha> i dont' think we want to get into a place where fluentd suddenly breaks everythign
14:13:15 <Tengu> jbadiapa: is fluentd going to last? I think I heard that one might be dropped in favor of something else.
14:13:23 <mwhahaha> additionally we don't have a native way to configure fluentd endpoint
14:13:42 <mwhahaha> i think this is a larger issue without having a complete logging solution that is managable via tripleo
14:13:59 <jbadiapa> there are couple of things here
14:14:00 <mwhahaha> that includes both agent and server
14:14:18 <Tengu> we should maybe use container standard logging with journald+rsyslog...
14:14:42 <jbadiapa> first, no matter where we going we need to have a proper logging solution, currently the solution is fluentd
14:15:35 <jbadiapa> but it is not ok at all, I meant there are several services that are not currently configured
14:15:51 <jbadiapa> and others they are not configured at all
14:16:19 <jbadiapa> BTW, enable fluentd would be temporally
14:16:24 <mwhahaha> jbadiapa: do we have any documentation around what is required to properly integrate a service with fluentd?
14:16:57 <jbadiapa> mwhahaha, I dont think so
14:16:58 <mwhahaha> if it' snot permanent, it's just going to break again and even just enabling it doesn't mean we have testing
14:17:12 <mwhahaha> jbadiapa: so we need to resolve that first if we expect folks to integrate with fluentd
14:19:08 <jbadiapa> ok
14:19:44 <openstackgerrit> Merged openstack/tripleo-heat-templates master: Add reflection of RpcPort to health checks  https://review.openstack.org/583629
14:19:45 <openstackgerrit> Merged openstack/python-tripleoclient stable/rocky: Use tripleo-admin user for minor updates  https://review.openstack.org/599364
14:20:36 <mwhahaha> jbadiapa: so we need fluentd integration docs, need a way to configure/deploy fluentd somewhere (undercloud?) in order to have the scenarios log to a central place, and then we'd need to update the CI framework to support pulling logs from this central place rather than each node
14:20:36 <jbadiapa> I agree that we need to doc how everything should be configured and what we expect form the different services
14:21:05 <mwhahaha> IMHO, a centralized logging on the undercloud could improve the overall CI logging but it's not a small undertaking
14:21:24 <jaosorior> jbadiapa: what's the plan for Stein? is it still to use fluentd or are we switching to rsyslog?
14:21:27 <shardy> how would that work if we go to all-in-one for the scenario jobs?
14:21:58 <mwhahaha> jbadiapa: can you fluentd to localhost?
14:23:06 <jbadiapa> jaosorior, the idea is adding rsyslog for stein and deprecate fluentd on the next release
14:23:22 <shardy> I guess we can work that out, I just wondered given the PTG topic of reworking the scenario jobs to use only one node
14:24:13 <jaosorior> shardy: I guess it shouldn't be too problematic if the server that the services are sending logs to is in the same host.
14:24:31 <jbadiapa> shardy, It would do but I havent tested TBH
14:24:52 <shardy> Ok cool, something to keep in mind I guess
14:25:33 <jbadiapa> mwhahaha, sorry but I'm not sure what you meant
14:26:03 <mwhahaha> jbadiapa: if you have a single node, can you configure the services to send to a local fluentd instance?
14:26:19 * mwhahaha isn't that familar with the general configuration expected with fluentd
14:26:52 <openstackgerrit> Sorin Sbarnea proposed openstack/tripleo-quickstart master: Avoid silent parsing failure of provisioned.csv  https://review.openstack.org/595795
14:27:39 <jbadiapa> mwhahaha, you don't need to send the logs to nowhere, currently the gate is not sending the log anywhere
14:28:01 <mwhahaha> it wouldn't be nowhere, it would be localhost
14:28:07 <mwhahaha> anyway i guess that's a quesiton for another time
14:29:57 <jaosorior> jbadiapa: anything else you wanna bring up on this topic?
14:30:27 <jbadiapa> well, I guess that enable fluentd on all scenarios it not accepted, but at least it's necessary on the scenario we are currently working
14:31:22 <jaosorior> what scenario is that?
14:32:21 <jbadiapa> It depends on the service
14:33:24 <jbadiapa> and then disable it, otherwise is impossible to be checked on the gate
14:33:49 <jbadiapa> we can move on, I'll try to explain it on the mailing list
14:33:54 <jaosorior> understood, gotta test it out somehow. Is using depends-on enough?
14:33:56 <jaosorior> lets do that
14:34:08 <jaosorior> #topic Rocky RC2
14:34:11 <jaosorior> mwhahaha: ^^
14:34:12 <mwhahaha> #link https://launchpad.net/tripleo/+milestone/rocky-rc2
14:34:23 <mwhahaha> so we haven't yet released RC2, we had some CI issues last week and such
14:34:36 <mwhahaha> Hopefully this week we can land all the bits, i k now we had a few critical open bugs
14:35:04 <mwhahaha> I just went through and moved a bunch of bugs that were < Critical but targeted to RC2.  Please make sure to target any new stuff to stein-1 unless it's specific to the RC
14:35:31 <mwhahaha> I'm going to go through the bugs and kickout anything not specifically rocky targeted later today
14:35:51 <EmilienM> https://review.openstack.org/#/c/598167/ is imho a blocker
14:35:53 <mwhahaha> Please let us know if there are any major Rocky related items otherwise we'll cut RC2 and move on
14:35:56 <EmilienM> I couldn't deploy rocky without it
14:36:10 <Tengu> +1
14:36:11 <EmilienM> yatin -1 it but we need to approve it asap
14:36:18 <EmilienM> owalsh: ^ you ok?
14:36:20 <mwhahaha> Can we get some nova/ironic folks to eyeball it?
14:36:24 <Tengu> can confirm, same issue on my side
14:36:28 <mwhahaha> It seems like a placement regression or something
14:36:31 <mwhahaha> if it's only deprecated
14:36:47 <Tengu> mwhahaha: I've added a comment in order to explain what I had on rocky
14:36:49 <mwhahaha> I checked and the referenced patch is not in stable/rocky
14:37:06 <Tengu> yeah, it's on master
14:38:00 <owalsh> EmilienM: https://review.openstack.org/598167 is required in rocky?
14:38:08 <openstackgerrit> Martin André proposed openstack/tripleo-heat-templates master: WIP Refactor openshift services for composable roles  https://review.openstack.org/599618
14:38:33 <Tengu> owalsh: apparently yes. I can't deploy my lab without it because the filters don't properly detect RAM and Disk amount
14:38:37 <EmilienM> owalsh: well for me it was required
14:38:47 <EmilienM> I couldn't deploy (this morning) my overcloud
14:39:05 <Tengu> there's some trickery in there - Ram is detected as 0MB, same for Disk....
14:39:41 <Tengu> and as it requires at least 4GB for the ram (and some more disk than the 0MB detected), it prevents deploy :(.
14:40:20 <mwhahaha> so sounds like we need to track down that issue
14:40:26 <openstackgerrit> Derek Higgins proposed openstack-infra/tripleo-ci master: [WIP] Testing ansible-ml2 job  https://review.openstack.org/582294
14:40:27 <mwhahaha> yes i agree if it's real that's a blocker for rc2
14:40:38 <mwhahaha> anyway that's it on the RC2 front, let's make sure rocky works :D
14:40:51 <jaosorior> thanks mwhahaha
14:41:08 <jaosorior> #topic FYI: Blueprint to use newer ceph versions during Stein (fultonj)
14:41:16 <jaosorior> fultonj: ^^
14:41:35 <fultonj> jaosorior: yes, so we have a blueprint
14:41:42 <fultonj> to test newer versions of ceph during stein
14:41:49 <owalsh> EmilienM: didn't think it was necessary, the nova patch is on master only. Either way +1 from me.
14:41:50 <fultonj> wanted to let people know the plan via the blueprint
14:42:04 <fultonj> jaosorior: and wanted you to ack
14:42:21 <fultonj> probably doesn't need a full discussion at PTG
14:42:26 <fultonj> so i bring it up here
14:42:29 <jaosorior> #link https://blueprints.launchpad.net/tripleo/+spec/ceph-mimic-nautilus
14:42:36 <jaosorior> thanks fultonj
14:42:49 <jaosorior> sounds good
14:42:53 <fultonj> thanks
14:43:22 <jaosorior> #topic Squad status
14:43:24 <jaosorior> ci
14:43:26 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting
14:43:28 <jaosorior> upgrade
14:43:30 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status
14:43:32 <jaosorior> containers
14:43:34 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-containers-squad-status
14:43:36 <jaosorior> integration
14:43:38 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-integration-squad-status
14:43:40 <jaosorior> ui/cli
14:43:42 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status
14:43:44 <jaosorior> validations
14:43:46 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-validations-squad-status
14:43:48 <jaosorior> networking
14:43:50 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-networking-squad-status
14:43:52 <jaosorior> workflows
14:43:54 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-workflows-squad-status
14:43:56 <jaosorior> security
14:43:58 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-security-squad
14:44:00 <jaosorior> edge
14:44:02 <jaosorior> #link https://etherpad.openstack.org/p/tripleo-edge-squad-status
14:44:27 <jaosorior> Any updates from any squad?
14:45:00 <Tengu> there's still the validation framework spec to validate :)
14:45:22 <Tengu> there's an open discussion regarding some wording/features, would be great to get that sorted out.
14:45:36 <Tengu> (so linked to the validation team I guess mostly)
14:46:12 <jaosorior> #topic bugs & blueprints
14:46:14 <jaosorior> #link https://launchpad.net/tripleo/+milestone/rocky-rc2
14:46:16 <jaosorior> #link https://launchpad.net/tripleo/+milestone/stein-1
14:46:18 <jaosorior> For Rocky we currently have 1 blueprints and about 36(+23) open Launchpad bugs.
14:46:20 <jaosorior> For Stein we currently have 27 blueprints open in Launchpad.
14:46:22 <jaosorior> Bugs: 36 rocky-rc1, 740 (+26) stein-1.  103 (-3) open Storyboard bugs.
14:46:24 <jaosorior> #link https://storyboard.openstack.org/#!/project_group/76
14:46:26 <jaosorior> Tengu: ack, gotta re-read your validations spec.
14:46:34 <Tengu> jaosorior: thanks :)
14:47:07 <openstackgerrit> Sorin Sbarnea proposed openstack/tripleo-quickstart master: Remove duplicate dictionary keys  https://review.openstack.org/597547
14:48:26 <jaosorior> #topic projects releases or stable backports
14:48:40 <jaosorior> I guess this was already covered by mwhahaha's topic.
14:49:05 <jaosorior> #topic specs
14:49:07 <jaosorior> #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open
14:49:18 <jaosorior> any other specs that need reviews besides Tengu's ?
14:49:29 <jaosorior> either way, please review the open ones in the link provided above ^^
14:50:42 <jaosorior> #topic open discussion
14:50:48 <jaosorior> Anything else folks want to bring up to the meeting?
14:52:38 <openstackgerrit> Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Avoid fatal task execution when falling back to QEMU  https://review.openstack.org/597523
14:54:14 <jaosorior> alright folks!
14:54:16 <jaosorior> thanks for joining
14:54:22 <openstackgerrit> Merged openstack/python-tripleoclient master: Add fixtures instead mocks  https://review.openstack.org/581342
14:54:30 <jaosorior> see you at the PTG, else, in two weeks
14:54:33 <jaosorior> #endmeeting