13:00:27 <lhinds> #startmeeting Security Squad Meeting
13:00:28 <openstack> Meeting started Wed Mar  7 13:00:27 2018 UTC and is due to finish in 60 minutes.  The chair is lhinds. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:31 <openstack> The meeting name has been set to 'security_squad_meeting'
13:00:41 <jaosorior> lhinds: thanks!
13:00:46 <lhinds> hey jaosorior .. shardy owalsh
13:01:01 <lhinds> let's give it a min for others to arrive
13:01:05 <jaosorior> So, this would be the first TripleO Security Squad meeting
13:01:10 <jaosorior> yeah, that sounds like a good idea
13:01:12 <lhinds> yay!
13:01:31 <owalsh> hey
13:01:43 <jaosorior> owalsh: :D
13:01:56 <lhinds> hey owalsh
13:02:02 <moguimar> I'm here for the meeting as well o/
13:02:05 <jaosorior> owalsh, lhinds: Lets wait a little bit, since shardy had mentioned he was interested.
13:02:09 <jaosorior> moguimar: hey! nice!
13:02:13 <lhinds> hey moguimar , good to have you
13:02:22 <moguimar> thanks lhinds
13:03:08 <lhinds> ok, jaosorior I guess we can kick off if ok with you
13:03:13 <jaosorior> that's fine with me
13:03:20 <lhinds> #topic agenda
13:03:25 <lhinds> #link https://etherpad.openstack.org/p/tripleo-security-squad
13:03:41 <lhinds> Topics
13:03:43 <lhinds> Group Kickoff
13:03:45 <lhinds> General organization
13:03:47 <lhinds> Work items
13:03:49 <lhinds> Any other business
13:03:55 <jaosorior> Right, so we have an etherpad which lhinds posted above. At the bottom of it we set up a small agenda for today ^^
13:04:03 <lhinds> anyone need to add an agenda item, please do add to the pad
13:04:31 <lhinds> #topic Group Kickoff
13:04:42 <lhinds> jaosorior: over to yourself oz
13:04:49 <jaosorior> great
13:05:31 <jaosorior> So, as some of you might know, we have Squads in TripleO, which is similar to Special Interest Groups (SIGs); and it's a group of folks dedicated to working on a certain topic.
13:05:56 <jaosorior> We decided to finally start the Security Squad, based on several tasks that have come up from different sources
13:06:13 <jaosorior> Also, to gather a bit more attention and hopefully contributors upstream, to work on security
13:06:53 <jaosorior> As mentioned in the etherpad, we gathered some tasks that we think should be addressed on the security side, and we'll be having meetings every week to see the progress of these
13:07:12 <jaosorior> but folks can also use the meeting to bring up questions, or things they think should be addressed
13:08:00 <jaosorior> I think that would pretty much cover the overview of the group, does that make sense to folks?
13:08:19 <moguimar> yes
13:08:34 <lhinds> sounds good to me jaosorior , anyone want jaosorior  to clairfy anything or have any questions?
13:08:48 <owalsh> sounds good to me too
13:08:52 <lhinds> s/clairfy/clarify
13:09:08 <jaosorior> As mentioned in the etherpad, note that this is TripleO specific, so any general OpenStack security topics should be covered in the OpenStack Security SIG; of which lhinds is the PTL.
13:09:22 <bogdando> btw, does anyone know which tiny dwarfs mutate that https://review.openstack.org/#/c/550100/1/environments/undercloud.yaml@45 into IP ??
13:09:50 <lhinds> #topic General organization
13:10:41 <lhinds> do you have anything on the topic jaosorior ? or already covered
13:10:45 <jaosorior> For now, given that the group of people is small, I think we can track the work in the etherpad, is that fine for people? If more people get interested and start working on things, eventually we could use something else. But it seems to me that this would be a good start.
13:11:02 <lhinds> jaosorior: +1
13:11:15 <lhinds> etherpad is fine for starters
13:11:25 <owalsh> +1
13:11:45 <jaosorior> alright, so that was easy :D
13:11:47 <jaosorior> next
13:11:50 <moguimar> +1
13:11:56 <lhinds> #topic Work Items
13:12:20 <jaosorior> lhinds and me added some topics (or proposals) to the etherpad already; which would be some main topics to cover
13:12:48 <lhinds> jaosorior: we can walk through each one as a topic if you like?
13:12:57 <jaosorior> yeah
13:13:02 <lhinds> #topic CI Security Job
13:13:30 <jaosorior> So, maybe the CI Security Job is not the best one to have on top, but it's something we would ultimately want.
13:13:52 <lhinds> good point, so there is no priority to the order
13:14:38 <jaosorior> it would cover encryption everywhere (either TLS or IPSec), SELinux everywhere, among other security related features
13:15:01 <jaosorior> the reason it's mentioned it should be RHEL based is because of SELinux
13:15:46 <jaosorior> I brought that topic at the PTG, and folks mentioned that testing SELinux on CentOS would be quite problematic, since any issue in the policy would break us, and we can't get fixes for that as fast as we wished
13:15:52 <jaosorior> and those fixes get earlier on RHEL
13:16:15 <jaosorior> so, to test this as fast as possible (including policy fixes), we concluded that having a RHEL based job would be the best idea.
13:16:38 <jaosorior> weshay mentioned that there plans to get such a job, so it would be a matter of waiting for it to be available (and helping out if possible)
13:17:06 <jaosorior> I would also like to see yolanda's hardened images tested as part of this, but I'm not sure if it would be feasable
13:17:39 <owalsh> we would need oooq support for * I assume
13:17:39 <jaosorior> So, point being, I don't think we can work on this yet, until we get a RHEL job in RDO CI
13:17:45 <jaosorior> owalsh: correct
13:18:10 <jaosorior> any questions/feedback on this?
13:18:32 <lhinds> just an idea, if folks are interested in working or initally monitoring an item, they can mark there names (added 'interested' section to work items)
13:18:42 <lhinds> *their
13:19:32 <jaosorior> lhinds: alright, maybe we can move to the next item
13:19:49 <lhinds> #topic SELinux everywhere
13:20:13 <lhinds> unless that was already covered above?
13:20:21 <jaosorior> I can elaborate
13:20:23 <jaosorior> so
13:20:47 <jaosorior> although we do set SELinux to enforce downstream; we don't do so for docker
13:21:18 <jaosorior> and with the current way we deploy/use containers, we can't actually enable it
13:21:47 <bogdando> #link https://review.openstack.org/#/c/517383/
13:21:48 <jaosorior> I added some notes here http://jaormx.github.io/2018/selinux-and-docker-notes/ regarding what needs to be done for containers in general
13:21:58 <jaosorior> right, bogdando had started that job at some point
13:22:04 <jaosorior> and I think it would be great to continue it
13:22:29 <jaosorior> the proposal is: * get SELinux to enforce for docker and disable it explicitly for all containers * Work container by container to make them work with SELinux
13:22:52 <jaosorior> and for testing I would like it to be in the RHEL job, this way we can catch policy issues early and fix them without breaking CI for too long.
13:23:08 <bogdando> yeah, I think that should work for dev/test cycles
13:23:35 <jaosorior> bogdando: would you get some cycles to continue working on this? I'll also dive into it
13:24:07 <bogdando> if I have some spare time from undercloud containers effort
13:24:10 <owalsh> IIRC the containers that caused most problems with selinux were nova_libvirt and nova_migration_target (sshd)
13:24:31 <owalsh> as the services detect selinux is enabled/disabled and behaviour changes
13:24:33 <jaosorior> owalsh: would certainly use your help getting those working (if possible)
13:25:05 <owalsh> jaosorior: sure, no expect on selinux but I'm willing to help
13:25:14 <jaosorior> thanks
13:25:33 <jaosorior> any questions/feedback about the SELinux for containers proposal?
13:26:06 <owalsh> so it requires the CI security job?
13:26:32 <owalsh> or at least oooq support for local dev/testing?
13:26:33 <jaosorior> owalsh: we can start without it, but ultimately we want it tested in CI
13:26:39 <openstackgerrit> Jiri Stransky proposed openstack/tripleo-upgrade master: New update prepare command  https://review.openstack.org/550472
13:26:53 <jaosorior> owalsh: yeah, at least oooq support would be great
13:27:50 <jaosorior> owalsh: once we start putting patches for it, we need to add instructions on that etherpad for other folks to try it out
13:28:31 <owalsh> jaosorior: ack
13:28:57 <jaosorior> #topic Security Hardening
13:29:07 <jaosorior> lhinds: oh, seems you gotta do it
13:29:16 <lhinds> #chair jaosorior
13:29:17 <openstack> Current chairs: jaosorior lhinds
13:29:21 <lhinds> try again oz.
13:29:24 <jaosorior> #topic Security Hardening
13:29:29 <lhinds> coolio
13:29:33 <jaosorior> thanks!
13:29:40 <jaosorior> lhinds: can you elaborate on that one? since you started that work.
13:30:36 <openstackgerrit> Jiri Stransky proposed openstack/tripleo-quickstart-extras master: Overcloud update CI with tripleo-upgrade role  https://review.openstack.org/542201
13:30:41 <lhinds> Sure, so this generally covers security changes needed that harden tripleO deployments, but are not substantial undertakings (generally) that require tracking multiple parts of a proposed change
13:30:48 <openstackgerrit> Jiri Stransky proposed openstack-infra/tripleo-ci master: Add multinode-overcloud-update playbook to run list  https://review.openstack.org/547058
13:31:04 <lhinds> so for example, a file permission, config value, insecure service that does not need running etc..
13:31:29 <owalsh> lhinds: so it's this the sort of stuff that comes from corporate security policies? e.g no telnet/rsh
13:31:32 <lhinds> in turn a lot of them are required for security compliance stuff, such as DISA STIG etc.
13:31:41 <lhinds> owalsh: yes, very much!
13:32:14 <lhinds> but could also be something around general hardening.. aka https://bugs.launchpad.net/tripleo/+bug/1665308
13:32:15 <openstack> Launchpad bug 1665308 in tripleo "Implement auth on mongodb" [High,Triaged]
13:33:08 <lhinds> so something could start by being logged into security hardening, but become an individual work item if it has a large scope.
13:33:38 <lhinds> as per the etherpad, there is a tag we use to track
13:33:42 <lhinds> #link https://bugs.launchpad.net/tripleo/+bugs?field.tag=security-hardening
13:34:17 <lhinds> I need to have a wash up of these, as a few I think can be closed now
13:34:27 <lhinds> that's all, unless any questions?
13:34:36 <owalsh> lhinds: some overlap with the realtime work e.g different kernel so I'll add my name to the list
13:34:48 <lhinds> owalsh: great! sounds good
13:35:06 <owalsh> also know my way around the hardened images since I basically copied what yolanda did when adding realtime images
13:35:14 <lhinds> some of the stuff I have done can be seen here:
13:35:25 <lhinds> #link https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/security_hardening.html
13:35:57 <lhinds> owalsh: just noted the SSH stuff needs an update in the above docs list, from your nova migration work
13:36:20 <owalsh> lhinds: sounds like a concrete work item :-)
13:36:29 <lhinds> owalsh: thx!
13:37:05 <lhinds> I think we can go onto Secrets Mgmgt jaosorior ?
13:37:14 <jaosorior> #topic Secret Management for TripleO
13:37:25 <jaosorior> So, this one is a big fish
13:38:14 <jaosorior> for compliance, we shouldn't be storing credentials in cleartext (or at least aim to reduce them(
13:38:15 <jaosorior> )
13:39:00 <jaosorior> But the aim is to have all of the undercloud/overcloud passwords stored in safe medium (be it Barbican, Custodia, Vault, Ansible Vault... or whatever is necessary)
13:39:33 <jaosorior> I think a good start for this would be to go through a deployment, identify all the sensitive data, who uses it and how is it used.
13:39:54 <jaosorior> document that (maybe in another etherpad)
13:40:05 <jaosorior> and from there move on to come up with proposals on how to store them
13:40:14 <owalsh> jaosorior: is the goal to use an existing system on the cus network, or to deploy our own?
13:40:14 <lhinds> jaosorior: do you think we should do that publically?
13:41:04 <jaosorior> lhinds: I think it should be, as much as possible.
13:41:31 <jaosorior> owalsh: maybe deploy our own (use Barbican on the undercloud for instance)
13:41:39 <lhinds> jaosorior: cool, if anything is more nasty, we should embargo it..we can go over that when we look at doing the audit
13:41:40 <jaosorior> owalsh:  it would depend on what needs to access what.
13:41:50 <jaosorior> lhinds: definitely
13:42:18 <jaosorior> owalsh: for instance, we still use puppet/hieradata for passwords and such, so maybe in those cases we need to look for backends for hiera that store them encrypted (hiera-eyaml or heira-vault)
13:42:23 <jaosorior> * hiera-vault
13:42:44 <jaosorior> and if that's not possible to use, then we would need to write our own.
13:43:56 <jaosorior> or, for config-download, stuff is stored in yaml files; so we might need to use barbican in the ansible templates, or, if it's not possible, use ansible Vault.
13:44:02 <owalsh> ack
13:44:05 <jaosorior> So yeah, we need to check out all the data first, and figure out what possibilities there are.
13:44:19 <owalsh> so a ton of work
13:44:25 <jaosorior> yeah
13:44:27 <jaosorior> that's a heavy one
13:44:51 <jaosorior> questions/feedback?
13:45:54 <moguimar> if that topic is so big, shouldn't we break it in sub topics?
13:46:08 <moguimar> in etherpad I mean
13:46:35 <lhinds> moguimar: a new pad? might be a good idea once things kick off
13:48:13 <moguimar> not a new pad, but something more than a title and a list of goals
13:48:34 <moguimar> something like breaking the goals in subtasks
13:49:08 <moguimar> that could be achived in 1 or 2 days or work
13:49:09 <lhinds> ack, so yes that would be the next evolution of the work item.
13:49:22 <moguimar> +1
13:49:38 <moguimar> got it
13:50:09 <owalsh> while I'm interested, I doubt I'll have much bandwidth to spare for this so leaving my name off the list
13:50:21 <jaosorior> I think it would be easier once we actually start identifying the sensitive data in a typical deployment. Cause right now, yes, I have some in mind, but I cannot name all of them from the top of my head.
13:51:05 <jaosorior> so, once we actually go through the deployment and identify items, we can start coming up with action items and strategies for the different pieces
13:51:06 <owalsh> jaosorior: I guess you just need to pick one to prototype it
13:51:08 <lhinds> so key item is the audit and collecting areas that need addressing?
13:51:32 <jaosorior> cause probably we can't do the same for mistral, swift and ansible
13:51:38 <jaosorior> lhinds: yes
13:52:30 <lhinds> jaosorior: any ideas for a pad name?
13:52:44 <jaosorior> lhinds: the same name as the topic maybe
13:52:47 <d0ugal> FWIW, there have been a couple of attempts to solve this in Mistral but they have stalled a few times. Mostly due to it being a hard problem to solve :)
13:52:54 <lhinds> jaosorior: k
13:53:27 <jaosorior> d0ugal: if you have some proposal and would need feedback, or would like to discuss the challenges, you could bring it up to this meeting :D
13:53:48 <jaosorior> or to the OpenStack Security SIG, since it's kinda general OpenStack and not TripleO specific
13:53:58 <jaosorior> d0ugal: but yeah, would be glad to help
13:54:17 <d0ugal> yup, sounds good. I wasn't that close to the work so I need to learn more, but I'll try and do that and then get help
13:54:32 <d0ugal> The result of the PTG discussion was that we need external help/advice :)
13:54:59 <honza> mwhahaha: what spec of mine do you want me to move to rocky?  i only have one open, and it's already targeted to rocky afaik
13:55:26 <lhinds> d0ugal what sort of help / advice is needed?
13:55:52 <lhinds> (I got hit with flu, so never made it)
13:56:22 <d0ugal> lhinds: good question. I need to figure out what questions we need help with - but I think it was at the level of general advice
13:56:51 <d0ugal> meaning that we were unsure of the best way forward
13:57:04 <lhinds> d0ugal: ack.
13:57:11 <d0ugal> (sorry, very vague. I'll try and learn more)
13:57:29 <lhinds> shall we track this within the squad so we can help stay on top and give a hand?
13:57:50 <jaosorior> +1
13:58:17 <lhinds> Thinking it might be good for the squad, as this is more around how ooo uses mistral, then core mistral concerns..?
13:58:17 <d0ugal> Good question. I guess I should make sure there is a central bug/blueprint for this in Mistral and then you can keep an eye on it
13:58:22 <openstackgerrit> Chuck Short proposed openstack/os-net-config master: Remove mox dependency  https://review.openstack.org/550477
13:59:05 <lhinds> #action track mistral (audit?) and have a central bug / blueprint
13:59:22 <lhinds> hmm, I thought that was a meetbot command.
13:59:45 <mwhahaha> honza: that's fine. I just pinged the people with open specs
13:59:47 <lhinds> d0ugal: are you ok to lodge a bug / bp? even if just a skeleton intially?
13:59:56 <honza> mwhahaha: gotcha, thanks
14:00:00 <d0ugal> lhinds: sure, I suspect we have one somewhere already. I'll dig around.
14:00:07 <lhinds> d0ugal: thx!
14:00:30 <jaosorior> lhinds: should we move to the next topic?
14:00:32 <lhinds> oh we are on the hour.. jaosorior threat analysis we can skip
14:00:46 <lhinds> did you want to do public tls quickly jaosorior ?
14:00:53 <jaosorior> sure
14:01:00 <jaosorior> #topic Public TLS by default
14:01:19 <jaosorior> Right, so, for Rocky I would like to have public TLS enabled by default in both the undercloud and the overcloud
14:01:31 <jaosorior> shouldn't be particularly a lot of work and it would be a nice thing to have
14:01:49 <openstackgerrit> Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: FFU upgrade issue with the release loop var.  https://review.openstack.org/549035
14:01:49 <openstackgerrit> Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: DNM: Add debug and don't fail on validation for ffu.  https://review.openstack.org/549269
14:01:50 <openstackgerrit> Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: Fix non working repo upgrade command for ffu.  https://review.openstack.org/550478
14:01:53 <lhinds> sgtm
14:02:12 <jaosorior> moguimar: I see you were interested, so we can pair it up if you want, and expand on that topic. Didn't write too much about it.
14:02:21 <shardy> Sounds good but we should try to improve the experience when using upstream builds with self-signed certs I think
14:02:23 <moguimar> sure
14:02:26 <openstackgerrit> Sergii Golovatiuk proposed openstack/tripleo-heat-templates stable/ocata: DO NOT MERGE: Testing I7bcc18445f3263128a9bff29a057f172af143c90  https://review.openstack.org/550480
14:02:33 <jaosorior> shardy: ??
14:02:34 <moguimar> TLS is something I have some background
14:02:58 <shardy> jaosorior: the stuff we looked at yesterday, you can't access the undercloud APIs from outside of the undercloud without copying certs etc
14:03:07 <shardy> we should make that clearer, at least in docs
14:03:53 <jaosorior> shardy: sounds like good feedback, added it to the notes in the etherpad
14:04:23 <openstackgerrit> Tim Rozet proposed openstack/tripleo-heat-templates stable/queens: Fixes OpenDaylight healthcheck/GUI feature  https://review.openstack.org/550481
14:04:32 <openstackgerrit> Tim Rozet proposed openstack/tripleo-common stable/queens: Fixes OpenDaylight healthcheck  https://review.openstack.org/550482
14:05:35 <slagle> EmilienM: what is supposed to install the heat packages when using undercloud containers?
14:05:37 <jaosorior> shardy: I don't think there's much we can do outside of writing docs to make that part work though. As we're not able to issue publicly trusted certs for the undercloud (we could... but it's not meant to be publicly accessible)
14:05:38 <lhinds> so we are overtime now..we can cover Limit TripleO users as a first item for next meeting?
14:05:46 <jaosorior> lhinds: sounds good
14:06:01 <jaosorior> Alright folks, thanks for joining!
14:06:15 <lhinds> yep, great first meeting !
14:06:22 <EmilienM> slagle: python-tripleoclient*
14:06:25 <lhinds> off to a really good start, thanks for the idea jaosorior
14:06:32 <lhinds> #endmeeting