17:00:57 <hyakuhei> #startmeeting Security 17:00:59 <openstack> Meeting started Thu May 7 17:00:57 2015 UTC and is due to finish in 60 minutes. The chair is hyakuhei. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:00 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:01:03 <openstack> The meeting name has been set to 'security' 17:01:05 <elmiko> heyo/ 17:01:08 <hyakuhei> woo! tmcpeak and nkinder! 17:01:17 <tmcpeak> life is good! 17:01:17 <hyakuhei> Hey elmiko glad to have you here again 17:01:23 <elmiko> =) 17:01:55 <hyakuhei> I guess we'll wait a few minutes for people to join and then we'll get started 17:01:59 <hyakuhei> Everyone ready for the summit? 17:02:04 <elmiko> woot! 17:02:14 <tmcpeak> :'( 17:02:22 <chair6> g'day 17:02:39 <nkinder> trying to be ready for the summit... :) 17:03:04 <hyakuhei> tkelsey: you here? 17:03:56 <sicarie> o/ 17:04:07 <tkelsey> hyakuhei: hey yeah sorry 17:04:12 <tkelsey> multi tasking meetings 17:04:16 <hyakuhei> Great, well welcome everyone :) 17:04:31 <hyakuhei> I don't have a hard agenda today, been scrambling to try to get things ready for the summit 17:04:49 <hyakuhei> So lets spin out an agenda now 17:04:49 <dave-mccowan> o/ 17:04:57 <hyakuhei> * security.openstack.org 17:05:01 <tmcpeak> new bandit version 17:05:05 <hyakuhei> * summit - meetup/lunch 17:05:07 <hyakuhei> * bandit 17:05:15 <hyakuhei> * Anchor roadmap - tkelsey 17:05:16 <tmcpeak> security of requirements 17:05:27 <hyakuhei> elaborate tmcpeak? 17:05:41 <tmcpeak> we're pulling in who knows what packages, most of them aren't signed on PyPI 17:05:43 <elmiko> hyakuhei: ossn? 17:05:54 <hyakuhei> Great 17:06:12 <hyakuhei> It'd be nice to get completed OSSNs up to 50 by the summit 17:06:33 <elmiko> i'd like to tackle another one, but need some guidance on which would be a good one 17:06:35 <nkinder> tmcpeak: hence the reason to use signed system packages instead of pip :) 17:06:54 <hyakuhei> cool 17:06:55 <tmcpeak> nkinder: yeah… that's definitely one approach 17:07:01 <hyakuhei> nkinder: back in your RedHat box. 17:07:09 <tmcpeak> lol 17:07:10 <nkinder> well, any distro 17:07:14 <hyakuhei> lol I know 17:07:28 <hyakuhei> There's a lot of downsides to system packages though 17:07:43 <hyakuhei> not least being system wide... signed PIP would be just lovely 17:07:52 <elmiko> that would be cool 17:07:56 <tmcpeak> it exists today, just less than 10% of requirements are doing it 17:08:09 <elmiko> ah 17:08:39 <tmcpeak> so.. I guess do we want to start with this? 17:08:51 <hyakuhei> #topic Bandit / Security Requirements 17:09:05 <hyakuhei> Might as well start here. So what's the impact/concern for Bandit 17:09:24 <tmcpeak> no impact or concern for Bandit, this is another one of my flighty quests 17:09:30 <tmcpeak> separate items :) 17:09:49 <tmcpeak> what I'm talking about merges nicely with some stuff gmurphy is looking at 17:09:52 <hyakuhei> ok cool 17:10:00 <tmcpeak> which first? 17:10:04 <hyakuhei> Bandit 17:10:15 <tmcpeak> cool - so we've pushed 0.11.0 about half an hour ago 17:10:38 <tmcpeak> finally fixes bundling of config, along with chair6's changes, and two new plugins (fletcher's xml and tkelsey's assert) 17:10:39 <elmiko> a little bit of interesting side not for bandit, 17:10:50 <tmcpeak> oh yeah, elmiko: do share 17:10:51 <elmiko> *note 17:11:00 <elmiko> #link https://bugzilla.redhat.com/show_bug.cgi?id=1217857 17:11:00 <openstack> bugzilla.redhat.com bug 1217857 in Package Review "Review Request: bandit - A framework for performing security analysis of Python source code" [Medium,New] - Assigned to nobody 17:11:10 <elmiko> a coworker proposed bandit for inclusing in fedora 17:11:16 <tmcpeak> wooot! 17:11:17 <tkelsey> heh awesome :D 17:11:19 <elmiko> he's been using it on all sorts of non-openstack stuff 17:11:25 <tkelsey> nice 17:11:31 <fletcher> niice 17:11:42 <tmcpeak> elmiko: you should have him drop by the channel and say hi, would be nice to talk to him 17:11:54 <elmiko> tmcpeak: will do! 17:11:59 <tmcpeak> elmiko: awesome! 17:12:23 <tmcpeak> so bknudson, dave-mccowan, et-al, you should all be good to move to latest Bandit, nothing will have changed since you're using your config file 17:12:57 <chair6> let's say nothing *should* have changed, but i'd suggest verifying that bold statement :) 17:13:03 <dave-mccowan> tmcpeak are there any recommended config file changes for more coverage? 17:13:24 <tmcpeak> dave-mccowan: we can look at creating a new config file to include some of the new plugins 17:14:05 <tmcpeak> chair6: yeah, nothing *should* have changed. Guess we can introduce new bugs, etc… but we haven't intentionally messed with anything that the gates will care about 17:14:46 <tmcpeak> so anything else anyone wants to say on Bandit before I rathole #OSSG ? 17:15:01 <tkelsey> FYI gmurphy poked me to add a new test for assert, so thats in the new version 17:15:15 <tmcpeak> ++ 17:15:18 <hyakuhei> +1 17:15:49 <elmiko> nice 17:15:58 <tmcpeak> cool - so security of requirements 17:16:20 <tmcpeak> something that has been stealing sleep from me off and on since I started working with OpenStack is the upstream packages which are used 17:16:29 <tmcpeak> I know gmurphy has been looking at this also 17:16:32 <tmcpeak> and probably many of you 17:16:37 <tmcpeak> so we have 17:16:48 <tmcpeak> 1) 250 packages which are dragged in at various times in OpenStack deployments 17:17:00 <tmcpeak> 2) no formal vetting process of the security of these packages 17:17:16 <tmcpeak> 3) packages are pushed to PyPI, which supports package signing, yet less than 10% of our requirements are signed 17:17:46 <tmcpeak> this of course opens the door to all kinds of issues - somebody modifies what a developer is pushing in flight to PyPI and our deployments ship with a backdoor, malware, etc 17:17:57 <tmcpeak> somewhere along the connection route, somebody tampers with the package 17:18:13 <tmcpeak> a tampered package can run code on your system with the privilege level that you run as by modifying setup.py 17:18:28 <elmiko> is there some advice about creating signed pacakges? 17:18:33 <tmcpeak> none of this is new, Paul McMillan has talked about it, but I think it's time we look at doing something 17:18:41 <gmurphy> think this is relevant - https://github.com/pypa/pip/issues/1035 17:18:44 <tmcpeak> elmiko: that's one of the things I'd like to propose we work on 17:18:51 * gmurphy just stumbled on it. 17:19:00 <elmiko> i wonder if we could start small by starting to sign some of our packages, regardless of requirements. 17:19:03 <tmcpeak> hound the developers of everything that is a requirement of OpenStack to sign their packages 17:19:07 <elmiko> tmcpeak: +1 17:19:08 <nkinder> so you are mostly focused on the authenticity of the package (vs the quality/security within it) 17:19:18 <tmcpeak> nkinder: I'm really concerned with both 17:19:24 <nkinder> sure 17:19:25 <tmcpeak> I'd like to propose systematic auditing also 17:19:38 <tmcpeak> but signed packages seems like an achievable goal in the mid term 17:19:58 <tmcpeak> operating system packages are all signed, why wouldn't OpenStack packages be? 17:20:22 <tmcpeak> I wrote a little script this morning that parses the PyPI json information for all of the projects listed in global-requirements 17:20:26 <tmcpeak> that's how I know less than 10% are signed 17:20:38 <tmcpeak> so I propose 1) come up with the easiest path to signing 17:20:48 <tmcpeak> 2) start tracking down the authors of the packages which aren't signed 17:20:52 <tkelsey> that makes sense tmcpeak, though we cant really enforce signing or anything, best we can do is work with the upstream maintainers 17:21:11 <tmcpeak> tkesley: you're right, we can't enforce - we can just help 17:21:13 <nkinder> signing is definitely a good goal that will at least prevent something from being tampered with in transit... 17:21:18 <tkelsey> tmcpeak: sure 17:21:34 <tkelsey> nkinder: +1 17:21:36 <elmiko> could we start by ensuring that all openstack packages are signed? 17:21:38 <nkinder> but what does the signing really indicate about the contents of the package? That's sort of the big difference with OS packages. 17:21:43 <gmurphy> that being said pip cannot verify the authenticity of the packages based on the issue i posted before. 17:21:45 <hyakuhei> Is there an open source (trusted/free) code signing org we could use? 17:21:59 <hyakuhei> nkinder: Lots of customers want the big bad NSA out of their code 17:22:03 <hyakuhei> even if the code is crappy 17:22:15 <hyakuhei> not that OpenStack is but signing is never a stamp of quality 17:22:20 <tmcpeak> gmurphy: right, pip doesn't do it. You'd need to validate before pip gets it 17:22:24 <nkinder> there's an authority that signs it that claims something about the contents, then you can ensure you at least have what they produced (good or bad) 17:22:47 <tmcpeak> nkinder: right, that's kind of the immediate goal I'd like to drive towards 17:22:54 <nkinder> hyakuhei: yeah, understood. Just pointing out that signing only solves one small (but important) piece of the problem. 17:23:05 <tmcpeak> if I'm getting a package from somebody trying to subvert me, I'm kind of screwed without thorough auditing 17:23:29 <tmcpeak> you really need to have some level of trust in your upstream dependencies (in general) 17:23:32 <gmurphy> also this pep - https://www.python.org/dev/peps/pep-0458/ 17:24:00 <tmcpeak> gmurphy: ++ I'll have to read into these 17:24:12 <elmiko> interesting 17:24:13 <tmcpeak> so I think when we look at the whole problem - it's way too big 17:24:28 <tmcpeak> we have to start somewhere, and I think a reasonable somewhere is ensuring our little corner of PyPI is signed 17:25:02 <tmcpeak> also once we can get the % significantly higher, we can start applying pressure on the holdouts 17:25:13 <gmurphy> agree. 17:25:31 <tmcpeak> so of the 220 packages which aren't signed, let's say 50% will do if if we can reduce their effort level to less than 2 hours 17:25:40 <tmcpeak> now we have 110 packages to go, we hound them some more 17:25:43 <elmiko> tmcpeak: +1 to our little corner 17:25:57 <gmurphy> although i'd like for us to consider doing code audits when new depednencies are introduced to openstack/requirements too. 17:26:01 <tmcpeak> get it to 50, then we can go to them and say "look guys, we really like what you're doing, but if you can't sign this package we'll have to look for alternatives" 17:26:02 <tmcpeak> etc 17:26:10 <tmcpeak> gmurphy: definitely! 17:26:34 <gmurphy> or at least a bandit run.. 17:26:44 <gmurphy> anyway.. carry on i've got a call to jump on 17:26:56 <tmcpeak> gmurphy: that would be easy… download all requirements, run Bandit with the keystone gate, file bugs 17:27:12 <tmcpeak> then we have at least a starting baseline 17:27:17 <gmurphy> yeah. 17:27:26 <hyakuhei> Could this be a potential bandit feature? To run through the project and it's requirements and report on which are available signed / unsigned ? 17:27:27 <gmurphy> well i was trying to work towards that 17:27:44 <gmurphy> just need to get a few more cycles to have something up and running 17:27:50 <tmcpeak> hyakuhei: sure.. i'd see it as a sidekick script to Bandit, but yeah 17:27:52 <elmiko> nice idea 17:28:26 <tmcpeak> sweet, so yeah, that's kind of what gmurphy and I have been kicking around 17:28:27 <tmcpeak> thoughts? 17:28:37 <tmcpeak> like it, hate it, don't care? 17:28:41 <gmurphy> if we also had cve info for python packages we could report on known vulns etc 17:28:59 <tkelsey> sounds reasonable to me, its going in the right direction for sure 17:29:05 <elmiko> i like the idea of bandit doing some sort of check to see what is signed. i also get where nkinder is coming from about the quality of contents, but i think this would be a great first step. 17:29:16 <nkinder> yeah, we need both 17:29:22 <nkinder> one step at a time :) 17:29:23 <elmiko> nkinder: +1 17:29:40 <tmcpeak> sweet 17:29:54 <tmcpeak> I'll probably keep digging, report back with more next week? 17:30:12 <hyakuhei> I don't think we'll have a meeting next week 17:30:17 <tmcpeak> oh yeah, summit 17:30:21 <hyakuhei> yeh 17:30:24 <tmcpeak> even more time then :D 17:30:26 <hyakuhei> #topic summit 17:30:56 <hyakuhei> As always, I'll try to convince HP to pay for a security meal, though looking at the calendar that'll be hard 17:31:06 <hyakuhei> Any thoughts on when? 17:32:16 <elmiko> that's like the toughest question so far lol 17:32:16 <tmcpeak> these are some well-fed/paid engineers when you can't get anybody interested in a free meal ;) 17:32:19 <nkinder> It's hard to say... 17:33:09 <nkinder> I'd say to just pick a day and see who yells :) 17:33:24 <hyakuhei> lol ok so maybe I'll work something else out, pinch some beers from the Booth or something :P 17:33:38 <hyakuhei> One of the talks I'm doing at the summit is around full stack security 17:33:52 <hyakuhei> Basically I'm presenting all the security options that exist for an OpenStack deployment today 17:33:57 <hyakuhei> and might be available tomorrow 17:34:10 <elmiko> sounds cool 17:34:11 <hyakuhei> Is anyone interested in reviewing my notes? I'm sure there are a bunch of gaps 17:34:42 <hyakuhei> This could include everything from Bios Passwords and PXE isolation to django configuration for Horizon 17:34:47 <tkelsey> hyakuhei: you can ping them to me if you like :) 17:34:56 <tmcpeak> hyakuhei: I'm interested 17:35:04 <nkinder> sounds like a lot to fit into one talk :) 17:35:33 <hyakuhei> It is 17:35:51 <hyakuhei> It'll be a whistle stop tour of technologies and how they can fit together today and what might be around the corner tomorrow 17:36:03 <hyakuhei> Or as I like to call it, how to make your cloud really secure but really broken 17:36:31 <hyakuhei> ok, tkelsey want to spend a few minutes talking about the future of Anchor? 17:36:41 <tkelsey> ok dokie 17:36:48 <tkelsey> just need to find the link 17:37:02 <hyakuhei> #topic Anchor 17:37:07 <hyakuhei> #link https://etherpad.openstack.org/p/Anchor_Project_Roadmap 17:37:09 <hyakuhei> This one ? 17:37:20 <tkelsey> so, I put down a few thoughts into ... yeah that one :P 17:37:45 <tkelsey> this is preliminary and not in priority order or anything 17:38:04 <tmcpeak> looks good 17:38:05 <tkelsey> anyone who is interested please take a look and add you notes as necessary 17:38:13 <tmcpeak> also - you have custom X509 code? ;) 17:38:21 <tmcpeak> have you broken the cardinal rule and rolled your own crypto? 17:38:39 <tkelsey> tmcpeak: well we have custom python wrapper around OpenSSL lol 17:38:47 <tkelsey> so no crypto 17:38:56 <tmcpeak> ahh, ok.. that's (mostly) fine 17:39:24 <tkelsey> tmcpeak: no its not lol :P its on the list to remove as soon as a dedicated lib is available (pyca/cryptography) 17:39:25 <tmcpeak> looks like a solid roadmap 17:39:46 <tmcpeak> yeah, well I thought you implemented X509 in Python.. that would be bad 17:39:50 <tkelsey> i need to check up with reaperhulk at the summit as to the status of the code in cyrptography 17:40:05 <tkelsey> heh yeah that would be 17:40:41 <tmcpeak> tkelsey: solid docs would probably be your #1 17:40:48 <tkelsey> yeah for sure 17:40:53 <tmcpeak> then you can really start selling it 17:41:05 <elmiko> i see the barbican plugin on there, i know there has been some talk about a cert manager plugin for castellan. might that be part of a discussion? 17:41:18 <tkelsey> in terms of priority it is top, a lot of the work I have done for the Vancouver talk will end up as docs 17:41:33 <tkelsey> elmiko: +1 sure 17:41:53 <elmiko> i think it's still a topic for debate in castellan, just curious 17:41:59 <tkelsey> I didnt know Castellan had a direct cert manager plugin 17:42:05 <tkelsey> or plans for it 17:42:09 <elmiko> well, talk mainly 17:42:19 <tkelsey> interesting to know, thanks for mentioning it 17:42:21 <hyakuhei> Are octavia pushing for that? 17:42:40 <elmiko> i think that is where some of the talk is originating from, octavia and neutron-lbaas 17:42:57 <elmiko> it sounded kinda controversial though 17:44:18 <tkelsey> well i guess we can look into whats going on, it wont hurt to add it to the roadmap plan 17:44:23 <tkelsey> since its mostly brainstorming :) 17:44:24 <hyakuhei> Makes sense 17:44:40 <hyakuhei> Though Anchor probably isn't useful for neutron 17:44:52 <hyakuhei> atleast not for doing LBaaS over the edge 17:44:56 <elmiko> yea, most likely it would take some talk with the barbican/castellan folks anyways. 17:45:24 <hyakuhei> cool, we pretty much blew through the makeshift agenda 17:45:31 <hyakuhei> #topic Any other business 17:45:42 <tmcpeak> security guidance! 17:45:54 <tmcpeak> gmurphy mentioned this last week I'm sure, but I wasn't around 17:45:59 <tmcpeak> we've converted them to ReST 17:46:10 <tmcpeak> we just need final editing and we can publish them (finally) 17:46:16 <hyakuhei> oooh 17:46:22 <elmiko> hyakuhei: could i get the link for open bugs needing OSSNs please =) 17:46:31 <tmcpeak> gmurphy: you still here? 17:46:37 <hyakuhei> ossn bugs? 17:46:56 <hyakuhei> #link https://bugs.launchpad.net/ossn 17:46:58 <elmiko> i thought the list of ossns needing writeup came from launchpad? 17:47:03 <hyakuhei> :) 17:47:13 <elmiko> hehe, that was easy 17:47:17 <nkinder> Who can get this fixed? https://security.openstack.org/ 17:47:24 <tkelsey> #link https://bugs.launchpad.net/ossn/+bugs 17:47:25 <nkinder> it's sort of embarrassing... 17:47:30 <hyakuhei> ooer 17:47:44 <gmurphy> the +s will come soon 17:47:47 <tkelsey> nkinder: yikes :-/ 17:47:47 <tmcpeak> #link https://github.com/gcmurphy/python-sec-guidance/tree/master/source/guidelines 17:47:48 <nkinder> seems like plain http on https 17:47:51 <gmurphy> there is a ticket for it somehwere 17:48:00 <tmcpeak> hah, woah 17:48:28 <gmurphy> fungi: knows about this. (the https bit) 17:48:40 <tmcpeak> that's bad news bears 17:48:52 <fungi> https://review.openstack.org/155099 17:48:58 <gmurphy> fungi: thx 17:49:10 <tmcpeak> ahh 17:49:26 <fungi> #link https://review.openstack.org/#/c/155099/ 17:49:35 <fungi> need to remember my meetbot macros there ;) 17:51:15 <hyakuhei> :) 17:51:17 <tmcpeak> gmurphy: so what's needed to get the developer guidance on security.openstack.org ? 17:51:34 <tmcpeak> would be cool to have those in time for summit, then we (you guys) can plug them 17:52:03 <tkelsey> and link them in Bandit docs 17:52:13 <tmcpeak> tkelsey: + 17:53:06 <tmcpeak> fungi: can we create a linked section somewhere on security.openstack.org for the development guidance? 17:53:53 <fungi> we can. at the moment it would need to be committed to the openstack/ossa git repo 17:54:06 <fungi> but i can help usher it through with gmurphy's assistance reviewing 17:54:14 <tmcpeak> fungi: please do 17:54:25 <tmcpeak> I'd love to get them in time for summit, not sure how feasible that is 17:54:43 <hyakuhei> fungi: I'd be happy to see it there for now and then create a repo other than the OSSA one later 17:54:51 <fungi> longer term i think we want to rearrange the security.o.o site so that we can publish to it from multiple related git repos 17:54:51 <hyakuhei> Or maybe you know, the security-doc repo 17:55:00 <hyakuhei> that exists for security docs and whatnot :P 17:55:14 <tkelsey> lol 17:55:22 <hyakuhei> fungi: That could work too 17:55:47 <tmcpeak> yeah, that makes sense 17:55:48 <fungi> yep, release+summit time is just a bit of a scramble for larger undertakings like that, but can work on a better collaboration model for the content there once we're back from vancouver 17:56:03 <tmcpeak> fungi: fair enough 17:56:42 <hyakuhei> +1 17:56:54 <hyakuhei> we're approaching time now people, anything else to address?# 17:57:55 <hyakuhei> I guess that'll do it then! Until after the summit guys! Looking forward to seeing you all once again! 17:58:02 <hyakuhei> #endmeeting