| Daviey | viraptor: Yeah, i'll do that tomorrow. Bed is calling me | 00:01 |
|---|---|---|
| Daviey | viraptor: But if you have time, do try that devstack plugin ^^ | 00:01 |
| openstackgerrit | Dave Walker proposed openstack/anchor: [WIP] Initial commit of devstack plugin https://review.openstack.org/206264 | 00:03 |
| *** markvoelker has joined #openstack-security | 00:09 | |
| *** jmckind has joined #openstack-security | 00:27 | |
| *** salv-orlando has joined #openstack-security | 00:33 | |
| *** salv-orlando has quit IRC | 00:37 | |
| *** openstackgerrit has quit IRC | 00:46 | |
| *** openstackgerrit has joined #openstack-security | 00:47 | |
| *** sicarie__ has quit IRC | 00:49 | |
| *** elo1 has joined #openstack-security | 00:51 | |
| *** dave-mcc_ has joined #openstack-security | 00:53 | |
| *** elo has quit IRC | 00:54 | |
| *** bpokorny_ has joined #openstack-security | 00:55 | |
| *** dave-mccowan has quit IRC | 00:57 | |
| *** bpokorny has quit IRC | 00:58 | |
| *** jmckind has quit IRC | 01:05 | |
| *** jmckind has joined #openstack-security | 01:07 | |
| *** jmckind has quit IRC | 01:09 | |
| *** sicarie__ has joined #openstack-security | 01:09 | |
| *** sicarie__ has quit IRC | 01:11 | |
| *** bpokorny_ has quit IRC | 01:26 | |
| *** salv-orlando has joined #openstack-security | 01:44 | |
| *** salv-orlando has quit IRC | 01:48 | |
| *** lexholden has quit IRC | 01:51 | |
| *** sigmavirus24 has quit IRC | 02:07 | |
| *** jhfeng has joined #openstack-security | 02:20 | |
| *** viraptor has quit IRC | 02:24 | |
| *** jhfeng has quit IRC | 02:33 | |
| *** bpokorny has joined #openstack-security | 02:51 | |
| *** bpokorny has quit IRC | 02:51 | |
| *** bpokorny has joined #openstack-security | 02:52 | |
| *** bpokorny has quit IRC | 02:52 | |
| *** bpokorny has joined #openstack-security | 02:56 | |
| *** tkelsey has joined #openstack-security | 03:05 | |
| *** tkelsey has quit IRC | 03:10 | |
| *** rmarathu has quit IRC | 04:02 | |
| *** markvoelker has quit IRC | 04:04 | |
| *** bpokorny has quit IRC | 04:11 | |
| *** bpokorny has joined #openstack-security | 04:11 | |
| *** bpokorny has quit IRC | 04:14 | |
| *** rmarathu_ has joined #openstack-security | 04:19 | |
| *** bpokorny has joined #openstack-security | 04:20 | |
| *** rmarathu has joined #openstack-security | 04:20 | |
| *** bpokorny has quit IRC | 04:22 | |
| *** rmarathu_ has quit IRC | 04:24 | |
| *** rmarathu has quit IRC | 05:02 | |
| *** rmarathu has joined #openstack-security | 05:02 | |
| *** markvoelker has joined #openstack-security | 05:05 | |
| *** tmcpeak has quit IRC | 05:07 | |
| *** markvoelker has quit IRC | 05:10 | |
| *** jamielennox is now known as jamielennox|away | 05:52 | |
| *** viraptor has joined #openstack-security | 05:53 | |
| *** salv-orlando has joined #openstack-security | 05:55 | |
| *** jamielennox|away is now known as jamielennox | 05:56 | |
| *** cx has joined #openstack-security | 06:41 | |
| cx | hla | 06:42 |
| *** cx has quit IRC | 06:46 | |
| *** salv-orlando has quit IRC | 06:59 | |
| *** markvoelker has joined #openstack-security | 07:06 | |
| *** sdake has joined #openstack-security | 07:07 | |
| *** markvoelker has quit IRC | 07:11 | |
| openstackgerrit | Darren Chan proposed openstack/security-doc: Renamed Future section and added domain information https://review.openstack.org/199393 | 07:11 |
| *** elo1 has quit IRC | 07:17 | |
| *** elo has joined #openstack-security | 07:22 | |
| *** browne has quit IRC | 07:35 | |
| *** alex_klimov has joined #openstack-security | 07:38 | |
| *** salv-orlando has joined #openstack-security | 07:50 | |
| *** sdake has quit IRC | 07:57 | |
| *** maikol1 has joined #openstack-security | 08:06 | |
| maikol1 | hola quien para | 08:07 |
| maikol1 | hablar | 08:07 |
| *** shohel has joined #openstack-security | 08:09 | |
| Daviey | rmarathu: if you install bandit using pip, it should install stevedore aswell. | 08:11 |
| maikol1 | NADIA PARA HABLAR | 08:11 |
| *** maikol1 has left #openstack-security | 08:12 | |
| *** shohel has quit IRC | 08:18 | |
| *** hyakuhei has joined #openstack-security | 08:33 | |
| *** y_sawai has joined #openstack-security | 08:44 | |
| *** markvoelker has joined #openstack-security | 09:07 | |
| *** markvoelker has quit IRC | 09:11 | |
| *** tkelsey has joined #openstack-security | 09:13 | |
| *** dg_ has joined #openstack-security | 09:15 | |
| *** y_sawai has quit IRC | 09:29 | |
| *** shohel has joined #openstack-security | 09:36 | |
| *** dg_ has quit IRC | 09:53 | |
| *** lexholden has joined #openstack-security | 10:03 | |
| *** alex_klimov has quit IRC | 10:29 | |
| *** salv-orl_ has joined #openstack-security | 10:34 | |
| *** salv-orlando has quit IRC | 10:37 | |
| *** salv-orl_ has quit IRC | 10:47 | |
| *** salv-orlando has joined #openstack-security | 11:00 | |
| *** salv-orl_ has joined #openstack-security | 11:01 | |
| *** salv-orlando has quit IRC | 11:05 | |
| *** alex_klimov has joined #openstack-security | 11:06 | |
| *** rmarathu has quit IRC | 11:07 | |
| *** markvoelker has joined #openstack-security | 11:08 | |
| *** markvoelker has quit IRC | 11:13 | |
| *** hyakuhei has quit IRC | 11:37 | |
| *** hyakuhei has joined #openstack-security | 11:38 | |
| *** hyakuhei has quit IRC | 11:38 | |
| *** dave-mcc_ has quit IRC | 11:49 | |
| *** shohel1 has joined #openstack-security | 11:58 | |
| *** rmarathu has joined #openstack-security | 11:58 | |
| *** shohel has quit IRC | 12:00 | |
| *** markvoelker has joined #openstack-security | 12:09 | |
| *** shohel1 has quit IRC | 12:11 | |
| *** markvoelker has quit IRC | 12:14 | |
| *** markvoelker has joined #openstack-security | 12:29 | |
| *** bknudson has joined #openstack-security | 12:30 | |
| *** shohel has joined #openstack-security | 12:32 | |
| *** edmondsw has joined #openstack-security | 12:33 | |
| *** jmckind has joined #openstack-security | 12:36 | |
| *** dave-mccowan has joined #openstack-security | 12:44 | |
| *** tmcpeak has joined #openstack-security | 12:47 | |
| *** y_sawai has joined #openstack-security | 12:50 | |
| *** y_sawai has quit IRC | 12:54 | |
| *** bknudson has quit IRC | 12:59 | |
| *** lexholden has quit IRC | 13:02 | |
| *** jmckind has quit IRC | 13:11 | |
| *** edmondsw has quit IRC | 13:14 | |
| *** lexholden has joined #openstack-security | 13:17 | |
| *** browne has joined #openstack-security | 13:19 | |
| *** bknudson has joined #openstack-security | 13:20 | |
| rmarathu | can anybody help me to understand profiles, plugins in bandit.yaml file here - https://github.com/openstack/keystone/blob/master/bandit.yaml please? | 13:21 |
| tkelsey | hello rmarathu whats up? | 13:22 |
| tkelsey | so the profile is given a name and the list under "include" is a list of tests to run. When you use bandit with the -p option and the profile name, it will only run the tests that are listed | 13:23 |
| tkelsey | so in that example, if running with "-p keystone_conservative" bandit would only run.. | 13:23 |
| tkelsey | blacklist_calls - blacklist_imports - request_with_no_cert_validation - exec_used - set_bad_file_permissions - subprocess_popen_with_shell_equals_true - linux_commands_wildcard_injection - ssl_with_bad_version | 13:23 |
| tkelsey | and nothing else | 13:23 |
| tkelsey | the names like "set_bad_file_permissions" are the plugin test names | 13:25 |
| tkelsey | I have started documenting all of them here: https://review.openstack.org/#/c/205505/4/docs/source/tests/index.rst | 13:26 |
| rmarathu | tkelsey: thank you for the immediate response | 13:27 |
| tkelsey | no problem rmarathu :) | 13:27 |
| rmarathu | i have few questions, let me put all together and post...give me few min | 13:28 |
| rmarathu | :) | 13:28 |
| tkelsey | ok sure | 13:28 |
| *** singlethink has joined #openstack-security | 13:31 | |
| *** edmondsw has joined #openstack-security | 13:51 | |
| *** rmarathu_ has joined #openstack-security | 13:53 | |
| *** rmarathu has quit IRC | 13:55 | |
| *** rmarathu_ is now known as rmarathu | 13:55 | |
| *** sigmavirus24_awa has joined #openstack-security | 13:57 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:57 | |
| *** maikol1 has joined #openstack-security | 13:59 | |
| maikol1 | hola | 14:00 |
| *** shohel has quit IRC | 14:00 | |
| maikol1 | quien | 14:00 |
| maikol1 | para hablar | 14:00 |
| *** shohel has joined #openstack-security | 14:00 | |
| *** viraptor has quit IRC | 14:04 | |
| *** jmckind has joined #openstack-security | 14:14 | |
| *** maikol1 has left #openstack-security | 14:16 | |
| *** alex_klimov has quit IRC | 14:22 | |
| *** alex_klimov has joined #openstack-security | 14:23 | |
| rmarathu | tkelsey: these are the questions I have, please help | 14:24 |
| rmarathu | 1. keystone_conservative and keystone_verbose are the profiles? and those names are user defined and can be given any name? | 14:24 |
| rmarathu | 2. include - list mentions the name of plugins? the bandit help lists the plugins and these are the from that list right?(ex: blacklist_calls,blacklist_imports etc) | 14:24 |
| rmarathu | 3. under those plugins, there is mention of some data like this - | 14:24 |
| rmarathu | blacklist_calls: | 14:24 |
| rmarathu | bad_name_sets: | 14:24 |
| rmarathu | - pickle: | 14:24 |
| rmarathu | qualnames: [pickle.loads, pickle.load, pickle.Unpickler, | 14:24 |
| rmarathu | cPickle.loads, cPickle.load, cPickle.Unpickler] | 14:24 |
| rmarathu | message: "Pickle library appears to be in use, possible security issue." | 14:24 |
| rmarathu | - marshal: | 14:24 |
| rmarathu | qualnames: [marshal.load, marshal.loads] | 14:25 |
| rmarathu | message: "Deserialization with the marshal module is possibly dangerous." | 14:25 |
| rmarathu | - md5: | 14:25 |
| rmarathu | qualnames: [hashlib.md5] | 14:25 |
| rmarathu | message: "Use of insecure MD5 hash function." | 14:25 |
| rmarathu | - mktemp_q: | 14:25 |
| rmarathu | ----------- | 14:25 |
| rmarathu | blacklist_imports: | 14:25 |
| rmarathu | bad_import_sets: | 14:25 |
| rmarathu | - telnet: | 14:25 |
| rmarathu | imports: [telnetlib] | 14:25 |
| rmarathu | level: ERROR | 14:25 |
| rmarathu | message: "Telnet is considered insecure. Use SSH or some other encrypted protocol." | 14:25 |
| tmcpeak | ramarathu: 1) yes | 14:25 |
| rmarathu | what goes they specify? | 14:25 |
| tmcpeak | 2) yes | 14:26 |
| elmiko | rmarathu: for future reference, please use something like http://paste.openstack.org/ for code listings =) | 14:26 |
| tkelsey | rmarathu: ok that may have been better in a pastebin :) | 14:26 |
| elmiko | jinx ;) | 14:26 |
| tkelsey | elmiko: heh yeah :) | 14:26 |
| tkelsey | rmarathu: so lets see now | 14:26 |
| tmcpeak | 3) they specify plugin specific parameters, in the case of blaclist_calls we have qualnames which lists the qualified name of the function we're testing, ie libname.function_name, and the messasge it should display if its found | 14:27 |
| tkelsey | for question 1, yes, thats right. They can be anything | 14:27 |
| tkelsey | for question 2) yes, thats right | 14:27 |
| tmcpeak | rmarathu: tkelsey is working on plugin specific documentation as we speak that should make this more clear | 14:27 |
| tkelsey | question 3, that is plugin specific configuration data, some stuff got documented in this patch: https://review.openstack.org/#/c/204136/ | 14:28 |
| tkelsey | but it is being reworked, there may still be good info for you though | 14:28 |
| tkelsey | rmarathu: there will be full data on all this soon as a few patches I have in the works land and I can work on it again :) | 14:29 |
| tkelsey | for now, ask here or read the source if its not covered by that patch I linked | 14:30 |
| *** shohel has quit IRC | 14:32 | |
| tmcpeak | Daviey, sigmavirus24, browne | 14:33 |
| tmcpeak | ping | 14:33 |
| Daviey | here | 14:33 |
| sigmavirus24 | pong | 14:33 |
| browne | hi | 14:34 |
| tmcpeak | have you guys seen the XML thing? | 14:34 |
| tkelsey | hey guys | 14:34 |
| sigmavirus24 | tmcpeak: the patch? yes | 14:34 |
| sigmavirus24 | I'm on the fence about it | 14:34 |
| tkelsey | so, I cant really continue work on the docs effort until the fate of that patch is decided | 14:34 |
| tmcpeak | so I guess more generally we should decide the future of blacklist calls | 14:34 |
| tkelsey | since it will have a major impact on the names of plugins etc | 14:34 |
| Daviey | tmcpeak: Yeah, it was me that complained about the code -> config :) | 14:34 |
| tmcpeak | and then apply our decision universally | 14:34 |
| tmcpeak | Daviey: right | 14:35 |
| tkelsey | to blacklist or not, that is the question | 14:35 |
| tmcpeak | so as most active members I'm hoping we can come to some agreement, and reviews are not.. interactive enough | 14:35 |
| *** shohel has joined #openstack-security | 14:35 | |
| tmcpeak | I've also sent an email to Fletcher asking him to come | 14:35 |
| tmcpeak | but it's 7:30 over there so he probably hasn't gone to bed yet :P | 14:35 |
| tkelsey | I have a 30 min slot now, then im out for the day | 14:36 |
| tmcpeak | ok for starters, does anybody think it makes sense to have mix-and-match | 14:36 |
| tkelsey | but lets try and chat it over now | 14:36 |
| tkelsey | no | 14:36 |
| tmcpeak | as in some checks go in blacklist and some are separate functions? | 14:36 |
| * Daviey is in a conference call... but not one that i am terribly involved with. | 14:37 | |
| tkelsey | no, its just confusing for users and confusing for contributors | 14:37 |
| tkelsey | we should have a clear choice, one way or the other | 14:37 |
| tmcpeak | haha Daviey | 14:37 |
| tmcpeak | right, IMO mix and match is arbitrary | 14:37 |
| rmarathu | tkelsey: ok I will look through the patch and ask questions | 14:37 |
| tkelsey | rmarathu: ok cool | 14:37 |
| tmcpeak | one additional fringe option is we could list unsafe XML functions in a separate unsafe XML config, but that seems to be going backwards rather than forwards | 14:38 |
| browne | i actually would rather have plugins rework so that a exploiter of bandit just specified checks, not plugin names. similar to flake8 | 14:38 |
| Daviey | What do you chaps think about having a layered config? Ie, default values that can be overridden by user config? | 14:38 |
| browne | and move the profiles out of bandit.yaml | 14:38 |
| Daviey | That would perhaps solve the issue of configuration ability, but also keeping sane defaults and simple main conifg. | 14:38 |
| tkelsey | Daviey: +10 yes | 14:38 |
| tmcpeak | ok two simultaneous ideas at once | 14:38 |
| tmcpeak | simultaneous=at once | 14:39 |
| tmcpeak | sorry, early :P | 14:39 |
| tmcpeak | anyway | 14:39 |
| Daviey | I also agree with browne.. | 14:39 |
| tmcpeak | oh ok | 14:39 |
| tkelsey | lets stick to one thing, though I have that config on my TODO list Daviey | 14:39 |
| tmcpeak | so same idea | 14:39 |
| Daviey | I am not sure either of those ideas contrasts tho | 14:39 |
| tmcpeak | browne: what's the difference between specifying checks and plugin names? | 14:40 |
| Daviey | a plugin can support more than 1 test.. Is a check another name for a test? | 14:40 |
| sigmavirus24 | Daviey: actually they way we're registering plugins 1 plugin name should be 1 check. A check can probably have multiple tests | 14:41 |
| browne | tmcpeak: well, pretty much the same, but would like it available on the command line. bandit -x S101, S102 … where S101 was a well defined code for a check, rather than a plugin name | 14:41 |
| sigmavirus24 | i.e. something that checks if a function is called, if it has x, y, z keyword arguments passed to it | 14:41 |
| sigmavirus24 | browne: that would be plausible, certainly | 14:41 |
| tmcpeak | browne: oh ok, gotcha | 14:41 |
| Daviey | sigmavirus24: Maybe i am wrong, but I thought a few plugins had multiple functions? But bubbled up as one test? | 14:41 |
| tmcpeak | so some way or organizing better | 14:41 |
| sigmavirus24 | Would need to rework some stuff though | 14:41 |
| * Daviey groks code | 14:42 | |
| tmcpeak | yeah, that's a fairly big push | 14:42 |
| sigmavirus24 | Daviey: perhaps. I'm not intimately certain of all of them | 14:42 |
| sigmavirus24 | s/certain/familiar/ | 14:42 |
| tmcpeak | I agree though, as Bandit gets bigger that's the best way to keep organization | 14:42 |
| tmcpeak | so injections are in the 100 range, etc | 14:42 |
| tmcpeak | unsafe functions 200 | 14:42 |
| tmcpeak | like that? | 14:42 |
| Daviey | funnily enough, https://github.com/openstack/bandit/blob/master/bandit/plugins/xml.py | 14:42 |
| tkelsey | OK, are we not getting off the point now? we can name/map/whatever stuff for sure. But the question is many distinct tests, vs a few configurable ones | 14:43 |
| sigmavirus24 | tmcpeak: we can figure those out later | 14:43 |
| sigmavirus24 | == tkelsey | 14:43 |
| browne | sure, kinda separate item | 14:43 |
| tmcpeak | right | 14:43 |
| tmcpeak | so onto this | 14:43 |
| tmcpeak | tests defined in config vs tests defined in tests | 14:44 |
| sigmavirus24 | (Daviey: those are individual plugins as far as our plugin system is concerned: https://github.com/openstack/bandit/blob/master/setup.cfg#L97..L114) | 14:44 |
| Daviey | I *think* we are all agreed this doesn't belong in the primary bandit.yaml? | 14:44 |
| sigmavirus24 | Daviey: no, I don't think we *all* agreed on that | 14:44 |
| Daviey | sigmavirus24: right, yes - that is true. | 14:44 |
| tkelsey | Daviey: whats that? | 14:44 |
| sigmavirus24 | tkelsey: the xml config bits for blacklist calls | 14:44 |
| *** jmckind has quit IRC | 14:45 | |
| Daviey | tkelsey: Growing the bandit.yaml by 68 lines for a no-op is pretty distasteful IMO. | 14:45 |
| sigmavirus24 | So, I'm personally in agreement that as it currently is, the xml module is reimplementing blacklist calls and that it makes sense for those to be included as blacklist calls in the config | 14:45 |
| tkelsey | right, I actually have no problem with the list being in the config. But I am OK with putting it some place else, so long as that place is not in a bunch of functions :P | 14:45 |
| tmcpeak | the nice part about blacklist functions was that it recognizes a bunch of tests all do the *exact* same thing and just moved it to YAML definitions | 14:46 |
| tkelsey | Daviey: maybe this highlights a problem with the config then? rather than the approach | 14:46 |
| Daviey | tkelsey: I'm wondering if there should be a yaml config in the library, then a more sparse one that inherits the library config. | 14:46 |
| Daviey | That would satisfy my main concern | 14:46 |
| sigmavirus24 | Daviey: yeah that's what we were talking about previously | 14:47 |
| browne | the config is kinda problematic because each time it changes, then a project using it needs to change their copy | 14:47 |
| sigmavirus24 | Kind of like having a bunch of ansible roles with their own variables that you can override from a global variables file, yes? | 14:47 |
| tmcpeak | browne: yeah, agreed | 14:47 |
| Daviey | sigmavirus24: right! | 14:47 |
| *** jmckind has joined #openstack-security | 14:47 | |
| sigmavirus24 | browne: yeah that's kind of the main problem with the current patch, right? | 14:47 |
| tmcpeak | sigmavirus24, Daviey: yeah I like that | 14:47 |
| sigmavirus24 | It will hurt anyone who's turned off the xml checks | 14:47 |
| tkelsey | a "hidden" config stashed some place in the lib might be surprising to people dont you think? | 14:48 |
| tkelsey | im not adverse to it though | 14:48 |
| sigmavirus24 | That said, are people turning that off in the projects we know are using bandit? | 14:48 |
| tmcpeak | tkelsey: yeah, I think it needs to be explicit: [OVER-RIDING CONFIG FROM xyz.yaml] | 14:48 |
| sigmavirus24 | tkelsey: yes, and I'm not sure it has to be hidden | 14:48 |
| browne | sigmavirus24: yep, i leaning towards having a solution such that projects don't have to have a bandit.yaml | 14:48 |
| Daviey | tkelsey: we currently have a hidden config :) | 14:48 |
| tmcpeak | WE DO?! awesome :) | 14:48 |
| tkelsey | Daviey: for sure, but since we are talking about improvements ... :0 | 14:49 |
| tkelsey | :) | 14:49 |
| sigmavirus24 | browne: right and if we start moving towards flake8, we don't need the yaml bits that we currently have if we don't want them =P | 14:49 |
| sigmavirus24 | (i.e., one less dependency :^D) | 14:49 |
| tmcpeak | ok so seems like most of us like this idea, is anybody not sold? | 14:49 |
| tkelsey | sigmavirus24: could you put a spec/doc/words together about how this might work? | 14:49 |
| tkelsey | its an interesting idea | 14:49 |
| Daviey | Actually... the child config could declare what it inherits from | 14:49 |
| Daviey | similar to how we handle the word-list now | 14:50 |
| sigmavirus24 | tkelsey: we should be more explicit about "this", we aren't writing javascript ... yet | 14:50 |
| sigmavirus24 | Daviey: like jinja templates? =P | 14:50 |
| Daviey | INVENT ALL THE THINGS | 14:50 |
| tkelsey | so, I have 9mins here lol | 14:51 |
| tmcpeak | yeah.. tkelsey still isn't unblocked | 14:51 |
| sigmavirus24 | tkelsey: send an email to the list about the idea and I'll see if I can blueprint/spec something out for us | 14:51 |
| tkelsey | lets agree that the config sucks, we need a new one, how about specific vs configured ? | 14:51 |
| tmcpeak | let's get a path forward so tkelsey can finish docs while he has momentum | 14:51 |
| sigmavirus24 | I'm in favor of the patch although I'm concerned about the backwards compatibility problems. That said, semver is what we follow and we've not promised stability (within semver) until 1.0 | 14:51 |
| Daviey | WHat is the blocking question ? | 14:52 |
| tmcpeak | the blocking question is how should the XML tests be documented | 14:52 |
| tmcpeak | and for that matter, the blacklist calls | 14:52 |
| tkelsey | no, its how should this be done going forward | 14:52 |
| tkelsey | generic configured tests or specific names functions | 14:52 |
| tkelsey | *named | 14:52 |
| tmcpeak | tkelsey: well what's actually holding up your doc effort? | 14:53 |
| Daviey | So it is more how we need to style the config. | 14:53 |
| tmcpeak | wanting to avoid the rework I suppose? | 14:53 |
| tkelsey | I cant document stuff thats going to be ripped out/changed | 14:53 |
| tmcpeak | yeah, fair enough | 14:53 |
| Daviey | Well you can.... Having a basis of current reality isn't so bad | 14:53 |
| tmcpeak | well, sadly it seems like we're about to make a big push one way or the other - and docs are going to be stale soon anyway | 14:54 |
| Daviey | As in, it would be smaller to change to the new world than writing from scratch, right? | 14:54 |
| tkelsey | well maybe its, I wont rather than cant. But same difference unless someone wants to step up and do it :P | 14:54 |
| tmcpeak | Daviey: it's a huge doc effort, would suck to rework | 14:54 |
| Daviey | Ok, i think i underestimated how much doc this would require | 14:54 |
| tmcpeak | tkelsey: how about this - you push the doc structure, leave document the functionality of our tests, and then if we have to do some copy paste later so be it | 14:55 |
| tmcpeak | ^ that without the word "leave" | 14:55 |
| *** sdake has joined #openstack-security | 14:55 | |
| tkelsey | that isnt answering the question. Thats just leaving it haging | 14:55 |
| tkelsey | *hanging | 14:55 |
| Daviey | Would it be a really bad time to suggest we consider switching to oslo.config? | 14:56 |
| tkelsey | I also want to say "this is how you do X" and I cant unless we have a clear choice | 14:56 |
| tkelsey | Daviey: yes, stick to the point please | 14:56 |
| tmcpeak | tkelsey: well I guess the other choice is just to leave the docs unfinished for now since we don't have a resolution | 14:56 |
| tkelsey | well with people like rmarathu asking for them I dont think thats a good idea | 14:57 |
| tmcpeak | we could pick a time, say tomorrow at 9 PST to meet in here and then come to a conclusion one way or another | 14:57 |
| tkelsey | im not sure how we can expect people to use bandit more whole sale without docs | 14:57 |
| tmcpeak | tkelsey: what do you suggest? | 14:57 |
| Daviey | I really think the current implementation is OK with inheritance of config support. | 14:58 |
| Daviey | That is my only blocker with the branch as is. | 14:58 |
| tkelsey | we come to a decision on what we do vs named funcs or configured tests. I write the docs accordingly. We worry about fixing our config in a nice way after | 14:58 |
| tmcpeak | ok well config might be another big doc change | 14:58 |
| tkelsey | not as big as the plugins | 14:59 |
| Daviey | So other than the big config file, what other snags do we have with the branch? | 14:59 |
| tkelsey | ok im in a meeting now, sorry, out fo time | 14:59 |
| tmcpeak | yeah, moving out blacklist_calls, blacklist_imports into separate/overridden configs works for me | 14:59 |
| tmcpeak | allright tkelsey, thanks | 15:00 |
| *** fletcher_ has joined #openstack-security | 15:00 | |
| tmcpeak | fletcher_ just in time :) | 15:00 |
| Daviey | tmcpeak: did you grasp any other issues? | 15:00 |
| tmcpeak | Daviey: I'm honestly fine either way as long as we can be consistent | 15:00 |
| rmarathu | Having good documentation is better choice, that helps users like me to try things and give back something back to community in the form of feedback or bugs :) | 15:00 |
| tmcpeak | rmarathu: agreed :) | 15:01 |
| fletcher_ | Helllllo, all :) | 15:01 |
| Daviey | rmarathu++ | 15:01 |
| rmarathu | :) | 15:01 |
| tmcpeak | fletcher_: we're trying to figure out where we want to put "blacklist calls" going forward | 15:01 |
| fletcher_ | So i was going to craft a response to Tim's message this morning | 15:02 |
| sigmavirus24 | Opinion Daviey will hate me for: Those xml tests belong in blacklist_calls and that's where they should be. The rest of the config topics being discussed are tangential and not helpful at the moment. I'm +2 on the review as it is | 15:02 |
| tmcpeak | I think we can all agree that your XML checks are basically blacklist calls and imports, and it would be better to be consistent. Either call them as such, or move all the existing blacklist calls and imports into functions | 15:02 |
| Daviey | I can't see any blocker with the current blacklist_calls implementation on the branch | 15:02 |
| rmarathu | apart from keystone yaml file do we have yaml configuration files for other components like nova, cinder, glance,neutron etc? | 15:02 |
| fletcher_ | but the tl;dr is, managing a config is harder than post processing, and were still left with the issue of not having any context on these calls | 15:02 |
| rmarathu | please share the link if they are already existing for openstack gate tests | 15:02 |
| fletcher_ | we claim these are 'medium' severity and 'high' confidence - we can't say high confidence at all when we flag things like urllib.open("https://www.google.com") | 15:03 |
| tmcpeak | rmarathu: yeah, although Keystone is the best example and what we point people to, most just copied it | 15:03 |
| fletcher_ | If the goal is Bandit to be a Static Analyzer, not just an advanced grep, then it seems we should take advantage of the context and check for things like that | 15:03 |
| Daviey | sigmavirus24: No, i don't hate that.. I'm saying that i *am* ok with it, providing the user config complexity remains no worse. | 15:03 |
| Daviey | which is why the side conversation of configuration IS relevant | 15:03 |
| sigmavirus24 | Daviey: I didn't say they weren't relevant | 15:04 |
| sigmavirus24 | I said they weren't helpful | 15:04 |
| Daviey | sigmavirus24: Well, that i disagree with | 15:04 |
| fletcher_ | tmcpeak: so in my opinion, blacklist_calls itself should be deprecated. It's not doing Static Analyzer things, leads to a lot of false positives, and makes management more difficult | 15:04 |
| tmcpeak | fletcher_: agreed, some plugins will want to really dig in to AST | 15:04 |
| sigmavirus24 | They're side-tracking the discussion from what's important to discuss so we can help unblock tkelsey | 15:04 |
| rmarathu | tmcspack: thank you | 15:04 |
| tmcpeak | fletcher_: well blacklist_calls makes it easy for people who know they have unsafe functions to track the use of those functions without having to be a Bandit dev | 15:05 |
| fletcher_ | for example, if I maintain my own bandit.yaml config and we decide to add a new function to the blacklist_calls list (say XML things), then I'm left to either do some weird merging, or miss out on the new check | 15:05 |
| tmcpeak | fletcher_: yeah, I think we're all agreeing that is an issue. Daviey/sigmavirus24 are suggesting that we allow over-ridden configs | 15:06 |
| tmcpeak | so basically bandit.yaml is base, and on top of that you can add keystone.yaml | 15:06 |
| tmcpeak | Daviey, sigmavirus24: is that a fair description? | 15:06 |
| Daviey | sigmavirus24: We haven't really distilled what is the blocking part for tkelsey. As far as i understand the only thing standing in the way is config file complexity? | 15:06 |
| Daviey | fletcher_ brings up the issue of blacklist being dropped.. but that doesn't block this branch, does it? | 15:07 |
| fletcher_ | In my opinion, making it easier for people to add functions to a blacklist versus generating a ton of un-context aware warnings with inaccurate confiences seems to be the wrong approach | 15:07 |
| fletcher_ | I agree it makes it easier, but at the cost of the tool's accuracy and usability | 15:07 |
| sigmavirus24 | fletcher_: so at the moment no one is writing plugins for bandit except for us | 15:07 |
| fletcher_ | I've had this opinion for a while and was going to bring it up at the meeting or whatever, but we just happen to get into it :) | 15:07 |
| sigmavirus24 | That said, the xml plugin *currently* is acting exactly like blacklist_calls | 15:08 |
| fletcher_ | totally agree with that | 15:08 |
| sigmavirus24 | But it's reimplementing that logic for each of its functions | 15:08 |
| *** bpokorny has joined #openstack-security | 15:08 | |
| sigmavirus24 | So while blacklist_calls is probably not ideal for most of the checks we currently have there, those xml functions/checks/whatever you want to call them are duplicating logic and acting the same way | 15:08 |
| fletcher_ | my point is, we should be moving aware from blacklist_calls. While the first effort at XML is essentially the same functionality, it's the basis to add things like context aware checking | 15:08 |
| Daviey | sigmavirus24: I am working on an external deployment specific plugin right now. | 15:08 |
| fletcher_ | s/aware/away | 15:09 |
| sigmavirus24 | fletcher_: the thing with blacklist_calls is that it gives us flexibility to develop those xml-specific plugins without breaking people | 15:09 |
| sigmavirus24 | So while right now it's a hammer | 15:09 |
| sigmavirus24 | It's a hammer that people can use until we have a chisel | 15:09 |
| fletcher_ | So is the claim that a lot of people add things to the blacklist_calls that we currently aren't checking for? | 15:09 |
| fletcher_ | And those people are incapable of writing proper plug-ins? | 15:09 |
| fletcher_ | I'm just trying to understand. blacklist_calls seems, to me, to be the obviously wrong approach if Bandit is a Static Analyzer | 15:10 |
| tmcpeak | providing people with at least an option to easily define functions that they want to detect the usage of is useful | 15:10 |
| *** bpokorny has quit IRC | 15:10 | |
| sigmavirus24 | fletcher_: so consider blacklist_calls the entry drug | 15:10 |
| tmcpeak | Bandit can be used as a static analyzer, but really it's just a tool to solve problems, and one such problem is "audit the usage of x, y, and z" | 15:10 |
| sigmavirus24 | People add stuff to it and realize it isn't powerful enough | 15:11 |
| tmcpeak | in fact we're going to use that very capability for our crypto audit | 15:11 |
| sigmavirus24 | Then they write their own plugin | 15:11 |
| Daviey | fletcher_: Where are you saying that blacklist functionality belongs? | 15:11 |
| sigmavirus24 | Daviey: I think fletcher_ wants it dead altogether | 15:11 |
| sigmavirus24 | It isn't dynamic enough | 15:11 |
| fletcher_ | I'm saying that things like XML, urllib, md5, pickle and the other blacklist calls are inaccurate and resultin fatigue | 15:11 |
| Daviey | So.. if i wanted to opt-out of a test, is it by config? | 15:11 |
| fletcher_ | i can agree having blacklist_calls for random functions people want to add, would be useful | 15:11 |
| sigmavirus24 | So the thing is you're saying that `urllib.open('https://google.com/')` should or shouldn't be flagged? | 15:12 |
| fletcher_ | but moving a bunch of stadnard calls to it seems counter to how we could use the AST and really make Bandit rock | 15:12 |
| tmcpeak | fletcher_: well what additional analysis are you going to do on a md5 call to know if it's an issue? | 15:12 |
| sigmavirus24 | tmcpeak: I was wondering the same about urllib.open and the xml functions | 15:12 |
| fletcher_ | fair enough, maybe md5 was a bad example. I'd still imagine a statc string, versus a variable named '.*pass.*' might warrant a higher confidence | 15:13 |
| tmcpeak | right, it seems in some cases just detecting usage of potentially unsafe functions is the best you can do and adds some value | 15:13 |
| fletcher_ | sigmavirus24: shouldn't | 15:13 |
| fletcher_ | sometimes, but a lot of these could | 15:13 |
| fletcher_ | and, imo, we shouldn't be adding functions to the list that could | 15:14 |
| tmcpeak | fletcher_: so it seems that some functions could definitely take advantage of additional detection like that, but in some cases you can only detect a usage which does in itself provide some value to draw attention for pentesting etc | 15:14 |
| fletcher_ | (i'm not being short, if it reads I am; just a million windows open at once) | 15:14 |
| tmcpeak | :) | 15:14 |
| sigmavirus24 | fletcher_: why wouldn't urllib.open('https://something.com') be a problem? | 15:14 |
| fletcher_ | they clami is that we should audit protocol on that | 15:15 |
| tmcpeak | so personally I use Bandit for two completely different flows - gate test, I want to make sure devs aren't checking in crap and pentesting, where I can accept false positives as they are easy to rule out and sometimes draw my attention to cool issues | 15:15 |
| fletcher_ | if it's a static string, then I would flag that has confidence low, versus if it was a variable | 15:15 |
| fletcher_ | e.g. urllib.open(requiest.iv.get('url')) | 15:15 |
| tmcpeak | that's why configuration is good, there's a pentest config and a gate config | 15:15 |
| sigmavirus24 | fletcher_: ah, but you're missing the point where (except for very very very very recent versions of urllib(2)) it doesn't do HTTPS verification and that in and of itself is really bad, and we're all very confident about that | 15:16 |
| fletcher_ | imo, Bandit should be at it's core a Python security static analyzer | 15:16 |
| fletcher_ | so maybe its a conflict of philosophy | 15:16 |
| sigmavirus24 | or maybe that particular example is a really bad one given the terrible-ness that is urllib.open | 15:17 |
| sigmavirus24 | ;) | 15:17 |
| sigmavirus24 | (disclaimer: I'm a core developer of requests) | 15:17 |
| tmcpeak | we're kind of down in the weeds a bit | 15:17 |
| fletcher_ | "Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected." | 15:17 |
| fletcher_ | that is the message, unrelating to HTTPS at all | 15:17 |
| tmcpeak | removing the ability to check function calls to make Bandit fit in more with the traditional static analysis concept doesn't make a lot of sense | 15:18 |
| sigmavirus24 | fletcher_: so the message is bad, not the check | 15:18 |
| fletcher_ | tmcpeak: agreed! I was thinking an in person convo would be better :) | 15:18 |
| fletcher_ | No, i disagree with that as well | 15:18 |
| tmcpeak | yeah, sadly IRC is probs the best we can do currently :) | 15:18 |
| fletcher_ | in general, the blacklist_calls checks are unaware and report confidences and severities that are inaccurate | 15:18 |
| tmcpeak | fletcher_: well we're also getting ready to use Bandit to audit all the places in the code where crypto is used | 15:19 |
| tmcpeak | we can't do that without configurable blacklist calls | 15:19 |
| tmcpeak | in pentesting I've also found a function is insecure, added it to blacklist calls, and used that to trace through the end-to-end exploit | 15:19 |
| fletcher_ | so I also totally agree we should provide that functionality, the ability to add a simple list of function calls. I'm saying we shouldn't continuously add functions (like XML and pickle) checks that are unaware to the list | 15:19 |
| fletcher_ | it should be a very minimal (ideally empty) list that you can add to | 15:20 |
| fletcher_ | imo | 15:20 |
| tmcpeak | fletcher_: ok I see some merit in that | 15:20 |
| tmcpeak | fletcher_: so let's talk about the XML calls | 15:20 |
| tmcpeak | what are we going to add to those, or are they finished? | 15:20 |
| tmcpeak | if they're finished, I'm in favor of moving all of the calls into a config file that is separate from the main bandit.yaml | 15:21 |
| fletcher_ | the current diff that was suggested was to delete them from plugins and move them to blcaklist_calls, so they're already done | 15:21 |
| tmcpeak | fletcher_: were you planning to add more? | 15:21 |
| *** dwyde has joined #openstack-security | 15:21 | |
| tmcpeak | contextual awareness, etc? | 15:21 |
| fletcher_ | all the XML checks exist now, I had planned to add some of that yes | 15:21 |
| tmcpeak | fletcher_: ok cool, that in that case they should certainly live in code, as blacklist calls will not support adding additional contextual awareness | 15:22 |
| fletcher_ | tbh, I don't think that should be the blocker on the diff if everyone besides me thinks blacklist_calls is the right choice | 15:22 |
| fletcher_ | I think we're kind of at a fork in the road on deciding what blacklist_calls and what it should be going forward | 15:22 |
| fletcher_ | sounds dramatic, lol | 15:23 |
| tmcpeak | well I'm ok with that being the line we draw: "is it a simple function qualname check? if so —> blacklist calls", "are we checking additional factors or will we in the future? —> separate functions" | 15:23 |
| sigmavirus24 | tmcpeak: so I would say, "Will we in the future?" is a bad question | 15:23 |
| fletcher_ | Wouldn't it be better to have them all in plug-ins? If people want to contribute, it becomes much easier than ripping things out and starting completely fresh | 15:23 |
| Daviey | I suppose it is ok for a plugin to begin life as a blacklist call then migrate over to a separate plugin. | 15:24 |
| sigmavirus24 | Because saying "Okay let's duplicate the blacklist calls functionality until someone sometime in the ambiguous future adds to this" | 15:24 |
| tmcpeak | Daviey: I agree, it's a simple move | 15:24 |
| sigmavirus24 | == Daviey | 15:24 |
| tmcpeak | sigmavirus24: sure, that makes sense | 15:24 |
| fletcher_ | fair enough, but that seems like it just means "continue to add to blacklist_calls" | 15:24 |
| fletcher_ | Maybe we start requiring plug-ins to be more context aware | 15:24 |
| tmcpeak | fletcher_: I don't want to go down that road, contextual aware isn't proven and I want to keep a low barrier to entry for newer users | 15:25 |
| *** bpokorny has joined #openstack-security | 15:25 | |
| sigmavirus24 | also, a reminder, a plugin can begin its life as a third-party thing | 15:25 |
| sigmavirus24 | and that can be as contextually aware or not | 15:25 |
| tmcpeak | so I'm ok with the approach that "if blacklist calls/blacklist imports fills your need, it belongs there, otherwise separate function" | 15:26 |
| sigmavirus24 | and based on adoption we can absorb it upstream | 15:26 |
| fletcher_ | tmcpeak: i can agree with that, but it seems like a that point you just write your own config for your own functions. landing unaware checks in the main tool in the main repo is destined to make the tool less accurate | 15:26 |
| tmcpeak | I do want blacklist calls, imports to be over-rideable like I believe Daviey was suggesting so bandit.yaml doesn't balloon and projects don't get stuck having to merge everytime Bandit changes | 15:27 |
| Daviey | right | 15:27 |
| tmcpeak | fletcher: yeah I think you can just write your own config files instead | 15:27 |
| tmcpeak | or own plugins works too | 15:27 |
| fletcher_ | Sure, but now the tool is inherently noisey | 15:27 |
| fletcher_ | with a bunch of checks that are inaccurate | 15:27 |
| tmcpeak | fletcher_: yeah, that's why we have profiles - if you don't like the results from a plugin, disable it | 15:28 |
| Daviey | fletcher_: Flipping things on their head slightly towards your concern, what if a plugin is a test which exposes attributes such as "tests_blacklist".. then the type of test isn't the identifier, but the attributes a test exposes is the identifier. Not saying it is worth the effort, but is that the direction you were thinking? | 15:28 |
| tmcpeak | Daviey: I didn't follow that | 15:28 |
| * sigmavirus24 either | 15:29 | |
| Daviey | Hmm, i need a whiteboard. | 15:29 |
| Daviey | Don't worry... carry on. | 15:29 |
| fletcher_ | tmcpeak: fair enough, i'm not overly familiar with profiles. that essentially leaves us in a situation of "oh, is this too inaccurate and nosiey for you, tweak it down and manage". It seems we should have the opposite approach "this tool will only report really relevant things, but you can add noisier checks easily if you'd like" | 15:29 |
| tmcpeak | fletcher_ true although adding in stuff seems harder than removing it in my mind | 15:30 |
| Daviey | fletcher_: That is a risky approach IMO, as people will start off with a small test suite and just make it even smaller. | 15:30 |
| tmcpeak | profiles actually support the ability to exclude: "oh I hate the x test" - boom: new profile, excluded | 15:31 |
| Daviey | Or are people turning up more knobs themselves with their own profiles ? | 15:31 |
| tmcpeak | nobody actually uses it currently, but it works | 15:31 |
| tmcpeak | Daviey: basically the profile the gates are using is based on Keystone, which we wrote and provided to them | 15:32 |
| fletcher_ | I think this comes back to this tool being tied to OpenStack and that workflow versus making it useful to the Security community as a whole. If I'm random developer X, I want to check my Python code with Bandit, I run it and get a bunch of useless messages, I'm way less likely to use it in the future | 15:32 |
| *** browne has quit IRC | 15:32 | |
| fletcher_ | i might be bikeshedding at this point :) | 15:33 |
| tmcpeak | fletcher_: yeah, although like I said I use it for pentesting as well | 15:33 |
| tmcpeak | security engineer doing code review and gate test are our two main use cases, and we want to make it equally useful for both. That is a major design goal | 15:34 |
| Daviey | tmcpeak: How do you filter 'too much noise' from useful stuff? | 15:34 |
| tmcpeak | Daviey: in my use I just leave it all on. False positives are trivial to spot, but sometimes they catch something interesting | 15:35 |
| *** sdake has quit IRC | 15:35 | |
| tmcpeak | I've actually seen the hardcoded password test find something good | 15:35 |
| tmcpeak | although it generates pure crap 95% + of the time | 15:35 |
| Daviey | tmcpeak: you just grep / paginate through it? | 15:35 |
| tmcpeak | yep | 15:35 |
| tmcpeak | paginate | 15:35 |
| fletcher_ | In my opinion, false positives and fatigue are some of the most crucial pain points for tools like this. Ultimately leading to developers ignoring any warnings returned by the tool | 15:36 |
| fletcher_ | if I'm running this on all commits to a repo, and it complains on almost everyone about things that aren't issues, it becomes problematic (not claiming Bandit is that noisey) | 15:36 |
| tmcpeak | fletcher_: that's why it's configurable though, if you think the x test sucks, create a profile that doesn't include it or removes it | 15:37 |
| fletcher_ | and yes, I could tweak this and do different things, but that makes the barrier to entry pretty high to have something useful | 15:37 |
| fletcher_ | whoops, simpatico :) | 15:37 |
| tmcpeak | what's crap for your use might be good for me though, or I might be willing to tolerate the false positives for the payout of getting that good issue | 15:37 |
| fletcher_ | getting good issues and reporting high false positibves aren't related, imo | 15:37 |
| fletcher_ | eliminating false positives, doesn't mean you will miss the good ones | 15:38 |
| tmcpeak | yeah, it pretty much does | 15:38 |
| fletcher_ | how so? | 15:38 |
| tmcpeak | in any system if you tune the detection too specifically than you will get false negatives | 15:38 |
| tmcpeak | that's kind of the age old tradeoff, isn't it? | 15:38 |
| sigmavirus24 | fletcher_: so, I would have to say that this the default argument (would almost go so far as to say strawman) against *EVERY* linter | 15:39 |
| fletcher_ | Sure, i agree with regards to 'too specifically'. But that's the balance we have to try and find | 15:39 |
| fletcher_ | right now we're just erring on the side of now confidence or specificity | 15:39 |
| fletcher_ | s/now/no | 15:39 |
| sigmavirus24 | Right | 15:39 |
| tmcpeak | sure so if we get better plugins, everything gets better | 15:39 |
| sigmavirus24 | Right | 15:39 |
| fletcher_ | agreed | 15:39 |
| tmcpeak | I think the question is, how do we want to go ahead with what we have | 15:39 |
| sigmavirus24 | Until we get those plugins, we should continue to use the hammer we have though | 15:40 |
| fletcher_ | but allowing people to add anything to blacklist_calls doesn't move us towards that | 15:40 |
| Daviey | So i guess we need to document what knobs are available to users? I mean, suggest starting off with just HIGH confidence issues.. and expand on to custom Profiles for needs? | 15:40 |
| tmcpeak | Daviey: sure, we can bundle some profiles too | 15:40 |
| tmcpeak | "SourceReview" and "Gate" | 15:40 |
| tmcpeak | as examples | 15:40 |
| sigmavirus24 | So, gertty is another tool that's built buy the openstack community that wants to not just target the openstack community | 15:40 |
| sigmavirus24 | They currently have a bunch of different example configs (openstack, plain gerrit, others) | 15:41 |
| sigmavirus24 | We could do something similar and have the keystone one be an example, etc. | 15:41 |
| fletcher_ | My vote: blacklist_calls should be a minimal list and we should work on getting most functions out of it (even if intially they are just like XML plug-in now, mimicing blacklist_call functionality). People should be able to easily extend this list for functionality. | 15:41 |
| sigmavirus24 | My experience makes me lean the opposite way | 15:42 |
| sigmavirus24 | People will not improve something unless it's really annoying to them | 15:42 |
| sigmavirus24 | People are more inclined to scratch their own itches | 15:42 |
| fletcher_ | people won't use it or pay attention to it if it's annoying | 15:42 |
| sigmavirus24 | I disagree | 15:42 |
| fletcher_ | that's my entire point | 15:42 |
| tmcpeak | so like I said, I'm a Bandit dev obviously, but I also use Bandit in both ways that I believe we care to support | 15:43 |
| tmcpeak | pentester and gate | 15:43 |
| tmcpeak | I'm not in favor of anything that makes either of those uses more difficult than it needs | 15:43 |
| tmcpeak | and it seems like bundling example profiles for both solves both uses | 15:43 |
| fletcher_ | I agree, we should support both of those | 15:43 |
| fletcher_ | I think this ignores the facts that the blacklist_calls are inherently inaccurate, and this doesn't do anything to solve that | 15:44 |
| tmcpeak | additionally let's keep blacklist_calls because it's useful, but move it to a separate config so bandit.yaml doesn't get huge and projects don't constantly have to merge their changes in with new bandit.yaml | 15:44 |
| tmcpeak | fletcher_: tolerably so for some uses though | 15:44 |
| Daviey | My first experience of Bandit was flood fatigue.. I had NFI how i'd even start to use it on real code that was't written from the start being tested against Bandit. | 15:44 |
| fletcher_ | We can probably curb this discussion until we meet up in seattle if you want tho, and obviously, I'm not a blocking vote anyways :) | 15:44 |
| tmcpeak | and to be clear, they aren't inaccurate, they're completely accurate. It just doesn't mean anything in some cases | 15:44 |
| fletcher_ | a confidence of HIGH is inaccurate when it doesn't mean anything | 15:45 |
| fletcher_ | by definition | 15:45 |
| fletcher_ | maybe that's just a matter of tweaking the way we report blacklist_calls though | 15:45 |
| tmcpeak | "we're 100% sure that you are calling this function" it's severity that doesn't mean anything | 15:45 |
| fletcher_ | Ah, you're right, I got those flopped | 15:46 |
| tmcpeak | ok so to summarize - here's where we are: | 15:46 |
| tmcpeak | 1) we're in debate about usefulness about blacklist_calls but we're going to leave it for now. Fletcher, I and anybody else that wants to will go out for beers one night in Seattle and figure it out :P | 15:46 |
| tmcpeak | 2) tests that don't implement any functionality beyond blacklist_calls should use that | 15:47 |
| tmcpeak | 3) tests that do implement functionality beyond blacklist_calls should be in a function by themselves. We will move them at the time that the functionality is extended | 15:47 |
| *** elo has quit IRC | 15:47 | |
| Daviey | +1 | 15:47 |
| tmcpeak | 4) config needs to be over-rideable, bandit.yaml is not the single source of config - blacklist_calls is an initial candidate for things that should be moved to a separate file | 15:48 |
| tmcpeak | fletcher_, Daviey, browne, sigmavirus24, tkelsey: everybody happy with 1-4? | 15:48 |
| tmcpeak | ^ | 15:48 |
| tmcpeak | or "can live with it" | 15:48 |
| tmcpeak | ? | 15:48 |
| fletcher_ | I disagree with #2 (as I'm sure everyone knows), but I can agree that seems to be the conesnsus | 15:48 |
| * fletcher_ can live with it for now | 15:48 | |
| tmcpeak | fair enough :) | 15:49 |
| Daviey | tmcpeak: succulently summarised.. +4 | 15:49 |
| tmcpeak | great | 15:49 |
| fletcher_ | boom, good convo | 15:49 |
| tmcpeak | fletcher_: thanks for dropping in | 15:49 |
| sigmavirus24 | Yeah I can live with it I think | 15:50 |
| sigmavirus24 | tmcpeak: I'd rather be able to be part of the discussion though in Seattle even if remotely, so doing this over beers might not be the most productive =P | 15:50 |
| Daviey | tmcpeak: So with this, do we consider tkelsey unblocked? | 15:50 |
| tmcpeak | sigmavirus24: ok cool we'll get you in conversation for sure | 15:51 |
| tmcpeak | Daviey: yeah, I believe so :) | 15:51 |
| Daviey | great! | 15:51 |
| tmcpeak | yeah cool, good stuff | 15:51 |
| tmcpeak | so buy in from browne, tkelsey and I think we're gtg | 15:51 |
| * sigmavirus24 is also being pulled in 3 directions right now so please excuse the delayed responses | 15:52 | |
| tmcpeak | sigmavirus24: cool, thanks for the time | 15:52 |
| sigmavirus24 | tmcpeak: no problem | 15:55 |
| tkelsey | Heh, so my next meeting just got cancelled :) Im back | 15:59 |
| tmcpeak | just in time :) | 15:59 |
| tkelsey | reading over the scrollback, seems like we got to a good place ? | 15:59 |
| tkelsey | I like you summation tmcpeak | 16:00 |
| *** fletcher_ has quit IRC | 16:00 | |
| tkelsey | fletcher_: I would be happy to work with you around point 2, if you have specific things you want to see | 16:00 |
| tmcpeak | fletcher has left the building | 16:00 |
| tmcpeak | we'll figure it out in Seattle | 16:01 |
| tmcpeak | and bridge in sigmavirus24, Daviey etc for the discussion | 16:01 |
| tkelsey | ok cool. Sorry I had to drop out before, 'murrica woke up :) | 16:01 |
| tkelsey | thanks for continuing things in my abscence tmcpeak | 16:02 |
| *** lexholden has quit IRC | 16:02 | |
| tmcpeak | sure, sounds like we have a good way forward here | 16:02 |
| sigmavirus24 | So Daviey my only concern that I have with merging/overriding config is that people are going to want something more than what we're going to offer | 16:03 |
| sigmavirus24 | It's a thing I've seen in Flake8 that has broken things in the past | 16:03 |
| sigmavirus24 | People have radically different ideas of how configs should be merged/overlayed/extended/etc and no one will be happy, so this will *always* be a problem | 16:04 |
| sigmavirus24 | As soon as we start merging things, I think it'll be a bad time for us | 16:04 |
| sigmavirus24 | Unless we come to a design decision early and stay firm on it unless something really enticing is proposed | 16:04 |
| sigmavirus24 | (Which isn't exactly friendly seeming to new contributors, but at this point I'm worried more about not getting stuck in the hell that is the extents to which configuration management can be stretched in lint/analysis tooling) | 16:05 |
| Daviey | sigmavirus24: Yeah, i think we'd need a spec for it TBH | 16:07 |
| sigmavirus24 | Yeah | 16:07 |
| sigmavirus24 | Put it like this, people (I won't name names) wanted Flake8 to look at some global config, then their personal config, then the project config and then subproject configs | 16:08 |
| sigmavirus24 | The order of operations there might seem natural that each level closer to the project overrides the previous but they wanted the project config to override the subproject config | 16:08 |
| tmcpeak | Daviey, sigmavirus24: yes, this is a perfect use for a spec | 16:10 |
| *** shohel has quit IRC | 16:11 | |
| sigmavirus24 | Yeah | 16:12 |
| sigmavirus24 | I think we should discuss exactly how far we want to actually go with configuration of bandit and its behaviour | 16:12 |
| Daviey | sigmavirus24: Yeah, good point. Nobody will ever be happy :) | 16:12 |
| sigmavirus24 | Some people will want to just remove something from blacklist calls which in their personal config would mean copying everything but | 16:13 |
| sigmavirus24 | Which means then when we add new things that they want, they need to be aware of that and copy those too | 16:13 |
| sigmavirus24 | (if they want them) | 16:13 |
| sigmavirus24 | The way to make that not be a problem would be to allow them to blacklist blacklist_calls | 16:13 |
| sigmavirus24 | Which then becomes a really bad headache | 16:13 |
| Daviey | hah, i was about to say that. Sounds wrong! | 16:14 |
| sigmavirus24 | Yeah | 16:14 |
| sigmavirus24 | Which lends more credibility to fletcher's argument that blacklist_calls is bad, although in a different direction (configurability rather than accuracy) | 16:14 |
| *** rmarathu has quit IRC | 16:15 | |
| tmcpeak | well no, I think they don't have to copy everything but | 16:16 |
| tmcpeak | I think we're talking about moving blacklist calls to its own config for starter | 16:16 |
| tmcpeak | so if they want to remove something, they just go remove it | 16:16 |
| tmcpeak | comment it out, etc | 16:16 |
| sigmavirus24 | Okay | 16:16 |
| sigmavirus24 | So having per-plugin config files? | 16:17 |
| sigmavirus24 | That might be ... interesting | 16:17 |
| Daviey | tmcpeak: Are you saying, cp $stockconfig $myconfig ; sed 's/somethingihate//' $myconfig ? | 16:17 |
| *** Windir has quit IRC | 16:20 | |
| *** alex_klimov has quit IRC | 16:20 | |
| Daviey | tmcpeak: Also, is that hardcoded password issue you found on an open source project? | 16:20 |
| *** sdake has joined #openstack-security | 16:22 | |
| *** Windir has joined #openstack-security | 16:22 | |
| tmcpeak | Daviey: yeah, pretty much that | 16:23 |
| Daviey | tmcpeak: With that model, will people bother pulling in a new config getting latest goodness? | 16:23 |
| *** rmarathu has joined #openstack-security | 16:23 | |
| *** browne has joined #openstack-security | 16:25 | |
| tmcpeak | Daviey: most interesting bug wasn't in open source, but there's also this: https://bugs.launchpad.net/nova/+bug/1420457 | 16:25 |
| openstack | Launchpad bug 1420457 in OpenStack Compute (nova) "Insecurely generated session ID in vpn_ping()" [Low,Fix released] - Assigned to Tony Breeds (o-tony) | 16:25 |
| tmcpeak | Daviey: maybe, they should, but if they don't that's kind of on them | 16:25 |
| tmcpeak | some people intentionally don't adopt new flake8 plugins too | 16:26 |
| tmcpeak | workflow could be as easy as "get new profiles" go back and comment the ones I don't like | 16:26 |
| tmcpeak | 20 seconds of effort | 16:26 |
| Daviey | true | 16:27 |
| Daviey | That does go someway of reducing complexity | 16:28 |
| tmcpeak | yeah, shouldn't be too bad | 16:29 |
| sigmavirus24 | tmcpeak: so re flake8 plugins: there are lots of 3rd party ones | 16:36 |
| sigmavirus24 | No one pulls those in by default | 16:36 |
| sigmavirus24 | Further, Hacking is now taking advantage of Flake8's ability to have checks off by default | 16:36 |
| tmcpeak | sigmavirus24: it seems like even stock ones though people are constantly disabling some | 16:36 |
| sigmavirus24 | Although that needs a bit more work in Flake8 to allow them to be enabled | 16:37 |
| sigmavirus24 | tmcpeak: right, that's actively disabling rather than not pulling in new ones ;) | 16:37 |
| sigmavirus24 | the language confused me | 16:37 |
| sigmavirus24 | carry on | 16:37 |
| tmcpeak | oh right, we might be saying the same thing | 16:37 |
| sigmavirus24 | yeah | 16:37 |
| sigmavirus24 | different language, same idea | 16:37 |
| tmcpeak | I'm saying workflow is disable some, then at some point "give me all the new bandit" which drops new configs and then you can go and copy over your disabling | 16:38 |
| sigmavirus24 | Right | 16:38 |
| sigmavirus24 | ignore= in Flake8 land | 16:38 |
| tmcpeak | yeah | 16:39 |
| sigmavirus24 | and that would work well if browne's idea of having static codes assigned to checks is something we want | 16:48 |
| sigmavirus24 | then you can just say ignore=XML100124 =P | 16:48 |
| * sigmavirus24 is making up codes and names for the same of illustration | 16:48 | |
| browne | sigmavirus24: +1 | 16:52 |
| openstackgerrit | Merged openstack/anchor: Check for exception code and not type https://review.openstack.org/206257 | 16:53 |
| tmcpeak | for sure, that seems to tie in nicely | 16:57 |
| *** dwyde has quit IRC | 17:00 | |
| *** elo has joined #openstack-security | 17:04 | |
| *** tkelsey has quit IRC | 17:22 | |
| *** dwyde has joined #openstack-security | 17:27 | |
| *** nkinder has quit IRC | 17:32 | |
| *** markvoelker has quit IRC | 17:47 | |
| *** jmckind has quit IRC | 17:55 | |
| *** jmckind has joined #openstack-security | 17:59 | |
| *** nkinder has joined #openstack-security | 18:01 | |
| *** b10n1k has joined #openstack-security | 18:14 | |
| *** singleth_ has joined #openstack-security | 18:20 | |
| *** singlethink has quit IRC | 18:23 | |
| *** markvoelker has joined #openstack-security | 18:42 | |
| *** markvoelker_ has joined #openstack-security | 18:46 | |
| *** markvoelker has quit IRC | 18:48 | |
| *** singlethink has joined #openstack-security | 19:08 | |
| *** voodookid has joined #openstack-security | 19:09 | |
| *** singleth_ has quit IRC | 19:11 | |
| *** markvoelker_ has quit IRC | 19:29 | |
| *** singleth_ has joined #openstack-security | 19:29 | |
| *** singlet__ has joined #openstack-security | 19:31 | |
| *** singlethink has quit IRC | 19:32 | |
| *** singleth_ has quit IRC | 19:34 | |
| *** elo1 has joined #openstack-security | 19:55 | |
| *** elo has quit IRC | 19:57 | |
| *** b10n1k has quit IRC | 20:01 | |
| *** b10n1k has joined #openstack-security | 20:02 | |
| *** alex_klimov has joined #openstack-security | 20:18 | |
| *** y_sawai has joined #openstack-security | 20:21 | |
| *** dwyde has quit IRC | 20:30 | |
| *** salv-orlando has joined #openstack-security | 20:31 | |
| *** salv-orl_ has quit IRC | 20:34 | |
| *** lexholden has joined #openstack-security | 20:39 | |
| *** tkelsey has joined #openstack-security | 21:11 | |
| *** dwyde has joined #openstack-security | 21:12 | |
| *** tkelsey has quit IRC | 21:16 | |
| *** y_sawai has quit IRC | 21:18 | |
| *** bknudson has quit IRC | 21:59 | |
| *** bpokorny_ has joined #openstack-security | 22:01 | |
| *** bpokorny has quit IRC | 22:04 | |
| *** bpokorny_ has quit IRC | 22:04 | |
| *** bpokorny has joined #openstack-security | 22:04 | |
| *** dwyde has quit IRC | 22:07 | |
| *** dwyde has joined #openstack-security | 22:16 | |
| *** jmckind has quit IRC | 22:18 | |
| openstackgerrit | Darren Chan proposed openstack/security-doc: Renamed Future section and added domain information https://review.openstack.org/199393 | 22:28 |
| *** bpokorny has quit IRC | 22:29 | |
| *** bpokorny has joined #openstack-security | 22:29 | |
| *** singlet__ has quit IRC | 22:40 | |
| *** dwyde has quit IRC | 22:41 | |
| *** alex_klimov has quit IRC | 22:43 | |
| *** elo has joined #openstack-security | 22:48 | |
| *** elo1 has quit IRC | 22:52 | |
| *** bpokorny has quit IRC | 22:52 | |
| *** bpokorny has joined #openstack-security | 22:53 | |
| *** viraptor has joined #openstack-security | 23:02 | |
| *** voodookid has quit IRC | 23:06 | |
| *** edmondsw has quit IRC | 23:07 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 23:07 | |
| *** maikol1 has joined #openstack-security | 23:11 | |
| maikol1 | quien para hablar | 23:12 |
| *** dave-mccowan has quit IRC | 23:14 | |
| *** maikol1 has left #openstack-security | 23:15 | |
| openstackgerrit | Dave Walker proposed openstack/anchor: [WIP] Initial commit of devstack plugin https://review.openstack.org/206264 | 23:18 |
| Daviey | viraptor: Upstream pecan worked on the non-determinsitic 500 issue today.. but working around it still makes sense i think (to support older versions) - https://github.com/ryanpetrello/pecan/commit/f52d3b4fe2aca0a09fb313e60dcd63ee0a1c88e0 | 23:22 |
| Daviey | But it was good to see they validated that it was a bug in pecan, not our stuff. | 23:22 |
| viraptor | I was just going to send you the link :) | 23:22 |
| Daviey | hah | 23:23 |
| viraptor | yeah, yesterday on a call we decided to accept the patch as it is to unbreak the CI, but I'll try to make a nicer version today | 23:23 |
| viraptor | when global requirements move past this patch in pecan, we could drop our version | 23:24 |
| Daviey | Yeah, that is true | 23:25 |
| Daviey | viraptor: what do you want to improve about the patch? | 23:25 |
| viraptor | maybe make it a bit closer to what pecan does - actually raise the right exception every time for consistency, instead of using .abort() | 23:27 |
| viraptor | I like the idea of tests working explicitly on the right exception as they did before | 23:27 |
| Daviey | viraptor: meh, I prefer using object wrappers aswell for the application itself.. but standardising on error code in tests feels reasonable to me. | 23:30 |
| *** elo1 has joined #openstack-security | 23:36 | |
| *** jmckind has joined #openstack-security | 23:37 | |
| *** elo has quit IRC | 23:38 | |
| *** salv-orlando has quit IRC | 23:52 | |
| *** sdake has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!