18:01:08 <morganfainberg> #startmeeting Keystone
18:01:08 <openstack> Meeting started Tue Mar  3 18:01:08 2015 UTC and is due to finish in 60 minutes.  The chair is morganfainberg. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:13 <openstack> The meeting name has been set to 'keystone'
18:01:17 <amakarov> morganfainberg, you made it!
18:01:29 <morganfainberg> #topic Bandit - a security focused static code analysis tool
18:01:30 <bknudson> hi
18:01:34 <morganfainberg> lets just get right into it
18:01:37 <morganfainberg> tmcpeak1 o/
18:01:47 <tmcpeak1> hi all, I'm tmcpeak - I usually hang out in #openstack-security
18:01:48 <ayoung> Can you do Static analysis on Python?
18:01:54 <tmcpeak1> sure
18:01:59 <ayoung> tmcpeak1, lies
18:02:01 <ayoung> Ha
18:02:05 <tmcpeak1> lmao
18:02:06 <tmcpeak1> well
18:02:06 * ayoung jaded
18:02:07 <tmcpeak1> kinda
18:02:10 <tmcpeak1> :D
18:02:17 <morganfainberg> ayoung, i can look at code in python... thats the same thing right? :P
18:02:22 <tmcpeak1> so for those that haven't heard of Bandit, it's here
18:02:28 <gyee> just use the @deprecation tag :)
18:02:33 * stevemar sneaks in late
18:02:35 * morganfainberg quiets up for tmcpeak1 to continue.
18:02:35 <tmcpeak1> https://github.com/stackforge/bandit
18:02:41 <topol> static analysis for python must be tough with all the dynamic binding
18:02:42 <tmcpeak1> #link https://github.com/stackforge/bandit
18:03:03 <tmcpeak1> and #link https://wiki.openstack.org/wiki/Security/Projects/Bandit
18:03:19 <tmcpeak1> so basically the idea here is to call attention to low hanging fruit in code that is checked in
18:03:31 <tmcpeak1> we have about 40 tests
18:03:33 <ayoung> tmcpeak1, I assume that this is being run against Keystone, and any weaknesses found will be treated as a security bug until triaged?
18:03:38 <tmcpeak1> so far, many of them are very very noisy
18:03:45 * lbragstad walks in late
18:03:59 <tmcpeak1> yes, so the reason I'm here is that I'd like Keystone to volunteer to run Bandit in a gate
18:04:08 <tmcpeak1> bknudson expressed some interest in this
18:04:11 <dolphm> tmcpeak1: as in, false positives?
18:04:18 <ayoung> tmcpeak1, hmmm
18:04:25 <ayoung> Gate is the wrong place
18:04:31 <joesavak> o/ : )
18:04:34 <ayoung> we don't want to publically announce any problems
18:04:36 <tmcpeak1> so the profile that I've made for Keystone is conservative and currently doesn't have any results
18:04:37 <bknudson> I've been working on a tox env -- https://review.openstack.org/#/c/157930/
18:04:43 <ayoung> here's the issue
18:04:50 <tmcpeak1> ayoung: these are publicly available anyway, Bandit is opensource
18:04:53 <bknudson> the point of putting it in the gate is to stop bad code getting in to begin with.
18:04:55 <ayoung> lets say Bandit gets smart about some new problem
18:04:56 <tmcpeak1> you can just pull it down and find the same thing the gate test finds
18:05:07 <ayoung> and the gate now announces to the world :Keystone has vuln X
18:05:26 <tmcpeak1> yeah, but that is very publicly available anyway
18:05:37 <ayoung> tmcpeak1, not really
18:05:39 <dolphm> tmcpeak1: it sounds like you're implying we have violations against the full suite of tests?
18:05:40 <tmcpeak1> any attacker would already have access to that information
18:05:45 <marekd> after the vuln is fixed and commited
18:05:54 <ayoung> tmcpeak1, this should be run, but....
18:05:58 <tmcpeak1> dolphm: yeah, the full suite of tests include some very noisy ones that need to be tuned
18:06:08 <ayoung> OK...so it would be something like this:
18:06:11 <lbragstad> what about publishing those results to a private page?
18:06:22 <ayoung> 1.  run it privately, file bugs, get Keystone as secure as Bandit can say
18:06:23 <tmcpeak1> I've created a bandit config file for use with Keystone
18:06:25 <tmcpeak1> http://pastebin.com/SeBkRvCS
18:06:28 <tmcpeak1> #link http://pastebin.com/SeBkRvCS
18:06:30 <bknudson> the plan is to have bandit work very similar to pep8
18:06:40 <tmcpeak1> please check out that link
18:06:43 <ayoung> 2.  No upgrades to Bandit in Gate until private runs have the same level of fix
18:06:52 <bknudson> as with pep8, a new bandit wouldn't be run against keystone in the gate without a requirements bump.
18:07:07 <ayoung> tmcpeak1, you understand my concern?
18:07:15 <tmcpeak1> ayoung: absolutely
18:07:27 <ayoung> We do things like this internally, too.  We run tools like Valgrind etc.  And this is fantastic
18:07:30 <morganfainberg> this means bandit would need to be locked separately from requirements
18:07:31 <tmcpeak1> so with what in my opinion is a sensible profile, Keystone already has no vulnerabliities
18:07:37 <tmcpeak1> here's output of recentish Keystone
18:07:38 <morganfainberg> since global reqs overrides requirements.txt
18:07:39 <ayoung> but we don't publish the results.  We consume them internally and fix
18:07:41 <tmcpeak1> #link http://pastebin.com/U0Fpjar1
18:07:46 <morganfainberg> tmcpeak1, i'm running it locally right now on keystone fwiw
18:07:47 <morganfainberg> ;)
18:07:52 <dstanek> sorry - here now. traffic was a killer
18:08:09 <tmcpeak1> morganfainberg: oh cool, you're already using it?
18:08:25 <tmcpeak1> so yeah, some of the tests in the full suite generate *tons* of false positives
18:08:31 <morganfainberg> tmcpeak1, just ran it
18:08:37 <morganfainberg> tmcpeak1, took me 30 seconds to enable
18:08:44 <tmcpeak1> the idea with the profile I pasted above is that it will really only flag things you don't want in your code
18:08:53 <morganfainberg> so, i see sqla issues and complaining about popen
18:08:56 <morganfainberg> mostly
18:09:05 <tmcpeak1> morganfainberg: use the above linked profile please
18:09:06 <ayoung> popen is awesome!
18:09:07 <tmcpeak1> sql tests are noisy
18:09:08 <morganfainberg> and binding to all interfaces (desired)
18:09:09 <dolphm> tmcpeak1: please be clear - you're suggesting that there's no value in reporting the violations that bandit is currently seeing in keystone?
18:09:15 <bknudson> earlier it found a problem where regular random was used, so I posted a fix for that: https://review.openstack.org/#/c/157990/
18:09:37 <tmcpeak1> dolphm: no, not at all.  I'd like to see folks using Bandit on a regular (manual) basis to close bugs
18:09:48 <tmcpeak1> I also see value in an automated gate process which prevents egregious checkins
18:09:50 <tmcpeak1> please see this
18:09:50 <bknudson> and it also complained about the use of 0.0.0.0 for default bind host: https://review.openstack.org/#/c/157975/
18:10:09 <tmcpeak1> (before I put this, please note this is all publicly available info, I'm not disclosing 0 days)
18:10:10 <bknudson> but I didn't have tmcpeak1's profile.
18:10:15 <tmcpeak1> #link http://pastebin.com/G7iVFGZD
18:10:21 <tmcpeak1> bknudson: I just made it this morning :)
18:10:38 <tmcpeak1> we'd like to prevent this kind of code from getting into Keystone
18:10:48 <bknudson> just be glad we're not trove.
18:11:05 <raildo> l
18:11:16 <ljfisher> ayoung, so presumably something real caught in a gate test wouldn’t actually get merged until fixed. Would you still consider that a vulnerability in Keystone even if it hasn’t been merged? (I’ve also been hacking on Bandit)
18:11:22 <morganfainberg> so tmcpeak1 , with that profile i see no violations?
18:11:34 <tmcpeak1> yeah, here's an output of Keystone with that profile
18:11:41 <morganfainberg> ljfisher, the concern isn't new code. we agree that is fine
18:11:41 <tmcpeak1> command to run Bandit against Keystone with verbose profile: bandit -c keystone.yaml -p keystone_verbose ~/Documents/projects/keystone/ -r
18:11:48 <tmcpeak1> Keystone Bandit conservative run: http://pastebin.com/U0Fpjar1
18:12:00 <morganfainberg> ljfisher, the issue is old code when bandit gets smarter
18:12:02 <ayoung> ljfisher, so my concern is that we need to know about new vulnerabilities before they show up in the gate.  My concner is not with changes, but existing code.  So...I think this is OK...
18:12:12 <tmcpeak1> morganfainberg: for the gate we are pinning a version
18:12:14 <ayoung> but we need to have additional review done when Bandit gets update
18:12:18 <tmcpeak1> new Bandit functionality won't change your gate
18:12:30 <tmcpeak1> you'll pin to a version, and chose when to update
18:12:42 <tmcpeak1> same as you do with other dependancies
18:12:42 <ayoung> pinning version is good
18:12:44 <morganfainberg> tmcpeak1, pins in requirements.txt are overriden by global requirements
18:12:45 <morganfainberg> FYI
18:12:48 <nonameentername> o/
18:12:52 <morganfainberg> you can't pin in requirements.txt
18:12:59 <tmcpeak1> the pin will go into global requirements
18:13:03 <ayoung> M y concern is not Keystone, but Bandit.  Before Bandit posts a new update...we'd want to know what it caught.
18:13:08 <bknudson> the pin for hacking is handled differently.
18:13:31 <bknudson> it's not automatically updated.
18:13:48 <morganfainberg> bknudson, we'd need this to fall into that same category then
18:13:52 <ayoung> OK...Gate is a red herring.  I have no problem with running Bandit in gate
18:14:12 <tmcpeak1> cool :)
18:14:17 <tmcpeak1> ayoung: you still concerned about vulns though?
18:14:17 <morganfainberg> tmcpeak1, yeah i'd be fine with this being an additional check - i'd like this to not be lumped into pep8 though
18:14:28 <morganfainberg> tmcpeak1, if infra doesn't mine i'd like to see this as a specific job
18:14:33 <tmcpeak1> morganfainberg: same, I'd like it to be a separate job
18:14:36 <tmcpeak1> yeah
18:14:42 <morganfainberg> if there are concerns from infra we can put it in pep8
18:14:50 <ayoung> What we do need is to know about new classes of vulnerabilities before we make the tool general available.  And that is not just a Keystone issue
18:14:53 <morganfainberg> but my opinion is 1st go w/ separate job
18:14:59 <bknudson> the requirements for this job are different than pep8
18:15:08 <morganfainberg> bknudson, exactly
18:15:11 <bknudson> since bandit doesn't need all the test-requirements like pep8 does.
18:15:28 <morganfainberg> and having it clearly be a separate failure would be very good.
18:15:32 <ayoung> ++
18:15:35 <dolphm> tmcpeak1: have you successfully used bandit to identify and fix actual vulnerabilities?
18:15:40 <tmcpeak1> ayoung: I definitely hear what you're saying.  Another piece which I'm not mentioning here because it isn't directly related to my gate test mission is getting people  to use it against their own projects
18:15:52 <morganfainberg> tmcpeak1, so i'm not opposed to this being enabled for keystone.
18:15:54 <tmcpeak1> dolphm: absolutely.  One place I've had great success is trove
18:16:13 <tmcpeak1> let me dig up one for you
18:16:34 <morganfainberg> tmcpeak1, i assume the .yaml will be hosted in keystone's tree [for sanity sake]
18:16:47 <tmcpeak1> https://bugs.launchpad.net/trove/+bug/1349939
18:16:49 <openstack> Launchpad bug 1349939 in Trove "Multiple vulnerabilities in Couchbase implementation of restore strategy" [Critical,Fix released] - Assigned to Amrith (amrith)
18:16:55 <tmcpeak1> so this bug in Trove was… really bad
18:16:57 <morganfainberg> not that i expect ot be changing it a bunch, but to save you a headache of updating.
18:17:08 <tmcpeak1> I found this using Bandit and then tracing through the code
18:17:10 <morganfainberg> for every project in the future.
18:17:22 <ayoung> tmcpeak1, this has the potential to be a Pandora's box.  But the box is already open I guess.
18:17:30 <tmcpeak1> this would have resulted in VMT process if Trove had been in production already
18:17:41 <bknudson> no OSSA only because it was in master.
18:17:49 <bknudson> so it caught a bug before release.
18:17:51 <tmcpeak1> morganfainberg: yes, I'd really like for each project to maintain their own profile
18:17:57 <bknudson> so... thanks!
18:17:59 <morganfainberg> tmcpeak1, cool
18:18:20 <tmcpeak1> ayoung: yeah, box is open :)
18:18:57 <morganfainberg> tmcpeak1, so step 1 for Keystone, - experimental job in gate, lets get the keystone tree in shape for this, step 2 - non-vote, once we're happy - voting
18:18:57 <tmcpeak1> cool, so I didn't want to take up all your time
18:19:02 <amrith> tmcpeak1, the bug you mention has been fixed in trove, yes?
18:19:07 <amrith> I remember doing it a while ago.
18:19:09 <tmcpeak1> morganfainberg: perfect
18:19:22 <tmcpeak1> amrith: yeah, you fixed it
18:19:24 <morganfainberg> tmcpeak1, ping me when you have the reviews to change project-config up, i'll +1 them as needed
18:19:29 <amrith> tmcpeak1, thx
18:19:34 <bknudson> we need an official release of bandit, also.
18:19:41 <tmcpeak1> morganfainberg: perfect, thank you
18:19:57 <morganfainberg> lets make sure bandit is released/in pypi etc
18:20:01 <tmcpeak1> yeah, there are a couple of more steps we need, but I'll be reaching out to a few of you to push this along when we're ready
18:20:19 <tmcpeak1> want to make sure the user (your) experience is rock-solid
18:20:37 <bknudson> so it's going to be clean to begin with
18:20:43 <dolphm> tmcpeak1: i just hope it doesn't prove to be a noisey tool spewing false positives at every contributor & reviewer
18:20:43 <bknudson> any new changes will eventually go through bandit
18:20:55 <morganfainberg> tmcpeak1, thanks. and make sure you get the experimental stuff in infra first, so we can do "check expirimental" as we implement the yaml etc in our tree
18:20:59 <bknudson> and if it's flagging things we need to decide if it's a false positive.
18:21:10 <tmcpeak1> dolphm: if it is, we (Bandit team) have completely failed.  We're going to test the crap out of it before we ask you guys to use it
18:21:23 <dolphm> tmcpeak1: is there a feature similar to # noqa for flake8 ?
18:21:24 <bknudson> There's a #nosec comment you can put on a line to tell bandit to ignore it.
18:21:30 <tmcpeak1> ^ :)
18:21:47 <tmcpeak1> our goal is not to have you guys using #nosec to get it to shut up
18:22:06 <tmcpeak1> nosec should basically be like "yeah we understand this isn't good from security but it's really ok because xyz"
18:22:22 <morganfainberg> there are cases we need to noqa
18:22:27 <morganfainberg> there will be limited cases to nosec
18:22:27 <dolphm> tmcpeak1: i'd expect a small handful of legitimate # nosec with explanations
18:22:36 <tmcpeak1> I've found bugs like that before too.. In nova they were generating a session ID in an uncryptographically safe way, turns out it didn't matter
18:22:40 <morganfainberg> but those will be the exception to the rule
18:22:59 <tmcpeak1> so that's a perfect use for #nosec followed by a comment about why it doesn't matter
18:23:01 <morganfainberg> tmcpeak1, ok i look forward to seeing this become available.
18:23:24 <tmcpeak1> the Nova guys actually had pushed a comment explaining why it isn't a security risk, so next guy doesn't come and panic about uncrypto session ID
18:23:34 <tmcpeak1> cool, sounds good!
18:23:57 <dolphm> tmcpeak1: # nosec should require an inline comment at least!
18:24:03 <tmcpeak1> definitely
18:24:09 <morganfainberg> tmcpeak1, yeah was just going to suggest that
18:24:13 <dolphm> tmcpeak1: # nosec: {why}
18:24:22 <morganfainberg> make it so #nosec can't be just bare on the line
18:24:28 <dolphm> morganfainberg: ++
18:24:35 <tmcpeak1> yeah, that's an awesome idea
18:24:51 <dolphm> morganfainberg: although you probably need more than just an inline comment to justify such a "why"
18:24:51 <tmcpeak1> I'll add that to our dev list :)
18:25:01 <tmcpeak1> yeah, they're probably going to be somewhat involved
18:25:15 <tmcpeak1> honestly with current #nosec handling I'm not sure the best way to do it, but I agree we should require something
18:25:34 <dolphm> tmcpeak1: maybe # nosec requires a comment on the preceeding line?
18:25:46 <tmcpeak1> we're dealing with AST and tying together multi-line comments is… at least a little challenging
18:26:05 <tmcpeak1> I'll throw it on our backlog though
18:26:06 <dolphm> tmcpeak1: that's the fun!
18:26:17 <morganfainberg> tmcpeak1, dolphm throws down the gauntlet... ;)
18:26:30 <tmcpeak1> and in the mean time we can have project reviewers throw liberal -1's for promiscuous #nosec usage
18:26:39 <tmcpeak1> it shall be done :D
18:26:54 <morganfainberg> ok so we'll look for the infra changes and then the stuff to make bandit a reality
18:27:00 <morganfainberg> tmcpeak1, thanks!
18:27:04 <tmcpeak1> sounds great!
18:27:05 <tmcpeak1> thanks all
18:27:11 <morganfainberg> #topic Specs that will need Feature Freeze Exceptions (FFE) and/or be delayed until Liberty
18:27:22 <morganfainberg> So, it's that sad time of the cycle
18:27:29 <bknudson> :(
18:27:35 <rodrigods> :(
18:27:35 <raildo> :(
18:27:35 <morganfainberg> time to punt things to either FFE or next cycle
18:27:46 <morganfainberg> #link https://launchpad.net/keystone/+milestone/kilo-3
18:28:49 <stevemar> i never understood why this is a sad time... it'll go in a few week later is all :P
18:28:52 <morganfainberg> currently on the chopping block [for FFE unless it is gating today, no particular order]: x509, Domain-SQL, Reseller, Improve list role assignments filtering performance
18:28:57 <henrynash> So I really want to keep domain specific configs in Kilo….we are doing pretty well on reviews…but not sure all will get in by the 5th
18:29:34 <morganfainberg> if those are not gating today, they are getting a -2 (procedural) and require a FFE after k3 cuts, or a delay to liberty
18:29:40 <stevemar> fwiw i do think the domain-sql stuff is close
18:29:56 <morganfainberg> i think domain-sql is the best option for FFE, reseller is a close second
18:30:16 <morganfainberg> the others i'm not optimistic they will be accepted for FFE
18:30:23 <morganfainberg> remember FFE isn't just us, keystone-core
18:30:23 <marekd> there is also this https://blueprints.launchpad.net/openstack/?searchtext=idp-id-registration
18:30:27 <morganfainberg> it's also rel-mngmt
18:30:34 <marekd> and review that is held off by a bug.
18:30:35 <rodrigods> for reseller, we need more eyes on it though, 0 reviews so far
18:31:13 <rodrigods> we are about to send the last change of the chain
18:31:15 <morganfainberg> marekd, and that one i think will be pushed
18:31:30 <morganfainberg> marekd, as much as i'd like to see it.
18:31:41 <henrynash> morganfainberg: assuming the reviewers are happy with my updates, the core underpinnings of domain-SQL may well be gating today, not just the “switch on” at the top
18:31:55 <morganfainberg> henrynash, and thats why i said gating today
18:32:03 <morganfainberg> if it's gating actively, we'll make sure it lands
18:32:10 <marekd> morganfainberg: me too, i wanted to discuss it later (post-meeting)
18:32:14 <henrynash> morganfainberg: ok, agreed
18:32:17 <morganfainberg> if it isn't lets plan a FFE
18:32:31 <morganfainberg> that one the FFE would be "yeah we couldn't land this in k3, but it's ready to go"
18:32:49 <henrynash> morganfainberg: agreed
18:33:12 <stevemar> henrynash, i'll take another look at your stuff today, but i looked at the comments you had made and i think you addressed all my concerns
18:33:25 <henrynash> stevemar: great, thx
18:33:32 <morganfainberg> other than that.
18:33:41 <morganfainberg> i have nothing else for the meeting
18:33:55 <morganfainberg> #topic Open Discussion
18:33:56 <marekd> ok, so maybe i can use some of this time?
18:34:00 <morganfainberg> go for it
18:34:06 <ayoung> henrynash, https://review.openstack.org/#/c/160872/3  make a stand alone patch and I'd be ahppy to +2A it
18:34:20 <ayoung> if depends on Add API support for domain config  and there is no need for that
18:34:36 <stevemar> ayoung, i'll rebase it on master and we can push it through?
18:34:39 <marekd> https://review.openstack.org/#/c/152156/ this review is actually kind of blocked, it passes the unittests but will probably explode with mysql
18:34:47 <ayoung> stevemar, go for it
18:34:48 <henrynash> ayoung: true..but then the follow on patches can’t depend on two things
18:34:48 <marekd> do every devstack installation following master
18:34:52 <marekd> the bug is here:
18:35:04 <ayoung> henrynash, reverse the order
18:35:13 <marekd> #link https://bugs.launchpad.net/keystone/+bug/1426334
18:35:14 <openstack> Launchpad bug 1426334 in Keystone "DB migration problem with federation extension" [Undecided,In progress] - Assigned to Marco Fargetta (marco-fargetta)
18:35:14 <ayoung> put the trivial changes at the front of the patch-stack
18:35:25 <henrynash> ayoung: ok….yes, I’ll push it to the front, ++
18:35:37 <marekd> and as i talked with Marco, adding a migration script will not help
18:35:39 <ayoung> thanks
18:35:39 <stevemar> henrynash, hit that cherry pick button
18:35:50 <marekd> as sanity_check in oslo_db will raise an exception.
18:35:55 <raildo> If someone want review the reseller patches :)
18:35:58 <raildo> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/reseller,n,z
18:36:02 <morganfainberg> marekd, and marco has done some work there.
18:36:03 <henrynash> stevemar: it’s standalne, justa reorder of dependant pacthes
18:36:23 <morganfainberg> marekd, i think we need docstrings for his fix and a bug against oslo to reference
18:36:38 <morganfainberg> marekd, my major complaint with his patch
18:36:43 <marekd> https://review.openstack.org/#/c/159803/5/keystone/common/sql/migration_helpers.py ?
18:36:48 <marekd> morganfainberg: ^^
18:36:51 <morganfainberg> though the code could probably be claeaned up some
18:36:52 <morganfainberg> marekd, yeah
18:36:59 <bknudson> I wonder why the db migration problem doesn't show up in the gate?
18:37:35 <marekd> sqlite maybe ?
18:37:41 <stevemar> bknudson, ^
18:37:41 <marekd> or theu use mysql?
18:38:16 <marekd> morganfainberg: oh, i see, he is esentually recreating all the tables
18:38:26 <morganfainberg> bknudson, because the gate default assumes utf-8 iinodb
18:38:38 <morganfainberg> this is a case where someone has misconfigured their mysql then pushed in keystone
18:38:41 <morganfainberg> then get wedged
18:38:44 <marekd> but if healthy actually? it may move lots of data in some rare cases.
18:38:46 <bknudson> morganfainberg: I thought they changed the gate defaults just to catch this
18:38:48 * jamielennox sneaks in the back
18:38:57 <morganfainberg> bknudson, clearly not :P
18:39:10 <morganfainberg> or we did something wonky
18:39:49 <bknudson> we've made fixes in the past to set tables to InnoDB
18:40:02 <morganfainberg> the issue here is it related to FKs. so it's ugly
18:40:03 <bknudson> I don't think we've tried to fix the charset.
18:40:11 <morganfainberg> bknudson, we did, migraiton 37
18:40:14 <marekd> morganfainberg: for the utf-8 patch we would then need to remove 001, 002 files
18:40:15 <morganfainberg> migration*
18:40:40 <morganfainberg> marekd, no, we fix them in-place as we have then we do the exceptional case when it's broken to catch anyone who's run them
18:40:52 <morganfainberg> marekd, look at the _Fix_migration_37 example
18:40:55 <stevemar> this is all sorts of ugly
18:41:01 <morganfainberg> it's not pretty but it unwedges the deployer
18:41:13 <morganfainberg> but we need to figure out how to catch this earlier.
18:41:25 <morganfainberg> maybe a hacking check to ensure table definitions in migrations have innodb/utf8?
18:41:40 <morganfainberg> or a way to make the sanity check less sane in an exceptional case so it can be fixed.
18:41:51 <bknudson> we do have test already for table structure
18:42:00 <morganfainberg> bknudson, but not in migration.
18:42:04 <bknudson> and I think there's a "live" test for innodb.
18:42:23 <morganfainberg> yeah. anyway this needs some eyes to maek sure we solve it this time ;)
18:43:39 <marekd> morganfainberg: where is that _Fix_migration_37 ?
18:43:48 <morganfainberg> in migration_helpers
18:43:53 <bknudson> marekd: line 142
18:43:59 <bknudson> https://review.openstack.org/#/c/159803/5/keystone/common/sql/migration_helpers.py
18:44:23 <morganfainberg> marekd, https://github.com/openstack/keystone/blob/master/keystone/common/sql/migration_helpers.py#L142-L181
18:45:15 <marekd> morganfainberg: ok, i will add a docstring
18:45:26 <morganfainberg> so i think we could make that code in marco's patch better by dropping the FK constraints and, fixing the table, and re-adding the constraints
18:45:30 <morganfainberg> rather than recreating the tables
18:45:49 <morganfainberg> less data mangling involved
18:46:28 <marekd> this need to be done all today?
18:46:34 <morganfainberg> marekd, that is a bug
18:46:39 <morganfainberg> that can happen anytime it lands
18:46:59 <morganfainberg> the dependant BP can't land until afterwards and might be FFE *or* just pushed until liberty
18:47:13 <morganfainberg> so i'd say fix the bug, ask for FFE if we want that in kilo
18:47:28 <marekd> ++
18:47:44 <morganfainberg> but start with the bug fix, which can land anytime before rc [sooner *is* better]
18:48:31 <morganfainberg> anything else?
18:48:43 <morganfainberg> before we close up the meeting and let infra have the channel early?
18:48:43 <marekd> how do i ask for FFE? ml-thread?
18:48:47 <marekd> with some explanation
18:48:49 <morganfainberg> marekd, yeah
18:49:18 <morganfainberg> why this should land in Kilo, why it wasn't able to land before FF, what the status of the code is
18:49:19 <morganfainberg> etc
18:49:25 <marekd> sure.
18:49:29 <morganfainberg> ok enjoy your spare 10minutes!
18:49:33 <morganfainberg> go get a cup of coffee :)
18:49:37 <henrynash> morganfainberg: remind me, do we ship python-keystoneclient with the serber or separately?
18:49:40 <morganfainberg> jamielennox, you probably need the coffee
18:49:49 <henrynash> (server even)
18:49:52 <morganfainberg> henrynash, separate, but we try and do a final release around RC
18:49:57 <jamielennox> morganfainberg: i could have stayed in bed :)
18:50:03 <henrynash> morganfainberg: ok
18:50:08 <morganfainberg> that is the releasr that ships with the named integrated release
18:50:11 <marekd> jamielennox: it's....4am now?
18:50:24 <morganfainberg> also expect a new ksm / ksc to go out shortly into march here
18:50:29 <jamielennox> no, nearly 6 - sydney time now
18:50:44 <morganfainberg> i'll be bugging people for ksc reviews and ksm stuff if we have something to release.
18:50:48 <morganfainberg> post k3 that is
18:50:54 <morganfainberg> thanks everyone!
18:50:58 <morganfainberg> #endmeeting