11:00:41 <oneswig> #startmeeting scientific-sig
11:00:41 <openstack> Meeting started Wed Jan 16 11:00:41 2019 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot.
11:00:42 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
11:00:43 <oneswig> ahoy
11:00:44 <openstack> The meeting name has been set to 'scientific_sig'
11:01:01 <oneswig> #link agenda for today https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_January_16th_2019
11:01:13 <janders> gday! :)
11:01:19 <daveholland> morning
11:01:27 <oneswig> Evening janders! Morning daveholland
11:01:38 <janders> it's so hot here today my roller blinds half melted..
11:01:47 <priteau> Hello.
11:01:54 <oneswig> Hi priteau! :-)
11:02:04 <verdurin> Morning.
11:02:05 <janders> swapped em out for wooden venetian blinds. If these start burning, it's time to move somewhere cooler..
11:02:31 <oneswig> janders: I'm finding that hard to relate to... could do with some warmth here!
11:02:40 <oneswig> Hi verdurin, is Callum with you?
11:02:46 <janders> We'd give away 15 degrees celcius any time
11:02:56 <janders> maximum of 41C today
11:03:43 <Xi0s> hi, Callum here
11:03:48 <belmoreira> o/
11:03:53 <oneswig> Xi0s: Hi Callum, thanks for coming.
11:04:01 <oneswig> Hi belmoreira!
11:04:06 <oneswig> OK let's get going
11:04:14 <oneswig> #topic Denver CFP
11:04:33 <oneswig> There is an HPC/GPU/AI track again
11:04:44 <oneswig> Deadline for submissions 23rd January, not far off
11:05:24 <oneswig> This time the Foundation have asked the community to assist with submission review and feedback
11:05:49 <oneswig> Blair has kindly offered to respond to anyone wanting a second opinion on a proposal.
11:06:12 <oneswig> In IRC channel #scientific-wg, or next week's meeting at 2100 UTC
11:06:30 <oneswig> He's not here today as he's in NZ and it's midnight
11:06:52 <oneswig> #link Open Infra Summit CFP https://www.openstack.org/summit/denver-2019/call-for-presentations/
11:07:08 <janders> That's great. I haven't started yet but definitely aiming to submit something.
11:07:12 <noggin143_> Does Blair need some help? If so, I'd be happy to help out with a third opinion
11:08:04 <oneswig> Thanks noggin143_! I'm sure that would be appreciated.  Hopefully we'll have some good material to review
11:08:54 <oneswig> While you're here noggin143_...
11:09:02 <oneswig> #topic OpenStack days at CERN
11:09:14 <oneswig> belmoreira: noggin143_: I hear you're planning an event!
11:09:49 <belmoreira> we will have an OpenStack Days CERN on the 27th of May
11:10:16 <belmoreira> the registration page should be available soon
11:10:44 <oneswig> Will it be on Indico?
11:11:06 <belmoreira> the theme of the event will be "Accelerating Science with OpenStack"
11:11:54 <belmoreira> at same time we will open the call for abstracts for lighting talks
11:12:47 <belmoreira> oneswig: the registration will be on eventbrite, but abstracts submissions will be on indico
11:13:00 <oneswig> belmoreira: will you announce on openstack-discuss when the registration page is online?
11:14:05 <belmoreira> not sure if openstack-discuss is really for this announcements. But it will be in many places
11:14:25 <oneswig> OK, thanks belmoreira noggin143_ - will look out for it
11:14:26 <belmoreira> I would like to invite all the Scientific SIG to this event
11:14:37 <janders> would you consider having remote presenters?
11:16:14 <belmoreira> janders: we didn't discuss that... but I'm not considering it
11:16:43 <oneswig> OK, thanks belmoreira
11:17:07 <oneswig> #topic Open Infra Days London
11:17:08 <noggin143_> recordings are also under discussion for later watching
11:17:36 <oneswig> The date is set for the London OpenStack event - 1st April
11:17:45 <oneswig> #link Open Infra Days London https://openinfradays.co.uk/
11:18:26 <oneswig> A scientific presentation track is planned
11:18:52 <oneswig> I hope we can get lots of participation from SIG members from this region
11:19:25 <oneswig> Exact details on scheduling/submission are TBD
11:19:55 <oneswig> That's all I have on that currently
11:20:11 <oneswig> #topic AAI for Medical Research Computing
11:20:34 <oneswig> We have a guest speaker today - Xi0s from Oxford Big Data Institute
11:20:45 <oneswig> Thanks for coming Xi0s
11:21:08 <oneswig> I saw his talk at CIUK 2018 and found it really informative
11:21:08 <Xi0s> No problem, I hope the content will be of some use to folks
11:21:28 <oneswig> #link Presentation slides https://www.dropbox.com/s/kvszibl5c0ijk13/AAI%20in%20Rescomp.pptx?dl=0
11:21:57 <oneswig> Xi0s: could you give a bit of context on what you were wanting to do?
11:21:57 <noggin143_> thanks, i'll pass it on to our AAI team
11:22:58 <Xi0s> Sure, so we're moving from simple genomics compute into a world of increasingly sensitive data requirements as we expand to support medical sciences at the university, and as such we've been building a lot of new services to support this
11:23:50 <oneswig> This line looks key today: "Single user identity and password store across all services"
11:23:59 <Xi0s> Core to this is a new OpenStack deployment, for which we wanted to implement a proper AAI underneath that could do 2FA for anyone and everyone in an enforceable and audit-able way
11:24:53 <Xi0s> exactly that, so single identity across the cluster, with a single password and single 2FA token registration, rather than per-service implementations
11:25:05 <oneswig> What was the second factor - a token to a mobile?
11:25:28 <Xi0s> it depends, mobile token (google authenticator) is easy, but has it's limitations
11:25:55 <Xi0s> no less, a boss that uses a Nokia 3310 - so we also support physical tokens (Yubikey for example)
11:26:14 <noggin143_> did you need a special sshd or was the 2FA standard?
11:26:22 <Xi0s> standard
11:26:42 <Xi0s> one of the driving requirements was that we can't afford the pain of distributing custom clients to users
11:28:13 <Xi0s> so openssh on the clientside supports the multi-factor auth as does the sshd on the server side
11:28:20 <oneswig> The second factor is described as optional (slide 13), what happens if it is not supplied?
11:29:20 <Xi0s> depends on the authentication protocol being used, but in summary, when the auth server receives the request, it will check your account as to whether 2FA is required (at the account or host or other level) and then act appropriately based on that
11:30:07 <Xi0s> because sshd<->sssd integration is pretty good, when you authentcate using ssh the client will (more or less) be aware as to whether your 2FA is required or not, and inform the user as to whether they have to specify it or not
11:30:37 <oneswig> So you can still have machine-to-machine service accounts which can authenticate in a non-interactive way
11:31:21 <Xi0s> yes, in theory, though we've not seen that in practice yet
11:31:45 <oneswig> How does this play with things like Ansible?
11:31:47 <Xi0s> support for ssh-keys and certificate-based auth is in there too
11:32:27 <Xi0s> in what context? (our deployment of all the actors in the AAI was done by ansible - though we don't use any service provided in this for credential store for ansible)
11:33:02 <oneswig> If I wanted to configure instances deployed with 2FA enabled, for example.
11:33:54 <oneswig> (as a user)
11:34:19 <Xi0s> interesting question - in theory if you are deploying 2FA you should apply it to _all_ instances and services you run as an admin, else you have a weak link that could be exploited
11:34:46 <daveholland> is the 2FA service intended for user-created instances/websites; or did you apply it to the OpenStack API/Horizon too?
11:35:06 <Xi0s> it will be applied to the API/Horizon, but also available for user-created instances
11:35:51 <Xi0s> so, because we have a number of auth-proxies, we can more or less support whatever service the user wants to enable, whether its a web-service or desktop login, etc
11:36:27 <Xi0s> however, this is a manual process, to ensure that you don't end up exposing user credentials to a maliciously crafted service
11:36:50 <Xi0s> (see: OAuth credentials leak from Google Docs ~2017?)
11:37:03 <oneswig> Where is the policy on whether an account requires 2FA stored?  Is it possible to be more selective (eg, using source address)?
11:38:25 <Xi0s> policy is done at the central auth server (freeIPA), and in _theory_ you could have step-up authentication either at a service level, host level, or indeed source address (though the latter is not implemented) - however due to limitations on the LDAP authentication protocol and the _current_ intergration with the web-proxy we have to either force on or force off for an account
11:39:01 <Xi0s> which, for us, meets the requirement of flagging users who have access to potentially sensitive data - however there is more work to be done on these tools for a more versatile service
11:39:56 <Xi0s> the key limitation (for us) is that Keycloak<-->sssd integration doesn't account for password expiry
11:40:08 <oneswig> You mention federated accounts - what can you do for those?
11:41:04 <Xi0s> so I love federation, the idea of not having to deal with user-password enquiries makes me excited - so we do have support for federated identity to a point
11:41:36 <Xi0s> so, allowing github or facebook authentication and linking those identities to our internal identities so that users can authenticate from an external source
11:41:54 <oneswig> EGI Checkin?
11:42:08 <Xi0s> yes, for sure
11:42:18 <Xi0s> however, it would only work for the web-based authentication at the moment
11:42:48 <oneswig> true.  Although I thought Indigo-DataCloud had a solution for the command line - noggin143_?
11:42:49 <Xi0s> while we have plans to support non-web federation, there is no fully versed solution for this (other than X509 certs, which I don't think is the future there)
11:43:23 <oneswig> Do you use levels of assurance in your user/role mappings?
11:43:43 <Xi0s> ldap-facade is also around, but basically there is no clientless solution that does not expose credentials to the "service"
11:44:07 <Xi0s> LoA is not something we use, because we don't have any way of passing that kind of data to the service as yet
11:44:28 <Xi0s> and realistically, unless the service can respond dynamically to the LoA there's no benefit to passing it as a number, though storing it is good for internal audit purposes
11:44:57 <oneswig> Do you also manage posix user ids as part of this service?
11:45:22 <Xi0s> yes, the identity service manages all of those within a set range
11:45:47 <oneswig> nice :-)
11:46:19 <Xi0s> another challenge we are about to face is how we integrate the userID mapping of a service like Lustre in a sensible fashion
11:46:36 <janders> I was about to ask about exactly that...
11:46:39 <janders> do you use manila?
11:46:42 <Xi0s> despite multiple meetings on the topic, i still have yet to gain clarity on how best to manage that
11:47:39 <Xi0s> not something we've explored yet - we're in discussions to have a test-deployment of a 'secure lustre' so we can see how to get things flowing right
11:47:52 <verdurin> janders: that's still to be decided
11:48:19 <Xi0s> personally, I dont see how kerberos tickets dont just solve the problem out of the box, and why an ID translation service is required
11:48:20 <janders> cool!
11:48:42 <oneswig> Xi0s: verdurin: have you seen the work of daveholland and team on Lustre/OpenStack integration: http://bit.ly/2stawpN
11:48:50 <Xi0s> but that could still be my naivety about lustre still
11:49:31 <verdurin> oneswig: yes, I've seen several iterations of their talk
11:49:38 <oneswig> thought so...
11:49:41 <verdurin> One of the reasons why we're looking at Lustre now
11:50:35 <oneswig> Do we have more questions for Xi0s?
11:50:55 <oneswig> Is your configuration documented in more detail than the presentation?
11:51:17 <oneswig> How would you suggest other people replicate your work?
11:51:20 <daveholland> what user feedback have you had so far? (usability, reliability, features, ...)
11:52:19 <Xi0s> very little so far, we're in a horrible transition world right now where we're running the new auth in parallel with our existing auth for the batch HPC, so i dare say if I was to poll now, feedback would be pretty poor
11:53:11 <Xi0s> if you wanted to replicate, we're not in a position to share a playbook just yet, but hopefully one day that could be of value
11:53:24 <daveholland> understood :) I would be interesting in seeing configuration too
11:53:51 <oneswig> Thanks Xi0s, I hope you'll keep us updated if you do get to publish more information
11:54:06 <Xi0s> Sure thing
11:54:26 <oneswig> #topic OpenStack Scientific SIG at ISC?
11:54:41 <oneswig> #link This ISC https://www.isc-hpc.com/
11:55:14 <oneswig> For a couple of years SIG members have successfully run a BoF at Supercomputing in the US
11:55:38 <oneswig> There was a suggestion that something similar ought to be done for ISC (in Frankfurt).
11:55:59 <oneswig> I don't think that can be decided now, (there's about a month before the submission deadline)
11:56:20 <oneswig> I'm interested to know if any SIG members attend?
11:56:57 <janders> I'd give it about 30% probability at the moment. Will see what's the interest in the team though
11:57:14 <daveholland> We sometimes send someone, no decision yet for this year though
11:57:44 <oneswig> I am aware of a few others who usually go.  If there's enough to run a BoF, I'll ask around
11:58:20 <verdurin> I'm closer to going than I have been.
11:58:20 <oneswig> verdurin: do you go to ISC?
11:58:24 <oneswig> ah, snap
11:59:13 <oneswig> OK, I will follow up in 2 weeks
11:59:18 <oneswig> #topic AOB
11:59:32 <oneswig> We are practically on the hour but anything to raise?
12:00:00 <janders> I wonder if you have any observations of RDMA latency fluctutations for small message sizes. Was playing with SRIOV/IB a bit again. Next week?
12:00:24 <janders> I should have some decent benchmarks by then
12:00:30 <oneswig> janders: I'd be interested to hear more on that.  I have an old blog post on bandwidth but didn't cover latency.
12:00:40 <janders> great!
12:00:48 <janders> I mostly focused on bandwidth till now, too
12:00:53 <oneswig> Let's get it onto the agenda, thanks janders
12:00:56 <janders> ok! let's chat more on that next week
12:00:58 <janders> thanks guys!
12:01:03 <oneswig> OK, time to close
12:01:08 <oneswig> #endmeeting