21:00:26 <oneswig> #startmeeting scientific-sig
21:00:27 <openstack> Meeting started Tue Apr 17 21:00:26 2018 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:30 <openstack> The meeting name has been set to 'scientific_sig'
21:00:40 <trandles> hello
21:00:46 <oneswig> #link agenda for today - https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_17th_2018
21:00:53 <oneswig> greetings trandles
21:01:02 <oneswig> see my mail?
21:01:19 <trandles> just now, thx
21:01:36 <trandles> I'll give it a read and provide feedback ASAP (after this meeting of course ;) )
21:01:38 <oneswig> excellent!  Michael did a great job last week
21:02:03 <oneswig> I saw you've got some contributions from the Sanger Centre in the UK - are they active users?
21:03:13 <trandles> Yeah, I think Sanger is (Matthew Vernon perhaps?)
21:03:14 <oneswig> I liked your paper from FOSDEM, very useful and informative.
21:03:32 <oneswig> How was the conference?
21:03:32 <trandles> Oh, that's the paper from SC17 btw, I didn't realize FOSDEM republished it
21:03:40 <martial> sorry I am late
21:03:48 <oneswig> Hey martial, no problem!
21:03:52 <oneswig> #chair martial
21:03:53 <openstack> Current chairs: martial oneswig
21:03:59 <trandles> Wish I could have attended FOSDEM but alas...
21:04:00 <oneswig> It's just us and trandles currently
21:04:25 <oneswig> trandles: was Charliecloud presented by Reid at FOSDEM, or nobody?
21:04:27 <martial> that's cool
21:04:46 <trandles> nobody that I know of
21:04:57 <trandles> I just sent Reid the FOSDEM link
21:05:00 <martial> Tim, do you know if Bob and Mike are going to SC18 or OpenStack summit Berlin?
21:05:11 <trandles> I do not know
21:05:24 <trandles> I think that's up in the air at the moment
21:05:35 <martial> I will reach out to them, and offer to redo the OpenStack BoF
21:06:01 <martial> for SC18
21:06:12 <oneswig> I'd be going to Berlin OpenStack instead of SC18
21:06:16 <trandles> oneswig: It looks like someone named Georg Rath gave a scientific containers overview and provided several papers
21:06:18 <trandles> https://fosdem.org/2018/schedule/event/containers_scientific/
21:06:31 <martial> it is very unfortunate that both are exactly at the same time
21:06:44 <trandles> I'm much rather go to Berlin but I have other program requirements for attending SC18 :(
21:07:07 <oneswig> trandles: handy link, thanks - I'll compare notes!
21:07:21 <trandles> martial: is it ok if I send you something small via USPS for you to take to Vancouver on my behalf?
21:07:41 <martial> yes of course
21:08:08 <martial> I will send you my address
21:08:20 <trandles> cool, thx
21:09:08 <oneswig> trandles: got some interest over here in the boot-to-ramdisk idea.
21:09:26 <trandles> the more the merrier
21:09:32 <oneswig> One of the team is looking at ways of updating the RAID controller firmware - it's boot-to-ramdisk or boot-from-volume
21:09:58 <trandles> I have to admit to having a tab opened to the storybook entry and another to the ironic spec wiki page and I haven't had a chance to actually work on it yet
21:10:37 <oneswig> No problem.  I'm on a train tomorrow & thursday - I could get a first draft up if you're snowed under.
21:11:12 <martial> snowed?
21:11:17 <trandles> is there an easy way to collaborate on the spec?  Maybe once it's been submitted?
21:12:00 <oneswig> Absolutely - you commit, I commit, it's tennis...
21:12:03 <martial> Since we have little people, this will have less impact (will post it on the forum/ML too), but Bob asked me to share: Workshop on Container-based systems for Big data, Distributed and Parallel computing (CBDP’2018) https://sites.google.com/view/cbdp18/home Co-located with Euro-Par 2018 http://europar2018.org August 27-28 2018, Torino, Italy
21:12:36 <oneswig> trandles: the main thing is to keep amending the same commit (and making sure we don't tread on each others toes)
21:12:50 <oneswig> martial: looks interesting
21:13:03 <oneswig> Turin is a lovely setting :-)
21:14:21 <trandles> Another trip I wish I could take.  Lucy and I are heading to Singapore and Thailand for two weeks, departing September 3, so Turin the week before is out.
21:14:51 <martial> unless you leave from Italy :)
21:14:57 <oneswig> Some people have all the fun.
21:15:29 <trandles> So, oneswig, what do you have in the way of HPCAC round-up other than the container write-up?
21:15:29 <oneswig> I think I'll be camping that weekend :-)
21:15:47 <oneswig> Another good year for the conference
21:16:03 <oneswig> Plenty of focus this year on containerisation (hence the writeup)
21:16:19 <oneswig> #link http://www.stackhpc.com/the-state-of-hpc-containers.html
21:16:39 <oneswig> But a couple of interesting talks on storage and AI as well.
21:17:04 <martial> pretty cool too
21:17:10 <oneswig> There's a huge focus on applying HPC techniques to AI, a study published by Baidu in particular seems to attract a lot of one-upmanship
21:18:07 <oneswig> A new form factor for NVMe was described - the "ruler" - up to 32 TB on something that looks like (you guessed it) a ruler
21:18:56 <oneswig> it takes up 4x PCIe lanes.  An AMD EPYC has 128 PCIe lanes => up to 1PB of NVMe in 1U (but no PCIe left for high-speed networking...)
21:19:30 <priteau> A new blog post from oneswig, nice!
21:19:49 <priteau> Hi everyone
21:19:49 <oneswig> There was another interesting talk on high-density compute for exascale - liquid cooled
21:20:06 <trandles> Is RDMA to NVMe available in virtual machines?  Do any hypervisors support it?
21:20:12 <oneswig> Hi priteau, welcome!  Yep that's what I've been working on the last couple of days :-)
21:20:53 <martial> Hey Pierre, welcome :)
21:21:02 <priteau> I will read it tomorrow while drinking my morning coffee ;-)
21:21:05 <oneswig> trandles: depends on what level the nvme is exposed.  I'd do it through an SR-IOV RDMA NIC in the instance.  The alternative would be NVME-over-fabrics into the hypervisor then p155ed up the wall by QEMU
21:21:30 <priteau> Sorry I can't stay longer for this meeting, but will read the logs
21:21:44 <oneswig> The issue with the former approach would be exposing a storage network to tenants (potentially)
21:22:21 <trandles> yeah...but it makes me think you could provide burst buffer-like capabilities to VM-based applications
21:22:54 <oneswig> There's a huge, huge potential being opened up around that.
21:24:05 <oneswig> trandles: what's the news on OpenHPC & Charliecloud?  I looked and saw the PR but there's nothing in the package repos yet
21:24:11 <trandles> it's another barrier being knocked down to HPC scale cloud-based scientific workloads
21:24:31 <trandles> So there was a review today for Charliecloud inclusion in OpenHPC
21:24:43 <trandles> I had been told, erroneously, that it was already approved.
21:25:07 <trandles> The review surfaced some questions.  Reid and Michael participated on the call and I haven't had a chance to debrief them
21:25:37 <oneswig> Ah - so not just yet?
21:26:30 <trandles> The 20 second summary is that they think it's a go, but the issue hasn't been updated one way or the other in git
21:27:03 <oneswig> The user namespace stuff seems to be a concern, in that exposes new kernel code paths for probing from unprivileged users.  What's the view on that - are the low-hanging issues all found and fixed to LANL's satisfaction?
21:27:26 <trandles> We believe so.
21:27:33 <trandles> we do mitigate some things
21:28:04 <trandles> a lot of the recent CVE's around user namespace haven't been with user namespace itself, but user namespace made it easier to attack another vulnerability
21:28:35 <trandles> so we disable namespaces not required for Charliecloud (i.e. everything but user and mount)
21:29:25 <oneswig> Makes good sense but perhaps not for the general use case (or OpenHPC)?
21:29:41 <trandles> the biggest one being network...but network is useless for our unprivileged use case because you still need real root privs to add the required network interface to the namespace to make it usable
21:30:33 <trandles> you can't disable namespaces on the Cray kernel btw...
21:30:41 <oneswig> Right - I saw that - unprivileged containers work for this use case but not for most others
21:30:52 <oneswig> Cray let you rebuild the CLE kernel??
21:30:59 <trandles> no, don't have to
21:31:03 <trandles> they're shipping 4.4
21:31:36 <oneswig> Ah OK - so this isn't about unsetting CONFIG_*_NS then?
21:32:13 <trandles> we'd have to recompile and do that, unset the NS stuff, or get rid of the unshare system call, patch clone, etc...
21:32:45 <trandles> namespaces are pretty deeply embedded in the 4.+ kernel
21:33:09 <oneswig> Well you wouldn't want a half measure
21:33:58 <trandles> the RHEL-based and upstream (in 4.something...) kernels include a /proc/ interface for specifying the allowed number of each namespace
21:34:32 <oneswig> Was there something about nested depth as well?
21:35:00 <trandles> not sure about that one
21:35:10 <trandles> look in /proc/sys/user/ btw...
21:35:28 <oneswig> trandles: I'm on a mac
21:35:31 <oneswig> :-)
21:35:52 <trandles> when you get the chance ;)
21:36:20 <jmlowe> whew, finally caught one here after the change to daylight savings
21:36:26 <oneswig> Does OpenHPC already package either Shifter or Singularity?
21:36:31 <oneswig> Hi jmlowe
21:36:38 <trandles> Singularity is in OpenHPC AFAIK
21:36:44 <trandles> Hi Mike
21:36:54 <jmlowe> Hello!
21:37:14 <martial> hey Mike, welcome ;)
21:37:23 <oneswig> There was a question earlier - are you planning on attending either SC2018 or OpenStack Berlin?
21:37:39 <martial> (yep that was me)
21:38:33 <jmlowe> breaks my heart but I don't see how I can justify Berlin over SC18
21:39:05 <martial> want to have a redo of the OpenStack BoF?
21:39:12 <martial> (at SC18?)
21:39:16 <trandles> One last comment on user namespaces.  We took our unprivileged approach because we didn't want to wade into the privilege escalation/de-escalation morass.  We rely on the kernel for security, which we already do implicitly.
21:39:20 <jmlowe> Sure
21:39:40 <martial> jmlowe: do you know if Bob is going to SC18 too?
21:39:44 <oneswig> trandles: I'm with you on that.
21:41:07 <martial> trandles: Met with Ms Lovett last week at the Docker Federal summit
21:41:26 <trandles> oh good
21:42:28 <trandles> yeah, that's another thing that's maybe not widely known yet, but some of us (LANL, NERSC, CSCS) have been talking with Docker since SC17.  We want them to take Charliecloud and Shifter and consider them reference implementations of acceptable HPC container runtimes.
21:43:36 <martial> she spoke about that, and your help. They have made a lot of progress I am told. I have to follow up with her, I would love her to do a couple presentations with us
21:43:43 <oneswig> Seems wise to work from a common ecosystem, specialised where necessary
21:44:26 <martial> not ready to announce at DockerCon in June (I will be there)
21:44:45 <trandles> We'd much rather go to a vendor-supported solution in the long term.
21:45:21 <martial> I think she would be happy/interested to talk about their progress too
21:45:48 <trandles> I'm hoping to meet with her in May when I'm out in the Bay Area for other meetings.
21:45:59 <martial> she is in DC
21:46:35 <trandles> Hrm, Christian said he thought she'd be out west when I am.  I need to email them. ;)
21:47:33 <martial> am sure if you are in the area she can be around
21:49:08 <martial> Stig, should I ask her if she want to present to our group?
21:49:46 <martial> after all if the tech is builtin :)
21:49:48 <oneswig> Seems like a good idea to me - but perhaps when they have something concrete to discuss?
21:50:17 <martial> that's cool :)
21:51:10 <oneswig> A common runtime that meets these requirements (and scales to the simultaneous launch of 10,000 instances) would be excellent.
21:51:48 <oneswig> trandles: on that note, have you looked at CRI-O or similar?
21:52:14 <trandles> Michael has been looking at CRI-O, but I can't speak to where he's at in that effort.
21:52:35 <oneswig> He mentioned something left-field - bproc was it?
21:52:49 <trandles> that I don't konw
21:52:51 <trandles> *know
21:53:14 <oneswig> I may have misheard him
21:54:35 <oneswig> Time is creeping up on us, was there more to cover?  (checks the agenda...)
21:55:56 <oneswig> I'm not sure there is.  I think the action is on martial to submit a BoF for SC2018, correct?
21:56:28 <oneswig> And trandles and I will work on a spec for boot-to-ramdisk
21:56:39 <trandles> I promise
21:56:53 <oneswig> As do I :-)
21:57:16 <martial> yes Sir
21:57:43 <oneswig> excellent :-)
21:57:59 <trandles> martial: if you send me your mailing address I'd appreciate it :)
21:58:48 <martial> Tim, yes am going to do that now :)
21:59:34 <martial> done
21:59:37 <oneswig> Anyone else see this? https://insidehpc.com/2018/04/accelerating-ceph-rdma-nvme/
22:00:12 <oneswig> Bizarrely, iWARP based
22:00:25 <oneswig> OK, time is upon us
22:00:27 <trandles> I had not, thanks for the pointer
22:00:36 <oneswig> going going...
22:00:43 <martial> ohhh really interesting
22:00:49 <oneswig> #endmeeting