15:00:24 <zhipeng> #startmeeting openstack-cyborg
15:00:25 <openstack> Meeting started Wed Jan  3 15:00:24 2018 UTC and is due to finish in 60 minutes.  The chair is zhipeng. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:28 <openstack> The meeting name has been set to 'openstack_cyborg'
15:00:51 <zhipeng> meetbot still on vacation ?
15:01:11 <Li_Liu> Happy New Year guys!
15:01:17 <Yumeng__> Hi
15:01:23 <jkilpatr> happy new year
15:02:05 <openstack> zhipeng: Error: Can't start another meeting, one is in progress.  Use #endmeeting first.
15:03:25 <openstack> zhipeng: Error: Can't start another meeting, one is in progress.  Use #endmeeting first.
15:03:33 <jkilpatr> meetbot seemed to have worked the first time?
15:03:49 <zhipeng> ah looks like my network's problem
15:04:27 <zhipeng> who do we have here?
15:04:32 <zhipeng> #topic Roll Call
15:04:40 <zhipeng> #info Howard
15:04:49 <jkilpatr> #info Justin
15:05:30 <mpaolino> #info Michele
15:06:51 <zhipeng> and also #info liliu and yumeng
15:07:07 <zhipeng> #topic Queens Release Prep
15:07:26 <zhipeng> folks entering Jan we are officially in the mode for release prep
15:07:28 <vipparthy> here is Vipparthy
15:07:42 <zhipeng> so we need to accelerate our progress
15:08:08 <zhipeng> let's run through the open patches, see what is still missing and what could be closed
15:08:18 <zhipeng> #link https://review.openstack.org/#/q/status:open+project:openstack/cyborg
15:08:34 <zhipeng> #info code freeze is Feb 5
15:08:57 <zhipeng> #info 1. Internal API spec
15:09:01 <zhipeng> #link https://review.openstack.org/525598
15:09:12 <zhipeng> jkilpatr any update on this one ?
15:09:50 <jkilpatr> zhipeng, waiting on feedback, I'm pretty happy with it.
15:10:11 <zhipeng> i think we should keep it open as we flash out the api/conductor/agent extension update ?
15:10:29 <jkilpatr> sounds good to me, talk about what you want in the review.
15:11:15 <zhipeng> no problem, I think we should go back to the spec after we land zhuli's API patch
15:11:36 <zhipeng> ok next up
15:11:56 <zhipeng> #info 2. CRUD api extension
15:12:07 <zhipeng> #link https://review.openstack.org/527396
15:13:03 <Li_Liu> #info Li
15:13:22 <zhipeng> TL; DR for this patch is that
15:13:52 <zhipeng> for Queens we will need a working API/DB that implements the resource provider data model
15:14:06 <zhipeng> this patch is the attempt to achieve that
15:14:16 <jkilpatr> I'll review today
15:14:27 <crushil> zhipeng, The generic driver depends on this patch. Once this patch merges, I can add the implementation for the CRUD to the generic driver too
15:14:38 <crushil> And it doesn't have unit tests yet
15:14:45 <zhipeng> so I think as I mentioned before, the implementation should align with jkilpatr's internal API spec
15:15:15 <zhipeng> crushil ya I saw your comment
15:15:36 <zhipeng> I will work with zhuli and let's land this patch this week
15:15:55 <crushil> I have already added create functionality to the generic driver
15:16:00 <jkilpatr> I need to look at my own commit again, it's been a while.
15:16:09 <zhipeng> i know :P
15:16:34 <zhipeng> okey without further a due
15:16:42 <zhipeng> #info 3. generic driver
15:16:47 <zhipeng> #link https://review.openstack.org/525057
15:16:56 <zhipeng> crushil take away
15:17:51 <crushil> So, as I already mentioned this patch depends on https://review.openstack.org/527396 and once that patch merges, I can add functionality for the other use cases for Cyborg.
15:19:20 <zhipeng> we also need unit test for this one right ? I mean later
15:19:29 <crushil> yup
15:20:27 <zhipeng> how generic should we expect for the driver when we release it ? Can we have very simple demo on NVIDIA GPU ?
15:21:07 <crushil> That's the goal
15:21:23 <crushil> We can do a simple POC at the OS summit
15:21:43 <zhipeng> cool
15:22:06 <zhipeng> ok now let's move on two the relative two new specs
15:22:19 <zhipeng> #info 4. FPGA data modeling spec
15:22:29 <vipparthy> can you please plan for Demo online, so we can join and view functionality , Thank you
15:22:29 <zhipeng> #link https://review.openstack.org/526559
15:22:50 <zhipeng> this is actually the spec that zhuli is implementing in his patch
15:23:12 <zhipeng> vipparthy we could discuss that at Dublin PTG for sure
15:23:27 <zhipeng> Li Liu you there ?
15:23:33 <Li_Liu> yup
15:23:36 <vipparthy> Thank you Zhipeng
15:23:37 <zhipeng> Li_Liu
15:24:02 <zhipeng> could you briefly introduce the spec ?
15:24:07 <Li_Liu> sure
15:24:13 <zhipeng> so that everyone gets up to speed
15:25:57 <Li_Liu> Basically, the spec introduces a way to describe hierarchy accelerator structure
15:26:08 <Li_Liu> One of the most typical usage is FPGA
15:26:28 <Li_Liu> We plan to add a new table called Deployables
15:27:16 <Li_Liu> It represent any accelerators that can be discovered/deployed
15:27:54 <Li_Liu> The key is it has 2 pointers, parent_id and root_ir
15:28:02 <Li_Liu> root_id*
15:28:27 <Li_Liu> parent_id points to the parent Deployable row
15:28:40 <Li_Liu> root_id points to the root Deployable row
15:29:16 <Li_Liu> This way, we can form nested tree structures
15:29:28 <Li_Liu> For instance, in the case of FPGA
15:30:02 <Li_Liu> Each FPGA can have multiple Physical Functions
15:30:29 <Li_Liu> Each Physical Functions can have multiple Virtual Functions
15:32:13 <Li_Liu> Also, we have a Attribute table which stores k-v pairs referencing back to the Deployables table
15:32:45 <zhipeng> is this aligned with the nested resource provider spec ?
15:32:50 <Li_Liu> this way, we can track arbitrary type of attributes for generic types
15:32:55 <zhipeng> (which expected to be done in Queens)
15:33:20 <Li_Liu> It does, actually
15:33:40 <zhipeng> cool
15:34:00 <zhipeng> thx for the intro, everyone plz help review the spec
15:34:08 <Li_Liu> the Deployable is counterpart to resource providers in NOVA
15:34:13 <zhipeng> zhuli will also reflect the changes in the spec on the fly
15:34:19 <Li_Liu> Thanks a lot guys
15:35:30 <zhipeng> moving on to the next
15:35:43 <zhipeng> #info 5. Intel's FPGA device driver
15:35:56 <zhipeng> #link https://review.openstack.org/530720
15:36:23 <zhipeng> shaohe_feng_ could you help introduce the spec ?
15:38:45 <shaohe_feng_> zhipeng: yes
15:39:43 <shaohe_feng_> zhipeng: this spec is going to provide the fpga information to the Li_Liu's spec
15:40:27 <shaohe_feng_> zhipeng: and support a program interface for FPGA
15:41:06 <shaohe_feng_> When cyborg agent starts or does resource check periodically, the cyborg FPGA driver should enumerate the list of the FPGA devices, and report the details of all available FPGA accelerator on the host. Such as BDF, PID, VID, IMAGE_ID, PF/VF type.
15:41:33 <zhipeng> this has any impact on your work @crushil ?
15:41:37 <shaohe_feng_> When user uses empty FPGA regions as its accelerator, cyborg agent should call driver's program() interface. Cyborg agent should provide BDF of VF, and local image path to the driver. More details ref
15:44:05 <crushil> I'm not sure
15:44:17 <crushil> I need to look at the spec
15:44:55 <shaohe_feng_> zhipeng: I'm not familiar with other vendor's FPGA. Not sure we need a virtual FPGA driver interface.
15:45:58 <zhipeng> crushil okey
15:46:22 <zhipeng> shaohe_feng_ I think Li_Liu could help you with the insight of other vendor implementation
15:46:23 <shaohe_feng_> o, sorry. the driver should can program any FPGA regions not only empty region
15:46:51 <shaohe_feng_> zhipeng: that's greate
15:47:10 <shaohe_feng_> will update the spec.
15:47:26 <zhipeng> cool
15:47:39 <zhipeng> okey then we have the last non-priority patch
15:47:49 <zhipeng> #info 6. SPDK driver code
15:48:01 <zhipeng> #link https://review.openstack.org/513704
15:48:02 <shaohe_feng_> the agent decide which region need to be re-program.
15:49:03 <zhipeng> the SPDK code is developed by people in my team, so if we could squeeze it in Queens then it would be great, if not it is also not an issue
15:49:13 <zhipeng> will ping you guys when it is ready
15:49:23 <zhipeng> #topic Queens Priorities
15:49:40 <jkilpatr> proof of concept?
15:49:40 <zhipeng> I would suggest the following priorities for this month
15:49:53 <zhipeng> jkilpatr ya like that
15:50:10 <zhipeng> we have patches need to be merged in spdk in the first place to make it work
15:50:27 <zhipeng> but it is non-priority so no hurry on that one
15:50:55 <zhipeng> i will handle that quietly on my side :P
15:51:05 <zhipeng> okey back to priority suggestion
15:51:30 <zhipeng> #info suggestion 1: land LiLiu and zhuli's patch this week
15:52:00 <zhipeng> i know there are a lot of reviews to be done, but let's try to crunch it this week
15:52:17 <zhipeng> so that crushil and justin's work could move ahead
15:52:58 <zhipeng> #info suggestion 2: we land justin's and shaohe's spec by the end of next week
15:53:38 <zhipeng> meaning that once we agreed upon the API layer implementation this week, we could crunch out the conductor/agent/driver interface pretty much next Wed
15:54:14 <zhipeng> and we should be okey to have a full picture for the internal API spec, as well as shalhe's driver spec to align with it
15:54:31 <Li_Liu> sounds great
15:54:42 <zhipeng> #info suggestion 3: at the moment we keep the vendor driver code in-tree since there are not a lot
15:54:43 <jkilpatr> I'll give it a go.
15:54:51 <jkilpatr> agreed
15:55:00 <jkilpatr> no reason to make things more complicated until it becomes an issue.
15:55:07 <zhipeng> yep
15:55:29 <zhipeng> anyone else comment on my three suggestions ?
16:02:52 <zhipeng> crushil ?
16:03:18 <crushil> 3
16:03:43 <crushil> We should keep it in-tree
16:03:53 <crushil> We are not in that position yet to move it out of tree
16:04:03 <vipparthy> zhipeng, who can help me to identify driver specs to be written for different FPGA's in our team ?
16:04:06 <zhipeng> crushil exactly
16:04:14 <crushil> And apologies for delays. I'm in a conference right now
16:04:27 <zhipeng> no problem :)
16:05:09 <zhipeng> vipparthy you could go over the log, there are info points that could help you identify it
16:06:28 <zhipeng> so if there are no other issues, I will close the meeting on time-ish :P
16:06:43 <zhipeng> let's go to work and try to deliver an awesome first release
16:07:05 <crushil> \o/
16:09:23 <zhipeng> #endmeeting