15:01:19 <zhipeng> #startmeeting openstack-cyborg
15:01:19 <openstack> Meeting started Wed Jun 14 15:01:19 2017 UTC and is due to finish in 60 minutes.  The chair is zhipeng. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:22 <openstack> The meeting name has been set to 'openstack_cyborg'
15:01:37 <zhipeng> #topic BP discussion
15:01:55 <zhipeng> #info the api spec has been updated to reflect the latest comment
15:02:05 <zhipeng> #link https://review.openstack.org/#/c/445814/
15:03:14 <zhipeng> plz help review it, would be nice to merge it by the end of this week
15:04:22 <zhipeng> and then I think I will take over the cyborg-nova interaction spec
15:06:08 <zhipeng> any thoughts on the api spec so far ?
15:06:13 <crushil> zhipeng, Will do
15:06:26 <crushil> Need to look at the API spec
15:06:58 <zhipeng> okey no problem
15:07:05 <zhipeng> moving on to the next topic
15:07:12 <zhipeng> #topic code development process
15:07:21 <zhipeng> crushil how's the code going ?
15:08:22 <crushil> It's going. I have pushed the WIP up for the driver. I will fill out my code pending your work and jkilpatr's work
15:08:54 <jkilpatr> morning everyone.
15:09:10 <jkilpatr> Conductor is very stubby working on getting it hooked up to rabbit today.
15:09:11 <zhipeng> morning :)
15:09:45 <zhipeng> crushil have you submitted the code yet ?
15:10:10 <jkilpatr> yup he did on sudnay
15:10:15 <jkilpatr> sunday*
15:10:19 <crushil> Yup
15:10:38 <crushil> Morning jkilpatr
15:10:44 <zhipeng> oh did not see that yet
15:11:13 <zhipeng> well there is a thought when I update the api spec today
15:11:23 <jkilpatr> I have a patch up for an internal accelerator object because I need one in both the conductor and the agent and didn't want to duplicate code.
15:11:49 <zhipeng> that the generic driver should morph into a standalone library for local and remote accelerator attach/detach
15:12:19 <jkilpatr> so that means we have attach/detach library and then the pci passthrough driver build ontop of it?
15:12:23 <zhipeng> like os-brick
15:12:35 <jkilpatr> ok no idea what os-brick is
15:13:31 <zhipeng> yes, just import the lib and use it
15:13:31 <zhipeng> so the background story is
15:13:31 <zhipeng> back in the days
15:13:31 <zhipeng> every cinder driver has to do attach/detach on its own
15:13:39 <zhipeng> either iSCSI, FC, or what have you
15:14:11 <zhipeng> then cinder project extract that part of code out and formed a new sub project called os-brick
15:14:23 <zhipeng> bascially provide a common lib for the attach/detach
15:14:39 <zhipeng> the benefit is that, you could use it even Nova is not involved
15:14:54 <zhipeng> for example attach a volume for baremetal host
15:15:45 <zhipeng> and cinder driver could just include the lib and call the functions they need
15:17:36 <crushil> zhipeng, So, are you proposing we should move our generic driver code eventually into something like OS-brick?
15:17:59 <zhipeng> yes that is my current thinking
15:18:12 <zhipeng> in the near future
15:18:46 <crushil> I don't understand why though?
15:19:27 <jkilpatr> for the sake of abstraction
15:19:38 <jkilpatr> instead of having to attach.nvidiagraphicsdriver
15:19:49 <zhipeng> coz currently you are defining the basic operations for the drivers right ?
15:19:53 <jkilpatr> we would do attach(nvidiagraphics driver object)
15:20:15 <zhipeng> and at the end of the day, most of the attach/detach operstions will be the same for most of the drivers
15:20:24 <zhipeng> yep
15:20:51 <jkilpatr> too much abstraction is counterproductive and I see that in a lot of openstack projects, but this level makes sense.
15:21:40 <crushil> Ok, we can talk more about it as the design progresses
15:21:58 <jkilpatr> I find it easier to write somthing monolithic then I understand how to break it out easier
15:22:08 <zhipeng> yes of course :)
15:22:34 <zhipeng> haha
15:23:07 <zhipeng> enough of the microservice hype
15:24:35 <jkilpatr> tech has a bad habit of coming up with ideas that are great in very specific situations then applying them to everything and realizing their bad ideas to follow blindly
15:24:42 <jkilpatr> in general do what makes sense.
15:25:09 <jkilpatr> Anyways back on topic. RPC vs messaging vs direct linking
15:25:19 <zhipeng> indeed
15:25:47 <jkilpatr> rpc places the responsibility for error handling into the caller, so for example if I where to have the conductor call attach via rpc then the conductor would need to handle possible errors (at least the way I understand it)
15:26:08 <jkilpatr> in general I think API -> conductor should be RPC
15:26:19 <jkilpatr> and conductor -> agents should be message passing
15:26:32 <jkilpatr> and then agents -> drivers should be just direct linking (we import the python modules on the same machine)
15:27:06 <zhipeng> i think it makes sense on the agent->driver side
15:27:08 <jkilpatr> adding a messaging/rpc server to every driver is just bad design, what if you load 100 drivers? do you just have 100 new servers that need to run and listen
15:27:20 <zhipeng> but considering scaling problem, conductor should still rpc to agent ?
15:29:45 <jkilpatr> so rpc would be less messaging load sure, but then what if the command fails, we have to put the error handling into the conductor
15:29:51 <jkilpatr> where I'd argue it really does not belong
15:30:23 <jkilpatr> I guess we could have a wrapper to handle all failures on the agent. but I'm still not super happy with the idea.
15:30:33 <jkilpatr> opinions on this are appreciated.
15:31:28 <zhipeng> as far as I know, when we deploy cyborg, conductor is more than likely on a different node than the agent
15:31:40 <zhipeng> so even taking into consideration of the error handling
15:31:50 <zhipeng> RPC would still be a preferred choice
15:51:11 <jkilpatr> ok then.
15:52:10 <jkilpatr> I still need to figure out how oslodb works so that the conductor can save stuff out.
15:53:07 <zhipeng> right :)
15:55:26 <jkilpatr> so we should probably define when we want to merge these components, do we want to go all the way to first POC with everything in reviews?
15:56:15 <zhipeng> i think i could try
15:57:55 <jkilpatr> it's going to make all up tests a little frustrating (trying to get all the components together) but I'm fine with that.
15:59:50 <zhipeng> well we definitely will be expecting failures :P
15:59:52 <zhipeng> for sure
16:37:13 <ttk2[m]> zhipengh: did you forget to end the meeting again?
20:53:22 <zhipengh[m]> Ahhh shit ...
02:17:30 <zhipeng> #link https://twitter.com/nopainkiller/status/876941606846320642
02:17:51 <zhipeng> fyi I just did a OVS Orbit podcast with Ben Plaff at LinuxCon Beijing
02:18:09 <zhipeng> on OVS and Cyborg, the episode will likely be online around early July
02:18:18 <zhipeng> would be a good publicity for Cyborg :)
02:18:43 <zhipeng> and I also update our master slide deck a bit to reflect a possible relationship between Cyborg and OVS
02:18:50 <zhipeng> as well as Cyborg and Cillium
02:21:03 <zhipeng> #link https://docs.google.com/presentation/d/1xXjqlMaXjXe7mn-iwoETGcTPnTbqCDfpaT6yqlZ8srE/edit?usp=sharing
02:21:11 <zhipeng> and if i remember correctly ..
02:21:14 <zhipeng> #endmeeting