22:01:56 <adrian_otto> #startmeeting Solum Team Meeting
22:01:56 <openstack> Meeting started Tue Jun 24 22:01:56 2014 UTC and is due to finish in 60 minutes.  The chair is adrian_otto. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:01:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:01:59 <openstack> The meeting name has been set to 'solum_team_meeting'
22:02:07 <adrian_otto> #link https://wiki.openstack.org/wiki/Meetings/Solum Our Agenda
22:02:12 <adrian_otto> #topic Roll Call
22:02:15 <adrian_otto> Adrian Otto
22:02:22 <muralia> murali allada
22:02:25 <james_li> james li
22:02:27 <ravips> Ravi Sankar Penta
22:02:28 <aratim> Arati Mahimane
22:02:32 <datsun180b> Ed Cranford
22:02:41 <PaulCzar> Paul Czarkowski
22:02:41 <angu5> o/
22:02:45 <julienvey> Julien Vey
22:02:47 <mkam> Melissa Kam
22:02:49 <devkulkarni> Devdatta Kulkarni
22:03:30 <adrian_otto> is angu5 Mr Salkeld?
22:03:42 <dimtruck_> Dimitry Ushakov
22:04:00 <datsun180b> he is
22:04:02 <angu5> yip
22:04:13 <adrian_otto> welcome everyone
22:04:31 <adrian_otto> you may chime in at any time to be recorded in our attendance
22:04:41 <adrian_otto> #topic Annoncements
22:04:54 <adrian_otto> do any team members have any announcements for us today?
22:05:08 <angu5> mistralclient was released
22:05:11 <angu5> :-)
22:05:16 <adrian_otto> whoot!
22:05:21 <adrian_otto> version 0.0.??
22:05:29 <julienvey> 0.0.4
22:05:42 <adrian_otto> #link https://pypi.python.org/pypi/mistral/0.0.4
22:05:50 <adrian_otto> excellent!
22:05:55 <angu5> but still not sync'd
22:05:55 <datsun180b> oh i was looking for the client there
22:05:59 <adrian_otto> Thanks to those of you helping with that!
22:06:14 <adrian_otto> any other announcements?
22:06:24 <julienvey> small announcement, I found the problem with the spec repo https://review.openstack.org/#/c/102246/
22:06:37 <julienvey> at least I hope so :P
22:06:51 <adrian_otto> julienvey: you mean why zuul is not picking up and doing anything?
22:06:55 <adrian_otto> or some other problem?
22:06:59 <julienvey> yes
22:07:04 <adrian_otto> oh, what did you learn?
22:07:18 <datsun180b> zuul didn't know to
22:07:20 <adrian_otto> aha
22:07:22 <julienvey> we just missed a change in zuul config file
22:07:33 <adrian_otto> my patch for the stackforge setup must have missed that?
22:07:46 <adrian_otto> ok, that makes sense
22:07:52 <adrian_otto> thanks julienvey!
22:08:02 <adrian_otto> ok, let's proceed to action items
22:08:08 <adrian_otto> #topic Review Action Items
22:08:59 <adrian_otto> devkulkarni to work with adrian_otto to document the LP devleopment approach in BP+task+wiki to clearly outline our approach, and how it solves each use case.
22:09:17 <adrian_otto> devkulkarni indicated he plans to discuss this with me this week
22:09:19 <devkulkarni> I have added details to the custom-language-pack bp
22:09:37 <adrian_otto> Oh, good, so we can review that
22:09:49 <devkulkarni> https://blueprints.launchpad.net/solum/+spec/custom-language-packs .. yes I will discuss with adrian_otto and anyone else who is interested
22:09:53 <adrian_otto> should we put that in our parking lot to look at during Open Discussion today?
22:10:01 <adrian_otto> if we have some time left
22:10:08 <devkulkarni> sounds good. we can also revisit that time to time
22:10:21 <adrian_otto> next action item is:
22:10:22 <adrian_otto> ratim to follow up with noorul to plan a solution to https://review.openstack.org/#/c/96928/
22:10:24 <adrian_otto> aratim to follow up with noorul to plan a solution to https://review.openstack.org/#/c/96928/
22:10:42 <adrian_otto> that looks like it's still under review
22:11:01 <aratim> yes, but the issue we had is now resolved
22:11:01 <adrian_otto> angu5: you have a pending question
22:11:14 <angu5> really?
22:11:34 <aratim> angu5 is suggesting we need not store the stack_id in component
22:11:42 <aratim> which is what the patch is for
22:11:45 <angu5> resource_uri
22:11:59 <adrian_otto> https://review.openstack.org/#/c/96928/12/solum/deployer/handlers/heat.py line 158
22:12:55 <angu5> I am not very stressed, we are moving to pipeline
22:13:11 <angu5> not sure how long that is going to live in the current form
22:13:34 <adrian_otto> angu5: you have a -1 vote on this patch. Do you want to move it to a -0 vote?
22:13:47 <angu5> not really
22:14:02 <angu5> I am sure aratim can answer
22:14:09 <adrian_otto> because I don't understand the objection. I am I the only one?
22:14:25 <julienvey> the question has not been answered yet
22:14:33 <julienvey> I understand angus
22:14:35 * ravips doesn't understand the context :(
22:14:37 <angu5> the stack_id is in the stack result
22:14:59 <angu5> so I was asking why we need a different field to store stack_id
22:15:05 <aratim> angu5: do you mean we should remove the stack_id field from component ?
22:15:15 <aratim> since we already have resource_uri
22:15:19 <julienvey> the suggestion is to parse the resource_uri to extract the stack id from it
22:15:26 <angu5> just asking if we can get the stack_id from the stack[id]
22:15:35 <devkulkarni> ravips: background.. we are talking about how to track heat stack that is created for an assembly within solum
22:15:51 <ravips> devkulkarni: ok
22:16:03 <angu5> aratim: if it is not possible - no problem
22:16:20 <aratim> that could be done, but I think requires to undo many things
22:16:25 <aratim> that the patch does
22:16:39 <aratim> the patch intends to add that field to component
22:16:54 <angu5> well, that is the point
22:16:56 <adrian_otto> ok, so I'll consider this action item completed
22:17:06 <angu5> we will barely need the patch
22:17:10 <devkulkarni> aratim: is it fair to say that the bug report said 'lets add stack id to component' and so that approach was followed
22:17:37 <aratim> yes devkulkarni, we had created that bug to add the stack_id
22:17:39 <angu5> bug should declare the problem, not the solution
22:17:39 <devkulkarni> If so, I would blame it on the bug report ;)
22:17:47 <aratim> :)
22:17:58 <devkulkarni> angu5: well, we have not followed that many bugs really
22:18:11 <devkulkarni> s/that/that for/
22:18:47 <devkulkarni> if the action item is completed then are we abandoning the patch?
22:18:53 <angu5> maybe we can move on
22:19:08 <angu5> devkulkarni: the problem is we want the stack_id
22:19:14 <angu5> not name
22:19:21 <adrian_otto> this can be concluded in review the comment stream
22:19:28 <angu5> totally
22:19:32 <devkulkarni> ok. sounds good.
22:19:32 <aratim> angu5: storing the stack_id makes it easier
22:19:37 <aratim> check https://review.openstack.org/#/c/96928/12/solum/deployer/handlers/heat.py
22:19:41 <aratim> line 202
22:20:23 <angu5> we can take this off line
22:20:38 <adrian_otto> #topic Review Tasks
22:21:14 <adrian_otto> note that we agreed in our last meeting that I would call out key priorities as status items, not necessarily from blueprints
22:21:42 <devkulkarni> yes. I remember that.
22:21:42 <adrian_otto> for those who were not present at the prior meeting I'd like your input on this
22:22:22 <adrian_otto> do you have any objections to having standing agenda items to get status on development of key features, similar to what we used to do for the "Review Blueprints" part of the meeting agenda?
22:22:40 <angu5> fine by me
22:22:46 <ravips> nope
22:22:46 <adrian_otto> the agenda should name the Stacker who we expect to give an update
22:23:00 <julienvey> yep
22:23:01 <adrian_otto> and if you will not be present, just identify an alternate to provide an update
22:23:23 <adrian_otto> the purpose for this approach is to keep the whole team informed of our milestones and progress on the most important bits of work we are doing
22:23:52 <adrian_otto> so I'd look for topics like "Pipeline" and "Build Farm"
22:24:26 <angu5> I can give status on pipeline
22:24:34 <adrian_otto> are any of you prepared to offer updates for each of those? I know  our review queue was a bit backed up over the past week pending some critical path items, like the Mistral release
22:24:42 <julienvey> I'll talk about build farm and git after angus
22:24:45 <adrian_otto> ok, angu5, please proceed
22:24:51 <adrian_otto> thanks julienvey
22:24:59 <angu5> so mistalclient released
22:25:18 <angu5> I have the auth sorted (I hope)!
22:25:28 <angu5> the build job is mostly done
22:25:33 <datsun180b> nice
22:25:43 <angu5> I need to do the heat mistral plugin
22:25:51 <angu5> and do final testing
22:25:57 <adrian_otto> auth = chained trusts to allow solum to act upon resources in a heat stack when solum is not being acted upon by an interactive user, without requiring us to store bearer tokens.
22:26:06 <angu5> but hopefully we can get some more patches in
22:26:23 <angu5> adrian_otto: yip
22:26:43 <adrian_otto> for example, when a commit hits a git repo, and a webhook fires to trigger a re-execution of the tasks in the pipeline
22:27:18 <angu5> that is fine as mistral makes the trust at workbook create time
22:27:23 <adrian_otto> angu5: are there any areas where team members can help you?
22:27:36 <adrian_otto> or where that would make sense
22:27:59 <angu5> people are welcome "git review -d "
22:28:03 <angu5> and test it out
22:28:27 <angu5> the heat plugin should be easy
22:28:38 <angu5> I was fighting mostly with auth
22:28:45 <angu5> hopefully that is out the way
22:28:55 <adrian_otto> which functionality will belong in a heat plugin?
22:29:07 <angu5> just updating the stack
22:29:46 <angu5> we need to think about the services / heat template generation
22:30:09 <angu5> maybe make the services/ a bit like the infra/
22:30:23 <angu5> and take a provider template
22:30:37 <angu5> and we then can generate the template
22:30:55 <angu5> (to get 'add ons')
22:31:15 <angu5> I can post an initial spec with some ideas
22:31:32 <angu5> (maybe someone can run with that)
22:31:49 <adrian_otto> thanks angu5. Anything more on Pipelines?
22:31:54 <angu5> that's all
22:32:14 <adrian_otto> ok, julienvey please update the team on Build Farm
22:32:28 <julienvey> So, for the build farm, we have some reviews in progress for the solum part
22:32:45 <julienvey> I was trying to work with Drone but they don't have an API :(
22:33:04 <adrian_otto> maybe Jenkins?
22:33:08 <julienvey> it's in progress on their side so I'll wait a little before choosing another solution
22:33:20 <julienvey> yes, Pierre should start working on Jenkins sson
22:33:23 <julienvey> soon
22:33:24 <angu5> can't we just detect the tests?
22:33:40 <julienvey> it's for the initial config
22:33:44 <julienvey> users/repos...
22:33:56 <julienvey> we identified 3 alternatives
22:34:09 <julienvey> 1/ going with the API
22:34:17 <julienvey> 2/ SSH to the instances
22:34:23 <julienvey> 3/ guest agent
22:34:29 <julienvey> (in order of preference)
22:34:39 <angu5> marconi?
22:35:00 <adrian_otto> that would fit into guest agent I expect
22:35:03 <julienvey> yeah, I have to admit I haven't really look at it yet
22:35:39 <angu5> so install the builder api into the instance
22:35:53 <julienvey> angu5: that's 3/
22:36:06 <julienvey> actually 3 is "Install something on the instance"
22:36:21 <julienvey> whereas 1 and 2 are "Do everything remotely"
22:36:37 <adrian_otto> were these options outlined in the spec proposal you have in review, julienvey?
22:36:43 <julienvey> yes
22:36:54 <angu5> if we install the builder api to the instance, then we can reuse the mistral pluging
22:36:56 <adrian_otto> good, so we can discuss the merits of each there
22:37:15 <angu5> (auth is an issue)
22:37:29 <julienvey> If everyone is ok with that, I am too
22:37:49 <devkulkarni> yes. lets discuss it on the spec
22:37:54 <angu5> ok
22:38:15 <julienvey> https://review.openstack.org/#/c/100539/
22:38:23 <adrian_otto> julienvey: can you give us a quick sense of our plan for securing the build hosts, and the interaction between them and Solum?
22:38:25 <devkulkarni> thanks julienvey
22:38:59 <julienvey> it will depend on what we choose
22:39:16 <julienvey> if 1/ we need to secure the API, but don't know how yet
22:39:39 <julienvey> if 2/ it's just a keypair to add when the instance start so no problem
22:40:02 <julienvey> if 3/ the instance will pick the jobs itself, so again, not a problem
22:40:35 <adrian_otto> if we take option 2, let's be sure that every host has a unique keypair
22:40:41 <angu5> I like a pulling option aproach
22:40:52 <julienvey> :)
22:41:18 <angu5> maybe marconi in a really simple guest agent
22:41:57 <angu5> jenkins is an option too I guess
22:42:09 <julienvey> one question, what's the advantage of marconi over a classic queue system ?
22:42:24 <adrian_otto> julienvey: it has multi-tenancy
22:42:26 <angu5> you over ampq
22:42:33 <angu5> yeah that
22:42:40 <adrian_otto> so you don't need a separate instance of rabbit per tenant
22:43:02 <angu5> but you would need creds on the host
22:43:08 <angu5> (not so nice)
22:43:09 <PaulCzar> yeah, means we can use the openstack provided services rather than trying to run and manage queuing systems per customer
22:43:13 <adrian_otto> you just have a simple queue (or set of queues) per tenant, and SOlum just needs one instance of Marconi to manage everyone
22:43:42 <adrian_otto> and in the case where the cloud provider offers Marconi as a hosted service, you just use that
22:43:53 <julienvey> there's one more question that need answer, is the build farm "per-tenant" or "solum's"
22:43:57 <adrian_otto> and Solum would not need it's own.
22:44:13 <angu5> julienvey: +1 to per tenant
22:44:14 <PaulCzar> if its Solums … then we should just use rabbit …  if its per-tenant then Marconi
22:44:19 <julienvey> Thank you, I'll give a try to marconi
22:44:20 <adrian_otto> julienvey: ideally we should allow either
22:44:29 <angu5> https://github.com/openstack/python-marconiclient/blob/master/examples/claims.py
22:44:46 <adrian_otto> but I think it's more important to *allow* build farms per tenant
22:44:56 <adrian_otto> and perhaps use that as the first iteration
22:45:20 <adrian_otto> and then optimize on that in follow-up iterations to allow a shared build farm with the additional security concerns addressed
22:45:26 <angu5> I need to go take kids to school
22:45:28 <adrian_otto> thoughts on this?
22:45:32 <adrian_otto> thanks angu5
22:45:34 <julienvey> ok, it will be easier to implement "per tenant" I think, so i'm good with it :)
22:45:44 <julienvey> bye angu5
22:45:54 <adrian_otto> just try to keep the shared farm use case in mind while implementing
22:45:58 <PaulCzar> a public cloud would probably want to own the build farm for all for resource consolidation and allowing for cheaper users to not need to run a bunch of VMs for a small application
22:46:02 <adrian_otto> so we don't have to forklift it later
22:46:13 <julienvey> sure
22:46:36 <adrian_otto> ok, thanks julienvey
22:46:46 <adrian_otto> any more details on Build Farm?
22:46:52 <julienvey> no
22:47:15 <PaulCzar> julienvey: I see some git-push BPs from you … are you actively working on that ?
22:47:17 <adrian_otto> ok, any other tasks that team members would like to review together?
22:47:44 <julienvey> yeah, we are working on both build farm and git, it's quite the same mechanisms
22:47:51 <adrian_otto> or suggestions for standing status topics for upcoming meetings?
22:48:05 <PaulCzar> I assume you know that there’s a POC of it already in the contrib ?  - https://github.com/stackforge/solum/tree/master/contrib/example-gitpush
22:48:17 <PaulCzar> in case that can help save time …  it’s very much a MVP
22:48:31 <julienvey> yes, it did, I copied a lot of code from it
22:48:37 <PaulCzar> cool
22:49:07 <adrian_otto> #topic Open DIscussion
22:49:59 <adrian_otto> #link https://blueprints.launchpad.net/solum/+spec/custom-language-packs Custom Language Packs
22:50:07 <adrian_otto> we can take a moment to look over that
22:50:30 <devkulkarni> yes, please do. I will create a spec out of this soon.
22:50:39 <devkulkarni> in the mean while you can add comments to the summary section.
22:50:39 <adrian_otto> looks like this can be submitted as a spec review
22:50:48 <adrian_otto> so I'll look forward to that
22:50:57 <devkulkarni> ok. so
22:51:04 <devkulkarni> the comments can wait for the spec.
22:51:04 <PaulCzar> devkulkarni: it seems to be pretty VM specific …   no docker stuff ?
22:51:22 <devkulkarni> PaulCzar: yes, it identify that custom language packs are VMs.
22:51:28 <devkulkarni> s/it/I/
22:51:35 <devkulkarni> for docker
22:51:52 <devkulkarni> we need to make solum work with Docker lps
22:51:55 <devkulkarni> are we there yet?
22:52:07 <ravips> devkulkarni: whats the scope of language packs? where/how do we store/retrieve service endpoints provided by lp
22:52:09 <devkulkarni> in any case, that can be a separate spec
22:52:26 <PaulCzar> devkulkarni: lp-cedarish is both a VM and a docker LP …  so I’d say we’re there
22:52:30 <devkulkarni> ravips: what do you mean by service endpoints provided by an lp?
22:53:14 <adrian_otto> PaulCzar: +1
22:53:17 <devkulkarni> PaulCzar: lp-cedarish is a framework to support heroku build packs. I would not call it out as a docker-based language pack necessarily
22:53:33 <devkulkarni> lp-cedarish for VM
22:53:36 <devkulkarni> still uses dibs
22:53:43 <devkulkarni> so in what way it is docker based?
22:53:45 <adrian_otto> DiB
22:54:14 <ravips> devkulkarni: lp can be webservice/db + base ubuntu/fedora, how does the user app knows the db/webservice info
22:54:46 <devkulkarni> ravips: definition of a lp imo is bit restricted than that.
22:54:46 <adrian_otto> ravips: that is part of the plan file that is given when the app is registered with Solum
22:55:11 <adrian_otto> the LP is a one-host architecture by default
22:55:12 <PaulCzar> devkulkarni: lp-cedarish only uses DiB if you’re pure VMs
22:55:18 <devkulkarni> lp is base image + packages and libraries that you can use to construct a running assembly
22:55:25 <PaulCzar> if you’re using the docker driver …  it doesn’t touch DiB
22:55:45 <devkulkarni> PaulCzar: yes I know that. and custom lps for VMs I am saying we do exactly that
22:56:07 <devkulkarni> for custom lps for docker, we can do something else, but the bp does not identify that part yet.
22:56:36 <PaulCzar> devkulkarni: okay.  I’m somewhat confused by the BP :)
22:56:51 <PaulCzar> devkulkarni: we can hash it out offline
22:56:59 <devkulkarni> PaulCzar: sure
22:57:05 <adrian_otto> devkulkarni: an LP could be a git repo URL to a repo that contains a Dockerfile
22:57:13 <ravips> devkulkarni: adrian_otto: ok, so LP only provides necessary libs/pkgs and wiring betweeen the app and other services provided by lp is presented in plan file?
22:57:20 <adrian_otto> and the LP image is created on the fly the first time it is used
22:58:03 <devkulkarni> adrian_otto: sure. the mechanism to specify what can be installed as part of the LP creation could be a Dockerfile or a 'prepare' script
22:58:04 <adrian_otto> that would make LP's really lightweight and dynamic
22:58:10 <devkulkarni> sure
22:59:20 <PaulCzar> so I am somewhat of the opinion that a person who wants to write their own LP might want to build the image / docker image themselves and upload it and just specify entrypoints for run, test, etc
22:59:21 <devkulkarni> ravips: yes
22:59:25 <adrian_otto> ok, time to wrap up
22:59:30 <ravips> devkulkarni: ok, thx
22:59:31 <PaulCzar> rather than have solum build it for them
22:59:36 <adrian_otto> thanks everyone for attending
22:59:41 <datsun180b> PaulCzar: ++ to that
22:59:42 <devkulkarni> PaulCzar: have identified that
22:59:52 <devkulkarni> but the bp is with the assumptin
22:59:56 <paulmo_> See you!
23:00:00 <devkulkarni> that someone want to really build a lp within solum
23:00:06 <adrian_otto> any lurgers wanting to be recorded in attenance today? chime in now
23:00:22 <devkulkarni> I can make that clear at the beginning of the BP
23:00:22 <adrian_otto> #endmeeting