21:00:15 <devkulkarni> #startmeeting Solum Team Meeting
21:00:16 <openstack> Meeting started Tue Jun 30 21:00:15 2015 UTC and is due to finish in 60 minutes.  The chair is devkulkarni. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:19 <openstack> The meeting name has been set to 'solum_team_meeting'
21:00:28 <devkulkarni> #topic Roll Call
21:00:35 <gpilz> Gil Pilz
21:00:36 <devkulkarni> Devdatta Kulkarni
21:00:37 <muralia> murali allada
21:00:46 <datsun180b> ed cranford
21:00:53 <devkulkarni> here is our agenda for today: #link https://wiki.openstack.org/wiki/Meetings/Solum#Agenda_for_2015-06-30_2100_UTC
21:01:07 <devkulkarni> hi gpilz, muralia, datsun180b
21:01:14 <muralia> hey all
21:01:29 <dimtruck> Dimitry Ushakov
21:01:41 <devkulkarni> hi dimtruck
21:02:10 <mkam> Melissa Kam
21:02:15 <devkulkarni> hi mkam
21:02:41 <devkulkarni> hi james_li
21:02:45 <james_li> james li
21:03:02 <devkulkarni> #topic Announcements
21:03:47 <devkulkarni> this was from last week: https://review.openstack.org/#/c/190949/
21:04:11 <devkulkarni> Solum is now in the big tent
21:04:26 <devkulkarni> we have follow up actions to change our repository locations etc.
21:04:27 <muralia> woohoo
21:04:28 <mkam> hooray!
21:04:34 <muralia> congrats everyone
21:04:48 <devkulkarni> yes, congratulations to everyone
21:04:56 <gpilz> wow! what a nice tent!
21:05:06 <datsun180b> so roomy
21:05:13 <devkulkarni> gpilz, datsun180b :)
21:05:38 <devkulkarni> once adrian_otto gets back we can sync up with him to figure out the next steps
21:05:53 <devkulkarni> till then we continue with our repositories being in stackforge
21:05:58 <datsun180b> i imagine the first step will be to create the repos in openstack
21:06:03 <devkulkarni> yes
21:06:22 <devkulkarni> I hope there is a checklist of things to do
21:07:05 <devkulkarni> but, should be fine I guess without it too.. there are other experienced folks who can help us with this transition
21:07:18 <devkulkarni> ok, any other announcements?
21:07:52 <devkulkarni> for those who are just joining or joined a bit late, here is the agenda for today: #link https://wiki.openstack.org/wiki/Meetings/Solum#Agenda_for_2015-06-30_2100_UTC
21:08:27 <devkulkarni> ok, I am going to proceed to status updates
21:08:46 <devkulkarni> #topic Status Updates
21:08:59 <devkulkarni> (datsun180b): Update on the recent issues with our gates (and the fixes)
21:09:07 <devkulkarni> floor is yours datsun180b
21:09:29 <datsun180b> 1. oslo.* migration. thanks to doug, we're oslo.* free
21:09:57 <datsun180b> 2. a little bit of mistral's own growing pains, but those appear to be taken care of
21:10:25 <devkulkarni> cool. so the gates are green now, right?
21:10:28 <datsun180b> 3. a change to requirements.txt syntax to help push for better nailing-down of dependencies in the future
21:10:45 <datsun180b> the gates are so green, my app-resource review was merged just before this meeting
21:11:08 <devkulkarni> datsun180b: regarding point #3, would we need to change anything on our end to start using the new requirements.txt syntax?
21:11:23 <datsun180b> so double thanks to dhellmann for his help in the last week, and to renat for diving into mistral
21:11:38 <datsun180b> devkulkarni: we already have, since we're using a newer pbr
21:11:49 <devkulkarni> and lifeless for helping us with the requirements stuff
21:11:51 <devkulkarni> oh okay
21:12:01 <datsun180b> right yes all of that
21:12:23 <devkulkarni> cool.. thanks datsun180b for being our gate watcher
21:12:30 <datsun180b> gate is green, app resource is merged, santa is real, and tomorrow is christmas
21:12:35 <datsun180b> i yield the floor
21:12:39 <devkulkarni> thanks datsun180b
21:12:49 <devkulkarni> next up is muralia
21:12:51 <devkulkarni> (muralia): Updates on delete bug fix
21:13:09 <muralia> Yes, I had twp patches out there this week
21:13:39 <muralia> 1) fix for a bug that was leaving behind log files in a users swift account once an app was deleted.
21:14:03 <muralia> this was leaving behind orphaned files in the users account and they were getting charged for storage.
21:14:10 <muralia> the patch got merged this morning.
21:14:44 <muralia> 2) I implemented support for using the swiftclinet to download operator LPs.
21:15:02 <muralia> there were reliability issues when downloading a large docker image using wget.
21:15:18 <devkulkarni> I remember discussing about second point on the ML
21:15:27 <muralia> Yes, I've made swiftclient the default option for downloading LPs
21:15:34 <muralia> This patch got merged this morning too
21:15:38 <muralia> Thats it for me
21:15:43 <devkulkarni> and the wget option also still exists, right?
21:16:07 <muralia> yes it does. Its an option in the config file
21:16:17 <devkulkarni> cool, thanks for the updates muralia
21:16:22 <devkulkarni> next I can go
21:16:28 <devkulkarni> (devkulkarni): Update on solum app update feature
21:16:36 <devkulkarni> I have been working on app update feature
21:16:52 <devkulkarni> I have a review up: https://review.openstack.org/#/c/194297/
21:17:03 <devkulkarni> and also a spec: https://review.openstack.org/#/c/189929/
21:17:21 <devkulkarni> the spec outlines an update protocol which is implemented in the patch
21:17:34 <devkulkarni> currently I am working through the tests
21:17:49 <devkulkarni> also need to rebase with master
21:18:02 <devkulkarni> should have an updated patch set later today or tomorrow
21:18:24 <devkulkarni> please take a look at the code patch whenever you get a chance
21:18:36 <devkulkarni> thats it for me
21:18:54 <devkulkarni> james_li: you are next
21:18:56 <devkulkarni> (james_li): Update on bash to python conversion work
21:19:40 <james_li> i am working on converting the bash scripts of build-lp, build-app, unittest-app to python code
21:19:51 <james_li> by leveraging docker-py
21:20:36 <james_li> a few additional tasks are also addressed in this work, including:
21:22:02 <james_li> 1. security: adding timeout for running user's scirpts for unittesting and building
21:23:12 <james_li> resource constraint on memory, disk and cpu usage on docker containers by leveraging some new features in recent docker releases.
21:23:30 <devkulkarni> nice
21:23:49 <devkulkarni> james_li: does this stuff work on our vagrant setup?
21:24:03 <james_li> devkulkarni: yes
21:24:06 <devkulkarni> do we need to update the Vagrantfile to install latest version of docker
21:24:09 <devkulkarni> oh, cool
21:24:30 <james_li> nova-docker always try to install the latest docker release
21:24:36 <datsun180b> https://github.com/rackerlabs/vagrant-solum-dev/pull/50 merge that and we'll have what james is using
21:24:46 <james_li> datsun180b: thanks
21:25:12 <james_li> 2. managing docker containers during unittesting and building
21:25:12 <devkulkarni> done
21:25:47 <james_li> which means user can use CLI to interrupt an ongoing unittest or building process
21:25:58 <james_li> thats all for me
21:26:05 <devkulkarni> oh, is that already part of your current set of patches?
21:26:29 <devkulkarni> I did not realize that the CLI stop switch is also being tackled in your patches
21:26:32 <james_li> will be a new patch
21:26:39 <devkulkarni> james_li: ok, makes sense
21:26:49 <devkulkarni> cool, good stuff james_li
21:26:55 <james_li> but current patch paves road for that
21:27:04 <devkulkarni> yep, have seen the current patches
21:27:10 <devkulkarni> they look good
21:27:27 <devkulkarni> thanks james_li
21:27:33 <james_li> sure
21:27:35 <devkulkarni> mkam: you want to go next?
21:27:57 <devkulkarni> dimtruck: how about you?
21:28:05 <dimtruck> we can go
21:28:10 <devkulkarni> ok
21:28:22 <dimtruck> since the last time we spoke, created a set of language packs
21:28:27 <dimtruck> including nodejs and wordpress
21:28:46 <devkulkarni> do you have links to their git repos?
21:28:51 <dimtruck> working through a ci/cd workflow with a side application and using solum to deploy
21:28:52 <dimtruck> yup!
21:28:54 <dimtruck> one sec
21:29:04 <mkam> https://github.com/rackspace-solum-samples/solum-languagepack-wordpress
21:29:05 <mkam> https://github.com/rackspace-solum-samples/solum-languagepack-nodejs
21:29:12 <dimtruck> bam!
21:29:32 <dimtruck> https://github.com/rackspace-solum-samples/solum-ui << this is the ui
21:29:38 <dimtruck> err, sample application
21:30:00 <devkulkarni> what lp are you using for the solum-ui?
21:30:05 <dimtruck> nodejs
21:30:19 <devkulkarni> *looking at the nodjs lp*
21:30:24 <dimtruck> the language pack installs node 0.10.38, npm 1.2, grunt, and karma
21:30:40 <dimtruck> https://github.com/rackspace-solum-samples/solum-languagepack-nodejs/blob/master/bin/build.sh
21:30:56 <dimtruck> npm 2.10*
21:31:09 <devkulkarni> dimtruck: what is the base image (FROM buildpack-deps:jessie)
21:31:16 <dimtruck> validating that the workflow is clean
21:31:16 <devkulkarni> ?
21:31:25 <dimtruck> that's jessie - debian 8
21:31:34 <dimtruck> the latest and greatest!
21:31:41 <devkulkarni> ok, but why buildpack-deps?
21:31:54 <devkulkarni> is it coming from one of the heroku style buildpacks?
21:31:57 <dimtruck> nope
21:32:00 <dimtruck> docker hub
21:32:23 <devkulkarni> I see
21:32:25 <dimtruck> it's another version of https://registry.hub.docker.com/u/nodesource/jessie/
21:32:35 <dimtruck> i could have used this one just as well :)
21:33:15 <dimtruck> here's the buildpack-deps repo: https://registry.hub.docker.com/u/library/buildpack-deps/
21:33:37 <devkulkarni> cool, thanks for the links
21:34:09 <devkulkarni> dimtruck: would these lps work in vagrant setup?
21:34:18 <dimtruck> i don't see why not
21:34:19 <datsun180b> why shouldn't they
21:34:20 <devkulkarni> *need to try these out*
21:34:24 <dimtruck> i have not tested them in vagrant
21:34:25 <dimtruck> right
21:34:30 <devkulkarni> ok
21:34:38 <devkulkarni> cool, awesome stuff dimtruck
21:35:09 <devkulkarni> ok, any more updates from the team?
21:35:25 <dimtruck> before next week, we'll work to send out an operations guide
21:35:29 <dimtruck> for solum deployments
21:35:39 <devkulkarni> dimtruck: oh, that will be awesome!!
21:35:41 <dimtruck> i can update the team of its status next meeting
21:35:50 <devkulkarni> there was a request for that on the irc
21:35:57 <devkulkarni> dimtruck: sounds good
21:36:24 <devkulkarni> #topic Review action items
21:36:41 <devkulkarni> There were no action items from last meeting
21:37:09 <devkulkarni> moving on to the next topic
21:37:16 <devkulkarni> #topic Discussion topics
21:37:27 <devkulkarni> (devkulkarni): Plugin architecture for deployer
21:37:48 <devkulkarni> so first item that I want to discuss is building a plugin architecture for Solum
21:38:26 <devkulkarni> this stems from the need to be able to use different kinds of heat resources in different operational environments.
21:39:02 <devkulkarni> for instance, in rackspace environment we want to use the rackspace cloud load balancer resource, whereas on vagrant we want to use neutron
21:39:45 <devkulkarni> with the plugin architecture, we should be able to choose such different heat resources based on the operational environment of solum
21:39:57 <james_li> jinja for HOT templating?
21:40:02 <devkulkarni> any thoughts on how we might start moving in that direction?
21:40:10 <devkulkarni> james_li: good point
21:40:25 <devkulkarni> but the problem is more than just choosing the right resource in the HOT
21:40:36 <devkulkarni> we may also need to selectively choose the code itself
21:40:48 <james_li> yeah like flavor ...
21:41:05 <devkulkarni> for instance, in the app update protocol, I modify the HOT in the code
21:41:32 <devkulkarni> that code will be different based on whether the environment is rax or something else
21:41:51 <james_li> solum should keeps a set of base HOT templates, and we can plugin stuff to it on the fly
21:41:59 <devkulkarni> regarding flavor (and other such attributes) — we should be able to control them via CLI
21:42:06 <devkulkarni> james_li: yep
21:42:49 <devkulkarni> I know that heat has a plugin architecture
21:42:49 <muralia> devkulkarni: are we going to add these new options to the CLI as flags?
21:42:58 <devkulkarni> muralia: good question
21:42:59 <james_li> the pluggable stuff could also be something from user's input
21:43:06 <muralia> I'm concerned that the CLI will have way too many options and flags.
21:43:07 <devkulkarni> probably not as flag
21:43:11 <devkulkarni> yep
21:43:26 <devkulkarni> I am thinking that we will need to enhance the app structure
21:43:26 <muralia> ok. this seems like an advanced option, so we can put it in the planfile
21:43:31 <devkulkarni> that datsun180b is defining
21:43:37 <muralia> ok
21:43:37 <devkulkarni> appfile
21:43:45 <datsun180b> more like "has defined"
21:43:52 <datsun180b> i'm excited it merged today
21:44:14 <devkulkarni> james_li: yes, some of the parts of the pluggable code could come from user input, such as the flavor
21:44:23 <devkulkarni> datsun180b: yeah, it finally merged
21:44:27 <devkulkarni> congrats :)
21:45:02 <datsun180b> the format should be clearer than the planfile format
21:45:24 <datsun180b> and i don't think there's much translation between what's in the sample and what the api expects
21:45:26 <devkulkarni> so about pluggable architecture.. any thoughts on how to actually go about breaking our code? james_li?
21:45:32 <james_li> devkulkarni: we don't need to worry too much about supporting user input now, we need to build some fundamental things. how to support user experience is another issue
21:45:48 <devkulkarni> james_li: agree
21:46:34 <james_li> devkulkarni: as far as I can think now, it probably just a deployer change
21:46:39 <devkulkarni> based on your recent work on lp_handlers, are there any insights for us on the pluggability front stemming from your experience there?
21:46:59 <devkulkarni> yes, it will be only limited to deployer for now
21:47:35 <devkulkarni> I was going to look into how heat has designed their plugin architecture
21:47:40 <james_li> yeah, I think we can target for it after your current work is done.
21:47:49 <devkulkarni> randallburt might be a good person to reach out to in this regard
21:47:58 <james_li> it makes sense to me
21:48:12 <devkulkarni> james_li: yep, it will be a follow up work
21:48:35 <devkulkarni> ok, let me take an action item to pursue this with randallbut
21:49:19 <devkulkarni> #action devkulkarni to check how to make solum's heat deployer pluggable to support different kinds of resources in different operational environments
21:49:53 <devkulkarni> the next item on discussion topics that I had was: (devkulkarni): Coordination between multiple deployers
21:50:04 <devkulkarni> so here is some background on this
21:50:45 <devkulkarni> in an operational setting with multiple deployers, it is possible that consecutive app actions are taken up by different deployers to work on
21:51:08 <devkulkarni> we have faced race conditions when there is no coordination between the deployers
21:51:42 <devkulkarni> example of a race condition is when two instances of apps remain even when an app is deleted
21:52:03 <devkulkarni> currently, we deal with such issues by using the Solum database as the point of coordination
21:52:38 <devkulkarni> what I wanted to discuss is — is using db for this purpose enough?
21:53:05 <devkulkarni> or do we want to investigate going the route of distributed coordination protocols?
21:53:26 <devkulkarni> the latter will increase complexity of our code a whole lot
21:53:35 <muralia> does the db option implemented now give us what we need?
21:54:26 <devkulkarni> it does, as long as we don't have install of solum that uses different dbs
21:54:34 <gpilz> this seems like an area that is only going to become a problem if solum becomes popular
21:54:55 <gpilz> i would suggest focussing on more near-term issues
21:55:14 <datsun180b> one of those good problems to have
21:55:15 <gpilz> DB coordination will hold up for a while
21:55:35 <devkulkarni> gpilz: from operational pov this may become an issue sooner, if an operator decided so have a single solum API with data distributed across DCs
21:55:55 <devkulkarni> agree on DB coordination should hold up for now
21:56:11 <devkulkarni> ok, we can revisit this later
21:56:18 <james_li> devkulkarni: what do we need to figure out between two app updates to ensure there is NO race condition?
21:56:47 <devkulkarni> james_li: basically the actions of the deployers need to be ordered
21:56:47 <james_li> timestamps?
21:57:06 <devkulkarni> yeah, they could help.
21:57:15 <devkulkarni> in fact, we are utilizing timestamps right now
21:57:29 <devkulkarni> but our granularity is only seconds
21:57:40 <james_li> the classic Lamport protocol uses timestamps to ensure the cause-effect relationships between events.
21:57:48 <devkulkarni> so if there are updates within a second then we are still going to run into a race
21:58:24 <muralia> should we just change the granularity then?
21:58:32 <devkulkarni> james_li: yes, that is correct. but the pratical implementation may have limitations like above
21:58:46 <devkulkarni> in which case, we would need to use something else to order the requests
21:58:59 <devkulkarni> muralia: yes, we should do that.. I think there is bug for that
21:59:08 <muralia> ok
21:59:13 <james_li> datsun180b suggested to use incremental
21:59:25 <james_li> auto-increase
21:59:25 <devkulkarni> ok, we are at the almost at the end of our meeting time
21:59:33 <devkulkarni> lets continue in solum
21:59:43 <devkulkarni> james_li: what is auto-increase?
21:59:49 <james_li> #solum
21:59:52 <devkulkarni> james_li: ok
22:00:02 <devkulkarni> ok, thanks all for joining today
22:00:08 <devkulkarni> have a happy long weekend
22:00:12 <devkulkarni> #endmeeting