15:01:41 <jimbaker> #startmeeting craton
15:01:43 <openstack> Meeting started Mon Sep 26 15:01:41 2016 UTC and is due to finish in 60 minutes.  The chair is jimbaker. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:47 <openstack> The meeting name has been set to 'craton'
15:01:55 <jimbaker> #topic rollcall
15:02:10 <jimbaker> syed_, hi, hope you had a nice weekend
15:02:34 <jimbaker> anything you need to ping me on?
15:02:46 <syed_> jimbaker: hi, yes it was good, i hope you had a good weekend too
15:03:03 <syed_> not really for now, trying to get the keystone thing fixed, since i need to get it going for horizon peoples
15:03:23 <syed_> was able to get authentication from the server, but when i do the client api call, i get an error
15:03:23 <jimbaker> there's beautiful fall foliage in the high country of colorado right now
15:04:23 <syed_> ohh great
15:04:38 <syed_> yeah fall is beautiful, i was in chicago over the weekends visiting my aunt !
15:05:16 <jimbaker> it was good to see the the colors of the aspen - mostly yellow - as well as other foliage against the first snow of the season, at around 3000m
15:05:47 <syed_> woah, thats sounds really cool
15:05:55 <syed_> but well, there is no fall in texas
15:06:01 <syed_> haha
15:06:02 <jimbaker> syed_, sounds good about keystone work
15:06:43 <jimbaker> getting the middleware setup, plus dockerization of supporting services, is going to very helpful
15:07:16 <syed_> yes it indeed will be, so i am trying to grasp as much as possible to get these things going asap
15:07:24 <syed_> i guess the client side of code is going pretty well
15:07:28 <syed_> we got hosts and regions up
15:07:32 <syed_> cell is out there
15:07:37 <syed_> then we will be left with variables
15:07:49 <syed_> which i have briefly started taking a look into
15:08:26 <jimbaker> syed_, yeah, first parts of var support should be easy, but absolutely essential
15:09:43 <syed_> i saw reviews of ansible integration, they look good
15:10:11 <jimbaker> syed_, that's an important part of the variable work as well
15:10:22 <syed_> hmm ..
15:10:35 <jimbaker> i'm glad we decided that we didn't need to do more substantive work on inventory modeling...
15:10:39 <jimbaker> at least at this time
15:10:43 <syed_> yeap
15:11:48 <jimbaker> my work on the two level task scheduling is going to need for sulo to finish up on the parent hierarchy
15:12:17 <jimbaker> (i might take that bit of code back from him at this point...)
15:12:32 <jimbaker> syed_, on the client side with respect to variables
15:12:51 <sigmavirus> o/
15:13:00 <jimbaker> besides the ansible inventory endpoint, plus the json export script from ansible
15:13:09 <jimbaker> sigmavirus, hi!
15:14:40 <jimbaker> looks like a small meeting today, so we can use this to mostly discuss dev issues
15:14:56 <syed_> +1
15:15:32 <jimbaker> the only two admin/community things outstanding are
15:16:14 <jimbaker> 1. sigmavirus started the process of getting a logbot set up for #craton
15:16:24 <jimbaker> s/two/few
15:17:06 <jimbaker> 2. we were able to finalize a time for tues dev meetings focused on the client & ecosystem at 1600 UTC
15:17:49 <jimbaker> 3. i have a talk tomorrow for the colorado openstack meetup - this will be a chance to get some warmup in for next month's talk in barcelona
15:18:43 <jimbaker> 4. last week we talked about pushing more community involvement, and i was going to write an email. haven't done that. but i will promote tomorrow; and will get to writing more emails
15:18:56 <jimbaker> i think that's it
15:19:03 <jimbaker> jovon, hi!
15:19:03 <syed_> 3- good luck for tomorrow jim, yes it will be great going for barcelona !
15:20:05 <jimbaker> syed_, thanks! i have some slides from jcannava, plus some other stuff. will focus on finalizing this pres today & tomorrow
15:20:38 <jimbaker> #topic dev this week
15:21:41 <jovon> oi! jim
15:21:42 <jimbaker> sigmavirus, i guess this work for you continues to help shepherd through the cratonclient reviews, right?
15:23:18 <jimbaker> while we wait for sigmavirus... jovon?
15:23:28 <jimbaker> how are things going with you?
15:23:39 <jimbaker> palendae, welcome
15:23:50 <palendae> o/
15:24:14 <jovon> Thing are well thanks. Bug fixing as usual
15:25:03 <jimbaker> jovon, are you planning to update the review in https://review.openstack.org/#/c/363171/ ?
15:26:31 <jimbaker> just want to know where we are on that. maybe something smaller to start with?
15:26:34 <jovon> yes. the merger of surveys patch must have slipped me
15:26:42 <jovon> turvey*
15:27:18 <jovon> it will take a simple edit of the index.rst and the patch will be completed
15:27:22 <jovon> my apologie
15:27:40 <jimbaker> jovon, no worries, i'm just trying to keep this review process managed. i let it go out of whack, don't want to repeat
15:28:45 <jimbaker> right now, i think we are in good shape - sulo has no pending reviews, just an experiment + a review blocked on other work. so that's pretty good on the core side
15:29:32 <jimbaker> and sigmavirus has been helping land chrisspencer's work
15:29:53 <jimbaker> palendae, any help you need on reviews on the ansible integration side?
15:30:42 <palendae> I have a few outstanding right now, but they're not directly related. I should have a couple this week
15:30:51 <palendae> The export argument review merged
15:31:08 <jimbaker> palendae, awesome to hear about that merge. please reach out when you are ready
15:31:26 <jimbaker> i found the modeling discussion last week to be extremely helpful
15:31:43 <palendae> Good
15:32:55 <jimbaker> i will briefly mention what i have been looking at
15:33:49 <jimbaker> first, i'm going to complete the api refactor, now that region_id vs region consistency issues have been addressed, since this was conflicting
15:34:21 <jimbaker> this should remove a lot of boilerplate, and make it easier to get some other changes, like 204 for delete
15:34:38 <jimbaker> (not so difficult now, but just more boilerplate to workaround)
15:35:36 <jimbaker> second, my focus has been on the taskflow backend; and how to accommodate tracking more information, especially with respect to linking an overall taskgraph with a subtraskgraph
15:35:50 <jimbaker> so divide work up to the cabinet level in one graph
15:36:14 <jimbaker> and then do the actual workflow tasks against the cabinet in another graph
15:36:43 <jimbaker> so i have worked out (most of) the modeling, and plan to implement this week
15:37:35 <jimbaker> once that's in place, we can start hooking up some of the api stuff that sulo was exploring in https://review.openstack.org/#/c/355938/
15:38:32 <jovon> you've been busy. good work !
15:38:35 <jimbaker> so this will result in scalable, resumable workflows that are data center aware. all buzzwords will be checked off :)
15:38:56 <jimbaker> jovon, thanks :)
15:39:33 <jimbaker> syed_, any updates on your side?
15:39:39 <syed_> well as i told
15:39:49 <syed_> looking into client side variable things up
15:40:07 <syed_> have installed docker locally, so working on functional testing
15:40:09 <jimbaker> syed_, so planning on adding that to cratonclient support?
15:40:29 <syed_> but for now i am trying to get the keystone thing done for horizon folks
15:40:31 <syed_> so they can play with client, and now more things are out there, so that will be great for them
15:41:11 <syed_> for now i am getting an understanding of what and how that works in craton, but mostly focusing towards getting docker-py working,
15:41:18 <syed_> i am expecting keystone thing to be done by today eod
15:41:29 <jimbaker> syed_, right sounds like the right priorities. 1. keystone; 2. functional testing; 3. variables
15:41:33 <syed_> yes
15:42:37 <jimbaker> with respect to variables, one thing that we need to figure out is how to rollup exceptional situations in the variables. this seems like a problem to be done in the core. and it needs to be extensible
15:42:54 <syed_> Hmm ..
15:43:35 <jimbaker> one thing that occurred to me is that we could use the virtualized variable model we have talked about, and use that here
15:44:17 <jimbaker> so in a nutshell, variables have a resolution model that is built on https://docs.python.org/3/library/collections.html#collections.ChainMap
15:44:47 <syed_> Ahh i see
15:44:49 <jimbaker> (which in turn is modeled on how locals, globals are resolved in python)
15:45:49 <jimbaker> the virtualized variable model is really simple here. we can install a PluginMap (which we write) in this ChainMap, which can do arbitrary things
15:46:32 <jimbaker> this follows the CS adage that the solution to all problems is another level of indirection (except indirection itself...)
15:48:40 <jimbaker> anyway... what if we had an ExceptionalPluginMap, which much like a generated column in a sql database, summarizes other variables? we can then ask it for the 'exceptions' key or whatever on a given host, and it looks at the reported variables for say osa-security
15:49:27 <jimbaker> the only requirements for an ExceptionalPluginMap is that it supports the abstract class for a read-only python Map
15:49:54 <jimbaker> anyway... just a thought here
15:49:58 <syed_> Hmm
15:50:00 <syed_> let me look into that
15:50:03 <syed_> will come up with this
15:50:18 <jimbaker> syed_, it's a fun problem to think about
15:50:44 <syed_> indeed it is :)
15:51:14 <jimbaker> the good thing is, you can indefinitely block to do any necessary lookups. the bad thing is, we still have to care about what that means for a client, so it's not quite right :)
15:51:56 <jimbaker> but maybe it can be mostly right. anyway, something to think about
15:52:31 <jimbaker> syed_, in looking at this - i would suggest the following
15:53:27 <jimbaker> for this example: we have in the host variables the information captured from the osa-security role audit
15:53:38 <syed_> okay
15:53:54 <jimbaker> given those variables - what does it mean to rollup to "out-of-compliance"
15:54:19 <palendae> Is Craton handling that logic?
15:54:39 <palendae> I didn't think it was
15:54:42 <jimbaker> palendae, so we have to handle that logic, or so i assume
15:54:57 <jimbaker> but we cannot know in advance all possibilities
15:55:16 <jimbaker> so that's why i want to generalize this as a plugin
15:55:50 <palendae> Fair
15:56:03 <jimbaker> palendae, but maybe audits from osa can have enough information on output that we can label accordinglu
15:56:16 <jimbaker> sounds likes a good discussion for ansible integration :)
15:56:42 <palendae> Yezah
15:57:10 <palendae> I ask because my assumption would have been that Craton stores the information, than somthing external queries and decides on compliance
15:57:31 <jimbaker> palendae, right - this is the rackspace public cloud model that we have adopted
15:57:57 <jimbaker> we got a general variables model, guess what we can stuff it with whatever looks useful :)
15:58:12 <palendae> Ok
15:58:21 <jimbaker> the problem is, we want to hook it up to a console like horizon
15:58:35 <jimbaker> but to do that, horizon doesn't know about compliance
15:58:55 <jimbaker> so this originally external query needs to be pushed into craton core, without making it tightly coupled
15:59:47 <jimbaker> time to wrap up
15:59:57 <jimbaker> good follow on discussion on #craton
16:00:00 <jimbaker> #endmeeting