14:59:05 <jimbaker> #startmeeting craton
14:59:06 <openstack> Meeting started Mon Jul 11 14:59:05 2016 UTC and is due to finish in 60 minutes.  The chair is jimbaker. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:59:11 <openstack> The meeting name has been set to 'craton'
14:59:30 <jimbaker> today's agenda can be found here: https://etherpad.openstack.org/p/craton-meetings
14:59:54 <jimbaker> sulo, hi
15:00:02 <jimbaker> sarob cannot make it
15:00:13 <sulo> o/
15:00:26 <jimbaker> o/
15:00:35 <sigmavirus> o/
15:01:35 <jimbaker> #topic user stories
15:02:01 <jimbaker> we have been working against the user stories: https://specs.openstack.org/openstack/openstack-user-stories/user-stories/proposed/fleet-management.html
15:03:42 <jimbaker> kencjohnston and KrishR have curated these stories, and we have agreed they are broadly applicable to fleet management and craton specifically
15:04:46 <jimbaker> because the stories are high level, to make them actionable i believe we need to focus on representative stories; define acceptance criteria; and build out accordingly
15:04:52 <jimbaker> turvey, o/
15:04:55 <jimbaker> cmspence, o/
15:04:59 <cmspence> o/
15:05:02 <turvey> o/
15:05:50 <jimbaker> the specific story that seems to be most straightforward for us to work on is FLT005
15:06:01 <jimbaker> FLT005 - As a cloud operator, I need to check my deployed physical resources against a set of policies and rules, so that I can meet security, availability and other requirements
15:06:59 <jimbaker> i believe auditing with the OSA security role lets us go deep. also we can work with mhayden on it
15:07:43 <jimbaker> sulo, we are pretty close to having OSA container modeling working against craton inventory, right?
15:07:52 <sulo> yeah
15:08:10 <sulo> so that .. and in general a model to allow network devices too
15:08:44 <sulo> so we can do things that are raised here :https://docs.google.com/document/d/1N3f_H3bjZ1a08wkxRLCMt5ANXRGmFAOEUZ_mtI3BYxw/edit
15:09:00 <sulo> i.e being able to use switches / cabs
15:09:13 <sulo> that is useful for osa as well btw
15:09:37 <jimbaker> sulo, so your PR re device hierarchies is essential for this to happen?
15:09:50 <sulo> yes
15:09:50 <jimbaker> with respect to OSA containers?
15:09:58 <jimbaker> ack
15:10:07 <sulo> jimbaker: so like i was discussing with you in #craton
15:10:45 <sulo> the question is: do we want to model each of these as separate models vs doing how i did in that par .. with a column tracking host_type
15:10:45 <jimbaker> sulo, my questions for you and the team, and not necessarily something we can decide on now
15:11:03 <jimbaker> sulo, yeah, that's an additional question to mine then :)
15:11:08 <sulo> jimbaker: yeah .. we can resume that discussion in craton
15:11:34 <sulo> jimbaker: my point is as soon as we agree on a model . . i think we can solve for both those items
15:11:34 <jimbaker> no worries, we just want to publicly capture these questions
15:11:55 <jimbaker> i would add: 1) do we fully capture as a DAG?
15:12:13 <jimbaker> 2) do variables inherit in this hierarchy?
15:12:46 <jimbaker> (if we model as a DAG, we get much closer to oneops core modeling, which will help convergence there)
15:13:04 <jimbaker> anyway, i think we should resolve these questions this week
15:13:18 <jimbaker> so that seems like an action item to me
15:14:03 <sulo> so DAG model is close to what we are trying to do through devices like the pr right now ?
15:14:53 <jimbaker> sulo, yes
15:15:46 <sulo> jimbaker: so what does fully capturing DAG involve ?
15:15:55 <sulo> just trying to understand really
15:16:06 <sulo> are we missing something on that .. ?
15:16:16 <jimbaker> sulo, sure - you just need a src, dest associative table
15:16:44 <jimbaker> and outside of the db you need to support computing things like transitive closure
15:16:59 <jimbaker> and ensuring it's actually acyclic
15:17:09 <sulo> ok
15:17:49 <sulo> jimbaker: but we also need variable hierarchy right ? so i guess we do both ?
15:17:53 <jimbaker> it seems more complicated if we want to have variable overrides participate in the DAG, so my question of whether we need that modeling power
15:18:19 <jimbaker> sulo, right, we are obviously thinking of similar things :)
15:18:53 <jimbaker> so i don't know the answers. just want to ask the questions to we can model the things we care about
15:19:09 <jimbaker> anyway, two action items here
15:19:19 <jimbaker> #action define acceptance criteria for FLT005
15:19:35 <jimbaker> #action jimbaker define acceptance criteria for FLT005
15:20:08 <jimbaker> #action jimbaker sulo finalize modeling of device hierarchy, considering DAG, variables
15:20:18 <jimbaker> look good?
15:20:29 <sulo> yes
15:21:25 <jimbaker> #topic inventory fabric
15:22:28 <jimbaker> sulo, this is a term i started using to explain to stakeholders what craton is trying to do in terms of managing sources of truth for inventory
15:22:57 <sulo> jimbaker: heh, cool
15:23:10 <jimbaker> it's not trying to do everything... but perhaps it can integrate with arbitrary other systems, via plugins
15:23:36 <sulo> jimbaker: so i've had a few meetings with osa folks last week on how we can achieve this
15:24:00 <sulo> there is a good discussion going on here also :https://github.com/rackerlabs/craton/issues/12
15:24:10 <jimbaker> so if we have a network inventory system like arista; or nova is managing the scheduling of VMs to hosts... that's the role of the fabric
15:24:20 <jimbaker> and presumably virtualized variables
15:24:45 <jimbaker> https://github.com/rackerlabs/craton/issues/109
15:25:18 <jimbaker> sulo, so any updates from OSA on this from that email thread?
15:25:41 <jimbaker> (it
15:25:51 <sulo> well lots of feedback during my meeting .. and on that issue i posted above
15:26:04 <jimbaker> sure, just wanted a summary here
15:26:30 <sulo> but the main point to start with is : being able to support config data such that we can generate a ansible inventory to run a playbook
15:26:56 <sulo> and to do that we need to address quite a few issues
15:27:23 <sulo> like 1. this should be templated enough that inital variable generation
15:27:35 <sulo> and host and group generation is super easy
15:27:56 <sulo> 2. we support variable override in the way you have captured in various issues already
15:28:23 <sulo> 3. we support containers and deives as in my pr
15:28:27 <sulo> so we are moving towards it
15:28:43 <sulo> but there is some more work to be done .. mostly on config/variables side
15:28:59 <sulo> so the plan is for us to get the base set working
15:29:06 <sulo> then demo to them and go from there
15:29:20 <jimbaker> sulo, yeah. it seems to me this will start impacting OSA, with respect to templating
15:29:56 <sulo> oh yeah .. and 4. support handling secrets ;)
15:30:56 <jimbaker> seems reasonable. i think handling secrets should not be so bad, we can adopt a vault-style model
15:31:01 <jimbaker> (ansible vault)
15:31:30 <jimbaker> but definitely details there to get right
15:31:46 <sulo> yeah
15:32:35 <jimbaker> on my side, i spent some time on the variable refactoring. it should work, and it almost does, but i'm missing something on the sqlalchemy side in terms of how relationships get converted to the appropriate collection type
15:33:02 <jimbaker> anyway, i'm sure it will be easy enough to fix
15:33:18 <sulo> jimbaker: does that include variables blame support ?
15:33:37 <jimbaker> sulo, yeah, that would just work once i have it refactored
15:33:44 <sulo> cool
15:34:02 <jimbaker> i'm just trying to preserve k,v access through a dict mapping, as opposed to seeing a list of variables
15:34:49 <jimbaker> anyway, i know much more about sqlalchemy at this point :)
15:35:59 <jimbaker> let's move on
15:36:05 <jimbaker> #topic audit workflows
15:37:12 <jimbaker> sulo, so the OSA work is a good complement here, since it enables the actual FLT005 stuff with OSA security role
15:37:37 <sulo> jimbaker: yeah, so i did test this a little bit
15:37:42 <jimbaker> at this point, we need to take gus' work on taskflow integration
15:37:49 <jimbaker> and really start building it out
15:37:58 <sulo> we need to discuss the architecture a bit more detail in one of the dev meeting
15:38:12 <jimbaker> sulo, sounds good to me
15:39:05 <jimbaker> i know gus looked into TF's jobboard, and that seems the right approach for scale; but i'm more concerned about task generation/scheduling
15:39:21 <jimbaker> since we only have this as TODOs so far
15:40:05 <jimbaker> should we just have that as the focus of this thurs dev meeting, with research this week?
15:40:17 <sulo> +1
15:41:15 <jimbaker> cmspence, sigmavirus, turvey - any thoughts?
15:41:57 <jimbaker> #topic integrations
15:42:17 <cmspence> jimbaker: +1
15:42:22 <jimbaker> i talked with mike tamayo from OSIC/intel
15:42:24 <turvey> Works for me
15:42:31 <jimbaker> (sounds good!)
15:42:57 <jimbaker> and he will be committing resources to help on horizon integration, which will be optional for operators
15:43:22 <jimbaker> but certainly useful for all sorts of use cases, even if many operators will often want to use the CLI or python client
15:43:41 <sigmavirus> sorry jimbaker, got stuck in another meeting
15:43:48 <jimbaker> sigmavirus, no worries
15:43:59 <jimbaker> so we are looking at wireframes by july 23
15:44:54 <sulo> jimbaker: details on this integration work ? is this going upstream etc ?
15:45:05 <jimbaker> re oneops integration, after a good meeting with walmart labs on june 28, we are waiting to follow up. sarob will help organize
15:45:05 <sulo> what are we expecting on horizon end?
15:45:50 <jimbaker> sulo, it should be a horizon plugin that can consume craton's REST APIs
15:46:07 <sulo> ok
15:46:13 <sigmavirus> I really think that should be put off until we have a stable API
15:46:20 <sulo> yeah agree
15:46:39 <jimbaker> sigmavirus, i understand your concern, however we are getting stakeholder feedback to do this earlier than later
15:47:02 <turvey> We'll make sure that those implementing understand the API is still in flux
15:47:39 <jimbaker> turvey, +1
15:47:44 <sulo> jimbaker: so even if we use the current api what is horizon going to show ? basically a UI for inventory to start with ?
15:48:12 <jimbaker> sulo, yes, and then augment with say being able to use the workflow APIs, as we develop those
15:48:59 <jimbaker> so being able to inspect the environment, config vars, etc; then as we add workflows, kick those off
15:49:27 <sulo> right, yeah in general sound good, given it takes UI work away from this team ;)
15:49:31 <jimbaker> interesting points here is that we will need to consider how to show say audit results - what convention/config makes it possible to flag things
15:49:42 <jimbaker> sulo, correct, we want this work done
15:49:51 <jimbaker> and it is a separate team of resources :)
15:50:06 <jimbaker> having good APIs is our responsibility regardless
15:50:55 <turvey> It also gives is an early consumer of the API, which could be useful.
15:50:56 <jimbaker> going back to FLT005 - this will help in completeness on that story. also on demoable things for barcelona :)
15:51:01 <jimbaker> turvey, +1
15:51:20 <jimbaker> time to move on!
15:51:29 <jimbaker> #topic admin
15:51:48 <jimbaker> we need to reschedule the dev meeting on thurs
15:52:29 <jimbaker> sulo cannot make it at all, and i believe it would be best to accommodate gus by my doing a 1:1 with him
15:53:22 <jimbaker> cmspence, sigmavirus, sulo, turvey - i'm proposing something around 1600 UTC, no earlier than 1530 UTC on thursdays
15:53:41 <sulo> sounds good to me
15:53:49 <jimbaker> i will send out a doodle poll on the list to get specific times. but would this basically work for all of you?
15:54:06 <turvey> Sure
15:54:14 <cmspence> ya, that should be fine
15:54:48 <jimbaker> sigmavirus, ?
15:55:08 <jimbaker> the other item here is that sarob has basically completed the project patch
15:55:20 <jimbaker> we need to do a final review
15:55:26 <sulo> oh its not complete .. i just https://review.openstack.org/#/c/332470/4
15:55:32 <sigmavirus> jimbaker: I have a weekly meeting Thursday mornings, I probably won't make that either
15:55:36 <sulo> i was going to ping him on that
15:55:47 <jimbaker> sigmavirus, then we need to figure a time to make it work for you
15:55:54 <sigmavirus> jimbaker: meh
15:55:58 <sulo> jimbaker: sigmavirus: different day ?
15:55:59 <sigmavirus> I'm not on the project full time for more months
15:56:03 <sigmavirus> we can shift if necessary then
15:56:38 <jimbaker> sigmavirus, yeah, i just want to make sure that you, sulo, cmspence, turvey and me can make it
15:56:52 <sulo> jimbaker: sigmavirus: as long as its not very late UTC time i am fine with any date btw.
15:57:13 <jimbaker> sulo, thanks for being flexible!
15:57:24 <jimbaker> we just didn't want to push it too much
15:57:27 <sigmavirus> Let's go with thursday for now. I'm doing mental utc math
15:57:34 <sulo> heh
15:57:54 <jimbaker> sigmavirus, so i am making it work with jason c's meeting
15:58:03 <jimbaker> we should be in our staff meeting :)
15:58:15 <sigmavirus> jimbaker: I'm not in your staff meeting though =P
15:58:33 <jimbaker> ok, then it's just sulo and i have to be there
15:58:50 <jimbaker> but regardless, let's find a time over on #craton, then put it out there on the mailing list to finalize
15:58:55 <sulo> we are running out of time here .. move this to #craton ?
15:59:03 <jimbaker> yes, time to end meeting
15:59:07 <jimbaker> #endmeeting