14:00:02 <mrhillsman> #startmeeting osops
14:00:03 <openstack> Meeting started Mon Feb 27 14:00:02 2017 UTC and is due to finish in 60 minutes.  The chair is mrhillsman. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:07 <openstack> The meeting name has been set to 'osops'
14:00:23 <mrhillsman> #topic Roll Call
14:00:28 <yankcrime> o/
14:00:54 <mrhillsman> morning everyone, osops meeting for those who are available :)
14:01:07 <mrhillsman> agenda is here #link https://etherpad.openstack.org/p/osops-meeting
14:02:27 <mrhillsman> anyone else here besides myself and yankcrime
14:02:32 <mrhillsman> for the meeting that is :)
14:03:12 <robputt> afternoon all
14:03:13 <robputt> :-)
14:03:27 <yankcrime> hi robputt !
14:03:39 <robputt> hey yankcrime, :-)
14:04:06 <mrhillsman> hey robputt
14:04:14 <cemason> i'm here as well  :)
14:04:16 <mrhillsman> cemason welcome!
14:04:36 <mrhillsman> #topic State of the Repos
14:04:46 <mrhillsman> so as you can see on the agenda
14:05:00 <mrhillsman> the repos can use some love i think
14:05:11 <mrhillsman> but overall standardization
14:05:46 <mrhillsman> unfortunately none of the folks from osic who have been working on this are in the meeting
14:06:17 <mrhillsman> but here is some idea
14:06:22 <mrhillsman> https://github.com/osic/ops-generic
14:07:50 <mrhillsman> something like, version the tool works with
14:07:54 <mrhillsman> known issues
14:08:05 <mdorman> o/ sorry for late
14:08:05 <robputt> having all the tools in one repo kinda breaks alot of stuff
14:08:19 <robputt> for example, you can't do a tagged version release of a given tool
14:08:19 <yankcrime> are we talking about defining a new taxonomy and doing away with the existing one?
14:08:38 <mrhillsman> nothing definitive
14:08:58 <mrhillsman> definitely doing something with the README
14:09:10 <mrhillsman> but that means each tool would have to be in a folder
14:09:55 <mrhillsman> if the current taxonomy works with some slight changes great
14:10:28 <mrhillsman> robputt agreed, i think that repo is just an example of structure for a repo
14:10:29 <yankcrime> i like the idea of 'contrib' but agreed wrt the proliferation of tools and versioning
14:10:33 <mrhillsman> we can keep all the ones we have
14:10:56 <mrhillsman> i like versioning as well
14:11:13 <robputt> I have to be honest, I dislike the structure
14:11:17 <mrhillsman> any recommendations on how
14:11:38 <robputt> there are a number of problems with it, such as, you are unable to use ansible in it's intended form loading modules via ansible-galaxy
14:11:53 <robputt> and if you want one tool you end up having this whole bloated repo
14:12:02 <robputt> my gut instinct tells me each tool should be its own repo
14:12:19 <mrhillsman> robputt do you mind taking action to recommend new structure
14:12:25 <robputt> with a tagged version release which is compatible with each openstack release, but I also appreciate that means we end up managing alot of repos
14:13:11 <robputt> its hard to say mrhillsman, we are in danger of having lots of little repos
14:13:22 <yankcrime> i think it might put people off contributing simple scripts as well
14:13:28 <robputt> I can suggest a structure, but I think it would be unpopular as it would add additional management overhead
14:13:47 <robputt> yep, it's getting the balance right, its a complex one
14:13:48 <yankcrime> really i think you'd want a couple of people to volunteer to curate something like robputt's suggesting
14:13:51 * yankcrime nods
14:14:11 <robputt> ohhh for sure you'd need to have people actively act as owners of the repos
14:14:23 <robputt> we'd kind of have a "product owner" kind of head for each
14:14:46 <mrhillsman> with them being little someone could "product own" more than one yes?
14:14:55 <robputt> yeah of course
14:15:09 <mrhillsman> and with a standard would that make it easier to do so?
14:15:14 <robputt> there is a bit of me which says leave it as it is, and get the master branch to a point when it works for all scripts against the latest release
14:15:22 <robputt> and tag that as a "Works with Newton" release
14:15:49 <yankcrime> that sounds like a pragmatic way forward
14:15:53 <mrhillsman> +1
14:16:00 <robputt> but it means we need to get all scripts in the repo in a known working state for a given openstack release
14:16:13 <mrhillsman> we can work on that in osic
14:16:26 <mrhillsman> so all agreed on that?
14:16:34 <robputt> we can then backport it to previous versions in a branch for each Openstack version
14:16:53 <robputt> yep I would agree
14:17:01 <robputt> I think the overhead of having a repo for each script is too much
14:17:14 <robputt> but versioned releases some how for the repos are pretty damn important I think
14:17:14 <mrhillsman> cool
14:17:29 <mrhillsman> we can discuss further definitely
14:17:42 <mrhillsman> i think getting it to works with x is a good first
14:18:04 * yankcrime nods
14:18:12 <mrhillsman> cool, moving on
14:18:48 <mrhillsman> #agreed get all tools to works with X status
14:18:54 <mdorman> +1
14:18:58 <mrhillsman> #topic Wiki Cleanup
14:19:38 <mrhillsman> i want to discuss this with uc as it relates to the purpose of osops beyond the repos
14:19:52 <mrhillsman> and i think after such a discussion we can clean some of the stuff up on the wiki
14:20:07 <mrhillsman> just seems like stuff is everywhere
14:20:31 <yankcrime> what's the general UC steer on this stuff?
14:20:34 <yankcrime> or is that what you want to prompt?
14:20:46 <mrhillsman> that is what i want to prompt
14:20:51 <yankcrime> righto
14:21:09 <mrhillsman> i have an idea but could be totally off
14:21:28 <mrhillsman> wondering if anyone here has reviewed it and has any thoughts
14:21:36 <mrhillsman> or even tried to use it hehe
14:21:57 <robputt> no real thoughts
14:22:03 <mrhillsman> looks to need some tlc in general
14:22:10 <robputt> It would be nice to flesh it out with some product catalogue kind of style thing
14:22:30 <mrhillsman> hmmm...interesting
14:22:40 <robputt> for a new visitor "A repo of curated tools and scripts." doesn't really mean much when he is looking for some specific script
14:22:42 <mrhillsman> noting that
14:22:48 <mrhillsman> agreed
14:22:58 <yankcrime> yeah, good idea
14:23:08 <robputt> if we could list all of our scripts, which repo they live in, a status for the last few openstack releases of known compatability and maybe a roadmap for additions / improvements
14:23:36 <mrhillsman> and actually, that could help with not having to have complex repo setup
14:24:04 <mrhillsman> could standardize around the 'product catalogue'?
14:24:30 <mrhillsman> good stuff robputt
14:25:00 <mrhillsman> ok, anything else here?
14:25:08 <nishpatwa007> robputt I would like to add
14:26:14 <nishpatwa007> So https://github.com/osic/ops-generic is a more structured implementation of the some of the functionalities of ops tools contrib with reusable common libraries
14:27:15 <robputt> I see...
14:27:47 <mrhillsman> i'm going to go on to the next topic everyone unless we need to spend more time here
14:27:48 <nishpatwa007> in that way its easier to add more scripts
14:27:50 <robputt> that makes me think of something else for either the wiki / github repos actually...
14:28:03 <robputt> each script could do with usage instructions / sample configs
14:28:08 <mdorman> or at least it’s supposed to be.  in theory things from the contrib repo are supposed to be “promoted” up to the generic repo.   but in practice i don’t think taht is happening much
14:28:25 <nishpatwa007> Yes each tool has a readme for the usage and the requirements
14:28:42 <nishpatwa007> so each directory is an autonomous entity
14:29:13 <robputt> yep, we need to add that because at the moment stuff like orphaned floating ip tool has no requirements.txt or a readme
14:29:23 <robputt> so for a new comer it maybe be a bit of a struggle to work out how to use it
14:29:45 <robputt> anyway I agree next topic, we are rabbit holing around what standards a tool needs to be included in generic rather than contrib
14:29:53 <mrhillsman> ok cool
14:30:00 <mrhillsman> we can discuss further of course later
14:30:03 <nishpatwa007> cool
14:30:08 <mrhillsman> #topic Feedback Loop
14:30:20 <mrhillsman> #link https://github.com/openstack/openstack-user-stories
14:30:26 <mrhillsman> so we have these user stories
14:31:01 <mrhillsman> we definitely cannot "force" work i would say but more so influence it
14:31:24 <mrhillsman> the product team however would like operators to review the user stories
14:31:45 <mrhillsman> and from a technical/operator/end-user/etc standpoint ensure they make sense
14:32:15 <mrhillsman> these would basically bubble up to the UC who then advocate for the work
14:32:55 <mrhillsman> having operators review the user-story i believe the idea is it adds more value
14:33:14 <mdorman> +1 we should be helping review that stuff, and this is a place where we can influence new features on the front-end
14:33:23 <mrhillsman> yep
14:33:36 <mrhillsman> i will send an email to the ML at large
14:33:47 <mrhillsman> but wanted to get any feedback from you fine folks
14:34:09 <mrhillsman> #action mrhillsman email Ops ML regarding PWG user story reviews
14:34:12 <robputt> no real opinion on this, user stories should come from all of the places in my opinion
14:34:13 <robputt> :-)
14:34:23 <mrhillsman> agreed
14:34:32 <yankcrime> ops are users too! ;)
14:34:50 <mrhillsman> lol, yes, yes we are
14:35:10 <mrhillsman> ok cool, on to the next one
14:35:19 <mrhillsman> #topic Documentation
14:35:56 <mrhillsman> so, i think some folks have a misunderstanding around this
14:36:04 <mrhillsman> or maybe it has not been clear
14:36:25 <mrhillsman> ops folks are not asking to be taxed with doing the actual docs work
14:36:45 <mrhillsman> but rather review and feedback, filing bugs, etc
14:36:58 <robputt> I personally think the ops based documentation for Openstack could be improved across the board.
14:37:00 <mrhillsman> asettle am i wrong?
14:37:06 <asettle> Hey!
14:37:24 <robputt> there are lots of small bits that need tweaking to make stuff work as expected for new / inexperienced operators
14:37:25 <mrhillsman> docs folks are in agreement robputt afaik
14:37:27 <asettle> So, it's a 50/50, you're not wrong ;) but operators *do* have specialty knowledge that devs and tech writers just do not have.
14:37:35 <asettle> We do request/expect those gaps are often filled.
14:37:47 <asettle> Whether that be by bug reporting (letting us konw a change, an error, etc) or actually patching it up yourself.
14:37:56 <robputt> I am happy to submit patches
14:38:04 <asettle> robputt: that's the spirit ;)
14:38:07 <mrhillsman> +1
14:38:15 <mrhillsman> same here
14:38:15 <robputt> but I have no clue about how the docs repos / workflow works
14:38:17 <asettle> If I can help, I will. I'm Alex o/ for those who haven't met me.
14:38:20 <asettle> I'm the docs PTL
14:38:27 <robputt> is it largely the same as any other OS project?
14:38:29 <asettle> robputt: exactly the same as dev. If you need help with it though, let me help you work it through
14:38:34 <asettle> No changes robputt :)
14:38:34 <mrhillsman> it is the same as any other ^
14:38:38 <robputt> cool :-)
14:38:38 <asettle> https://github.com/openstack/openstack-manuals
14:38:53 <asettle> We like to ensure there's minimal difference.
14:38:57 <robputt> well in this case I have some changes for the Newton Debian 8 manual which I picked up the other day when deploying some stuff
14:38:57 <robputt> :-)
14:39:02 <mrhillsman> hehe
14:39:04 <asettle> robputt: the install guide?
14:39:07 <robputt> yep
14:39:17 <asettle> So, we just lost our Debian guy. So, please, patch away by all means.
14:39:22 <mrhillsman> there's some issues around debian i read
14:39:31 <asettle> Debian is no longer, in summary.
14:39:32 <mrhillsman> the packaging guy is gone or something
14:39:39 <asettle> We had one guy who was helping the team to package and manage the guides is gone.
14:39:43 <robputt> damn
14:39:56 <mrhillsman> yeah, hopefully it will get worked out
14:39:58 <asettle> The docs team already managers 3 distributions (well, 5) so keeping on debconf and debian was a hard slog.
14:40:00 <mrhillsman> there is a long thread about it
14:40:07 <asettle> So, we've had to drop it until another package manager comes our way.
14:40:14 <robputt> I can't promise packages, I hate packaging Python stuff inside of anything over the PyPi, but the docs I can submit updates for sure
14:40:28 <asettle> robputt: love your stuff :) it's not up in Ocata :( But Newton would love some attention if you have the time.
14:40:51 <asettle> I'm actually in the process of looking at the ops-guide we have up and will be owrking with our team leads to improve it and the admin guide
14:40:55 <asettle> https://docs.openstack.org/ops-guide/operations.html
14:41:05 <asettle> https://docs.openstack.org/admin-guide/
14:41:13 <asettle> If anyone's interested in helping out these guys with me, please let me know :)
14:42:21 <mrhillsman> awesome, i'm going to go to the next topic because of timing
14:42:26 <asettle> Please do :)
14:42:26 <mrhillsman> thanks asettle ;)
14:42:45 <robputt> np, asettle i'll poke you later about docs stuff I have a few more queries
14:42:53 <mrhillsman> going to skip over ancillary tooling if no one objects
14:42:56 <asettle> robputt: jump on over to our chan if you have any questions :) #openstack-doc
14:43:37 <mrhillsman> and osic like environments
14:43:46 <robputt> well
14:43:47 <mrhillsman> #topic Underutilized Projects
14:43:48 <robputt> hang on
14:43:54 <mrhillsman> ok, going back
14:43:56 <mrhillsman> to which one?
14:43:56 <robputt> lets chat about osic like envs quickly
14:43:58 <robputt> :P
14:44:10 <mrhillsman> #topic OSIC-like Environments
14:44:15 <robputt> so I currently have "RobCloud" my personal Openstack playground
14:44:43 <robputt> which massively sucks but I try stuff out in there which I want in a vanilla openstack deployment
14:45:05 <robputt> So, what opportunity is there for us getting access to OSIC like envs for testing out stuff
14:45:13 <robputt> in particular emerging Openstack projects?
14:45:20 <mrhillsman> anyone can request resources from osic
14:45:28 <mrhillsman> and for emerging projects even the more
14:45:33 <robputt> ok... but what about if we want to have control plane access?
14:45:51 <robputt> or are we expected to build a cloud ontop of the cloud for testing?
14:46:11 <mrhillsman> you get baremetal nodes
14:46:17 <robputt> ohhh ok cool :-)
14:46:30 <mrhillsman> and can do what your heart desires with tnem :)
14:46:42 <robputt> tbh its probably the same overhead as what I have now then for trying stuff out if you get bare metal
14:46:48 <mrhillsman> beyond that though, there are other options as well and i was hoping to discuss
14:47:03 <robputt> so these bare metal nodes, are they Ironic driven?
14:47:11 <mrhillsman> so you want openstack already live
14:47:29 <mrhillsman> they are ironic driven more than likely by now or definitely will be
14:47:36 <robputt> ok
14:47:37 <robputt> :-)
14:47:39 <mrhillsman> there is work to move all of the nodes under ironic
14:47:45 <mrhillsman> some are, some are not unfortunately
14:47:48 <robputt> let me take this offline mrhillsman
14:47:52 <mrhillsman> ok
14:48:09 <mrhillsman> there is also cloud native foundation and few other such osic-like environments
14:48:14 <robputt> but I would like to make use of OSIC for my Openstack adventures rather than my current setup as its a bit crummy ;-)
14:48:22 <mrhillsman> i was wondering if anyone knows of any, has any, is planning on having any, etc
14:49:19 <mrhillsman> i have a contact at AT&T who mentioned having something like osic there
14:49:22 <mrhillsman> but not sure what that means
14:49:28 <mrhillsman> so i will go on to the next
14:49:33 <mrhillsman> time running out hehe
14:49:39 <mrhillsman> #topic Underutilized Projects
14:49:54 <robputt> Yep, we need to get in here :-)
14:49:55 <robputt> big time
14:50:01 <mrhillsman> #link https://etherpad.openstack.org/p/ops-adopt-a-project-pike
14:50:28 <mrhillsman> so beyond the general issue of projects not having enough feedback and new ones needing more than those that have been around
14:50:44 <robputt> its not so much feedback
14:50:52 <mrhillsman> we started an effort in osic to help those that folks want to use
14:50:57 <mrhillsman> but just need that little extra push
14:51:03 <robputt> I would like us to get very early involvement so we can shape a product around operational acceptance from day one.
14:51:07 <mrhillsman> and i think the thought is good across the board
14:51:51 <mrhillsman> how can we get that to work
14:51:59 <mrhillsman> i assume we need test environments
14:52:05 <mrhillsman> tying into the osic-like topic
14:52:14 <mrhillsman> though they do not have to be large
14:52:24 <mrhillsman> but devstack would not be sufficient
14:52:34 <mrhillsman> from an operational standpoint
14:52:50 <klindgren> So we tried ironic.  But honestly the "friction" between the ironic team and the nova team and the pace of getting something out there that was not crazy pants - was untenable.
14:52:51 <robputt> It doesn't need to be a large deployment, but silly stuff like
14:52:57 <robputt> "how does it scale"
14:53:04 <robputt> is often not even covered in new projects at all
14:54:02 <mrhillsman> klindgren i'd like to hear more
14:54:14 <klindgren> Something as simple as configuring a raid array for r10 or r5 or r1 & 5.  The answer was "use seperate pools for each raid array.
14:54:59 <klindgren> Which means that I need to have 3 different config/pools of hardware for 1 piece of hardware
14:55:28 <klindgren> for requests to be pulled from.  Ironic was build some placement api thing, for NOVa , but that took like 2 cycles - not even sure where it is in ocata
14:56:03 <mrhillsman> would like to hear more offline klindgren as it could be an issue going forward in general
14:56:18 <klindgren> Its also super easy to get an ironic node stuck
14:56:21 <mrhillsman> and probably something that needs to pushed up for discussion
14:56:30 <klindgren> reboot it while deploying an image to it - it will bascially never come back
14:56:40 <mrhillsman> so we have not so new projects and very new projects
14:56:45 <klindgren> must go to the db to clear the state, and remove the vm from nova
14:57:10 <yankcrime> early ops engagement for new and emerging projects would be a great idea imo
14:57:15 <robputt> we have a whole load of ugly around our ironic deployment :-(
14:57:48 <mrhillsman> i'd like to hear more about how that looks like
14:57:55 <robputt> well, just for example
14:57:58 <robputt> if we look at Picasso
14:57:58 <mrhillsman> new and emerging
14:58:03 <robputt> which is an uber new project
14:58:15 <robputt> and a pretty important one in my opinion, but maybe thats just my love for FaaS
14:58:24 <mrhillsman> #link https://wiki.openstack.org/wiki/Picasso
14:58:34 <robputt> it currently for me isn't really heading in the direction of operational acceptance in my opinion...
14:58:39 <klindgren> Some of the variables are one-sizes fits all (like the stty number to use for serial console on serial over lan) - but all hardware is snowflakes.  So I actually have 2 peices of hardware that need different stty numbers.  No way to handle that in ironic.
14:59:02 <robputt> For example, it is not configurable like a standard Openstack service, it doesn't use Oslo libs for stuff like DB / Messaging
14:59:09 <mrhillsman> i'm going to have to kill our meeting at 9 but feel free to keep going
14:59:17 <mrhillsman> i will capture things for next meeting
14:59:26 <robputt> it doesn't have a pluggable backend for using with multiple function runners etc...
14:59:34 <robputt> to me, as an operator I expect things to use the standard libs
14:59:40 <robputt> and to behave in an Openstacky kind of way
14:59:52 <mrhillsman> and we have to have buy-in from those projects no doubt
14:59:53 <robputt> once you have configured 1 OS project you should be able to configure them all
15:00:00 <yankcrime> so really our engagement should be at the blueprint stage
15:00:06 <robputt> yes totally
15:00:20 <mrhillsman> #endmeeting