14:00:02 #startmeeting osops 14:00:03 Meeting started Mon Feb 27 14:00:02 2017 UTC and is due to finish in 60 minutes. The chair is mrhillsman. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:07 The meeting name has been set to 'osops' 14:00:23 #topic Roll Call 14:00:28 o/ 14:00:54 morning everyone, osops meeting for those who are available :) 14:01:07 agenda is here #link https://etherpad.openstack.org/p/osops-meeting 14:02:27 anyone else here besides myself and yankcrime 14:02:32 for the meeting that is :) 14:03:12 afternoon all 14:03:13 :-) 14:03:27 hi robputt ! 14:03:39 hey yankcrime, :-) 14:04:06 hey robputt 14:04:14 i'm here as well :) 14:04:16 cemason welcome! 14:04:36 #topic State of the Repos 14:04:46 so as you can see on the agenda 14:05:00 the repos can use some love i think 14:05:11 but overall standardization 14:05:46 unfortunately none of the folks from osic who have been working on this are in the meeting 14:06:17 but here is some idea 14:06:22 https://github.com/osic/ops-generic 14:07:50 something like, version the tool works with 14:07:54 known issues 14:08:05 o/ sorry for late 14:08:05 having all the tools in one repo kinda breaks alot of stuff 14:08:19 for example, you can't do a tagged version release of a given tool 14:08:19 are we talking about defining a new taxonomy and doing away with the existing one? 14:08:38 nothing definitive 14:08:58 definitely doing something with the README 14:09:10 but that means each tool would have to be in a folder 14:09:55 if the current taxonomy works with some slight changes great 14:10:28 robputt agreed, i think that repo is just an example of structure for a repo 14:10:29 i like the idea of 'contrib' but agreed wrt the proliferation of tools and versioning 14:10:33 we can keep all the ones we have 14:10:56 i like versioning as well 14:11:13 I have to be honest, I dislike the structure 14:11:17 any recommendations on how 14:11:38 there are a number of problems with it, such as, you are unable to use ansible in it's intended form loading modules via ansible-galaxy 14:11:53 and if you want one tool you end up having this whole bloated repo 14:12:02 my gut instinct tells me each tool should be its own repo 14:12:19 robputt do you mind taking action to recommend new structure 14:12:25 with a tagged version release which is compatible with each openstack release, but I also appreciate that means we end up managing alot of repos 14:13:11 its hard to say mrhillsman, we are in danger of having lots of little repos 14:13:22 i think it might put people off contributing simple scripts as well 14:13:28 I can suggest a structure, but I think it would be unpopular as it would add additional management overhead 14:13:47 yep, it's getting the balance right, its a complex one 14:13:48 really i think you'd want a couple of people to volunteer to curate something like robputt's suggesting 14:13:51 * yankcrime nods 14:14:11 ohhh for sure you'd need to have people actively act as owners of the repos 14:14:23 we'd kind of have a "product owner" kind of head for each 14:14:46 with them being little someone could "product own" more than one yes? 14:14:55 yeah of course 14:15:09 and with a standard would that make it easier to do so? 14:15:14 there is a bit of me which says leave it as it is, and get the master branch to a point when it works for all scripts against the latest release 14:15:22 and tag that as a "Works with Newton" release 14:15:49 that sounds like a pragmatic way forward 14:15:53 +1 14:16:00 but it means we need to get all scripts in the repo in a known working state for a given openstack release 14:16:13 we can work on that in osic 14:16:26 so all agreed on that? 14:16:34 we can then backport it to previous versions in a branch for each Openstack version 14:16:53 yep I would agree 14:17:01 I think the overhead of having a repo for each script is too much 14:17:14 but versioned releases some how for the repos are pretty damn important I think 14:17:14 cool 14:17:29 we can discuss further definitely 14:17:42 i think getting it to works with x is a good first 14:18:04 * yankcrime nods 14:18:12 cool, moving on 14:18:48 #agreed get all tools to works with X status 14:18:54 +1 14:18:58 #topic Wiki Cleanup 14:19:38 i want to discuss this with uc as it relates to the purpose of osops beyond the repos 14:19:52 and i think after such a discussion we can clean some of the stuff up on the wiki 14:20:07 just seems like stuff is everywhere 14:20:31 what's the general UC steer on this stuff? 14:20:34 or is that what you want to prompt? 14:20:46 that is what i want to prompt 14:20:51 righto 14:21:09 i have an idea but could be totally off 14:21:28 wondering if anyone here has reviewed it and has any thoughts 14:21:36 or even tried to use it hehe 14:21:57 no real thoughts 14:22:03 looks to need some tlc in general 14:22:10 It would be nice to flesh it out with some product catalogue kind of style thing 14:22:30 hmmm...interesting 14:22:40 for a new visitor "A repo of curated tools and scripts." doesn't really mean much when he is looking for some specific script 14:22:42 noting that 14:22:48 agreed 14:22:58 yeah, good idea 14:23:08 if we could list all of our scripts, which repo they live in, a status for the last few openstack releases of known compatability and maybe a roadmap for additions / improvements 14:23:36 and actually, that could help with not having to have complex repo setup 14:24:04 could standardize around the 'product catalogue'? 14:24:30 good stuff robputt 14:25:00 ok, anything else here? 14:25:08 robputt I would like to add 14:26:14 So https://github.com/osic/ops-generic is a more structured implementation of the some of the functionalities of ops tools contrib with reusable common libraries 14:27:15 I see... 14:27:47 i'm going to go on to the next topic everyone unless we need to spend more time here 14:27:48 in that way its easier to add more scripts 14:27:50 that makes me think of something else for either the wiki / github repos actually... 14:28:03 each script could do with usage instructions / sample configs 14:28:08 or at least it’s supposed to be. in theory things from the contrib repo are supposed to be “promoted” up to the generic repo. but in practice i don’t think taht is happening much 14:28:25 Yes each tool has a readme for the usage and the requirements 14:28:42 so each directory is an autonomous entity 14:29:13 yep, we need to add that because at the moment stuff like orphaned floating ip tool has no requirements.txt or a readme 14:29:23 so for a new comer it maybe be a bit of a struggle to work out how to use it 14:29:45 anyway I agree next topic, we are rabbit holing around what standards a tool needs to be included in generic rather than contrib 14:29:53 ok cool 14:30:00 we can discuss further of course later 14:30:03 cool 14:30:08 #topic Feedback Loop 14:30:20 #link https://github.com/openstack/openstack-user-stories 14:30:26 so we have these user stories 14:31:01 we definitely cannot "force" work i would say but more so influence it 14:31:24 the product team however would like operators to review the user stories 14:31:45 and from a technical/operator/end-user/etc standpoint ensure they make sense 14:32:15 these would basically bubble up to the UC who then advocate for the work 14:32:55 having operators review the user-story i believe the idea is it adds more value 14:33:14 +1 we should be helping review that stuff, and this is a place where we can influence new features on the front-end 14:33:23 yep 14:33:36 i will send an email to the ML at large 14:33:47 but wanted to get any feedback from you fine folks 14:34:09 #action mrhillsman email Ops ML regarding PWG user story reviews 14:34:12 no real opinion on this, user stories should come from all of the places in my opinion 14:34:13 :-) 14:34:23 agreed 14:34:32 ops are users too! ;) 14:34:50 lol, yes, yes we are 14:35:10 ok cool, on to the next one 14:35:19 #topic Documentation 14:35:56 so, i think some folks have a misunderstanding around this 14:36:04 or maybe it has not been clear 14:36:25 ops folks are not asking to be taxed with doing the actual docs work 14:36:45 but rather review and feedback, filing bugs, etc 14:36:58 I personally think the ops based documentation for Openstack could be improved across the board. 14:37:00 asettle am i wrong? 14:37:06 Hey! 14:37:24 there are lots of small bits that need tweaking to make stuff work as expected for new / inexperienced operators 14:37:25 docs folks are in agreement robputt afaik 14:37:27 So, it's a 50/50, you're not wrong ;) but operators *do* have specialty knowledge that devs and tech writers just do not have. 14:37:35 We do request/expect those gaps are often filled. 14:37:47 Whether that be by bug reporting (letting us konw a change, an error, etc) or actually patching it up yourself. 14:37:56 I am happy to submit patches 14:38:04 robputt: that's the spirit ;) 14:38:07 +1 14:38:15 same here 14:38:15 but I have no clue about how the docs repos / workflow works 14:38:17 If I can help, I will. I'm Alex o/ for those who haven't met me. 14:38:20 I'm the docs PTL 14:38:27 is it largely the same as any other OS project? 14:38:29 robputt: exactly the same as dev. If you need help with it though, let me help you work it through 14:38:34 No changes robputt :) 14:38:34 it is the same as any other ^ 14:38:38 cool :-) 14:38:38 https://github.com/openstack/openstack-manuals 14:38:53 We like to ensure there's minimal difference. 14:38:57 well in this case I have some changes for the Newton Debian 8 manual which I picked up the other day when deploying some stuff 14:38:57 :-) 14:39:02 hehe 14:39:04 robputt: the install guide? 14:39:07 yep 14:39:17 So, we just lost our Debian guy. So, please, patch away by all means. 14:39:22 there's some issues around debian i read 14:39:31 Debian is no longer, in summary. 14:39:32 the packaging guy is gone or something 14:39:39 We had one guy who was helping the team to package and manage the guides is gone. 14:39:43 damn 14:39:56 yeah, hopefully it will get worked out 14:39:58 The docs team already managers 3 distributions (well, 5) so keeping on debconf and debian was a hard slog. 14:40:00 there is a long thread about it 14:40:07 So, we've had to drop it until another package manager comes our way. 14:40:14 I can't promise packages, I hate packaging Python stuff inside of anything over the PyPi, but the docs I can submit updates for sure 14:40:28 robputt: love your stuff :) it's not up in Ocata :( But Newton would love some attention if you have the time. 14:40:51 I'm actually in the process of looking at the ops-guide we have up and will be owrking with our team leads to improve it and the admin guide 14:40:55 https://docs.openstack.org/ops-guide/operations.html 14:41:05 https://docs.openstack.org/admin-guide/ 14:41:13 If anyone's interested in helping out these guys with me, please let me know :) 14:42:21 awesome, i'm going to go to the next topic because of timing 14:42:26 Please do :) 14:42:26 thanks asettle ;) 14:42:45 np, asettle i'll poke you later about docs stuff I have a few more queries 14:42:53 going to skip over ancillary tooling if no one objects 14:42:56 robputt: jump on over to our chan if you have any questions :) #openstack-doc 14:43:37 and osic like environments 14:43:46 well 14:43:47 #topic Underutilized Projects 14:43:48 hang on 14:43:54 ok, going back 14:43:56 to which one? 14:43:56 lets chat about osic like envs quickly 14:43:58 :P 14:44:10 #topic OSIC-like Environments 14:44:15 so I currently have "RobCloud" my personal Openstack playground 14:44:43 which massively sucks but I try stuff out in there which I want in a vanilla openstack deployment 14:45:05 So, what opportunity is there for us getting access to OSIC like envs for testing out stuff 14:45:13 in particular emerging Openstack projects? 14:45:20 anyone can request resources from osic 14:45:28 and for emerging projects even the more 14:45:33 ok... but what about if we want to have control plane access? 14:45:51 or are we expected to build a cloud ontop of the cloud for testing? 14:46:11 you get baremetal nodes 14:46:17 ohhh ok cool :-) 14:46:30 and can do what your heart desires with tnem :) 14:46:42 tbh its probably the same overhead as what I have now then for trying stuff out if you get bare metal 14:46:48 beyond that though, there are other options as well and i was hoping to discuss 14:47:03 so these bare metal nodes, are they Ironic driven? 14:47:11 so you want openstack already live 14:47:29 they are ironic driven more than likely by now or definitely will be 14:47:36 ok 14:47:37 :-) 14:47:39 there is work to move all of the nodes under ironic 14:47:45 some are, some are not unfortunately 14:47:48 let me take this offline mrhillsman 14:47:52 ok 14:48:09 there is also cloud native foundation and few other such osic-like environments 14:48:14 but I would like to make use of OSIC for my Openstack adventures rather than my current setup as its a bit crummy ;-) 14:48:22 i was wondering if anyone knows of any, has any, is planning on having any, etc 14:49:19 i have a contact at AT&T who mentioned having something like osic there 14:49:22 but not sure what that means 14:49:28 so i will go on to the next 14:49:33 time running out hehe 14:49:39 #topic Underutilized Projects 14:49:54 Yep, we need to get in here :-) 14:49:55 big time 14:50:01 #link https://etherpad.openstack.org/p/ops-adopt-a-project-pike 14:50:28 so beyond the general issue of projects not having enough feedback and new ones needing more than those that have been around 14:50:44 its not so much feedback 14:50:52 we started an effort in osic to help those that folks want to use 14:50:57 but just need that little extra push 14:51:03 I would like us to get very early involvement so we can shape a product around operational acceptance from day one. 14:51:07 and i think the thought is good across the board 14:51:51 how can we get that to work 14:51:59 i assume we need test environments 14:52:05 tying into the osic-like topic 14:52:14 though they do not have to be large 14:52:24 but devstack would not be sufficient 14:52:34 from an operational standpoint 14:52:50 So we tried ironic. But honestly the "friction" between the ironic team and the nova team and the pace of getting something out there that was not crazy pants - was untenable. 14:52:51 It doesn't need to be a large deployment, but silly stuff like 14:52:57 "how does it scale" 14:53:04 is often not even covered in new projects at all 14:54:02 klindgren i'd like to hear more 14:54:14 Something as simple as configuring a raid array for r10 or r5 or r1 & 5. The answer was "use seperate pools for each raid array. 14:54:59 Which means that I need to have 3 different config/pools of hardware for 1 piece of hardware 14:55:28 for requests to be pulled from. Ironic was build some placement api thing, for NOVa , but that took like 2 cycles - not even sure where it is in ocata 14:56:03 would like to hear more offline klindgren as it could be an issue going forward in general 14:56:18 Its also super easy to get an ironic node stuck 14:56:21 and probably something that needs to pushed up for discussion 14:56:30 reboot it while deploying an image to it - it will bascially never come back 14:56:40 so we have not so new projects and very new projects 14:56:45 must go to the db to clear the state, and remove the vm from nova 14:57:10 early ops engagement for new and emerging projects would be a great idea imo 14:57:15 we have a whole load of ugly around our ironic deployment :-( 14:57:48 i'd like to hear more about how that looks like 14:57:55 well, just for example 14:57:58 if we look at Picasso 14:57:58 new and emerging 14:58:03 which is an uber new project 14:58:15 and a pretty important one in my opinion, but maybe thats just my love for FaaS 14:58:24 #link https://wiki.openstack.org/wiki/Picasso 14:58:34 it currently for me isn't really heading in the direction of operational acceptance in my opinion... 14:58:39 Some of the variables are one-sizes fits all (like the stty number to use for serial console on serial over lan) - but all hardware is snowflakes. So I actually have 2 peices of hardware that need different stty numbers. No way to handle that in ironic. 14:59:02 For example, it is not configurable like a standard Openstack service, it doesn't use Oslo libs for stuff like DB / Messaging 14:59:09 i'm going to have to kill our meeting at 9 but feel free to keep going 14:59:17 i will capture things for next meeting 14:59:26 it doesn't have a pluggable backend for using with multiple function runners etc... 14:59:34 to me, as an operator I expect things to use the standard libs 14:59:40 and to behave in an Openstacky kind of way 14:59:52 and we have to have buy-in from those projects no doubt 14:59:53 once you have configured 1 OS project you should be able to configure them all 15:00:00 so really our engagement should be at the blueprint stage 15:00:06 yes totally 15:00:20 #endmeeting