20:01:20 #startmeeting tc 20:01:21 Meeting started Tue Feb 7 20:01:20 2017 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:23 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:25 The meeting name has been set to 'tc' 20:01:29 o/ 20:01:42 Hi everyone! 20:01:44 Our agenda for today is at: 20:01:47 #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee 20:01:55 remember to use #info #idea and #link liberally to make for a more readable summary 20:02:02 #topic Deprecate postgresql in OpenStack 20:02:06 #link https://review.openstack.org/427880 20:02:10 sdague: o/ 20:02:21 quick intro ? 20:02:21 right, this came up on the mailing list again recently 20:02:47 the basic issue is that pg is really very unevenly supported in our community 20:02:58 we dropped the bulk of the gate testing 5 months ago 20:03:01 and almost entirely untested 20:03:05 right, that 20:03:14 the number of actual users is small (per survey results) 20:03:22 from the comments on the thread and review, it appears some people still use it though 20:03:32 SUSE's OpenStack product uses postgres and tests if heavily 20:03:35 sdague: +1 making this explicit for users, it keeps surpising people 20:03:47 and with only so many resources to go around, I think we should say once and for all that this isn't an upstream supported thing 20:04:13 ttx: some people still use a number of things 20:04:25 I'm sure more use nova-net than postgresql 20:04:31 quick point of info...Huawei uses a DB based on Postgres and that mean DT, China Mobile, etc. 20:04:43 So, so big users, or soon to be big. 20:04:45 this seems like one of those things like stable branches and requirements, where folks want to use it but don't contribute to it 20:04:49 We've done this with other tools technologies in the community 20:04:50 the trick will be coming up with a soft enough way to say we're not actively testing it and won't know what we're breaking without coming out and implying that it just won't work 20:04:52 yah 20:04:57 what dhellmann said 20:05:02 Rockyg : what is "a database based on postgres"? 20:05:08 o/ 20:05:10 the problem isn't anything specifically against postgres - it's that it breaks and nobody cares 20:05:12 we have removed support for qpidd, for example 20:05:20 dhellmann: sounds like a postgres fork 20:05:20 Iwait a sec... 20:05:28 postgres is obviously a fine database platform 20:05:52 G 20:05:55 so, nothing against postgres but unless people raise their hands and help maintaining it, I think I'm fine with removing support for it 20:05:57 mordred: +1 this is more our inability to support both 20:05:58 Guass 20:06:00 db 20:06:03 but if people think it's important from an _upstream_ perspective, people need to pony up to support it 20:06:03 I still prefer postgres over mysql because I don't get angry every time I have to figure out how to set up a user 20:06:06 flaper87: +1 20:06:07 GaussDB 20:06:13 mordred: i'm not sure that's accurate. i think it just takes longer to raise since it doesn't get surfaced as early 20:06:18 So is there an option for one of these users to step up and offer to fully support it? 20:06:18 to be fair, if someone wanted to come upstream and "contribute to making postgres work", it's super unclear what that person should do, other than waiting for it to break and jumping on it 20:06:20 dhellmann: funny, that's one of my big beefs when I try to use postgres :) 20:06:26 mordred : heh 20:06:31 I wish we'd use an abstraction layer that lets us ignore it 20:06:34 dhellmann: heh, I have the opposite problem. I can never remember how to add postgres users :) 20:06:43 ttx: ideally that's what sqlalchemy would be 20:06:54 ttx: well - so this is my other opinino, where I become more opinionated 20:06:56 So, this just got raised with Huawei by Gordon. I'll need to spin them up for support in community 20:06:58 e.g. should that person go contribute patches to optimize better for postgres, if that's what is chosen? 20:07:02 it feels like it leaks implementation details up though 20:07:04 i think we also need to keep in mind that the term "support" in our (upstream) case just means we do or don't actively try to avoid breaking something, distinct from distributors "supporting" some combined solution they're actively testing and taking contracts from customers over 20:07:11 ttx: should be oslo.db and/or sqlalchemy for that matter 20:07:15 ttx: abstraction layers are fine when you don't need optimizations or are using basic common parts 20:07:29 large scale things on rdbms' should be able to use advanced features of their rdbms 20:07:32 but at some point you have to decide you are either using your technology stack effectively 20:07:34 but apparently it's not abstract enough because there are specific optimizations 20:07:34 sdague: the question then becomes, how much do we rely on those advanced features 20:07:41 unfortunately, that is not the place where databases are similar enough to abstract 20:07:45 or just abstracting for abstracting sakes 20:08:00 contribs/patches welcome, 3rd party ci welcome, no CI jobs in community? 20:08:26 mordred: sdague: +1, I think that is the real reason to do this, not that people aren't willing to work on it 20:08:33 but does that let us optimize for mysql? 20:08:49 I mean, currently we are (about to) saying that projects can assume an oslo.db database is present in an openstack installation 20:08:51 given that there are distributors preferring postgresql over mysql for openstack, i wonder whether us officially stating that we don't support it will cause them to rethink that or will just resort in more downstream patching/forks so they don't have to switch to mysql 20:08:56 dims: I would like to open the doors for our team to optimize for _a_ database 20:08:58 Shoud we say "MySQL" instead ? 20:09:05 mordred: ++ 20:09:08 mordred : agree 20:09:19 I believe the inertia is there for "A database" to be mysql 20:09:25 especially because that also opens up optimizing the HA scenarios into the environment 20:09:29 but I would be just as happy if a database was postgres 20:09:37 sdague: exactly 20:09:43 And if we say that we write code for MySQL only, then should ditching PG support (in projects that support it) follow deprecation policies ? 20:09:48 If we have to specify one, can it at least be an Open database? Like MariaDB? 20:09:58 so I had a question, currently its MySQL "like" as in does it include MySQL+galera, MariaDB, MariaDB+galera, etc 20:10:11 johnthetubaguy: I think mysql-like is the right answer 20:10:14 edleafe: the differences between maria and mysql are miniscule 20:10:27 the optimizations across those sets would be pretty close 20:10:27 edleafe: if we get to the point where they matter, let's deal with it then 20:10:28 mordred: how long will that be the case? 20:10:30 sdague: doesn't it expose some implementation details that would require extra testing though ? 20:10:33 sdague: yeah, I wasn't sure how to expess that I guess 20:10:34 mordred: in production, sure. I meant the licensing 20:10:39 over time i have a feeling mysql, mariadb and the other mysql forks will diverge to the point where we would still need to "pick one" 20:10:44 ttx: possibly, but we have to start somewhere 20:10:46 fungi: ++ 20:10:49 so 2 more releases with no postgres in our CI, then we optimize for mysql-like databases? 20:11:01 we'll have the same problem again just withing a smaller family 20:11:11 edleafe: has mysql stopped being gpl and I didn't noticed? 20:11:15 fungi: ++ 20:11:18 is this 'optimising' still all hypothetical? 20:11:30 So we could do a two-stage thing. Say now that in the future only MySQL will be supported. Then in two releases ditch all PG support 20:11:33 the concern with future divergence supports my assertion that the resolution should state "yes mysql" not "no postgresql" 20:11:38 if we say "mysql-like" does that include the clustering thing the folks from oracle are interested in us supporting? 20:11:43 cdent: yes 20:11:44 cdent: ++ 20:12:05 ++ 20:12:05 because that seems to have some schema constraints 20:12:06 cdent: yes, I thought the same thing 20:12:20 so, I'm fine adjusting the language as long as we are getting to the same place before we all rage quit 20:12:21 (not that I really want to vote for mysql if the reason for it is because we want "optimizations" for our little databases ;) ) 20:12:24 dhellmann: it's a great question. I would hope so - but I think we need to think about it 20:12:30 cdent: I would likely retrofit it in the "base services" stuff, so it would be "yes MySQL" 20:12:35 gordc: I have read reports of some tools not working with pg. Nothing very widely deployed. 20:12:39 mordred: IANAL, but stuff like this makes me concerned: https://www.mysql.com/about/legal/licensing/foss-exception/ 20:12:47 cdent: when I say "optimizations" I do not mean like speed optimizations 20:12:54 edleafe: that's been there literally for forever 20:13:00 dhellmann: I would rather we picked a clustering one because thats more restrictive, like MariaDB + galera, but yeah, there are differences 20:13:02 some operators won't want to remove pg from their cloud, they probably have customers running clouds in production, they'll have to maintain the deployments. Not sure we've thought about it until now 20:13:12 mordred: And I've not liked it for literally forever 20:13:12 edleafe: it allows foss projects that link against libmysqlclient to not need to gpl their application 20:13:22 persia: it is being used so i imagine it's just the lag between product team noticing and patching it upstream 20:13:44 sdague: does the two-stage thing I posted above make sense ? 20:13:44 EmilienM: but I think they did that based on an unrealistic expectation of what is happening in upstream 20:13:48 johnthetubaguy : right, my point is if we're worried about managing support for 1 database in order to optimize for it, having 2 clustering solutions with different requirements seems like it's going to be a hard thing to sell 20:14:14 dhellmann: that feels like the same problem in a slightly different flavor 20:14:16 cdent: when I say optimizations- I mean what sdague mentioned - being able to bake in more assumptions about how HA and friends work 20:14:19 dtroyer : yep 20:14:32 AJaeger: what is your longterm roadmap at Suse, to keep pg? Have you already prepare a migration etc? 20:14:32 sdague: i.e. make a statement now, then start actively ditching things starting in two releases 20:14:43 because HA stories and whatnot are where the differences between things like mysql and postgres diverge MASSIVELY 20:14:45 dhellmann: in my head I was supporting the top 4 most deployed databases, as best we can, I know that means we will have to do things differently to whats happening today 20:14:49 mordred : it's also a matter of training folks in the community to recognize the sorts of things that can/must be done for optimization 20:14:53 ttx: I think that the dev tooling should dump things now, we stopped testing with it in Newton 20:14:55 (it would be great to also hear from operators) 20:14:56 dhellmann: yup 20:15:03 EmilienM: no migration planned yet. 20:15:07 EmilienM: we have no plans to migrate from postgres either 20:15:14 in pike we should remove it from devstack 20:15:28 sdague: apparently doesn't prevent people from using it. They are surely at risk, but not a reason to remove features with no warning 20:15:36 what gordc and AJaeger said is a concern we can't ignore it 20:15:37 EmilienM: but this discussion here started already discussion 20:15:44 johnthetubaguy : I guess we need to be clear about what "support" means there, because I think 4 may be too big of a number 20:15:49 sdague: "we stopped testing with it in Newton" <- which we is this? nova? openstack? 20:15:53 sdague: not sure what you mean by dev tooling, might be saying the same thing I say 20:15:56 jroll: openstack 20:16:00 jroll: openstack 20:16:04 jroll: we removed the dsvm postgres jobs 20:16:08 dhellmann: +1 I meant to ask that question, I am not quite sure what that means yet either 20:16:10 the main base layer jobs were deleted 20:16:12 all of them? 20:16:18 huh, TIL 20:16:19 pretty much 20:16:25 gate-tempest-dsvm-ironic-pxe_ipmitool-postgres-ubuntu-xenial-nv 20:16:28 ironic tests it, apparently 20:16:32 except apparently for ceilometer, since that's how the issue came up 20:16:34 jroll: we had it in ceilometer which is where we always find the postgres issues 20:16:41 neutron has a periodic job for psql 20:16:46 and ceilometer has a job up to remove it as well 20:16:54 jroll: we didn't delete *postgres* just the job that ran everywhere 20:16:55 has a change I mean 20:16:59 jroll: this came up because ceilometer still tries to do some tests with postgres and there were recent changes which broke it because we don't generally integration test with that any longer 20:17:02 so projects that had one offs kept them 20:17:06 mtreinish: so "we" is the integrated gate. not openstack. 20:17:08 er, what gordc said 20:17:14 fungi: :) 20:17:27 fungi: yep, I've been reading the thread 20:17:35 ISTM that there's a lot more going on with postgresql in the community, dev and deployer, than we originally thought 20:17:41 reference discussion of dropping gate job http://lists.openstack.org/pipermail/openstack-dev/2016-August/thread.html#101892 20:17:47 dhellmann: I actually don't think that's true 20:17:55 thingee: thanks for that 20:17:56 sdague: basically I agree that we should align documentation with reality (now). I'd like to give a transition period before we actively break anything for people for which that may actually be working fine 20:18:10 sdague : I hear you saying we're not testing it, and I hear 3 projects saying they are 20:18:11 of which there seems to be a lot 20:18:11 ttx: ok, but define actively break 20:18:18 ttx: we seem to be breaking things anyway 20:18:21 dhellmann : the main thing i am worrying about is moving all the existing user deployments.. 20:18:25 the lack of postgres representation in the user survey has been very consistent 20:18:26 (because of not testing it) 20:18:27 to be clear, I'm not advocating for or against dropping postgres here, I just want to be clear that we're talking about the integrated gate, not openstack as a whole 20:18:28 right, we took away all that testing 20:18:30 mordred: ++ 20:18:45 sdague: like remove code that works around pgsql glitches 20:18:48 mordred: ++ 20:18:54 dims: what about the ones who don't want to leave pg? 20:18:54 (not in tests, in service code) 20:18:56 jroll : my impression is that we're talking about encouraging all projects to drop psql support 20:19:04 that's what the end of the resolution says 20:19:06 EmilienM : right 20:19:14 I think, like with everything else, we should provide a transition plan out of psql 20:19:22 dhellmann: I think the resolution encourages all projects to ignore it 20:19:23 it would be interesting to get more information on wy some operators are deploying with postgresql, and even moreso why some distributions chose to standardize their openstack designs on it 20:19:30 flaper87: my thoughts too 20:19:32 * flaper87 doesn't like to assume it'll all be good 20:19:32 dhellmann: sorry, I mean when we talk here about how we've stopped testing it 20:19:45 dhellmann: yeah I prefer to think the way cdent put it. yes mysql. 20:19:47 so, I don't think there will be a transition plan 20:19:48 ttx: "And that the recommendation is they remove it from their code, documentation, and testing at the earliest convenience." 20:19:50 I understand the "why" of the proposal. I'm just very worried about our community who use pg, like Suse or Huawei and probably much more 20:20:08 sdague: ++ 20:20:09 the fact is, it's level of community support is very low 20:20:13 when we say drop support, do we mean actively break it, or make it low priority and let fixes in still, or deprecate then break it? 20:20:30 EmilienM: that's why we have a deprecation procedure 20:20:36 flaper87: yes that but until it's planned, we shouldn't remove support or jobs like we're doing 20:20:40 dhellmann: right, that's the part I disagree with. 20:20:41 johnthetubaguy: i think the current state is the 2nd 20:20:42 saying "we have to have a transition plan" means we actually need a ramp up on it, to have any idea that it works, plus transition tooling, plus... docs? 20:20:45 sdague: transition plan, 1 cycle notice.. tomatos, tomatos 20:20:50 johnthetubaguy: " make it low priority and let fixes in " seems the least harmful 20:20:53 EmilienM: yup 20:20:54 in reality it's a small percentage of users, and we can drop support for it gracefully 20:20:56 manila gate tests do ensure postgres compatibility, but if the rest of the projects start allowing postgres bugs to go unfixed we'll eventually have to drop support too because we won't be able to test 20:20:58 dhellmann: I disagree with actively breaking it without a deprecation period 20:20:59 sdague : so another way is to get folks to show up to help? (dunno how, but...) 20:21:11 dims: but we don't want other folks to show up 20:21:27 sdague : it is not clear that everyone feels that way. I'm unconvinced. 20:21:29 * jroll points out that switching database engines is WAY harder to do than a normal feature or config deprecation, which is what our deprecation process is designed for 20:21:29 I'm also not advocating for keeping pgsql, but this decision will further prevent the user survey changing or for it to become the defacto 20:21:31 ttx: I think I am with you, I would also say "formal deprecation" period 20:21:33 right. we _want_ to be able to do better things with HA and similar stories so that we can provide a better product for our operators 20:21:39 jroll: ++ 20:21:40 jroll : true 20:21:54 mordred: ++ 20:21:55 I suggest to have at least "low prio and get bugs in " for the transition period. 20:21:55 folks probably have more things built on top of openstack 20:22:04 because of the fact we did support pgsql (probably without many testing), we can't just throw it away like this 20:22:04 and better compatibility on the API 20:22:10 jroll: +1 20:22:13 mordred: I agree with the end goal. I disagree with actively breaking people for which that might accidentally work 20:22:14 * AJaeger likes to be able to upstream patches 20:22:20 sdague: yes compat on the API 20:22:21 EmilienM: you assume that we did support pg officially 20:22:26 at least not without a deprecation notice/period 20:22:28 AJaeger ++ 20:22:35 ttx: sure. I do not have any problem with deprecation periods 20:22:46 That's the crux of my issue with the requirement of making this super graceful to get rid of it 20:22:52 sdague: we did or we didn't. The reality is some Clouds probably in production are using it and we can't ignore it now 20:23:12 EmilienM: ++ 20:23:13 the reason pg was in the gate, was because I spent 2 months making that happen in 2013 over some unrelated licensing concerns 20:23:21 the way I see this, the deprecation period here would help concerned folks get together I find the best way forward, the really important job we need to do here is wave the big red flag that something needs doing 20:23:33 johnthetubaguy : ++ 20:23:34 johnthetubaguy: ++ 20:23:42 we collectively allowed people who figured out how to get openstack running with postgres to add sections to official deployment documentation saying how to do it, for example 20:23:43 agree johnthetubaguy 20:23:50 we could also say "unless someone shows up to actually test it, we'll remove support for it" 20:23:54 fungi : ++ 20:23:59 fungi: we did, without any resolution 20:24:01 "because in OpenStack, what's not tested is broken" 20:24:10 ttx: what does 'test it' mean? 20:24:12 and this is about setting direction that we want out of that game 20:24:20 is it allowed to directly go ask people who we know are running it to pony up resource to support it? the passive option of waiting doesn't seem great 20:24:25 gordc: maintain functioning gate ? 20:24:31 ttx: I'd rather we didn't say that, to be quite honest, because there is still a ceiling on how good we can make openstack when we're trying to engineer around 2 different wildly-divergent database backends 20:24:33 cdent: I honestly don't want to go down that road 20:24:37 mordred: ++ 20:24:41 ttx: "test it" and also "proactively maintain it" 20:24:52 mordred: that is a good point. Like I said, I agree with the end goal 20:24:57 ttx: hardware resources? or development? iirc, the original reason for removing was mainly due to hardware constraints 20:24:58 nobody in the world who is running LARGE systems does it on "either mysql or postgres, kinda whichever" 20:25:03 I'd be OK with the idea of saying we don't want maintenance if we have an extended deprecation period 20:25:07 that's the big point, by making this an abstraction layer instead of a foundation, we fundamentally limit openstack progress 20:25:08 worse, when trying to engineer around nuances of johnthetubaguy's four different database backends ;) 20:25:12 sdague: ++ 20:25:31 fungi: better than five though right ;) 20:25:35 sdague: so we should not say we remove pgsql because it's broken, but because it prevents us from relying on advanced DB features 20:25:37 fungi : right, I think we're going to end up having to pick a clustering solution, too 20:25:38 and, I hear, people get frustrated and rant about pace of openstack progress some times 20:25:44 gordc: I think the biggest challenge is to maintain CI jobs running well with pg (and deal with failures when they happen, like we currently do with mysql) 20:25:56 so, I think there's agreement on the goal but not entirely on how we get there 20:26:05 dhellmann: yes. being able to pick one of those would be amazing and allow us to move the needle forward 20:26:11 should we proceed with voting either here or on the review? 20:26:14 sdague: are they saying we are too fast or too slow ? Because I hear both 20:26:23 ttx: I hear both at the same time many times :) 20:26:32 EmilienM: periodic gate like neutron has now? i know the bugs we find at product end have been pushed back into upstream 20:26:34 Being more opinionated about things like this (messaging, etc too) will let us speed up somewhat 20:26:51 but there's a noticeable lag between upstream patch to product team 20:26:51 sdague: +1 we will have to be more opinionated to go faster, and keep going 20:27:02 I feel we're starting to say the same things so I'd rather move onto the next phase of the discussion 20:27:10 flaper87: ++ 20:27:14 flaper87: yes please 20:27:18 so what do we write on the big red flag? 20:27:27 johnthetubaguy: a sonnet? 20:27:34 Sounds like there is some consensus on the end goal, but diagreement on the way to state it and how fast to proceed 20:27:34 so, should we vote on whether we should have a deprecation period here or on the review ? 20:27:44 mordred: if that makes people read it, it could totally work :) 20:27:47 flaper87: I propose we continue on the review 20:27:48 I think a good first step would be make sure install guide says mysql 20:27:49 johnthetubaguy: :) 20:27:52 ttx: ++ 20:27:52 ttx: I like that 20:27:53 I don't think any of us would disagree with that 20:27:59 it will take some time to hammer out 20:28:13 the changing existing pg deployments is the hard part 20:28:21 jroll: It does already 20:28:23 ttx: maybe we should also tee up a session at the Forum to discuss this 20:28:30 mordred : ++ 20:28:32 jroll: and only MySQL/MariaDB 20:28:33 ++ 20:28:33 also as has been pointed out, this will serve as prior art for the message bus discussions still to come 20:28:34 mordred, ++ 20:28:35 AJaeger: it doesn't mention postgres at all? 20:28:37 oh nice 20:28:44 fungi: ++ 20:28:51 also at ops midcycle 20:28:55 jroll: Correct -t he Install Guide shows basically one path - and opinoined way to install 20:28:55 jroll: it says postgres is also supported in install guide 20:28:56 #action mordred to propose a pgsql discussion at PTG (and probably another one at Forum) when time comes 20:29:03 jroll: again, what gets exposed in the community around pg is really inconsistent 20:29:06 mordred: thanks 20:29:06 * jroll reads conflicting information 20:29:07 but all the steps are for mysql 20:29:07 ttx: wow. I shoudl have seen that coming 20:29:12 mordred: I'm fast 20:29:18 OK, I propose we move on 20:29:21 +1 20:29:21 sdague: yeah so we should clean up stuff that new deployments would use to just mysql 20:29:24 yes 20:29:25 I think we need something before then, though, but we can talk on the review about that 20:29:26 and go from there with existing stuff 20:29:28 jroll: yep 20:29:35 then==forum 20:29:35 i still mostly want to know _why_ people are chosing the less popular option here, so that we have better data on whether it's just an arbitrary choice or there are definite behavior/performance needs not being met by the default option 20:29:42 let's iterate on review, and discuss next topic 20:29:56 dolphm: around ? 20:29:56 sounds good 20:30:06 fungi: +1 me too 20:30:08 ttx: o/ 20:30:34 #topic Disallow downtime of controlled resources during upgrades 20:30:37 #link https://review.openstack.org/404361 20:30:42 The language in the latest draft looks good to me 20:30:52 I guess the main unknown is how exactly you would validate "accessibility of a controlled resource" throughout the upgrade process 20:31:00 Some grenade code ? 20:31:29 ttx: some tests, yes, could be grenade 20:31:34 an ssh session being open, kept coming up, just saying that for reference really 20:31:35 ttx: grenade does poll things while the services are shutdown 20:31:42 i'm hoping to answer that at the PTG, but yes... my thought is to extend/adapt the resources that grenade creates beforre an upgrade, to also be exercised during the upgrade 20:32:16 mtreinish: so, if that becomes an asynchronous an ongoing validation process, i think ti satisfies the tag for the projects it covers 20:32:34 that seems like a reasonable approach, and I think we can add details to the tag with some examples later when we have some 20:32:43 mtreinish: ok, so enough to assess "accessibility of a controlled resource" 20:32:45 I think that's going to be the basic challenge, everything we do in testing today is sequential 20:33:25 because it's not just the validation, it's how the actual upgrades are done as well 20:33:38 sdague: +1 was just tying something like that 20:34:21 ttx: well it's not a continous check more after thigns are shutdown can I still ping server x 20:34:40 mtreinish: I think that the new wording is good with that 20:35:12 ttx: i disagree, the keyword being "throughout" the upgrade process 20:35:22 * ttx looks up latest patchset 20:35:34 today, the accessibility of controlled resources is tested basically once during the upgrade process 20:35:46 dolphm: well, 3 times 20:35:46 ttx: L59 https://review.openstack.org/#/c/404361/8/reference/tags/assert_supports-accessible-upgrade.rst 20:36:05 before services drop, when they are down, after they are upgraded 20:36:11 sdague: once before the upgrade begins, once during the upgrade process, and once after the upgrade? 20:36:11 Should we add some precision so that testing 3 times is enough ? 20:36:24 I just want to remove the uncertainty there 20:36:24 one could imagine perhaps opening an ssh connection to a vm in a thread/subprocess, then doing the upgrade, then making sure the ssh session doesn't drop 20:36:36 on the ssh session idea... ssh can resume seamlessly after rather lengthy periods of total packet loss 20:36:39 mordred: or that 20:36:42 I don't think we need to solve the implementation ideas in this meeting, do we? 20:36:48 nope 20:36:52 nope 20:36:53 not evne a little bit :) 20:37:01 well, I think we do want to have an implementation before the tag though 20:37:10 ok, so we can judge that when the tag is proposed 20:37:18 to a specific project 20:37:20 that's true 20:37:21 I'm more musing on that by way of exploring what sorts of things this could be wanting to require 20:37:21 because if we think it's testable, we should work that out first 20:37:26 I'm not sure we do. We've clearly agreed on "throughout". Different projects may have different ways of testing that. 20:37:30 you can ssh to an instance, suspend the instance for an hour or more (in the right network environments at least), resume the session and then go back to using that same socket without it ever disconnecting 20:37:30 we get to revise the tag until someone gets the tag? 20:37:45 I suspect we'll refine the tag definition when a project actually asks for it :) 20:37:46 dhellmann: ++ 20:37:47 plus, we all need to realize that testing is a model 20:37:53 sdague: I still wonder if it has value in saying "hey we don't test this thing yet" 20:37:54 johnthetubaguy : or after, if it's a backwards-compatible change 20:37:59 the point isn't to catch everything 20:38:04 * ttx is good with that. Baby steps 20:38:09 sdague: ++ 20:38:10 the point is to catch representative errors that people missed in code review 20:38:19 If you feel good, pile up +1s and I'll approve now 20:38:38 sdague: +1 20:38:50 right, which is why i think it's sufficient to just test sample (once, twice, whatever) during the process and assume things weren't offline in between and miraculously came back just in time for the next poll 20:39:04 though that's also a good way to sneak nondeterminsitic failures in 20:39:08 One more +1 to approval 20:39:14 Any objection ? 20:39:18 I feel we should merge what we have, and adjust it if we need to before we approve the first project to get the tag 20:39:24 johnthetubaguy: yes 20:39:25 johnthetubaguy: that seems fine 20:39:30 fungi : ++ to using reasoning to apply good, minimal testing practices 20:39:33 +1 20:39:39 +1 20:39:41 johnthetubaguy: +1, baby steps 20:39:41 Alright, approved 20:39:53 Thanks dolphm ! 20:39:59 dhellmann: and keeping an eye out for races. polling-related test like that are a great surface area to grow race conditions on 20:40:00 ++ thanks dolphm 20:40:03 #topic Progress on stalled reviews 20:40:07 fungi : true 20:40:08 * Status update on Golang CTI 20:40:11 #link https://review.openstack.org/410355 20:40:14 dtroyer: how is that going ? 20:40:21 cool, landing these tags has certainly driven a lot of conversations in the right direction (reasonable testing practices, as someone just mentioned) 20:40:23 so thank you all! 20:40:29 fungi, ++ 20:40:52 I need to get back to it, runup to release has consumed me, but I think it's pretty close 20:40:53 dolphm : thanks for working on this 20:41:17 * dims needs to review that golang_cti in depth 20:41:23 dtroyer: ok, so WIP but you'll pick it up soon 20:41:26 I kept flaper87's doc in mind and IIRc there may only be minor changes to fit that 20:41:27 On that same topic... 20:41:30 flaper87: do we want to grandfather in that step 1 for golang addition (the use case analysis) based on the Swift case ? 20:41:37 Did you find any volunteer to file that ? 20:42:02 I sent the email out and reached out but no one signed up for it AFAICT 20:42:08 unless I missed something entirely obvious 20:42:21 I would love to do a first runtest of this process with Swift 20:42:24 I've been operating under the assumption that we can refer to that as necessary 20:42:25 ok, maybe reach out to the ones that should be interested see if they still care 20:42:42 ttx: erm, done that already 20:42:51 * flaper87 shrugs 20:42:56 ok. Maybe we could tie loose ends at PTG 20:43:02 * Status update on Driver discoverability / need for driver team recognition 20:43:10 A couple weeks ago we tabled that discussion while waiting for (some) progress on driver discoverability 20:43:11 yes, I'm hoping for that and feel free to action me on it 20:43:18 I was wondering when we would be comfortable to reconsider those various options for driver team recognition 20:43:41 #action flaper87 to tie loose ends at PTG re: step 1 for golang adoption (use case analysis) 20:43:54 So when is that ? post-PTG ? Later ? 20:44:14 thingee: I suspect there won't be much visible progress on discoverability before a while ? 20:44:26 +!1 for post PTG to take the drivers discussion again 20:44:35 +1* 20:44:46 Also a bit of PTG beers might convince the ones or the others 20:44:49 yeah, post-PTG 20:44:56 beers works too 20:45:39 it sounded like there was a fair amount of interest from the www site admins on revamping/improving the driver marketplace and solving the need to automatically aggregate driver details from structured data under the control of individual teams 20:45:48 ttx: not yet 20:45:54 fungi: yes, but will likely take some time to implement 20:45:58 agreed 20:46:16 it seems we have a path forward on that front at least, if not a timeline (yet anyway) 20:46:21 ok, let's revive it post-PTG and use that week to informally discuss it 20:46:27 ttx: current noticable progress is neutron moving towards the same common format that nova started for listing drivers. I would like to start communication with other projects to follow 20:46:45 thingee: sounds like a good step forward! 20:46:52 cinder as I believed from various people in the team back in barcelona are also going to be moving towards this same common format 20:46:52 * Progress on Introduce assert:supports-api-compatibility 20:46:59 thingee : is neutron listing drivers that are outside of its repos? 20:47:04 #undo 20:47:05 Removing item from minutes: #action flaper87 to tie loose ends at PTG re: step 1 for golang adoption (use case analysis) 20:47:10 arh 20:47:12 once that's in place the foundation will work with the community on displaying things in the market place with the common format. 20:47:18 #action flaper87 to tie loose ends at PTG re: step 1 for golang adoption (use case analysis) 20:47:23 ttx: hah :) 20:47:26 ttx: stop taking work away from me 20:47:32 no wait, do it! take work away from me 20:47:34 I need #subtopic 20:47:39 dhellmann: yes 20:47:42 that meetingbot always surprises 20:47:44 thingee : ok, good 20:47:51 everytime I undo it's never what I want either 20:47:51 #link https://review.openstack.org/418010 20:47:59 ttx: patches accepted :) 20:48:03 This one was blocked while the API-WG refreshes their "API changes guidelines" that this proposal references 20:48:12 yeah, still working on that 20:48:23 it's a fun mailing list thread 20:48:35 in the rush of release, has been slow generating the necessary input to get a full picture of the context 20:48:38 mordred: already need to add Slack-like 'notify me when I'm mentioned in another channel the bot lurks in" feature 20:48:41 dhellmann: https://review.openstack.org/#/c/318192/ 20:48:44 but it's getting there 20:48:50 thingee: EDONOTWANT 20:48:52 gah 20:48:57 ttx: EDONOTWANT 20:49:19 mordred: ? 20:49:20 mordred: would let us get rid of common meeting channels though :) 20:49:23 thingee : maybe we can move some of that css and templating into oslosphinx when we have 2 that are similar enough 20:49:50 cdent: ok cool, the other review is stalled while we wait for that step to complete 20:50:01 dhellmann: yes there is already duplication happening with the script that generates it. 20:50:12 ttx: yeah, that's the somewhat regrettable but desired outcome 20:50:13 ttx: to be fair a lot of projects have been living under the current guideline for a while 20:50:23 it was a tc approved thing back in the day 20:50:34 Cinder patch for driver info: https://review.openstack.org/#/c/371169/ 20:50:38 mtreinish: so technically we could pursue both in parallel ? 20:50:40 smcginnis: excellent! 20:50:49 err or ppb: https://wiki.openstack.org/wiki/Governance/Approved/APIStability 20:50:54 ttx, mtreinish I don't think that a good idea 20:50:56 thingee : either oslosphinx or a new project, whatever's easier 20:51:03 the existing guidelines are flat out wrong in many ways 20:51:04 I'd also like to point out we have driver information automatically pulled out of the source and published here: http://docs.openstack.org/developer/cinder/drivers.html 20:51:08 mtreinish: archeowiking is not that great 20:51:41 and i think the modern reality of the big tent and the concept of tags changes the concepts of stability quite a bit 20:51:44 * ttx keeps on adding ETOOOLD notices to pages that some people exhume 20:51:54 ttx: sure, I'm just saying this tag was more my attempt to modernize the rules projects have been using 20:51:55 at least the discussion thus far certainly suggests that to be the case 20:51:56 smcginnis: very nice 20:52:12 ok, so let's see where cdent's modernization effort takes us first 20:52:19 shouldn't be too much longer 20:52:20 #topic Open discussion 20:52:21 go cdent! 20:52:26 * Board+TC workshop on "OpenStack Futures" 20:52:32 So finally it looks like this will happen March 8-9 in Boston 20:52:33 smcginnis: although hmm, I was kind of hoping we'd have one way to generate these. This seems different from neutron did with the .ini file 20:52:40 cdent: to be fair, I think that 90% of the guideline is pretty much going to be the same 20:52:41 a.k.a. "week of March 6 in Boston" in the poll 20:52:44 nice thanks ttx 20:52:49 mmh, does dates are horrible for me :( 20:52:50 cdent: only 10% really gets tweaked 20:52:54 There would be a 1-day workshop with the board on March 8, followed by some common evening activity (which should include food) 20:52:54 I won't be able to attend 20:52:56 smcginnis: how is nova generating theirs? 20:53:01 sdague: that's not the impression I've had from discussion, actually 20:53:02 #info all milestone-based projects have tagged their RC1 and branched stable/ocata 20:53:07 * flaper87 thought we were voting for March 6th 20:53:14 which is why I've been stalling following through on rubber stamping it 20:53:15 On March 9 the Board will hold a traditional board meeting, while interested TC members would work on the "TC vision", maybe with a facilitator from ZingTrain if gothicmindfood can arrange that 20:53:19 thingee: I'll have to see what they did. I'd rather not maintain an ini file, but if that makes everyone consistent, then all for it. 20:53:22 because there's lots of input that contradicts the cw 20:53:28 flaper87: frankly, no date was perfect 20:53:29 flaper87: :( 20:53:38 but yes, we assumed it was ok with you based on that vote 20:53:44 Those TC members not interested in working on the TC vision nor in the board meeting could go home early 20:54:03 I'm confirming all the details and will send something out 20:54:08 as we need to RSVP 20:54:08 smcginnis: I'll take a look at nova and bring it up in the thread. 20:54:15 thingee: Cool! 20:54:25 bah :( 20:54:26 Will very likely happen in DLA Piper offices, which means we have to RSVP 20:54:27 thingee: nova's is generated with a custom sphinx extension, johnthetubaguy wrote it 20:54:40 oh, sorry, danpb originally wrote it 20:54:41 The address is 33 Arch Street 20:54:45 sdague: https://review.openstack.org/#/c/371169/14/doc/ext/driver_support_matrix.py ? 20:54:53 yeah, its evolved from dan's 20:54:54 thingee: yeh 20:55:06 ttx: ok cool 20:55:10 ini file was used because parser was already in python 20:55:25 and, the alternatives were actually pretty much just a gross 20:55:39 thingee: the plan was to move that into some oslo lib, not quite done that bit yet 20:55:54 let me know if you have any questions. We coupled both things to make the travel more worthwhile and avoid yet another trip for TC visioning 20:56:10 thingee: I haven't looked at the rendering code for the driver stuff, but it sounds like the sort of thing sphinxcontrib-datatemplates would be good for 20:56:29 ttx: how firm are the dates? 20:56:46 ttx: will a formal invite (or something else) be sent out ? 20:56:48 yeah, it's right by boston common. at least there are lots of places to eat ;) 20:57:01 dhellmann: my understanding is that they are final. I'd like to doublecheck that though, since it's VERY fresh 20:57:06 like 5min ago 20:57:09 ttx; ok 20:57:12 ttx: can you send details asap so we can book flights, etc? 20:57:15 I'll wait to book a flight :-) 20:57:26 That is my plan. After all I'm the one coming from the furthest away 20:57:29 dhellmann: thingee: there are some nova OSIC folks looking at tidying ours up again next cycle, let me know if there is a better direction to consider 20:57:30 * stevemar still needs to be book his ptg flight 20:57:39 * flaper87 is literally going on PTO on the 8th, something that doesn't happen often 20:57:46 smcginnis: http://git.openstack.org/cgit/openstack/nova/tree/doc/source/support-matrix.ini 20:57:54 ttx: well, some of the board members are probably attending from asia ;) 20:58:08 flaper87 : enjoy the time off! 20:58:11 fungi: I won't send them any confirmation though 20:58:13 flaper87: well deserved 20:58:18 ttx: fair enough 20:58:32 thingee: Looks like I could probably adapt our automated way to generate most of that ini. 20:58:43 flaper87: I don't expect anything final to come out of those two days. Will be missing you though 20:58:55 Anything else, anyone ? 20:59:41 Oh, re: PTG we have an ethercalc instance now, so I'll be setting up booking forms for the discussion room and other shared resources 20:59:49 not forms, more like tables 21:00:08 so we'll be able to start booking extra stuff 21:00:14 https://ethercalc.openstack.org/ <-- yay 21:00:18 ttx: the info will all be here right? https://wiki.openstack.org/wiki/Governance/Foundation/8Mar2017BoardMeeting 21:00:19 and we are out of time 21:00:34 stevemar: mayyyybe ? I'll send something to the TC list 21:00:41 ttx: thanks 21:00:52 mordred: yes, please test and crash it so that we know if we can rely on it or not :) 21:01:02 and we are out of time 21:01:06 Thanks everyone 21:01:08 #endmeeting