15:00:41 <noonedeadpunk> #startmeeting openstack_ansible_meeting 15:00:42 <openstack> Meeting started Tue May 18 15:00:41 2021 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:45 <openstack> The meeting name has been set to 'openstack_ansible_meeting' 15:00:51 <noonedeadpunk> #topic rollcall 15:00:54 <noonedeadpunk> o/ 15:02:06 <jnamdar> I'm getting a 404 on /v1 or /v2 15:06:48 <noonedeadpunk> #topic office hours 15:07:12 <noonedeadpunk> I can't actually recall new bugs that were submitted recentl;y, so skipping that topic 15:08:05 <noonedeadpunk> I want to return to the topic of dropping branches. For some reason I thought that ocata will be just dropped, but seems like we need to mark it eol 15:08:56 <noonedeadpunk> So I wonder if we should limit to ocata? or take pike with it? as at the end it feels up to us to pick stuff to drop 15:09:56 <noonedeadpunk> we kind of discussed that later and stopped only on Ocata 15:10:28 <noonedeadpunk> so I will proceed with marking it as eol 15:10:48 <noonedeadpunk> and abandon all patches that were submitted for it 15:10:56 <admin0> i am in the process of upgrading one env from ocata -> latest , so please dont delete files :D 15:11:17 <noonedeadpunk> oh... 15:11:38 <noonedeadpunk> I was going to drop branch and tags related to ocata 15:11:52 <admin0> how can i request more time on this 15:12:05 * admin0 will send pizzas :) 15:12:16 <noonedeadpunk> about what time period are we talking about? 15:12:29 <admin0> can i pm ? 15:12:51 <noonedeadpunk> um, yes, but descision has to be taken in channel :) 15:13:31 <admin0> there is a very important cloud .. but in ocata that needs to be upgraded .. i am setting up a lab to replicate that this week .. within 15-20 days i will be able to tell .. 15:14:26 <noonedeadpunk> I can imagine how challenging it would be to spawn ocata deployment in 2021 15:14:39 <admin0> it is 15:15:00 <admin0> if its deleted, if there can be way how I can clone an earlier branch/tag or preserve it , that would be great 15:15:27 <admin0> is asking till end of the year a big deal ? 15:15:27 <noonedeadpunk> I think you will still be able to checkout to sha from master.... 15:15:35 <jrosser> the tags arent deleted are they? 15:15:55 <admin0> can i bribe you with a pizza to keep it till end of year :) 15:16:14 <admin0> and beer 15:16:15 <spatel> curious what is the rush to delete old tags? 15:16:20 <noonedeadpunk> jrosser: but tags are on branch? Actually along with sha's... 15:16:50 <spatel> why don't we keep them if they don't cost $$ 15:17:02 <admin0> i have one critial infrastructure handling some government infrastrucutre to be upgraded from ocata -> latest till end of year 15:17:19 <admin0> so if it can be kept till end of year, i would be very grateful 15:17:32 <admin0> "an undisclosed" government will be very grateful 15:17:41 <jrosser> i was trying to follow the discussion in other channels about this 15:18:29 <noonedeadpunk> admin0: I think that would be super challenging also because really other projects are dropping their branches super actively nowadays 15:18:50 <noonedeadpunk> I wasn't following discussions, but was following ML 15:19:24 <jrosser> when l look in the nova repo (on github) there are still tags for <branch>-em on really ancient branches 15:19:34 <jrosser> but the branches themselves are removed 15:19:41 <jrosser> afaik it's just a pointer to a sha 15:20:10 <noonedeadpunk> yeah, it's a pointer to sha.... but sha in specific tree? 15:20:34 <noonedeadpunk> I mean if you merge old branch with master, then SHA will be valid 15:20:56 <jrosser> though the point about the other repos is totally valid for admin0 case 15:21:12 <jrosser> it's just generally happening that the old branches are being removed 15:21:46 <admin0> if its have to be dropped, i cannot stop it .. but i would need some help to figure out what i can do now to clone or do stuff so that i will be able to deploy ocata 15:22:10 <noonedeadpunk> actually now I wonder if I did right by checkout to stable/train for EM... I think I should have checkout to train-em instead 15:22:28 <noonedeadpunk> admin0: well, there's no rush for us specifically 15:22:47 <jrosser> but look i think this isnt necessarly a problem 15:22:48 <jrosser> https://github.com/openstack/nova/tree/kilo-eol 15:22:59 <jrosser> the code is there, the branch is deleted 15:23:10 <jrosser> but the tags remain as references 15:23:16 <noonedeadpunk> then I've missed about hob git works... 15:24:31 <noonedeadpunk> well, ok, then it solves everybody problems I think 15:24:39 <jrosser> yes, i think it's fine 15:24:59 <noonedeadpunk> admin0: you will be able to checkout to ocata-em instead of stable/ocata 15:25:08 <admin0> that is fine 15:25:28 <admin0> can i checkout tags also ? or are tags also gone 15:25:30 <admin0> i usually checkout tags 15:25:32 <admin0> like x.y.z 15:25:49 <admin0> i don't checkout the branches like stable/X 15:25:52 <noonedeadpunk> now I don't know:) if em is working then tags should be also fine 15:26:14 <noonedeadpunk> I think I should actually do more research then 15:26:30 <jrosser> theres em and eol 15:26:41 <jrosser> some projects already em and eol stein 15:26:42 <noonedeadpunk> ah! 15:26:52 <noonedeadpunk> ok, now I got how this works:) 15:27:04 <admin0> as long as tags remain, i am not too worried about branch-names 15:27:13 <noonedeadpunk> I mean new tag is created on top of the branch that is dropped 15:27:21 <jrosser> yeah 15:28:01 <admin0> so tags will work ? 15:28:03 <noonedeadpunk> nova has all tags in place 15:28:43 <noonedeadpunk> well, anyway we can probably postpone dropping ocata as well? 15:29:32 <noonedeadpunk> I think nothing would happen if we drop branch somewhere autumn with X release? Especially it's so important 15:29:56 <jrosser> this sounds fine 15:30:15 <admin0> thank you 15:31:13 <noonedeadpunk> ok, cool. 15:32:11 <noonedeadpunk> next thing I want to discuss is how we see root CA generation. I mean - should we disable SSL or don't enable CA when variables not set, or jsut create dummy default CA? 15:32:50 <noonedeadpunk> As I think we should then say about these variables as like required ones? 15:33:17 <noonedeadpunk> and underline how important is to set them both during upgrade to W and for new setup? 15:34:18 <noonedeadpunk> fwiw W deadline is July 2 15:34:38 <jrosser> i guess that always creating the internal self-signed CA is good 15:34:52 <jrosser> even when there is a proper certificate for the public endpoint 15:35:29 <jrosser> it would still be needed for rabbitmq regardless, unless the deployer also provided their own certificates for each rabbit node 15:35:46 <noonedeadpunk> let's probably then include reference to the sample with these variables _everywhere_ - in deploy guide, in aio guide, in release notes... 15:36:16 <jrosser> the problem we have is that there is so many possible ways that this could be done 15:36:26 <jrosser> and i think we have to choose a "sensible default", as usual 15:37:33 <jrosser> if we want to move on and do further work for ssl on galera and internal endpoint then the certs become a key component 15:37:55 <noonedeadpunk> yeah. but the main concern here is that rotate CA is not so easy I guess. And in case it's missed during upgrade, you will get your production deployment with dummy info in CA 15:38:18 <jrosser> i think that rotating the intermediate is actually quite easy 15:38:44 <jrosser> the root CA itself is another matter 15:38:46 <noonedeadpunk> and info is not in root but in intermediate? 15:39:16 <noonedeadpunk> I just thought that all these org stuff and country and etc are in root? 15:39:24 <jrosser> root signs intermediate, intermediate signs server cert 15:39:59 <jrosser> so if there is a requirement to replace the trust chain for whatever reason, then thats easy up to the point of wanting to rotate the actual root CA certificate 15:40:43 <noonedeadpunk> Ah, ok, openstack_pki_authorities contain both root and intermediate 15:40:53 <jrosser> yes, as many of either as you want 15:40:55 <jrosser> it's a list 15:41:14 <noonedeadpunk> and I think intermediate can be even skipped for $reason ? 15:41:17 <jrosser> so i had in mind that we could have intermediates for services, SSH certs, <whatever> 15:41:34 <noonedeadpunk> yeah, that make sense 15:41:45 <jrosser> then you can split the risk/cost of replacing an intermediate easily without having to change everything 15:42:09 <noonedeadpunk> well, that's in case we have intermediate per service? 15:42:23 <jrosser> and as it stands now, you could extend the list of intermediates trivially 15:42:46 <jrosser> and have new certs made off the new intermediate, and their trust chain will validate against the original root CA 15:43:10 <jrosser> so keeping the root CA super-safe is perhaps key 15:43:44 <noonedeadpunk> sounds like it is 15:43:49 <jrosser> and thats something i've not tested here, supplying the root CA, plus intermediate and it's key out-of-band to the PKI role 15:43:58 <jrosser> as that would be the totally best way 15:45:16 <jrosser> i think thats it should be a case of copying the files to the right dirs in /etc/openstack_deploy/pki/roots/ and then not defining the variable that creates CA in the PKI role 15:46:57 <jrosser> tbh this is why i kind of paused at the documentation part 15:47:07 <noonedeadpunk> ok, thinking about reasonable defaults, probably you was right about placing https://review.opendev.org/c/openstack/openstack-ansible/+/788031/7/tests/roles/bootstrap-host/files/user_variables_pki.yml to group_vars instead 15:47:09 <jrosser> there is something self-contained we need for CI 15:47:27 <jrosser> and then there is whatever points we need for overriding these variables for production deployments 15:47:49 <jrosser> that might want a self-contained CA, or derived from a company CA, or whatever else 15:49:07 <noonedeadpunk> right, agred 15:49:09 <noonedeadpunk> *agreed 15:49:15 <jrosser> yes, so setting openstack_pki_authorities: [] in user_variables would stop any new CA being made 15:50:39 <jrosser> this potential complexity is a reason to keep this simple just for rabbit/haproxy for W and try to shake out some of the production use-cases during X 15:52:28 <noonedeadpunk> yeah, the point with docs, is to make ppl aware that they will get `Example Corporation` CA by default, and they really need to change that 15:52:54 <noonedeadpunk> as for upgrades I bet this will be confusing 15:53:02 <noonedeadpunk> also 15:53:16 <noonedeadpunk> should we enable ssl for all endpoints by default? 15:53:38 <jrosser> perhaps during X cycle? 15:53:52 <noonedeadpunk> yeah, might be. 15:54:16 <jrosser> i'm not sure how that should be done at haproxy 15:54:17 <noonedeadpunk> As I guess we need to write upgrade path for that 15:54:34 <jrosser> i mean it can be done today of course with setting all endpoints to SSL 15:54:50 <jrosser> be if the same certificate is the right thing to do on the inside and outside 15:55:05 <jrosser> *but if 15:55:24 <noonedeadpunk> which is only the case with wildcards Iguess... 15:55:40 <noonedeadpunk> and with lets encrypt that would be tough 15:55:43 <jrosser> if we want to eventually also do mutual-tls on the internal endpoint then we must use the internal CA i think 15:56:01 <jrosser> and seperatly have LE or whatever on the external 15:56:40 <noonedeadpunk> yeah, let's leave it for now, agree 15:57:13 <noonedeadpunk> I just thought that we can just use internal ca for internal/admin endpoint by default 15:57:27 <jrosser> yes, i think that would be a good thing 15:57:29 <noonedeadpunk> and do mutual-tls later on 15:57:44 <jrosser> we can put an IP: SAN in there and it should just work with what we have already 16:00:54 <noonedeadpunk> #endmeeting