| *** ysandeep|out is now known as ysandeep|rover | 01:21 | |
| *** ysandeep|rover is now known as ysandeep|rover|bbl | 02:08 | |
| *** ysandeep|rover|bbl is now known as ysandeep|rover | 04:23 | |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 07:29 | |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 07:40 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Fix distro support option for Octavia https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/839028 | 07:49 |
|---|---|---|
| *** ysandeep|rover is now known as ysandeep|rover|lunch | 08:01 | |
| jrosser | good morning | 08:21 |
| damiandabrowski[m] | hi! | 08:28 |
| *** ysandeep|rover|lunch is now known as ysandeep|rover | 09:02 | |
| jrosser | i do not really understand the releasenotes build process here https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/773984/ | 09:38 |
| jrosser | probably the releasenotes job only runs when we add a new reno, and something there is broken | 09:39 |
| damiandabrowski[m] | at least i can reproduce `releasenotes/source/unreleased.rst:13:Inline literal start-string without end-string.` error locally | 09:59 |
| damiandabrowski[m] | but it's confusing as this file has only 5 lines | 09:59 |
| damiandabrowski[m] | ahhh found it | 10:03 |
| damiandabrowski[m] | added my review | 10:04 |
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-python_venv_build master: Split venv_rebuild functionality https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/773984 | 10:07 |
| jrosser | damiandabrowski[m]: aaaaahhh thanks! It would have been helpful if the error said it was an error in a releasnote file :/ | 10:08 |
| damiandabrowski[m] | yeah it wasn't so clear. btw. how does it work, who can edit Your change? everyone or only a core reviewer? :D | 10:10 |
| jrosser | i think anyone logged into gerrit can update a patch | 10:11 |
| damiandabrowski[m] | ahh, thanks! | 10:12 |
| *** dviroel|out is now known as dviroel | 11:31 | |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 11:32 | |
| mgariepy | yep anyone can update any patch. i personally don't really care if anyone update my patches. some ppl might tho. | 11:36 |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 11:37 | |
| mgariepy | you had good vacation damiandabrowski[m] ? | 11:40 |
| SiavashSardari | hey everyone | 11:44 |
| SiavashSardari | we are using proxy variables in user_variables.yml to use http proxy in our deployment. the other day I had some issues on some of our nodes accessing each other. | 11:44 |
| SiavashSardari | The root cause was interesting, we are having 24 containers on each controller and /etc/environment actually are using PAM and according to https://github.com/linux-pam/linux-pam/blob/master/modules/pam_env/pam_env.c#L47 setting env vars in PAM has a limit of 1024 characters. | 11:44 |
| SiavashSardari | while i was on the issue I figured out setting /etc/environment on lxc_container_create, might be redundant because lxc containers are already starting using env vars set by PAM. | 11:45 |
| jrosser | i know allllllll about this :) | 11:45 |
| damiandabrowski[m] | mgariepy: thanks, I visited NYC. It was awesome but because i was walking ~15km every day, now I'm more tired than before my vacation :D | 11:45 |
| mgariepy | LOL :) | 11:45 |
| SiavashSardari | may be we should update https://opendev.org/openstack/openstack-ansible/src/branch/master/etc/openstack_deploy/user_variables.yml#L143 | 11:46 |
| jrosser | SiavashSardari: well it is complicated unfortunately | 11:46 |
| jrosser | https://docs.openstack.org/openstack-ansible/latest/user/limited-connectivity/index.html#practice-b-proxy-access-to-internet-resources | 11:46 |
| SiavashSardari | all_containers is too large | 11:46 |
| jrosser | for any proper deployment i would not recommend making global proxy configurations on the nodes *at all* | 11:47 |
| SiavashSardari | @jrosser, Thanks, I've read the docs. but unfortunately setting up the proxy confs is crucial to us. | 11:48 |
| jrosser | well i know, my deployments are also behind http proxies | 11:48 |
| jrosser | and i wrote those docs after having to work around the exact thing you descrive | 11:49 |
| jrosser | the best approach for larger deployments is to use the deployment_environment_variables stuff to set the proxies only during the playbook runs | 11:49 |
| jrosser | and to make specific config outside the scope of openstack-ansible for apt.conf proxies and whatever else you need | 11:50 |
| jrosser | also, you will find that the support for no_proxy is pretty patchy, not all things take notice of it | 11:50 |
| SiavashSardari | I know but /etc/env is soooo much easier :D | 11:50 |
| jrosser | and so a whole bunch of unexpected traffic is u-bend around your http proxy without you realising | 11:51 |
| jrosser | i can't really put it another way other than to say that the global proxy config is really easy, but ultimately breaks badly for medium/large deployments | 11:52 |
| SiavashSardari | but many processes won't use http_proxy var | 11:52 |
| jrosser | you don't need it at runtime, thats the thing | 11:52 |
| jrosser | all the openstack config files are given the urls of the services directly, or via the loadbalancer | 11:53 |
| SiavashSardari | yep you're right about the run time but we still need proxy to upload db remote backup, monitoring alerts, etc. | 11:54 |
| SiavashSardari | it seems there is no easy way to have it all | 11:55 |
| *** ysandeep|rover is now known as ysandeep|rover|mtg | 11:55 | |
| SiavashSardari | I'll try your way then. Thanks @jrosser | 11:55 |
| mgariepy | damiandabrowski[m], don't forget the workflow +1 :) | 11:56 |
| jrosser | SiavashSardari: imho the best thing to do is to make specific config for the things that really need it, like APT or pip or whatever else where you can adjust a config file | 11:56 |
| SiavashSardari | btw is there any effort for replacing ceph-ansible with adm in OSA? we're going to start the migration planing for our deployments, this month. | 11:59 |
| damiandabrowski[m] | mgariepy: ahhh yeah :| thanks | 12:00 |
| jrosser | the proxy stuff was a big bunch of work to get everything right so I also made a CI job for it and it uses this config https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2#L292-L304 | 12:01 |
| jrosser | that is a very severe test as it removes eth0 from the containers so there is no possibility of non-proxied traffic | 12:02 |
| jrosser | SiavashSardari: regarding ceph-adm I don't know really other than we mentioned noticing that ceph-ansible is being deprecated | 12:03 |
| jrosser | one of the things we discussed was the need to clarify the openstack-ansible documentation about the intent of ceph deployed with OSA | 12:05 |
| jrosser | like is it just a test fixture we use for CI, or is it something that we "support" that end users should have confidence in using | 12:05 |
| SiavashSardari | @jrosser, tnx for the proxy stuff, I will try your way. but removing eth0 sounds a little bit extreme to me | 12:12 |
| jrosser | oh sure :) that is purely for the test fixture - but it does validate that the deployment can be functional in this type of setup with no global proxy | 12:14 |
| opendevreview | Christian Rohmann proposed openstack/openstack-ansible-galera_server master: Move slow_query_log_file to location handled by logrotation https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/841415 | 12:14 |
| jrosser | it was necessary to do that in the CI jobs because the test nodes are not actually behind a proxy - the squid process runs on the same node and there is also a real external interface | 12:14 |
| jrosser | so it can work "by accident" if eth0 is still present in that situation | 12:15 |
| SiavashSardari | on the ceph part, yeah I remember the weekly was couple of weeks ago. I'm with @damiandabrowski[m] on this one, which mentioned having ceph integrated with osa is an advantage osa has | 12:17 |
| SiavashSardari | there are a lot of complexity to what to do with docker networks and lxc bridges in a shared controller and mon deployments. | 12:19 |
| jrosser | also someone has to be prepared to invest the time to figure it all out, too | 12:25 |
| jrosser | i just have no idea how cephadm would work out with existing things in openstack-ansible, like the LXC stuff, support for proxies, support for ssh bastions for the deploy host..... | 12:26 |
| SiavashSardari | I think we will get to those parts maybe next month. I will keep you posted | 12:29 |
| jrosser | are you going to deploy OSA and cephadm stuff onto the same nodes? | 12:30 |
| jrosser | personally i also don't like having one of the ceph nodes effectively become a deployment node | 12:31 |
| jrosser | *another | 12:31 |
| *** ysandeep|rover|mtg is now known as ysandeep|rover | 12:37 | |
| SiavashSardari | @jrosser, currently we've deployed ceph using osa which means ceph-mons are lxc containers on controllers. our goal is to replace ceph-ansible with adm. | 12:47 |
| SiavashSardari | we are going to try and error some scenarios in dev env before deciding our final solution and deploy on other deployments. | 12:48 |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 13:21 | |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 13:31 | |
| opendevreview | Merged openstack/openstack-ansible-os_octavia master: Fix distro support option for Octavia https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/839028 | 13:35 |
| opendevreview | Merged openstack/ansible-role-pki master: Fix variable names for certificate CSR fields https://review.opendev.org/c/openstack/ansible-role-pki/+/830793 | 13:41 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq / erlang versions https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/840284 | 13:49 |
| opendevreview | Merged openstack/openstack-ansible-os_octavia master: Make octavia_provider_network better configurable https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/787336 | 13:52 |
| jrosser | damiandabrowski[m]: did you plan to look at the glusterfs stuff? | 13:53 |
| jrosser | it's possible to run https://review.opendev.org/c/openstack/openstack-ansible/+/837589 in an AIO with SCENARIO=aio_infra and you should get a 3-node gluster cluster running to have a play with it | 13:54 |
| jrosser | aio_lxc_infra | 13:54 |
| damiandabrowski[m] | thanks, I'll give it a try this week | 13:55 |
| jrosser | it would be worth trying out repo container delete/recreate playbooks | 13:59 |
| damiandabrowski[m] | okok | 14:00 |
| jrosser | and also we probably need to document what to do when a physical node is replaced/re-installed | 14:00 |
| jrosser | some of that is talked about here https://docs.rackspace.com/support/how-to/recover-from-a-failed-server-in-a-glusterfs-array/ | 14:01 |
| damiandabrowski[m] | You're right, we may put some info here: https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks.html | 14:12 |
| mgariepy | that's a lot of steps haha | 14:14 |
| jrosser | it's not as bad as it looks | 14:14 |
| mgariepy | and i guess it will need to be done when doing OS upgrade and reinstalling the server. | 14:14 |
| jrosser | my playbook does some of that | 14:14 |
| jrosser | and there is a bind mount in /openstack/glusterfs, if you can restore that on a re-installed host and re-run the repo-server playbook it should be automatic | 14:15 |
| mgariepy | ok | 14:15 |
| jrosser | this is able to recover from a container destroy/create https://github.com/openstack/openstack-ansible-plugins/blob/master/roles/glusterfs/tasks/main.yml#L102-L121 | 14:17 |
| jrosser | but it relies on the contents of /openstack/gluster for the needed UUID | 14:17 |
| jrosser | would be really great if some other eyes can get on this | 14:18 |
| *** spotz_ is now known as spotz | 14:18 | |
| mgariepy | i'll try it. spawning the instance then i need to head up to a meeting :D | 14:20 |
| opendevreview | Merged openstack/openstack-ansible master: Add CSP headers for img-src and worker-src https://review.opendev.org/c/openstack/openstack-ansible/+/841154 | 14:22 |
| opendevreview | Merged openstack/openstack-ansible master: Bump ansible-core version to 2.12.5 https://review.opendev.org/c/openstack/openstack-ansible/+/839312 | 14:41 |
| opendevreview | Merged openstack/openstack-ansible master: Add documentation reminder to set external VIP to FQDN for real deployments https://review.opendev.org/c/openstack/openstack-ansible/+/840360 | 14:41 |
| opendevreview | Merged openstack/openstack-ansible master: Install rally only when rally_install is set https://review.opendev.org/c/openstack/openstack-ansible/+/840686 | 14:41 |
| *** dviroel is now known as dviroel|lunch | 15:09 | |
| *** ysandeep|rover is now known as ysandeep|rover|out | 15:26 | |
| damiandabrowski[m] | https://review.opendev.org/c/openstack/ansible-hardening/+/835733 that's weird | 15:30 |
| damiandabrowski[m] | 'Change 840353 in project openstack/openstack-ansible-openstack_hosts does not share a change queue with 835733 in project openstack/ansible-hardening' | 15:31 |
| damiandabrowski[m] | how have we finally implemented integrated queue? https://review.opendev.org/c/openstack/openstack-ansible/+/836657 | 15:31 |
| jrosser | we did not do this yet | 15:38 |
| *** dviroel|lunch is now known as dviroel | 16:03 | |
| damiandabrowski[m] | ahh okok. I've noticed that the change it depends on is not merged yet | 16:47 |
| damiandabrowski[m] | so I'd understand an 'This change depends on a change that failed to merge.' error, but I can't explain this one | 16:47 |
| admin1 | i have cert key and ca_cert .. what is the pem and do i have to provide all of them ? is pem a cat of cert and ca_cert ? | 18:21 |
| *** dviroel is now known as dviroel|afk | 20:53 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!