| *** nurdie has joined #openstack-ansible | 00:08 | |
| *** noonedeadpunk has quit IRC | 00:08 | |
| *** noonedeadpunk has joined #openstack-ansible | 00:10 | |
| *** nurdie has quit IRC | 00:12 | |
| *** tosky has quit IRC | 00:18 | |
| *** gyee has quit IRC | 00:36 | |
| *** nurdie has joined #openstack-ansible | 00:40 | |
| *** rfolco has joined #openstack-ansible | 00:43 | |
| *** waverider has quit IRC | 00:45 | |
| *** nurdie has quit IRC | 00:45 | |
| *** rfolco has quit IRC | 00:47 | |
| *** gregwork has quit IRC | 00:48 | |
| *** cshen has joined #openstack-ansible | 01:15 | |
| *** cshen has quit IRC | 01:19 | |
| ThiagoCMC | admin0, finished a new deployment, same "Something went wrong!" at Horizon's login! :-O | 01:59 |
|---|---|---|
| ThiagoCMC | Found the error: "neutronclient.common.exceptions.ServiceUnavailable: <html><body><h1>503 Service Unavailable</h1>" :-P | 02:05 |
| ThiagoCMC | My openstack_user_config produced a broken neutron config, error: "neutron_lib.exceptions.NetworkVlanRangeError: Invalid network VLAN range: 'external:5' - 'Need exactly two values for VLAN range'." - Interesting... I thought that it would d a check up somehow lol | 02:12 |
| ThiagoCMC | My idea was to be able to create 2 neutron external networks, then, I tried: `range: "5,196"`, under provider_networks... Not, I changed it to `range: "5:5,196:196"` ... Errors gone! Feeling stupid because I complained here but I broke it. :-P | 02:25 |
| *** nurdie has joined #openstack-ansible | 02:42 | |
| *** spatel has joined #openstack-ansible | 02:44 | |
| *** nurdie has quit IRC | 02:46 | |
| *** cshen has joined #openstack-ansible | 03:15 | |
| *** cshen has quit IRC | 03:19 | |
| *** lemko has quit IRC | 04:25 | |
| *** lemko has joined #openstack-ansible | 04:25 | |
| *** nurdie has joined #openstack-ansible | 04:43 | |
| *** nurdie has quit IRC | 04:48 | |
| *** cshen has joined #openstack-ansible | 05:15 | |
| *** cshen has quit IRC | 05:20 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-ansible | 05:33 | |
| openstackgerrit | zhangboye proposed openstack/openstack-ansible master: Replace deprecated UPPER_CONSTRAINTS_FILE variable https://review.opendev.org/c/openstack/openstack-ansible/+/765239 | 05:48 |
| *** pto has joined #openstack-ansible | 06:03 | |
| stduolc | I try to run OSA on three node, which is ubuntu 18.04. I get the error like this: http://paste.openstack.org/show/800679/ , what can i do? | 06:19 |
| *** nurdie has joined #openstack-ansible | 06:26 | |
| *** nurdie has quit IRC | 06:32 | |
| *** spatel has quit IRC | 06:33 | |
| *** cshen has joined #openstack-ansible | 06:39 | |
| *** cshen has quit IRC | 06:44 | |
| *** d34dh0r53 has quit IRC | 07:09 | |
| *** d34dh0r53 has joined #openstack-ansible | 07:09 | |
| *** cshen has joined #openstack-ansible | 07:24 | |
| *** rpittau|afk is now known as rpittau | 07:27 | |
| *** cshen has quit IRC | 07:29 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not warn about incorrect group naming https://review.opendev.org/c/openstack/openstack-ansible/+/760952 | 07:39 |
| *** miloa has joined #openstack-ansible | 07:45 | |
| *** shyamb has joined #openstack-ansible | 08:06 | |
| *** cshen has joined #openstack-ansible | 08:08 | |
| *** luksky has joined #openstack-ansible | 08:22 | |
| *** jbadiapa has joined #openstack-ansible | 08:23 | |
| *** nurdie has joined #openstack-ansible | 08:28 | |
| *** nurdie has quit IRC | 08:33 | |
| *** tosky has joined #openstack-ansible | 08:34 | |
| *** redrobot has quit IRC | 08:41 | |
| jrosser | morning | 08:42 |
| jrosser | stduolc: it looks like something may have gone wrong with creating the container base image | 08:47 |
| jrosser | have you had any errors before that point and had to re-run the playbooks? | 08:47 |
| *** andrewbonney has joined #openstack-ansible | 08:49 | |
| *** shyamb has quit IRC | 08:58 | |
| jrosser | noonedeadpunk: i put some replies on here https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/764449, we can get gshippey to double check this on a real deployment? | 09:13 |
| jrosser | theres also a couple of patches to look at here for the tripleo folk on os_tempest https://review.opendev.org/q/project:openstack%252Fopenstack-ansible-os_tempest+status:open | 09:15 |
| *** shyamb has joined #openstack-ansible | 09:19 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 09:22 | |
| noonedeadpunk | oh, yes, you're right about RuntimeDirectory. but galera_mariadb_backups_user is commented out while I think it should not? and probably we should be creating this user as well then? | 09:31 |
| noonedeadpunk | we can merge this actually and do this as follow up patch | 09:32 |
| *** pto_ has joined #openstack-ansible | 09:32 | |
| *** shyamb has quit IRC | 09:32 | |
| jrosser | yes you are right about the defaults there - didnt spot that | 09:33 |
| jrosser | oh wait hold on, it's a comment about the variable below? | 09:34 |
| jrosser | perhaps it should be on the end of the next line down | 09:34 |
| noonedeadpunk | I'm blind.... | 09:34 |
| noonedeadpunk | uh :( soorrryyy | 09:35 |
| jrosser | i asked gshippey to add a comment because the existing variable names are not totally clear about db user vs. unix user | 09:35 |
| *** pto has quit IRC | 09:35 | |
| noonedeadpunk | shame on me :( | 09:35 |
| jrosser | its ok :) | 09:35 |
| noonedeadpunk | I was sure I did search defaults file and find no variables, but probably it was just a dream | 09:36 |
| noonedeadpunk | would be awesome to merge https://review.opendev.org/q/topic:%22run_condition%22+(status:open) https://review.opendev.org/q/topic:%22osa%252Foctavia_notifications%22+(status:open) and https://review.opendev.org/q/topic:%22api_threads%22+(status:open) | 09:39 |
| jrosser | we have the mariadb backups working in a real environment now btw | 09:39 |
| noonedeadpunk | I tested them in sandbox and things were fine for me as well | 09:40 |
| jrosser | the existing bind mount from the galera container makes the backups accessible by a user on the host which is neat | 09:40 |
| noonedeadpunk | haven't tested at scale this implementation, but used to be running mariabackup manually on high loaded db and it was great | 09:40 |
| jrosser | thats why we had to add the var to set the GID of the backups so that the db dump was only readable by the right user | 09:40 |
| noonedeadpunk | regarding bind mount - I think https://review.opendev.org/c/openstack/openstack-ansible/+/752059 is pretty useful.... | 09:41 |
| jrosser | yes, thats on our backlog of things to test becasue i think we have the same issue of overloaded mounts | 09:41 |
| jrosser | and missing journals being collected | 09:42 |
| jrosser | the patch looks fine but just not had chance to try it out yet | 09:42 |
| noonedeadpunk | yeah... as journals are available only inside LXC that way | 09:42 |
| jrosser | yes, and it's not cool putting journalbeat or whaever agent into each container | 09:42 |
| jrosser | would rather just do it once on the host | 09:43 |
| noonedeadpunk | and btw, I think your python collection script will be handy since we will have all lxc journals on host - it won't need lxc module which was the issue | 09:43 |
| jrosser | yes thats a good idea | 09:43 |
| jrosser | it did provide an easy way to enumerate the containers at access the journals | 09:44 |
| jrosser | but it sure this can be refactored | 09:44 |
| jrosser | we'd have to get the container names somehow, as thats currently done with the lxc python library | 09:45 |
| noonedeadpunk | eventually I already can't recall what exactly I was doing with that patch, but idea was to get a regexp where more long path would be inserted before smaller one if exists. but I feel it has some corner cases though... | 09:45 |
| noonedeadpunk | well, we can always do `lxc-ls` with popen | 09:46 |
| noonedeadpunk | since module is pretty problematic to get since it's not provided with pip iirc? | 09:46 |
| jrosser | yes it is very old (2012) on pypi.org | 09:48 |
| noonedeadpunk | btw looking at this patch again, while it fix issue with logs, I think we'd rather should define bind mounts at one place only, to be ably to properly sort list of required ones... | 09:49 |
| *** luksky has quit IRC | 09:51 | |
| jrosser | is this currently defined in many places? | 09:52 |
| noonedeadpunk | yeah, and that's why it raised at the first place | 09:56 |
| noonedeadpunk | so in each playbook as pre step we do bind mounts which uses this https://opendev.org/openstack/openstack-ansible/src/branch/master/playbooks/common-tasks/os-lxc-container-setup.yml#L49-L60 | 09:57 |
| noonedeadpunk | but also we have a default set https://opendev.org/openstack/openstack-ansible-lxc_container_create/src/branch/master/tasks/lxc_container_config.yml#L44-L53 | 09:58 |
| jrosser | this is probably creation-time vs. add-later | 10:06 |
| stduolc | jrosser: I run it again, and get the same error. Which version of OSA should I use? | 10:07 |
| jrosser | stduolc: which version do you use currently? | 10:08 |
| *** MickyMan77 has joined #openstack-ansible | 10:17 | |
| stduolc | stable/ussuri aecc608cdd90050922ea9e3bb3cd8ef7d09b68ac | 10:18 |
| stduolc | I change to 21.2.0 and have a try. | 10:18 |
| *** nurdie has joined #openstack-ansible | 10:29 | |
| *** nurdie has quit IRC | 10:34 | |
| MickyMan77 | Hi all, is there anyone that have a working example config to connect openstack with FreeIPA ? | 10:39 |
| *** rfolco has joined #openstack-ansible | 10:59 | |
| *** shyamb has joined #openstack-ansible | 11:08 | |
| *** shyamb has quit IRC | 11:08 | |
| *** shyamb has joined #openstack-ansible | 11:09 | |
| *** shyam89 has joined #openstack-ansible | 11:09 | |
| *** shyam89 has quit IRC | 11:09 | |
| noonedeadpunk | MickyMan77: isn't it just ldap? | 11:11 |
| MickyMan77 | yes | 11:12 |
| noonedeadpunk | pretty sure this example was working one https://docs.openstack.org/openstack-ansible-os_keystone/latest/configure-keystone.html#implementing-ldap-or-active-directory-backends | 11:12 |
| noonedeadpunk | you will need a bit different mappings for groups and users, but it's pretty doable | 11:13 |
| noonedeadpunk | would be awesome if you could share your config in case you get freeipa working so we could add it to the docs | 11:13 |
| MickyMan77 | ok.. will do that.. | 11:14 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone stable/ussuri: Adding tags to federated openid support using auth_mod_openidc https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/765289 | 11:17 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone stable/train: Adding tags to federated openid support using auth_mod_openidc https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/765290 | 11:18 |
| *** shyamb has quit IRC | 11:22 | |
| *** luksky has joined #openstack-ansible | 11:22 | |
| *** spatel has joined #openstack-ansible | 11:34 | |
| *** spatel has quit IRC | 11:39 | |
| *** shyamb has joined #openstack-ansible | 11:45 | |
| *** shyamb has quit IRC | 12:08 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not print btrfs error in log https://review.opendev.org/c/openstack/openstack-ansible/+/765318 | 12:09 |
| jrosser | we still have this patch relating to OIDC https://review.opendev.org/c/openstack/openstack-ansible/+/760431 | 12:13 |
| jrosser | feels kind of hacky really not sure if there is a neater approach | 12:14 |
| noonedeadpunk | I can recall I found something super wird, like having contract in keystone role which is never triggered or smth like that | 12:14 |
| jrosser | yes, theres some ugly stuff here | 12:14 |
| jrosser | possibly a file that should have been removed when the contract was moved out to the playbook | 12:15 |
| noonedeadpunk | you suggest doing cleanup as a separate patch? | 12:17 |
| *** mgariepy has quit IRC | 12:20 | |
| noonedeadpunk | I have super weird feeling about this patch tbh - partially because I do not fully understand complexity we have, and other part why we have `hosts: keystone_all` and later on `run_once: yes` in https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/760429/1/tasks/main_keystone_federation_sp_idp_setup.yml#1 | 12:20 |
| noonedeadpunk | and can't we just set `tasks_from: keystone_federation_sp_idp_setup.yml`? | 12:20 |
| *** nurdie has joined #openstack-ansible | 12:30 | |
| noonedeadpunk | jrosser: eventually if you're ok with current state - we can merge it and iterate later | 12:31 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: Fix magnum_service_user_domain_name https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/765320 | 12:34 |
| *** nurdie has quit IRC | 12:35 | |
| admin0 | quick question -- br-lbaas .. is it supposed to be only in controllers ? or also in network nodes ? | 12:36 |
| admin0 | there are controllers, network nodes, compute nodes and some hosts who do network+compute :) | 12:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Updated from OpenStack Ansible Tests https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/758751 | 12:45 |
| jrosser | admin0: as usual it is up to you :) | 12:55 |
| jrosser | if you do the lbaas mgmt network as a flat network then you need it everywhere | 12:55 |
| jrosser | if you do it as a vlan network then you only need br-lbaas on your controllers for the lxc to connect to | 12:55 |
| jrosser | there is no particular right answer, you need to design something appropriate for your environment | 12:56 |
| jrosser | noonedeadpunk: i agree there is complexity | 12:59 |
| jrosser | and perhaps it can be made clearer, we've got a role: with tasks_from, but the tasks need to be conditional | 13:00 |
| jrosser | so the condition is in main_keystone_federation_sp_idp_setup.yml if it then further includes keystone_federation_sp_idp_setup.yml | 13:01 |
| noonedeadpunk | ut we don't use keystone_federation_sp_idp_setup.yml anywhere except main_keystone_federation_sp_idp_setup ? | 13:01 |
| jrosser | thats right | 13:01 |
| jrosser | are the variables in the conditional available in the playbook? i was maybe thinking they were only defined in the role | 13:02 |
| jrosser | but that might be incorrect | 13:02 |
| noonedeadpunk | ah, ok, yes, might be the case actually... | 13:02 |
| *** shyam89 has joined #openstack-ansible | 13:02 | |
| jrosser | really i just did this patch very quickly as someone was stuck with this | 13:03 |
| jrosser | and maybe it can be much nicer somehow | 13:03 |
| noonedeadpunk | the main concern is what we should do with https://opendev.org/openstack/openstack-ansible-os_keystone/src/branch/master/tasks/keystone_db_sync.yml#L72-L98 ? | 13:04 |
| noonedeadpunk | jsut drop this? | 13:04 |
| jrosser | these are the tasks that perhaps are never run | 13:05 |
| pto_ | Why does the os_manila tasks get invoked, when its not configured? E.g. TASK [os_manila : Copy manila configs]: fatal: [os_infra1]: FAILED! => {"msg": "'null' is undefined"} | 13:06 |
| *** pto_ is now known as pto | 13:06 | |
| noonedeadpunk | I believe you should have manila group in your inventory in order for role to be invoked | 13:07 |
| pto | noonedeadpunk: Interesting, i have manilla groups... wonder why I have them... Anyway, thanks for claryfing | 13:12 |
| pto | noonedeadpunk: Is there an easy way to remove the groups, or do I need to edit the openstac_inventory.json manually? | 13:15 |
| *** mgariepy has joined #openstack-ansible | 13:15 | |
| noonedeadpunk | no, unfortunatelly no easy way to drop groups - only possible to drop hosts from inventory | 13:15 |
| pto | noonedeadpunk: thx! | 13:25 |
| *** mugsie has quit IRC | 13:32 | |
| *** waverider has joined #openstack-ansible | 13:37 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_adjutant master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/764656 | 13:39 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Reduce number of processes on small systems https://review.opendev.org/c/openstack/openstack-ansible-os_sahara/+/764651 | 13:57 |
| *** spatel has joined #openstack-ansible | 13:57 | |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_nova master: Fix templating issue due to conditional https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/765332 | 14:16 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 14:17 | |
| *** cshen has quit IRC | 14:17 | |
| *** shyam89 has quit IRC | 14:23 | |
| mgariepy | jrosser, https://review.opendev.org/c/openstack/nova/+/758761 do you know anyone who can help with this one ? oct 19 > december 3.. | 14:25 |
| jrosser | hmm not really - i think theres maybe only a small group who can approve nova patches on stable branches | 14:28 |
| *** nurdie has joined #openstack-ansible | 14:32 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Add Zun CI requirement to Zuul required projects https://review.opendev.org/c/openstack/openstack-ansible/+/763177 | 14:33 |
| openstackgerrit | Merged openstack/openstack-ansible master: Decrease amphora image RAM for CI https://review.opendev.org/c/openstack/openstack-ansible/+/765134 | 14:33 |
| *** nurdie has quit IRC | 14:36 | |
| mgariepy | jrosser, so the solution is to fork nova and use that? | 14:45 |
| mgariepy | that's annoying... | 14:45 |
| *** nurdie has joined #openstack-ansible | 14:45 | |
| jrosser | if you are needing that patch locally, then the cleanest way is indeed to fork nova and cherry pick it onto the stable branch you want | 14:46 |
| jrosser | kind of exactly like https://github.com/bbc/nova/tree/bbc-ussuri-21.0.1 in fact for precisely the same reason | 14:47 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_nova master: Add etcd3gw module https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/765336 | 14:50 |
| noonedeadpunk | jamesdenton: have you explicitly set etcd as cache backend for nova? | 15:03 |
| noonedeadpunk | as I think we don't have etcd3 package installed anywhere | 15:04 |
| *** pcaruana has quit IRC | 15:04 | |
| *** pcaruana has joined #openstack-ansible | 15:05 | |
| jamesdenton | i did not | 15:06 |
| noonedeadpunk | hm.... | 15:08 |
| noonedeadpunk | I think the only place where etcd might be required https://docs.openstack.org/nova/ussuri/configuration/config.html#cache.backend | 15:09 |
| noonedeadpunk | but we are using memcached | 15:09 |
| noonedeadpunk | so it's super weird... | 15:09 |
| jamesdenton | i haven't checked logs in zuul. i just upgraded this environment from 21.2.0 -> master | 15:10 |
| jamesdenton | full error on instance create: http://paste.openstack.org/show/800707/ | 15:11 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Fix use of deprecated image_cache_manager_interval https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/765338 | 15:12 |
| noonedeadpunk | hm....... | 15:12 |
| noonedeadpunk | and you have `backend = oslo_cache.memcache_pool` in nova.conf? | 15:16 |
| jamesdenton | yes | 15:17 |
| noonedeadpunk | I think that it's actually nothing wrong with our set of packages, as etc3d should be loaded fro caching at all | 15:17 |
| jrosser | this depends how the plugin framework works i guess | 15:18 |
| jamesdenton | same error here: https://zuul.opendev.org/t/openstack/build/0af9364cc9094b5096899ed7693a9f60/log/logs/host/nova-conductor.service.journal-10-50-46.log.txt | 15:18 |
| jrosser | if it loads all the possibilities up front | 15:18 |
| jrosser | or only the specific one you need due to the config | 15:19 |
| jamesdenton | https://zuul.opendev.org/t/openstack/build/0af9364cc9094b5096899ed7693a9f60/log/logs/host/nova-conductor.service.journal-10-50-46.log.txt#2431 | 15:19 |
| jamesdenton | seems cosmetic at this point | 15:19 |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 15:27 | |
| *** cshen has joined #openstack-ansible | 15:31 | |
| *** redrobot has joined #openstack-ansible | 15:35 | |
| spatel | noonedeadpunk: hatop is broken for python3 look like we need to find other source to get it. | 15:38 |
| spatel | https://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/defaults/main.yml#L105 | 15:38 |
| spatel | I just clone this latest source and it does have support of python3 - https://github.com/jhunt/hatop | 15:38 |
| *** waverider has quit IRC | 15:39 | |
| *** waverider has joined #openstack-ansible | 15:39 | |
| *** waverider has quit IRC | 15:40 | |
| *** waverider has joined #openstack-ansible | 15:40 | |
| noonedeadpunk | spatel: so we can just set https://github.com/jhunt/hatop/archive/v0.8.0.zip for url and update sha? | 15:46 |
| spatel | sounds like a good idea | 15:50 |
| *** miloa has quit IRC | 15:50 | |
| spatel | and pretty quick fix | 15:50 |
| spatel | do you want me to cut the patch or you already did :) | 15:50 |
| noonedeadpunk | I haven't and feel free to place it | 15:50 |
| spatel | lets me give it a shot what is sha for? | 15:51 |
| spatel | to validate integrity ? | 15:51 |
| noonedeadpunk | ah, sorry, ment sha256 checksum for haproxy_hatop_download_checksum | 15:52 |
| noonedeadpunk | yep exactly | 15:52 |
| spatel | should i create that hash on my system and update config right? | 15:52 |
| noonedeadpunk | yep | 15:52 |
| spatel | cool | 15:53 |
| noonedeadpunk | a pity that it's not on pypi but it's even easier at current state of things | 15:55 |
| *** macz_ has joined #openstack-ansible | 15:56 | |
| *** fanfi has joined #openstack-ansible | 15:58 | |
| *** waverider has quit IRC | 16:03 | |
| *** waverider has joined #openstack-ansible | 16:10 | |
| *** mgariepy has quit IRC | 16:10 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/ussuri: Adding tags to federated openid support using auth_mod_openidc https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/765289 | 16:13 |
| *** pcaruana has quit IRC | 16:16 | |
| noonedeadpunk | dmsimard: was able to reproduce issue on vm http://paste.openstack.org/show/800711/ | 16:17 |
| noonedeadpunk | however it seems pretty random for me | 16:18 |
| openstackgerrit | Satish Patel proposed openstack/openstack-ansible-haproxy_server master: Python3 supported version of hatop https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/765348 | 16:18 |
| dmsimard | noonedeadpunk: was it following a playbook that had been interrupted (i.e, ctrl+c) or something like that ? | 16:19 |
| noonedeadpunk | well, at first yes. but as you can see from paste I can just re-run same command and it will rase later | 16:19 |
| noonedeadpunk | like 1 out of 5-10 times ara fails | 16:20 |
| noonedeadpunk | can place your key on vm if it helps | 16:21 |
| *** pcaruana has joined #openstack-ansible | 16:21 | |
| dmsimard | noonedeadpunk: I can't look right away but it could be helpful to troubleshoot: https://github.com/dmsimard.keys | 16:21 |
| noonedeadpunk | I think that maybe ara still tries to write down result when new task launched | 16:21 |
| dmsimard | noonedeadpunk: when it's happened to me, re-running a playbook afterwards worked | 16:22 |
| dmsimard | I'm not sure what locks/unlocks the database | 16:22 |
| dmsimard | need to investigate | 16:22 |
| noonedeadpunk | yeah, but like when I just spamming re-run it fails again in some time | 16:22 |
| *** waverider has quit IRC | 16:28 | |
| openstackgerrit | Satish Patel proposed openstack/openstack-ansible-os_octavia master: Removing spare_amphora_pool_size option https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765351 | 16:45 |
| ThiagoCMC | Morning! :-P | 16:47 |
| ThiagoCMC | I'm trying to upload images to Glance using ansible, and I'm seeing errors like this: `"changed": false, "extra_data": null, "msg": "Image creation failed: Unable to establish connection to https://mycloud.domain.com:9292/v2/images/62d2bd86-1838-4ced-ade8-6abee63daded/file: ('Connection aborted.', OSError(\"(32, 'EPIPE')\"))"` | 16:48 |
| ThiagoCMC | I have to repeat, repeat, and repeat again, until it suddenly works! | 16:48 |
| ThiagoCMC | Any idea? | 16:48 |
| noonedeadpunk | sounds pretty common | 16:48 |
| ThiagoCMC | Ouch | 16:49 |
| noonedeadpunk | but can;t actually recal what exactly it was... | 16:49 |
| *** rpittau is now known as rpittau|afk | 16:49 | |
| ThiagoCMC | Do you know if it works better I just use Bash and `openstack image ..` commands, instead of Ansible `os_image` module? | 16:50 |
| noonedeadpunk | iirc it was the same | 16:51 |
| ThiagoCMC | Hmm... ok | 16:52 |
| ThiagoCMC | Thanks! | 16:52 |
| ThiagoCMC | =) | 16:53 |
| noonedeadpunk | and backend for glance is ok? | 16:53 |
| *** mgariepy has joined #openstack-ansible | 16:53 | |
| *** luksky has quit IRC | 16:54 | |
| noonedeadpunk | I can recall somebody was asking here the same question about a month or so ago, but he found the issue | 16:54 |
| noonedeadpunk | but reallyy can't recall what is was | 16:54 |
| ThiagoCMC | Oh no... lol | 16:55 |
| admin0 | ThiagoCMC, i use terraform for the initial setup .. whenver its ready to test | 16:55 |
| admin0 | upload image, create instance, use ssh keys, create volume, mount volume etc | 16:55 |
| ThiagoCMC | noonedeadpunk, I'll run the upload in a `while true` :-D | 16:55 |
| noonedeadpunk | I don't think it matters | 16:55 |
| *** dave-mccowan has quit IRC | 16:55 | |
| ThiagoCMC | admin0, that's neat! Learning Terraform is in my TODO list! | 16:55 |
| ThiagoCMC | I havd it in my desktop, to create libvirt VMs for fun | 16:56 |
| ThiagoCMC | *had | 16:56 |
| admin0 | let me upload the script and a short howto and you will be able to run that to certify/quick-test every new aio or build | 16:56 |
| ThiagoCMC | Awesome! | 16:56 |
| ThiagoCMC | BTW, I'm thinking about contributing with your openstackfaq website! ^_^ | 16:57 |
| ThiagoCMC | I'm becoming a ninja on this! LOL | 16:57 |
| admin0 | it will be great | 16:58 |
| noonedeadpunk | ThiagoCMC: oh, well, in that case it was smth about auth in glance logs | 17:00 |
| noonedeadpunk | http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2020-11-09.log.html#t2020-11-09T11:51:36 | 17:00 |
| ThiagoCMC | Nice! BTW, sometimes, the error change to: `{"changed": false, "extra_data": null, "msg": "Image creation failed: Image checksum verification failed"}` O_o | 17:06 |
| noonedeadpunk | I think it's smth going on weird with the storage tbh... | 17:06 |
| ThiagoCMC | True... Well, Ceph is HEALTH_OK | 17:07 |
| noonedeadpunk | oh, ok, it's ceph | 17:07 |
| ThiagoCMC | Yep... The only difference from the defaults in my setup, is that the pool `images` is compressed. But, I did this before and it works. | 17:08 |
| noonedeadpunk | and what if you create another sample pool but regular one, and point glance to it just to test if smth will change? | 17:08 |
| ThiagoCMC | Sure, I can do that | 17:09 |
| ThiagoCMC | I can also try to disable the compression =P | 17:09 |
| ThiagoCMC | Same problem without compression... :-( | 17:15 |
| *** tosky has quit IRC | 17:29 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Use synchronize module for rings distribution https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/765354 | 17:32 |
| *** klamath_atx has joined #openstack-ansible | 17:42 | |
| *** pcaruana has quit IRC | 17:45 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Python3 supported version of hatop https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/765348 | 17:48 |
| spatel | noonedeadpunk: thank you to fix hatop pathc :) | 18:01 |
| openstackgerrit | Satish Patel proposed openstack/openstack-ansible-os_octavia master: Removing spare_amphora_pool_size option https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765351 | 18:06 |
| ThiagoCMC | noonedeadpunk, the `while true` "solved" it! LOL images fails then works... when all gone, control+c. LOL | 18:07 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Fix linter errors https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/755576 | 18:14 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Drop octavia v1 api options https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765138 | 18:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Cleanup octavia configuration https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765148 | 18:15 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Replace deprecated UPPER_CONSTRAINTS_FILE variable https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/762061 | 18:19 |
| spatel | folks, adding region: us-west-1 in user_variables.yml will change region for everyone right? | 18:20 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 18:22 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/606143 | 18:23 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/606143 | 18:27 |
| *** andrewbonney has quit IRC | 18:29 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/606143 | 18:29 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/606143 | 18:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Updated from OpenStack Ansible Tests https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/754177 | 18:47 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/754411 | 18:48 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/606143 | 18:50 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/606143 | 18:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Replace deprecated UPPER_CONSTRAINTS_FILE variable https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/762061 | 18:53 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Ensure that we create Octavia resources only once https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765375 | 18:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/754428 | 19:25 |
| *** kleini has quit IRC | 19:35 | |
| *** luksky has joined #openstack-ansible | 19:42 | |
| spatel | noonedeadpunk: any update about victoria release? | 19:47 |
| *** cshen has quit IRC | 19:53 | |
| *** cshen has joined #openstack-ansible | 20:00 | |
| ThiagoCMC | spatel, riiight?! LOL | 20:23 |
| ThiagoCMC | :-P | 20:23 |
| spatel | ThiagoCMC: just waiting for it so hope next week we will see some movement. | 20:25 |
| ThiagoCMC | I can't wait to update my cloud, and also to have Senlin! | 20:28 |
| *** sshnaidm|afk has quit IRC | 20:30 | |
| spatel | +1 | 20:31 |
| spatel | you can use master branch if just playing in lab | 20:31 |
| ThiagoCMC | It's both my lab and my production env hehehe | 20:32 |
| spatel | How big your prod env? | 20:32 |
| ThiagoCMC | 5 controllers, 6 Ceph OSD and 3 Compute Nodes (powerful AMD 3950X with NVMe) | 20:33 |
| spatel | 5 contollers and only 3 compute.. hmm? | 20:33 |
| spatel | I am running my cloud in 3 controller and 328 computes... haha | 20:34 |
| ThiagoCMC | Yeah, my controlers are kind old... The Compute are new machines, I'll buy more maybe in the end of 2021 | 20:34 |
| spatel | curious why do you need 5? | 20:34 |
| ThiagoCMC | Nice! | 20:34 |
| ThiagoCMC | I do 5 because 3 is too low for quorum (if I lost 1, I'll stay with 2, which is bad) | 20:35 |
| ThiagoCMC | So, 5 give me some space to move things around | 20:35 |
| spatel | but in 5 if you loose 1 then 4 also create issue right? | 20:36 |
| spatel | I have 6 controller in new datacenter 3x infra and 3x api | 20:36 |
| spatel | i did reboot 1 node many time and my cluster survive with 2 node (not sure what issue i will see with 2 node) | 20:37 |
| ThiagoCMC | I believe that yes, 4 might bring problems lol | 20:37 |
| ThiagoCMC | With 2~4~6 .. The quorum might hit some sort of impasse (I believe) | 20:38 |
| spatel | as far as no split brain thing should work fine with 2 node. | 20:38 |
| spatel | ThiagoCMC: here is the summary of my both cloud - https://ibb.co/0B5SCCQ | 20:41 |
| ThiagoCMC | Cool! | 20:42 |
| *** waverider has joined #openstack-ansible | 20:48 | |
| *** tosky has joined #openstack-ansible | 20:55 | |
| *** openstackgerrit has quit IRC | 21:08 | |
| *** mrda has quit IRC | 21:15 | |
| admin0 | 5 controllers in your test/home lab is an overkill :) but if you have resources, go for it :) | 21:15 |
| *** mrda has joined #openstack-ansible | 21:17 | |
| *** openstackgerrit has joined #openstack-ansible | 21:29 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not warn about incorrect group naming https://review.opendev.org/c/openstack/openstack-ansible/+/760952 | 21:29 |
| *** rfolco has quit IRC | 21:34 | |
| *** cshen has quit IRC | 21:35 | |
| *** kleini has joined #openstack-ansible | 21:36 | |
| ThiagoCMC | admin0, right? =P | 21:48 |
| ThiagoCMC | I just wanna make sure that I'll never, ever lose my Ceph Mons... lol - Neither the Galera! But I've faced problems with Galera befora, it's harder than Ceph for me | 21:48 |
| admin0 | the only time i lost galera was when gnocchi used all the inodes and galera got corrupt coz the system was out of inodes :( | 21:50 |
| admin0 | thankfully i had backups made every morning | 21:50 |
| ThiagoCMC | Good to know! | 21:51 |
| admin0 | ThiagoCMC, try trove with postgresql :) | 21:51 |
| ThiagoCMC | lol | 21:51 |
| admin0 | designate with powedns backend | 21:52 |
| ThiagoCMC | That's for sure! | 21:52 |
| admin0 | i am giving you blog topics | 21:52 |
| ThiagoCMC | Why Postgres? MariaDB is so cool! | 21:52 |
| admin0 | so .. starting with mariadb | 21:52 |
| ThiagoCMC | I have an awesome MariaDB HA setup with VIP | 21:52 |
| ThiagoCMC | But not sure if makes sense when with Trove. | 21:53 |
| admin0 | postgres has postgis :) | 21:53 |
| admin0 | your osa mariadb will be completey diff from trove | 21:53 |
| ThiagoCMC | Hmm... ok. Never head of postgis =P | 21:54 |
| admin0 | https://postgis.net/features/ | 21:55 |
| *** waverider has quit IRC | 21:56 | |
| ThiagoCMC | admin0, damn! Does Postgis uses GPU?! lol | 21:56 |
| ThiagoCMC | geometry, geography, raster... O_O | 21:56 |
| *** waverider has joined #openstack-ansible | 21:56 | |
| admin0 | uncovered blog topics: trove ( mariadb, postgres) , designate (bind(done), powerdns) , kubernetes ( since magnum failed, might post kubeone+terraform method) | 21:58 |
| *** cshen has joined #openstack-ansible | 22:03 | |
| ThiagoCMC | Ok, I'll take a look at the Designate + Bind9 and try with PowerDNS | 22:03 |
| admin0 | try trove please | 22:04 |
| admin0 | first | 22:04 |
| admin0 | coz i have not tried | 22:04 |
| admin0 | designate + bind9, i have it ready | 22:04 |
| ThiagoCMC | Okdok | 22:08 |
| ThiagoCMC | BTW, here is how I do MariaDB HA primary/primary: https://github.com/tmartinx/svauto/blob/main/ansible/roles/mariadb/tasks/main.yml | 22:09 |
| admin0 | our good friends here in this channel are already doing a good job to give us mariadb for osa --why do you reinvent the wheel :) | 22:10 |
| admin0 | invent your own wheel | 22:10 |
| ThiagoCMC | Because I need it for projects based on 2 servers only. | 22:11 |
| admin0 | i fear split-brain of the maria that will make your brain -- split :) | 22:11 |
| ThiagoCMC | LOL | 22:12 |
| jrosser | if you only have 2 it is guaranteed split brain? a failed node can never rejoin on its own | 22:14 |
| admin0 | let his brain split on his own :) | 22:14 |
| ThiagoCMC | It won't split. | 22:16 |
| ThiagoCMC | ;-) | 22:16 |
| *** spatel has quit IRC | 22:16 | |
| jrosser | we built 2x haproxy 2x keyckoak 2x galera | 22:18 |
| ThiagoCMC | 2 node Galera? | 22:19 |
| jrosser | and on the first time we restart one of the galera it all went very very bad | 22:19 |
| ThiagoCMC | Oh, damn | 22:19 |
| jrosser | so straight away deployed a third node | 22:19 |
| *** rfolco has joined #openstack-ansible | 22:19 | |
| ThiagoCMC | The solution I found looks good though! I believe that the VIP thing will avoid the split brain | 22:20 |
| ThiagoCMC | The services doesn't know the individual IPs, only the VIP, and keepalived takes care of it. But, this is a prototype, I have no advanced checks nor anything fancy on keepalived. | 22:21 |
| ThiagoCMC | I like its simplicity! | 22:21 |
| ThiagoCMC | I used it for Icinga, Kea DHCP and a few others... | 22:22 |
| *** rfolco has quit IRC | 22:24 | |
| *** jbadiapa has quit IRC | 22:25 | |
| jrosser | you are really looking there at how the client interacts with the db, just the same as OSA uses haproxy/keepalived | 22:25 |
| jrosser | but this is worth reading https://galeracluster.com/library/kb/two-node-clusters.html | 22:25 |
| ThiagoCMC | Thanks! | 22:27 |
| jrosser | "when one node fails, the remaining node will become non-operational" <- this is exactly what happend to us | 22:27 |
| jrosser | do maintainance on one node -> whole cluster down | 22:28 |
| ThiagoCMC | This doesn't happen with the setup I found | 22:29 |
| ThiagoCMC | Here is my extremely simple keepalived for it: https://github.com/tmartinx/svauto/tree/main/ansible/roles/keepalived-mysql | 22:30 |
| admin0 | you can't let go of svauto can you :) | 22:35 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Update octavia messaging options https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765166 | 22:36 |
| ThiagoCMC | LOLOL | 22:36 |
| *** rfolco has joined #openstack-ansible | 22:42 | |
| *** luksky has quit IRC | 23:03 | |
| *** kukacz has quit IRC | 23:04 | |
| *** kukacz has joined #openstack-ansible | 23:06 | |
| *** rfolco has quit IRC | 23:17 | |
| *** nurdie has quit IRC | 23:22 | |
| *** nurdie has joined #openstack-ansible | 23:44 | |
| *** nurdie has quit IRC | 23:49 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!