| *** threestrands has joined #openstack-ansible | 00:35 | |
| *** jhesketh has joined #openstack-ansible | 02:56 | |
| *** cshen has joined #openstack-ansible | 03:45 | |
| *** cshen has quit IRC | 03:49 | |
| *** evrardjp has quit IRC | 04:23 | |
| *** udesale has joined #openstack-ansible | 04:38 | |
| *** evrardjp has joined #openstack-ansible | 05:41 | |
| *** shyamb has joined #openstack-ansible | 05:48 | |
| *** shyamb has quit IRC | 06:04 | |
| *** miloa has joined #openstack-ansible | 06:04 | |
| *** shyamb has joined #openstack-ansible | 06:05 | |
| *** fghaas has joined #openstack-ansible | 06:17 | |
| *** cshen has joined #openstack-ansible | 06:41 | |
| *** jbadiapa has joined #openstack-ansible | 06:49 | |
| *** shyamb has quit IRC | 06:53 | |
| *** shyamb has joined #openstack-ansible | 06:53 | |
| *** thuydang has joined #openstack-ansible | 06:59 | |
| *** shyamb has quit IRC | 07:03 | |
| *** pcaruana has joined #openstack-ansible | 07:12 | |
| *** rpittau|afk is now known as rpittau | 07:12 | |
| *** threestrands_ has joined #openstack-ansible | 07:18 | |
| *** threestrands_ has quit IRC | 07:18 | |
| *** threestrands has quit IRC | 07:21 | |
| masterpe | I removed elasticsearch cluster with openstack-ansible /opt/openstack-ansible-ops/elk_metrics_6x/site.yml -e 'elk_package_state="absent"' --tags package_install | 07:31 |
|---|---|---|
| masterpe | Like described on https://github.com/openstack/openstack-ansible-ops/tree/master/elk_metrics_6x | 07:32 |
| masterpe | But the groups are like elastic-logstash_hosts still exists | 07:33 |
| masterpe | How to remove the groups? | 07:33 |
| *** udesale_ has joined #openstack-ansible | 07:47 | |
| *** udesale has quit IRC | 07:50 | |
| jrosser | masterpe: you need to remove the files in env.d which were added for the ELK hosts | 07:50 |
| jrosser | remove the host entries from openstack_user_config.yml | 07:50 |
| jrosser | then use scripts/inventory_manage.py to remove the hosts and groups from the inventory | 07:50 |
| kleini | I tried now to add python 3 library to uWSGI role. The deployment fails. I don't see the uwsgi playbook running anywhere at least not for Keystone: http://paste.openstack.org/show/792392/. There is a task in uwsgi/tasks/uwsgi_install.yml named "Create uWSGI system group". I can not find this task name in the run, therefore I don't think uwsgi playbook is used for Keystone. So how to proceed to fix the missing python 3 | 08:21 |
| kleini | library for uWSGI based services? | 08:21 |
| *** sshnaidm has joined #openstack-ansible | 08:22 | |
| *** shyamb has joined #openstack-ansible | 08:23 | |
| kleini | Found the import_role: name: uwsgi in some OS daemon deployments. Keystone is an exception. | 08:25 |
| kleini | https://review.opendev.org/#/c/720546/1 <- this is then required for me for Train | 08:26 |
| masterpe | jrosser: I have done that, if I do a inventory-manage.py -r elastic-logstash_hosts, then a inventory-manage.py -G the group is still there | 08:33 |
| *** cshen_ has joined #openstack-ansible | 08:45 | |
| *** shyam89 has joined #openstack-ansible | 08:45 | |
| *** cshen has quit IRC | 08:48 | |
| *** shyamb has quit IRC | 08:48 | |
| masterpe | If I start dynamic_inventory.py I get the error KeyError: u'elastic-logstash_hosts' | 08:54 |
| *** tosky has joined #openstack-ansible | 09:01 | |
| jrosser | have you removed the hosts from the inventory as well as the group? | 09:03 |
| masterpe | The group elastic-logstash_hosts is not empty | 09:19 |
| masterpe | In this group it has the physical controllers | 09:20 |
| *** shyam89 has quit IRC | 09:20 | |
| masterpe | This machine is also in use for cinder | 09:20 |
| *** shyam89 has joined #openstack-ansible | 09:20 | |
| *** gregwork has quit IRC | 09:24 | |
| *** this10nly has joined #openstack-ansible | 09:29 | |
| kleini | jrosser: adding libpython3.6 to the uwsgi_distro_packages still does not solve the problem for me. install method for uwsgi is still source for me an then this package list is not considered at all as python_venv_build is running. Unfortunately that does not install libpython3.6 on the target container. I will try to understand what python_venv_build does but some help maybe solves the issue faster for me. | 09:30 |
| *** shyam89 has quit IRC | 09:32 | |
| *** shyamb has joined #openstack-ansible | 09:35 | |
| *** shyam89 has joined #openstack-ansible | 09:45 | |
| *** shyamb has quit IRC | 09:48 | |
| *** rpittau is now known as rpittau|bbl | 10:15 | |
| *** shyam89 has quit IRC | 10:33 | |
| jrosser | kleini: have you tried replacing this with the python3 library? https://github.com/openstack/openstack-ansible-os_keystone/blob/master/vars/debian.yml#L23 | 10:42 |
| jrosser | becasue as i understand it this is specific to the keystone role | 10:43 |
| kleini | I replaced that and keystone deployment works then. This topic is fine. I try now to fix all the other uWSGI based services with fixes in the ansible-role-uwsgi. That role still uses source install method and yet I have not clue, where to tell it to install libpython3.6 | 10:44 |
| jrosser | i am still surprised that it fails | 10:45 |
| kleini | still the same problem as last week. uwsgi binary is missing libpython3.6m.so.1.0 | 10:45 |
| jrosser | which OS are you using? | 10:47 |
| kleini | http://paste.openstack.org/show/792404/, Ubuntu 18.04 LTS | 10:48 |
| kleini | http://paste.openstack.org/show/792393/ <- placement deployment log | 10:49 |
| jrosser | i have that library available http://paste.openstack.org/show/792405/ | 10:50 |
| jrosser | then when i look in the container base image libpython3.6m is present http://paste.openstack.org/show/792406/ | 10:52 |
| jrosser | this is why i am hesitating to just patch roles to fix this, i'd rather understand actually whats happening here | 10:53 |
| kleini | I totally agree with you, that is is necessary to understand, what is the cause. My base container does not contain libpython3.6m.so.1.0 | 10:54 |
| kleini | so there is the cause. | 10:54 |
| jrosser | i will test this in a VM | 10:58 |
| jrosser | kleini: this is train branch? | 11:00 |
| kleini | stable/train | 11:00 |
| kleini | I would say, problem is here: https://opendev.org/openstack/openstack-ansible-nspawn_hosts/src/branch/master/vars/ubuntu-18.04.yml#L33 there only libpython2.7 is listed | 11:02 |
| jrosser | oh man - nspawn :( | 11:07 |
| kleini | yes, nspawn, of course! | 11:07 |
| kleini | nothing's better than that ;-) | 11:08 |
| jrosser | so none of this is to do with the uwsgi or other roles | 11:08 |
| jrosser | it needs fixing there | 11:08 |
| kleini | just added libpython3.6 there and will test with placement now | 11:08 |
| kleini | does LXC use /var/lib/machines, too? | 11:09 |
| jrosser | you'll need to force rebuilding of the base image, and delete the existing container | 11:09 |
| jrosser | the OSA lxc role is set up to keep the container image in the same place | 11:10 |
| jrosser | if you really want to use nspawn then i'd recommend adding and maintaining a CI job using nspawn | 11:11 |
| jrosser | becasue this is not getting any testing at all | 11:11 |
| kleini | I have no idea, how to setup a CI job using nspawn? Maybe I need to switch to LXC. I see some _zfs playbooks in the LXC role. Does it support ZFS storage? That would be really nice! | 11:13 |
| jrosser | i use lxc + zfs | 11:14 |
| kleini | for all productive target hosts - as long as they are physical hardware - I use Ubuntu 18.04 on ZFS root and all my filesystems, partitions, hardware failures are gone! | 11:14 |
| jrosser | do you PXEboot those? | 11:15 |
| kleini | nope, still USB stick and SSH console for initial setup | 11:15 |
| kleini | we're too small | 11:15 |
| jrosser | zfs + lxc http://paste.openstack.org/show/792407/ | 11:16 |
| kleini | and placement uWSGI stuff is up and running | 11:17 |
| jrosser | well we should patch the nspawn role for that | 11:17 |
| jrosser | ubuntu + pxeboot + zfsroot is sad situation - no real support in debian-installer for zfsroot | 11:18 |
| jrosser | so i use very small md raid1 for root then mount a raidz1 over the important bits | 11:19 |
| kleini | root ZFS with mirrored bpool and rpool satisfies all my needs | 11:23 |
| *** thuydang has quit IRC | 11:24 | |
| kleini | jrosser: lxc_hosts/var/ubuntu-18.04.yml lists python3-dev. So -dev packages are necessary as you said, no -dev packages on productive containers. | 11:28 |
| kleini | ? | 11:28 |
| *** this10nly has quit IRC | 11:31 | |
| *** this10nly has joined #openstack-ansible | 11:33 | |
| *** cshen_ has quit IRC | 11:35 | |
| jrosser | kleini: ideally the -dev packages are not in the base image because they are not needed in most of the containers | 11:41 |
| *** cshen has joined #openstack-ansible | 11:41 | |
| jrosser | i have a patch from a very long time ago which was never merged that removes all that https://review.opendev.org/#/c/613078/19/vars/ubuntu-18.04.yml | 11:41 |
| *** cshen has quit IRC | 11:46 | |
| *** rh-jelabarre has joined #openstack-ansible | 11:50 | |
| openstackgerrit | Marcus Klein proposed openstack/openstack-ansible-nspawn_hosts master: Python 3 library is needed for uWSGI. https://review.opendev.org/721246 | 11:51 |
| kleini | ^^^ my most minimal fix | 11:52 |
| *** rpittau|bbl is now known as rpittau | 12:03 | |
| *** cshen has joined #openstack-ansible | 12:14 | |
| *** udesale_ has quit IRC | 12:34 | |
| *** irclogbot_1 has quit IRC | 13:06 | |
| *** irclogbot_0 has joined #openstack-ansible | 13:07 | |
| *** thuydang has joined #openstack-ansible | 13:08 | |
| *** thuydang has quit IRC | 13:38 | |
| *** thuydang has joined #openstack-ansible | 13:54 | |
| *** miloa has quit IRC | 13:56 | |
| -openstackstatus- NOTICE: Zuul is temporarily offline; service should be restored in about 15 minutes. | 13:57 | |
| *** cshen has quit IRC | 14:25 | |
| *** this10nly has quit IRC | 14:28 | |
| *** cshen has joined #openstack-ansible | 14:45 | |
| *** rh-jelabarre has quit IRC | 14:58 | |
| Nick_A | We have nova_resume_guests_state_on_host_boot set to False for a specific compute host in our openstack_user_config.yml file. Should nova.conf on the compute host be changed to False after running playbooks? How do we confirm that value is correctly applied? | 15:01 |
| *** mgariepy has joined #openstack-ansible | 15:03 | |
| jrosser | Nick_A: you should check this https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/master/releasenotes/notes/removed-variables-ce24689e104e537c.yaml | 15:03 |
| dmsimard | jrosser, mnaser, noonedeadpunk: btw I tested the latest release of ara with https://review.opendev.org/#/c/696634/ -- data: https://api.trunk.demo.recordsansible.org/?name=&path=openstack-ansible&label= | 15:09 |
| *** rh-jelabarre has joined #openstack-ansible | 15:09 | |
| dmsimard | and in API form: https://api.trunk.demo.recordsansible.org/api/v1/playbooks?path=openstack-ansible | 15:09 |
| Nick_A | jrosser thank you | 15:10 |
| Nick_A | Second, unrelated question if anyone has time: what timeouts or other settings might we be overlooking that would sometimes cause errors like this when taking snapshots? http://paste.openstack.org/show/792417/ | 15:13 |
| *** mmethot has quit IRC | 15:19 | |
| *** redkrieg has joined #openstack-ansible | 15:25 | |
| redkrieg | If I set a number of things in nova_nova_conf_overrides under a file called user_nova.yml and then set one specific key in that under host_vars for a host in my openstack_user_config.yml, will ansible merge those dictionaries for that host or do I need to specify each key modified in user_nova.yml under host_vars for the host I want to tweak? | 15:28 |
| *** yolanda has joined #openstack-ansible | 15:28 | |
| *** mmethot has joined #openstack-ansible | 15:34 | |
| jrosser | you can only specify config overrides in one place | 15:35 |
| jrosser | ^ bad choice of words :) | 15:36 |
| jrosser | if you define them in more than one place then the highest precedence in ansible terms will win | 15:36 |
| jrosser | user_*.yml is passed on the ansible-playbook command line with the -e flag so that will always override anything you have in host/group vars | 15:37 |
| *** gyee has joined #openstack-ansible | 15:40 | |
| *** gshippey has joined #openstack-ansible | 15:42 | |
| *** cshen has quit IRC | 15:45 | |
| -openstackstatus- NOTICE: Gerrit will be restarted to correct a misconfiguration which caused some git mirrors to have outdated references. | 15:46 | |
| *** ianychoi_ is now known as ianychoi | 15:53 | |
| redkrieg | thank you, I'll see if I can find another way to set this per-host then | 15:57 |
| *** KeithMnemonic has joined #openstack-ansible | 16:00 | |
| *** rpittau is now known as rpittau|afk | 16:07 | |
| *** yolanda has quit IRC | 16:17 | |
| *** cshen has joined #openstack-ansible | 16:18 | |
| *** thuydang has quit IRC | 16:21 | |
| *** fghaas has quit IRC | 16:22 | |
| *** cshen has quit IRC | 16:23 | |
| *** yolanda has joined #openstack-ansible | 16:29 | |
| *** evrardjp has quit IRC | 16:35 | |
| *** evrardjp has joined #openstack-ansible | 16:35 | |
| *** jbadiapa has quit IRC | 16:37 | |
| jrosser | redkrieg: you can define the config override in group_vars which itself could include a new variable you define for each host, or maybe use a default() as well | 16:41 |
| *** sshnaidm is now known as sshnaidm|afk | 16:59 | |
| redkrieg | jrosser: is this the right way to do that? http://paste.openstack.org/show/792423/ | 17:05 |
| redkrieg | assuming the nova_nova_conf_overrides contained the rest of what I want in there as well | 17:06 |
| noonedeadpunk | dmsimard: so we're supposed to use recordansible instead of ara_report folder to look for the results? | 17:18 |
| noonedeadpunk | or it's just a dnm part? | 17:19 |
| dmsimard | noonedeadpunk: I don't have the funding to host an openstack-wide instance :p | 17:19 |
| dmsimard | that was just a dnm/test | 17:20 |
| noonedeadpunk | ah, ok:) | 17:20 |
| dmsimard | there's nothing preventing you from standing up your own within OSA, though | 17:20 |
| noonedeadpunk | so yeah, at least now we now what playbook we're talking about, which is nice. And it seems the reverted order is pretty ok too | 17:20 |
| noonedeadpunk | I think at this time we're probably comfortable with storing with test results... As I'm not sure that this overall format is somehow informatve. | 17:21 |
| noonedeadpunk | Like we mostly need to look at report at specific job against specific distro | 17:22 |
| dmsimard | noonedeadpunk: right, so if you look at https://api.demo.recordsansible.org/ you'll see there are labels that can be used to categorize things | 17:22 |
| dmsimard | per project, per job, could have per zuul buildset too | 17:23 |
| dmsimard | or per patchset/change | 17:23 |
| noonedeadpunk | yeah, I see, just not sure how handy it is, unless there's possibility to get direct link from zuul results page to this | 17:26 |
| noonedeadpunk | And also hosting question is quite interesting as well :p | 17:26 |
| *** yolanda has quit IRC | 17:26 | |
| *** cshen has joined #openstack-ansible | 17:40 | |
| *** evrardjp has quit IRC | 17:44 | |
| *** cshen has quit IRC | 17:45 | |
| jrosser | hmm neutron-tempest-plugin...... now that needs python >= 3.6 | 17:48 |
| jrosser | lots and lots of broken stable branches now | 17:48 |
| *** evrardjp has joined #openstack-ansible | 17:49 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest stable/train: Pin neutron-tempest-plugin to 0.9.0 for Train branch. https://review.opendev.org/721319 | 17:57 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Bump SHAs for stable/train https://review.opendev.org/721090 | 17:58 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest stable/stein: Pin neutron-tempest-plugin to 0.9.0 for Train branch. https://review.opendev.org/721320 | 18:00 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/stein: Bump SHAs for stable/stein https://review.opendev.org/721091 | 18:01 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/721089 | 18:03 |
| openstackgerrit | Marcus Klein proposed openstack/openstack-ansible-nspawn_hosts master: Python 3 library is needed for uWSGI. https://review.opendev.org/721246 | 18:05 |
| *** prometheanfire has quit IRC | 18:15 | |
| *** fghaas has joined #openstack-ansible | 18:21 | |
| *** jrosser has quit IRC | 18:41 | |
| *** jrosser has joined #openstack-ansible | 18:42 | |
| *** yolanda has joined #openstack-ansible | 18:46 | |
| *** yolanda has quit IRC | 18:46 | |
| *** prometheanfire has joined #openstack-ansible | 18:56 | |
| *** jmccrory has quit IRC | 19:08 | |
| *** jmccrory has joined #openstack-ansible | 19:09 | |
| *** fghaas has quit IRC | 19:19 | |
| *** cshen has joined #openstack-ansible | 19:40 | |
| *** cshen has quit IRC | 19:45 | |
| *** thuydang has joined #openstack-ansible | 19:47 | |
| *** gshippey has quit IRC | 19:47 | |
| *** gshippey has joined #openstack-ansible | 21:07 | |
| *** cshen has joined #openstack-ansible | 21:41 | |
| *** cshen has quit IRC | 21:45 | |
| *** cshen has joined #openstack-ansible | 22:18 | |
| *** tosky has quit IRC | 22:19 | |
| *** cshen has quit IRC | 22:23 | |
| *** mrda has quit IRC | 22:31 | |
| *** mrda has joined #openstack-ansible | 22:33 | |
| *** threestrands has joined #openstack-ansible | 23:18 | |
| *** gshippey has quit IRC | 23:27 | |
| *** thuydang has quit IRC | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!