| *** spatel has joined #openstack-ansible | 01:26 | |
| *** spatel has quit IRC | 02:01 | |
| *** mgoddard has quit IRC | 03:33 | |
| *** mgoddard has joined #openstack-ansible | 03:33 | |
| *** miloa has joined #openstack-ansible | 05:39 | |
| *** halali_ has joined #openstack-ansible | 05:43 | |
| *** luksky has joined #openstack-ansible | 06:45 | |
| *** Adri2000 has joined #openstack-ansible | 06:53 | |
| *** andrewbonney has joined #openstack-ansible | 07:21 | |
| *** rpittau|afk is now known as rpittau | 07:26 | |
| *** hindret has joined #openstack-ansible | 08:08 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Re-add NoLogPasswords ansible-lint rule https://review.opendev.org/c/openstack/openstack-ansible/+/795062 | 08:30 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Re-add NoLogPasswords ansible-lint rule https://review.opendev.org/c/openstack/openstack-ansible/+/795062 | 08:36 |
| *** sshnaidm|afk is now known as sshnaidm | 08:51 | |
| *** sshnaidm has quit IRC | 08:58 | |
| *** sshnaidm has joined #openstack-ansible | 08:59 | |
| jonher | https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/794199 should have addressed the comments | 09:17 |
| noonedeadpunk | lgtm | 09:29 |
| jonher | great | 09:35 |
| jonher | io saw your additional comments, should we also remove extras/os-cloudkitty-install.yml since that is also in integrated repo? then there is no more extras folder. either way we updated the install.yml in integrated repo so it's not the same currently | 10:30 |
| noonedeadpunk | yeah, I think we don't need extras at all nowadays | 10:34 |
| opendevreview | Jonathan Herlin proposed openstack/openstack-ansible-os_cloudkitty master: Cloudkitty role cleanup and config updates https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/794199 | 10:34 |
| noonedeadpunk | but we can drop it with follow=up as well | 10:34 |
| noonedeadpunk | jonher: also ther're trailing whitespaces in reno | 10:35 |
| jonher | oh, i'll fix that | 10:35 |
| opendevreview | Jonathan Herlin proposed openstack/openstack-ansible-os_cloudkitty master: Cloudkitty role cleanup and config updates https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/794199 | 10:36 |
| jonher | extras removed and whitespaces fixed | 10:36 |
| *** ioni has quit IRC | 11:38 | |
| *** luksky has quit IRC | 11:42 | |
| *** luksky has joined #openstack-ansible | 11:42 | |
| jrosser | looking at the badly failing centos-8 jobs it looks like somehow the centos-8-stream packages are trying to install on a not-stream centos-8 | 11:52 |
| *** odyssey4me has quit IRC | 11:52 | |
| jrosser | oh well also CentOS 8.4.2105 seems to be a thing now | 11:53 |
| jrosser | ansible_distribution_version: '8.4' | 11:56 |
| *** ioni has joined #openstack-ansible | 11:58 | |
| *** ioni has quit IRC | 11:59 | |
| *** tosky has joined #openstack-ansible | 11:59 | |
| noonedeadpunk | oh damn | 12:03 |
| noonedeadpunk | that's what it about :( | 12:03 |
| noonedeadpunk | I actually thought it's some repo issue :( | 12:04 |
| *** ioni has joined #openstack-ansible | 12:05 | |
| jrosser | i'm not actually sure what it is now tbh | 12:12 |
| noonedeadpunk | as like iptables from i686 was trying to be installed | 12:15 |
| noonedeadpunk | which could not be any centos 8 | 12:15 |
| noonedeadpunk | *libselinux | 12:16 |
| jrosser | hmm well thats actually is libselinux-devel-2.9-4.el8_3.i686 | 12:17 |
| jrosser | 8_3 rather than 8_4 or something | 12:17 |
| jrosser | it should be wanting this instead http://rpmfind.net/linux/RPM/centos/8.4.2105/baseos/x86_64/Packages/libselinux-devel-2.9-5.el8.x86_64.html | 12:18 |
| *** wonder has joined #openstack-ansible | 12:19 | |
| noonedeadpunk | also I clean forgot what I recently propesed that assumed 8.3 to be latest one.... | 12:19 |
| jrosser | oh some talk in #opendev about mirror storage being full | 12:20 |
| noonedeadpunk | yeah, indeed... | 12:20 |
| *** ioni has quit IRC | 12:25 | |
| *** wonder is now known as ioni | 12:32 | |
| admin1 | if i need to change port of any service, which file has the ports data ? | 12:33 |
| *** odyssey4me has joined #openstack-ansible | 12:42 | |
| noonedeadpunk | um, it's in defaults/main.yml | 12:44 |
| noonedeadpunk | but yu will also need to change haproxy endpoints as well, as we don't use variables for ports data there | 12:44 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_panko stable/victoria: Fix wsgi_venv path for panko-api https://review.opendev.org/c/openstack/openstack-ansible-os_panko/+/795039 | 12:52 |
| *** spatel has joined #openstack-ansible | 12:52 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_panko stable/ussuri: Fix wsgi_venv path for panko-api https://review.opendev.org/c/openstack/openstack-ansible-os_panko/+/795040 | 12:52 |
| admin1 | noonedeadpunk, thanks | 12:53 |
| *** odyssey4me has quit IRC | 12:54 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum stable/victoria: Define region for Magnum trust https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/795043 | 12:54 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum stable/ussuri: Define region for Magnum trust https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/795044 | 12:54 |
| noonedeadpunk | ok, found patch where I messed up with 8,3 - https://opendev.org/openstack/openstack-ansible-os_nova/commit/e1e078350a5657be09c688430ad600a89977775d | 12:59 |
| noonedeadpunk | not sure though if fixed edk2-ovmf is part of 8.4 and stream though | 12:59 |
| *** odyssey4me has joined #openstack-ansible | 13:00 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts master: Add CentOS 8.4 support https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/795102 | 13:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Add CentOS 8.4 support https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/795103 | 13:06 |
| *** mgariepy has quit IRC | 13:27 | |
| *** mgariepy has joined #openstack-ansible | 13:28 | |
| *** jnamdar has joined #openstack-ansible | 13:32 | |
| *** miloa has quit IRC | 13:48 | |
| spatel | noonedeadpunk any idea how to resolve this issue - https://zuul.opendev.org/t/openstack/build/3e57c803e81b42e999bae665e03fab2c | 14:34 |
| opendevreview | Satish Patel proposed openstack/openstack-ansible-os_neutron master: Add ovn clustering support https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/794266 | 14:39 |
| spatel | no able to pinpoint error from CI logs | 14:45 |
| jrosser | spatel: https://zuul.opendev.org/t/openstack/build/3e57c803e81b42e999bae665e03fab2c/log/job-output.txt#21080-21085 | 14:53 |
| spatel | I already dig into massive logs file because of size hard to understand which section is broken | 14:55 |
| spatel | https://zuul.opendev.org/t/openstack/build/3e57c803e81b42e999bae665e03fab2c | 14:56 |
| jrosser | i'm giving you a link to line 21080 to 21085 | 14:57 |
| spatel | hmm my browser didn't take me there.. | 14:57 |
| jrosser | find the place where things fail, i usually search for "RUN END" in the log and find the one with failed: 1 just before it | 14:58 |
| jrosser | no it won't | 14:58 |
| jrosser | but those lines will be hightlighted yellow if you scroll down to them | 14:58 |
| jrosser | i think this came up last week - that the services are trying to be stopped, but they do not exist | 14:58 |
| spatel | Found it :) | 14:59 |
| spatel | Let me debug and see | 14:59 |
| spatel | i can spin up in my lab | 14:59 |
| spatel | It would be good if i can login into CI build vm to debug | 15:00 |
| jrosser | you can request a hold on a job if you want to do that | 15:00 |
| jrosser | normally the node is deleted right after the logs are collected | 15:00 |
| spatel | no worry, let me debug it in my lab must be something silly going on here. | 15:01 |
| noonedeadpunk | just to say that - hold can be asked in #opendev | 15:01 |
| noonedeadpunk | but it's really more for exception when issue can't be reproduced locally | 15:02 |
| jrosser | spatel: also | 15:02 |
| jrosser | TASK [os_neutron : Install ovn-northd packages] | 15:02 |
| jrosser | skipping: [aio1_neutron_server_container-20dcbd39] | 15:02 |
| jrosser | ^ like find the root cause of this i think | 15:02 |
| spatel | Let me build it in my lab first | 15:02 |
| jrosser | it should just need exactly the same config as CI with your patch applied to os_neutron after bootstrap_ansible.sh | 15:03 |
| jrosser | spatel: it skips installing ovn-northd on aio1_neutron_server_container-20dcbd39, which feels correct | 15:13 |
| jrosser | it then tries to restart it on aio1_neutron_server_container-20dcbd39 which fails | 15:14 |
| jrosser | but we have a container aio1_neutron_ovn_northd_container-ef8e38fe which is where it should actually be | 15:14 |
| noonedeadpunk | So, we're anyway in circle dependescy with centos. As we need to land https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/795103 https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/795102 and https://review.opendev.org/c/openstack/openstack-ansible/+/794795 but they all kind of depend on each other | 15:26 |
| noonedeadpunk | So I'd push setting CentOS to NV now | 15:26 |
| noonedeadpunk | to land all required changes | 15:26 |
| *** ThiagoCMC has quit IRC | 15:39 | |
| *** rpittau is now known as rpittau|afk | 16:01 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Move CentOS 8 jobs to NV https://review.opendev.org/c/openstack/openstack-ansible/+/795153 | 16:16 |
| *** ThiagoCMC has joined #openstack-ansible | 16:38 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Gather extra facts for healthcheck playbooks https://review.opendev.org/c/openstack/openstack-ansible/+/794795 | 16:58 |
| *** djhankb has quit IRC | 17:00 | |
| *** djhankb has joined #openstack-ansible | 17:01 | |
| *** andrewbonney has quit IRC | 17:04 | |
| noonedeadpunk | aio on 8.4 passed relatively clean, except really some small nits | 17:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Add CentOS 8.4 support https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/795103 | 17:18 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Gather extra facts for healthcheck playbooks https://review.opendev.org/c/openstack/openstack-ansible/+/794795 | 17:20 |
| *** odyssey4me has quit IRC | 17:44 | |
| spatel | jrosser back now, sorry i got busy in company work | 18:00 |
| spatel | you said, <jrosser> it should just need exactly the same config as CI with your patch applied to os_neutron after bootstrap_ansible.sh | 18:01 |
| spatel | i thought CI and manual building should be close | 18:02 |
| jrosser | did you use lxc locally? | 18:07 |
| spatel | what do you mean lxc locally? | 18:17 |
| spatel | i didn't start yet to build aio | 18:17 |
| spatel | just kickstarted new ubuntu box | 18:22 |
| spatel | going to build aio on it with my patch | 18:22 |
| jrosser | i mean, when you were testing the OVN stuff did you export SCENARIO=aio_metal, i remember you found that to be much quicker, as theres no lxc | 18:52 |
| jrosser | the metal deploy lets you get away with a ton of errors about ansible groups as all the components are collapsed onto the same node | 18:53 |
| jrosser | those only show up with LXC deploys, which is why i wondered in the previous tests you did if it was with or without containers | 18:53 |
| *** ioni has quit IRC | 18:54 | |
| spatel | earlier i build AIO-OVN using metal but never build with LXC (i did multi-node deployment using LXC though) | 18:58 |
| spatel | jrosser ^ | 18:58 |
| spatel | let me try building AIO using LXC in my lab to see what issue we encounter | 18:59 |
| spatel | last time i did this SCENARIO='aio_ovn_metal' | 19:14 |
| spatel | where is aio_ovn_metal defined ? | 19:14 |
| spatel | jrosser ^^ | 19:14 |
| jrosser | spatel: it’s kind of parsed, not defined anywhere | 19:15 |
| spatel | oh | 19:15 |
| spatel | should i be doing aio_ovn_lxc ? | 19:15 |
| jrosser | yes try that | 19:16 |
| spatel | ok | 19:16 |
| spatel | why don't we use apply_security_hardening: True for AIO deployment :) | 19:23 |
| spatel | why don't we use apply_security_hardening: False for AIO deployment :) | 19:23 |
| spatel | I meant false | 19:23 |
| spatel | Every time i build aio and forgetting to set false and later its painful to work and doing testing because of lots of security stuff | 19:24 |
| spatel | also it takes little time to add sec stuff | 19:25 |
| noonedeadpunk | ok, awesome, centos8 seems "unblocked" with https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/795102 | 19:57 |
| noonedeadpunk | it's just symlink to 8.3 just in case | 19:59 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/ussuri: Disable magnum zero nodes test. https://review.opendev.org/c/openstack/openstack-ansible/+/795137 | 20:02 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum stable/ussuri: Define region for Magnum trust https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/795044 | 20:03 |
| spatel | noonedeadpunk i thought centos is gone gone | 20:16 |
| spatel | jrosser i hit the same issue what CI hit so its good | 20:48 |
| spatel | debugging started.... | 20:48 |
| spatel | hmm why its looking for ovs services on neutron-server, they don't be there -http://paste.openstack.org/show/806436/ | 20:51 |
| spatel | it should be on ovn container so look like something is messed up here | 20:51 |
| spatel | my patch has some issue, it doesn't know where to run... | 20:54 |
| spatel | i need to set when condition here | 20:54 |
| spatel | jrosser i have added when condition here so why it still trying to run this command on neutron-server ? - http://paste.openstack.org/show/806438/ | 21:06 |
| jrosser | spatel: the services are defined here https://github.com/openstack/openstack-ansible-os_neutron/blob/master/vars/main.yml#L346 | 21:07 |
| spatel | https://github.com/openstack/openstack-ansible-os_neutron/blob/master/vars/main.yml#L490 | 21:08 |
| spatel | its there | 21:08 |
| jrosser | is this right? - neutron_services['neutron-ovn-northd']['group'] in group_names | 21:08 |
| jrosser | isnt group names all the groups | 21:09 |
| spatel | i took example from other playbooks code | 21:09 |
| jrosser | feels like saying "is this group name in all the group names" -> always true | 21:09 |
| spatel | i took that example from here - https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/tasks/providers/ovn_config.yml#L27 | 21:10 |
| jrosser | oh hold on "group_names List of groups the current host is part of" | 21:11 |
| jrosser | so i think you need to add debug here to look at what group_names contains | 21:12 |
| spatel | ok let me try -vvvv | 21:12 |
| spatel | do i need when condition under block: code? | 21:13 |
| spatel | i don't think so | 21:13 |
| jrosser | this could be an inventory structure issue | 21:14 |
| jrosser | look at which containers are in which groups, there should be neutron_ovn_northd group | 21:14 |
| jrosser | but i would add a debug task to print group_names near your block: | 21:14 |
| spatel | look at this inventory output - http://paste.openstack.org/show/806439/ | 21:16 |
| spatel | we do have neutron_ovn_northd group | 21:16 |
| jrosser | ok right so we need to debug | 21:18 |
| jrosser | i still say you need to print group_names to see what is there | 21:19 |
| spatel | how do i print group_name in playbook ? | 21:19 |
| jrosser | add a debug: task | 21:19 |
| jrosser | var: group_names | 21:19 |
| spatel | msg: | 21:19 |
| spatel | ok | 21:19 |
| opendevreview | Merged openstack/openstack-ansible-tests stable/victoria: Workaround linting issue https://review.opendev.org/c/openstack/openstack-ansible-tests/+/792906 | 21:20 |
| spatel | jrosser here is the debug output - http://paste.openstack.org/show/806440/ | 21:21 |
| spatel | i am not seeing northd here | 21:22 |
| spatel | why this line saying service_en: False ? - https://github.com/openstack/openstack-ansible-os_neutron/blob/master/vars/main.yml#L492 | 21:24 |
| jrosser | spatel: we need to work out why the task runs when we think it should not | 21:27 |
| spatel | hmm | 21:28 |
| spatel | also why its not able to find group | 21:28 |
| jrosser | now try printing the other part of the expression, the left hand side | 21:28 |
| jrosser | because it’s running against neutron server host | 21:28 |
| spatel | neutron_services['neutron-ovn-northd']['group'] ? | 21:28 |
| jrosser | which, correctly, is not in the northd group | 21:28 |
| jrosser | yes | 21:28 |
| spatel | ok | 21:28 |
| spatel | may be this is meant to run for metal not lxc | 21:29 |
| spatel | debug output print - "neutron_services['neutron-ovn-northd']['group']": "neutron_ovn_northd" | 21:30 |
| spatel | that is correct | 21:30 |
| spatel | but that group isn't part of this list - http://paste.openstack.org/show/806440/ | 21:31 |
| spatel | BRB | 21:31 |
| *** luksky has quit IRC | 21:34 | |
| admin1 | doesn't ansible honor .ssh/config host key files ? | 21:34 |
| admin1 | i have a host where i need to give another key for root .. its in config so ssh host works fine | 21:35 |
| admin1 | but when i run the ansible playbook, it does not use the config it seems | 21:35 |
| admin1 | is there a way to tell osa to use a specific key when running playbooks ( doing ssh as root ) | 21:35 |
| jrosser | ansible pretty much always just leverages your local ssh config | 21:36 |
| *** spatel has quit IRC | 21:42 | |
| *** tosky has quit IRC | 22:04 | |
| admin1 | thanks jrosser.. i had used c1: ip: .. and ssh c1 worked as root .. but ansible was doing ssh root@Ip directly | 22:15 |
| jrosser | yeah, it uses whatever is in the inventory | 22:15 |
| admin1 | basically ssh infra1 worked as root ssh infra1's ip did not worked | 22:15 |
| admin1 | all working now .. thanks | 22:15 |
| *** Adri2000 has quit IRC | 22:51 | |
| *** Adri2000 has joined #openstack-ansible | 22:54 | |
| *** Adri2000 has quit IRC | 23:14 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!