| *** nicolasbock has quit IRC | 00:06 | |
| *** nicolasbock has joined #openstack-ansible | 00:06 | |
| *** cshen has joined #openstack-ansible | 00:17 | |
| *** nicolasbock has quit IRC | 00:20 | |
| *** gyee has quit IRC | 00:33 | |
| *** macz has quit IRC | 00:36 | |
| *** cshen has quit IRC | 00:41 | |
| *** cshen has joined #openstack-ansible | 00:46 | |
| *** cshen_ has joined #openstack-ansible | 01:06 | |
| *** cshen has quit IRC | 01:08 | |
| *** born2bake has quit IRC | 01:31 | |
| *** openstackstatus has joined #openstack-ansible | 01:43 | |
| *** ChanServ sets mode: +v openstackstatus | 01:43 | |
| *** cshen_ has quit IRC | 01:50 | |
| *** nicolasbock has joined #openstack-ansible | 02:18 | |
| *** cshen has joined #openstack-ansible | 02:23 | |
| *** nicolasbock has quit IRC | 02:26 | |
| *** nicolasbock has joined #openstack-ansible | 02:27 | |
| *** cshen has quit IRC | 02:27 | |
| *** nicolasbock has quit IRC | 02:44 | |
| *** lixingxing has joined #openstack-ansible | 02:44 | |
| *** nicolasbock has joined #openstack-ansible | 02:51 | |
| *** cshen has joined #openstack-ansible | 02:54 | |
| *** idlemind has quit IRC | 02:55 | |
| *** lixingxing has quit IRC | 03:15 | |
| *** nicolasbock has quit IRC | 03:21 | |
| *** cshen has quit IRC | 03:25 | |
| *** rohit02 has joined #openstack-ansible | 03:27 | |
| *** cshen has joined #openstack-ansible | 03:31 | |
| *** nurdie has joined #openstack-ansible | 03:43 | |
| *** cshen has quit IRC | 03:52 | |
| *** cshen has joined #openstack-ansible | 04:00 | |
| *** cshen has quit IRC | 04:08 | |
| *** nurdie has joined #openstack-ansible | 04:12 | |
| *** nurdie has quit IRC | 04:16 | |
| *** goldyfruit_ has quit IRC | 04:47 | |
| *** nurdie has joined #openstack-ansible | 04:52 | |
| *** nurdie has quit IRC | 04:56 | |
| *** cshen has joined #openstack-ansible | 05:07 | |
| *** nurdie has joined #openstack-ansible | 05:22 | |
| *** nicolasbock has joined #openstack-ansible | 05:23 | |
| *** cshen has quit IRC | 05:27 | |
| *** nurdie has quit IRC | 05:27 | |
| *** cshen has joined #openstack-ansible | 05:28 | |
| *** irclogbot_2 has quit IRC | 05:34 | |
| *** irclogbot_0 has joined #openstack-ansible | 05:37 | |
| *** pcaruana has joined #openstack-ansible | 05:59 | |
| rohit02 | hi team .....how we can cinder volume backend both ceph and lvm in stein......getting error for lvm "lvm driver not support active active configuration" | 06:01 |
|---|---|---|
| rohit02 | plzz help me if any one having idea | 06:04 |
| *** kimhindart has joined #openstack-ansible | 06:10 | |
| *** radeks has joined #openstack-ansible | 06:16 | |
| *** radeks has quit IRC | 06:18 | |
| *** radeks has joined #openstack-ansible | 06:19 | |
| *** cshen has quit IRC | 06:31 | |
| *** gokhani has joined #openstack-ansible | 06:54 | |
| jrosser | rohit02: I don’t think the lvm driver can do active/active | 06:55 |
| jrosser | can you explain a bit why you need both? | 06:55 |
| *** rohit02 has quit IRC | 06:57 | |
| *** rohit02 has joined #openstack-ansible | 06:57 | |
| *** waxfire has joined #openstack-ansible | 07:08 | |
| *** hwoarang has quit IRC | 07:12 | |
| *** hwoarang has joined #openstack-ansible | 07:16 | |
| *** jawad_axd has joined #openstack-ansible | 07:17 | |
| *** fungi has quit IRC | 07:20 | |
| *** kimhindart has quit IRC | 07:21 | |
| *** nurdie has joined #openstack-ansible | 07:23 | |
| *** nicolasbock has quit IRC | 07:24 | |
| *** cshen has joined #openstack-ansible | 07:24 | |
| *** nurdie has quit IRC | 07:28 | |
| *** shyamb has joined #openstack-ansible | 07:32 | |
| *** luksky has joined #openstack-ansible | 07:34 | |
| *** shyamb has quit IRC | 07:44 | |
| *** nicolasbock has joined #openstack-ansible | 07:48 | |
| *** yolanda has quit IRC | 07:55 | |
| *** int-0x21 has joined #openstack-ansible | 07:55 | |
| *** nicolasbock has quit IRC | 07:58 | |
| *** nicolasbock has joined #openstack-ansible | 07:58 | |
| *** kopecmartin|pto is now known as kopecmartin|sick | 08:05 | |
| *** nicolasbock has quit IRC | 08:07 | |
| *** cshen has quit IRC | 08:08 | |
| *** tosky has joined #openstack-ansible | 08:10 | |
| *** yolanda has joined #openstack-ansible | 08:11 | |
| *** nicolasbock has joined #openstack-ansible | 08:16 | |
| *** nicolasbock1 has joined #openstack-ansible | 08:19 | |
| *** kimhindart has joined #openstack-ansible | 08:19 | |
| *** nicolasbock has quit IRC | 08:20 | |
| *** arxcruz|off is now known as arxcruz | 08:21 | |
| *** nicolasbock1 has quit IRC | 08:24 | |
| *** nicolasbock has joined #openstack-ansible | 08:25 | |
| openstackgerrit | kourosh vivan proposed openstack/openstack-ansible master: Fix requirements_git_url call https://review.opendev.org/693011 | 08:27 |
| *** nicolasbock has quit IRC | 08:34 | |
| openstackgerrit | kourosh vivan proposed openstack/openstack-ansible-os_keystone master: Add possibility to overwrite public repo https://review.opendev.org/692872 | 08:38 |
| *** kimhindart has quit IRC | 09:03 | |
| *** kimhindart has joined #openstack-ansible | 09:06 | |
| *** mathlin has quit IRC | 09:12 | |
| *** kimhin has joined #openstack-ansible | 09:23 | |
| *** miloa has joined #openstack-ansible | 09:23 | |
| *** nurdie has joined #openstack-ansible | 09:24 | |
| *** kimhindart has quit IRC | 09:26 | |
| *** kimhin has quit IRC | 09:28 | |
| *** nurdie has quit IRC | 09:29 | |
| miloa | Morning | 09:30 |
| int-0x21 | Morning | 09:37 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance stable/train: Drop common-db tag from db_sync task https://review.opendev.org/693146 | 09:42 |
| jrosser | morning | 09:47 |
| *** admin0 has joined #openstack-ansible | 09:47 | |
| *** kimhindart has joined #openstack-ansible | 09:59 | |
| *** kimhindart has quit IRC | 10:02 | |
| *** DanyC has joined #openstack-ansible | 10:08 | |
| *** DanyC has joined #openstack-ansible | 10:09 | |
| *** DanyC has quit IRC | 10:10 | |
| *** ianychoi has quit IRC | 10:13 | |
| *** DanyC has joined #openstack-ansible | 10:14 | |
| *** DanyC has quit IRC | 10:42 | |
| *** luksky has quit IRC | 10:56 | |
| *** sep has quit IRC | 10:59 | |
| *** sep has joined #openstack-ansible | 11:02 | |
| *** DanyC has joined #openstack-ansible | 11:03 | |
| *** noonedeadpunk[h] has joined #openstack-ansible | 11:24 | |
| miloa | On stable/rocky I've got a problem with panels, horizon_install.yml install all packages without filtering if panel is set to true or false, as packages install panels in "enabled" repo the panels are by default present, althrougt the py file of panel is removed from repo "enabled" because pyc and pyo are still present. Perhaps it would be better to use package install and remove in place of the | 11:25 |
| miloa | actual panel enabled task in horizon_post_install.yml for distrib install method. | 11:25 |
| *** nurdie has joined #openstack-ansible | 11:25 | |
| fresta | Is there documentation anywhere on how to best configure the public/floating network? I would like to use the same interface and subnet for both api stuff and floating ips. | 11:27 |
| *** noonedeadpunk[h] has quit IRC | 11:29 | |
| *** nurdie has quit IRC | 11:30 | |
| jrosser | fresta: you can configure the public network any way you like, there isn't really a definition of "best" | 11:31 |
| jrosser | however i would say that sharing the subnet for floating IP and API stuff is only really something that should be done in a lab environment | 11:31 |
| jrosser | certainly never anywhere you care about the security of the openstack control plane | 11:32 |
| openstackgerrit | Andriy Shevchenko proposed openstack/openstack-ansible-os_blazar master: fix tox python3 overrides https://review.opendev.org/661545 | 11:35 |
| fresta | This is for a lab env so security is not that important. I created a bridge "br-public" and assigned a public ip for control plane api stuff. But when giving neutron that bridge as a flat network I get """Error: Can not enslave a bridge to a bridge.""" | 11:36 |
| fresta | jrosser: Maybe I shouldn't use a bridge for that? | 11:37 |
| jrosser | you cannot give a bridge to neutron like that | 11:37 |
| jrosser | it must be an interface | 11:37 |
| jrosser | fresta: i would read this https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/openstack_interface.cfg.prod.example#L108-L124 | 11:38 |
| jrosser | there is an example config there where the native (untagged) traffic on br-vlan is connected via a veth to a special interface eth12 | 11:39 |
| jrosser | eth12 is then specified to neutron as the physical interface (even though in this case its not physical) | 11:40 |
| jrosser | so if you really want to use br-mgmt also for the public network, you can do the same trick | 11:40 |
| *** noonedeadpunk[h] has joined #openstack-ansible | 11:41 | |
| jrosser | but make sure that you have specified the addresses you want to use for public/floating IP here https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.prod.example#L7-L11 | 11:41 |
| jrosser | otherwise the OSA dynamic inventory is likley to allocate container IP where you don't want them | 11:42 |
| noonedeadpunk[h] | mornings | 11:45 |
| *** luksky has joined #openstack-ansible | 11:45 | |
| fresta | jrosser: Aha that's clever. I was hoping i could do all network config with netplan but I don't think it supports veth (among a lot of other things). | 11:45 |
| fresta | But I will give this a try. Thanks a lot! | 11:45 |
| openstackgerrit | Andriy Shevchenko proposed x/monitorstack master: Update home-page https://review.opendev.org/630452 | 11:46 |
| jrosser | fresta: no worries - let us know if it works out | 11:47 |
| noonedeadpunk[h] | why is this ^ under our governance? | 11:49 |
| gshippey | noonedeadpunk[h]: can you regenerate the SHA bump for master, https://review.opendev.org/#/c/689649/1 this one is stale | 11:49 |
| noonedeadpunk[h] | gshippey: oh, yep, sure. I guess I've missed master yestarday | 11:50 |
| *** luksky has quit IRC | 11:56 | |
| openstackgerrit | Merged x/monitorstack master: Update home-page https://review.opendev.org/630452 | 12:00 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/693163 | 12:13 |
| *** rohit02 has quit IRC | 12:13 | |
| *** sshnaidm|off is now known as sshnaidm|bbl | 12:14 | |
| openstackgerrit | Mikael Loaec proposed openstack/openstack-ansible-os_horizon stable/rocky: [WIP]Fix panels enable/disable for distro install. https://review.opendev.org/692804 | 12:17 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/693163 | 12:17 |
| jrosser | noonedeadpunk: this is subtle error https://review.opendev.org/#/c/692872/5 | 12:29 |
| *** rohit02 has joined #openstack-ansible | 12:29 | |
| jrosser | i wonder if UTF-8 is slipping into our code un-noticed...... | 12:29 |
| *** ansmith_ has quit IRC | 12:33 | |
| noonedeadpunk[h] | jrosser: yeah, that may happen actually, but I'm not sure how to check that right now... I guess ansible-lint is not checking this | 12:34 |
| jrosser | noonedeadpunk[h]: i only found what was going on by cherry-picking the patch and pasting it here http://pygments.org/demo/6935167/ | 12:37 |
| jrosser | it nicely highlights the location of the warnings | 12:37 |
| noonedeadpunk[h] | I guess I have some extension for vs code for checking that - but it's nice tool | 12:39 |
| * noonedeadpunk[h] adds to bokkmarks | 12:39 | |
| noonedeadpunk[h] | I think we need to add some check to the linters check | 12:41 |
| noonedeadpunk[h] | But it's worth doing while migrating to the integrated linters check - to edit them in one place instead of every tox | 12:41 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Restart mysql when package is installed https://review.opendev.org/693043 | 12:43 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server stable/train: Restart mysql when package is installed https://review.opendev.org/693172 | 12:44 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server stable/stein: Restart mysql when package is installed https://review.opendev.org/693173 | 12:44 |
| *** luksky has joined #openstack-ansible | 13:03 | |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server master: Ansible 'search' is a test not a filter https://review.opendev.org/693061 | 13:08 |
| guilhermesp | mornings | 13:08 |
| jrosser | o/ | 13:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server stable/train: Ansible 'search' is a test not a filter https://review.opendev.org/693177 | 13:13 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server stable/stein: Ansible 'search' is a test not a filter https://review.opendev.org/693178 | 13:14 |
| fresta | jrosser: I created a veth pair, connected one end to br-public and specified the other for neutron. Works perfectly. Thanks again! | 13:26 |
| jrosser | fresta: excellent | 13:29 |
| *** ansmith_ has joined #openstack-ansible | 13:33 | |
| *** nurdie has joined #openstack-ansible | 13:41 | |
| noonedeadpunk[h] | jrosser: so https://github.com/ansible/ansible/pull/64512 worked for my test environment. Wondering how master works with rabbitmq 3.8 now... | 13:47 |
| jrosser | did we bump rabbitmq to 3.8? | 13:49 |
| *** henryb has joined #openstack-ansible | 13:55 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest stable/queens: Don't download image when it's already there https://review.opendev.org/693182 | 14:09 |
| *** goldyfruit_ has joined #openstack-ansible | 14:10 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/train: Bump SHAs for stable/train https://review.opendev.org/693008 | 14:16 |
| openstackgerrit | Merged openstack/openstack-ansible stable/stein: Bump SHAs for stable/stein https://review.opendev.org/693002 | 14:16 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Bump SHAs for stable/pike https://review.opendev.org/689652 | 14:16 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Use the cached cirros image for tests run from this repo https://review.opendev.org/693185 | 14:21 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance stable/train: Drop common-db tag from db_sync task https://review.opendev.org/693146 | 14:22 |
| openstackgerrit | kourosh vivan proposed openstack/openstack-ansible-os_keystone master: Add possibility to overwrite public repo https://review.opendev.org/692872 | 14:24 |
| jrosser | int-0x21: ThiagoCMC: this is merged https://review.opendev.org/#/c/693008/, so stable/train of OSA should now have all the dependancies pointing correctly to Train repos | 14:25 |
| *** radeks has quit IRC | 14:27 | |
| jrosser | noonedeadpunk[h]: guilhermesp i dont understand why this is proceeding to the gate https://review.opendev.org/#/c/692607/ becasue it looks like the same change is already merged in the stable/train SHA bunp | 14:30 |
| noonedeadpunk[h] | I dunno actually, as it was placed before the bump | 14:31 |
| guilhermesp | hum i guess this needed a -1 before the stable/train sha bump? | 14:33 |
| guilhermesp | or maybe we just missed that | 14:36 |
| jrosser | perhaps - i was expecting to see a conflict or "cannot merge" | 14:36 |
| int-0x21 | jrosser, oh nice i will check it out | 14:36 |
| guilhermesp | yeah not sure... this was +w yesterday | 14:36 |
| int-0x21 | Been doing some testing with manual install of openstack to get a better feel for it and am back to ansible now when i have more control of what is actualy happening | 14:36 |
| int-0x21 | master still has the " no filter named 'ips_in_ranges' " for ceph mon it seems | 14:37 |
| *** rohit02 has quit IRC | 14:37 | |
| guilhermesp | jrosser: should we abandon or wait zu to execute? it already failed one job so, it's not gonna merge | 14:38 |
| guilhermesp | zuul* | 14:38 |
| *** goldyfruit___ has joined #openstack-ansible | 14:38 | |
| *** rohit02 has joined #openstack-ansible | 14:38 | |
| *** goldyfruit_ has quit IRC | 14:40 | |
| *** sshnaidm|bbl is now known as sshnaidm|rover | 14:41 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Fix typos in the yml files https://review.opendev.org/692956 | 14:45 |
| int-0x21 | Putting ipaddrs_in_ranges.py into /opt/openstack-ansible/playbooks/filter_plugins/ solved the ceph_mon issue | 14:51 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_container_create master: Fix typos in the yml files https://review.opendev.org/692773 | 14:51 |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install master: Update the source link to the lastest one https://review.opendev.org/692776 | 14:52 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server master: Fix the format of the bugs link https://review.opendev.org/692774 | 14:55 |
| noonedeadpunk[h] | guilhermesp: jrosser was right, https://review.opendev.org/#/c/692607/1 is not needed as this sha is laready merged. https://opendev.org/openstack/openstack-ansible/src/branch/stable/train/ansible-role-requirements.yml#L219 | 14:58 |
| *** ansmith has joined #openstack-ansible | 15:01 | |
| *** sum12 has left #openstack-ansible | 15:02 | |
| *** ansmith_ has quit IRC | 15:03 | |
| *** ansmith has quit IRC | 15:31 | |
| *** ansmith has joined #openstack-ansible | 15:31 | |
| *** rohit02 has quit IRC | 15:33 | |
| *** rohit02 has joined #openstack-ansible | 15:34 | |
| *** prometheanfire has quit IRC | 15:38 | |
| *** prometheanfire has joined #openstack-ansible | 15:39 | |
| *** rohit02 has quit IRC | 15:39 | |
| *** gyee has joined #openstack-ansible | 15:46 | |
| openstackgerrit | Mikael Loaec proposed openstack/openstack-ansible-os_horizon stable/rocky: [WIP]Fix panels enable/disable for distro install. https://review.opendev.org/692804 | 15:46 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: [DNM] test patch https://review.opendev.org/693203 | 15:48 |
| ThiagoCMC | Hey guys, todays bootstrap-ansible.sh (stable/train) is failing, two or three errors: git openstack-ansible-os_placement fatail: reference is not a tree, same with git ansible-role-uwsgi, then, in the end, the script can't find "/var/lib/lxc" withn the deployment host but, no containers is supposed to be running at the deployment server. | 15:58 |
| *** ansmith has quit IRC | 16:00 | |
| *** noonedeadpunk[h] has quit IRC | 16:00 | |
| *** macz has joined #openstack-ansible | 16:04 | |
| jrosser | dirk: i see this on two different CI providers now https://zuul.opendev.org/t/openstack/build/629e4415c3d94bb8a46c03a5747c2051/log/job-output.txt#13213 | 16:12 |
| jrosser | maybe an upstream repo issue there? | 16:12 |
| *** cshen has joined #openstack-ansible | 16:22 | |
| *** ansmith has joined #openstack-ansible | 16:22 | |
| *** DanyC has quit IRC | 16:32 | |
| *** ivve has quit IRC | 16:34 | |
| jrosser | ThiagoCMC: i'm not sure what is going on for you - i have just bootstrap-ansible on a VM and all the repos clone OK? http://paste.openstack.org/show/785859/ | 16:45 |
| *** miloa has quit IRC | 16:52 | |
| *** DanyC has joined #openstack-ansible | 17:07 | |
| *** DanyC has quit IRC | 17:12 | |
| guilhermesp | ok so i think then we can proceed to abandon 692607 jrosser noonedeadpunk | 17:16 |
| jrosser | yes i think so | 17:16 |
| jrosser | looks like we need some help from our suse friends.... * seems broken | 17:17 |
| guilhermesp | how's more susish nowadays ? | 17:17 |
| *** spatel has joined #openstack-ansible | 17:21 | |
| spatel | I hate when you running playbooks and you found its breaking because some remote fedora repo is down | 17:22 |
| spatel | is there any good way to make it failover to next repo or try other ? | 17:22 |
| *** jawad_axd has quit IRC | 17:23 | |
| *** andrea15 has joined #openstack-ansible | 17:26 | |
| *** jawad_axd has joined #openstack-ansible | 17:29 | |
| *** melwitt is now known as jgwentworth | 17:31 | |
| *** jawad_axd has quit IRC | 17:32 | |
| *** DanyC has joined #openstack-ansible | 17:36 | |
| ThiagoCMC | jrosser: which branch are you on? | 17:36 |
| jrosser | i made new VM and checked out stable/train just an hour or two ago | 17:37 |
| ThiagoCMC | Here is my output: https://pastebin.com/WNUCjiQE - error at lines 6, 35 and 57 until almost the end... | 17:39 |
| spatel | why running openstack-ansible setup-hosts.yml --limit infra01 doing nothing | 17:41 |
| jrosser | did you start from fresh or is this a git fetch / git checkout stable/train / ./scripts/bootstrap-ansible.sh on top of a previous deployment | 17:41 |
| spatel | but if i run "openstack-ansible setup-hosts.yml" only it works | 17:41 |
| ThiagoCMC | jrosser: it's not 100% fresh, I'll rebuild the OSA Deployment Server tonigh and try again. | 17:42 |
| jrosser | a quick woraround would be to just delete the things that are failing from /etc/ansible/roles and re-bootstrap | 17:42 |
| guilhermesp | hum i see ThiagoCMC is already using the recent merged bump shas https://review.opendev.org/#/c/693008/ | 17:44 |
| guilhermesp | yeah, i think maybe deleting the roles and let the bootstrap script clone then again might work | 17:44 |
| *** cshen has quit IRC | 17:45 | |
| guilhermesp | failing roles to be more precise | 17:45 |
| ThiagoCMC | Hmm... got it! I'll try that now. | 17:46 |
| jrosser | guilhermesp: i wish i understood why this happens | 17:46 |
| jrosser | sometimes we bump the SHA and the exact same thing happens in the CI jobs | 17:47 |
| jrosser | usually for external repos like gnocchi so i feel like it is something to do with the way the git history is structured | 17:47 |
| guilhermesp | when we change a sha, or overide it, we expect the bootstrap-script to re-clone it right? or checkout it to the specified sha | 17:48 |
| jrosser | ultimately it's this https://github.com/openstack/openstack-ansible/blob/master/scripts/get-ansible-role-requirements.yml#L98-L111 | 17:49 |
| guilhermesp | ok so might be a module issue or a git issue itself. | 17:51 |
| jrosser | so yes, we expect bootstrap-ansible to get the /etc/ansible/roles/* to the correct state regardless of if it's fresh or an upgrade | 17:51 |
| jrosser | absolutely yes | 17:51 |
| ThiagoCMC | AHA! The "rm -rf /etc/ansible/roles/os_placement and uwsgi fixed the issue! | 17:51 |
| jrosser | guilhermesp: ^ so there i guess we know that there are cases where it works for fresh clones but not for moving an existing role forward to a new SHA | 17:52 |
| jrosser | but only for some roles, some times :( | 17:52 |
| guilhermesp | that's the only clue we have | 17:52 |
| guilhermesp | "stderr": "fatal: reference is not a tree: c42b0d95ff4ab8c4ca00404df04f2eaaff431c9c\n", "stderr_lines": ["fatal: reference is not a tree: c42b0d95ff4ab8c4ca00404df04f2eaaff431c9c"], "stdout": "", "stdout_lines": []} | 17:52 |
| guilhermesp | os_placement example ^ | 17:53 |
| jrosser | i think you get the same error if after finding one of those you cd to the directory and git show <SHA> | 17:53 |
| jrosser | but it would be good to confirm that | 17:53 |
| guilhermesp | ThiagoCMC: do you still have a failing repo or did you rm -rf all failing roles? | 17:54 |
| ThiagoCMC | I did the "rm -rf /etc/ansible/roles/os_placement /etc/ansible/roles/uwsgi" (the failing roles) and the bootstrap worked! | 17:55 |
| ThiagoCMC | I'm also seeeing something related to Python3 not being in use: "Not overwriting existing python script /opt/ansible-runtime/bin/python (you must use /opt/ansible-runtime/bin/python3)" - can I ignore that? =) | 17:56 |
| ThiagoCMC | Yeah, never mind, the python is a symlink to python3, sorry about the buzz. | 17:57 |
| ThiagoCMC | For some reason, stable/train is creating the neutron_server_container-xxx within the Network Nodes (where I only expect the neutron-*-agents as metal). It's ignoring my /etc/openstack_deploy/env,d/neutron.yml file. I tried jrosser suggestions, no effect. | 18:02 |
| ThiagoCMC | It works on stable/stein! | 18:02 |
| guilhermesp | jrosser: that's the suse issues you were mentioning https://zuul.opendev.org/t/openstack/build/55e278d3b56143c79c2049a283301a39/log/job-output.txt#3981 ? | 18:09 |
| jrosser | I saw a whole bunch of failing to install packages from the suse repo / mirror | 18:10 |
| d34dh0r53 | wrt to mnaio, people are still using it but I think they have their own workarounds for things, it's generally still usable but needs a little TLC | 18:23 |
| d34dh0r53 | jamesdenton, jrosser ^ | 18:24 |
| *** DanyC has quit IRC | 18:24 | |
| jrosser | d34dh0r53: always good to roll in fixed for those workarounds..... | 18:26 |
| jamesdenton | thanks, d34dh0r53. i had every intention of getting to it today, but real work™ got in the way | 18:35 |
| d34dh0r53 | :) no worries, while you're both here, any objections to me making 18.04 the default image for the vms? | 18:36 |
| *** jgwentworth is now known as melwitt | 18:36 | |
| jamesdenton | i demand it | 18:36 |
| jamesdenton | :D | 18:36 |
| d34dh0r53 | lol, excellent | 18:36 |
| cjloader | jamesdenton: d34dh0r53 o/ | 18:39 |
| d34dh0r53 | howdy cjloader | 18:39 |
| jamesdenton | hi there | 18:39 |
| *** tosky has quit IRC | 18:48 | |
| *** int-0x21 has quit IRC | 18:52 | |
| *** strattao has quit IRC | 19:15 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 19:27 | |
| *** spatel has quit IRC | 19:28 | |
| *** ivve has joined #openstack-ansible | 19:51 | |
| *** nurdie has quit IRC | 20:01 | |
| *** nurdie has joined #openstack-ansible | 20:02 | |
| *** ansmith has quit IRC | 20:13 | |
| *** nurdie has quit IRC | 20:19 | |
| *** nurdie has joined #openstack-ansible | 20:19 | |
| *** nurdie has quit IRC | 20:24 | |
| *** spatel has joined #openstack-ansible | 20:31 | |
| spatel | any idea what is this error for | 20:31 |
| -spatel- fatal: [ostack-eng-infra-1-2_keystone_container-0889527d -> 172.28.17.12]: FAILED! => { | 20:31 | |
| -spatel- "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", | 20:31 | |
| spatel | jrosser: any idea? its happening during "Create DB for services" task | 20:35 |
| jrosser | i would be setting the no_log to false otherwise you'll never see the underlying error | 20:36 |
| spatel | jrosser: i can see error unable to connect database, check your password or account | 20:44 |
| spatel | jrosser: does galera create services accounts or keystone going to create service account? | 20:45 |
| spatel | jrosser: this is the error i am getting http://paste.openstack.org/show/785866/ | 20:48 |
| spatel | what is this "reading initial communication packet" ? | 20:48 |
| jrosser | maybe first check that 172.28.17.12 is the host you expect that task to be delegated to | 20:53 |
| jrosser | and also if you expect this "login_host": "172.28.16.9", | 20:54 |
| spatel | i can see mysql is up and all 3 cluster nodes registerd | 20:54 |
| spatel | 172.28.16.9 is my HAproxy VIP (internal) | 20:54 |
| spatel | do i need to tell mysql to allow host access from 172.28.16.9 ? | 20:54 |
| jrosser | if that is your VIP then that should be done already, i guess | 20:57 |
| jrosser | spatel: so for example, from my utility container i can use the mysql cli to interact with the database | 20:58 |
| spatel | let me try from my util box | 20:59 |
| jrosser | and if this works that will prove you can access the db via the VIP | 20:59 |
| -spatel- $ mysql -h 172.28.16.9 -u root | 20:59 | |
| -spatel- ERROR 2013 (HY000): Lost connection to MySQL server at 'handshake: reading inital communication packet', system error: 11 | 20:59 | |
| spatel | look like something is wrong | 21:00 |
| spatel | I am running HAproxy on Infra nodes* | 21:00 |
| spatel | infra-1/2 | 21:00 |
| spatel | I am running tcpdump on mysql and not seeing any request coming on it from LB | 21:05 |
| spatel | i believe something is wrong with LB | 21:05 |
| jrosser | looks like it, yes | 21:05 |
| spatel | crap!! haproxy saying galera-back down | 21:08 |
| spatel | but i can see its up and i can access from HAproxy node | 21:09 |
| spatel | let me poke... | 21:09 |
| spatel | from infra node i can do "telnet 172.28.17.12 3306" | 21:10 |
| spatel | what is this? | 21:12 |
| spatel | server ostack-eng-infra-1-2_galera_container-d11cc5d0 172.28.17.12:3306 check port 9200 inter 12000 rise 1 fall 1 | 21:12 |
| spatel | why saying check port 9200 ? | 21:12 |
| *** nurdie has joined #openstack-ansible | 21:13 | |
| spatel | jrosser: ^^ | 21:13 |
| ioni | spatel, that's xinetd from galera | 21:15 |
| ioni | it runs in galera container | 21:15 |
| ioni | ssh into galera container and check if is running | 21:16 |
| *** goldyfruit___ has quit IRC | 21:16 | |
| ioni | if is running, then look at /etc/xinetd.d/mysqlchk | 21:16 |
| ioni | maybe only_from wasn't detected correctly | 21:16 |
| spatel | ioni: you are right | 21:17 |
| spatel | only_from = 172.28.16.7 172.28.16.8 172.28.16.12 172.28.16.13 172.28.16.14 172.28.1.216 172.28.15.166 172.28.15.69 127.0.0.1 | 21:17 |
| spatel | i can't see HAproxy IP address in this list | 21:17 |
| *** nurdie has quit IRC | 21:18 | |
| ioni | i used to have galera_monitoring_allowed_source: "0.0.0.0/0" in user_variables.yml | 21:18 |
| ioni | to avoid this issue in the past | 21:18 |
| spatel | look like... i have to do that | 21:18 |
| spatel | ioni: thanks for point that out | 21:19 |
| ioni | spatel, your welcome my friend :) | 21:19 |
| jrosser | this ought to be default? https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/galera_all.yml#L35 | 21:23 |
| *** gshippey has quit IRC | 21:27 | |
| *** pcaruana has quit IRC | 21:27 | |
| spatel | ioni: do you know how to run specific tag to update xinited ? | 21:28 |
| spatel | i don't want to run whole playbook to just update xinted :) | 21:29 |
| spatel | jrosser: Do you know? | 21:29 |
| ioni | spatel, not really, i don't really use tags seens most of the time, when i try, they don't work as expected | 21:29 |
| spatel | is it safe to run galere playbook ? | 21:30 |
| spatel | ioni: ^ | 21:32 |
| ioni | or you can use edit the file manually on couples of containers :D | 21:32 |
| spatel | ioni: i want to see what happen :) | 21:33 |
| jrosser | also depends how much you want to find why it is broken | 21:33 |
| jrosser | this could be just the first of a few things that somehow doesnt get the VIP address properly | 21:33 |
| *** nurdie has joined #openstack-ansible | 21:34 | |
| *** nurdie has quit IRC | 21:35 | |
| *** nurdie has joined #openstack-ansible | 21:36 | |
| spatel | jrosser: i thought playbook will be smart enough to add HAproxy IPs but it didn't :( | 21:36 |
| jrosser | right, but it should | 21:36 |
| jrosser | and i linked you to the group_vars which will set it | 21:36 |
| jrosser | so its a judgement, either its just broken, which is unlikley as there are lots of multinode deploys out there working, or there is something in your environment/vars causing this | 21:37 |
| jrosser | or a corner case which we need to fix | 21:38 |
| spatel | Oh yess.. interesting.. | 21:38 |
| jrosser | just adding a debug: var: galera_monitoring_allowed_source | 21:39 |
| spatel | you know what i have "galera_monitoring_allowed_source:" in user_variable.yml may be i am overwriting stuff :) | 21:39 |
| jrosser | followed by fail: task into the galera playbook would be reasonable way to see whats happening | 21:39 |
| jrosser | ahha! | 21:39 |
| spatel | damn it... that is the issue | 21:40 |
| jrosser | so for things like this i always first go to codesearch.openstack.org | 21:40 |
| spatel | i wish no matter what you have in user_variable.yml it should force add haproxy group | 21:40 |
| jrosser | and put the var name in, and straight away up pops everywhere it is defined / used | 21:40 |
| *** nurdie has quit IRC | 21:40 | |
| spatel | jrosser: i love it ----> codesearch.openstack.org | 21:41 |
| spatel | is this dedicated for openstack-ansible or whole universe ? | 21:42 |
| jrosser | openstack everything | 21:42 |
| spatel | its very handy though :) i can see how to troubleshoot now | 21:42 |
| jrosser | it is this https://github.com/hound-search/hound | 21:43 |
| spatel | nice jquery :) | 21:44 |
| spatel | jrosser: Do you guys using Cell_v2 ? | 21:50 |
| spatel | I have zero experience with cell deployment that is why building 3rd isolated cluster :( | 21:50 |
| spatel | I hate doing.. because it required lots of work | 21:51 |
| jrosser | no we don't do that | 21:52 |
| spatel | how do you running multiple cloud? | 21:59 |
| *** henryb has quit IRC | 22:05 | |
| *** tosky has joined #openstack-ansible | 22:10 | |
| *** spatel has quit IRC | 22:36 | |
| *** nicolasbock has joined #openstack-ansible | 22:47 | |
| *** nicolasbock has quit IRC | 23:00 | |
| *** nicolasbock has joined #openstack-ansible | 23:01 | |
| *** nicolasbock has quit IRC | 23:12 | |
| *** nicolasbock has joined #openstack-ansible | 23:21 | |
| *** nicolasbock has quit IRC | 23:21 | |
| *** nicolasbock has joined #openstack-ansible | 23:21 | |
| *** ansmith has joined #openstack-ansible | 23:24 | |
| *** nicolasbock has quit IRC | 23:31 | |
| *** nicolasbock has joined #openstack-ansible | 23:32 | |
| *** nicolasbock has quit IRC | 23:36 | |
| *** tosky has quit IRC | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!