| *** waxfire has quit IRC | 00:00 | |
| *** waxfire has joined #openstack-ansible | 00:00 | |
| *** spatel has joined #openstack-ansible | 01:17 | |
| *** spatel has quit IRC | 01:21 | |
| *** spatel has joined #openstack-ansible | 02:22 | |
| *** nurdie has quit IRC | 03:37 | |
| *** nurdie has joined #openstack-ansible | 03:43 | |
| *** nurdie has quit IRC | 03:57 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-ansible | 05:33 | |
| *** spatel has quit IRC | 05:47 | |
| *** jawad_axd has joined #openstack-ansible | 06:48 | |
| *** jawad_axd has quit IRC | 06:53 | |
| noonedeadpunk | admin0: it feels like some networking issue between rabbit containers | 09:47 |
|---|---|---|
| noonedeadpunk | the main issue here `Mnesia could not connect to any nodes.` which is stdout from rabbit itself | 09:47 |
| admin0 | noonedeadpunk, i have done iperf ( got 9.3gbit) . tcpdumps .. i fail to find any issues | 09:47 |
| noonedeadpunk | and you would need to re-run rabbitmq_install with `-e rabbitmq_upgrade=true` | 09:47 |
| noonedeadpunk | I think it connects by hostname - have you checked that you have valid hosts file there? | 09:48 |
| admin0 | yes .. from all 3 containers, i can ping all other 2 containers by its hostname | 09:49 |
| admin0 | the ips and hostname checks out | 09:49 |
| admin0 | i followed this also manually .. https://gist.github.com/niedbalski/69a72103adad4f0f9609a0857c9810a4 | 09:49 |
| admin0 | but stuck in the same situation | 09:50 |
| admin0 | i validated the .erlang secret using md5sum .. they are also in sync | 09:52 |
| noonedeadpunk | I'm wondering if smth is wrong with rabbitmq version you got. but we are bumping it to specific version to get the same one each run... | 09:52 |
| admin0 | even re-formatted the controllers, and destroyed and re-initialized the whole setup completely to give it a new name/ip | 09:53 |
| admin0 | and everything else works just fine .. (haproxy, galera) .. but always stuck in this rabbitmq | 09:53 |
| admin0 | in the same time, while being stuck at this, i have setup the same exact ( copied the inventory ) to set it up in 4 differnt places and locations and they work fine | 09:54 |
| admin0 | just this one specific build is puzzeling | 09:54 |
| noonedeadpunk | I think at this point you should have rabbitmq running on c1? can you telnet to it's port from c2 rabbit contianer? | 09:54 |
| admin0 | i am able to telnet from c2 -> c1 | 09:58 |
| admin0 | and also from c3 -> c1 | 09:58 |
| admin0 | i get a connected msg | 09:58 |
| noonedeadpunk | Well, I'm not sure what to suggest... Maybe trying to do tcpdump while running `rabbitmqctl join_cluster "rabbit@c1-rabbit-mq-container-e1d2ff91"` or enable debug logs https://www.rabbitmq.com/logging.html#debug-logging | 10:01 |
| admin0 | i did the tcpdump . it shows the ports connecting and getting response .. ( only that part0 | 10:02 |
| admin0 | i did not tried to dump that traffic | 10:02 |
| noonedeadpunk | it would be encrypted anyway | 10:14 |
| noonedeadpunk | I'm wondering about MTU?:) | 10:16 |
| noonedeadpunk | as all networking might work until it got encrypted because of mtu.... | 10:17 |
| admin0 | i just realized something | 10:29 |
| admin0 | the baseip was set to 10.0.0.0/22 range .. while our lxdbr0 is at 10.0.3.0/24 which overlaps | 10:29 |
| admin0 | changing all to 10.100 and retrying | 10:29 |
| admin0 | but it would surprise me that on setup-infra, eveything else will work except rabbit | 10:30 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/ussuri: Move openstack-ansible-uw_apache centos job to centos-8 https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/765928 | 11:25 |
| *** tosky has joined #openstack-ansible | 11:48 | |
| admin0 | noonedeadpunk, thanks for the pointers and helping | 11:57 |
| admin0 | changing that conflicting ip worked | 11:57 |
| openstackgerrit | Merged openstack/openstack-ansible master: Bump OSA for RC1 https://review.opendev.org/c/openstack/openstack-ansible/+/766957 | 12:08 |
| noonedeadpunk | admin0: oh, I'd never thought about that! | 12:48 |
| *** spatel has joined #openstack-ansible | 12:50 | |
| *** spatel has quit IRC | 12:54 | |
| admin0 | funny to me is only rabbit failed on the clustering step .. rest all of them just worked fine | 12:57 |
| admin0 | how to enable (glance) image upload via horizon | 13:05 |
| admin0 | i know we have to do some variable, but i don't recall that variable | 13:05 |
| noonedeadpunk | `horizon_images_upload_mode: legacy` | 13:17 |
| admin0 | thanks noonedeadpunk | 13:22 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Continue Wallaby release development https://review.opendev.org/c/openstack/openstack-ansible/+/768016 | 15:07 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Continue Wallaby release development https://review.opendev.org/c/openstack/openstack-ansible/+/768016 | 15:16 |
| *** spatel has joined #openstack-ansible | 15:41 | |
| *** spatel has quit IRC | 15:47 | |
| *** jawad_axd has joined #openstack-ansible | 16:17 | |
| *** spatel has joined #openstack-ansible | 17:49 | |
| *** spatel has quit IRC | 17:54 | |
| *** lemko4 has joined #openstack-ansible | 18:20 | |
| *** lemko has quit IRC | 18:20 | |
| *** lemko4 is now known as lemko | 18:20 | |
| *** nurdie has joined #openstack-ansible | 19:19 | |
| *** macz_ has joined #openstack-ansible | 20:29 | |
| *** mrda has quit IRC | 21:11 | |
| *** jawad_axd has quit IRC | 21:13 | |
| *** jawad_axd has joined #openstack-ansible | 21:15 | |
| *** jawad_axd has quit IRC | 21:20 | |
| *** mrda has joined #openstack-ansible | 21:21 | |
| *** ianychoi_ has joined #openstack-ansible | 21:30 | |
| *** ianychoi has quit IRC | 21:33 | |
| *** jawad_axd has joined #openstack-ansible | 21:51 | |
| *** jawad_axd has quit IRC | 21:56 | |
| *** jawad_axd has joined #openstack-ansible | 22:28 | |
| *** jawad_axd has quit IRC | 22:33 | |
| *** dave-mccowan has joined #openstack-ansible | 23:09 | |
| *** nurdie has quit IRC | 23:38 | |
| *** yann-kaelig has joined #openstack-ansible | 23:46 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!