| *** threestrands has joined #openstack-ansible | 00:05 | |
| *** threestrands has quit IRC | 00:05 | |
| *** threestrands has joined #openstack-ansible | 00:05 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/pike: Update filters to support over-commit https://review.openstack.org/516429 | 00:14 |
|---|---|---|
| *** chyka has quit IRC | 00:14 | |
| *** lbragstad has quit IRC | 00:28 | |
| *** savvas_ has quit IRC | 00:33 | |
| *** chyka has joined #openstack-ansible | 00:46 | |
| *** chyka has quit IRC | 00:50 | |
| *** newmember has quit IRC | 01:07 | |
| *** hw_wutianwei has joined #openstack-ansible | 01:16 | |
| *** japestinho has joined #openstack-ansible | 01:40 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Add dynamic table for our tested Scenarios https://review.openstack.org/520294 | 03:02 |
| openstackgerrit | Merged openstack/openstack-ansible master: Do not implement reload for novnc/spice service https://review.openstack.org/522526 | 03:04 |
| openstackgerrit | Guoqiang Ding proposed openstack/openstack-ansible-os_neutron master: Update the doc link https://review.openstack.org/522520 | 03:25 |
| openstackgerrit | Guoqiang Ding proposed openstack/openstack-ansible-os_neutron master: Update the doc link https://review.openstack.org/522520 | 03:31 |
| openstackgerrit | Guoqiang Ding proposed openstack/openstack-ansible-os_neutron master: Update the doc link for release notes https://review.openstack.org/522697 | 03:38 |
| cloudnull | any cores around mind giving this a tap through https://review.openstack.org/#/c/521331 | 03:58 |
| cloudnull | ThomasS: sorry there's a good lot of us off for the US holiday | 03:58 |
| cloudnull | ThomasS: what release is this ? | 03:59 |
| cloudnull | have you tried just rebuilding all of the rabbitmq containers? | 03:59 |
| cloudnull | `openstack-ansible lxc-container-destroy.yml lxc-container-create.yml rabbitmq-install.yml --limit shared-infra_hosts:rabbitmq_all` | 04:00 |
| cloudnull | is this a new deployment? | 04:01 |
| cloudnull | if troubleshooting, can you resolve the noted hostnames of all rabbitmq nodes within the rabbitmq container ? | 04:01 |
| * cloudnull running off again as I too am out for the holiday however ThomasS let us know how it goes. | 04:03 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Add nspawn container driver https://review.openstack.org/477017 | 04:06 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Modify LXC container create to JIT create https://review.openstack.org/490976 | 04:06 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Add nspawn container driver https://review.openstack.org/477017 | 04:06 |
| *** savvas_ has joined #openstack-ansible | 04:23 | |
| *** savvas_ has joined #openstack-ansible | 04:23 | |
| *** savvas_ has quit IRC | 04:27 | |
| *** udesale has joined #openstack-ansible | 04:52 | |
| *** germs1 has quit IRC | 04:59 | |
| *** germs has joined #openstack-ansible | 04:59 | |
| *** germs has quit IRC | 05:00 | |
| *** threestrands_ has joined #openstack-ansible | 05:02 | |
| openstackgerrit | Guoqiang Ding proposed openstack/openstack-ansible-os_horizon master: Update the doc theme https://review.openstack.org/519896 | 05:03 |
| *** threestrands has quit IRC | 05:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Do not implement reload for novnc/spice service https://review.openstack.org/522510 | 05:12 |
| *** threestrands_ has quit IRC | 05:14 | |
| *** udesale__ has joined #openstack-ansible | 05:32 | |
| *** udesale has quit IRC | 05:35 | |
| *** gkadam has joined #openstack-ansible | 05:38 | |
| *** gkadam has quit IRC | 05:50 | |
| *** gkadam has joined #openstack-ansible | 05:51 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Converge neutron agents onto Baremetal https://review.openstack.org/454450 | 06:08 |
| prometheanfire | go to bed | 06:14 |
| *** gkadam has quit IRC | 06:19 | |
| ThomasS | hi | 06:24 |
| ThomasS | i am still struggeling with the install | 06:25 |
| ThomasS | so far after running destroy-containers playbook i started over with setup-hosts | 06:26 |
| *** cjloader has joined #openstack-ansible | 06:26 | |
| ThomasS | then the apt repo was not working | 06:26 |
| ThomasS | i was not sure why so i started "troublehsooting" on infra1 | 06:27 |
| ThomasS | cleared iptables rules | 06:27 |
| ThomasS | killed dnsmasq | 06:27 |
| ThomasS | at the end it turned out that it was a config in apt.conf.d which pointed to the repo container which was gone | 06:27 |
| ThomasS | the i re-ran setup hosts which worked perfect | 06:28 |
| ThomasS | the i did setup-infra | 06:28 |
| ThomasS | containers got no IP on infra1 | 06:28 |
| ThomasS | i figured out that the dnsmasq was still killed and the iptables were missing | 06:29 |
| ThomasS | i did iptables-save / iptables-restore from infra2 to infra1 | 06:29 |
| ThomasS | and copied the commandline from dnsmasq from infra2 to infra1 and just strated it | 06:30 |
| prometheanfire | why not run setup-everything? | 06:34 |
| ThomasS | i am following the guide :-) | 06:34 |
| *** cjloader has quit IRC | 06:35 | |
| *** cjloader has joined #openstack-ansible | 06:36 | |
| ThomasS | problem is that all worked great expect the rabbit container on the first node said that there was a mismatch with hostnames | 06:36 |
| prometheanfire | iirc hostnames are not removed with lxc-container-delete | 06:37 |
| ThomasS | where can i set the internal container network? | 06:37 |
| ThomasS | yes hosts was still bloated :-) | 06:38 |
| prometheanfire | openstack-user-config.yml iirc | 06:38 |
| ThomasS | i guess which config option? | 06:39 |
| *** cjloader has quit IRC | 06:40 | |
| ThomasS | the container net is set | 06:40 |
| ThomasS | 10.0.100.101 | 06:41 |
| ThomasS | but the internal is 10.0.3.0 | 06:41 |
| ThomasS | which is in use | 06:41 |
| ThomasS | very strange did not find it | 06:44 |
| ThomasS | i think i found it | 06:45 |
| ThomasS | lxc_net_address: 10.0.3.1 | 06:45 |
| *** udesale has joined #openstack-ansible | 06:59 | |
| *** udesale__ has quit IRC | 07:00 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [WIP] Implement iptables management https://review.openstack.org/512099 | 07:01 |
| *** jvidal has joined #openstack-ansible | 07:02 | |
| ThomasS | mhh the new ip is not picked up yet and configured in the iptables | 07:03 |
| ThomasS | setup-hosts is responsible for this right? | 07:03 |
| ThomasS | also dnsmasq is still listening on 10.0.3.1 | 07:04 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [WIP] Implement iptables management https://review.openstack.org/512099 | 07:07 |
| ThomasS | ok it is all in the lxc_hosts role | 07:10 |
| ThomasS | when is it run? | 07:10 |
| ThomasS | can i enforce a re-config of the network? | 07:10 |
| ThomasS | also changed the dhcp range now | 07:10 |
| *** pcaruana has joined #openstack-ansible | 07:12 | |
| ThomasS | strange | 07:17 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [WIP] Implement iptables management https://review.openstack.org/512099 | 07:17 |
| ThomasS | the new lxc network config is not picked up even after running the lxc_hosts playbook | 07:18 |
| *** Taseer has quit IRC | 07:20 | |
| *** Taseer has joined #openstack-ansible | 07:20 | |
| ThomasS | maybe someone has a hint | 07:20 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Do not implement reload for novnc/spice service https://review.openstack.org/522731 | 07:29 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Do not implement reload for novnc/spice service https://review.openstack.org/522732 | 07:29 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Update the doc link https://review.openstack.org/522520 | 07:30 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Update the doc link for release notes https://review.openstack.org/522697 | 07:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/newton: Do not implement reload for novnc/spice service https://review.openstack.org/522734 | 07:36 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Do not implement reload for novnc/spice service https://review.openstack.org/522735 | 07:37 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/ocata: Do not implement reload for novnc/spice service https://review.openstack.org/522736 | 07:38 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/newton: Do not implement reload for novnc/spice service https://review.openstack.org/522737 | 07:39 |
| *** hamza has joined #openstack-ansible | 07:45 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar stable/ocata: Initial Zuul v3 role jobs https://review.openstack.org/514299 | 07:47 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally stable/pike: Constraint ansible installation https://review.openstack.org/522501 | 07:52 |
| openstackgerrit | Merged openstack/openstack-ansible master: Add infra healthchecks https://review.openstack.org/495492 | 07:53 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Adds CentOS 7 support https://review.openstack.org/522541 | 08:02 |
| *** michelv has joined #openstack-ansible | 08:03 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Update the doc theme https://review.openstack.org/519896 | 08:12 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_server master: Implement pypiserver and pypi proxy cache https://review.openstack.org/521754 | 08:15 |
| *** mbuil has joined #openstack-ansible | 08:17 | |
| *** armaan has joined #openstack-ansible | 08:31 | |
| *** gkadam has joined #openstack-ansible | 08:42 | |
| *** armaan has quit IRC | 08:49 | |
| *** mardim__ has joined #openstack-ansible | 08:52 | |
| ThomasS | so on setup infrastruture i get this error | 08:54 |
| ThomasS | https://pastebin.com/eV11zpfe | 08:54 |
| ThomasS | one of the 3 rabbit mq containers fails on the task Apply rabbitmq policies] | 08:55 |
| ThomasS | the host file on infra1 looks fine | 08:55 |
| openstackgerrit | Dimitrios Markou proposed openstack/openstack-ansible master: Add openvswitch_nsh group to inventory https://review.openstack.org/517269 | 08:55 |
| *** armaan has joined #openstack-ansible | 08:57 | |
| ThomasS | https://pastebin.com/BaZkY9vk | 08:57 |
| ThomasS | and the host file in the rabbit container looks like this | 08:57 |
| ThomasS | https://pastebin.com/6gTD1VAd | 08:58 |
| ThomasS | i am getting nuts :-( | 08:58 |
| *** armaan has quit IRC | 08:58 | |
| *** armaan has joined #openstack-ansible | 08:59 | |
| ThomasS | i am open for any hint | 08:59 |
| ThomasS | thnaks | 08:59 |
| evrardjp | what's the issue? | 09:12 |
| evrardjp | ThomasS: destroy the containers for rabbit, and re-build them, then reinstall rabbit. Should work :) | 09:15 |
| evrardjp | don't try to debug that hard, these are rabbitmq containers. | 09:16 |
| *** mrch has joined #openstack-ansible | 09:26 | |
| *** shardy has joined #openstack-ansible | 09:29 | |
| ThomasS | i destroyed all containers | 09:29 |
| ThomasS | and restaretd | 09:29 |
| ThomasS | :-( | 09:29 |
| ThomasS | so container-destroy - setup-hosts - setup-infra | 09:30 |
| ThomasS | https://pastebin.com/eV11zpfe | 09:30 |
| ThomasS | this looks very strange | 09:30 |
| *** holmsten has joined #openstack-ansible | 09:30 | |
| ThomasS | i am really getting nuts | 09:32 |
| ThomasS | the hosts file on the host infra1 also looks ok | 09:32 |
| ThomasS | https://pastebin.com/BaZkY9vk | 09:32 |
| ThomasS | and also in the container | 09:32 |
| ThomasS | https://pastebin.com/6gTD1VAd | 09:32 |
| ThomasS | destroying again and again will not fix it unfortunatly | 09:33 |
| *** Miouge has left #openstack-ansible | 09:33 | |
| *** Miouge has joined #openstack-ansible | 09:33 | |
| evrardjp | you can do more targetted plays, to avoid running the whole thing again and again | 09:34 |
| evrardjp | on top of it you can limit to rabbitmq | 09:34 |
| yifei | Does OSA support to configure the NTP server? I found that in OSA ocata configures different sources in the hosts in my environment. | 09:34 |
| yifei | Nov 24 01:19:16 host5 chronyd[32038]: Selected source 162.210.111.4 | 09:35 |
| yifei | Nov 24 01:19:20 host4 chronyd[32057]: Selected source 204.2.134.164 | 09:35 |
| evrardjp | ThomasS: I don't see what's wrong in your pastebin | 09:35 |
| evrardjp | ThomasS: could you paste your openstack_user_config and your user_* variables somewhere? | 09:35 |
| ThomasS | can do it, have destroyed the rabbit mq container and done setup-hosts -l infra1 and setup-infra -l infra1 -l infra1_rabbit_mq_container-315b2af6 | 09:36 |
| evrardjp | yifei: you should have an ntp indeed. I think the ansible-hardening role configures chronu | 09:36 |
| evrardjp | chrony* | 09:36 |
| ThomasS | sure | 09:36 |
| ThomasS | one second | 09:36 |
| yifei | evrardjp: Do you mean I need to configure NTP server in user_variables.yml? | 09:37 |
| evrardjp | yifei: I didn't say that | 09:37 |
| evrardjp | doesn't it work by default? | 09:37 |
| yifei | evrardjp: sorry, didn 't catch your idea | 09:38 |
| evrardjp | yifei: what is the issue? | 09:38 |
| yifei | evrardjp: the chrony source in different hosts are not the same | 09:38 |
| evrardjp | yifei: yes, you need a properly configured ntp | 09:38 |
| ThomasS | https://paste.ee/p/ib8Nc openstack_user_vars | 09:39 |
| yifei | evrardjp: do you know the variable name? | 09:39 |
| evrardjp | yifei: https://github.com/openstack/ansible-hardening/blob/master/templates/chrony.conf.j2#L22-L24 ? | 09:39 |
| evrardjp | yifei: https://github.com/openstack/ansible-hardening/blob/master/doc/source/getting-started.rst | 09:40 |
| ThomasS | user vars | 09:40 |
| ThomasS | https://paste.ee/p/c4Vjn | 09:40 |
| ThomasS | they are pretty much the example | 09:40 |
| yifei | evrardjp: Got it. Thanks a lot | 09:40 |
| evrardjp | yifei: so if you didn't follow the documentation and have overriden your own, you are probably using the defaults: | 09:40 |
| evrardjp | https://github.com/openstack/ansible-hardening/blob/master/defaults/main.yml#L265-L269 | 09:40 |
| evrardjp | yifei: it should be alright I think | 09:40 |
| openstackgerrit | Dimitrios Markou proposed openstack/openstack-ansible-os_neutron master: Add OvS-NSH support https://review.openstack.org/517259 | 09:41 |
| yifei | evrardjp: I did not do any configuration, but different sources came out | 09:41 |
| yifei | in my environment | 09:41 |
| evrardjp | yifei: yes, see my paste on defaults. | 09:41 |
| evrardjp | but you can override with what you need :) | 09:42 |
| openstackgerrit | Dimitrios Markou proposed openstack/openstack-ansible master: Add openvswitch_nsh group to inventory https://review.openstack.org/517269 | 09:43 |
| evrardjp | ThomasS: it looks ok. | 09:43 |
| ThomasS | thanks | 09:43 |
| evrardjp | ThomasS: so what is your issue? | 09:43 |
| ThomasS | the rabbit mq container tasks fail | 09:43 |
| evrardjp | could you post a log about the failure? | 09:44 |
| evrardjp | please re-run the rabbitmq playbook | 09:44 |
| ThomasS | first error onRead rabbit cookie] * | 09:44 |
| ThomasS | https://pastebin.com/eV11zpfe | 09:44 |
| evrardjp | openstack-ansible rabbitmq-install.yml | 09:44 |
| evrardjp | ThomasS: on first run that's normal | 09:44 |
| evrardjp | as you can see the task is rescued | 09:44 |
| ThomasS | yes looks like it | 09:45 |
| evrardjp | the deploys continue and doesn't show as failed. | 09:45 |
| ThomasS | but the hostname mismatch is the problem | 09:45 |
| evrardjp | the summary still shows failed=1 because ansible doesn't report rescued tasks yet | 09:45 |
| yifei | evrardjp: Thank you | 09:45 |
| ThomasS | yes but the second error stops the deploy | 09:45 |
| ThomasS | https://pastebin.com/eV11zpfe | 09:45 |
| evrardjp | so you say: | 09:46 |
| evrardjp | 1) you have the openstack_user_config file there, and the user_* variables | 09:46 |
| odyssey4me | yifei the sources for ntp use a pool, so yes they'll be different on every host unless you change the use of the ntp pools | 09:46 |
| ThomasS | yes | 09:47 |
| ThomasS | user_varaiables and secrets | 09:47 |
| evrardjp | 2) you destroyed your rabbitmq containers, re-created them, and re-ran the rabbitmq playbook | 09:47 |
| ThomasS | yes and also all containers | 09:47 |
| evrardjp | and you get an issue with a problematic hosts | 09:47 |
| ThomasS | so something basic is broken i guess? | 09:47 |
| yifei | odyssey4me: if i define "security_ntp_servers", then it will use the one I configured right? | 09:47 |
| odyssey4me | yifei these addresses (the defaults) are anycast: https://github.com/openstack/ansible-hardening/blob/422b793b804bcac6fa4059d70ce368c5beab9d5d/defaults/main.yml#L265-L269 | 09:48 |
| odyssey4me | yifei yes | 09:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Add ability to set the default index https://review.openstack.org/522761 | 09:48 |
| evrardjp | ThomasS: not on our side, it works everywhere. There is pretty much something wrong in your environment | 09:48 |
| evrardjp | are your /etc/hosts corrects on the physical nodes themselves? | 09:48 |
| ThomasS | i guess so but what? | 09:48 |
| ThomasS | for me it looks like it | 09:48 |
| ThomasS | here is the hosts file from infra1 | 09:48 |
| evrardjp | ThomasS: I don't know, I don't know what you do with your hosts :) | 09:49 |
| ThomasS | https://pastebin.com/BaZkY9vk | 09:49 |
| ThomasS | :-( | 09:49 |
| ThomasS | the hosts file in the problematic container looks like this | 09:49 |
| ThomasS | https://pastebin.com/6gTD1VAd | 09:49 |
| evrardjp | what's this L94 on your first hosts file? | 09:50 |
| yifei | odyssey4me: but in my another environment, using the default ones, it failed to create VM due to the clocks are not synchronized, will deep into it. Any way, thank you fist | 09:50 |
| ThomasS | L94? | 09:50 |
| ThomasS | paste error | 09:51 |
| ThomasS | the ( is not in the file | 09:51 |
| evrardjp | k | 09:51 |
| evrardjp | odyssey4me: if you have an idea on ThomasS issue, I have none. | 09:51 |
| evrardjp | ThomasS: which version are you running? | 09:52 |
| ThomasS | 16.0.3 | 09:52 |
| *** electrofelix has joined #openstack-ansible | 09:52 | |
| ThomasS | i messed up the hostfile the first run and then did a reinstall as odyssey4me adviced | 09:53 |
| ThomasS | and also stopped the stupid job to copy over the hosts file :-) | 09:53 |
| odyssey4me | evrardjp I've seen another report of the rabbit policy task failing too, but only one. And we aren't seeing it in the gates. I think it's a run_once task, which is why it only fails on one host. | 09:54 |
| evrardjp | I am deploying a 16.0.3 right now, to compare the hosts file. It will take a while. Please ping me back a little later. | 09:54 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Add ability to set the default index https://review.openstack.org/522761 | 09:54 |
| odyssey4me | But yes, it would seem that the cluster isn't ready to accept the policy being applied, so the start point would be to check the health of the cluster and to figure out why it's not ready | 09:55 |
| evrardjp | ThomasS: could you ping me in like 1h? | 09:55 |
| odyssey4me | previously there was a job overwriting the hosts file that was interfering, but apparently that's fixed now | 09:55 |
| ThomasS | i will try to | 09:55 |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/newton: Depth argument is prome to failure for ansible<2.3 https://review.openstack.org/522508 | 09:56 |
| evrardjp | ThomasS: in the meantime, maybe worth restarting manually, see what it gives. | 09:56 |
| ThomasS | yes and the rabbit task worked there! | 09:56 |
| odyssey4me | so I dunno - possibly best to dig into some rabbitmq logs to see what's going on with the cluster, and use google to help understand how to debug a failed rabbit cluster | 09:56 |
| evrardjp | ThomasS: I agree with odyssey4me there ^ | 09:56 |
| ThomasS | mhhh ok thanks | 09:56 |
| ThomasS | the strange think is that the first time with the broken hosts file worked fine | 09:57 |
| evrardjp | ThomasS: do you have inconsistencies between the host files and dns? | 09:57 |
| evrardjp | just wondering... | 09:58 |
| ThomasS | no not in this enviroment | 09:58 |
| ThomasS | the hosts use a public dns | 09:58 |
| evrardjp | logan-: is there an issue with your CI, or is there a neutron change that broke it? | 09:58 |
| odyssey4me | ThomasS rabbitmq is *very* sensitive to dns resolution issues, so it's *most* likely the cause | 09:58 |
| evrardjp | odyssey4me: thanks for the vote on https://review.openstack.org/#/c/522542/ | 09:58 |
| ThomasS | i guess so | 09:58 |
| odyssey4me | evrardjp oh sure - I thought it looked related to ansible 2.4 | 09:59 |
| evrardjp | :) | 09:59 |
| ThomasS | the hostfiles look correct or not? | 09:59 |
| evrardjp | yeah next commit is to test ansible 2.4. | 09:59 |
| ThomasS | so what can i try? | 09:59 |
| evrardjp | ThomasS: like odyssey4me said, manually connect to that node, trying to restart rabbit, checking the logs. | 09:59 |
| evrardjp | trying to see if there is no resolution issue in the container etc. | 10:00 |
| odyssey4me | ThomasS you'll need to debug the cluster start failure - or at least confirm whether the cluster is up | 10:00 |
| odyssey4me | this is going to be something you need to understand to run the cluster operationally anyway, so think of it as a learning experience | 10:00 |
| evrardjp | ThomasS: https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks/rabbitmq-maintain.html | 10:00 |
| evrardjp | might help you | 10:01 |
| *** armaan has quit IRC | 10:02 | |
| *** armaan has joined #openstack-ansible | 10:03 | |
| ThomasS | ok | 10:03 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_rally stable/newton: Updated from OpenStack Ansible Tests https://review.openstack.org/522764 | 10:03 |
| ThomasS | so at least only infra1 is online | 10:03 |
| ThomasS | why is it resolving 127.0.1.1? | 10:05 |
| ThomasS | that looks wired | 10:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally stable/newton: Constraint ansible installation https://review.openstack.org/522765 | 10:06 |
| ThomasS | other question | 10:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally stable/newton: Used cached git sources and enable depends-on https://review.openstack.org/518293 | 10:06 |
| ThomasS | how can re-confgure the lxc network and dnsmasq? | 10:07 |
| ThomasS | even the lxc_hosts playbook does not do it | 10:07 |
| ThomasS | wanted to change the internal ip range because 10.0.3.0 can lead to problems | 10:08 |
| ThomasS | changed vars in openstack_user_config | 10:08 |
| ThomasS | # lxc network #lxc_net_address: 172.16.16.1 #lxc_net_netmask: 255.255.255.0 #lxc_net_dhcp_range: 172.16.16.1,172.16.16.253 | 10:08 |
| ThomasS | commented them out because setup-hosts was not changing dnsmasq and iptables | 10:09 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Lighter ceph gates https://review.openstack.org/522766 | 10:09 |
| ThomasS | i think this is the root cause of the problem | 10:09 |
| ThomasS | because the containers resolve through 10.0.3.1 | 10:09 |
| ThomasS | and yesterday while debugging we killed the dnsmasq | 10:10 |
| ThomasS | and i "just" started it again by coping the commandline from from infra2 | 10:10 |
| ThomasS | so to sum it up, what is responsible for setting this up? | 10:11 |
| ThomasS | it looks not like setup-hosts playbook :-) | 10:11 |
| evrardjp | robcresswell: do you know if it's possible to test the dashboard plugin of project x without running project x? Like deploying the horizon fwaas plugin but not having configured fwaas in neutron at all. | 10:12 |
| evrardjp | I know it will probably burp when using it, but it shouldn't fail in the deploy part, right? | 10:12 |
| evrardjp | (and on top of it, how do we test that) | 10:12 |
| odyssey4me | evrardjp the horizon role does exactly that today as far as I know | 10:13 |
| evrardjp | oh you mean we are testing all our plugins directly there? | 10:13 |
| odyssey4me | it does a deployment, then tests just the login I think | 10:13 |
| odyssey4me | we could probably enable the plugins - it'll most likely work, at least in terms of convergence | 10:14 |
| ThomasS | any idea guys or where i can look? | 10:14 |
| evrardjp | odyssey4me: yeah, showing that the dashboard is there | 10:14 |
| evrardjp | is probably enough* | 10:14 |
| *** mardim__ has quit IRC | 10:14 | |
| evrardjp | odyssey4me: yeah my goal is to have a "dashboard" scenario, but still not overlap with the translations one. On top of it the translations is only a periodic | 10:14 |
| odyssey4me | ThomasS I'm not sure why you want to adjust those, but it can be set using the vars in https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/defaults/main.yml#L60-L78 as you have already discovered | 10:15 |
| ThomasS | ok and how do i reconfigure? | 10:16 |
| ThomasS | placed them in openstack_user_config | 10:16 |
| ThomasS | and then ran setup-hosts | 10:16 |
| odyssey4me | ThomasS make sure that your net range and dhcp range match, obviously, and also make sure that you don't have clashes between those and the ranges in your openstack_user_config | 10:16 |
| odyssey4me | ThomasS these should not be set in openstack_user_config - they should be set in user_variables | 10:17 |
| ThomasS | ok | 10:17 |
| ThomasS | and then which playbook das the reconfigure of the iptables masq rules and dnsmasq? | 10:17 |
| odyssey4me | all variable overrides except those that go into the inventory should be set in user_variables... if you put them into openstack_user_config then they become persistant in the inventory which is a hassle later | 10:17 |
| ThomasS | omk | 10:18 |
| ThomasS | thanks for clarifing this | 10:18 |
| odyssey4me | ThomasS these settings kinda need to be set right from the start - I'm not sure if reconfiguring a running environment will work very well | 10:18 |
| odyssey4me | you may have to destroy all the containers, change these, and run setup-hosts again | 10:18 |
| ThomasS | ok will do | 10:19 |
| ThomasS | and you are sure that dnsmasq and iptables are set? | 10:19 |
| ThomasS | because i tried this.... | 10:19 |
| odyssey4me | with all this mucking about of the foundaitonal elements, it might also be a good idea for you to rebuild the hosts you're working on | 10:19 |
| *** mardim__ has joined #openstack-ansible | 10:19 | |
| odyssey4me | networking configs go wrong, and stay broken unless you understand how to fix them properly... which requires understanding everything that's being done in these plays... and you're still learning | 10:20 |
| ThomasS | oh :-( | 10:20 |
| ThomasS | really? | 10:20 |
| ThomasS | openstack-ansible lxc-containers-destroy.yml | 10:21 |
| odyssey4me | so I would suggest rebuilding the hosts if you can - if not then you may have to work around the issue by doing a reboot after reconfiguring or something like that | 10:21 |
| ThomasS | will do this and then setup-everything | 10:21 |
| odyssey4me | at this stage I don't know - you are going to have to take responsibility for doing your own troubleshooting... we know that the tools work - we're confident because they're tested daily and used by many installations... so what you're hitting is some kind of edge case specific to your environment and configuration | 10:22 |
| odyssey4me | it's usually best to figure out your desired config in a test environment where you can rebuild your hosts quickly, because it will take some time to get all the network configs and other things right and you'll want to rebuild/retest many, many times before you actually think you have your production config right. | 10:23 |
| ThomasS | yes you are totally right | 10:30 |
| ThomasS | i will give it one more try and hope the dnsmasq is reconfigured with the new settings | 10:30 |
| ThomasS | if not, it is configured somewhere else | 10:31 |
| ThomasS | does it make sense to clean the ansible_facts dir? | 10:32 |
| ThomasS | or how can i rebuild the inventory? | 10:32 |
| *** hw_wutianwei has quit IRC | 10:33 | |
| *** cjloader has joined #openstack-ansible | 10:38 | |
| *** pradiprwt has joined #openstack-ansible | 10:38 | |
| openstackgerrit | Manuel Buil proposed openstack/openstack-ansible-os_neutron master: Fix the neutron_package_list condition https://review.openstack.org/522775 | 10:40 |
| *** cjloader has quit IRC | 10:42 | |
| pradiprwt | Hi, I facing issue while installing nova it is getting failed, here is console error>> http://paste.openstack.org/show/627311/ . I observed in one of infra node few containers are STOPPED state, manually I tried to start but it is not starting | 10:43 |
| pradiprwt | anyone have idea how I can solve this | 10:43 |
| *** DanyC has joined #openstack-ansible | 10:49 | |
| odyssey4me | pradiprwt read the container log to see if it helps you figure out why the container won't start | 10:51 |
| odyssey4me | failing that, destroy the container and rebuild it | 10:51 |
| pradiprwt | odyssey4me : Is it possible to destroy the single container and rebuild it ? | 10:52 |
| odyssey4me | pradiprwt yes: openstack-ansible lxc-containers-destroy.yml --limit <container name> | 10:53 |
| odyssey4me | then: openstack-ansible lxc-containers-create.yml --limit lxc_hosts,<container name> | 10:53 |
| pradiprwt | great | 10:53 |
| pradiprwt | lxc_hosts will be IP ? | 10:54 |
| odyssey4me | no, that's the group name - no need to translate it into anything | 10:57 |
| pradiprwt | ok | 10:57 |
| odyssey4me | only replace what's between the <> | 10:57 |
| pradiprwt | I have deleted the infra1_nova_conductor_container-45b7a6ea container | 10:57 |
| pradiprwt | but it getting fail to create | 10:58 |
| pradiprwt | created :) | 10:58 |
| odyssey4me | and so you have to do what I first suggested - figure out why | 10:59 |
| odyssey4me | once it's recreated, you'll need to run the os-nova-install.yml playbook again to configure nova on it | 10:59 |
| pradiprwt | Container creation in progress, it is creating container in the host which has problem | 11:00 |
| *** michelv has quit IRC | 11:02 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally stable/newton: Used cached git sources and enable depends-on https://review.openstack.org/518293 | 11:04 |
| pradiprwt | odyssey4me : The container creation playbook is fridge from last 10 min, container is stopped state in first infra node | 11:06 |
| pradiprwt | There is no any useful log in logfile | 11:08 |
| odyssey4me | pradiprwt well, you're going to have to find the information somewhere - it's up to you to figure it out | 11:08 |
| pradiprwt | sure | 11:09 |
| *** michelv has joined #openstack-ansible | 11:09 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Use Ansible 2.4 https://review.openstack.org/522778 | 11:10 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Fix ansible-lint test failures for ironic https://review.openstack.org/522779 | 11:10 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Use Ansible 2.4 https://review.openstack.org/522778 | 11:11 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Use Ansible 2.4 https://review.openstack.org/522778 | 11:13 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic master: Resolve ansible-lint test failures https://review.openstack.org/522780 | 11:13 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic master: Use group_names to check a host belongs to group https://review.openstack.org/520987 | 11:14 |
| *** yifei has quit IRC | 11:17 | |
| evrardjp | ThomasS: http://paste.openstack.org/show/627318/ so I think your generated hosts file is fine. You have another issue maybe. | 11:17 |
| ThomasS | ok | 11:21 |
| ThomasS | thanks for the debug | 11:21 |
| ThomasS | doing resinstall with different lxc network now | 11:21 |
| ThomasS | dnsmasq and iptables are already fixed by setup-hosts | 11:22 |
| *** udesale has quit IRC | 11:24 | |
| mardim__ | hwoarang, odyssey4me evrardjp I think I identified why we have the transient error in ovs-nsh gate | 11:27 |
| mardim__ | regarding the headers installation | 11:27 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: [TEST] Check the existing global pip.conf contents https://review.openstack.org/522782 | 11:27 |
| evrardjp | mardim__: yeah? | 11:27 |
| mardim__ | I found why sometimes it succededs | 11:28 |
| mardim__ | and sometimes it fails | 11:28 |
| mardim__ | let me expalin what I am observing | 11:28 |
| evrardjp | odyssey4me: for your test you should probably add a debug task ;) | 11:28 |
| evrardjp | else it will just run cat on your host | 11:28 |
| odyssey4me | evrardjp hmm, good point | 11:28 |
| evrardjp | :p | 11:28 |
| mardim__ | when we have this it succeds TASK [os_neutron : Install neutron role packages] ****************************** | 11:29 |
| mardim__ | 2017-11-24 10:22:35.243503 | ubuntu-xenial | Friday 24 November 2017 10:22:35 +0000 (0:00:02.868) 0:33:57.993 ******* | 11:29 |
| mardim__ | 2017-11-24 10:23:25.974055 | ubuntu-xenial | changed: [server1] | 11:29 |
| mardim__ | 2017-11-24 10:23:26.207848 | ubuntu-xenial | changed: [server2] | 11:29 |
| mardim__ | 2017-11-24 10:25:24.405204 | ubuntu-xenial | changed: [localhost] | 11:29 |
| mardim__ | 2017-11-24 10:25:32.663068 | ubuntu-xenial | changed: [agents1] | 11:29 |
| mardim__ | 2017-11-24 10:25:33.947642 | ubuntu-xenial | changed: [agents2] | 11:29 |
| evrardjp | mardim__: yes? | 11:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic master: Resolve ansible-lint test failures https://review.openstack.org/522780 | 11:30 |
| mardim__ | evrardjp, I am trying to paste :P | 11:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic master: Use group_names to check a host belongs to group https://review.openstack.org/520987 | 11:30 |
| odyssey4me | mardim__ use pastebin or a similar paste service for log pasting please | 11:31 |
| evrardjp | mardim__: paste.openstack.org | 11:31 |
| mardim__ | ok | 11:31 |
| odyssey4me | paste.openstack.org or gist or whatever else | 11:31 |
| evrardjp | :) | 11:31 |
| evrardjp | odyssey4me: great minds think alike! | 11:31 |
| evrardjp | :D | 11:32 |
| mardim__ | odyssey4me, evrardjp here :) https://hastebin.com/pomubepesa.rb | 11:33 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: [TEST] Check the existing global pip.conf contents https://review.openstack.org/522782 | 11:33 |
| mardim__ | when it runs like that it fails | 11:33 |
| mardim__ | the problem is in the order for some reason | 11:33 |
| *** hamza has quit IRC | 11:34 | |
| *** hamza has joined #openstack-ansible | 11:34 | |
| odyssey4me | evrardjp and rally for newton is back to a working state with https://review.openstack.org/522765 and https://review.openstack.org/518293 :) | 11:34 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: tasks: Remove unused apache configuration https://review.openstack.org/522785 | 11:34 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: Add support for the openSUSE Leap distributions https://review.openstack.org/522786 | 11:34 |
| odyssey4me | mardim__ I would not be surprised if kernel packages need to be installed on the host before they're installed on the containers | 11:36 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: Add support for the openSUSE Leap distributions https://review.openstack.org/522786 | 11:36 |
| hwoarang | that's a very strange bug | 11:37 |
| hwoarang | or behavior... | 11:37 |
| odyssey4me | hwoarang off-hand, does opensuse not do a conf.d directory for nginx at all, or is it just in a different location to centos/ubuntu ? | 11:37 |
| mardim__ | hwoarang, odyssey4me evrardjp I am thinking.... | 11:38 |
| odyssey4me | hwoarang for reference, see the opensuse results in https://review.openstack.org/#/c/521754/ | 11:38 |
| hwoarang | odyssey4me: i think it doesn't use one by default | 11:38 |
| mardim__ | odyssey4me, hwoarang evrardjp can i somehow in ansible to define the order for the hosts in inventory ? | 11:39 |
| mardim__ | odyssey4me, hwoarang evrardjp so i can replicate ? | 11:39 |
| mardim__ | the behaviour | 11:39 |
| odyssey4me | hwoarang ah, I see we change the nginx.conf to include conf files from there so I can just create the dirs | 11:40 |
| hwoarang | yep | 11:40 |
| hwoarang | i had to do that in keystone i think | 11:40 |
| odyssey4me | hwoarang ah, thanks for that: https://github.com/openstack/openstack-ansible-os_keystone/blob/master/tasks/keystone_nginx.yml#L43 | 11:41 |
| hwoarang | wow i am surprised by my memory | 11:41 |
| hwoarang | :) | 11:41 |
| evrardjp | odyssey4me: do we try rally with ansible 2.3 in newton? | 11:41 |
| hwoarang | good good | 11:41 |
| evrardjp | :p | 11:41 |
| evrardjp | it would work, but we have a different version in O | 11:42 |
| odyssey4me | evrardjp I haven't tried to actually use it - but it builds | 11:42 |
| evrardjp | :) | 11:42 |
| evrardjp | I don't really care. | 11:42 |
| evrardjp | it is just weird to have ocata with a different version now | 11:43 |
| hwoarang | mardim__: do you really need linux-headers everywhere? | 11:43 |
| hwoarang | or just the host? because the kmp is loaded only on the host right? | 11:44 |
| hwoarang | so you shouldn't install all that in the containers | 11:44 |
| hwoarang | or not? | 11:44 |
| evrardjp | mardim__: yes it's possible to change the order on the inventory | 11:44 |
| mardim__ | hwoarang, hm that's a good point let me think | 11:44 |
| evrardjp | I mean in your playbook | 11:44 |
| evrardjp | but that's not something you should do. | 11:44 |
| hwoarang | afaik you can't actually load a kernel module in the container. the kernel and its modules are shared with the host | 11:45 |
| hwoarang | so loading it on the host is enough | 11:45 |
| hwoarang | i believe... | 11:45 |
| mardim__ | evrardjp, I wanted just to replicate so I can identify the error | 11:45 |
| evrardjp | mardim__: what hwoarang said makes sense. | 11:45 |
| mardim__ | hwoarang, yes you are right that's why I added that to my patch name: Restart OvS-NSH service to load the kmod inside the container | 11:46 |
| mardim__ | command: lxc-attach -n "{{ item }}" -- service openvswitch-switch restart | 11:46 |
| mardim__ | ignore_errors: True | 11:46 |
| mardim__ | with_items: "{{ groups[neutron_services['neutron-dhcp-agent']['group']] }}" | 11:46 |
| mardim__ | when: | 11:46 |
| mardim__ | - ovs_nsh_support | bool | 11:46 |
| mardim__ | - ("openvswitch_nsh" in group_names) | 11:46 |
| evrardjp | mardim__: I already commented on that task | 11:47 |
| evrardjp | you should probably not do that. | 11:47 |
| evrardjp | odyssey4me: did you see this already? It's an infra thing | 11:47 |
| evrardjp | http://logs.openstack.org/periodic/git.openstack.org/openstack/openstack-ansible/master/openstack-ansible-upgrade-ceph-ubuntu-xenial/f835b74/job-output.txt.gz#_2017-11-24_06_08_06_547867 | 11:47 |
| *** armaan has quit IRC | 11:47 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_server master: Implement pypiserver and pypi proxy cache https://review.openstack.org/521754 | 11:48 |
| mardim__ | evrardjp, I should fo this because in xci the neutron agents containers are executed before the controller host | 11:48 |
| *** michelv has quit IRC | 11:48 | |
| mardim__ | which means if I do not restart | 11:48 |
| mardim__ | the ovs inside the containers is in an error state | 11:48 |
| mardim__ | fo-> do | 11:49 |
| odyssey4me | evrardjp hmm, I'm guessing that we should take out the 'origin' specification there | 11:49 |
| odyssey4me | IIRC infra removes the origin git remote for some reason | 11:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Collect logs to a subfolder https://review.openstack.org/522794 | 11:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Fix order of script for upgrades https://review.openstack.org/522795 | 11:51 |
| evrardjp | odyssey4me: do we? | 11:51 |
| evrardjp | that sounds weird | 11:51 |
| odyssey4me | evrardjp it is, and I don't know the history - but that is what I noticed at some point was being done | 11:52 |
| odyssey4me | we don't need to checkout the branch from the origin anyway, we just need the branch | 11:52 |
| *** michelv has joined #openstack-ansible | 11:52 | |
| evrardjp | I am fixing it right now | 11:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally stable/newton: Constraint ansible installation https://review.openstack.org/522765 | 11:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_rally stable/newton: Used cached git sources and enable depends-on https://review.openstack.org/518293 | 11:53 |
| odyssey4me | evrardjp good suggestion - adjusted | 11:54 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Ensure Zuul can checkout the proper branch https://review.openstack.org/522797 | 11:54 |
| odyssey4me | how exciting - horizon is almost in a place where it can use conf files like other services: https://review.openstack.org/404735 :) | 11:54 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Do not force using the remote "origin" when testing upgrades https://review.openstack.org/522797 | 11:56 |
| evrardjp | odyssey4me: horizon dashboards are curently borked for us, I will fix that asaic | 11:57 |
| evrardjp | but that looks cool | 11:57 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: Add support for the openSUSE Leap distributions https://review.openstack.org/522786 | 11:57 |
| evrardjp | odyssey4me: What would be cool is that we can pass any random variable to deploy any random horizon dashboard | 11:58 |
| evrardjp | will write that down | 11:58 |
| evrardjp | that's easy when I think about it. | 11:59 |
| evrardjp | Darn I have another idea :) | 11:59 |
| evrardjp | hwoarang: thanks on the work on barbican that sounds very nice | 12:02 |
| evrardjp | should be in a scenario in our integrated gates! | 12:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Allow pip_install to run in repo-server playbook https://review.openstack.org/522022 | 12:07 |
| odyssey4me | evrardjp barbican's in the aio scenario ;) | 12:07 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Fix order of script for upgrades https://review.openstack.org/522795 | 12:07 |
| evrardjp | odyssey4me: oh | 12:07 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: Add support for the openSUSE Leap distributions https://review.openstack.org/522786 | 12:08 |
| hwoarang | too bad it doesn't work with mariadb 10.2 | 12:09 |
| odyssey4me | oh really? | 12:09 |
| evrardjp | odyssey4me: it's not listed on https://docs.openstack.org/openstack-ansible/latest/contributor/quickstart-aio.html | 12:09 |
| hwoarang | yep | 12:09 |
| hwoarang | odyssey4me: i added some links in the zuul job on why it's non-voting for opensuse. hopefully barbican will fix it soon... or mariadb... | 12:09 |
| odyssey4me | evrardjp oh, my bad - I thought it was | 12:10 |
| evrardjp | odyssey4me: did we mess up something in the commit there? https://github.com/openstack/openstack-ansible/blob/master/tests/vars/bootstrap-aio-vars.yml | 12:10 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_server master: Implement pypiserver and pypi proxy cache https://review.openstack.org/521754 | 12:10 |
| odyssey4me | evrardjp no, it was probably something we discussed (or I discussed with myself) | 12:10 |
| evrardjp | :) | 12:10 |
| odyssey4me | but never implemented | 12:10 |
| evrardjp | I don't remember with which project can barbican be used | 12:11 |
| evrardjp | I guess cinder and keystone? | 12:11 |
| odyssey4me | evrardjp heh, I knew it - https://review.openstack.org/#/c/486086/ | 12:12 |
| odyssey4me | barbican can be used with any project that uses secrets - swift, ironic, keystone, cinder | 12:12 |
| odyssey4me | not sure about ironic, but they may have got the integration done | 12:12 |
| evrardjp | odyssey4me: ha. | 12:13 |
| evrardjp | odyssey4me: I guess I am good to backport dstat all accross the board | 12:15 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Run dstat in gate tests https://review.openstack.org/522799 | 12:19 |
| evrardjp | odyssey4me: let's call this gate the gate of secrets! | 12:20 |
| evrardjp | https://wiki.guildwars.com/wiki/Gate_of_Secrets | 12:20 |
| evrardjp | or maybe better | 12:20 |
| evrardjp | the chamber of secrets! | 12:21 |
| hwoarang | evrardjp: can you fold this fix https://review.openstack.org/#/c/520615/ in your backport ? | 12:21 |
| hwoarang | gates always timeout i haven't managed to get it to master yet ... | 12:22 |
| evrardjp | :( | 12:23 |
| evrardjp | hwoarang: let's merge it in master first before backporting... Anyway this backport is already in a race condition with the ARA one | 12:23 |
| evrardjp | so one or the other will get your backport | 12:24 |
| *** DanyC has quit IRC | 12:24 | |
| hwoarang | ok | 12:24 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Set the default pip index to use the pypi cache when possible https://review.openstack.org/522802 | 12:27 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Add task to allow creation of provided roles https://review.openstack.org/518611 | 12:35 |
| ThomasS | hi | 12:36 |
| ThomasS | i did the re-run of setup-infra and it did not succeed | 12:36 |
| ThomasS | will do a second try | 12:37 |
| ThomasS | if this is not working out i will reinstall at least the controllers | 12:37 |
| ThomasS | is there any more info about dns setup required? | 12:37 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Set the default pip index to use the pypi cache when possible https://review.openstack.org/522802 | 12:38 |
| odyssey4me | ThomasS the containers all inherit dns config from the host, so as long as the host is good it should be fine | 12:38 |
| evrardjp | LOCI is now an officially approved project | 12:40 |
| odyssey4me | yeah, saw that - nice work there SamYaple | 12:41 |
| *** armaan has joined #openstack-ansible | 12:45 | |
| ThomasS | the host was good all the time | 12:49 |
| ThomasS | it has a host file for all the nodes and dns is out public | 12:50 |
| *** dave-mccowan has joined #openstack-ansible | 12:51 | |
| *** yifei has joined #openstack-ansible | 12:52 | |
| ThomasS | now looks like the rabbit cluster is online | 12:53 |
| evrardjp | hwoarang: could you vote on https://review.openstack.org/#/c/522542/ ? | 12:55 |
| evrardjp | very quick and easy vote | 12:55 |
| ThomasS | now ceph mon throughs an error with the release. i set for ceph_release_num: 12 | 12:55 |
| ThomasS | the error is a bit misleading for me | 12:56 |
| *** askb has quit IRC | 12:56 | |
| ThomasS | https://paste.ee/p/9H0jf | 12:56 |
| neith | Hello all,is it the expected behavior not to authorize to have 2 same rules in 2 secGRoups in the same project? | 12:56 |
| ThomasS | user_variables looks like this | 12:58 |
| ThomasS | https://paste.ee/p/oZ0tb | 12:58 |
| ThomasS | maybe i set i wrong :-) | 12:59 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Create cluster without using a failed task https://review.openstack.org/522810 | 13:00 |
| odyssey4me | evrardjp hwoarang ^ I've had enough of the confusion that rabbit cluster task causes... time to fix it. | 13:00 |
| odyssey4me | working on the cookie one now too. | 13:00 |
| evrardjp | odyssey4me: ok | 13:03 |
| evrardjp | better that than waiting for ansible 2.5 | 13:03 |
| ThomasS | :-) | 13:03 |
| ThomasS | i commented ceph_release_num: 12 in user_vars because it will pick luminous anyway | 13:04 |
| odyssey4me | evrardjp I think that using a task that will always fail is bad form regardless. | 13:04 |
| evrardjp | odyssey4me: you are 100% right. | 13:05 |
| *** jvidal has quit IRC | 13:05 | |
| evrardjp | a purist would have asked a module to check the state, but I don't care. | 13:05 |
| evrardjp | well I do, but not at that point. | 13:05 |
| mardim__ | hwoarang, evrardjp odyssey4me I think I found the solution to the problems with the kernel modules for ovs-nsh | 13:05 |
| mardim__ | the problem is | 13:06 |
| mardim__ | that iam using the neutron role to load the modules on physical hosts | 13:06 |
| evrardjp | is? | 13:06 |
| evrardjp | and? | 13:06 |
| mardim__ | But when we use the plain ovs | 13:06 |
| mardim__ | we use this role to load the modules on the physical hosts | 13:07 |
| mardim__ | https://github.com/openstack/openstack-ansible-openstack_hosts | 13:07 |
| mardim__ | which makes sense because this role will prepare the hosts | 13:07 |
| mardim__ | I ma thinking to use the same role so I can load the modules on the pysical hosts for ovs-nsh | 13:08 |
| mardim__ | hwoarang, evrardjp odyssey4me What do you think ? | 13:08 |
| evrardjp | I don't understand your point | 13:09 |
| evrardjp | https://github.com/openstack/openstack-ansible-os_neutron/blob/master/tests/test.yml#L17 | 13:10 |
| mardim__ | evrardjp, ok let me explain | 13:10 |
| mardim__ | evrardjp, when we use palin ovs without nsh support | 13:10 |
| mardim__ | plain* | 13:10 |
| *** m|y|k has joined #openstack-ansible | 13:11 | |
| mardim__ | we use this role: https://github.com/openstack/openstack-ansible-openstack_hosts | 13:11 |
| mardim__ | to load the kernel modules of ovs to the physical hosts | 13:11 |
| evrardjp | for me to understand what you are explaining, I need code :) | 13:11 |
| evrardjp | point me to the line and I will understand better than english | 13:11 |
| evrardjp | :) | 13:11 |
| mardim__ | ok sure :) | 13:12 |
| mardim__ | give me a sec | 13:12 |
| mardim__ | evrardjp, https://github.com/openstack/openstack-ansible-os_neutron/blob/master/tests/neutron-overrides-opendaylight.yml#L18-L21 | 13:13 |
| evrardjp | yes | 13:13 |
| mardim__ | there is more wait :) | 13:13 |
| mardim__ | evrardjp, https://github.com/openstack/openstack-ansible-openstack_hosts/blob/5674ee514c87e86473ad7b0fde8cbf3682b4eacb/defaults/main.yml#L38 | 13:14 |
| mardim__ | see ? | 13:14 |
| mardim__ | evrardjp, Do you see the connection when we use plain ovs withou nsh support ? | 13:14 |
| mardim__ | without* | 13:15 |
| mardim__ | evrardjp, ? | 13:17 |
| evrardjp | no I do not | 13:17 |
| evrardjp | the neutron role is using the openstack_hosts indeed | 13:17 |
| evrardjp | and by default empty | 13:17 |
| evrardjp | and because there is nothing in test-vars I understand why this could be a problem | 13:18 |
| evrardjp | but I don't understand how ovs works then | 13:18 |
| mardim__ | no wait | 13:18 |
| mardim__ | forget about ovs-nsh | 13:18 |
| evrardjp | oh maybe in tox | 13:19 |
| *** michelv has quit IRC | 13:19 | |
| evrardjp | yeah that's it | 13:19 |
| evrardjp | ok now I understand better. | 13:19 |
| mardim__ | what did you understand ? | 13:19 |
| evrardjp | how this works. | 13:19 |
| mardim__ | tell me so ican be sure we are in the same page | 13:19 |
| mardim__ | :) | 13:19 |
| evrardjp | what's your tox env? | 13:20 |
| evrardjp | opendaylight? | 13:20 |
| mardim__ | this is not related to tox | 13:20 |
| mardim__ | forget about testing | 13:20 |
| evrardjp | what's your issue? | 13:20 |
| mardim__ | Until now if we wanted to install the ovs | 13:20 |
| mardim__ | we should include this https://github.com/openstack/openstack-ansible-os_neutron/blob/master/tests/neutron-overrides-opendaylight.yml#L18-L21 | 13:21 |
| mardim__ | in the user_variables.yaml file | 13:21 |
| mardim__ | right ? | 13:21 |
| evrardjp | yes... which is done in tox | 13:21 |
| evrardjp | by giving the file to the cli | 13:21 |
| mardim__ | forget about testing I ma talking for a real deployment now | 13:22 |
| mardim__ | forget about CI | 13:22 |
| evrardjp | yes. | 13:22 |
| mardim__ | ok and the reason we included that in the user_variables.yaml | 13:22 |
| evrardjp | a real deploy of an integrated openstack-ansible that has user_* variables, that's what you are talking about? | 13:22 |
| mardim__ | exactly | 13:23 |
| evrardjp | ok please go on | 13:23 |
| mardim__ | the reason we incuded that in the user_variables.yaml file | 13:23 |
| mardim__ | is because we wanted | 13:23 |
| mardim__ | this role: | 13:23 |
| evrardjp | yes for openstack_hosts | 13:23 |
| mardim__ | exactly | 13:24 |
| evrardjp | so? | 13:24 |
| mardim__ | we can use the same role to load the kernel modules when we have the ovs-nsh case | 13:24 |
| evrardjp | is the group: "network_hosts" confusing you or? | 13:24 |
| mardim__ | we can use the same role to install the ovs-nsh kernel modules | 13:25 |
| evrardjp | yes, this is the purpose of the openstack_hosts role and that's part of our deployment.... | 13:25 |
| mardim__ | exactly | 13:25 |
| evrardjp | I still don't understand what's the issue/what's revolutionary | 13:25 |
| evrardjp | :p | 13:25 |
| mardim__ | I didn;t know that and I was trying to loda the ovs-nsh kernel modules from neutron role instead the openstack_hosts role | 13:25 |
| mardim__ | load* | 13:25 |
| evrardjp | if you are doing your own machine, yes you need your own user_* overrides | 13:26 |
| evrardjp | and if you want to include that into a openstack-ansible integrated gate test, you'd have to generate something like user_opendaylight.yml for example | 13:26 |
| mardim__ | No still you dodn't catch my point :( | 13:27 |
| mardim__ | didn't* | 13:27 |
| evrardjp | nope. | 13:27 |
| evrardjp | I told you, code is easier than english. | 13:27 |
| evrardjp | :) | 13:27 |
| mardim__ | hahah I know :) | 13:27 |
| *** armaan has quit IRC | 13:27 | |
| *** armaan has joined #openstack-ansible | 13:27 | |
| mardim__ | I will create the patches and I will add you as a reviewer | 13:28 |
| mardim__ | and you will understand ;) | 13:28 |
| evrardjp | there is no need to patch openstack_hosts. | 13:28 |
| mardim__ | evrardjp, Yes there is | 13:28 |
| evrardjp | for what? | 13:28 |
| mardim__ | because we will use that role to load the ovs-nsh kernel modules | 13:28 |
| evrardjp | so? | 13:28 |
| mardim__ | which is the proper thing to do | 13:29 |
| mardim__ | because we do that for plain ovs so far | 13:29 |
| ThomasS | fyi now the setup-openstack playbook is running :-) | 13:29 |
| odyssey4me | mardim__ evrardjp yeah, openstack_hosts does the kernel module loading for ovs in general IIRC so ti makes sense to do it there | 13:30 |
| evrardjp | mardim__: please patch something, we'll discuss in the review. But I am pretty sure there is NO change needed in openstack_hosts role | 13:30 |
| evrardjp | odyssey4me: I don't get what you mean | 13:30 |
| mardim__ | odyssey4me, exactly | 13:31 |
| evrardjp | https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/tasks/openstack_kernel_modules.yml | 13:31 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Use tempest plugin for functional testing https://review.openstack.org/518612 | 13:31 |
| evrardjp | is abstracted enugh | 13:31 |
| evrardjp | if you want an override, it's just a var | 13:31 |
| mardim__ | evrardjp, odyssey4me https://review.openstack.org/#/c/517259/21/vars/ubuntu-16.04.yml | 13:31 |
| odyssey4me | yep, just openstack_host_specific_kernel_modules needs to be appropriately adjusted - probably in group vars somewhere | 13:31 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Check and set cookie without using a failed task https://review.openstack.org/522814 | 13:31 |
| mardim__ | line 25 26 | 13:31 |
| mardim__ | I will move those packages to openstack_host role | 13:32 |
| evrardjp | odyssey4me: my point exactly. No need to change the openstack_hosts role | 13:32 |
| *** openstackgerrit has quit IRC | 13:32 | |
| evrardjp | this is all a docs change at best | 13:33 |
| evrardjp | for the integrated gates | 13:33 |
| evrardjp | and if you are doing a integrated scenario, then you need to define these user_* variables. | 13:33 |
| odyssey4me | hmm, might need to add a facility to add extra packages as well, otherwise just setting the kernel modules to load won't work | 13:33 |
| evrardjp | we already have that | 13:33 |
| odyssey4me | oh? | 13:34 |
| evrardjp | openstack_host_distro_packages | 13:34 |
| mardim__ | evrardjp, really ? | 13:34 |
| mardim__ | evrardjp, can you provide a link | 13:34 |
| odyssey4me | yes, but you don't want to override the whole list -just add a few more | 13:34 |
| mardim__ | ? | 13:34 |
| mardim__ | Also I need to add my ppa | 13:34 |
| evrardjp | odyssey4me: mmm yes. | 13:34 |
| mardim__ | because the packages I want to isntall is in a private ppa | 13:34 |
| mardim__ | tough decisions here :) | 13:35 |
| evrardjp | odyssey4me: we can't do that for now, not the way it's written right now. My patch solves that though, but you -1ed it :p | 13:35 |
| evrardjp | mardim__: you can still override that list of packages you want to install everywhere with your user_* files | 13:36 |
| evrardjp | odyssey4me: the elegance of my patch made it also group vars friendly :) | 13:36 |
| mardim__ | Yes but what about the private ppa | 13:36 |
| odyssey4me | evrardjp your patch doesn't change this situation at all | 13:36 |
| mardim__ | I am installing those packages form a private ppa | 13:36 |
| odyssey4me | it doesn't allow additional arbitrary repositories to be added | 13:36 |
| odyssey4me | it only does UCA, OR an external repo | 13:37 |
| odyssey4me | and that functionality is already in the role | 13:37 |
| evrardjp | odyssey4me: you are wrong | 13:37 |
| mardim__ | evrardjp, Do you have the link to your patch ? | 13:37 |
| mardim__ | ican help with the reviews | 13:38 |
| evrardjp | mardim__: for your use case we you shouldn't technically need my patch | 13:38 |
| evrardjp | in your case, it's just a doc change | 13:38 |
| evrardjp | you say to your users they need repo x installed on their host | 13:39 |
| evrardjp | then define in their user_* the proper kernel module loading | 13:39 |
| evrardjp | and the proper package installation | 13:39 |
| evrardjp | however if you want to review, here it is: | 13:39 |
| evrardjp | https://review.openstack.org/#/c/515397 | 13:39 |
| evrardjp | in your case: | 13:39 |
| evrardjp | https://review.openstack.org/#/c/515397/6/tasks/openstack_hosts_configure_apt.yml would be enough to configure your ppa | 13:39 |
| evrardjp | then the configuration of your metal node distro packages would happen there: https://review.openstack.org/#/c/515397/6/tasks/configure_metal_hosts.yml L77 | 13:40 |
| evrardjp | oh apparently odyssey4me is right, I forgot to do the proper override in openstack_host_metal_distro_packages | 13:41 |
| evrardjp | like it's done for the other keys (ppa etc) | 13:41 |
| evrardjp | let me fix that quick | 13:41 |
| mardim__ | evrardjp, I am thinking ... :) | 13:42 |
| *** ianychoi has quit IRC | 13:44 | |
| *** openstackgerrit has joined #openstack-ansible | 13:45 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Create cluster without using a failed task https://review.openstack.org/522810 | 13:45 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: [WIP] Add the ability to run the role on all hosts https://review.openstack.org/515397 | 13:45 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Check and set cookie without using a failed task https://review.openstack.org/522814 | 13:47 |
| mgariepy | morning everyone | 13:47 |
| evrardjp | morning mgariepy! | 13:47 |
| evrardjp | mgariepy: could you review https://review.openstack.org/#/c/522542/ ? | 13:48 |
| mgariepy | sure | 13:52 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Use Ansible 2.4 https://review.openstack.org/522778 | 13:54 |
| mgariepy | done. | 13:55 |
| evrardjp | thanks | 13:57 |
| *** Dinesh_Bhor has quit IRC | 13:57 | |
| evrardjp | odyssey4me: reviewed, proposed a simplification, not sure if you'll like it :p | 13:57 |
| odyssey4me | ja, thought of that - but I don't know what effect there is of changing the cookie while it's running | 14:00 |
| odyssey4me | let me do that in a subsequent patch so we can see if it's ok | 14:00 |
| *** SerenaFeng has joined #openstack-ansible | 14:01 | |
| odyssey4me | actually, may as well roll it in | 14:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Check and set cookie without using a failed task https://review.openstack.org/522814 | 14:04 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Fix ansible-lint test failures https://review.openstack.org/522829 | 14:07 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Check and set cookie without using a failed task https://review.openstack.org/522814 | 14:08 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server master: Create cluster without using a failed task https://review.openstack.org/522810 | 14:08 |
| *** dave-mccowan has quit IRC | 14:10 | |
| evrardjp | odyssey4me: yeah, los corones! | 14:14 |
| evrardjp | that was definitely inappropriate, sorry. | 14:16 |
| evrardjp | odyssey4me: haha "ANSIBLE0014 Environment variables don't work as part of command which is nonsense." | 14:18 |
| odyssey4me | yeah, been seeing that here and there - it definitely works, so I'm not sure why the lint thinks it doesn't | 14:19 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Improve lint test flexibility https://review.openstack.org/522832 | 14:24 |
| evrardjp | odyssey4me: yeah, don't know. | 14:25 |
| *** hamza has quit IRC | 14:25 | |
| *** lbragstad has joined #openstack-ansible | 14:28 | |
| mardim__ | evrardjp, I reviewd your patch :) | 14:30 |
| mardim__ | evrardjp, Also I can test your patch when it is ready so just tell me when is it :) | 14:32 |
| *** michelv has joined #openstack-ansible | 14:35 | |
| *** yifei has quit IRC | 14:35 | |
| *** yifei has joined #openstack-ansible | 14:35 | |
| *** cjloader has joined #openstack-ansible | 14:36 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: [WIP] Add the ability to run the role on all hosts https://review.openstack.org/515397 | 14:36 |
| evrardjp | mardim__: gates are testing it :D | 14:37 |
| evrardjp | if not, we are missing something in our tests :) | 14:37 |
| *** SerenaFeng has quit IRC | 14:38 | |
| mardim__ | evrardjp, ok then lets cross our fingers for the gates :) | 14:39 |
| *** cjloader has quit IRC | 14:41 | |
| evrardjp | :D | 14:41 |
| *** yifei has quit IRC | 14:44 | |
| *** yifei has joined #openstack-ansible | 14:44 | |
| *** armaan has quit IRC | 14:47 | |
| *** armaan has joined #openstack-ansible | 14:48 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-openstack_hosts master: Add the ability to run the role on all hosts https://review.openstack.org/515397 | 14:50 |
| odyssey4me | evrardjp not sure if you noticed, but we can't look at our ara results because the zuul ara report overwrites ours... so I did https://review.openstack.org/#/c/522794/1 to solve that | 14:53 |
| evrardjp | yeah I've noticed many things that are still broken since zuul v3 :D just making myself blind to them until I haven't fixed the fires. | 14:54 |
| evrardjp | I have fixed* | 14:54 |
| evrardjp | thanks for the patch! | 14:54 |
| evrardjp | will review that quick | 14:54 |
| evrardjp | odyssey4me: doesn't look like it works? | 14:55 |
| mardim__ | evrardjp, To be sure that we are in the same page | 14:55 |
| *** armaan has quit IRC | 14:55 | |
| mardim__ | if your patch gets merged | 14:56 |
| evrardjp | if :) | 14:56 |
| *** armaan has joined #openstack-ansible | 14:56 | |
| odyssey4me | evrardjp it is working, we don't generate an ara report if the test passes | 14:56 |
| mardim__ | I will be able through user variables | 14:56 |
| evrardjp | oh yeah true. We need to change that at some point, or at least generate the subunit | 14:56 |
| mardim__ | to add my private ppa with higher priority than others | 14:56 |
| mardim__ | to install the packages for kernel modules | 14:57 |
| evrardjp | and load them, yeah | 14:57 |
| evrardjp | in group vars | 14:57 |
| mardim__ | and also load my custom kernel modules | 14:57 |
| evrardjp | or in user_* | 14:57 |
| mardim__ | in the controller and compute hosts right ? | 14:57 |
| evrardjp | that's pretty much all nodes | 14:58 |
| evrardjp | :p | 14:58 |
| evrardjp | joking aside, yes | 14:58 |
| odyssey4me | evrardjp oh - I thought we were doing the subunit... looks like that's gone - lemme add that back | 14:58 |
| mardim__ | there are olaso the containers or ? | 14:58 |
| mardim__ | also* | 14:58 |
| evrardjp | odyssey4me: I don't see anything on health anymore, so I think we do not | 14:58 |
| mardim__ | evrardjp, if I add the private ppa to user_variables this will be added for all the nodes (physical and containers)? | 14:59 |
| mardim__ | or just physical ? | 14:59 |
| odyssey4me | evrardjp interesting - it appears that we never did for the roles | 15:00 |
| odyssey4me | anyway, will add it | 15:00 |
| evrardjp | oh I don't care about that for the roles | 15:01 |
| evrardjp | but why not indeed. | 15:01 |
| evrardjp | do you have an idea on why after an upgrade, my lxc containers have NO ips, except the galera one? | 15:01 |
| evrardjp | that's gonna be a fun mystery again | 15:02 |
| *** armaan has quit IRC | 15:04 | |
| *** armaan has joined #openstack-ansible | 15:05 | |
| evrardjp | fun stuff, I get kernel systraces! | 15:14 |
| *** yifei has quit IRC | 15:16 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Always generate ARA subunit results https://review.openstack.org/522849 | 15:26 |
| odyssey4me | evrardjp ^ | 15:26 |
| *** mgariepy has quit IRC | 15:29 | |
| evrardjp | questions asked! | 15:30 |
| *** mgariepy has joined #openstack-ansible | 15:32 | |
| *** tnogisto has quit IRC | 15:34 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Update all SHAs https://review.openstack.org/522850 | 15:36 |
| openstackgerrit | Merged openstack/openstack-ansible master: Create variable override tests https://review.openstack.org/522542 | 15:40 |
| *** armaan has quit IRC | 15:45 | |
| *** armaan has joined #openstack-ansible | 15:46 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Collect /etc files from the host and containers https://review.openstack.org/522852 | 15:47 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Fix in-tree version https://review.openstack.org/522853 | 15:49 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Fix in-tree version https://review.openstack.org/522853 | 15:50 |
| odyssey4me | dammit evrardjp the scripts should be doing that automatically for you, you should not have to remember | 15:50 |
| evrardjp | yeah. | 15:50 |
| evrardjp | worked for the rest. | 15:50 |
| odyssey4me | fix the script to ensure it does the right thing by default | 15:50 |
| evrardjp | yeah! | 15:51 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest stable/pike: Fix doc conf lint failures https://review.openstack.org/522854 | 15:55 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest stable/pike: Fix ansible-lint test failures https://review.openstack.org/522304 | 15:55 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.13 https://review.openstack.org/522856 | 16:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Always generate ARA subunit results https://review.openstack.org/522849 | 16:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Always generate ARA subunit results https://review.openstack.org/522849 | 16:03 |
| *** germs has joined #openstack-ansible | 16:05 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Collect /etc files from the host and containers https://review.openstack.org/522852 | 16:06 |
| *** mrch has quit IRC | 16:09 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/newton: Update all SHAs for 14.2.13 https://review.openstack.org/522858 | 16:11 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: test-log-collect.sh: Collect package and repository info on Ubuntu https://review.openstack.org/522860 | 16:16 |
| *** mardim__ has quit IRC | 16:19 | |
| *** armaan has quit IRC | 16:19 | |
| *** armaan has joined #openstack-ansible | 16:20 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: test-log-collect.sh: Collect package and repository info on Ubuntu https://review.openstack.org/522860 | 16:23 |
| *** mbuil has quit IRC | 16:25 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca stable/pike: Initial OSA zuul v3 role jobs https://review.openstack.org/516594 | 16:36 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade to RabbitMQ 3.6.14 https://review.openstack.org/522865 | 16:36 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Allow pip_install to run in repo-server playbook https://review.openstack.org/522022 | 16:38 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Set the default pip index to use the pypi cache when possible https://review.openstack.org/522802 | 16:39 |
| evrardjp | hwoarang: are you still there? | 16:41 |
| evrardjp | I am curious about something: http://logs.openstack.org/78/522778/4/check/openstack-ansible-deploy-aio-opensuse-423/1d4db6b/job-output.txt.gz#_2017-11-24_14_11_00_743526 | 16:41 |
| hwoarang | let me check | 16:45 |
| hwoarang | evrardjp: yeah it's the pgrep -f fix i am trying to get it | 16:45 |
| hwoarang | i was discussing it with logan- and odyssey4me the other day | 16:45 |
| evrardjp | ok | 16:45 |
| hwoarang | ansible has some troubles killing subshells from other processes | 16:46 |
| hwoarang | you will notice that centos7 has the same issue | 16:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Remove selfcheck.json https://review.openstack.org/522868 | 16:47 |
| openstackgerrit | Merged openstack/openstack-ansible master: scripts: scripts-library.sh: Use pgrep -f to find the dstat process https://review.openstack.org/520615 | 16:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Remove selfcheck.json https://review.openstack.org/522868 | 16:48 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-ops master: Validate the target directory exist before linking. https://review.openstack.org/522869 | 16:48 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing AIO on openSUSE https://review.openstack.org/507451 | 16:51 |
| *** chyka has joined #openstack-ansible | 16:54 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install master: Add ability to disable implementing pip.conf https://review.openstack.org/522871 | 16:55 |
| *** holmsten has quit IRC | 16:56 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Use Ansible 2.4 https://review.openstack.org/522778 | 16:57 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Set the default pip index to use the pypi cache when possible https://review.openstack.org/522802 | 16:58 |
| odyssey4me | evrardjp there we go, it's working now :) https://review.openstack.org/522849 | 17:01 |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server master: Fix ansible-lint test failures https://review.openstack.org/522829 | 17:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Fix ansible-lint test failures https://review.openstack.org/522875 | 17:03 |
| *** jbadiapa has quit IRC | 17:05 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/pike: Improve lint test flexibility https://review.openstack.org/522832 | 17:06 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Collect logs to a subfolder https://review.openstack.org/522794 | 17:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/pike: Collect logs to a subfolder https://review.openstack.org/522877 | 17:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/ocata: Collect logs to a subfolder https://review.openstack.org/522878 | 17:08 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/newton: Collect logs to a subfolder https://review.openstack.org/522879 | 17:09 |
| *** shardy has quit IRC | 17:13 | |
| *** germs has quit IRC | 17:14 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_monasca stable/pike: Updated from OpenStack Ansible Tests https://review.openstack.org/516299 | 17:15 |
| *** gkadam has quit IRC | 17:17 | |
| *** SerenaFeng has joined #openstack-ansible | 17:23 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Always quote the filesystem permissions https://review.openstack.org/518577 | 17:30 |
| odyssey4me | hwoarang evrardjp I have a bunch of patches ready for review and would appreciate it if you could take a peek: https://review.openstack.org/#/q/project:%22%255Eopenstack/(monitorstack%257Copenstack-ansible).*%22+status:open+label:Verified%253E%253D1%252Czuul+NOT+label:Workflow%253C%253D-1+NOT+label:Code-Review%253C%253D-1+owner:jesse-pretorius | 17:32 |
| odyssey4me | of course any other cores available can do the same | 17:32 |
| *** armaan has quit IRC | 17:38 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Fix doc conf lint failures https://review.openstack.org/522883 | 17:41 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Fix ansible-lint test failures https://review.openstack.org/522875 | 17:41 |
| evrardjp | odyssey4me: will do | 17:43 |
| spotz | odyssey4me: I'll take a look durin my next recording break. I can probably get a few done | 17:44 |
| spotz | have to keep stopping to fix typos in slides that went through QA.... | 17:44 |
| *** michelv has quit IRC | 17:44 | |
| odyssey4me | haha, thanks spotz - appreciate it | 17:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Fix lint test failures https://review.openstack.org/522875 | 17:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest stable/pike: Fix lint test failures https://review.openstack.org/522304 | 17:54 |
| *** pcaruana has quit IRC | 17:55 | |
| *** rogervn has joined #openstack-ansible | 17:56 | |
| rogervn | Hello guys, I´m having some trouble trying openstack-ansible behind a http and rpm proxies. | 17:57 |
| odyssey4me | rogervn have you seen https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-limited-connectivity.html ? | 17:58 |
| rogervn | I´ve been able to make it work for the setup-hosts part with the environment variables and editing the repos.d files. But the setup-infrastructure does´t seem to use the /etc/environment variables, so I´m having trouble to download get-pip. | 17:58 |
| *** SerenaFeng has quit IRC | 18:00 | |
| rogervn | Yes, I´ve set the user_variables.yml with the environment proxy variables. | 18:00 |
| rogervn | It also worked fine with the setup-hosts.yml. | 18:00 |
| odyssey4me | ok, that should also then work for the containers | 18:00 |
| *** germs has joined #openstack-ansible | 18:01 | |
| odyssey4me | rogervn have you read and understood https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/user_variables.yml#L113-L138 ? | 18:02 |
| odyssey4me | having no proxy set for the haproxy load balancers is important | 18:02 |
| *** michelv has joined #openstack-ansible | 18:04 | |
| *** germs1 has joined #openstack-ansible | 18:05 | |
| rogervn | I hadn´t read all that information yet, I just used the global_environment_variables. I´ll try some of the other tags. | 18:07 |
| *** germs has quit IRC | 18:07 | |
| rogervn | And yet I´ve net set any haproxy lbs, I´m trying to set a base to make changes, then I´ll deploy a working environment. | 18:08 |
| odyssey4me | you shouldn't use both for the same thing | 18:08 |
| odyssey4me | ok, but without any haproxy (or some lb) hosts nothing beyond setup-hosts will work | 18:09 |
| odyssey4me | from there on it will, very soon in setup-infra, expect to use the load balancer... that will happen once the repo server is setup | 18:09 |
| rogervn | Oh, i see. I´ll set the haproxy in my user_config then. | 18:11 |
| rogervn | Also, I´m having a lot of trouble to set my RPM proxy, because most of the code tries to use external RPM repositories. Is there any configs to only use the repos I have in the nodes? | 18:12 |
| odyssey4me | rogervn by default the repo server will setup a proxy too - you can either make that use your upstream proxy, or you can disable that | 18:14 |
| odyssey4me | all containers will inherit any of these files from the host: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/master/vars/redhat-7.yml#L46-L51 | 18:15 |
| odyssey4me | if you want more copied in (like perhaps your proxy conf), then you can add a list of files to copy to your user_variables: https://github.com/openstack/openstack-ansible-lxc_hosts/blob/670b47bbfbe8327dd5fb3c734e599cd03d16f5ea/defaults/main.yml#L115-L120 | 18:16 |
| odyssey4me | the proxy on the repo is configured with https://github.com/openstack/openstack-ansible-repo_server/blob/master/templates/acng.conf.j2 but you can disable it by settings this as false in your user_vars: https://github.com/openstack/openstack-ansible-repo_server/blob/master/defaults/main.yml#L42 | 18:18 |
| odyssey4me | I'm out for the w/end - cheers all! | 18:18 |
| rogervn | Thanks odyssey4me! That helped a lot! | 18:18 |
| rogervn | Have a nice weekend! | 18:19 |
| *** cjloader has joined #openstack-ansible | 18:31 | |
| *** KMT has joined #openstack-ansible | 18:39 | |
| KMT | Hello | 18:39 |
| KMT | https://bpaste.net/show/dc42da59b3d9 | 18:39 |
| KMT | I have two openstack-ansible setup. one ifra node and one compute node | 18:40 |
| KMT | now i am trying to add ceph stoarge | 18:40 |
| KMT | ceph cluster is install on separate nodes | 18:41 |
| mgariepy | KMT: ansible mon1 -m setup | 18:41 |
| mgariepy | does that work ? | 18:41 |
| KMT | try to fllow below link | 18:42 |
| KMT | https://www.openstackfaq.com/openstack-ansible-ceph/ | 18:42 |
| mgariepy | were you limiting the run ? | 18:42 |
| KMT | https://bpaste.net/show/dc42da59b3d9 | 18:42 |
| KMT | error message somehow complaiing like "has no attribute u'ansible_br_storage" | 18:43 |
| KMT | i am suspecting network connectivity | 18:43 |
| mgariepy | KMT, do you have a br-storage interface on this server ? | 18:44 |
| KMT | my infra and compute node as four vlan | 18:44 |
| KMT | no is br-storage is required on ceph cluster nodes ? | 18:44 |
| mgariepy | KMT:run : ansible mon1 -m setup | 18:45 |
| mgariepy | does that work ? | 18:45 |
| KMT | mgariepy> KMT, do you have a br-storage interface on this server ? which server ? ceph mon1 or osd ? | 18:45 |
| mgariepy | if it does, try re-running the other playbook. | 18:46 |
| KMT | no br-storage is availbe in ceph mon and osd. is it required that ceph node should have br-storage interface ? | 18:48 |
| KMT | no br-storage is NOT available | 18:48 |
| mgariepy | I think you have this error because ansible is tring to map info on mon1 and the fact is expired. doing the "ansible <host> -m setup" will refresh the fact then the other playbook should run correctly | 18:50 |
| *** electrofelix has quit IRC | 18:50 | |
| KMT | is it required that ceph node should have br-storage interface ? | 18:52 |
| mgariepy | I guess not. | 18:53 |
| mgariepy | the error shown in your pastebin is for mon1. | 18:53 |
| KMT | below is the out of mon1 -m setup | 18:54 |
| KMT | https://bpaste.net/show/1f2721d57a4b | 18:54 |
| mgariepy | what is in your /etc/openstack_deploy/openstack_user_config.yml ? | 18:56 |
| mgariepy | any mention of br-storage there? | 18:57 |
| *** dave-mccowan has joined #openstack-ansible | 18:58 | |
| KMT | https://bpaste.net/show/cd17d3df3205 | 18:58 |
| KMT | above is openstack_user_config.yml | 18:58 |
| mgariepy | line 57 | 18:58 |
| mgariepy | the config in opesntack_user_config.yml needs to match the configuraiton or your server. | 18:59 |
| KMT | yes i have br-storage on infra and compute node | 18:59 |
| KMT | i am confused how to fit ceph nodes network | 19:00 |
| mgariepy | https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/app-config-prod-ceph.html | 19:01 |
| KMT | thanks let me read and try it | 19:03 |
| mgariepy | starting at line 100, you define your storage_hosts, < https://bpaste.net/show/cd17d3df3205 | 19:03 |
| mgariepy | so compute01 and mon1 should have the br-storage network. | 19:03 |
| mgariepy | I do beleive app-config-prod-ceph is more up to date than the one on openstackfaq.com | 19:05 |
| mgariepy | KMT which version are you deploying ? | 19:07 |
| mgariepy | of ceph and openstack. | 19:07 |
| *** dave-mccowan has quit IRC | 19:08 | |
| *** armaan has joined #openstack-ansible | 19:14 | |
| *** armaan has quit IRC | 19:17 | |
| *** michelv has quit IRC | 19:18 | |
| *** dave-mccowan has joined #openstack-ansible | 19:21 | |
| *** hamza has joined #openstack-ansible | 19:22 | |
| KMT | ceph version 12 and openstack is pike | 19:24 |
| *** m|y|k has quit IRC | 19:29 | |
| mgariepy | ok | 19:32 |
| KMT | is the config m entioned in latest document https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/app-config-prod-ceph.html will be applicabe to pike version also ? | 19:41 |
| mgariepy | KMT which part? | 19:41 |
| mgariepy | KMT probably. | 19:42 |
| KMT | related to ceph config etc | 19:43 |
| mgariepy | probably. | 19:46 |
| *** dave-mccowan has quit IRC | 19:47 | |
| *** KMT has quit IRC | 19:49 | |
| *** savvas_ has joined #openstack-ansible | 19:59 | |
| *** savvas has quit IRC | 20:00 | |
| *** hamza has quit IRC | 20:04 | |
| *** ThomasSh has joined #openstack-ansible | 20:22 | |
| ThomasSh | hi | 20:22 |
| ThomasSh | just wanted to report back | 20:23 |
| ThomasSh | the deployment finished successfully :-) | 20:23 |
| rogervn | Hello, I´m having an issue with the setup-infrastructure playbook. I´ve configured user_variables to use proxy and so it succeeded in configuring the containers in setup-hosts playbook. | 20:29 |
| rogervn | But whenever I try the setup-infrastructure, the first container fails to set up pip because it doesn´t resolve the hostname, due to not using the http_proxy configured in /etc/environment. | 20:30 |
| rogervn | I´ve already attached directly to the failed container and everything is working, the proxy is working and /etc/enviroment loads up the variables correctly. For some reason, the ansible playbook shell is note using them. | 20:31 |
| rogervn | But it worked fine in setup-hosts to download pip packages and set up distro packages. | 20:32 |
| ThomasSh | hi rogervn | 20:37 |
| ThomasSh | i had the same error | 20:37 |
| ThomasSh | have done the install with direct inet access now | 20:37 |
| ThomasSh | never got it to work | 20:37 |
| rogervn | Yeah, that´s not an option for me... | 20:37 |
| ThomasSh | it looks like the system tries to use the proxy instead of pulling the pip from the repo server | 20:37 |
| rogervn | I´m fine by it using the proxy, but it doesn´t seem to be using it. Even with all the variables set up. | 20:38 |
| *** savvas_ has quit IRC | 20:39 | |
| *** savvas has joined #openstack-ansible | 20:40 | |
| *** dave-mccowan has joined #openstack-ansible | 20:45 | |
| *** savvas_ has joined #openstack-ansible | 20:57 | |
| *** savvas has quit IRC | 20:59 | |
| *** dave-mccowan has quit IRC | 21:00 | |
| *** savvas has joined #openstack-ansible | 21:10 | |
| *** savvas_ has quit IRC | 21:12 | |
| *** savvas has quit IRC | 21:23 | |
| *** pradiprwt has quit IRC | 21:49 | |
| *** cjloader has quit IRC | 21:59 | |
| *** savvas has joined #openstack-ansible | 22:06 | |
| *** savvas has quit IRC | 22:09 | |
| *** ianychoi has joined #openstack-ansible | 22:09 | |
| *** savvas has joined #openstack-ansible | 22:09 | |
| *** rogervn has quit IRC | 22:10 | |
| gun1x | what's the difference between stable/pike and 16.0.2? | 22:26 |
| gun1x | i guess 16.0.3 is still beta | 22:26 |
| gun1x | i keep getting confused about the versions and i never know which version is pike beta, which version is pike stable :D | 22:27 |
| gun1x | hmm i think i figured it out: https://github.com/openstack/openstack-ansible/releases | 22:28 |
| *** askb has joined #openstack-ansible | 22:31 | |
| gun1x | so, if i got this right: master is queens. stable/pike is the future 16.0.4... 16.0.3 is the latest release of stable/pike with consistent code. | 22:31 |
| *** savvas_ has joined #openstack-ansible | 22:41 | |
| *** savvas has quit IRC | 22:41 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible stable/pike: Revert "Add ceph_mons to ceph example" https://review.openstack.org/522903 | 22:45 |
| *** ThomasSh has quit IRC | 23:09 | |
| *** savvas_ has quit IRC | 23:14 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!