openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 00:01 |
---|---|---|
*** tosky has quit IRC | 00:14 | |
logan- | cloudnull: https://bugzilla.redhat.com/show_bug.cgi?id=1650342 :/ | 00:29 |
openstack | bugzilla.redhat.com bug 1650342 in systemd "systemd-networkd support in RHEL 8" [Unspecified,New] - Assigned to systemd-maint | 00:29 |
*** eumel8 has quit IRC | 00:57 | |
*** cshen has quit IRC | 01:07 | |
openstackgerrit | Merged openstack/openstack-ansible stable/queens: Move ARA install to end of bootstrap https://review.openstack.org/617786 | 01:10 |
openstackgerrit | Merged openstack/openstack-ansible stable/pike: Move ARA install to end of bootstrap https://review.openstack.org/617787 | 02:03 |
*** cshen has joined #openstack-ansible | 02:03 | |
*** cshen has quit IRC | 02:07 | |
openstackgerrit | brandon zhao proposed openstack/openstack-ansible-apt_package_pinning master: fix tox python3 overrides https://review.openstack.org/618935 | 02:39 |
openstackgerrit | brandon zhao proposed openstack/openstack-ansible-os_tacker master: fix tox python3 overrides https://review.openstack.org/618937 | 02:56 |
*** maddtux has joined #openstack-ansible | 03:27 | |
*** spatel has joined #openstack-ansible | 03:54 | |
spatel | cloudnull: ^^ | 03:54 |
spatel | are you there? | 03:54 |
spatel | what is wrong here? http://paste.openstack.org/show/735699/ | 03:55 |
spatel | everything is working fine.. but just noticed that when i was poking around some stuff to find out why horizon GUI is slow | 03:56 |
*** cshen has joined #openstack-ansible | 04:04 | |
*** spatel has quit IRC | 04:05 | |
*** udesale has joined #openstack-ansible | 04:09 | |
*** cshen has quit IRC | 04:09 | |
*** ivve has joined #openstack-ansible | 04:41 | |
*** chandankumar has joined #openstack-ansible | 04:55 | |
*** chandankumar is now known as chkumar|ruck | 04:56 | |
*** cshen has joined #openstack-ansible | 05:49 | |
openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_cloudkitty master: Configure cloudkitty-api to use uwsgi https://review.openstack.org/567168 | 05:57 |
*** cshen has quit IRC | 05:57 | |
*** radeks_ has joined #openstack-ansible | 05:59 | |
*** radeks_ has quit IRC | 06:00 | |
*** radeks_ has joined #openstack-ansible | 06:02 | |
*** czunker has joined #openstack-ansible | 06:13 | |
*** PrinzElvis has joined #openstack-ansible | 06:19 | |
*** chkumar|ruck has quit IRC | 06:37 | |
*** spsurya has joined #openstack-ansible | 06:39 | |
*** dkopper_ has joined #openstack-ansible | 06:57 | |
*** gkadam has joined #openstack-ansible | 07:02 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 07:06 |
*** chandankumar has joined #openstack-ansible | 07:14 | |
*** chandankumar is now known as chkumar|ruck | 07:14 | |
*** pcaruana has joined #openstack-ansible | 07:20 | |
*** cshen has joined #openstack-ansible | 07:25 | |
*** eumel8 has joined #openstack-ansible | 07:32 | |
*** pcaruana has quit IRC | 07:34 | |
*** dkopper__ has joined #openstack-ansible | 07:34 | |
*** dkopper__ has quit IRC | 07:37 | |
*** pcaruana has joined #openstack-ansible | 07:40 | |
*** dkopper_ has quit IRC | 07:50 | |
*** chandankumar has joined #openstack-ansible | 07:52 | |
*** shardy has joined #openstack-ansible | 07:55 | |
*** chkumar|ruck has quit IRC | 07:56 | |
*** chandankumar is now known as chkumar|ruck | 07:57 | |
*** rpittau has joined #openstack-ansible | 08:04 | |
*** dcdamien has joined #openstack-ansible | 08:06 | |
openstackgerrit | Frank Kloeker proposed openstack/openstack-ansible-haproxy_server master: Add feature Letsencrypt SSL certification https://review.openstack.org/586774 | 08:06 |
*** cshen has quit IRC | 08:42 | |
*** rodolof has joined #openstack-ansible | 08:44 | |
*** cshen has joined #openstack-ansible | 08:45 | |
*** ahosam has joined #openstack-ansible | 08:49 | |
*** priteau has joined #openstack-ansible | 08:54 | |
*** lbragstad has joined #openstack-ansible | 08:54 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 08:55 |
openstackgerrit | Erik Berg proposed openstack/openstack-ansible-os_neutron master: Apply provider-configuration for 'ml2.ovs.dvr' also https://review.openstack.org/618972 | 09:06 |
*** hamzaachi has joined #openstack-ansible | 09:09 | |
*** thuydang has joined #openstack-ansible | 09:09 | |
*** tosky has joined #openstack-ansible | 09:11 | |
*** DanyC has joined #openstack-ansible | 09:16 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 09:23 |
*** ahosam has quit IRC | 09:44 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 09:54 |
*** rodolof has quit IRC | 10:02 | |
*** chkumar246 has joined #openstack-ansible | 10:13 | |
*** ahosam has joined #openstack-ansible | 10:13 | |
*** chkumar|ruck has quit IRC | 10:15 | |
*** rodolof has joined #openstack-ansible | 10:16 | |
*** chkumar246 is now known as chkumar|ruck | 10:18 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 10:21 |
*** thuydang has quit IRC | 10:23 | |
*** hamzaachi has quit IRC | 10:26 | |
openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_cloudkitty master: Configure cloudkitty-api to use uwsgi https://review.openstack.org/567168 | 10:31 |
odyssey4me | o/ been sick since returning from berlin... morning all | 10:32 |
openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/queens: Add libpython2.7 as a required package. https://review.openstack.org/618779 | 10:34 |
*** nowster has joined #openstack-ansible | 10:34 | |
openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts stable/rocky: Add missing variable for nspawn networking https://review.openstack.org/616159 | 10:42 |
dcdamien | morning odyssey4me, I was right -> http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2018-11-19.log.html#t2018-11-19T14:30:44 | 10:45 |
*** chkumar246 has joined #openstack-ansible | 10:54 | |
*** chkumar|ruck has quit IRC | 10:57 | |
*** czunker has quit IRC | 11:00 | |
*** fghaas has joined #openstack-ansible | 11:00 | |
fghaas | odyssey4me (and cloudnull if you're awake already), you'll recall the discussion that we had in Berlin about not setting wsrep_sync_wait=1 (formerly wsrep_causal_reads=on) for Galera — see http://galeracluster.com/documentation-webpages/mysqlwsrepoptions.html#wsrep-sync-wait for details on that option. | 11:06 |
fghaas | I think I just thought up a scenario where this could get really, really painful and I wonder if you agree: haproxy directs mysql clients to a single backend node, because the others are set to backup. We do a simple tcp check there. Now, suppose our connections from mysql clients hover just around max_connections, so the primary backend's mysqld flaps between accepting and refusing connections. Wouldn't that mean that we woul | 11:06 |
fghaas | d be getting alternate writer nodes, causing all sorts of havoc with wsrep-sync-wait=0? | 11:06 |
evrardjp | can someone point me to the etherpad used in the summit's for "Deployment tools feedback" ? I can't find it. mnaser you were there, right? | 11:07 |
mnaser | yes i was there | 11:07 |
mnaser | i meant to send a summary soon | 11:07 |
mnaser | evrardjp: https://etherpad.openstack.org/p/BER-deployment-tools | 11:08 |
* mnaser goes back to starting day | 11:08 | |
*** ahosam has quit IRC | 11:08 | |
*** cshen has quit IRC | 11:08 | |
*** ahosam has joined #openstack-ansible | 11:08 | |
*** admin0 has quit IRC | 11:09 | |
nowster | jrosser: I now have blank Ubuntu 18.0.1 server to test your patches on. | 11:10 |
*** udesale has quit IRC | 11:11 | |
dcdamien | What's wrong with Zuul? Anybody knows? | 11:14 |
mnaser | what issues you having dcdamien ? | 11:17 |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 11:17 |
jrosser | nowster: nice - i've been prodding at this a bit more https://review.openstack.org/#/c/618305/ | 11:18 |
*** cshen has joined #openstack-ansible | 11:20 | |
dcdamien | mnaser: timeouts, not just on my checks | 11:20 |
nowster | jrosser: ta. | 11:21 |
dcdamien | mnaser: | 11:21 |
dcdamien | https://review.openstack.org/#/c/618822/ | 11:21 |
dcdamien | https://review.openstack.org/#/c/617785/ | 11:21 |
dcdamien | https://review.openstack.org/#/c/617053/ | 11:21 |
mnaser | dcdamien: thats weird. | 11:22 |
mnaser | did we recently merge something that affected things? | 11:22 |
evrardjp | fghaas: interesting use case, but the idea was to move out from tcp check and use real checks with a proper endpoint, but I am not fully aware of the situation you're talking about (branch and other technical details) | 11:23 |
dcdamien | maybe something with moving ARA to the end of bootstrap? | 11:24 |
evrardjp | fghaas: I will read more about it. Thanks for pointing to this | 11:24 |
evrardjp | dcdamien: queens had a timeout in installing galera | 11:25 |
evrardjp | I'd say there are a few things that might have slowed-down and need further/deeper analysis | 11:26 |
evrardjp | ty mnaser for the link | 11:27 |
evrardjp | but thats the comparison | 11:27 |
evrardjp | I am thinking the other session | 11:28 |
evrardjp | i cant really remember it though.. | 11:28 |
*** chkumar246 is now known as chkumar|ruck | 11:30 | |
*** rpittau_ has joined #openstack-ansible | 11:31 | |
*** rpittau has quit IRC | 11:34 | |
*** rpittau_ is now known as rpittau | 11:34 | |
dcdamien | evrardjp: when there is `TIMED_OUT` on check that it means that whole check was aborted because it took too long? | 11:35 |
*** ahosam has quit IRC | 11:44 | |
*** ahosam has joined #openstack-ansible | 11:44 | |
*** ahosam has quit IRC | 11:46 | |
jrosser | nowster: i have been looking at how we get your haproxy bug fixed upstream, would you be able to help out with a test case here? https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/1804069 | 11:47 |
openstack | Launchpad bug 1804069 in haproxy (Ubuntu) "haproxy fails on arm64 due to alignment error" [Undecided,Incomplete] | 11:47 |
nowster | jrosser: hmm... Let me think. | 11:48 |
jrosser | i expect they want a super simple config example and a curl that shows the zero length response | 11:50 |
nowster | yep... I can do | 11:51 |
jrosser | cool, thanks | 11:51 |
*** chkumar246 has joined #openstack-ansible | 11:53 | |
nowster | jrosser: of course it's not breaking on the tests I'm doing... | 11:55 |
*** chkumar|ruck has quit IRC | 11:56 | |
*** maddtux_ has joined #openstack-ansible | 11:56 | |
jrosser | nowster: of course :) | 11:56 |
*** gkadam has quit IRC | 11:56 | |
*** maddtux has quit IRC | 11:56 | |
*** pabelanger has left #openstack-ansible | 11:58 | |
*** noonedeadpunk[h] is now known as noonedeadpunk | 12:00 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 12:03 |
odyssey4me | well, that's nice - basically every single version of OSA we've released can no longer bootstrap without adjusting a-r-r: https://github.com/willshersystems/ansible-sshd/issues/98 | 12:04 |
odyssey4me | we've always used the x.y.z version tag instead of the vx.y.z tag, and they've changed to the latter | 12:05 |
*** gkadam has joined #openstack-ansible | 12:14 | |
guilhermesp | morning osa team :) | 12:14 |
openstackgerrit | git-harry proposed openstack/openstack-ansible stable/pike: Fix ansible-sshd version https://review.openstack.org/618995 | 12:14 |
*** aiyengar_ has joined #openstack-ansible | 12:15 | |
*** chkumar has joined #openstack-ansible | 12:15 | |
*** maddtux_ has quit IRC | 12:17 | |
*** maddtux has joined #openstack-ansible | 12:18 | |
*** chkumar246 has quit IRC | 12:18 | |
dcdamien | odyssey4me: can you please review https://review.openstack.org/#/c/617847/ ? | 12:21 |
*** aiyengar_ has quit IRC | 12:21 | |
openstackgerrit | git-harry proposed openstack/openstack-ansible stable/pike: Fix ansible-sshd version https://review.openstack.org/618995 | 12:22 |
jamesdenton | good morning | 12:22 |
*** asettle has joined #openstack-ansible | 12:25 | |
*** priteau has quit IRC | 12:28 | |
*** ahosam has joined #openstack-ansible | 12:35 | |
evrardjp | dcdamien: yes | 12:41 |
evrardjp | wow git-harry proposed something | 12:41 |
evrardjp | clickbait | 12:41 |
*** ahosam has quit IRC | 12:44 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Fix nova-lxd tests for bionic. https://review.openstack.org/616302 | 12:47 |
*** rodolof has quit IRC | 12:52 | |
*** chkumar246 has joined #openstack-ansible | 12:54 | |
*** chkumar246 is now known as chkumar|ruck | 12:54 | |
*** chkumar has quit IRC | 12:57 | |
*** maddtux has quit IRC | 13:00 | |
*** mcela1 has joined #openstack-ansible | 13:01 | |
mcela1 | Hello, anyone has experience with setting up networks for tenants using vlans? or is there any guide specifically targeting this? | 13:02 |
openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts stable/rocky: Add host only network option https://review.openstack.org/616160 | 13:04 |
jrosser | mcela1: start here with the bit of config that talks about "eth12" https://docs.openstack.org/openstack-ansible/rocky/user/prod/example.html | 13:05 |
nowster | jrosser: I've not forgotten the haproxy stuff. I'm bootstrapping openstack-ansible to the point where it fails, then I'll turn things off selectively until I have a minimal config that still breaks. | 13:05 |
*** DRoBeR has joined #openstack-ansible | 13:06 | |
jrosser | nowster: no worries :) | 13:06 |
*** ahosam has joined #openstack-ansible | 13:10 | |
*** priteau has joined #openstack-ansible | 13:10 | |
*** ahosam has quit IRC | 13:10 | |
mcela1 | jrosser: i have tried that configuration before. I can correctly create a provider network of type "vlan" with a given segmentation ID, but when i try to ping the gateway either from an instance, or from the network namespace of the router, the ARP responses of the gateway do not make it back to the instance/router namespace. | 13:11 |
*** ahosam has joined #openstack-ansible | 13:11 | |
*** strattao has joined #openstack-ansible | 13:12 | |
mcela1 | what is even better is that if i do a tcpdump of the bonded interface, i can see ARP requests from the instance/router namespace leaving (and being correctly vlan tagged), but when they come back from the gateway, they are "dropped" and i can't find why this happens | 13:12 |
*** ahosam has quit IRC | 13:15 | |
jamesdenton | mcela1 can you share the output of 'brctl show' on the controller/network/compute nodes and the output of 'openstack network show' for the network involved? Let's make https://pasted.tech/pastes a thing. | 13:15 |
mcela1 | absolutely | 13:16 |
openstackgerrit | Merged openstack/openstack-ansible master: [trivial] Point to the new openstack-discuss ML https://review.openstack.org/618758 | 13:17 |
mcela1 | jamesdenton: The output of brctl show is as follows: https://pasted.tech/pastes/7a6a073a08cc8e522143ca9d381eeb3b44715b82 | 13:17 |
jamesdenton | Thanks. | 13:18 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 13:19 |
jamesdenton | mcela1 Do you happen to know if you have a native vlan set on the bond0 interfaces on the switch/ | 13:22 |
jamesdenton | and have you confirmed the interfaces are configured as a trunk and not access port | 13:22 |
openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Add ansible architecture mapping for arm64 https://review.openstack.org/618500 | 13:22 |
jamesdenton | and what is vlan 1324? | 13:23 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 13:27 |
dcdamien | Are this kind of changes candidates for backport? https://review.openstack.org/#/c/617847/ | 13:33 |
mcela1 | jamesdenton: as far as i know the native vlan is not the 1324 (the one i am trying to use for tenants). The interfaces are correctly configured, as traffic leaves tagged and we have routed access to them for the configured vlans. We suspect a bad configuration on interfaces, but have tried several reinstalls with different configurations and can't get it to work. | 13:35 |
jamesdenton | mcela1 i noticed that bond0.1324 is in br-vxlan bridge, and vlan 1324 is also used for one of the neutron networks. That won't work and could be a source of trouble if that's the vlan you're working with | 13:36 |
jamesdenton | you'll have to either change the vlan used for vxlan overlay networks, or delete and recreate the neutron network and use a different vxlan id | 13:37 |
mcela1 | we will try to reinstall again with that configuration and will get back with results. Thank you very much!! | 13:41 |
openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Configure cloudkitty-api to use uwsgi https://review.openstack.org/567168 | 13:41 |
jamesdenton | a reinstall may be scorching the earth here, but feel free and let us know! | 13:41 |
odyssey4me | evrardjp jrosser wrt to https://review.openstack.org/613078 - one could debate efficiency versus explicit dependencies... I find that the 'efficient' approach of making a package install only once results in forgetting why it's there and it never being cleaned up when there is no longer a dep... versus rather making it explicitly install where it is required (in each role which requires it) so that the role is self sufficient and it | 13:43 |
odyssey4me | also gets cleaned up if it's not required | 13:43 |
odyssey4me | there are some exceptions, of course - that's why we have things like https://github.com/openstack/ansible-role-python_venv_build/commit/c54aa8117e79dd9b37b61189223eb4a08ad5e2a0 and may also need something in openstack_hosts | 13:44 |
jrosser | my preference would be to make the lxc_hosts package list as small as possible | 13:45 |
odyssey4me | jrosser agreed | 13:49 |
*** chkumar246 has joined #openstack-ansible | 13:49 | |
openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 13:50 |
*** andymccr has joined #openstack-ansible | 13:51 | |
*** chkumar|ruck has quit IRC | 13:52 | |
*** ansmith has joined #openstack-ansible | 13:56 | |
openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Add ansible architecture mapping for arm64 https://review.openstack.org/618523 | 13:57 |
*** savvas_ has joined #openstack-ansible | 13:58 | |
evrardjp | odyssey4me: I am fine with cleaning it up, and if necessary move it to openstack_hosts | 13:59 |
evrardjp | jrosser: ^ | 13:59 |
evrardjp | I'd rather be explicit | 13:59 |
evrardjp | and efficiency can come from openstack_hosts | 13:59 |
*** chkumar246 has quit IRC | 14:01 | |
openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 14:05 |
openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 14:08 |
*** eumel8 has quit IRC | 14:11 | |
openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 14:27 |
*** strattao has quit IRC | 14:29 | |
openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Add Neutron VPNaaS Dashboard to Horizon https://review.openstack.org/617847 | 14:37 |
*** DanyC has quit IRC | 14:39 | |
*** DanyC has joined #openstack-ansible | 14:40 | |
*** DanyC has quit IRC | 14:41 | |
*** DanyC has joined #openstack-ansible | 14:43 | |
noonedeadpunk | Can you please check this cherry-pick? https://review.openstack.org/#/c/616988/ | 14:46 |
*** DanyC has quit IRC | 14:47 | |
openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/rocky: Add libpython2.7 as a required package https://review.openstack.org/618783 | 14:52 |
*** nowster has quit IRC | 14:52 | |
*** chkumar246 has joined #openstack-ansible | 14:53 | |
evrardjp | I generetate the bugtraige page, could people update today meeting's agenda | 14:55 |
evrardjp | wow | 14:55 |
evrardjp | I generated the bugtriage etherpad, could people have a look at our wiki page for meetings, to ensure the agenda is up to date? :D <-- more english | 14:56 |
*** jcosmao has joined #openstack-ansible | 14:59 | |
*** udesale has joined #openstack-ansible | 15:05 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Fix nova-lxd tests for bionic. https://review.openstack.org/616302 | 15:06 |
*** nowster has joined #openstack-ansible | 15:09 | |
jrosser | evrardjp: i really struggled with what to do with those nova-lxd vars - theres two different pool concepts and at least two bugs to work around...... | 15:11 |
evrardjp | jrosser: I think you did good, my only point for a negative vote was the inability for me to grok the contents of the reno :) | 15:12 |
evrardjp | else we can keep it that way (the test var part I mean) | 15:12 |
evrardjp | jrosser: thanks for tackling this! | 15:12 |
arxcruz | evrardjp: hey, regarding https://review.openstack.org/#/c/618738 did you saw my comments? | 15:12 |
arxcruz | i really don't know how to do that only on os_tempest, since all os_tempest are configured on osa-tests and on os_tempest has only the templates | 15:13 |
jrosser | evrardjp: once the upstream bugs in nova-lxd are ironed out those vars should not be needed in the test any more - that will make things more obvious | 15:14 |
evrardjp | arxcruz: I did not yet | 15:14 |
evrardjp | arxcruz: could we discuss this a little later, I am in a few meetings :) | 15:14 |
arxcruz | evrardjp: sure | 15:15 |
evrardjp | jrosser: oh great | 15:15 |
*** mcela1 has left #openstack-ansible | 15:17 | |
openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts stable/rocky: Modify the network to ensure their more restrictive https://review.openstack.org/616161 | 15:22 |
*** jproulx has joined #openstack-ansible | 15:23 | |
spotz | Back in action if needed! | 15:24 |
evrardjp | arxcruz: https://github.com/openstack/openstack-ansible-os_neutron/blob/master/zuul.d/jobs.yaml#L16-L20 is an example for neutron. The var you're passing is then used for this: https://github.com/openstack/openstack-ansible-os_neutron/blob/master/tox.ini#L105-L111 | 15:25 |
evrardjp | ofc it means you need to create your own override yml file to trigger a part of the code. | 15:26 |
odyssey4me | spotz here are some backports which just need verification that they match master - https://review.openstack.org/#/q/topic:nspawn-rocky-fixes+status:open | 15:26 |
arxcruz | evrardjp: ok, thanks, i'll take a look at this | 15:26 |
evrardjp | The idea behind that was to make sure developers can run the same things as gates | 15:26 |
odyssey4me | hopefully with all those in, nspawn will be in a working state for rocky | 15:26 |
arxcruz | odyssey4me: hey, what was the name of the guy working on the zuul api ? | 15:26 |
spotz | odyssey4me: Ok I'll start going through them | 15:27 |
chkumar246 | arxcruz, do we want to run python-tempestconf and stackviz from rpm in centos-distro job? | 15:27 |
evrardjp | however, you're giving me ideas on how to change that -- we should probably consume said variables and template them for direct consumption. Deployers could just run the same kind of things. | 15:27 |
odyssey4me | arxcruz hmm, my brain's only half here (due to 'flu) - maybe evrardjp will remember - he's from aus | 15:27 |
evrardjp | but for the moment, let's just do it this way | 15:27 |
evrardjp | Josh? | 15:27 |
arxcruz | chkumar246: eventually | 15:27 |
evrardjp | Josh Hesketh? | 15:27 |
arxcruz | evrardjp: some guy was doing some work/research to have a rest api in zuul to return the playbook and all variables for reproduce | 15:28 |
evrardjp | https://review.openstack.org/#/c/607077/ | 15:28 |
evrardjp | arxcruz: ^ | 15:28 |
odyssey4me | evrardjp yep, that's the guy! | 15:28 |
odyssey4me | jhesketh :) | 15:28 |
spotz | odyssey4me: Did you start getting sick on Wednesday? | 15:28 |
evrardjp | odyssey4me: I gave the link of the WIP, even better! ;) | 15:29 |
odyssey4me | spotz not sure when, but it hit me hard on friday | 15:29 |
jrosser | chkumar246: we may have a bit of tempest brok here in our barbican role.... is that an easy fix somewhere? - http://logs.openstack.org/36/614336/2/check/openstack-ansible-functional-distro_install-ubuntu-bionic/386a134/job-output.txt.gz#_2018-11-19_20_59_40_832895 | 15:29 |
odyssey4me | oh nice - I see he's got WIP for a runner too :) https://review.openstack.org/#/c/607082/1 | 15:29 |
spotz | odyssey4me: I started a sore throat on Wednesday, then the nose on Thursday. evrardjp and kmarc came to the rescue with meds | 15:30 |
chkumar246 | jrosser, checking | 15:30 |
odyssey4me | spotz yep, sore throat was how it started on fri - then full blown 'flu over the weekend for me | 15:30 |
spotz | OSA flu! | 15:30 |
arxcruz | evrardjp: i own you more jagermaster | 15:31 |
evrardjp | own or owe? | 15:31 |
nowster | yuk! I developed "post nasal drip" after a heavy cold. | 15:31 |
nowster | And that wasn't "Conference Flu" either. | 15:31 |
evrardjp | folks you have to be sick BEFORE the conference, so you can give it better to your friends! | 15:32 |
evrardjp | :p | 15:32 |
arxcruz | evrardjp: next time you be in berlim, you have one shot! :) | 15:32 |
chkumar246 | jrosser, on puppet-side we came to know that ubuntu does not maintain tempest so it is better to run tempest from source | 15:33 |
jrosser | chkumar246: actually i linked you a nv distro install job there, but they all look unhappy | 15:33 |
chkumar246 | jrosser, https://review.openstack.org/615514 | 15:33 |
chkumar246 | jrosser, I am taking a look let me see how I can fix it | 15:34 |
openstackgerrit | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: Add centos-7 job with support to python-tempestconf https://review.openstack.org/619021 | 15:38 |
cloudnull | mornings all | 15:40 |
mgariepy | morning cloudnull | 15:40 |
cloudnull | o/ | 15:40 |
evrardjp | arxcruz: I forgot to say that those jobs should be added in jobs.yaml, and you should add said job name into projects :) | 15:41 |
*** dave-mccowan has joined #openstack-ansible | 15:41 | |
arxcruz | evrardjp: yes, i just realize that | 15:41 |
arxcruz | working on the patch :) | 15:41 |
*** cshen has quit IRC | 15:43 | |
*** hamzaachi has joined #openstack-ansible | 15:43 | |
openstackgerrit | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: Add centos-7 job with support to python-tempestconf https://review.openstack.org/619021 | 15:44 |
evrardjp | arxcruz: if you put [WIP] in the title, I won't bother you :p | 15:45 |
openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Added task to list tempest tests https://review.openstack.org/619024 | 15:45 |
openstackgerrit | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: [WIP] - Add centos-7 job with support to python-tempestconf https://review.openstack.org/619021 | 15:45 |
chkumar246 | jrosser, ^^ a proactive approach to find which went wrong here | 15:45 |
evrardjp | chkumar246 arxcruz thanks for the work done in tempest, I haven't said it enough! :p | 15:46 |
arxcruz | evrardjp: now you own me a jagermaister | 15:47 |
arxcruz | whatever it writes | 15:47 |
mnaser | cloudnull, DimGR, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum, darrenc, JRobinson__, colinmcnamara, thorst, adreznec, eil397, | 15:53 |
mnaser | qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, mnaser, nicolasbock, jrosser, cjloader, antonym, dcdamien | 15:53 |
evrardjp | jagermeister :) | 15:53 |
*** spatel has joined #openstack-ansible | 15:53 | |
mnaser | 7 minute warning :) | 15:53 |
spatel | cloudnull: morning! | 15:53 |
evrardjp | mnaser: thanks! | 15:53 |
chkumar246 | jrosser, the issue with the above failed part is tempest failed to find that tests | 15:53 |
spatel | is meeting going to start? | 15:53 |
* cloudnull goes to get coffee | 15:53 | |
spatel | so i don't spam you ... | 15:53 |
chkumar246 | jrosser, above patch will clearly say whether it is discovered or not | 15:54 |
spatel | i have this question if its easy to answer for you http://paste.openstack.org/show/735699/ | 15:54 |
chkumar246 | jrosser, can we add a depends on your patch? | 15:54 |
chkumar246 | with my patch | 15:54 |
cloudnull | spatel - try to stop/start_app else restart the rabbitmq container. | 15:56 |
spatel | but i wonder how everything is working... | 15:56 |
spatel | even rabbitmq is broken | 15:57 |
jrosser | chkumar246: sure | 15:57 |
cloudnull | spatel its likely only broken on that node | 15:57 |
spatel | i am seeing same behavior on all 3 nodes | 15:58 |
spatel | same error | 15:58 |
cloudnull | oh then everything is not working :) | 15:58 |
cloudnull | can you restart the rabbitmq process ? | 15:58 |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_barbican master: Remove unnecessary package install duplication https://review.openstack.org/614336 | 15:59 |
spatel | hmm! i think this is the behavior from beginning and i never paid attention... | 15:59 |
spatel | i can build machine launch etc.. | 15:59 |
spatel | let me restart R | 16:00 |
openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Use loop_var name in when clause https://review.openstack.org/616595 | 16:00 |
mnaser | damn woops | 16:00 |
mnaser | i always do this | 16:00 |
mnaser | #startmeeting openstack_ansible_meeting | 16:00 |
openstack | Meeting started Tue Nov 20 16:00:59 2018 UTC and is due to finish in 60 minutes. The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
spatel | just checked all 3 has same error | 16:01 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
*** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:01 | |
openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:01 |
mnaser | #topic roll call | 16:01 |
mnaser | o/ | 16:01 |
*** openstack changes topic to "roll call (Meeting topic: openstack_ansible_meeting)" | 16:01 | |
spatel | cloudnull: i will talk to you after this meeting :) | 16:01 |
mnaser | laggy bot | 16:01 |
mnaser | o/ | 16:01 |
cloudnull | o/ | 16:01 |
jrosser | o/ | 16:01 |
mnaser | quiet one | 16:02 |
evrardjp | o/ | 16:02 |
mnaser | okay so lets get started | 16:02 |
mnaser | #topic what happened since last meeting | 16:02 |
*** openstack changes topic to "what happened since last meeting (Meeting topic: openstack_ansible_meeting)" | 16:02 | |
mnaser | not much, i think. we were mostly all having a great time in berlin :) i had a really good deployemnt tool feedback session | 16:02 |
mnaser | i had a few action items that i carried out | 16:02 |
mnaser | also, project on-boarding and project update were amazing | 16:02 |
mnaser | clearly there is an amazing amonut of interest in OSA, so let's all keep up on that awesome work :D super motivating | 16:03 |
chkumar246 | arxcruz, thanks for suggestion it makes sense, I will update once dependent patch finishes the output | 16:03 |
mnaser | other things: we're working on os_placement and hopefully we'll land that eventually | 16:04 |
mnaser | it's cool because we're kinda one of the leaders in that so it would be nice for OSA to be "FIRST!" | 16:04 |
*** savvas_ has quit IRC | 16:04 | |
mnaser | other than that, jrosser brought up arm64 so maybe want to expand on that | 16:04 |
cloudnull | ++ | 16:04 |
jrosser | so, out of mostly curiosity i had a go on the new arm64 ci nodes | 16:05 |
odyssey4me | :) | 16:05 |
jrosser | and then entirely coincidentally nowster showed up here in the channel trying to do it for real | 16:05 |
mnaser | jrosser: if you need any support on arm stuff, i can get people your way :) | 16:05 |
jrosser | so we've been just poking at that a bit and seeing how far it gets | 16:05 |
jrosser | theres a very obvious and now fixed bug in haproxy | 16:06 |
mnaser | if you are going to be doing some work on that, i'd love to make an introduction to massimo who leads a lot of the open source / openstack stuff and he is a really really useful asset in terms of maybe even getting you resources to accomlpish the things you need | 16:06 |
jrosser | that would be great | 16:06 |
mnaser | wonderful, ill send that soon | 16:06 |
jrosser | the other thing is that the performance of the arm64 nodes isnt great, so we need someone to help take a look at that | 16:06 |
jrosser | i suspect the storage is slow | 16:07 |
mnaser | perfect, he'll actually be able to help with all that stuff | 16:07 |
evrardjp | there are other ppl you might want to reach, like linaro too, should you require some help | 16:07 |
jrosser | for anyone who is interested here is the test patch https://review.openstack.org/#/c/618305/ | 16:08 |
mnaser | also, i might be able to get you some.. faster resources for local testing | 16:08 |
mnaser | will follow up more in that email | 16:08 |
jrosser | cool :) | 16:08 |
mnaser | anything else from last week? | 16:09 |
guilhermesp | late o/ | 16:09 |
mnaser | :) | 16:09 |
guilhermesp | so I think you noticed that Ive been working on os_placement | 16:10 |
guilhermesp | here's the pr https://review.openstack.org/#/c/618820/ | 16:10 |
guilhermesp | I'm still building the basics of the role, but I still need to create the tests | 16:10 |
guilhermesp | seems that the placement api works under apache + uswgi | 16:11 |
guilhermesp | mnaser: mentioned that there is also red hat packages for distro install | 16:11 |
guilhermesp | is that right? | 16:11 |
mnaser | yes, rdo apparently publishment placement packages already | 16:11 |
*** udesale has quit IRC | 16:11 | |
guilhermesp | at the moment I'm focusing in source install | 16:12 |
guilhermesp | but good to know | 16:12 |
mnaser | yep lets get that and we can improve/iterate | 16:12 |
mnaser | anything else or shall we move onto bug triage? | 16:12 |
cloudnull | just a quick note about the summit -- odyssey4me and I were in fhaas training session (which was great!), if any folks in the channel here were in that session and want to follow up on some of the things we talked about in the all day session please reach out. | 16:12 |
mnaser | s/fhaas/fghaas/ you're welcome :p | 16:13 |
cloudnull | ^ | 16:13 |
cloudnull | :) | 16:13 |
mnaser | #topic bug triage | 16:13 |
*** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)" | 16:13 | |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1804204 | 16:13 |
openstack | Launchpad bug 1804204 in openstack-ansible "Quickstart: AIO in openstack-ansible ubuntu instructions" [Undecided,New] | 16:13 |
mnaser | yep | 16:14 |
mnaser | thats confirmed | 16:14 |
*** hamzy has quit IRC | 16:14 | |
evrardjp | valid | 16:14 |
evrardjp | low-hanging-fruit | 16:14 |
evrardjp | low ? | 16:14 |
mnaser | i'd say medium considering .. its broken | 16:15 |
cloudnull | something like this came up in the project onboarding ++ Id say medium | 16:15 |
mnaser | asked if paul can push up a patch | 16:15 |
mnaser | if not we can figure things out :) | 16:16 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1803444 | 16:16 |
openstack | Launchpad bug 1803444 in openstack-ansible "gnocchi playbook failing on 18.0.0" [Undecided,New] | 16:16 |
*** chkumar246 has quit IRC | 16:16 | |
mnaser | include vs import_playbook black magic | 16:16 |
mnaser | does anyone knows why the above might occur? | 16:16 |
mnaser | seems related to ceph_client too | 16:17 |
evrardjp | checking | 16:18 |
mnaser | wait is it because the include is include-ing it regardless? | 16:18 |
mnaser | because it looks like that deployer is using ceph | 16:18 |
mnaser | so that block should be entirely ignored | 16:18 |
evrardjp | I tried to avoid that playbook for a while :p | 16:18 |
* cloudnull reading | 16:18 | |
noonedeadpunk | hm, never faced with this one... | 16:18 |
nowster | mnaser: Just to let you know that I'm working with "Works On ARM" too (ie. Massimo). | 16:19 |
mnaser | nowster: oh awesome! | 16:19 |
cloudnull | fatal: [infra1_gnocchi_container-7ed445f6]: FAILED! => {"msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"} | 16:19 |
evrardjp | we should probably adapt the play to only run the appropriate role if a grouping of host was conditionally successfull | 16:19 |
mnaser | that fits perfectly then :) | 16:19 |
cloudnull | is generally an issue with facts | 16:19 |
cloudnull | like the OS was not found or there were no valid facts and ansible didnt regather | 16:19 |
evrardjp | cloudnull: I agree on the ceph client part | 16:19 |
mnaser | yeah so i think no facts were gathered for that because no tasks actually ran on the gnocchi container | 16:20 |
mnaser | at least in that specific section that it was breaking on | 16:20 |
mnaser | which was the setup-things workaround | 16:20 |
cloudnull | as for the "when" in the import playbook. | 16:20 |
cloudnull | im not sure that actually works any longer. | 16:20 |
evrardjp | we should maybe ask for specific runs | 16:20 |
evrardjp | cloudnull: yeah my point exactly | 16:21 |
evrardjp | cloudnull: well tbh I am not sure WHEN it worked :p | 16:21 |
cloudnull | IDK if import_playbook allows for vars to be passed through | 16:21 |
evrardjp | double pun inside! | 16:21 |
mnaser | cause an import isn't really meant to be conditional right? | 16:21 |
evrardjp | cloudnull: it was never intended for that | 16:21 |
cloudnull | ++ | 16:21 |
evrardjp | mnaser: right | 16:21 |
cloudnull | it worked with include because we took advantage of the overloaded include system | 16:21 |
evrardjp | yeah | 16:22 |
mnaser | is there include_playbook ..? | 16:22 |
evrardjp | nope | 16:22 |
evrardjp | only import | 16:22 |
mnaser | so what we need to do is move that logic into os-gnocchi right? | 16:22 |
evrardjp | but as I said | 16:22 |
cloudnull | include will load whatever it finds while import is more like python. | 16:22 |
evrardjp | we can move some part into the play | 16:22 |
evrardjp | and include_role | 16:22 |
cloudnull | ++ | 16:22 |
evrardjp | or the meta to end the playbook early | 16:23 |
evrardjp | (but I'd rather avoid that one) | 16:23 |
evrardjp | for the sake of the bug, I'd like to see a real run of gnocchi | 16:23 |
mnaser | is there a quick 2 minute explanation | 16:23 |
mnaser | as to why this even happens? | 16:23 |
evrardjp | to see if it happens again | 16:23 |
evrardjp | I mean of the gnocchi playbook | 16:24 |
mnaser | also noonedeadpunk mentions he hasnt ran into that.. do you use ceph? | 16:24 |
mnaser | do we wanna dig into this again next week? | 16:25 |
evrardjp | mnaser: that sounds reasonable | 16:26 |
noonedeadpunk | mnaser: yep, I'm running ceph | 16:26 |
francois | hi guys, I'm the author of this bug report and am available if you need any other info :) | 16:26 |
evrardjp | noonedeadpunk: with gnocchi? | 16:26 |
evrardjp | francois: oh great! | 16:26 |
evrardjp | perfect! | 16:26 |
openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Correct clustered gating job https://review.openstack.org/619033 | 16:26 |
evrardjp | francois: could you run just that gnocchi play again? | 16:26 |
evrardjp | openstack-ansible os-gnocchi-install.yml | 16:27 |
evrardjp | it should not fail :p | 16:27 |
francois | evrardjp: ok | 16:27 |
noonedeadpunk | yep. but, I'm not running swift... | 16:27 |
evrardjp | if you put the log somewhere that would be nice | 16:27 |
evrardjp | noonedeadpunk: that's the deal :) | 16:27 |
noonedeadpunk | And now I'm wondering, how I get it installed at all... | 16:27 |
evrardjp | francois: because if it repeats itself with that case, it's not a facts problem, but a playbook problem | 16:28 |
noonedeadpunk | oh, we're including gnocchi role twice... | 16:29 |
*** aedc has quit IRC | 16:29 | |
mnaser | francois: can you please update that bug with the info | 16:29 |
mnaser | and we'll cotninue | 16:29 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1803382 | 16:29 |
openstack | Launchpad bug 1803382 in openstack-ansible "unbound-client.yml failure depending on TERMCAP envvar content" [Undecided,New] | 16:29 |
*** aedc has joined #openstack-ansible | 16:29 | |
francois | mnaser: yeah, will do that | 16:29 |
mnaser | i ran into that bug too | 16:29 |
mnaser | it happened when i was running something in screen | 16:30 |
mnaser | i had *no* idea why it was happening | 16:30 |
francois | also mine :) | 16:30 |
mnaser | but it didn't ahppen anymore in screen | 16:30 |
francois | this one looks pretty weird! | 16:30 |
mnaser | i have no idea why it honestly happens | 16:30 |
evrardjp | I was told that jrosser had the same issue | 16:30 |
evrardjp | we can probably say confirmed then | 16:31 |
francois | an easy workaround is simply deleting this variable from /etc/openstack_deploy/ansible_facts/localhost | 16:31 |
dcdamien | I have the same issue too | 16:31 |
evrardjp | we should unset said thing in the connection plugin | 16:31 |
francois | but I'm quite interested in how it could happen :) | 16:31 |
mnaser | i will put this as confirmed/high | 16:31 |
mnaser | running things in screen is kinda.. important. | 16:31 |
evrardjp | tmux? | 16:32 |
evrardjp | :p | 16:32 |
noonedeadpunk | + | 16:32 |
evrardjp | never had an issue with tmux :p | 16:32 |
evrardjp | hahaha | 16:32 |
evrardjp | anyway | 16:32 |
guilhermesp | byobu rules | 16:32 |
evrardjp | yeah | 16:32 |
cloudnull | tmux for life! | 16:32 |
evrardjp | important | 16:32 |
mnaser | francois: do you have sometime to dig a little bit into that one? | 16:32 |
guilhermesp | byobu = tmux lover too | 16:32 |
evrardjp | I digged through a rabbit hole long ago with a rabbitmq issue of the env var that was passed | 16:32 |
evrardjp | but we changed our connection plugin now, and we are not relying on the same behaviours anymore | 16:33 |
evrardjp | so you're hitting an issue in ansible itself | 16:33 |
evrardjp | probably | 16:33 |
*** dcdamien has quit IRC | 16:33 | |
evrardjp | worth checking in the bugs there too | 16:33 |
mnaser | i put the workaround but yeah, i have no idea, we cant solve it now, just triaging :) | 16:33 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1803347 | 16:34 |
openstack | Launchpad bug 1803347 in openstack-ansible "Documentation mentions requirement of bridges on compute nodes" [Undecided,New] - Assigned to Mohammed Naser (mnaser) | 16:34 |
evrardjp | mnaser: yeah lgtm | 16:34 |
mnaser | hah i remember this | 16:34 |
noonedeadpunk | Yep, my question, sorry. | 16:34 |
evrardjp | :) | 16:34 |
mnaser | confirmed low | 16:34 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1803019 | 16:34 |
openstack | Launchpad bug 1803019 in openstack-ansible "Nova generates versioned_notifications but no one consume" [Undecided,New] - Assigned to KimMinsik (for-beatitudo) | 16:34 |
noonedeadpunk | Yeah, it's problem but more about Nova | 16:35 |
mnaser | in progress medium | 16:35 |
noonedeadpunk | I know the workaround to it | 16:35 |
noonedeadpunk | ah, ok | 16:35 |
*** DRoBeR has quit IRC | 16:36 | |
evrardjp | maybe documenting the workaround is good? | 16:36 |
noonedeadpunk | it's already a patch for it present... | 16:37 |
evrardjp | yeah we can talk about it in the review if you prefer | 16:37 |
noonedeadpunk | not at all. But it's exactly my way of resolving this. | 16:38 |
noonedeadpunk | but I've just overrided nova.conf with extra params | 16:38 |
mnaser | yep, its nice the patch has 'native' feature so itd be good to review it | 16:38 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1802742 | 16:39 |
openstack | Launchpad bug 1802742 in openstack-ansible "parameterized nova placement nginx error log level" [Undecided,New] | 16:39 |
mnaser | that seems like wishlist | 16:39 |
noonedeadpunk | and for ocata... | 16:40 |
mnaser | it probably still applies | 16:40 |
mnaser | wait it doesnt | 16:40 |
mnaser | we dropped nginx right?! | 16:40 |
noonedeadpunk | I think it's just plain uwsgi instead.. | 16:40 |
mnaser | yeah | 16:40 |
mnaser | you're right | 16:41 |
mnaser | meh confiremd/wishlist | 16:41 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1802466 | 16:41 |
openstack | Launchpad bug 1802466 in openstack-ansible "keystone cannot generate keystone.log file" [Undecided,New] - Assigned to KimMinsik (for-beatitudo) | 16:41 |
evrardjp | not everywhere, but for nova placement I don't know | 16:42 |
mnaser | placement is uwsgi | 16:42 |
mnaser | also thats cool, we got a fix for that too | 16:42 |
mnaser | +2d it | 16:42 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1802195 | 16:43 |
openstack | Launchpad bug 1802195 in openstack-ansible "ceph-rgw installation/configuration lacks idempotency, configuration changes break the setup" [Undecided,New] | 16:43 |
evrardjp | for the last one, I am worrying about container non container | 16:43 |
fghaas | evrardjp: how so? | 16:44 |
evrardjp | fghaas: well it was for the keystone cannot generate, which is not the last one, I am a little slow on typing :p | 16:44 |
evrardjp | sorry for being in 2 meetings at the same time :p | 16:45 |
fghaas | evrardjp: ack, thanks for clarifying :) | 16:45 |
mnaser | fghaas: is that an osa issue or ceph-ansible? | 16:45 |
noonedeadpunk | btw, about keystone - I don't have /var/log/keystone at all, as all log files in journalctl | 16:46 |
fghaas | it's an osa issue in that osa pulls in a state of ceph-ansible that was never meant for production. | 16:46 |
fghaas | I hope the commit messages explain that fairly well | 16:46 |
*** hamzaachi has quit IRC | 16:47 | |
fghaas | I think that bug is only in New because the actual thing to be fixed is in Rocky, and the fix in master is Related | 16:47 |
mnaser | fghaas: so does that mean its resolved ? | 16:47 |
fghaas | I certainly like to think so, and haven't heard anyone scream otherwise | 16:48 |
mnaser | ok cool | 16:49 |
evrardjp | noonedeadpunk: that sounds more likely this is why I am concered of said patch. | 16:49 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1802132 | 16:49 |
openstack | Launchpad bug 1802132 in openstack-ansible "OpenStack Ansible dashboard not able to log in with admin credentials" [Undecided,New] | 16:49 |
mnaser | sounds like a faield deploy | 16:50 |
mnaser | asked to rerun horizon install | 16:50 |
*** gkadam has quit IRC | 16:50 | |
mnaser | we'll review next week | 16:50 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1802070 | 16:50 |
openstack | Launchpad bug 1802070 in openstack-ansible "Add resource_filter field in cinder.conf for non-admin user can retrieve volume & snapshot volume" [Undecided,New] - Assigned to KimMinsik (for-beatitudo) | 16:50 |
mnaser | https://review.openstack.org/#/c/616113/ | 16:51 |
mnaser | already a patch | 16:51 |
mnaser | lets run this by the cinder team | 16:51 |
mnaser | ill do that | 16:51 |
mnaser | just to kinda figure out what is going on | 16:51 |
mnaser | but it should be in progress | 16:51 |
mnaser | #link https://bugs.launchpad.net/openstack-ansible/+bug/1801656 | 16:52 |
openstack | Launchpad bug 1801656 in openstack-ansible "fix tox python3 overrides" [Undecided,New] | 16:52 |
evrardjp | thanks for dealing with it mnaser | 16:53 |
mnaser | i guess we can keep that one | 16:53 |
mnaser | put it as confirmed | 16:53 |
mnaser | to track our py3 stuff | 16:53 |
evrardjp | ok | 16:53 |
evrardjp | fine for me | 16:53 |
evrardjp | it's almost all done anyway, isn't it? | 16:53 |
mnaser | it is mostly done i think | 16:53 |
noonedeadpunk | returning back to cinder, I thought, that policies resolves mentioned problem.. | 16:53 |
mnaser | #topic open discussion | 16:54 |
*** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)" | 16:54 | |
mnaser | we can talk about the bugs outside the meeting, but it does seem like it resolves things, but best to run by cinder team | 16:54 |
evrardjp | I don't have any topic for open discussion | 16:54 |
spotz | jungleboyj and the rest of the team should be home by now | 16:54 |
mnaser | while we have a few minutes, got anything on side of arxcruz re: os-tempest? | 16:55 |
evrardjp | except I hoped you had a nice trip back from the summit | 16:55 |
evrardjp | oh yeah probably worth checking that direction | 16:55 |
arxcruz | mnaser: we are starting to add a job to test python-tempestconf | 16:55 |
jungleboyj | Yep. I am. :-) | 16:55 |
arxcruz | and from there, we will add new functionalities on python-tempestconf, like override settings, rpm package, etc | 16:55 |
jungleboyj | What is the problem? | 16:56 |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_keystone master: Add libpython2.7 as a required package https://review.openstack.org/619040 | 16:56 |
guilhermesp | about os_placement, I'd really appreciate reviews and suggestions to finish the work https://review.openstack.org/#/c/618820/ :) | 16:56 |
mnaser | cool, thats awesome arxcruz | 16:56 |
mnaser | please send reviews | 16:56 |
arxcruz | mnaser: yup, evrardjp is already reviewing it :) | 16:57 |
mnaser | sweet | 16:57 |
mnaser | #endmeeting | 16:59 |
*** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/2xA1eZC" | 16:59 | |
openstack | Meeting ended Tue Nov 20 16:59:36 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:59 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-20-16.00.html | 16:59 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-20-16.00.txt | 16:59 |
mnaser | thanks everyone | 16:59 |
openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-11-20-16.00.log.html | 16:59 |
jungleboyj | mnaser: That bug looks like an issue in Cinder. The policies in code should handle doing that unless the file is available. | 16:59 |
spatel | cloudnull: do you have min? I am seeing very strange issue with rabbitmq http://paste.openstack.org/show/735821/ | 17:07 |
spatel | if you look into this output you will see "Error: unable to connect to node 'rabbit@ostack-controller-01': nodedown" | 17:07 |
spatel | its trying to connect wrong node or hostname.. | 17:07 |
spatel | who set that hostname? does openstack-ansible doing it? | 17:08 |
spatel | it seems bug somewhere... | 17:08 |
spatel | rabbitmq-server is running... but cluster not functional | 17:09 |
*** hamzaachi has joined #openstack-ansible | 17:10 | |
mnaser | jungleboyj: it looks like this isnt policy in code but something about resources | 17:11 |
jungleboyj | mnaser: Ok. I have added your bug to our meeting agenda tomorrow to see if we can get some input on our end. It seems strange to me. | 17:12 |
mnaser | jungleboyj: thanks :) | 17:14 |
jungleboyj | mnaser: No problem. | 17:14 |
*** fghaas has quit IRC | 17:15 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_keystone master: Add libpython2.7 as a required package https://review.openstack.org/619040 | 17:22 |
noonedeadpunk | guys, does anyone uses tooz coordination for cinder? | 17:23 |
openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca-ui master: Retire openstack-ansible-os_monasca-ui https://review.openstack.org/617204 | 17:23 |
evrardjp | I am done for the day | 17:26 |
evrardjp | ttyl everyone! | 17:26 |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 17:27 |
spatel | Folks, any idea how to fix hostname issue in rabbitmq | 17:27 |
*** tobias-urdin is now known as tobias-urdin_afk | 17:27 | |
noonedeadpunk | spatel if you've changed hostname, you'll probably have to re-add host to the cluster | 17:28 |
noonedeadpunk | but I'm not a big expert to be honest | 17:28 |
spatel | I didn't change anything... this is all build by OSA | 17:28 |
spatel | in LAB and Production same behavior.. connecting wrong hostname.. | 17:29 |
*** DanyC has joined #openstack-ansible | 17:29 | |
spatel | i am trying forget but still talking to wrong name | 17:30 |
openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 17:30 |
spatel | rabbitmqctl forget_cluster_node rabbit@ostack-controller-01 | 17:30 |
noonedeadpunk | Oh, found your paste. It seems, that rabbit is down in your container. Are you sure, that it was installed inside container? As it's name looks like it was installed bare metal... | 17:31 |
*** DanyC_ has joined #openstack-ansible | 17:31 | |
*** DanyC has quit IRC | 17:31 | |
spatel | I have hack /etc/hosts file and lie to it about hostname and now i am seeing this error http://paste.openstack.org/show/735824/ | 17:32 |
spatel | I think i should open ticket.. it seems BUG | 17:32 |
noonedeadpunk | rabbitmqctl won't work if rabbitmq service is down. | 17:32 |
spatel | rabbitmq-server.service is running.. | 17:33 |
spatel | only cluster status throwing error.. its not letting me stop/start_app also | 17:33 |
spatel | I have 200 node production and everything working.. so far.. but just today i found this issue.. its not cluster correctly... | 17:34 |
spatel | i wonder my cluster only talking to single node and not looking into cluster | 17:34 |
noonedeadpunk | This is how rabbitmqctl works. It asks only localhost node. If you wish to ask some another member of the cluster, rabbitmqadmin should do the trick | 17:35 |
odyssey4me | Time for me to head out before I feel any worse. Night all! | 17:36 |
*** shardy has quit IRC | 17:36 | |
*** cshen has joined #openstack-ansible | 17:37 | |
*** lbragstad has quit IRC | 17:38 | |
*** lbragstad has joined #openstack-ansible | 17:41 | |
noonedeadpunk | Or maybe I'm mistaken... | 17:43 |
*** spatel has quit IRC | 17:50 | |
*** spatel has joined #openstack-ansible | 18:03 | |
spatel | noonedeadpunk: i didn't get it what you trying to say | 18:06 |
*** cshen has quit IRC | 18:07 | |
*** cshen has joined #openstack-ansible | 18:08 | |
arxcruz | evrardjp: what am i doing wrong here? https://review.openstack.org/#/c/619021/3/zuul.d/jobs.yaml this vars isn't being set on the job... python-tempestconf isn't being invoqued | 18:15 |
spatel | I have summit bug: https://bugs.launchpad.net/openstack-ansible/+bug/1804283 | 18:17 |
openstack | Launchpad bug 1804283 in openstack-ansible "Rabbitmq Error: unable to connect to node 'rabbit@ostack-controller-01': nodedown" [Undecided,New] | 18:17 |
*** cshen has quit IRC | 18:17 | |
openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts stable/rocky: Correct host only routes https://review.openstack.org/616162 | 18:18 |
*** rpittau has quit IRC | 18:27 | |
*** DanyC_ has quit IRC | 18:27 | |
*** DanyC has joined #openstack-ansible | 18:27 | |
*** czunker has joined #openstack-ansible | 18:32 | |
spatel | This is scary!!! https://www.techsfo.com/blog/2013/06/rabbitmq-breaks-when-you-rename-hostname/ | 18:33 |
spatel | its saying you need to re-install rabbitmq to change hostname.. damn it | 18:33 |
openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon master: Implements masakari-dashboard to be installed in horizon https://review.openstack.org/619058 | 18:37 |
*** tobias-urdin_afk is now known as tobias-urdin | 18:38 | |
openstackgerrit | Andy Smith proposed openstack/openstack-ansible master: Add qdrouterd role for oslo.messaging backend deployment https://review.openstack.org/570261 | 18:39 |
*** cshen has joined #openstack-ansible | 18:40 | |
*** cshen has quit IRC | 18:45 | |
*** fghaas has joined #openstack-ansible | 18:46 | |
*** pcaruana has quit IRC | 18:47 | |
spatel | Oh! wait... this is interesting http://paste.openstack.org/show/735829/ | 18:55 |
spatel | cloudnull: noonedeadpunk ^^ | 18:55 |
spatel | noonedeadpunk: ^^ | 18:55 |
spatel | it works but with -n option | 18:55 |
*** cshen has joined #openstack-ansible | 18:56 | |
cloudnull | spatel so the hostname was changed? | 18:56 |
*** strattao has joined #openstack-ansible | 18:56 | |
spatel | no | 18:56 |
*** fghaas has quit IRC | 18:56 | |
spatel | i didn't touch anything... | 18:57 |
spatel | rabbitmqctl cluster_status <-- throwing error but | 18:57 |
spatel | rabbitmqctl -n rabbit@ostack-controller-01-rabbit-mq-container-1bf6ede2 cluster_status <-- is working | 18:57 |
cloudnull | so then the rabbitmq custler is running ok however you have to pass the node-name flag in order to query it ? | 18:57 |
spatel | yes.. | 18:58 |
spatel | but why do i need to do that? | 18:58 |
cloudnull | and rabbitmq is running in a container ? | 18:58 |
spatel | yes in LXC | 18:58 |
cloudnull | I assume the failing node is "ostack-controller-01" | 18:58 |
noonedeadpunk | Strange thing is about the hostname, which rabbitmqctl is trying to ask by default... | 18:58 |
cloudnull | I assume "ostack-controller-01" is the name set in /etc/hosts ? | 18:59 |
cloudnull | and /etc/hostname ? | 18:59 |
spatel | ostack-controller-01 <- is my host node | 18:59 |
cloudnull | can you also paste the output of `hostnamectl` ? | 18:59 |
spatel | http://paste.openstack.org/show/735830/ | 18:59 |
spatel | hostnamectl -> http://paste.openstack.org/show/735831/ | 19:00 |
openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible master: Added masakari-dashboard repo to openstack_services https://review.openstack.org/619062 | 19:01 |
cloudnull | it looks like the rabbitmqctl command thinks its hostname is "ostack-controller-01" | 19:01 |
cloudnull | what about the files in /etc/rabbitmq ? | 19:01 |
-spatel- [root@ostack-controller-01-rabbit-mq-container-1bf6ede2 rabbitmq]# grep ostack-controller-01 /etc/hosts | 19:01 | |
-spatel- 192.168.100.31ostack-controller-01 | 19:01 | |
cloudnull | is that hostname there? | 19:01 |
cloudnull | spatel that hosts entry, is that the ip address of your container ? | 19:02 |
spatel | /etc/rabbitmq/rabbitmq.config doesn't have any hostname definition anywhere | 19:02 |
spatel | container ip is 192.168.100.31 | 19:02 |
cloudnull | which is set in /etc/hosts | 19:03 |
cloudnull | can you change the name in the `/etc/hosts` file to the container name ? | 19:03 |
spatel | doing it.. | 19:03 |
spatel | oh! its already there.. | 19:05 |
-spatel- [root@ostack-controller-01-rabbit-mq-container-1bf6ede2 rabbitmq]# grep 192.168.100.31 /etc/hosts | 19:05 | |
-spatel- #192.168.100.31ostack-controller-01 | 19:05 | |
-spatel- 192.168.100.31 ostack-controller-01-rabbit-mq-container-1bf6ede2.openstack.local ostack-controller-01-rabbit-mq-container-1bf6ede2 ostack-controller-01_rabbit_mq_container-1bf6ede2 | 19:05 | |
cloudnull | oh so its a duplicate entry | 19:05 |
*** cshen has quit IRC | 19:05 | |
spatel | somehow rabbitmq hold that entry in DB somewhere so not leaving it.. | 19:06 |
spatel | cloudnull: ready for fun???? | 19:08 |
-spatel- [root@ostack-controller-01-rabbit-mq-container-1bf6ede2 rabbitmq]# echo $HOSTNAME | 19:08 | |
-spatel- ostack-controller-01.site5.vivox.com | 19:08 | |
spatel | what the heck is this? | 19:08 |
spatel | $HOSTNAME | 19:08 |
cloudnull | ? | 19:08 |
spatel | why echo $HOSTNAME giving wrong hostname on lxc container | 19:08 |
spatel | it should be container hostname right? | 19:09 |
cloudnull | it should . | 19:09 |
spatel | who set that up? | 19:09 |
spatel | let me try and see | 19:09 |
cloudnull | I have no idea | 19:09 |
spatel | export HOSTNAME=ostack-controller-01-rabbit-mq-container-1bf6ede2 | 19:10 |
spatel | it working now :) | 19:10 |
cloudnull | yay :) | 19:10 |
spatel | something is wrong in OSA... | 19:10 |
cloudnull | is "ostack-controller-01.site5.vivox.com" the FQDN on the physical host ? | 19:10 |
spatel | i deployed 3 cloud and all three has same issue.. | 19:10 |
spatel | yes ostack-controller-01 is physical host | 19:11 |
cloudnull | maybe those environment variables are being passed along into the lxc-attach command? | 19:11 |
noonedeadpunk | spatel: You might have forwarded ENV from host to container while entered container | 19:11 |
cloudnull | I know you can tell it to drop environment variable pass though. | 19:11 |
*** strattao has quit IRC | 19:12 | |
*** fghaas has joined #openstack-ansible | 19:12 | |
spatel | This is what i am doing | 19:12 |
spatel | lxc-attach -n ostack-controller-01_rabbit_mq_container-1bf6ede2 | 19:12 |
spatel | where i should tell to not use host ENV? | 19:13 |
spatel | I am using all default OSA.. i didn't customized anything so it must be hiden somewhere in OSA | 19:14 |
spatel | how about if OSA do following.. | 19:15 |
spatel | add NODENAME=rabbit@ostack-controller-01-rabbit-mq-container-1bf6ede2 in /etc/rabbitmq/rabbitmq-env.conf | 19:15 |
spatel | that should solve that issue no matter ENV is correct or wrong.. | 19:16 |
cloudnull | spatel try `lxc-attach --clear-env --name $CONTAINER` | 19:16 |
cloudnull | ^ we run that when running the playbooks | 19:17 |
spatel | cloudnull: that dropped ENV and now i am seeing correct echo $HOSTNAME | 19:18 |
spatel | do i need to add --clear-env everytime? | 19:19 |
spatel | what is the permanent solution for that | 19:19 |
*** asettle has quit IRC | 19:20 | |
*** fghaas has left #openstack-ansible | 19:24 | |
cloudnull | cool | 19:24 |
cloudnull | IDK of a permanent way to pass that flag every time. | 19:25 |
*** hamzy has joined #openstack-ansible | 19:25 | |
* cloudnull looking | 19:25 | |
openstackgerrit | Manuel Buil proposed openstack/openstack-ansible-os_neutron master: Bug fix: Bridge br-provider missing in dvr https://review.openstack.org/619070 | 19:26 |
cloudnull | spatel maybe something like https://pasted.tech/pastes/c1c75800c19d7caf6e073dbf141f9e9e88cc95e2 | 19:27 |
spatel | that is sweet!!! | 19:27 |
cloudnull | you'd need to write that into your bash profile | 19:27 |
cloudnull | but that would do it | 19:27 |
spatel | in short default behavior is to pass ENV to container | 19:28 |
cloudnull | it would seem so | 19:28 |
cloudnull | ive not seen that behavior on ubuntu / suse | 19:29 |
cloudnull | however it would seem that if that env var is set ($HOSTNAME) its passed through | 19:29 |
spatel | hmm! so look like one more CentOS bug | 19:30 |
cloudnull | we could add that alias to the lxc-hosts role | 19:32 |
cloudnull | for all os's which would help solve this in cent | 19:32 |
spatel | it would be good so folks won't get confused and get panic like me :) let me update BUG | 19:32 |
spatel | cloudnull: here is the bug if you want to go ahead and put that fix in https://bugs.launchpad.net/openstack-ansible/+bug/1804283 | 19:34 |
openstack | Launchpad bug 1804283 in openstack-ansible "Rabbitmq Error: unable to connect to node 'rabbit@ostack-controller-01': nodedown" [Undecided,New] | 19:34 |
cloudnull | spatel https://pasted.tech/pastes/e1b1e5e67f5e0283a6d56b04e2147c51a9754091 | 19:36 |
cloudnull | that was my global change | 19:37 |
spatel | beautiful!! | 19:37 |
spatel | i am pushing them in .bashrc | 19:37 |
*** cshen has joined #openstack-ansible | 19:39 | |
openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon master: Implements masakari-dashboard to be installed in horizon https://review.openstack.org/619058 | 19:44 |
jrosser | cloudnull: does that interfere with ansible environment: settings? | 19:47 |
openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 19:50 |
cloudnull | jrosser it doesnt look like it would | 19:55 |
cloudnull | in the plugins repo where we calll lxc-attach we use the full path | 19:55 |
cloudnull | so the alias shouldnt impact those tools | 19:55 |
cloudnull | I guess it might be something that could interact with a shell or command when were not using the plugins/modules. | 19:56 |
cloudnull | though idk | 19:56 |
jrosser | Thinking about it those environment: settings on tasks are probably created in the ansible dropped python anyway.... | 19:56 |
*** cshen has quit IRC | 20:14 | |
cloudnull | ^ yea i think thats true | 20:15 |
*** aedc has quit IRC | 20:15 | |
Miouge | After upgrading keepalived from 1.2.19 to 1.2.24 (Ubuntu 16.04) I lost the VRRP IPs. It turns out that in v1.2.20 keepalived exits if the VRRP interface has no IP, which is the case for my br-public (see relese notes https://github.com/acassen/keepalived/blob/master/ChangeLog#L3800). Should we have an assert for that? | 20:15 |
*** cshen has joined #openstack-ansible | 20:15 | |
*** dcdamien has joined #openstack-ansible | 20:16 | |
*** aedc has joined #openstack-ansible | 20:26 | |
*** ahosam has joined #openstack-ansible | 20:28 | |
openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible master: Add a check that keepalived interfaces have IPs https://review.openstack.org/619092 | 20:29 |
*** cshen has quit IRC | 20:29 | |
*** cshen has joined #openstack-ansible | 20:31 | |
*** mmercer has joined #openstack-ansible | 20:41 | |
*** czunker has quit IRC | 20:49 | |
*** priteau has quit IRC | 20:52 | |
openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Fix nova-lxd tests for bionic. https://review.openstack.org/616302 | 20:56 |
*** hamzaachi has quit IRC | 21:21 | |
*** ahosam has quit IRC | 21:23 | |
*** ahosam has joined #openstack-ansible | 21:24 | |
*** dave-mccowan has quit IRC | 21:28 | |
*** ahosam has quit IRC | 21:29 | |
*** hamzy has quit IRC | 21:42 | |
*** DanyC has quit IRC | 22:06 | |
*** dcdamien has quit IRC | 22:27 | |
*** spatel has quit IRC | 22:30 | |
*** dcdamien has joined #openstack-ansible | 22:36 | |
*** strattao has joined #openstack-ansible | 22:37 | |
*** strattao has quit IRC | 22:48 | |
*** dcdamien has quit IRC | 22:54 | |
*** cshen has quit IRC | 22:55 | |
*** mugsie has quit IRC | 23:06 | |
*** ivve has quit IRC | 23:08 | |
*** aedc has quit IRC | 23:47 | |
*** spatel has joined #openstack-ansible | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!