| *** cheng1 has quit IRC | 00:05 | |
| *** cheng1 has joined #openstack-ansible | 00:06 | |
| *** nocsoc has joined #openstack-ansible | 00:13 | |
| *** cheng1 has quit IRC | 01:03 | |
| *** cheng1 has joined #openstack-ansible | 01:04 | |
| *** cheng1 has quit IRC | 02:02 | |
| *** cheng1 has joined #openstack-ansible | 02:02 | |
| *** cheng1 has quit IRC | 02:21 | |
| *** cheng1 has joined #openstack-ansible | 02:22 | |
| *** jcath has joined #openstack-ansible | 02:27 | |
| *** spatel has joined #openstack-ansible | 03:07 | |
| *** spatel has quit IRC | 03:08 | |
| *** cheng1 has quit IRC | 03:26 | |
| *** cheng1 has joined #openstack-ansible | 03:30 | |
| *** cheng1 has quit IRC | 04:18 | |
| *** cheng1 has joined #openstack-ansible | 04:18 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-ansible | 04:33 | |
| *** pcaruana has quit IRC | 04:46 | |
| *** pcaruana has joined #openstack-ansible | 04:47 | |
| *** gyee has quit IRC | 05:11 | |
| *** cheng1 has quit IRC | 05:31 | |
| *** cheng1 has joined #openstack-ansible | 05:36 | |
| *** dpaclt has joined #openstack-ansible | 05:50 | |
| *** andrea15 has quit IRC | 05:51 | |
| *** cheng1 has quit IRC | 05:55 | |
| *** cheng1 has joined #openstack-ansible | 05:57 | |
| *** udesale has joined #openstack-ansible | 06:02 | |
| *** udesale has quit IRC | 06:17 | |
| *** udesale has joined #openstack-ansible | 06:18 | |
| *** jbadiapa has quit IRC | 06:23 | |
| *** jbadiapa has joined #openstack-ansible | 06:30 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: auto-detect interpreter. https://review.opendev.org/734672 | 06:32 |
|---|---|---|
| *** kaiokmo has joined #openstack-ansible | 06:50 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/stein: Bump os_tempest to include fix for pinning telemetry plugin to py2.7 version https://review.opendev.org/735065 | 06:56 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer stable/stein: Make user provided content deployable https://review.opendev.org/733387 | 06:57 |
| openstackgerrit | Erik Berg proposed openstack/openstack-ansible stable/train: ceph: remove invalid logging options https://review.opendev.org/735066 | 06:58 |
| *** cheng1 has quit IRC | 06:59 | |
| *** cheng1 has joined #openstack-ansible | 06:59 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] Test arm64 https://review.opendev.org/618305 | 07:13 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] Test arm64 https://review.opendev.org/618305 | 07:14 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] Test arm64 https://review.opendev.org/618305 | 07:16 |
| *** rpittau|afk is now known as rpittau | 07:18 | |
| *** jawad_axd has joined #openstack-ansible | 07:21 | |
| *** sshnaidm|afk is now known as sshnaidm | 07:23 | |
| *** tosky has joined #openstack-ansible | 07:38 | |
| jrosser | morning | 08:13 |
| jrosser | noonedeadpunk: are you working on galera? | 08:13 |
| jcath | hello, jrosser | 08:15 |
| jrosser | hello | 08:15 |
| jcath | I try aio with/without lxc today, it works, with stable/stein branch, thanks | 08:16 |
| *** jamesdenton has quit IRC | 08:16 | |
| jcath | but it really downloaded a lot, from many sources T_T | 08:16 |
| noonedeadpunk | jrosser: yeah, was just going to look at it | 08:18 |
| jrosser | noonedeadpunk: ok, if theres anything i can do thats not duplicate? | 08:18 |
| jrosser | jcath: yes it does. in my environment we mirror the apt repos, have local copies of the git repos and run devpi as a proxy to pypi | 08:19 |
| jrosser | devpi is the only thing that makes outgoing requests during a deployment | 08:19 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Fix Mariadb 10.3 -> 10.4 upgrade path https://review.opendev.org/734888 | 08:20 |
| janno | good morning | 08:20 |
| noonedeadpunk | I think that everything is just about it... But like role was passing in lxc in sandbox yestarday | 08:21 |
| jcath | about the mirror of apt/yum repos, is there a central file to change the urls? or I have to change the vars in different yml , like /etc/ansible/roles/openstack_hosts/vars/redhat-7.yml | 08:22 |
| jrosser | jcath: you should never have to change anything in /etc/ansible/roles | 08:22 |
| jrosser | each role has a sort of "external interface" which should be fairly well documented in /etc/ansible/roles/<role>/defaults/main.yml | 08:23 |
| jrosser | any of those variables can be overridden in /etc/openstack_deploy/user_<whatever>.yml | 08:23 |
| jcath | oh, i c. :D thanks, jrosser | 08:24 |
| jrosser | or you can make your own host_vars and group_vars directories in /etc/openstack_deploy just like you would do with any other ansible to provide host or group specific overrides | 08:24 |
| jcath | that's easier | 08:24 |
| *** itandops has joined #openstack-ansible | 08:26 | |
| *** itandops has quit IRC | 08:29 | |
| jrosser | noonedeadpunk: oh it was functional tests too wasnt it - check for cluster formation being unreliable | 08:31 |
| jcath | jrosser: ceph-ansible has a var call ceph_origin and support 'distro', it will install what's come with the distro. if openstack-ansible also something similar, it would save some time | 08:31 |
| noonedeadpunk | I think we should just wait for wsrep_cluster_size ==3 | 08:32 |
| noonedeadpunk | It's test and we know exactly what size we're expecting to see | 08:33 |
| noonedeadpunk | And was thinking to patch it in 734888 | 08:33 |
| noonedeadpunk | oh, or rebase 734888 on top of it... | 08:33 |
| jrosser | jcath: so there are two things, deploying the ceph cluster itself with ceph-ansible, (you can do that either yourself, or have OSA do it for you) | 08:34 |
| jrosser | jcath: and then there is setting up cinder/nova/glance etc with the appropriate client parts of ceph | 08:34 |
| jrosser | which of those do you mean? | 08:34 |
| jcath | I mean, deployment of ceph, it can install whatever come with the os distro, with the var ceph_origin => 'distro'. | 08:37 |
| jrosser | the client part is deployed with this ansible role https://github.com/openstack/openstack-ansible-ceph_client/blob/master/defaults/main.yml#L41 | 08:37 |
| jrosser | you should plan carefully here | 08:38 |
| jcath | ah.... T_T | 08:38 |
| jrosser | each openstack-ansible release supports a particular version of ceph-ansible | 08:38 |
| jrosser | versions of ceph-ansible deploy specific releases of ceph | 08:38 |
| jrosser | openstack ansible releases and ceph-ansible releases require specific versions of ansible | 08:39 |
| jrosser | your distro will provide a particular version of ceph | 08:39 |
| jrosser | ^ draw the venn diagram of that...... | 08:39 |
| jcath | ! complicated | 08:39 |
| openstack | jcath: Error: "complicated" is not a valid command. | 08:39 |
| jcath | so complicated | 08:40 |
| jcath | T_T | 08:40 |
| jrosser | you have a lot more freedom if you deploy ceph yourself with ceph-ansible and do not try to have OSA do that for you | 08:40 |
| jrosser | as you can decouple the ansible versions | 08:40 |
| jrosser | it's quite possible that for the particular OS you want to use and openstack release, you have no choice | 08:41 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Use more reliable option to get cluster ready state https://review.opendev.org/735079 | 08:43 |
| noonedeadpunk | jrosser: I think ^ should be more reliable if work at all :p | 08:43 |
| noonedeadpunk | (at least during internal playing with commands this should work...) | 08:44 |
| jcath | jrosser: thanks, I'll try again | 08:47 |
| jrosser | jcath: i think it's only complicated if you cannot compromise anywhere | 08:47 |
| jrosser | start with what you need to acheive and work out which options allow you to do that | 08:47 |
| jrosser | openstack release, ceph version, what youre prepared to mirror, seperate ceph installation or not..... all those things | 08:47 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Fix Mariadb 10.3 -> 10.4 upgrade path https://review.opendev.org/734888 | 08:48 |
| jcath | with aio_ceph, it seems it install the ceph mimic release, but the default value of roles/ceph_client/defaults/main.yml is "ceph_stable_release: luminous | 08:50 |
| jcath | " | 08:50 |
| *** itandops has joined #openstack-ansible | 08:51 | |
| jrosser | for which branch of openstack-ansible? | 08:51 |
| jcath | stable/stein | 08:51 |
| jrosser | https://opendev.org/openstack/openstack-ansible/src/branch/stable/stein/inventory/group_vars/all/ceph.yml#L28 | 08:52 |
| jrosser | the role default should probably be changed there but on stein in the AIO it is always overridden in the "integrated repo" during deployment | 08:53 |
| jcath | oh, see. I should check this, thanks, jrosser | 08:53 |
| jrosser | there are many many vars set in the openstack-ansible repo which override the role defaults to 'wire' all the roles together to make a coherent deployment | 08:53 |
| jrosser | heres a useful way to search for where the vars are set/used http://codesearch.openstack.org/?q=ceph_stable_release | 08:54 |
| jcath | I just only grep in /etc/openstack_deploy, forget to check there | 08:54 |
| jrosser | that only returns results for master branch, but you can click that through to the git repos and look at the different branches | 08:55 |
| jcath | thanks, frosser, sorry for my rookie question | 08:55 |
| jcath | s/frosser/jrosser/ | 08:55 |
| jrosser | jcath: no worries - theres a lot of moving parts here and it's easy to get lost | 08:56 |
| jcath | T_T | 08:56 |
| jrosser | OSA has a huga amount of customisation possible which means theres different routes depending on the needs of the deployment | 08:56 |
| jrosser | *huge | 08:56 |
| jcath | yeah, I c, it is quite flexible | 08:57 |
| jcath | that's the way a well opensource project grow up, many possibilities | 09:01 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-ceph_client stable/stein: Use the mimic release rather than luminous https://review.opendev.org/735083 | 09:02 |
| jrosser | jcath: ^ hopefully that change makes it less confusing :) | 09:02 |
| *** miloa has joined #openstack-ansible | 09:04 | |
| jcath | :D | 09:04 |
| *** nocsoc has quit IRC | 09:12 | |
| jcath | jrosser: a question about the neutron ml2 plugin type, with stable/stein, the default is linuxbridge, as I read the doc, it also support ovs with neutron_plugin_type -> ml2.ovs. does this release work well with ovs? | 09:13 |
| *** rpittau is now known as rpittau|bbl | 09:15 | |
| *** stingrayza has joined #openstack-ansible | 09:28 | |
| *** cheng1 has quit IRC | 09:31 | |
| *** also_stingrayza has quit IRC | 09:31 | |
| *** cheng1 has joined #openstack-ansible | 09:34 | |
| jrosser | jcath: whilst linuxbridge has been the default in OSA for a long time you can use OVS if you want to | 09:45 |
| *** andrea15 has joined #openstack-ansible | 09:50 | |
| *** cheng1 has quit IRC | 09:52 | |
| *** cheng1 has joined #openstack-ansible | 09:55 | |
| openstackgerrit | zhaoleilc proposed openstack/openstack-ansible master: Correct a typo in the document https://review.opendev.org/735098 | 10:03 |
| watersj | if you have limited nics how do you go about provider network (flat network). I have 3 br-mgmt, br-storage, br-vxlan. With kolla you just defined a "neutron nic" all traffic flat, vxlan, guest traffic uses it. | 10:32 |
| *** cheng1 has quit IRC | 10:35 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Use more reliable option to get cluster ready state https://review.opendev.org/735079 | 10:37 |
| jrosser | watersj: all neutron needs to know is which interface | 10:38 |
| jrosser | as an example, in the openstack-ansible AIO config you need some kind of invented external interface, so we hang eth12 off br-vlan to make a flat network (the untagged traffic on br-vlan) | 10:40 |
| jrosser | you can set it up any way you like, using any interface you like | 10:40 |
| *** cheng1 has joined #openstack-ansible | 10:40 | |
| jrosser | also you do not need to make br-mgmt br-storage and br-vxlan be dedicated nic, you can set those up however you like | 10:41 |
| jrosser | they can be vlans on the same physical interface if you like | 10:41 |
| jrosser | i think the difference with the reference you give from kolla is that for OSA, the expected "interface" that the deployer provides is br-mgmt, br-vxlan, br-storage and so on | 10:42 |
| jrosser | how you implement those bridges on the host is entirely up to you | 10:42 |
| watersj | jrosser, ok, thats my fault for not understanding host config | 10:43 |
| watersj | you can only have 1 bridge device per nic | 10:43 |
| watersj | when you saw inverted, is that tap/tun device? | 10:44 |
| jrosser | you can have many bridges per nic if you use a trunked 802.1q port to your switch | 10:44 |
| watersj | meaning each bridge is a vlan | 10:44 |
| jrosser | yes | 10:45 |
| jrosser | here is an example with a single bond to upstream switch(es) and each bridge on a different vlan of that https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/openstack_interface.cfg.singlebond.example | 10:46 |
| jrosser | here would be making a fake 'flat' network interface eth12 which is actually untagged traffic on br-vlan https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/openstack_interface.cfg.test.example#L68-L99 | 10:49 |
| watersj | jrosser, ty | 10:49 |
| jrosser | imho set this up how you need, don't just follow the example | 10:49 |
| watersj | jrosser, if you don't mind sharing, for your deployments what distro do you use? | 11:00 |
| jrosser | i am using ubuntu, but debian 10 is also a strong choice | 11:01 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Fix Mariadb 10.3 -> 10.4 upgrade path https://review.opendev.org/734888 | 11:05 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Return upgrade jobs to voting https://review.opendev.org/735110 | 11:07 |
| *** threestrands has quit IRC | 11:10 | |
| *** rpittau|bbl is now known as rpittau | 11:54 | |
| *** rh-jelabarre has joined #openstack-ansible | 11:57 | |
| *** rh-jelabarre has quit IRC | 11:58 | |
| *** rh-jelabarre has joined #openstack-ansible | 11:58 | |
| *** cheng1 has quit IRC | 11:59 | |
| *** cheng1 has joined #openstack-ansible | 12:00 | |
| *** dpaclt has quit IRC | 12:00 | |
| *** rh-jelabarre has quit IRC | 12:02 | |
| *** rh-jelabarre has joined #openstack-ansible | 12:02 | |
| *** rh-jelabarre has quit IRC | 12:03 | |
| *** pcaruana has quit IRC | 12:11 | |
| *** rh-jelabarre has joined #openstack-ansible | 12:14 | |
| *** fyx has quit IRC | 12:32 | |
| *** guilhermesp has quit IRC | 12:32 | |
| *** mnaser has quit IRC | 12:32 | |
| *** fyx has joined #openstack-ansible | 12:33 | |
| *** nicolasbock has quit IRC | 12:33 | |
| *** guilhermesp has joined #openstack-ansible | 12:33 | |
| *** nicolasbock has joined #openstack-ansible | 12:33 | |
| *** mnaser has joined #openstack-ansible | 12:33 | |
| *** itandops has quit IRC | 12:42 | |
| mgariepy | morning everyone | 12:44 |
| *** jamesdenton has joined #openstack-ansible | 12:46 | |
| *** arkan has joined #openstack-ansible | 12:48 | |
| arkan | hello expers, I have openstack installed, I can provision virtual mashines, but they don't have internet access | 12:49 |
| arkan | I'm using one nic | 12:49 |
| arkan | my question is, should I create a veth to be used with bridge (provider) | 12:50 |
| arkan | this is my netplan file | 12:53 |
| arkan | https://pastebin.com/CEjSny7q | 12:53 |
| arkan | ip netns exec qrouter-decf44e7-105e-4581-9e50-d64c6ed2430d ping 8.8.8.8 | 12:54 |
| arkan | connect: Network is unreachable | 12:54 |
| arkan | my openstack_user_config.yml https://pastebin.com/EDCmis2r | 12:55 |
| arkan | my user_variables.yml https://pastebin.com/H26Kskxg | 12:56 |
| jamesdenton | arkan it looks like neutron should be using the eno1 interface | 12:57 |
| jamesdenton | did you create a flat or vlan network? | 12:57 |
| arkan | flat | 12:57 |
| jamesdenton | ok | 12:57 |
| jamesdenton | so, eno1 is already connected to br-ext and can't be connected to the bridge neutron creates | 12:57 |
| *** itandops has joined #openstack-ansible | 12:58 | |
| jamesdenton | neutron will create a bridge for your flat network, brqXXXXXXX | 12:58 |
| jamesdenton | it will automatically try to connect eno1 to that bridge | 12:58 |
| arkan | the lxc, controller and compute nodes all have network access and internet | 12:58 |
| mgariepy | huh, jrosser noonedeadpunk weird.. https://c0969db12a174f8558fb-5ba6792c7427741c3f6f06d35721722e.ssl.cf1.rackcdn.com/734888/5/check/openstack-ansible-functional-debian-buster/8081375/logs/ara-report/result/9a2728ac-5b7a-47c7-b044-7904825be628/ | 12:58 |
| arkan | only the vm they don't have access | 12:58 |
| jamesdenton | right. the reason why is likely: eno1 is already connected to br-ext and can't be connected to the bridge neutron creates | 12:58 |
| noonedeadpunk | mgariepy: yeah, as it's on top of the change, that should fix that... | 12:59 |
| arkan | aha ! | 12:59 |
| noonedeadpunk | but ok.... | 13:00 |
| noonedeadpunk | will do another change to the check process... | 13:00 |
| arkan | jamesdenton: but how can I make this to work, because I need in the beginning internet access | 13:00 |
| arkan | jamesdenton: should I remove the connection between br-ext and eno1 ? | 13:01 |
| jamesdenton | well, there's two ways. 1) use vlan networks instead of a flat network, or 2) create a veth and connect one end to br-ext, and use the other for neutron. change host_bind_override to use that veth instead of eno1 | 13:01 |
| arkan | jamesdenton: or should I use other host_bind_override ? | 13:01 |
| arkan | aha | 13:02 |
| jamesdenton | but that will be a global change without some work. you'd need to do the same thing on all hosts | 13:02 |
| arkan | jamesdenton: I want to choose the second way | 13:02 |
| jamesdenton | kk | 13:02 |
| jamesdenton | creating a veth with netplan may be tricky, i dunno. most folks decide to re-install ifupdown2 | 13:03 |
| arkan | I thought to create it manually | 13:03 |
| jamesdenton | for testing, sure. it won't persist a reboot | 13:04 |
| arkan | jamesdenton: so can I use vlan instead of using host_bind_override ? | 13:05 |
| arkan | jamesdenton: I mean to choose your first solution | 13:05 |
| arkan | jamesdenton: if I use vlan instead of flat, there will be no need to use host_bind_override, right ? | 13:06 |
| jamesdenton | oh, yes. if you decide to create a vlan network instead then it should work as-is | 13:06 |
| jamesdenton | you still want host_bind_override, so neutron will know to build the vlan off eno1 | 13:06 |
| jamesdenton | it will create eno1.105 for neutron network using vlan 105 | 13:07 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Use more reliable option to get cluster ready state https://review.opendev.org/735079 | 13:08 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Fix Mariadb 10.3 -> 10.4 upgrade path https://review.opendev.org/734888 | 13:08 |
| arkan | jamesdenton: so if I provide br-ext with vlan16, and using in openstak_user_config.yml under network: container_interface: "eth12" host_bind_override": | | 13:09 |
| arkan | jamesdenton: so if I provide br-ext with vlan16, and using in openstak_user_config.yml under network: container_interface: "eth12" host_bind_override": "br-ext" | 13:09 |
| jamesdenton | do you only have a single vlan to use? | 13:10 |
| arkan | jamesdenton: yes, because I have to enter it in the router also, the router does not let me to specify range of vlans | 13:10 |
| arkan | jamesdenton: and it will be tagged | 13:11 |
| arkan | jamesdenton: the vlan that is not tagged (native) is in the 192.168.50.0/24 network has vlan 50 | 13:11 |
| jamesdenton | right, so my recommendation is to create something like vlan 40, and configure the router as the gateway for vlan 40. then create a neutron network using vlan 40 and the subnet you specified on the router. | 13:11 |
| jamesdenton | you could probably leave your configuration as-is, since it will build eno1.40 automatically and connect to the neutron bridge it creates | 13:12 |
| jamesdenton | that should Just Work™ | 13:12 |
| arkan | jamesdenton: so in this situation I will remove the ip from br-ext and add it to eno1 (physical int), then configure br-ext to use vlan | 13:15 |
| arkan | jamesdenton: after these changes, what playblook should I run ? | 13:15 |
| jamesdenton | no, you should be able to leave it as-is | 13:15 |
| jamesdenton | simply create a new neutron network: openstack network create --provider-network-type vlan --provider-segment 40 --provider-physical-network provider ... | 13:16 |
| arkan | aha | 13:17 |
| arkan | jamesdenton: great, I will try your recommendations, thanks a lot | 13:18 |
| jamesdenton | sure thing | 13:18 |
| *** owalsh_ is now known as owalsh | 13:20 | |
| jrosser | i think every time i've used a flat network i've regretted it and ended up changing it to vlan | 13:21 |
| jrosser | even when theres just one vlan | 13:21 |
| arkan | jamesdenton: should the bridge br-ext have an ip, I want to use 172.16.0.0/16 network with vlan 16 for it (already set in my router) ? | 13:24 |
| jamesdenton | the IP on br-ext needs to match whatever the native vlan is on the switchport connected to eno1 | 13:24 |
| jamesdenton | your VM won't be able to communicate with it directly, since the VM will be on VLAN 40 (in my example) | 13:25 |
| arkan | jamesdenton: so br-ext can use for example vlan 40 and also use the same network as the native vlan (vlan 50) which is 192.168.50.0/24, isn't right ? | 13:26 |
| arkan | jamesdenton: so can I give br-ext ip: 192.168.50.30/24 (on vlan 40) ? | 13:27 |
| arkan | jamesdenton: and set neutron as you wrote above | 13:28 |
| jamesdenton | if the native is 50, and what you have configured now is working, then i would leave it alone. Any network you configure within Neutron is independent of what you have configured on the host (mostly). it is simple using the eno1 interface as a vehicle for connectivity | 13:29 |
| jamesdenton | this is just further complicated by the use of a single interface for both host networking and neutron networking | 13:29 |
| arkan | jamesdenton: I don't have enough ports, that's why I'm using only one nic | 13:30 |
| arkan | jamesdenton: and I don't have switch | 13:30 |
| jamesdenton | that's fine, it will work. | 13:30 |
| jamesdenton | so your controller is directly connected to the router? | 13:31 |
| arkan | jamesdenton: yes | 13:31 |
| jamesdenton | how is your compute connected? | 13:31 |
| arkan | jamesdenton: also directly to the router | 13:31 |
| jamesdenton | interesting. | 13:31 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [WIP] Make upgrade jobs get role relevant versions https://review.opendev.org/710849 | 13:33 |
| *** kaiokmo has quit IRC | 13:33 | |
| jrosser | arkan: out of interest what router do you have? | 13:38 |
| arkan | jrosser: Cisco RV160W | 13:41 |
| arkan | unfortunately it does not use ios | 13:42 |
| arkan | I can not use serial cable and go to "en" mode ...etc | 13:42 |
| arkan | jrosser: I need to use the UI, it's based on Linux | 13:42 |
| jamesdenton | it has a built-in switch | 13:43 |
| noonedeadpunk | mgariepy: hmmmmm https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_64d/734888/6/check/openstack-ansible-functional-ubuntu-bionic-ssl/64d3b17/logs/ara-report/result/cd3110ae-73ae-431a-979d-4ee20327b09c/ | 13:51 |
| noonedeadpunk | so at this point,, I'm out of ideas.... | 13:53 |
| noonedeadpunk | out of log all members should be synced https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_64d/734888/6/check/openstack-ansible-functional-ubuntu-bionic-ssl/64d3b17/logs/openstack/container3/mariadb.service.journal.log.txt | 13:56 |
| noonedeadpunk | so like we have 3 members, but one of the member address is not in the wsrep_address_list | 13:58 |
| mgariepy | huh. | 14:04 |
| noonedeadpunk | only for 1 container | 14:09 |
| noonedeadpunk | in the same time others see it | 14:09 |
| jrosser | its the cluster that isnt | 14:09 |
| noonedeadpunk | it feels like mariadb bug tbh... | 14:09 |
| jrosser | :( | 14:09 |
| noonedeadpunk | and all of them have size 3 lol | 14:10 |
| noonedeadpunk | So like just wsrep_incoming_addresses don't get updated for some reason or smth | 14:10 |
| *** udesale_ has joined #openstack-ansible | 14:19 | |
| *** jawad_axd has quit IRC | 14:20 | |
| mgariepy | works for centos but not ubuntu.. | 14:21 |
| mgariepy | fun | 14:21 |
| *** udesale has quit IRC | 14:22 | |
| mgariepy | i'll try it in a vm. to see if i find something obvious. | 14:23 |
| noonedeadpunk | But it's really floating issue as well | 14:27 |
| noonedeadpunk | So it happens only _sometimes_.... | 14:27 |
| mgariepy | always fun to debug such issues. | 14:31 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [WIP] Make upgrade jobs get role relevant versions https://review.opendev.org/710849 | 14:48 |
| admin0 | one of my ceph mon (after reboot ) is listening to mgmt and not the public ( storage) network .. anyone seen this before ? | 14:51 |
| admin0 | its osa+ceph integrated install .. 18.1.8 | 14:52 |
| noonedeadpunk | can we get another vote for https://review.opendev.org/#/c/734834/ ? | 14:59 |
| mgariepy | done | 15:01 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: WIP Centos-8 support https://review.opendev.org/724721 | 15:03 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-tests master: Do not set alternatives before package is present https://review.opendev.org/731585 | 15:06 |
| *** maharg101 has quit IRC | 15:07 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: ironic: de-compose inventory https://review.opendev.org/735168 | 15:07 |
| *** itandops has quit IRC | 15:14 | |
| *** itandops has joined #openstack-ansible | 15:15 | |
| *** fghaas has joined #openstack-ansible | 15:15 | |
| *** sshnaidm_ has joined #openstack-ansible | 15:15 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/stein: Bump os_tempest to include fix for pinning telemetry plugin to py2.7 version https://review.opendev.org/735065 | 15:15 |
| *** sshnaidm has quit IRC | 15:16 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: inspector: fix service catalog creation https://review.opendev.org/735177 | 15:18 |
| *** maharg101 has joined #openstack-ansible | 15:21 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: ironic: de-compose inventory https://review.opendev.org/735168 | 15:33 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client stable/stein: Use the mimic release rather than luminous https://review.opendev.org/735083 | 15:48 |
| *** markvoelker has joined #openstack-ansible | 15:54 | |
| mgariepy | noonedeadpunk, 2 tries to have it fails. | 15:55 |
| *** mmethot_ has quit IRC | 15:55 | |
| mgariepy | i'll check if i can see something after lunch. | 15:55 |
| * noonedeadpunk crossed fingers | 15:55 | |
| *** markvoelker has quit IRC | 15:56 | |
| *** rpittau is now known as rpittau|afk | 15:58 | |
| *** gyee has joined #openstack-ansible | 16:03 | |
| *** fghaas has quit IRC | 16:11 | |
| mgariepy | noonedeadpunk, race condition ? simply restarting mysql does fix the issue. | 16:29 |
| *** mmethot_ has joined #openstack-ansible | 16:31 | |
| noonedeadpunk | argh | 16:31 |
| noonedeadpunk | ok, we can use block/rescue then//// | 16:32 |
| noonedeadpunk | or dunno | 16:32 |
| mgariepy | arf. | 16:32 |
| noonedeadpunk | that's really annoying | 16:32 |
| mgariepy | let me check a bit more :D) | 16:32 |
| noonedeadpunk | I'm not sure it's worth spending a lot of time... I guess cluster is really ok according to the logs and values I see | 16:37 |
| noonedeadpunk | It just skips updating participant ips for some reason but it does see them... | 16:38 |
| noonedeadpunk | btw would be interesting to check if changes got synced to the "failed" host | 16:38 |
| noonedeadpunk | if yes - then just let's workaround this issue | 16:38 |
| mgariepy | did you saw it on other container than container3 ? | 16:40 |
| noonedeadpunk | yep, on 2nd | 16:40 |
| noonedeadpunk | just never on 1st as it's master I guess | 16:40 |
| noonedeadpunk | https://c0969db12a174f8558fb-5ba6792c7427741c3f6f06d35721722e.ssl.cf1.rackcdn.com/734888/5/check/openstack-ansible-functional-debian-buster/8081375/logs/ara-report/result/9a2728ac-5b7a-47c7-b044-7904825be628/ | 16:41 |
| mgariepy | creating a db on either container2 or container3 does replicate to the other | 16:43 |
| noonedeadpunk | and it's on that state, right? ok, so we're good then | 16:43 |
| jrosser | it's sad thats broken | 16:43 |
| jrosser | we document it in our ops guide as how to check the cluster state | 16:44 |
| jrosser | but yeah, need to move forward..... | 16:44 |
| noonedeadpunk | If stuff is working pretty sure we shouldn't spend much time on it | 16:44 |
| jrosser | yes | 16:45 |
| noonedeadpunk | Hopefully this will be fixed in the next releases... | 16:45 |
| noonedeadpunk | Will probably sumbit a bug | 16:45 |
| *** sshnaidm_ is now known as sshnaidm | 16:46 | |
| *** sshnaidm is now known as sshnaidm|afk | 16:46 | |
| *** cheng1 has quit IRC | 16:47 | |
| mgariepy | http://paste.openstack.org/show/794663/ | 16:47 |
| jrosser | looks like we need this https://review.opendev.org/#/c/731457/5 | 16:47 |
| jrosser | i have a feeling there may be breakage in roles somewhere because of that, but we shall see | 16:47 |
| mgariepy | will need to patch for python interpreter | 16:48 |
| *** cheng1 has joined #openstack-ansible | 16:49 | |
| noonedeadpunk | or maybe we really should restart maria serially.... | 16:49 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Use more reliable option to get cluster ready state https://review.opendev.org/735079 | 16:52 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Use more reliable option to get cluster ready state https://review.opendev.org/735079 | 16:54 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_server master: Fix Mariadb 10.3 -> 10.4 upgrade path https://review.opendev.org/734888 | 17:03 |
| mgariepy | it does occur on centos7 as well ;) | 17:05 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [WIP] Make upgrade jobs get role relevant versions https://review.opendev.org/710849 | 17:06 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Stop installing python2.7 on debian/focal https://review.opendev.org/731457 | 17:09 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Add functional tests for ubuntu focal https://review.opendev.org/731251 | 17:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon stable/stein: Horizon configuration to support domains dropdown https://review.opendev.org/734834 | 17:17 |
| *** udesale_ has quit IRC | 17:19 | |
| *** itandops has quit IRC | 17:34 | |
| mgariepy | should we merge : https://review.opendev.org/#/c/734676/ | 17:34 |
| jrosser | mgariepy: that is making the focal tests pass i think, which is good | 17:35 |
| mgariepy | we will need to fix a bunch of host_vars in tests that set the interpreter to something tho. | 17:36 |
| mgariepy | https://github.com/openstack/openstack-ansible-galera_server/blob/master/tests/host_vars/localhost.yml#L19 | 17:38 |
| jrosser | oh my | 17:39 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-galera_server master: remove ansible_python_interpreter https://review.opendev.org/735204 | 17:39 |
| *** itandops has joined #openstack-ansible | 17:49 | |
| *** cheng1 has quit IRC | 17:59 | |
| *** cheng1 has joined #openstack-ansible | 18:00 | |
| jrosser | mgariepy: do you want to take the depends-on off https://review.opendev.org/#/c/734676/ to free it from the focal maridb patch? | 18:04 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-tests master: auto-detect interpreter https://review.opendev.org/734676 | 18:15 |
| mgariepy | done | 18:15 |
| noonedeadpunk | ok so https://review.opendev.org/#/c/710849/ seems works in terms of depends-on I think. But has some other issue with link to the repos... | 18:29 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-tests master: auto-detect interpreter https://review.opendev.org/734676 | 18:43 |
| mgariepy | anything needs some review and / or some help ? | 18:54 |
| jrosser | noonedeadpunk: that is odd - /openstack/src should always be a symlink to the zuul provided repos | 18:55 |
| jrosser | noonedeadpunk: though that does raise an interesting question for the keystone upgrades checks using OSA | 18:55 |
| jrosser | we probably can't do depends-on for keystone itself in an upgrade? | 18:57 |
| jrosser | oh maybe yes, if the zuul repo is prepared with master including the depends-on patch that should work | 18:57 |
| noonedeadpunk | we should add it to required ones I guess | 18:57 |
| noonedeadpunk | for zuul | 18:57 |
| noonedeadpunk | so that zull clone keystone... | 18:57 |
| jrosser | all the service repos already are | 18:58 |
| jrosser | thats whats failed for your patch | 18:58 |
| jrosser | the service repos not the ansible role repos | 18:58 |
| noonedeadpunk | oh, indeed https://opendev.org/openstack/openstack-ansible/src/branch/master/zuul.d/jobs.yaml#L96 | 18:58 |
| jrosser | this is how that works https://github.com/openstack/openstack-ansible/commit/b3932f0e8241344ef9ece8c028ebb70bd548defc | 18:58 |
| noonedeadpunk | I thought we have only our services | 18:58 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create stable/ussuri: Wait for systemd-tmpfiles-setup service to complete after container restart https://review.opendev.org/734089 | 18:59 |
| jrosser | /openstack/src always should be the zuul repos, then it works for metal and containers | 18:59 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [WIP] Make upgrade jobs get role relevant versions https://review.opendev.org/710849 | 18:59 |
| noonedeadpunk | so maybe the point is that I used trailing / | 19:00 |
| noonedeadpunk | so link or copy was incorrect | 19:00 |
| jrosser | is there anything to review merge today? | 19:01 |
| *** miloa has quit IRC | 19:03 | |
| jrosser | this is interesting https://zuul.opendev.org/t/openstack/build/9314f4dde5364ecfb87ecbefeebdd5c5/log/job-output.txt#3715-3720 | 19:04 |
| jrosser | different behaviour on the arm nodes? | 19:04 |
| *** pcaruana has joined #openstack-ansible | 19:05 | |
| noonedeadpunk | seems like that.... | 19:05 |
| noonedeadpunk | but image is different and it's probably prepared comletely different as well... | 19:06 |
| jrosser | noonedeadpunk: the arm images are the new 'clean' builds | 19:19 |
| jrosser | that will change in the x86 ones when https://review.opendev.org/#/c/734428/ merges which is anytime now | 19:20 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] Test arm64 https://review.opendev.org/618305 | 19:23 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer stable/stein: Make user provided content deployable https://review.opendev.org/733387 | 19:28 |
| mgariepy | i think this one is ready to get merged. https://review.opendev.org/#/c/734676/ | 19:29 |
| mgariepy | focal is failing because of the galera stuff and centos8 for.. you know centos8.. | 19:30 |
| mgariepy | zull is so slow. haha | 19:30 |
| *** fghaas has joined #openstack-ansible | 19:31 | |
| jamesdenton | mgariepy kick the tires on OVN yet? | 19:32 |
| mgariepy | jamesdenton, not quite yet | 19:33 |
| mgariepy | id like to have focal + ovn + U .. in 2-3 weeks.. | 19:33 |
| mgariepy | the tests on bionic aio seemed to work ok. | 19:34 |
| jamesdenton | ambitious | 19:34 |
| noonedeadpunk | jrosser: ouch... | 19:34 |
| mgariepy | well i would like not to have to distro-upgrade a new install a couple weeks in. | 19:34 |
| noonedeadpunk | but maybe at least setuptools will be present... | 19:34 |
| jamesdenton | looks like you're making good progress, though | 19:35 |
| noonedeadpunk | eventually we won't need this task once it merge | 19:35 |
| noonedeadpunk | So won't face with the issue | 19:35 |
| jrosser | noonedeadpunk: i've taken the pip removal off the arm job to see what happens | 19:37 |
| noonedeadpunk | oh, um... | 19:40 |
| noonedeadpunk | oh, yeah, you did it just now) | 19:41 |
| jrosser | its got past that https://zuul.openstack.org/stream/788be8169e1e40afa24b7c9ea1a41a71?logfile=console.log | 19:44 |
| mgariepy | i'll push a few patch for the localhost interpreter stuff. | 19:58 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/ansible-role-python_venv_build master: remove ansible_python_interpreter https://review.opendev.org/735224 | 19:58 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/ansible-role-systemd_mount master: remove ansible_python_interpreter https://review.opendev.org/735225 | 19:58 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/ansible-role-systemd_networkd master: remove ansible_python_interpreter https://review.opendev.org/735226 | 19:58 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/ansible-role-systemd_service master: remove ansible_python_interpreter https://review.opendev.org/735227 | 19:58 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-galera_server master: remove ansible_python_interpreter https://review.opendev.org/735228 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-haproxy_server master: remove ansible_python_interpreter https://review.opendev.org/735229 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-memcached_server master: remove ansible_python_interpreter https://review.opendev.org/735230 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-nspawn_container_create master: remove ansible_python_interpreter https://review.opendev.org/735231 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-os_aodh master: remove ansible_python_interpreter https://review.opendev.org/735232 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-os_rally master: remove ansible_python_interpreter https://review.opendev.org/735233 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-os_swift master: remove ansible_python_interpreter https://review.opendev.org/735234 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-rabbitmq_server master: remove ansible_python_interpreter https://review.opendev.org/735235 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-repo_server master: remove ansible_python_interpreter https://review.opendev.org/735236 | 19:59 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-tests master: remove ansible_python_interpreter https://review.opendev.org/735237 | 19:59 |
| mgariepy | there are still a few places where it will probably be needed to get removed tho. | 20:01 |
| *** fghaas has quit IRC | 20:17 | |
| *** cheng1 has quit IRC | 20:22 | |
| *** cheng1 has joined #openstack-ansible | 20:22 | |
| *** gyee has quit IRC | 20:36 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: ironic: de-compose inventory https://review.opendev.org/735168 | 20:37 |
| *** jcath has quit IRC | 20:39 | |
| *** itandops has quit IRC | 20:41 | |
| *** itandops has joined #openstack-ansible | 20:41 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: ironic: de-compose inventory https://review.opendev.org/735168 | 20:41 |
| *** mgariepy has quit IRC | 20:44 | |
| *** mgariepy has joined #openstack-ansible | 20:48 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: wip: split out to ironic-inspector-conductor https://review.opendev.org/735244 | 20:52 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: wip: split out to ironic-inspector-conductor https://review.opendev.org/735244 | 21:04 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: wip: split out to ironic-inspector-conductor https://review.opendev.org/735244 | 21:22 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: wip: split out to ironic-inspector-conductor https://review.opendev.org/735244 | 21:27 |
| *** cheng1 has quit IRC | 21:37 | |
| *** cheng1 has joined #openstack-ansible | 21:39 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: wip: split out to ironic-inspector-conductor https://review.opendev.org/735244 | 21:45 |
| *** markvoelker has joined #openstack-ansible | 21:49 | |
| *** markvoelker has quit IRC | 21:54 | |
| *** itandops has quit IRC | 22:12 | |
| *** threestrands has joined #openstack-ansible | 22:20 | |
| *** jbadiapa has quit IRC | 22:26 | |
| *** markvoelker has joined #openstack-ansible | 22:49 | |
| *** markvoelker has quit IRC | 22:54 | |
| *** tosky has quit IRC | 23:10 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: wip: split out to ironic-inspector-conductor https://review.opendev.org/735244 | 23:12 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!