Monday, 2020-09-14

*** cshen has joined #openstack-ansible00:18
*** cshen has quit IRC00:22
fridtjof[m]noonedeadpunk: i finished triaging, here's the bug report :D https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/189548701:15
openstackLaunchpad bug 1895487 in qemu (Ubuntu) "'qemu-img convert' performance regression when target is a LVM volume" [Undecided,New]01:15
watersjwhere do you look to see why your router on your flat network is not pingable?01:21
watersji am able to ping south side router (private side), but not public/flat side). Also from public net I can't ping router northside (sry got terminology wrong).01:23
*** spatel has joined #openstack-ansible01:58
*** cshen has joined #openstack-ansible02:18
*** cshen has quit IRC02:22
watersjphysical_interface_mappings = flat:eno2,vlan:br-vlan , is it ok having flat and vlan on same physical interface?02:24
*** rh-jelabarre has joined #openstack-ansible02:33
*** cshen has joined #openstack-ansible02:39
*** cshen has quit IRC02:43
*** rh-jelabarre has quit IRC03:25
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-ansible04:33
*** spatel has quit IRC04:37
*** cshen has joined #openstack-ansible04:39
*** cshen has quit IRC04:44
*** dasp has quit IRC05:48
*** fresta has joined #openstack-ansible06:12
*** miloa has joined #openstack-ansible06:18
*** pcaruana has joined #openstack-ansible06:20
noonedeadpunkfridtjof[m]: oh, thanks, that's interesting. fortunatelly I just skipped stein I did all upgrades R->T :)06:51
noonedeadpunkbut htat's really good to know06:51
*** pcaruana has quit IRC06:56
*** pcaruana has joined #openstack-ansible07:09
*** cshen has joined #openstack-ansible07:14
jrossermorning07:35
jrossernoonedeadpunk: i managed to reproduce the errors we get upgrading ansible past 2.9.9, this is really odd http://paste.openstack.org/show/797821/07:42
jrosser3 different behaviours on 3 different versions07:42
*** andrewbonney has joined #openstack-ansible07:42
*** MickyMan77 has joined #openstack-ansible07:57
*** shyamb has joined #openstack-ansible08:26
*** tosky has joined #openstack-ansible08:27
noonedeadpunkomg08:31
noonedeadpunkI feel veeeery comfortable on 2.9.9 :p08:31
noonedeadpunkjrosser: btw have you tried out 2.10?08:32
jrosseri have a wip patch and it fails the same08:32
noonedeadpunk'item' is undefined?08:33
noonedeadpunkI haven't read release notes yet, but maybe they've droped with_items and force to use loops now.....08:34
noonedeadpunkwill try to setup same based on your paste08:35
*** gokhani has joined #openstack-ansible08:50
*** sshnaidm|pto is now known as sshnaidm09:10
*** itandops has joined #openstack-ansible09:12
*** shyamb has quit IRC09:23
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/rocky: How to upgrade from xenial to bionic on rocky  https://review.opendev.org/71080309:32
*** spatel has joined #openstack-ansible09:38
*** shyamb has joined #openstack-ansible09:41
*** spatel has quit IRC09:44
*** gixx has quit IRC09:45
*** gixx has joined #openstack-ansible09:45
*** shyamb has quit IRC10:03
*** shyamb has joined #openstack-ansible10:03
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Install unzip package to unpack lxc image  https://review.opendev.org/75172410:07
fridtjof[m]noonedeadpunk: yeah, i'm planning to jump on train (lol) soon anyway, just want to have everything in working order before I do that. Right now, I just manually replaced qemu-img with the default bionic release ^^'10:14
noonedeadpunkhm, maybe we should bin corresponding package so it won't be installed from uca as well...10:17
noonedeadpunks/bin/pin10:17
watersjmy router's port is down(?) Stdout: Interface "qg-974bf301-dc" is down. VMs in my private network can't get out. I can ping gateway in private network just not getting out. On public side, ip of router is down.10:25
watersjlooking in NS everything seems ip ip netns exec qrouter-c05f804f-4cc1-4b9c-897b-a4d6e431fd77 ping [private vm, private gw, public gw] works10:26
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-haproxy_server master: Define http-01 params with already provided variables  https://review.opendev.org/75131610:38
noonedeadpunkwatersj: have you tried to restart affected l3 agent?10:39
noonedeadpunkas this feels as some l3 router configuration issue (in terms of iptables probably)10:40
jrossernoonedeadpunk: i may have a fix10:54
jrosseransible_host: "{{ hostvars[item]['ansible_host'] }}"10:54
jrosserbecomes10:54
jrosseransible_host: query('hostvars[item]', ansible_host)10:55
noonedeadpunkfeels like workaround...10:58
noonedeadpunkor it's preffered way of handling that nowadays?10:58
noonedeadpunk(I mean they suppose to do that way?)10:58
noonedeadpunkand how to handle ways when we need to descend deeper? ie ip addresses?10:59
jrossernot sure tbh, i was just going through bugs and found this https://github.com/ansible/ansible/issues/7109211:00
gokhanihi folks, our ssl is expired and I will renew it. I only use ssl at haproxy side. ıs it enough to change private.key and haproxy.pem files ? I don't want to run haproxy playbook.11:04
*** SecOpsNinja has joined #openstack-ansible11:07
watersjnoonedeadpunk, yes I have11:08
*** shyam89 has joined #openstack-ansible11:08
noonedeadpunkhaproxy.pem includes both certificate and private kay, so you need to update only /etc/ssl/private/haproxy.pem11:09
SecOpsNinjahi everyone. when trying to create a k8s cluster with magnum im getting "Failed to load default keystone auth policy: FileNotFoundError: [Errno 2] No such file or directory: '/etc/magnum/keystone_auth_default_policy.json'" but i don't find it in os_magnum ansible role. do i need to create it manually?11:10
noonedeadpunkwatersj: is port owner is correct as well?11:10
*** shyamb has quit IRC11:10
gokhaninoonedeadpunk, thanks I will change only .pem file.11:11
noonedeadpunkSecOpsNinja: hm........11:14
noonedeadpunklet me see11:14
watersjnoonedeadpunk, devive_owner = network:router_gateway (if that is what you mean)11:14
SecOpsNinjanoonedeadpunk, thanks. for what im seeing in the magnum.conf.j2 there isn't anything defined regarding kubernetes setion so i suposse its using the default  like stated were https://docs.openstack.org/magnum/latest/configuration/sample-config.html11:15
noonedeadpunkSecOpsNinja: yeah, we're missing it somehow o_O11:16
*** jbadiapa has joined #openstack-ansible11:16
noonedeadpunkhere's sample in the meanwhile https://opendev.org/openstack/magnum/raw/branch/stable/ussuri/etc/magnum/keystone_auth_default_policy.sample11:16
noonedeadpunkI guess you've enabled keystone auth somehow explicitly?11:18
noonedeadpunkie keystone_auth_enabled11:19
SecOpsNinjanoonedeadpunk, i was also cheking that the coumentation is not very updated because regarding magnum there is still reference to fedora-atomic-27 but we should be now using fedora-coreos (that s whats im trying to use in k8s master and node). Regarding that sample do i need to chamge dos $proJECT_ID variables or that are internal to magnum?11:19
noonedeadpunkyou shouldn't change variables I guess, but may want to adjust roles11:20
noonedeadpunkI mean I guess this file is not required unless keystone_auth_enabled is set explicitly to true11:20
*** shyam89 has quit IRC11:21
SecOpsNinjanoonedeadpunk, yep i didn't change the default settings in magnum. i only had to put the external public endpoint with valid tls so it was able to comunicate with keystone11:21
SecOpsNinjabut i will try find documentation regarding the point keystone_auth_Enabled11:22
noonedeadpunkI just get it from commit introduced it https://opendev.org/openstack/magnum/commit/59da4e25a6a31e296f8ad734395a79101576942411:22
noonedeadpunkmaybe it's set in the template level even11:23
noonedeadpunkyeah, it's template label lol https://docs.openstack.org/magnum/latest/user/#keystone-auth-enabled11:24
noonedeadpunkSecOpsNinja: ^11:24
noonedeadpunkanyway I think it's worth creating that file....11:24
SecOpsNinjayep i will try to first to create that file and  see if i can get it to work11:24
noonedeadpunknot sure about specific groups though....11:25
noonedeadpunkI think you can actually do not provide that flag to the template at the first place...11:25
noonedeadpunkbut yeah, creating file should also work11:25
*** shyamb has joined #openstack-ansible11:36
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum master: Add deployment of keystone_auth_default_policy  https://review.opendev.org/75176711:37
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum master: Simplify service creation  https://review.opendev.org/75176811:41
*** dave-mccowan has joined #openstack-ansible11:48
*** rh-jelabarre has joined #openstack-ansible11:50
*** rh-jelabarre has quit IRC11:51
*** rh-jelabarre has joined #openstack-ansible11:51
*** rh-jelabarre has quit IRC12:00
*** rh-jelabarre has joined #openstack-ansible12:00
openstackgerritJames Gibson proposed openstack/openstack-ansible-ops master: Change ansible tests to prefer Python3 over Python2 in vitualenv  https://review.opendev.org/75177312:15
*** shyam89 has joined #openstack-ansible12:29
*** shyamb has quit IRC12:31
*** shyam89 has quit IRC12:31
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Remove support for Centos-7  https://review.opendev.org/74212412:42
openstackgerritJames Gibson proposed openstack/openstack-ansible-ops master: Change ansible tests to prefer Python3 over Python2 in vitualenv  https://review.opendev.org/75177312:47
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Remove Centos-7 support  https://review.opendev.org/74210012:54
*** spatel has joined #openstack-ansible12:56
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Migrate LXC config keys to LXC3  https://review.opendev.org/74212212:56
*** lkoranda has joined #openstack-ansible12:59
spateljrosser: or noonedeadpunk - could you take care of this patch, I may not get time to touch it today - https://review.opendev.org/#/c/749379/13:01
noonedeadpunkspatel: I'd rather merge https://review.opendev.org/#/c/749540/3 instead13:02
noonedeadpunkI didn't notice yours one when was placing this:(13:02
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum master: Define condition for the first play host one time  https://review.opendev.org/75178013:04
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Define condition for the first play host one time  https://review.opendev.org/75178313:08
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_mistral master: Define condition for the first play host one time  https://review.opendev.org/75178413:10
*** cshen has quit IRC13:11
spatelnoonedeadpunk: i am also wondering what holding it to get merge?13:11
spatelmay be tomorrow meeting?13:11
spatelBRB13:12
*** spatel has quit IRC13:12
*** irclogbot_0 has quit IRC13:19
noonedeadpunkI gave my vote :p13:22
*** irclogbot_0 has joined #openstack-ansible13:24
*** mnaser has quit IRC13:32
*** mnaser has joined #openstack-ansible13:32
*** mnaser has quit IRC13:32
*** mnaser has joined #openstack-ansible13:32
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_neutron master: Define condition for the first play host one time  https://review.opendev.org/75179013:32
*** d34dh0r53 has joined #openstack-ansible13:54
*** sshnaidm is now known as sshnaidm|afk14:07
*** cshen has joined #openstack-ansible14:09
*** miloa has quit IRC14:16
*** baptistemm has joined #openstack-ansible14:29
baptistemmhello14:29
*** lkoranda has quit IRC14:36
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Ensure gzip is present to unpack lxc image  https://review.opendev.org/75172414:45
jrosserbaptistemm: hello! if you have a question just ask :)14:49
noonedeadpunkjrosser: btw have you seen https://bugs.launchpad.net/openstack-ansible/+bug/1895533 ?14:54
openstackLaunchpad bug 1895533 in openstack-ansible "/var/log bind mount overshadows /var/log/journal bindmount in lxc container setup" [Undecided,New]14:54
noonedeadpunkI just don't sure I understand... I guess we don't bind mount directory for journald? But like deployer does thet with lxc_container_bind_mounts or smth?15:00
noonedeadpunkoh, we actually do....15:03
jrosserwhere is that15:06
noonedeadpunkdunno, but have it in aio...15:06
noonedeadpunkwhich I spawned just with gate-check-commit.sh15:07
noonedeadpunkso in container I have http://paste.openstack.org/show/797842/15:07
noonedeadpunkbut I can't find how we do this at the moment:)15:08
noonedeadpunkjrosser: ok, that's how we do it https://opendev.org/openstack/openstack-ansible-lxc_container_create/src/branch/master/tasks/lxc_container_config.yml#L284-L29115:13
noonedeadpunktbh.....15:13
jrosserright now i'm not sure i understand what is wrong15:14
noonedeadpunkso we define bind mounts in 2 places...15:14
noonedeadpunkfirst one is during lxc_container_create, and the second one is common-tasks/os-lxc-container-setup.yml15:15
noonedeadpunkand we bind mount /var/log/journal, and on top of it /var/log15:15
noonedeadpunkwhich makes /var/log/journal overlapped15:15
noonedeadpunkso we'd rather mount /var/log and only after /var/log/journal....15:15
noonedeadpunkor smth like that15:15
noonedeadpunkso we should invent smth with ordering in lxc config....15:17
*** ianychoi has joined #openstack-ansible15:24
jamesdentonjrosser The results of the OVS provider network test, FYI: http://paste.openstack.org/show/797843/15:25
jrosserjamesdenton: wouldnt we expect to need to make a network: entry for the flat type anyway?15:31
jamesdentonyes, but as of right now the logic seems to expect two different bridges, one for vlan networks and another for (a) flat network. When in reality, it could be the same bridge.15:32
jamesdentonYou can set.. "flat_networks = *" in ml2_conf.ini to handle it15:34
jrosseroh - this isnt 'flat:br-provider' <- name:interface15:35
jrossernot type:interface15:35
jamesdentonwell, it's both. You potentially have 'vlan:br-provider' with type:vlan and 'flat:br-provider' with type:flat. The logic seems to create two bridge mappings, vlan:br-provider and flat:br-provider, which are not unique and the agent exists.15:36
*** cshen has quit IRC15:55
jrosserjamesdenton: humm, so you think we need a special case on 'flat_networks' in the template.....15:56
jrosserall feels a bit messy somehow15:56
jamesdentonseems a bit late to fix, no? :D16:06
jamesdentons/fix/change16:06
*** MickyMan77 has left #openstack-ansible16:16
*** spatel has joined #openstack-ansible16:26
spatelback16:28
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [reno] Ignore backported release note  https://review.opendev.org/75185516:32
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [reno] Ignore backported release note  https://review.opendev.org/75185516:33
baptistemmjrosser: yes I had but I was side-tracked by urgency. it's me who reported https://storyboard.openstack.org/#!/story/2008135 I wanted to see how the module was supposed to work.16:38
*** cshen has joined #openstack-ansible16:46
*** dasp has joined #openstack-ansible16:46
*** cshen has quit IRC16:57
*** spatel has quit IRC16:57
*** dasp has quit IRC16:58
*** dasp has joined #openstack-ansible17:00
*** andrewbonney has quit IRC17:31
*** SecOpsNinja has left #openstack-ansible17:35
*** Jeffrey4l has quit IRC18:08
*** spatel has joined #openstack-ansible18:35
*** Jeffrey4l has joined #openstack-ansible18:48
*** spatel has quit IRC18:52
*** cshen has joined #openstack-ansible18:53
*** Jeffrey4l has quit IRC18:55
*** Jeffrey4l has joined #openstack-ansible18:57
*** cshen has quit IRC18:57
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-lxc_hosts master: Wait for aria2c to finish  https://review.opendev.org/75172419:09
noonedeadpunkbaptistemm: I think it's worth asking in #openstack-ansible-sig19:12
watersjis it ok to have a vlan on the same interface as the flat network prior to OSA defined vlans ? I am tight on nics and split things out for bridges19:14
noonedeadpunkI do that yes. Eventually I'd say it's better to use some tagged interface as "flat", as flat!=vlan for neutron or smth...19:17
watersjflat is the default vlan 1,19:18
noonedeadpunkanyway neutron is going to pick up control on vlan interface to create new vlans on it, so probably having like bond0.20 for flat is ok19:18
noonedeadpunkI was facing issues when had vlan, vxlan and flat on the same physocal netwrok, in case flat was untagged (or with id1)19:19
noonedeadpunkbut maybe things have changes since then....19:19
baptistemmthanks noonedeadpunk19:40
*** spatel has joined #openstack-ansible20:38
*** spatel has quit IRC20:40
*** cshen has joined #openstack-ansible20:53
*** cshen has quit IRC20:58
jrosserlogan-: regarding https://github.com/openstack/openstack-ansible-galera_server/commit/3d405dfd52c0a5059cefd877fd578114bcdd912d21:15
jrosserwas there an ansible issue created for that?21:15
*** itandops has quit IRC21:16
jrosserlogan-: there is some related discussion here https://github.com/ansible/ansible/issues/7174521:23
logan-I don't think so. I was seeing it with the lxd connection plugin. I've actually been working around this using a different method now that doesn't require modifying roles.. http://paste.openstack.org/raw/797855/21:27
jrosserit's currently preventing us going past ansible 2.9.9, the galera role fails with the current code21:29
logan-Interesting. Something major must have changed between 2.9.9 -> 2.9.10. I have a totally different problem (but still connection related) that is blocking me from going past 2.9.9 in a project.21:31
jrossercould you comment on the ansible issue with your LXD workaround?21:31
logan-Sure21:32
jrosserseems there is something funky here21:32
logan-Yeah have you seen any problem with local connections? This is my >2.9.9 blocker http://paste.openstack.org/raw/797857/21:37
jrosserhmm no, nothing like that21:40
jrosserthough i've not got OSA beyond galera yet so there could be more surprises21:40
logan-If it helps I think it would be fine to revert https://github.com/openstack/openstack-ansible-galera_server/commit/3d405dfd52c0a5059cefd877fd578114bcdd912d21:47
*** tosky has quit IRC22:33
*** cshen has joined #openstack-ansible22:54
*** cshen has quit IRC22:58

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!