Thursday, 2020-04-23

*** watersj has joined #openstack-ansible00:19
fridtjof[m]noonedeadpunk: ah, thanks. i'll keep that in mind. it's not necessary though if I start off with just one compute host named "compute2", and then add "compute1" later, right?00:28
*** spatel has joined #openstack-ansible00:43
*** d34dh0r53 has quit IRC00:48
*** spatel has quit IRC00:48
openstackgerritMerged openstack/openstack-ansible master: Update global-requirement-pins  https://review.opendev.org/71774801:03
*** spatel has joined #openstack-ansible01:13
*** spatel has quit IRC01:18
watersjlooking at deployment guide of osa, does it use openvswitch or linux-bridge? If bridge can it be conf02:09
*** spatel has joined #openstack-ansible02:27
*** spatel has quit IRC02:29
*** spatel has joined #openstack-ansible02:29
*** spatel has quit IRC02:30
*** threestrands has joined #openstack-ansible02:33
*** hwoarang has quit IRC02:43
*** hwoarang has joined #openstack-ansible02:43
*** rh-jelabarre has quit IRC03:07
*** _mmethot_ has joined #openstack-ansible04:18
*** mmethot_ has quit IRC04:21
*** udesale has joined #openstack-ansible04:22
*** evrardjp has quit IRC04:35
*** evrardjp has joined #openstack-ansible04:35
*** cshen has joined #openstack-ansible05:00
*** cshen has quit IRC05:05
*** gyee has quit IRC05:29
*** cshen has joined #openstack-ansible06:00
*** this10nly has joined #openstack-ansible06:44
*** KeithMnemonic has quit IRC07:12
jrosserwatersj: you can choose linucbridge or ovs. the defaul setting if you don’t change anything is linuxbridge07:18
*** tosky has joined #openstack-ansible07:24
*** yolanda has joined #openstack-ansible07:24
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata  https://review.opendev.org/72106407:27
*** rpittau|afk is now known as rpittau07:39
admin0watersj, unless you know that your requirement needs an ovs, you can stick to linuxbridge08:00
Adri2000hello08:22
Adri2000I'm running queens (upgrade planned really soon :)) and after deploying the latest OSA version (from tag 17.1.16 to 17.1.17), swift-object-auditor doesn't want to start anymore, failing with ImportError: cannot import name viewkeys08:24
kleiniadmin0: what can be possible requirements to choose ovs? I am struggling, too, between choosing lb or ovs. I have a "handcrafted" OpenStack beginning with Icehouse release and there the install guides used ovs. Today they all use lb.08:25
Adri2000I tracked down the issue and found an ugly workaround that involves switching the order of two import lines in /openstack/venvs/swift-17.1.17/lib/python2.7/site-packages/swift/obj/auditor.py, "from eventlet import Timeout" and "from swift.obj import diskfile, replicator" (which in turns does "from six import viewkeys")08:25
Adri2000can't find what's wrong, can't see a significant diff between the 17.1.16 and 17.1.17 venvs that would explain the issue (eventlet and six versions didn't change) ... not sure how to debug further08:26
Adri2000summary of my strange problem: https://pastebin.com/6fLcVPsK08:29
*** threestrands has quit IRC08:33
*** sshnaidm|afk has quit IRC08:55
*** gshippey has joined #openstack-ansible08:56
noonedeadpunkAdri2000: I think that it might be already different version of six isntalled09:35
noonedeadpunkas we upper-constraints might be changed09:35
admin0kleini, for example, would you/users be doing EPA/NFV tests .. like connecting multiple t-rex traffic generators to multiple ports and trying to performance test an application on how it behaves on say 20gb , 50gb, 100gb traffic ?09:36
admin0generally speaking, if you are not crossing 10gbit in network traffic, lb works just fine ( simple to setup, work, troubleshoot, no headaches etc)09:37
admin0ovs is if you know your application will need it or need to do performance testing on the 20gbit+ network traffic, epa or nfv requirements etc09:37
kleiniour setup is target for performance tests of dovecot with 20G network connections currently. so ovs is maybe a good choice.09:38
admin0single dovecot instance is needed to cross 20G network barrier ?09:40
kleini384 core, 3TB memory cluster setup with scality storage09:41
admin0that does not answer :)09:41
admin0a cluster requirement is different from a per-instance requirement09:42
kleinilet's say it is a single IMAP server instance but a highly clustered cloudy setup. I am unfortunately not in the details.09:43
admin0use lb :)09:43
admin0thats easy to answer because i know dovecot09:43
kleiniwe develop dovecot :)09:43
admin0:D09:43
admin0then you know better in its architecture and how you would set it up in a SaaS or PaaS09:44
admin0type offering09:44
kleiniany concrete reason, why to use lb?09:44
admin0ease of user, setup, troubleshooting, less headaches09:44
kleiniokay, thanks. currently ovs is easier for me than lb, as I am familiar with ovs09:45
admin0well, it also depends on the other guys in the team .. the guy who gets call at 4am or does shift on holidays etc09:46
kleiniI will teach them ;-)09:46
admin0then its an easy choice right .. stick with ovs09:46
*** pcaruana has quit IRC09:50
*** sshnaidm has joined #openstack-ansible10:14
*** rpittau is now known as rpittau|bbl10:21
kleiniso one headache is currently, that OVS bridge with IP can not communication with macvlan type bridge in container. so host <-> container communication is broken there. I am still searching, what the cause could be.10:26
*** pcaruana has joined #openstack-ansible10:27
jrosserkleini: linuxbridge and OVS performance is basically the same - there is not a valid reason to choose between them on the grounds of that10:40
jrosserwe don't normally recommend OVS on the control plane nodes10:42
jrosserimho your control plane is being made too complex with nspawn + ovs + macvlan10:43
jrosserit is very much simpler with lxc + bridges10:43
kleiniso linux bridge is able to perform same as ovs on br-vxlan and br-vlan, so the tunnel bridge and the public network bridge? those will be the bridges with the most traffic between VMs10:51
jrosserkleini: watch the video here https://www.openstack.org/summit/denver-2019/summit-schedule/events/23525/pushing-packets-how-do-the-ml2-drivers-stack-up10:52
kleinithanks!10:53
jrosserit's totally fine to pick between ovs and linuxbridge on the compute/network hosts for tenant traffic - i just don't think its necessarily a decision to make based on performance10:53
jrosseryou make have a preference based on familiarity, and thats totally fine10:53
jrosserbut i think that control plane nodes should be considered differently10:54
kleiniI am not only talking about br-mgmt, br-vxlan, br-vlan but also about the ML2 driver. Regarding the ML2 driver I have the choice only once, when deploying the cloud. I can not change that later. That's why I'm hesitating and asking that much.10:58
kleiniwill watch the video10:58
*** watersj has quit IRC11:01
openstackgerritAndreas Jaeger proposed openstack/ansible-role-uwsgi master: Cleanup py27 support  https://review.opendev.org/72227711:03
*** thuydang has joined #openstack-ansible11:42
Adri2000noonedeadpunk: pip list, in both venvs, report the same version of six :(11:51
*** rh-jelabarre has joined #openstack-ansible11:52
*** rpittau|bbl is now known as rpittau12:02
*** udesale_ has joined #openstack-ansible12:20
*** udesale has quit IRC12:22
admin0how we deploy is use lb in the controllers and ovs in the computes12:35
admin0where some of the compute nodes are also network ndoes12:35
admin0controllers are pure controllers only12:35
kleiniand ovs as ml2 driver?12:49
jrosseradmin0: "how we deploy is use lb in the controllers" <- you mean you use bridges on the controller, not linuxbridge in the ml2 sense (there is room for confusion here!)12:51
openstackgerritMerged openstack/openstack-ansible-os_tempest master: Enable placeholder openstack-tox-molecule job  https://review.opendev.org/67385912:52
kleiniyes, there are bridges on the target hosts to be prepared and an ML2 driver. My current tests show, that OVS on the controller host for bridges breaks nspawn containers talking to the host through macvlan.13:03
openstackgerritMerged openstack/openstack-ansible-os_glance master: Update glance_store variables  https://review.opendev.org/71851313:06
jrosserkleini: it may be expected that you cant communicate with the host with macvlan - there is a similar situation for docker https://blog.oddbit.com/post/2018-03-12-using-docker-macvlan-networks/13:11
*** cshen has quit IRC13:12
*** udesale_ has quit IRC13:16
*** udesale has joined #openstack-ansible13:16
*** udesale has quit IRC13:18
*** udesale has joined #openstack-ansible13:19
kleinithanks for the link jrosser. every day new funny stuff but I learn a lot and get an idea in which directory I should go. thank you very much!13:32
jrosserno worries :)13:32
*** _mmethot_ has quit IRC13:34
jrosserkleini: more stuff for you! https://cloudnull.io/2017/06/nspawning-openstack-ansible/13:35
kleiniI met cloudnull at the Berlin infrastructure summit, talked with him about our current setup according to install guides and that I want to move to some more automation. and he mentioned that nspawn containers are supported but obviously it has still some problems13:41
jrosseryes, kevin did all the work on nspawn but he's now spending his time on tripleo13:43
jrosserthats kind of why we don't have anyone actively developing it13:43
*** mmethot has joined #openstack-ansible13:44
*** cshen has joined #openstack-ansible13:44
kleinithat's sad!13:48
*** cshen has quit IRC13:49
*** d34dh0r53 has joined #openstack-ansible13:54
*** cshen has joined #openstack-ansible13:57
admin0i meant  the br-vlan and br-vxlan on the controllers is done using lb .. not ovs ..  on the compute nodes, its done using ovs13:59
admin0an LTS version of os/osa would be good :)14:00
*** sshnaidm has quit IRC14:01
*** watersj has joined #openstack-ansible14:02
*** sshnaidm has joined #openstack-ansible14:02
*** this10nly has quit IRC14:06
kleinithat's a combination I am heading for now, too. LB on control plane (infra, controller, all containers) and OVS on data plane (compute & network nodes, no container at all)14:34
jrosserkleini: the default config puts the neutron servers on the network nodes - you should probably move those your controllers14:42
kleinidid that already according to your proposal some days ago14:46
kleinithanks again14:46
mnasernoonedeadpunk: can you help out and take care of setting up the ptg slots that can work for everyone?14:48
mnaseri can make myself available anymore, jrosser ?14:48
jrosserhmm?14:48
noonedeadpunkyeah, ok, wil check this out14:49
* noonedeadpunk didn't get the last part as well14:50
noonedeadpunkread through ML thread and looked through ethercalc... Either I don't understand smth or there're too much slots...14:59
noonedeadpunkand what type of team are we? horizontal I guess?15:01
noonedeadpunkjrosser: out of https://ethercalc.openstack.org/126u8ek25noy what do you think the most appropiate time?15:03
noonedeadpunklike I feel we can take slots on tuesday for sure15:04
noonedeadpunkdo we want to take monday as well?15:04
noonedeadpunk13-17 utc sounds ok for me, this will cover mnaser time I guess as well?15:05
noonedeadpunk21- midnight UTC fits we as well actually15:05
noonedeadpunkthe thing is that 13-17 UTC will be the same time with kolla and tripleo15:07
jrosserprobably 17:00 UTC is as late as would be good if we want lots of my team to join15:07
jrosseri.e before 17:00 UTC15:07
noonedeadpunkso I was thinking to take from 13 till 1715:08
noonedeadpunkoh wait15:08
noonedeadpunkjrosser: so what's your suggestion?15:08
* noonedeadpunk for second became unsure what UTC is15:09
jrosseri have to google "utc time" each time just to double check15:09
noonedeadpunkI think utc it's actually your native one :D15:10
jrosserexcept my wall clock is 1 hour later than that right now15:10
noonedeadpunkah, summer time, right15:10
noonedeadpunkso then we should take just till 16?15:11
jrosseri'm missing something here - do we need to choose particular days?15:12
noonedeadpunkyeah, from the first part of the schedule...15:12
noonedeadpunkhttp://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.html15:12
noonedeadpunkbut looking at tripleo we might take even several15:12
jrosserand we are concerned about the overlap with kolla/tripleo?15:14
noonedeadpunknot sure that we are... but maybe some ppl would like to join both15:15
jrosseri'm asking folk here if they have any preference15:20
noonedeadpunkAnd I'm probably need to look at https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-contrib-docs.html as well....15:21
*** sshnaidm has quit IRC15:22
noonedeadpunkmgariepy: do you have any preferences regarding ptg?15:24
*** pcaruana has quit IRC15:27
mgariepyi would be ok from 12:00pm utc to 8:00pm utc.15:39
mgariepybut i'm not 100% sure i'll join or have a lot of stuff to contribute.15:39
*** pcaruana has joined #openstack-ansible15:39
mnaseri can work with any timezone :)15:41
mgariepymnaser, you are a machine :) haha15:45
noonedeadpunkSo I'd say 13UTC-17UTC then, if we dont mind tripleo and kolla being at the same time15:51
noonedeadpunkand jrosser and gshippey would be able to join15:52
noonedeadpunkmnaser: do we want to book for 2 days or just 1 would be enough?15:52
guilhermespas mnaser i can also work with any timezone :)15:52
*** gyee has joined #openstack-ansible15:53
noonedeadpunkactually the only which I don't like is 4UTC - 8 UTC....15:53
jrosser13-17UTC works for us here15:54
noonedeadpunkok, nice15:54
jrosserbut it's cool to have it longer too, don't limit it to that15:54
gshippeyanytime works for me really - but less keen on being up through the night15:54
*** sshnaidm has joined #openstack-ansible15:55
noonedeadpunkjrosser: yeah, I'd agree to prolonge it but not sure if we're able to? I can also maybe take 21-00 - 22-00?15:56
*** udesale_ has joined #openstack-ansible15:57
noonedeadpunkLet's probably take tuesday as well?15:57
noonedeadpunkOk, we can't sign up for more than 4 hours...15:59
noonedeadpunkso, folks, maybe we should signup for 4 hours on tuesday and maybe another 4 hours at wendsday but in different time slot?15:59
*** udesale has quit IRC16:00
*** cshen has quit IRC16:02
noonedeadpunklet's say 13-00 - 17-00 on tuesday and 21-00 - 01-00 on wednesday?16:04
gshippeythat sounds good to me :) I'm quite happy to do the late slot as well16:05
noonedeadpunknot sure I want to pickup monday as pretty sure there will be some technical hiccup16:08
noonedeadpunkok, booked room516:11
mgariepyquick one, is anyone seen keystone keystone.token.token_formatters generating a token of 268 char when the limit should 255 ?16:11
noonedeadpunkmnaser: yould you like to fill in survey? https://openstackfoundation.formstack.com/forms/june2020_virtual_ptg_survey16:13
noonedeadpunkor I can do it eventually16:14
*** udesale_ has quit IRC16:23
noonedeadpunkok, submited16:27
*** thuydang has quit IRC16:29
noonedeadpunkcan I have another vote on https://review.opendev.org/#/c/722020/ and https://review.opendev.org/#/c/722021/ ?16:29
*** evrardjp has quit IRC16:35
*** cshen has joined #openstack-ansible16:35
*** evrardjp has joined #openstack-ansible16:35
*** cshen has quit IRC16:40
*** rpittau is now known as rpittau|afk16:49
mgariepyhuh, the len of the token is caused by ldap .. it seems.16:52
noonedeadpunkmgariepy: sorry never was facing smth like that16:54
mgariepyno worries, but my issue was something else, it's only a warning..17:08
mgariepynot an actual error.17:08
mgariepyuuid mapping in ldap stuff, but my issue was with app_credential about 14-18inch from my screen17:11
*** cshen has joined #openstack-ansible17:15
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Retire repo_build and pip_install roles  https://review.opendev.org/72202117:17
*** cshen has quit IRC17:20
openstackgerritMerged openstack/openstack-ansible-os_tempest stable/stein: Pin neutron-tempest-plugin to 0.9.0 for Stein branch.  https://review.opendev.org/72132017:27
openstackgerritMerged openstack/openstack-ansible-tests stable/queens: Retire repo_build and pip_install roles  https://review.opendev.org/72202017:42
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-tests stable/ocata: Retire repo_build and pip_install roles  https://review.opendev.org/72202217:43
*** sshnaidm is now known as sshnaidm|afk17:50
*** yolanda has quit IRC17:55
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-tests stable/ocata: Retire repo_build and pip_install roles  https://review.opendev.org/72202218:03
*** jbadiapa has quit IRC18:04
*** cshen has joined #openstack-ansible18:30
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Add extra controls for frontend redirects  https://review.opendev.org/72241918:34
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Allow extra parameters to be passed to certbot  https://review.opendev.org/72242018:34
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Allow multiple methods of installing certbot  https://review.opendev.org/72242118:34
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Add option to use distro provided certbot package  https://review.opendev.org/72242218:34
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Install self-signed certificate before certbot has run for the first time  https://review.opendev.org/72242318:34
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Add documentation for HA HAProxy with LetsEncrypt certificates.  https://review.opendev.org/72242418:34
jrossermnaser: noonedeadpunk i did a ton of work on the haproxy role ^18:35
jrosserthis lets you have letsencrypt with haproxy + keepalived, i need to test it yet with > 1 node but its looking good so far on a single one18:36
noonedeadpunkoh, cool, will take a look tomorrow18:36
jrosserreview welcomed there as it's quite a big set of changes18:36
* jrosser wanted to get something out there before my day off tomorrow :)18:37
*** KeithMnemonic has joined #openstack-ansible18:46
*** cshen has quit IRC19:06
*** gshippey has quit IRC19:09
*** cshen has joined #openstack-ansible19:33
*** vesper has joined #openstack-ansible19:37
*** vesper11 has quit IRC19:38
*** cshen has quit IRC19:38
*** mgariepy has quit IRC19:47
*** cshen has joined #openstack-ansible20:00
*** cshen has quit IRC20:04
*** mloza has quit IRC20:27
*** mloza has joined #openstack-ansible20:29
*** aedc has quit IRC20:49
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: Updated from OpenStack Ansible Tests  https://review.opendev.org/70947120:50
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_tempest stable/rocky: Pin neutron-tempest-plugin to 0.9.0 for Stein branch.  https://review.opendev.org/72249821:07
*** watersj has quit IRC22:04
*** ansmith has quit IRC22:17
*** tosky has quit IRC22:58
redkriegI'm having a hard time figuring out what I'm doing wrong with my openstack_user_config.yml file.  I want to avoid repeating each host definition in my custom group and compute_hosts.  I've been trying to do something with & and * as is used in the forthcoming example, but I am not getting the syntax right.  Is there documentation somewhere for using these to put the members of multiple groups23:46
redkriegin to one parent group?  http://paste.openstack.org/show/792627/23:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!