Tuesday, 2018-09-25

openstackgerritMichael Vollman proposed openstack/ansible-config_template master: Compare dict vars to determine changed  https://review.openstack.org/58253200:09
*** pmannidi has joined #openstack-ansible01:12
*** mrda has joined #openstack-ansible02:06
*** mrda has quit IRC02:09
*** Bhujay has joined #openstack-ansible02:29
*** dave-mccowan has quit IRC02:29
*** Bhujay has quit IRC02:30
*** Bhujay has joined #openstack-ansible02:30
*** dave-mccowan has joined #openstack-ansible02:50
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert refresh fact to strings  https://review.openstack.org/60495802:54
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Make the elasticsearch plugins optional  https://review.openstack.org/60486702:55
openstackgerritMerged openstack/openstack-ansible-ops master: Add the pip package hvac to support hashicorp vault  https://review.openstack.org/60483103:10
openstackgerritMerged openstack/openstack-ansible-ops master: Fix Kibana dashboard uploading for mirrors and proxies  https://review.openstack.org/60482703:10
spatelcloudnull: ^^03:14
spatelyou there?03:14
*** Bhujay has quit IRC03:41
*** udesale has joined #openstack-ansible03:57
*** mcarden has joined #openstack-ansible04:35
*** Bhujay has joined #openstack-ansible04:37
*** shyamb has joined #openstack-ansible05:14
*** lbragstad has quit IRC05:18
*** shyamb has quit IRC05:26
*** shyamb has joined #openstack-ansible05:31
*** spatel has quit IRC05:33
*** pcaruana has joined #openstack-ansible05:41
*** chkumar|off is now known as chkumar|ruck05:42
*** Bhujay has quit IRC06:00
openstackgerritMerged openstack/openstack-ansible-os_swift master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60412206:02
*** shyamb has quit IRC06:05
*** DanyC has joined #openstack-ansible06:08
*** shyamb has joined #openstack-ansible06:09
*** shyamb has quit IRC06:12
*** shyamb has joined #openstack-ansible06:13
*** Bhujay has joined #openstack-ansible06:14
*** Bhujay has quit IRC06:15
*** DanyC has quit IRC06:16
*** shyamb has quit IRC06:50
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_nova master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60408007:00
hwoarangspotz: hello. I am now :)07:00
*** shyamb has joined #openstack-ansible07:04
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_nova master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60408007:08
evrardjpmorning07:16
evrardjpI will try to find cycles for config_template today -- It's been quite a busy couple of weeks :)07:16
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_swift master: zuul.d: Update jobs for package installations  https://review.openstack.org/60498807:18
openstackgerritMerged openstack/openstack-ansible-os_blazar master: Define cache_timeout variable  https://review.openstack.org/60491707:27
*** hamzaachi has joined #openstack-ansible07:30
*** shyamb has quit IRC07:32
*** shyamb has joined #openstack-ansible07:45
*** shardy has joined #openstack-ansible07:52
*** tosky has joined #openstack-ansible07:54
*** shyamb has quit IRC07:56
openstackgerritMerged openstack/openstack-ansible-os_glance master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60313307:59
openstackgerritMerged openstack/openstack-ansible-os_tempest master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60313908:07
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server stable/rocky: Allow configuration of startup timeout  https://review.openstack.org/60499608:12
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server stable/queens: Allow configuration of startup timeout  https://review.openstack.org/60499808:13
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server stable/pike: Allow configuration of startup timeout  https://review.openstack.org/60499908:13
*** DanyC has joined #openstack-ansible08:16
arxcruzmnaser: odyssey4me hey, can you guys take a look at this http://logs.openstack.org/00/603100/27/check/openstack-ansible-functional-ubuntu-bionic/d0479e2/job-output.txt.gz#_2018-09-25_02_18_53_33158208:17
arxcruzi'm using the python_venv_build however it seems it's not creating the venv properly08:17
*** mrda has joined #openstack-ansible08:18
*** pmannidi has quit IRC08:35
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar stable/rocky: Define cache_timeout variable  https://review.openstack.org/60500708:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar stable/rocky: Fix functional tests for Blazar  https://review.openstack.org/60481308:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar stable/rocky: use include_tasks instead of include  https://review.openstack.org/60481408:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar stable/rocky: Add the MQ vhost/user creation and configuration  https://review.openstack.org/60481508:48
odyssey4mecores - anyon around to review https://review.openstack.org/604626 ?08:50
*** evrardjp_ has joined #openstack-ansible08:50
odyssey4mearxcruz: taking a look now08:50
*** evrardjp has quit IRC08:53
*** neith has joined #openstack-ansible08:53
*** shyamb has joined #openstack-ansible08:53
*** priteau has joined #openstack-ansible08:54
odyssey4mearxcruz: so it's doing the install as it would do if you pip installed any other way: http://logs.openstack.org/00/603100/27/check/openstack-ansible-functional-ubuntu-bionic/d0479e2/logs/ara-report/result/f9603dfe-dd11-49f2-8bf5-b7282299bdb9/08:57
odyssey4mearxcruz: mind if I tweak the patch to give some more diagnostics?08:57
odyssey4mearxcruz: oh, never mind, I see it09:02
odyssey4meI thought I did, then I realised I did not. :/09:04
odyssey4melemme fire up a test host to make inspection easier09:04
*** mwe1 has quit IRC09:11
*** evrardjp_ is now known as evrardjp09:15
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: AppArmor: Handle required profiles for neutron services  https://review.openstack.org/60469509:16
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60428609:16
*** Emine has joined #openstack-ansible09:18
*** dxiri has quit IRC09:29
openstackgerritMerged openstack/openstack-ansible stable/queens: Update all SHAs for 17.1.2  https://review.openstack.org/60462209:34
*** mwe1 has joined #openstack-ansible09:36
*** shyamb has quit IRC09:38
*** cshen has joined #openstack-ansible09:42
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: AppArmor: Handle required profiles for neutron services  https://review.openstack.org/60469509:51
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60428609:51
*** electrofelix has joined #openstack-ansible10:00
openstackgerritMerged openstack/openstack-ansible-repo_build stable/rocky: Remove galera_client meta-dependency  https://review.openstack.org/60448910:08
openstackgerritMerged openstack/openstack-ansible stable/rocky: Update all SHAs for 18.0.0  https://review.openstack.org/60461910:09
*** cshen has quit IRC10:12
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Revert "Revert "Run with KVM if we can in tests""  https://review.openstack.org/60472010:29
*** shyamb has joined #openstack-ansible10:29
*** dxiri has joined #openstack-ansible10:31
*** spatel has joined #openstack-ansible10:34
*** spatel has quit IRC10:38
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Update messaging notification configuration  https://review.openstack.org/60413110:42
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Disable neutron-vpnaas-agent  https://review.openstack.org/60465410:45
openstackgerritMerged openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.28  https://review.openstack.org/60462611:01
*** gokhani has joined #openstack-ansible11:02
*** shyamb has quit IRC11:12
*** pcaruana has quit IRC11:15
*** udesale has quit IRC11:17
spotzhwoarang: Need to run to the barn but when I set haproxy_ssl: false and openstack_external_ssl: false  I get a 503 on horizon. Any thoughts?11:27
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Add tags for beats setup tasks  https://review.openstack.org/60504411:30
arxcruzodyssey4me: hey, sure, go ahead, sorry, i was at doctor appointment11:32
odyssey4mearxcruz: no worries, I found the bug and made a review note11:32
spotzhwoarang: openstack_service_publicuri_proto: http dpesn't affect the 503 as it works alone11:32
spotzAnd with that I'll be back!11:32
*** sum12 has quit IRC11:34
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Fix journalbeat installation for mixed environments  https://review.openstack.org/60504511:35
*** sum12 has joined #openstack-ansible11:42
*** shyamb has joined #openstack-ansible11:48
arxcruzodyssey4me: (╯°□°)╯︵ ┻━┻11:57
arxcruzodyssey4me: seriously that's it?11:58
arxcruzfrom where i got this /var/log/openstack ?11:58
arxcruz┻━┻ ︵ \(°□°)/ ︵ ┻━┻11:58
odyssey4mearxcruz: yep, the /openstack/log directory would exist if it was using bare metal - but in this case it's a container, so /openstack/log exists on the host, not in the container11:58
odyssey4meso you're not going mad :p that is a path that exists, but it's not the right path ;)11:59
hwoarangspotz: hmm no idea right now12:07
openstackgerritMerged openstack/openstack-ansible-os_blazar stable/rocky: Define cache_timeout variable  https://review.openstack.org/60500712:29
*** shyamb has quit IRC12:31
*** strattao has joined #openstack-ansible12:31
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Add host metadata to core beats output  https://review.openstack.org/60505912:35
evrardjphey -- I'd like to discuss extended maintenance of Ocata at today's meeting12:36
odyssey4meevrardjp: ++12:39
odyssey4meAs far as I'm concerned, we can stop releasing it.12:39
evrardjpI'd like to point to EM tags of upstream projects, then tag EM for us, and keep the branch open12:40
evrardjpinstead of EOL12:40
evrardjpmaybe add a commit later to EOL should projects EOL vs keeping a branch open.12:41
odyssey4meevrardjp: yep, I'm good with that12:41
evrardjpbut at least after EM, we are kinda clear about the current state, without preventing extra patches by the community12:42
odyssey4meWe had a brief discussion at the PTG and agreed much the same.12:42
evrardjpoh I missed that part12:42
evrardjpwas that on the etherpad? I haven't seen it, else I'd have been there.12:42
evrardjp:p12:42
odyssey4meYeah, I think you were out at the time - and I guess we might have missed making a note in the etherpad.12:42
odyssey4meWe agreed to leave tests running as far as possible, but remove them if they're too troublesome - leaving only linting, then if those get too hard to maintain we can change them to a noop.12:43
evrardjpwell12:48
evrardjpI think that if tests break we should close the branch if no-one is willing to fix them12:48
evrardjpfor me it would even make sense to mark those tests nv when we reach EM12:49
evrardjpso that if people want their patch in, we say "it's fine, but fix the thing"12:49
evrardjpwith no real pressure12:50
openstackgerritMerged openstack/openstack-ansible-os_heat master: Update messaging notification configuration  https://review.openstack.org/60434812:55
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Update messaging configuration  https://review.openstack.org/60434012:59
*** johnthetubaguy has joined #openstack-ansible12:59
*** lbragstad has joined #openstack-ansible12:59
*** strattao has quit IRC13:07
*** lbragstad has quit IRC13:09
vollmancloudnull: When you have time can you take another look at https://review.openstack.org/#/c/582532/13:17
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310013:20
openstackgerritMerged openstack/openstack-ansible-os_neutron master: V2 driver is needed to work with ODL 9.0  https://review.openstack.org/60208613:20
*** lbragstad has joined #openstack-ansible13:23
*** lbragstad has quit IRC13:24
*** lbragstad has joined #openstack-ansible13:24
*** lbragstad has quit IRC13:26
odyssey4meevrardjp: Part of the premise of EM is that the branch stays - but there are no promises for tests. Anyone still using it has to do their own validation.13:26
*** lbragstad_ has joined #openstack-ansible13:26
*** lbragstad_ has quit IRC13:26
*** lbragstad has joined #openstack-ansible13:27
odyssey4meIt's only there to provide a way for collaborators doing patches downstream to share them upstream, rather than have everyone do their own fixes downstream and never share them13:27
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: AppArmor: Handle required profiles for neutron services  https://review.openstack.org/60469513:31
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60428613:31
noonedeadpunkhi folks13:36
noonedeadpunkevrardjp, is there any estimate for release of 18.0? As I'm going to put my OS into production and wondering if I have a change to upgrade it to rocky frst13:37
noonedeadpunk*have a chance13:37
evrardjpodyssey4me: that was my point :)13:41
evrardjpnoonedeadpunk: I would like to discuss this today13:41
evrardjpfor me, it's green on periodics, so I'd rather release.13:42
evrardjpbut odyssey4me has concerns and want another bump13:42
evrardjpI am fine with both, I just want to listen the concerns before submitting a final rc/release13:42
evrardjpI also need to do a release on friday so it should be long from that on13:42
noonedeadpunkthanks) I have several weeks to spare, so hope that release will take place by that time)13:44
odyssey4menice to see so much green :) https://review.openstack.org/#/c/604619/13:48
odyssey4mealso nice to see the build times are pretty low13:48
odyssey4mew00t - mysql-python is totally eliminated in that build :)13:50
noonedeadpunkYeah, that looks great) oh, btw, is there any good way to upgrade from xenial to bionic? As upgrade process for me looks like upgrading lxc_hots, than destroying/re-creating containers, and, hopefully, they should be created with bionic then13:51
odyssey4meevrardjp: With https://review.openstack.org/#/c/604619/ merged, that is not the SHA that needs releasing - so we need a new RC, don't we?13:52
odyssey4me*that is the sha that needs releasing13:52
odyssey4meThe last RC was not what we'd like to release - it still has all the mysql-python junk in it.13:53
evrardjplet me rephrase that -- you think mysql-python junk is a problem for releasing, and therefore only next release (friday) should be considered as final potential :)14:00
*** evrardjp has quit IRC14:03
*** evrardjp has joined #openstack-ansible14:03
odyssey4meevrardjp: ok, so you'll do another RC request on friday - so we have a few more days for validation?14:07
odyssey4meI know that for multi-node tests in master today we're getting failures in the glance role. I've not had a chance to dig into what's going on yet, but the issue may also be present in rocky... we'll have some tests kicked off soon to validate that.14:09
evrardjpodyssey4me: I will be on holiday this friday, so it might be even more than that : p14:17
odyssey4meevrardjp: I'd rather we did it before you went on holiday then ;)14:17
evrardjpI leave on thursday, I can submit the request on that day.14:18
evrardjpso we have enough time14:19
evrardjpwe'll always find bugs, right :)14:19
evrardjpthe question is, should they prevent a release14:19
*** vollman has quit IRC14:22
*** vollman has joined #openstack-ansible14:23
odyssey4meAs far as I know, the current stable/rocky is good enough to go with - perhaps we should do a release request tomorrow, and hope it gets released Thursday.14:23
odyssey4meon/by Thursday14:23
openstackgerritMerged openstack/openstack-ansible-galera_server stable/queens: Allow configuration of startup timeout  https://review.openstack.org/60499814:24
openstackgerritMerged openstack/openstack-ansible-galera_server stable/rocky: Allow configuration of startup timeout  https://review.openstack.org/60499614:24
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: AppArmor: Handle required profiles for neutron services  https://review.openstack.org/60469514:25
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60428614:25
canori01hey guys, I'm running Ubuntu 16 on my control plane and hypervisors with openstack-ansible queens. Would I be able to upgrade the OS on the metal nodes to Ubuntu 18 without affecting the openstack components running on the venvs?14:26
mnasercanori01: not yet14:27
spotzhwoarang: Ok interesting wasn't sure if you'd run into it before14:27
canori01mnaser: ah ok. But it will be possible in the future?14:28
mnasercanori01: in rocky official release, it should be14:28
mnaserbut we still haven't tested in palce upgrades and everything14:28
mnaserthere's a lot of testing to be done about OS upgrades14:29
*** dmsimard has quit IRC14:31
*** dmsimard has joined #openstack-ansible14:32
jrosserwe know that the lxc config files will need adjusting, for one thing14:33
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: AppArmor: Handle required profiles for neutron services  https://review.openstack.org/60469514:33
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60428614:33
*** spatel has joined #openstack-ansible14:34
spateljamesdenton: quick question14:34
spatelhow do i know remote instance running on what underly technology like DPDK, virtio or SRIOV ?14:35
spatelI have third-party remote cloud access, alternative of aws, trying to find how to find out what they are using under the guest VM14:36
*** aludwar has joined #openstack-ansible14:40
*** spatel has quit IRC14:40
*** strattao has joined #openstack-ansible14:41
*** hamzaachi has quit IRC14:45
*** hamzaachi has joined #openstack-ansible14:46
*** spatel has joined #openstack-ansible14:48
spateljamesdenton: ^^14:48
spatelsorry got disconnected, i need to get ZNC to make myself online :)14:49
jamesdentonspatel i'm not sure. if it's sr-iov, then your VM would indicate the driver (bnx2x, ixgbevf, etc) va virtio14:50
jamesdenton*versus14:50
*** dxiri has quit IRC14:50
spatelwhat you will see in DPDK ? (when run ethtool -i eth0 )?14:51
jamesdentonvirtio14:51
jamesdentonbut that isn't a good indicator14:51
*** dxiri has joined #openstack-ansible14:51
spatelI am seeing "driver: virtio_net" in remote cloud provider14:51
*** hamzaachi has quit IRC14:52
spateldoes that means its normal network ?14:52
mnaserso14:52
*** hamzaachi has joined #openstack-ansible14:52
jamesdentoni am not really familiar with AWS networking so I'm not 100% on what you would see14:53
mnaseri have a big oopsie14:54
mnaserright now if you have hostnames with dots in your ansible inventory14:54
spatelThis is not aws this is third-party cloud14:54
mnaseryou end up with hostnames like this: `ctl1-sjc1.vexxhost.us_octavia_server_container-1397ed94`14:54
jamesdentonahh ok14:54
*** hamzaachi has quit IRC14:54
mnasershould we be converting dots to underscores14:55
spatelI am planning to launch service in china and got evaluating clouds in china AWS vs local provider14:55
mnasershould we be enforcing no dots in the hostname in openstack_user_config14:55
*** hamzaachi has joined #openstack-ansible14:55
*** hwoarang has quit IRC14:56
*** skiedude has joined #openstack-ansible14:58
*** hwoarang has joined #openstack-ansible15:06
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: AppArmor: Handle required profiles for neutron services  https://review.openstack.org/60469515:07
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60428615:07
cloudnullmornings15:07
cloudnullvollman looking now15:07
*** udesale has joined #openstack-ansible15:12
*** hamzaachi_ has joined #openstack-ansible15:14
*** devx has joined #openstack-ansible15:14
*** dave-mccowan has quit IRC15:14
*** hamzaachi has quit IRC15:15
*** hwoarang has quit IRC15:17
*** chkumar|ruck is now known as chkumar|off15:17
openstackgerritMerged openstack/openstack-ansible-ops master: Fix journalbeat installation for mixed environments  https://review.openstack.org/60504515:20
openstackgerritMerged openstack/openstack-ansible-ops master: Convert refresh fact to strings  https://review.openstack.org/60495815:21
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: tasks: lxc_install_zypper: Use the 'state' option for external repos  https://review.openstack.org/60510215:21
openstackgerritMerged openstack/openstack-ansible-ops master: Add tags for beats setup tasks  https://review.openstack.org/60504415:24
*** strattao has quit IRC15:25
*** hwoarang has joined #openstack-ansible15:26
spotzmnaser cloudnull odyssey4me - Has anyone gotten an AIO to run without ssl and a cert and can you share? I get a 503 when I use haproxy_ssl: false and openstack_external_ssl: false though I might have something else wrong. This is queens15:27
cloudnullspotz ive not tired in quite awhile15:27
spotzcloudnull: hwoarang and logan- had given me those settings but I have no luck with them:(15:28
spotzI was gonna finally have a student lab on OSA!!!15:29
openstackgerritMerged openstack/ansible-config_template master: Compare dict vars to determine changed  https://review.openstack.org/58253215:30
*** strattao has joined #openstack-ansible15:32
odyssey4mespotz: that used to be all that needed to be set, but it seems it doesn't work any more :/ there's a bug somewhere I guess, which someone will have to track down15:32
spotzodyssey4me: Yeah I never did find where the 503 was coming from or you know I'd give more info and bug it:(15:33
odyssey4meI wonder whether setting 'openstack_external_ssl: false' will help? You'll likely have to redeploy haproxy & horizon15:33
odyssey4meand maybe the other services too15:34
mnaser503 seems like some other service is down15:34
mnaserso check haproxy state15:34
*** hamzaachi_ has quit IRC15:34
spotzmnaser possibility of Keystone, that's just bringing up the horizon login15:34
spotzlet me start with odyssey4me's changes15:35
*** Leo_m has joined #openstack-ansible15:36
*** hwoarang has quit IRC15:36
spotzodyssey4me: that one was already false but I'll give it another shot15:37
*** dxiri has quit IRC15:37
odyssey4mespotz: it'd be helpful to know whether CLI commands work - just getting a token, then trying each service in turn with something that's unique the the service, eg: glance - list images15:37
*** dave-mccowan has joined #openstack-ansible15:38
spotzodyssey4me: Ok let me get those settings back in place and will let you know. Looking for the commands:)15:38
*** hamzaachi has joined #openstack-ansible15:40
openstackgerritweizj proposed openstack/openstack-ansible-os_tacker master: Update messaging notification configuration  https://review.openstack.org/60510815:40
openstackgerritweizj proposed openstack/openstack-ansible-os_designate master: Update messaging notification configuration  https://review.openstack.org/60510915:40
openstackgerritweizj proposed openstack/openstack-ansible-os_glance master: Update messaging notification configuration  https://review.openstack.org/60511015:40
openstackgerritweizj proposed openstack/openstack-ansible-os_sahara master: Update messaging notification configuration  https://review.openstack.org/60511115:40
openstackgerritweizj proposed openstack/openstack-ansible-os_magnum master: Update messaging notification configuration  https://review.openstack.org/60511215:40
openstackgerritweizj proposed openstack/openstack-ansible-os_blazar master: Update messaging notification configuration  https://review.openstack.org/60511315:41
openstackgerritweizj proposed openstack/openstack-ansible-os_trove master: Update messaging notification configuration  https://review.openstack.org/60511415:41
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Make the elasticsearch plugins optional  https://review.openstack.org/60486715:42
evrardjphey15:44
evrardjpspotz: maybe an haproxy.cfg + keystone.conf could help us understand what's going on :)15:44
evrardjpif you can share those15:44
spotzevrardjp: I can go one better:)15:45
evrardjpnot sure what you meant there--- it works? :D15:47
*** hwoarang has joined #openstack-ansible15:47
spotzNope got distracted talking to cmurphy incoming PM:)15:49
*** wlmbasson has quit IRC15:52
openstackgerritweizj proposed openstack/openstack-ansible-os_trove master: Update messaging notification configuration  https://review.openstack.org/60511415:53
openstackgerritweizj proposed openstack/openstack-ansible-os_blazar master: Update messaging notification configuration  https://review.openstack.org/60511315:55
openstackgerritweizj proposed openstack/openstack-ansible-os_magnum master: Update messaging notification configuration  https://review.openstack.org/60511215:56
openstackgerritweizj proposed openstack/openstack-ansible-os_sahara master: Update messaging notification configuration  https://review.openstack.org/60511115:58
openstackgerritMerged openstack/openstack-ansible-ops master: Add host metadata to core beats output  https://review.openstack.org/60505915:59
mnasercloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum, darrenc, JRobinson__, colinmcnamara, thorst, adreznec, eil397,15:59
mnaserqwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, mnaser, nicolasbock, jrosser15:59
mnaser60 second warning15:59
mnaser:)15:59
openstackgerritweizj proposed openstack/openstack-ansible-os_designate master: Update messaging notification configuration  https://review.openstack.org/60510915:59
cjloadermnaser: i'm not on that list lol16:00
spotzIs it that time already?:)16:00
spotzcjloader: It's on the wiki page add yourself:)16:00
mnasercjloader: you can add yourself on the wiki https://wiki.openstack.org/wiki/Meetings/openstack-ansible16:00
mnaser^^16:00
spotz:)16:00
*** hamzaachi has quit IRC16:00
openstackgerritweizj proposed openstack/openstack-ansible-os_tacker master: Update messaging notification configuration  https://review.openstack.org/60510816:00
prometheanfireo/16:01
evrardjpo/16:01
hwoarango/16:01
mnaser#startmeeting openstack_ansible_meeting16:01
openstackMeeting started Tue Sep 25 16:01:26 2018 UTC and is due to finish in 60 minutes.  The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot.16:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:01
*** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)"16:01
mnaserround 216:01
openstackThe meeting name has been set to 'openstack_ansible_meeting'16:01
mnaser#topic rollcall16:01
*** openstack changes topic to "rollcall (Meeting topic: openstack_ansible_meeting)"16:01
mnasero/ ? :)16:02
hwoarango/16:02
guilhermespo/16:02
odyssey4meo/16:02
openstackgerritweizj proposed openstack/openstack-ansible-os_tacker master: Remove the unnecessary vao  https://review.openstack.org/60511716:02
mnaserok cool lets get rolling16:03
mnaser#topic Last week highlights16:03
*** openstack changes topic to "Last week highlights (Meeting topic: openstack_ansible_meeting)"16:03
mnaseri see some stale items there, i think the only relevant one is evrardjp's16:03
mnaserevrardjp: Making Ocata "extended maintenance" (This would add ocata-em tag, not closing the branch). See also: https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.html . No real value for us, what do we point to for upstream repos (IMO, the ocata-em upstream tags when available, else the stable branch it is attached to).16:03
openstackgerritweizj proposed openstack/openstack-ansible-os_tacker master: Remove the unnecessary verbose defined  https://review.openstack.org/60511716:03
mnaserevrardjp: would you like to expand on that?16:03
evrardjpyes16:04
evrardjpwell I guess we discussed this today with odyssey4me and it was discussed during PTG (sorry to have missed that conversation)16:05
_d34dh0r53_o/16:05
evrardjpThere are discussions on the ML about moving projects to EM for ocata16:05
mnaserit was a very brief discussion16:05
*** _d34dh0r53_ is now known as d34dh0r5316:05
evrardjpI suggest we move said projects to EM as soon as possible, instead of continuing to track stable/ocata16:06
spotzFor the ops side of the house are we going to be a tag vs branch?16:06
mnaserit looks like most maintainers/users dont seem to care16:06
evrardjpindeed16:06
evrardjpso the advantage of doing so is that we have a reference that's stable and meaningfull - never bumped until someone definitely wants to16:07
mnaserso do we just tag ocata-em and call it a day?16:07
evrardjpand it won't break anytime should EOL appear and the branch be deleted16:07
odyssey4meFor me, I'd just like to reach a stage where we update the docs to indicate the status (and provide reference to what it means) - and stop releasing it.16:07
*** pcaruana has joined #openstack-ansible16:07
*** hamzaachi has joined #openstack-ansible16:08
evrardjpmnaser: oh you mean just mark OSA as EM?16:08
spotzYeah my concern is folks looking for the wrong location so definitely want to make sure our docs are good16:08
evrardjpand not care about the branches?16:08
mnaserno, tag ocata-em and run jobs against ocata-em only16:08
mnaserit will break, eventually, inevitebly16:08
mnasermy question is16:08
mnaser(i should know this answer)16:08
odyssey4meIt's nice to have the branch there for those of us who'll still need to do patch updates. We have a ton of newton forks thanks to the EOL, and had to do a ton of tooling changes to switch where things pointed.16:09
evrardjpyou mean for upstream projects or OSA?16:09
mnaserif we don't follow stable policy16:09
mnaserit's kinda up to us to do what we want16:09
mnaseri dont think stable policy works well or makes sense for a deployment project16:09
odyssey4meAgreed, which is why we don't follow it.16:09
mnaseri really hope no one is doing ocata greenfield deployments :X16:09
evrardjpagreed mnaser16:09
openstackgerritMerged openstack/openstack-ansible-ops master: Change VM definition in deploy-vm mnaio playbook  https://review.openstack.org/60283216:09
evrardjpmnaser: there are ppl doing it though, but we can't do anything for them :p16:09
openstackgerritMerged openstack/openstack-ansible-ops master: Use group vars to reduce redundancy in host vars  https://review.openstack.org/60283416:10
openstackgerritMerged openstack/openstack-ansible-ops master: Enable setting mnaio disk size by pxe server group  https://review.openstack.org/60283516:10
openstackgerritMerged openstack/openstack-ansible-ops master: Make root partition size configurable  https://review.openstack.org/60285916:10
evrardjpahahah16:10
odyssey4meThe integrated build is pinned, so we don't need to do anything there. We can leave the roles as-is and if they break, we can fix them as needed.16:10
odyssey4meI'd suggest removing the periodic tests though.16:11
evrardjpok I will follow odyssey4me /mnaser 's proposition: Next release will be marked as EM instead of usual tag, and the rest stays the same. If someone wants to fix, (s)he can16:11
evrardjpI will clean things up: removing periodic testing -- changing the version name16:11
evrardjpmarking jobs as non voting16:11
spotzOk and give me a heads up and I'll try to get the docs patch up at the same time16:11
*** shyamb has joined #openstack-ansible16:11
evrardjpwe can discuss things in the review16:11
evrardjpspotz: you can push a patch already without the date I'd say.16:12
*** Emine has quit IRC16:12
odyssey4mewell, as for non-voting jobs - we know that no-one bothers to check them, so why not just remove them all and make patches just do doc builds and nothing more?16:12
mnaseri am in favour of dropping nv jobs.16:12
mnaserlets not waste resources16:12
mnasermaybe just linters16:12
odyssey4meAnyone using an EM project should be doing their own testing.16:12
mnaser++16:12
mnaserwell actually16:12
mnaserthings should still be backported so16:13
evrardjpthat should be written in a release note then16:13
mnaserpatches should be good16:13
evrardjpmaybe we keep the testing up until we can't?16:13
evrardjpthe branch would still be up so it's fine16:13
hwoarangwell 'em' gives the impression that someone maintains it. if you drop the jobs, then things start failing so the 'maintenance' is not sure accurate anymore16:13
mnaserdrop the testing till we cant16:13
odyssey4meyeah, the EM should definitely include a release note saying that it's EM and therefore no longer actively tested or developed16:13
mnasermeans someone has to go out and write a patch to drop the jobs16:13
mnaserwhen hey break16:13
mnaserthey*16:13
*** dave-mccowan has quit IRC16:13
odyssey4meyeah, I'm fine with leaving them as-is and removing them piecemeal as they break16:14
mnaseri dunno16:14
hwoarangthere is no good reason why things start failing when the openstack components are pinned16:14
odyssey4meI've already done some :p16:14
evrardjphwoarang: that's kinda the idea of EM though: until we can't merge things anymore, we keep branch open, else we close the branch16:14
mnaserhwoarang: we've seen plenty of things happen16:14
mnaserexternal dependencies changing url (ms + github stuff)16:14
mnasermariadb changing urls16:14
mnaseraaaaaall sort of things happen16:14
hwoarangyeah but if the branch is open you need some jobs to get an idea of how terrible the situation is :)16:14
spotzevrardjp: We've got docs where it mentions the location which is why if the branch goes away and we become a tag I'd rather wait16:15
evrardjphwoarang: :)16:15
mnaserwell see the thing is16:15
mnaserwe dont follow stable policy16:15
evrardjpthat's true16:15
mnaserso we have zero obligation of doing this16:15
evrardjpindeed16:15
mnasermy question is: is there someone here willing to step up to handle this16:15
evrardjpwe can close the branch right now if you want : p16:15
mnaserbecause it provides value for their organization16:15
mnaserif someone steps up to take care of the jobs, tagging em, etc, i'm all for it16:16
mnaserbut i dont think a deployment project being in EM makes a whole lot of sense, it does for the actual openstack services16:16
evrardjpI would prefer not work on this anymore :p16:16
hwoarangthen lets not do it :)16:16
mnaserwe could draft up a small email to ML16:16
mnaserand ask if someone wants to step up16:16
evrardjpI am not in agreement there -- deployers probably want their branch to live FOREVER16:17
evrardjpbut they will never get an incentive to upgrade -- which is a bad message :p16:17
mnaseragainst sending an email to ML?16:17
evrardjpkilling the branch is the ultimate incentive16:17
mnaserif someone wants it to live forever, they can pick up the work :-)16:17
evrardjpoh no I am for it : )16:17
evrardjpmnaser: well that was the idea of EM :D16:17
odyssey4memnaser: we (rax) need it, and will handle it16:17
evrardjpanyway16:17
mnaserwell there, no need of an ML post16:18
evrardjpwe spent much time for this, sorry for taking the time16:18
mnasernah, it's fine.16:18
spotzGood covos are worth the time16:18
odyssey4mewe only need a small subset of the roles16:18
spotzconvos even...16:18
mnaserin that case, we can go ahead with EM and given odyssey4me (or someone at rax) will be leading it then i'll leave it up to him/them to decide how they'd like to maintain it16:18
logan-i think we could leave gating in place for now and remove nv and periodics16:18
mnaser(as long as we all agree on it)16:19
evrardjpat least it's discussed and we know the interest of people is there -- we'll clarify the situation outside this meeting, summarize in the ML and take actions.16:19
odyssey4mebut we would prefer to keep it upstream rather than have to manage a downstream forks, right d34dh0r53 prometheanfire cjloader ?16:19
cjloaderyes, upstream is better16:19
evrardjpmnaser: agreed16:19
d34dh0r53absolutely16:19
prometheanfirealways16:19
mnaserawesome16:19
evrardjpunderstood -- so no EOL -- a path towards EM is better :)16:19
evrardjpor no EM but keeping things okayish16:20
mnaserso in that case, it's your call to see how best we deal with EM (and where  you want to maintain jobs or not)16:20
mnaseras long as we have some form of ownership, i'm happy to have it16:20
mnaser(shared responsibility is still a thing, we all help each other land stuff, but the bulk of removing any jobs or what not)16:20
mnasercool16:21
mnaseront other fun stuff16:21
mnaserany other comments about this?16:22
*** strattao has quit IRC16:22
evrardjpnone16:22
mnaseri guess not!16:22
mnaser#topic bug triage16:23
*** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)"16:23
mnaser#link https://bugs.launchpad.net/openstack-ansible/+bug/179378116:23
openstackLaunchpad bug 1793781 in openstack-ansible "Queens -> Rocky Upgrade breaks cinder/glance" [Undecided,New]16:23
hwoarangbased on last comment seems like a user config issue?16:24
mnaserindeed16:24
spotzYeah so DNF16:24
spotzor invalid16:24
mnaserquorum on invalid?16:24
*** shyamb has quit IRC16:25
hwoarangok16:25
mnaser#link https://bugs.launchpad.net/openstack-ansible/+bug/179338916:25
openstackLaunchpad bug 1793389 in openstack-ansible "Upgrade to Ocata: Keystone Intermittent Missing 'options' Key" [Undecided,New] - Assigned to Alex Redinger (rexredinger)16:25
mnaserhmm16:26
mnaserlooks like they pushed a fix16:26
mnaserbut nothing on master16:26
mnaseris alex on irc?16:26
evrardjpEM on ocata, it's time:p16:26
evrardjphahah16:26
mnaserits a patch that went straight to master16:26
mnasererr16:26
mnaserstraight to stable branches16:26
mnaserwhich means it will regress16:26
*** udesale has quit IRC16:27
hwoarangtrue. we should ask for proper backports16:27
mnaserd34dh0r53: https://review.openstack.org/#/c/604846/ mind taking off the +2 here because there is no master patch?16:27
evrardjpit started in queens apparently: https://review.openstack.org/#/c/604804/116:27
mnaserand that looks like a keystone bug too16:27
spotzWell unless there's no issue with master and it is only a backport?16:27
d34dh0r53my bad, should have looked at that closer16:27
mnaserno its an upgrade issue with keystone16:28
evrardjpwell master is different now16:28
evrardjpbut it doesn't prevent a master implementation16:28
mnaserconfirmed/medium and ask for an impl. in master?16:29
evrardjpalso it's very weird: why only the service tenant, and not the other keystone commands?16:29
mnaserit looks like a keystone bug to me honestly16:29
mnaserim tempted to put this down as invalid16:30
evrardjpor a conf issue16:30
mnaserOH16:30
mnaseryou know what this might be?!16:30
mnasermemcache not being flushed16:30
mnaserin the upgrade16:30
mnaserso the dict it pulls is invalid16:30
evrardjpmmm16:30
evrardjpwe have a restart in it16:30
mnasernot during the keystone play16:31
evrardjpoh wait16:31
evrardjphttps://github.com/openstack/openstack-ansible/blob/stable/ocata/scripts/run-upgrade.sh#L18516:31
evrardjpit is before keystone play16:31
evrardjpmnaser: I suggest we ask if memcached was properly flushed in the bug, see how it goes?16:32
odyssey4mebut is the bug reporter using the same upgrade tooling?16:32
evrardjpyeah my point odyssey4me16:32
evrardjpmark it incomplete until we know how the upgrade was triggered16:32
d34dh0r53I just asked him to join16:32
openstackgerritMerged openstack/openstack-ansible-ops master: MNAIO: Correct README regarding DATA_DISK_DEVICE  https://review.openstack.org/60406416:32
mnaserhmm16:33
cjloaderhe's on his way16:33
mnaserwe dont have much bugs16:33
*** Alex_Red1nger has joined #openstack-ansible16:33
*** strattao has joined #openstack-ansible16:33
d34dh0r53o/ Alex_Red1nger16:34
d34dh0r53do you need scrollback?16:34
spotzo/16:34
Alex_Red1nger\\o16:34
Alex_Red1ngerYes please16:34
mnaserAlex_Red1nger: we were talking about https://bugs.launchpad.net/openstack-ansible/+bug/179338916:34
openstackLaunchpad bug 1793389 in openstack-ansible "Upgrade to Ocata: Keystone Intermittent Missing 'options' Key" [Undecided,New] - Assigned to Alex Redinger (rexredinger)16:34
evrardjphey Alex_Red1nger , it's fun with bugs time currently : )16:34
d34dh0r53Alex_Red1nger: pasted it in slack16:34
spotzd34dh0r53: boo slack!16:34
d34dh0r53yep, boo slack16:34
d34dh0r53but didn't want to spam this changel16:35
d34dh0r53chanel16:35
mnasercut him some.......slack16:35
d34dh0r53ba dum cha16:35
Alex_Red1ngerHaha16:35
cjloaderd34dh0r53: spamalot much?16:35
d34dh0r53that's sir spamalot to you16:35
Alex_Red1ngerAnyway, this is definitely a keystone bug.16:35
spotzhehehe16:35
Alex_Red1ngerMy patch is meant as a work-around to expidite upgrades.16:35
skiedudeMight I ask what it means when my bug ticket is moved from open to INvalid with no comment of why?16:36
* spotz notes no nano in our containers is a major bug:)16:36
cjloaderspotz: +116:36
mnaserskiedude: got a link to one?16:36
skiedudehttps://bugs.launchpad.net/openstack-ansible/+bug/179378116:36
openstackLaunchpad bug 1793781 in openstack-ansible "Queens -> Rocky Upgrade breaks cinder/glance" [Undecided,Invalid]16:36
mnaserAlex_Red1nger: you were using the upgrade tooling?16:36
openstackgerritMerged openstack/openstack-ansible-ops master: MNAIO: Ensure consistent defaults  https://review.openstack.org/60405916:37
openstackgerritMerged openstack/openstack-ansible-ops master: MNAIO: Disable metering services by default  https://review.openstack.org/60403416:37
mnaserskiedude: sorry, i should have added a comment, it looks like you mentioned that it was something with your config so we assumed that it was just something to figure out in your config.16:37
skiedudeah ok, well I'm just assuming it is my config, I'm using the basic config options in the test production example, so nothing special16:37
Alex_Red1ngermnaser: Yeah, I ran into the bug during execution of run-upgrade.sh16:38
odyssey4meAlex_Red1nger: was part of the upgrade process to flush memcache, as done in every stable branch upgrade script? https://github.com/openstack/openstack-ansible/blob/stable/pike/scripts/run-upgrade.sh#L19216:38
skiedudeguess its not technically a bug, just wasn't sure where to put for support16:38
antonymmnaser: yeah, the error pops up when going from ocata to pike and it happens randomly, sometimes it passes successfully, other times, it just needs another run of the keystone playbook and it goes away16:38
antonym(using the run-upgrade.sh)16:38
mnaserim assuming the "it needs another run" really is16:38
odyssey4meaha, ok thanks for clarifying16:38
mnaserstuff getting cleared out of memcache16:38
odyssey4memaybe there needs to be some sort of check until it's cleared task?16:38
antonymyeah, it was kind of random16:39
Alex_Red1ngercacheing does seem like a likely cuplrit, since nothing in the code seems susceptible to a race.16:39
spotzskiedude: Best thing it so ask here with the understanding someone might not be available or the ML16:39
mnaseryeah im 99% sure its the memcache16:39
skiedudeunderstood, wheres a link to the ML?16:39
spotzskiedude: I'll PM you so as not to spam:)16:40
skiedudethanks!16:40
odyssey4memaybe the memcache flush needs to happen within the keystone role rather, because that flush happens while the old system is running16:41
mnaserexactly, i'd classify that as a bug in our deployment tooling16:41
mnasermore retries is just hoping that luck expires memcache stuff16:41
odyssey4mecould just have a flag to set, and use that flag in the upgrade script16:41
mnasermaybe as a handler16:41
mnaserwith a delegate_to16:41
odyssey4meyep, probably as a handler because that'll fire when the old one is shut down and the new one starts16:42
mnaseranytime we restart keystone16:42
mnaserwe do that16:42
odyssey4methat sounds like it'd be too much - and break rolling upgrades16:42
mnaseri dunno how much it'd break rolling upgrades16:42
mnasermemcache is just stateless16:42
mnaserit'll repopulate16:42
odyssey4meok, but doesn't it hold all the tokens and such?16:43
mnasernope, with fernet tokens16:43
odyssey4mewe've only ever needed this done on a major upgrade as far as I know, so it seems prudent to me to just have a flag and use it during the major upgrade16:43
mnaserthey dont get stored anywhere actually16:43
mnaserwith fernet tokens it validates it using the credential files16:44
odyssey4meanyway, those are details which can get sorted out in review16:44
*** spatel has quit IRC16:44
mnaseryeah.16:44
mnaserim not opposed to adding more retries if you think it helps16:44
mnaserit wont hurt16:44
mnaserAlex_Red1nger: just need a patch for master too16:44
mnaserif you can do that then we'll be able to push the rest16:44
odyssey4megiven the cache is for performance, it seems prudent to me not to flush the cache every time you restart keystone - but rather only when needed, otherwise we'd just be slowing the environment down16:45
mnasernot opposed to that either, or if we know its a major upgrade, we can figure out the details16:45
mnaserAlex_Red1nger: ill mark it as confirmed and we'll take a patch for master before we can land the backported ones16:45
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310016:46
mnaserskiedude: mailing list or irc (outside meeting hours) will be the best place to gather help :)16:46
mnaseraka in less than 15 minutes16:46
mnaser#link https://bugs.launchpad.net/openstack-ansible/+bug/179108516:46
openstackLaunchpad bug 1791085 in openstack-ansible "OVN Metadata Service Broken" [Undecided,New] - Assigned to James Denton (james-denton)16:46
Alex_Red1ngermnaser: OK, I'll move things over.16:46
*** mmercer has joined #openstack-ansible16:46
mnaserjamesdenton: do we need anytihng about that or just there to track progress?16:46
jamesdentoneventually, yes.16:47
jamesdentonit's #22 of #10016:47
spotzheehe16:47
mnaserso do we mark as wishlist? how do we track wip items usualy?16:47
jamesdentongood question. certainly not urgent16:48
mnaseri guess confirmed16:48
odyssey4meconfirm/in-progress I guess - priority probably low/medium16:48
mnaserbecause its confirmed broken.16:48
mnaserconfirmed/low it is16:48
mnaser#link https://bugs.launchpad.net/openstack-ansible/+bug/178238816:48
openstackLaunchpad bug 1782388 in openstack-ansible "Installing Multipath But Not Enabling In Nova Causes Volume Attachment Failures" [Undecided,New]16:48
jamesdentonthanks16:48
mnaserour trail16:48
mnaseri dont see byron around16:49
mnaseri guess i can leavei t as is16:50
mnaser#topic open discussion16:50
*** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)"16:50
mnaser10 fun minutes if anyone has anything fun16:50
cjloaderCan I possibly get some eyes on https://review.openstack.org/#/c/556586/?16:51
cjloaderThe designate spec16:51
odyssey4memnaser evrardjp did we want to chat about rocky releasing?16:51
evrardjpwe can16:51
mnaseroh sure16:51
mnaseri'll look at that spec soon cjloader16:51
mnaserlets see the state of rocky16:52
mnaserhttps://review.openstack.org/#/q/project:%255Eopenstack/openstack-ansible.*+is:open+branch:stable/rocky16:52
mnaserwe have some open things16:52
odyssey4meWe chatted a bit earlier. I suggested that we do the final RC tag tomorrow (Wednesday) and ask for the release based on it. JP's on holiday friday.16:52
evrardjpmnaser: when I proposed rocky was successfully building in periodics.16:52
mnaserim just trying to see if there is something really important to merge16:52
evrardjpmnaser: makes sense16:53
mnaserbut for the most part16:53
mnaserthey all seem mostly like cleanups16:53
odyssey4meAll the galera_client deps are gone from rocky, so I'm happy to go ahead16:53
mnasernothing too fundamental and important16:53
spotzAfter the meeting I need to know if we have anache conf overrides:)16:53
odyssey4methere are always bugs, and we release often, so I think it's time we released it into the wild16:53
spotzapache even16:53
mnaseri agree, it's in a good state, nothing pressing16:53
evrardjpodyssey4me: I agree there16:53
evrardjpjust for confirmation16:54
odyssey4meSo we can check periodics in the morning, and if it's good - we go for it16:54
evrardjpanother final rc, right? not reusing what's up16:54
spotz+116:54
mnaser++ i agree16:54
evrardjpok sounds good16:54
mnaseryes, whats up is pretty far behind16:54
mnaserdoesnt have all the galera_client stuff16:54
mnaseri think the galera_client stuff has helped drive deploy times so much and made it more stable16:54
mnaservery happy about that one16:54
spotzmnaser: But it's working well in SJC so far?16:54
mnaserwonderfully so16:55
evrardjpok16:55
spotzSweet:)16:55
mnaserall my rocky-deploy patches were merged too16:55
*** mmercer has quit IRC16:55
mnaserany other subjects? :)16:55
d34dh0r53pike16:55
mnaserpike release?16:55
d34dh0r53need to get https://review.openstack.org/#/c/604623/ going so that we can get pike updated16:55
openstackgerritAlex Redinger proposed openstack/openstack-ansible-os_keystone master: Increase retries to mitigate intermittent failure  https://review.openstack.org/60514616:56
mnaseri think release is on halt right now16:56
mnaserbecause of some issues with ssl?16:56
mnaserdoug posted something about it a few days ago on ML16:56
d34dh0r53ohh, must have missed that, I'll go look16:56
mnaseror maybe not, im not sure16:56
mnaserhttps://review.openstack.org/#/q/project:openstack/releases16:57
mnaseri see things merging16:57
mnaserd34dh0r53: you can ask in #openstack-release16:57
*** hamzaachi has quit IRC16:57
spotzI thought the comment there was specific with us and I was gonna blame mhayden16:57
d34dh0r53mnaser: yeah16:57
odyssey4meI asked earlier - no response yet16:57
evrardjpI think it's fixed now16:58
evrardjpthe releases issue I mean16:58
evrardjpsorry I haven't got the chance to continue on that -- a lot of fire still16:58
spotzI can poke Sean16:58
mnasercool16:58
openstackgerritAlex Redinger proposed openstack/openstack-ansible-os_keystone stable/rocky: Increase retries to mitigate intermittent failure  https://review.openstack.org/60514816:58
mnaserthats all folks?16:59
odyssey4methat's all from me17:00
d34dh0r53yep, I'm good17:00
mnaserthanks everyone17:00
mnaser#endmeeting17:00
*** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/2xA1eZC"17:00
openstackMeeting ended Tue Sep 25 17:00:31 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-09-25-16.01.html17:00
hwoarangthank you mnaser17:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-09-25-16.01.txt17:00
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-09-25-16.01.log.html17:00
mnaserthanks you!17:01
*** hwoarang has quit IRC17:05
antonymcould i get one more pair of eyes on this: https://review.openstack.org/#/c/597977/17:05
spotzOk do we have anywhere to overright the rewrite rule in the apache conf?:)17:06
openstackgerritMerged openstack/openstack-ansible master: Remove broken uptime tests during upgrade  https://review.openstack.org/60106517:06
spotzThis time I didn't get the 503, I suspect as I re-ran the hosrizon role along with haproxy17:06
*** Alex_Red1nger has quit IRC17:07
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/rocky: Remove broken uptime tests during upgrade  https://review.openstack.org/60515117:07
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Remove broken uptime tests during upgrade  https://review.openstack.org/60515217:11
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Remove broken uptime tests during upgrade  https://review.openstack.org/60515317:12
*** priteau has quit IRC17:13
odyssey4mehooray! ansible 2.5.10 will contain the switch to pymysql as a preferred driver for mysql: https://github.com/ansible/ansible/pull/4598217:15
*** hwoarang has joined #openstack-ansible17:19
mnaserodyssey4me: w00t, so no galera_client in galera container?17:20
mnaseralso does anyone which role sets the hostname for containers?17:20
odyssey4memnaser: it'll still be needed, but we can switch the package used by ansible and rest assured it'll work when we switch tp py317:21
odyssey4melxc_container_create sets the hostname, etc IIRC17:21
openstackgerritAndy Smith proposed openstack/openstack-ansible master: Add qdrouterd role for oslo.messaging backend deployment  https://review.openstack.org/57026117:24
mnaserodyssey4me: i mean we'll be able to drop it from galera_server container but utility might be the only last user?17:26
mnaseryeah im trying to find where in lxc_container_create17:26
mnaserah its a handler17:26
odyssey4memnaser: it still lays down the client config, which is fine - meh - the objective is not to drop the client role, but rather to ensure that we can move to py3 when we want to, and don't need to keep using that old lib17:27
mnaserodyssey4me: thing is on centos there are a bunch of inconsistent tasks right now with galera_client17:27
mnaserthat are not idempotent17:28
odyssey4meoh really? we should fix that17:28
mnaseryeah, i never got around it17:29
mnaserhttps://github.com/openstack/openstack-ansible-galera_client/blob/master/tasks/galera_client_install_yum.yml#L113-L12117:29
mnaserthats one17:29
*** dariko has joined #openstack-ansible17:33
*** shardy has quit IRC17:34
*** dxiri has joined #openstack-ansible17:38
spotzhwoarang odyssey4me - FYI Ok the apache config for horizon needs to be overridden I just edited it, and also the cookies have to be turned off in the horizon config17:42
spotzWe should probably put in overrides and such17:42
spotzor track down if we had them and why they don't work17:43
mgariepyfun. 3 host freeze when doing upgrade..17:44
spotzmgariepy: out of how many?:)17:46
odyssey4mespotz: if you can register a bug for it with the details of your hacks to fix it, then we can figure out a good patch17:46
odyssey4methanks for persisting17:46
spotzodyssey4me: I'll get that going, and if you or someone with patience wants to help I'll attempt to patch:)17:47
spotzBeen so long since I've done an ansible code patch:(17:47
mgariepy1 ctrl and 2 computes17:47
mgariepyof about 40ish computes/.17:48
mgariepyjust when upgrading rabbitmq cluster ..17:48
odyssey4memgariepy: just throw the computer out the window and go home :p17:55
*** vnogin has joined #openstack-ansible17:55
spotzYou know,,, 2 out of 40 aren't bad:)18:00
spotzSee look on the bright side!18:00
mgariepylol18:04
mnasernot that it's relevant for much longer18:04
mnaseranyone seen this http://paste.openstack.org/show/730739/18:04
mnaserwhen stopping a container?18:04
mgariepymnaser, i haven't18:06
*** vollman has quit IRC18:08
*** vnogin has quit IRC18:11
*** macza has joined #openstack-ansible18:11
spotzmnaser: I haven't stopped any, I've restarted services and re-run playbooks for my course18:23
mgariepymnaser, which version of lxc ?18:31
*** DanyC has quit IRC18:44
*** DanyC has joined #openstack-ansible18:50
*** dxiri has quit IRC18:56
*** mmercer has joined #openstack-ansible19:06
odyssey4meok folks, I'm out for the night - cheerio!19:15
*** vollman has joined #openstack-ansible19:16
*** spatel has joined #openstack-ansible19:17
spatelany idea why wsgi eating all of my memory on infra node and now my swap usage went up19:17
spotznight odyssey4me19:18
*** Leo_m has quit IRC19:18
*** Leo_m has joined #openstack-ansible19:19
*** DanyC has quit IRC19:29
*** electrofelix has quit IRC19:32
spateladmin0: ^^19:32
spatelyou there bro?19:32
spatelI was waiting for HugePage stuff19:33
*** DanyC has joined #openstack-ansible19:34
*** strattao has quit IRC19:38
*** hamzaachi has joined #openstack-ansible19:42
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add additional tests and update setupKibanaDashboard  https://review.openstack.org/60519419:42
*** dxiri has joined #openstack-ansible19:44
*** DanyC has quit IRC19:49
*** dariko has quit IRC20:00
maczais there any way to deploy an openstack service not in a venv?20:04
macza(with openstack ansible)20:07
cloudnullmacza you can install using the distro packages within rocky20:14
maczaahh dang just in rocky?20:19
cloudnullit was new in rocky.20:20
cloudnullosa has been a source based install tool since its inception20:21
maczagotcha. the issue i'm having is i have a package that doesn't install with pip, looks like apt install is the way to go.  any other workarounds? (running ocata)20:22
cloudnullhowever its been extended to do distro things now20:22
cloudnullis it like a cinder driver?20:23
cloudnullsomething that extends an openstack stervice ?20:23
cloudnull**service20:23
cloudnulli've seen folks install things like that using the apt package and then link it into the venv20:26
*** pcaruana has quit IRC20:43
*** DanyC has joined #openstack-ansible20:43
maczayeah its for an ironic driver20:45
maczarighto, yeah thats all i've seen so i'll give it a shot.  Thanks!20:48
cloudnullif you need anything on that front, ping out most of us have bouncers and would be happy to help - when we see the message :)21:01
maczagreat thanks :D21:05
*** skiedude has quit IRC21:26
maczanoice it worked, just dpkg -x the .deb and move the files in the python dir over to the venv python dir. that was easier than i expected21:32
logan-macza: yeah, for a little more resilient option you can symlink the module into the venv like https://github.com/openstack/openstack-ansible-os_nova/blob/1c222d60b35189db90506554a80f24d881451c37/tasks/drivers/kvm/nova_compute_kvm_install_source.yml#L37-L5621:35
*** spatel has quit IRC21:36
maczaohhh nice nice thanks21:42
*** hamzaachi has quit IRC21:44
*** hamzaachi has joined #openstack-ansible21:44
*** hamzaachi has quit IRC21:55
*** hamzaachi has joined #openstack-ansible21:56
*** hamzaachi has quit IRC21:56
*** hamzaachi has joined #openstack-ansible21:56
*** DanyC has quit IRC21:58
*** hamzaachi has quit IRC22:00
*** ram5391__ has joined #openstack-ansible22:08
ram5391__not new to openstack, but first time openstack-ansible newb here. I am trying to run the setup-hosts playbook, and I'm getting a weird error: File "/opt/openstack-ansible/osa_toolkit/generate.py", line 896, in _check_same_ip_to_multiple_host     ip = _value['ip'] KeyError: 'ip'22:23
ram5391__any ideas?22:23
*** dxiri has quit IRC22:28
openstackgerritMerged openstack/openstack-ansible-os_neutron master: zuul: Switch Xenial jobs to Bionic  https://review.openstack.org/60429722:29
*** Leo_m has quit IRC22:43
*** jralbert has joined #openstack-ansible22:45
*** jralbert has quit IRC22:54
*** vnogin has joined #openstack-ansible23:04
*** spatel has joined #openstack-ansible23:07
*** vnogin has quit IRC23:10
*** tosky has quit IRC23:11
*** spatel has quit IRC23:12
*** vollman has quit IRC23:13
*** macza has quit IRC23:17
*** macza has joined #openstack-ansible23:17
*** ram5391__ has quit IRC23:18
*** dave-mccowan has joined #openstack-ansible23:20
*** macza has quit IRC23:22
*** vnogin has joined #openstack-ansible23:25
*** vnogin has quit IRC23:30
*** vnogin has joined #openstack-ansible23:45
*** vnogin has quit IRC23:50
*** jralbert has joined #openstack-ansible23:52
jralbertI've installed an OpenStack deployment with OSA, and it's generated a set of public endpoints by IP (https://100.200.200.300:5000, say); but the SSL certificate I've got to secure the domain name those IPs are mapped to doesn't match, since it's specifying a domain name rather than an IP. This results in SSL errors when folks try to use the API interfaces. How do OSA deployers normally handle this?23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!