| *** kaiokmo has quit IRC | 00:27 | |
| *** gyee has quit IRC | 00:38 | |
| *** eumel8 has quit IRC | 00:54 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/queens: Revert "Run with KVM if we can in tests" https://review.openstack.org/589261 | 01:06 |
|---|---|---|
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/pike: Revert "Run with KVM if we can in tests" https://review.openstack.org/589262 | 01:06 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible stable/queens: Whitelist Octavia API https://review.openstack.org/589684 | 01:30 |
| guilhermesp | mnaser: ^ | 01:31 |
| mnaser | ty guilhermesp ! | 01:33 |
| *** gregoryo has joined #openstack-ansible | 01:54 | |
| *** Emine has quit IRC | 02:00 | |
| *** mpjetta has quit IRC | 02:07 | |
| cloudnull | mnaser logan- (or anyone else) mind giving https://review.openstack.org/#/c/588647 another look? | 02:11 |
| cloudnull | failing w/ source on cent due to mirror issues but otherwise passing the gate. | 02:12 |
| logan- | http://logs.openstack.org/47/588647/9/check/openstack-ansible-functional-distro_install-ubuntu-xenial/d44a41e/ failed also, but doesnt seem related to the patch | 02:14 |
| *** mpjetta has joined #openstack-ansible | 02:14 | |
| *** gkadam_ has joined #openstack-ansible | 02:24 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update retention process to use rollup https://review.openstack.org/589674 | 02:26 |
| *** gkadam__ has joined #openstack-ansible | 02:26 | |
| cloudnull | logan- ++ that seems like its a transient issue w/ the service db create. | 02:27 |
| *** cCain has quit IRC | 02:28 | |
| *** gkadam_ has quit IRC | 02:29 | |
| *** dxiri has quit IRC | 03:11 | |
| *** dxiri has joined #openstack-ansible | 03:12 | |
| *** yolanda has quit IRC | 03:14 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Revert "Work around nested virt issues on OVH test nodes" https://review.openstack.org/589309 | 03:32 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Revert "Work around nested virt issues on OVH test nodes" https://review.openstack.org/589310 | 03:32 |
| *** dxiri has quit IRC | 03:58 | |
| *** udesale has joined #openstack-ansible | 04:00 | |
| *** poopcat has quit IRC | 04:04 | |
| *** mmalchuk has quit IRC | 04:17 | |
| *** jenglisch has quit IRC | 04:27 | |
| *** Leo_m_ has quit IRC | 04:32 | |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Bump MariaDB minor version release https://review.openstack.org/584612 | 04:43 |
| *** shyambiradar has joined #openstack-ansible | 05:01 | |
| *** shyambiradar has quit IRC | 05:04 | |
| *** shyambiradar has joined #openstack-ansible | 05:04 | |
| *** shyambiradar has quit IRC | 05:14 | |
| *** shyambiradar has joined #openstack-ansible | 05:24 | |
| *** mmalchuk has joined #openstack-ansible | 05:28 | |
| *** Bhujay has joined #openstack-ansible | 05:35 | |
| *** sm806 has quit IRC | 05:37 | |
| *** udesale has quit IRC | 05:42 | |
| *** gkadam__ has quit IRC | 05:42 | |
| *** gkadam has joined #openstack-ansible | 05:44 | |
| *** galaxyblr has joined #openstack-ansible | 05:50 | |
| *** shyam89 has joined #openstack-ansible | 05:56 | |
| *** shyambiradar has quit IRC | 05:58 | |
| *** shyambiradar has joined #openstack-ansible | 05:59 | |
| *** sm806 has joined #openstack-ansible | 06:00 | |
| *** nicolasbock has joined #openstack-ansible | 06:03 | |
| *** yolanda has joined #openstack-ansible | 06:06 | |
| *** gkadam has quit IRC | 06:09 | |
| *** DanyC has joined #openstack-ansible | 06:10 | |
| openstackgerrit | Maksim Malchuk proposed openstack/openstack-ansible stable/queens: Add an ability to use custom PyPi index for deploy https://review.openstack.org/577653 | 06:14 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: Bump MariaDB minor version release https://review.openstack.org/584611 | 06:23 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Explain how to configure public endpoint https://review.openstack.org/582919 | 06:23 |
| *** gkadam has joined #openstack-ansible | 06:24 | |
| *** pcaruana has joined #openstack-ansible | 06:27 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert playbooks into roles https://review.openstack.org/589721 | 06:31 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update retention process to use rollup https://review.openstack.org/589674 | 06:33 |
| *** DanyC has quit IRC | 06:35 | |
| *** shyam89 has quit IRC | 06:35 | |
| *** shyambiradar has quit IRC | 06:36 | |
| ivve | hey, anyone knows if there is any docs on the live migration bandwidth story? | 06:36 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_zun master: add ubuntu bionic https://review.openstack.org/589722 | 06:39 |
| openstackgerrit | Andriy Shevchenko proposed openstack/ansible-config_template master: import zuul job settings from project-config https://review.openstack.org/589723 | 06:41 |
| ivve | i.e. where to raise the 1024kb/sec cap for live migrations.. seems to be the default | 06:43 |
| openstackgerrit | wangqi proposed openstack/openstack-ansible-os_horizon master: add ubuntu bionic release https://review.openstack.org/589725 | 06:44 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add bionic job to elk_metrics_6x testing https://review.openstack.org/589730 | 06:48 |
| *** vnogin has joined #openstack-ansible | 06:49 | |
| *** vnogin has quit IRC | 06:53 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add bionic job to elk_metrics_6x testing https://review.openstack.org/589730 | 06:54 |
| *** shyambiradar has joined #openstack-ansible | 06:58 | |
| *** luksky has joined #openstack-ansible | 06:59 | |
| *** arbrandes1 has joined #openstack-ansible | 07:01 | |
| evrardjp | hey | 07:02 |
| evrardjp | odyssey4me: jmespath is easy to build simple objects, but is not a replacement for the map filter :) | 07:03 |
| evrardjp | odyssey4me: when you mq patches are in I am submitting rc1. | 07:03 |
| evrardjp | jrosser: what's the status of that configuration item in cinder? That's what might break things. | 07:03 |
| evrardjp | odyssey4me: what do you think about bumping global requirement pins for master to the latest versions? | 07:04 |
| *** arbrandes has quit IRC | 07:04 | |
| jrosser | evrardjp: everything points to cinder volumes not having been restarted | 07:05 |
| evrardjp | jrosser: there seem to be a race condition leading to no restarts of cinder-volumes that might nbe worth analysis | 07:05 |
| jrosser | and the common systemd role stuff went in very recently so i was just lokking at that | 07:05 |
| jrosser | however ive got meetings all morning so will be later before i do more than just look at code | 07:06 |
| jrosser | but, it would explain why cinder/nova/tempest fail for bionic | 07:06 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert playbooks into roles https://review.openstack.org/589721 | 07:07 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add bionic job to elk_metrics_6x testing https://review.openstack.org/589730 | 07:08 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add bionic job to elk_metrics_6x testing https://review.openstack.org/589730 | 07:09 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Bump global requirement pins https://review.openstack.org/589733 | 07:09 |
| evrardjp | jrosser: yup, and maybe it would explain other issues. | 07:11 |
| jrosser | theres also something funky with the networkd setup on bionic too | 07:12 |
| jrosser | which is either its just broken or needs to wait for the interfaces to come up | 07:13 |
| jrosser | idk which yet | 07:13 |
| jrosser | when you build and aio by hand theres inevitably a delay between bootstrap-aio and needing br-mgmt working | 07:16 |
| jrosser | however in the integrated gate thats just a fraction of a second | 07:16 |
| *** cshen has joined #openstack-ansible | 07:17 | |
| evrardjp | we can maybe put a sleep() :) | 07:18 |
| evrardjp | it's normal that nics take time to be fully up | 07:18 |
| jrosser | i was goong to wait_for | 07:18 |
| *** tosky has joined #openstack-ansible | 07:29 | |
| *** jawadahmed has joined #openstack-ansible | 07:30 | |
| *** shyambiradar has quit IRC | 07:30 | |
| jawadahmed | /join ##mychannel | 07:35 |
| *** udesale has joined #openstack-ansible | 07:40 | |
| jrosser | evrardjp: http://paste.openstack.org/show/727603/ | 07:42 |
| *** pmannidi has quit IRC | 07:46 | |
| *** chason has quit IRC | 07:49 | |
| *** fghaas has joined #openstack-ansible | 07:49 | |
| *** hwoarang has joined #openstack-ansible | 07:50 | |
| *** jawadahmed is now known as Jay_ | 07:59 | |
| *** gillesMo has joined #openstack-ansible | 07:59 | |
| evrardjp | jrosser: that's not enough for me to fix things, but that's not encouraging indeed ... I thought when the dust of the mq settled we could have a look | 08:08 |
| jrosser | no i know | 08:08 |
| jrosser | i'm getting closer | 08:08 |
| jrosser | register: the result of config_template may give unexpected results, thats what i'm investigating now | 08:08 |
| jrosser | evrardjp: see this http://paste.openstack.org/show/727605/ | 08:10 |
| jrosser | see how it starts/stops the cinder-backup and cinder-volumes service | 08:10 |
| jrosser | but then runs the "Restart changed services" handler *twice* for cinder-backup and not at all for cinder-volumes | 08:10 |
| jrosser | oh, well skips actually | 08:11 |
| jrosser | anyway it's all around here somewhere | 08:11 |
| * jrosser meetings | 08:12 | |
| *** gregoryo has quit IRC | 08:18 | |
| *** shyambiradar has joined #openstack-ansible | 08:20 | |
| *** diliprenkila has joined #openstack-ansible | 08:23 | |
| diliprenkila | Hi all, | 08:24 |
| diliprenkila | I keep getting "MessagingTimeout: Timed out waiting for a reply to message ID fae51f03a7a74f589b376b8252cc28b4" in my neutron server and agents logs | 08:24 |
| *** Emine has joined #openstack-ansible | 08:25 | |
| diliprenkila | Do anybody encountered in the past : | 08:25 |
| fghaas | diliprenkila: this isn't really a question for the OSA channel as it's quite generic, but yes it's a very common issue. Things to check: network connectivity between your Neutron agents (compute and network nodes) and your RabbitMQ hosts (control nodes), CPU utilization of your control nodes, length of your RabbitMQ message queues. | 08:28 |
| *** phasespace has joined #openstack-ansible | 08:31 | |
| *** DanyC has joined #openstack-ansible | 08:36 | |
| *** electrofelix has joined #openstack-ansible | 08:38 | |
| *** shyam89 has joined #openstack-ansible | 08:42 | |
| *** shyambiradar has quit IRC | 08:45 | |
| *** shyam89 has quit IRC | 08:45 | |
| *** shyambiradar has joined #openstack-ansible | 08:45 | |
| *** hwoarang has quit IRC | 08:47 | |
| *** hwoarang has joined #openstack-ansible | 08:48 | |
| *** shyambiradar has quit IRC | 08:54 | |
| *** shyambiradar has joined #openstack-ansible | 08:54 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_tempest master: Run smoke and dashboard tests by default https://review.openstack.org/587617 | 08:59 |
| *** ssbarnea has quit IRC | 08:59 | |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: List ubuntu releases in release order https://review.openstack.org/589119 | 09:00 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: Add retries for apt https transport installation https://review.openstack.org/589118 | 09:00 |
| *** vnogin has joined #openstack-ansible | 09:01 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove pip_install role execution https://review.openstack.org/587835 | 09:04 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove pip from packages installed https://review.openstack.org/587849 | 09:04 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove pip from packages installed https://review.openstack.org/587849 | 09:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Eliminate installing pip on host/containers https://review.openstack.org/587853 | 09:08 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Eliminate installing pip on host/containers https://review.openstack.org/587853 | 09:09 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Simplify pip/setuptools/wheel prep for venvs https://review.openstack.org/589643 | 09:10 |
| *** ssbarnea has joined #openstack-ansible | 09:13 | |
| odyssey4me | evrardjp: so I'm slowing working towards eliminating the use of get-pip.py entirely and having only virtualenv on the host, coming from distro packages - everything else in the venv | 09:14 |
| odyssey4me | it's very close | 09:14 |
| odyssey4me | once that's done, I'd like to eliminate the pinning of pip/setuptools/wheel | 09:14 |
| odyssey4me | we get and use whatever virtualenv puts there is my target | 09:14 |
| evrardjp | odyssey4me: the only thing that matters to me is the greenlight to rc1 | 09:15 |
| evrardjp | as far as I know only the MQ is pending | 09:15 |
| odyssey4me | evrardjp: the mq work is all up - just waiting for reviews, and stubborn centos failures | 09:16 |
| evrardjp | a few one failed 15 minutes ago | 09:16 |
| evrardjp | neutron nova is the ones I am watching | 09:16 |
| evrardjp | odyssey4me: should we skip centos? | 09:17 |
| evrardjp | oh no it was the recheck 15 minutes ago, my bad | 09:17 |
| odyssey4me | there is one other body of work which would be good to get in if we can, but it's still a bit early stages | 09:17 |
| odyssey4me | yaya https://review.openstack.org/589733 worked | 09:17 |
| odyssey4me | *yay | 09:18 |
| odyssey4me | jrosser: could you review https://review.openstack.org/589733 if you have a moment? | 09:23 |
| odyssey4me | oh, and evrardjp or jrosser https://review.openstack.org/589630 needs another set of eyes, as the ironic role is currently broken | 09:24 |
| odyssey4me | evrardjp: this is the other body of work which it'd be ideal to include: https://review.openstack.org/#/q/topic:smart-sources+(status:open+OR+status:merged) | 09:25 |
| evrardjp | odyssey4me: keystone_messaging_enabled is when rpc is enabled, not notify? | 09:25 |
| evrardjp | wow that's confusing | 09:25 |
| odyssey4me | evrardjp: well, yes - we are going to have to rename the *_ceilometer_enabled var to something else in the future, as the notification queue is not just used for ceilometer | 09:26 |
| odyssey4me | but keystone can operate without messaging entirely | 09:26 |
| evrardjp | that's true | 09:27 |
| evrardjp | still the naming is confusing | 09:28 |
| evrardjp | but that's maybe me | 09:28 |
| evrardjp | odyssey4me: for the "use what's in the venv" I don't think it's a smart idea | 09:28 |
| evrardjp | because we'll have discrepencies between distros | 09:28 |
| evrardjp | although, it simplies the stories like gnocchi | 09:29 |
| evrardjp | which required setuptools x | 09:29 |
| evrardjp | odyssey4me: https://review.openstack.org/#/c/589733/1 is quite a bump -- pip 18! | 09:29 |
| odyssey4me | yep! | 09:30 |
| odyssey4me | but it appears to work, so let's do it | 09:30 |
| evrardjp | odyssey4me: Yeah I think master is fine, and queens should be alright too. | 09:31 |
| odyssey4me | so the thing is that for all distros now we have the things we need from all of them in terms of pip features and all - so we don't really care what the version is any more | 09:31 |
| evrardjp | there is maybe a tempest change that need backporting in Q, but let me try it. | 09:31 |
| evrardjp | odyssey4me: that's not 100% true | 09:31 |
| odyssey4me | oh? | 09:31 |
| evrardjp | have we solved the centos not presence of pip? | 09:31 |
| evrardjp | centos doesn't provide pip by default. | 09:32 |
| evrardjp | it provides virtualenv to a recent version | 09:32 |
| evrardjp | but pip needs to come from somewhere | 09:32 |
| evrardjp | it is maybe from rdo | 09:32 |
| evrardjp | but last time we tried, it failed, right? | 09:32 |
| odyssey4me | evrardjp: the host no longer needs pip at all | 09:32 |
| odyssey4me | all we need is virtualenv | 09:32 |
| evrardjp | that was the goal indeed | 09:33 |
| odyssey4me | virtualenv creates a venv with pip in it | 09:33 |
| evrardjp | I know that :) | 09:33 |
| odyssey4me | yep, so I've got patches up to remove pip entirely from the host | 09:33 |
| evrardjp | ok those were the one failing, right? | 09:33 |
| evrardjp | or am I wrong? | 09:33 |
| odyssey4me | they're not completely passing just yet - it's a WIP | 09:33 |
| odyssey4me | this is one that helps it along: https://review.openstack.org/#/c/589633/2 | 09:35 |
| odyssey4me | then https://review.openstack.org/589643 on top of it | 09:36 |
| evrardjp | jrosser: voting on https://review.openstack.org/#/q/topic:move-mq-setup-into-roles+(status:open) would be welcomed if you have time | 09:36 |
| odyssey4me | evrardjp: I really need help figuring this one out: https://review.openstack.org/#/c/582296/ | 09:37 |
| odyssey4me | I have a test host up and can add your key if you like | 09:38 |
| odyssey4me | I have confirmed that the magnum domain is there, so the task is lying - and perhaps it's something to do with scope. | 09:38 |
| evrardjp | I think always copy is something that should get removed | 09:38 |
| evrardjp | https://review.openstack.org/#/c/589643/2/defaults/main.yml | 09:38 |
| evrardjp | eventually | 09:38 |
| *** jonher has quit IRC | 09:43 | |
| *** luksky has quit IRC | 09:44 | |
| *** priteau has quit IRC | 09:45 | |
| *** priteau has joined #openstack-ansible | 09:45 | |
| *** hamzaachi has joined #openstack-ansible | 09:48 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-repo_build master: Simplify pip/setuptools/wheel prep for venvs https://review.openstack.org/589643 | 09:50 |
| *** shyam89 has joined #openstack-ansible | 09:55 | |
| *** shyambiradar has quit IRC | 09:57 | |
| *** cshen has quit IRC | 10:02 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove pip from packages installed https://review.openstack.org/587849 | 10:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove pip_install role execution https://review.openstack.org/587835 | 10:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts master: Remove pip from packages installed https://review.openstack.org/587849 | 10:05 |
| *** vnogin has quit IRC | 10:06 | |
| *** luksky has joined #openstack-ansible | 10:16 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Fix gate log collection https://review.openstack.org/589428 | 10:23 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Whitelist Octavia API https://review.openstack.org/589684 | 10:23 |
| *** madorn has quit IRC | 10:25 | |
| *** madorn has joined #openstack-ansible | 10:27 | |
| openstackgerrit | Merged openstack/openstack-ansible master: [docs] Add documentation for 'install_method' variable https://review.openstack.org/589398 | 10:28 |
| *** udesale has quit IRC | 10:30 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582296 | 10:33 |
| *** yolanda has quit IRC | 10:36 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582296 | 10:36 |
| odyssey4me | evrardjp: it would seem that centos builds are failing around 75% of the time, is that an accurate assessment? | 10:37 |
| odyssey4me | it seems to relate to mirrors or dns | 10:37 |
| odyssey4me | I'm about ready to make it non-voting so that we can get these things merged. | 10:38 |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia master: Allow tags to be used for MQ tasks https://review.openstack.org/589485 | 10:42 |
| evrardjp | odyssey4me: that sounds an accurate assessment we can use for today, so that when everything is in for MQ, we can revert and do our rc1 | 10:43 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Make CentOS tests non-voting https://review.openstack.org/589835 | 10:49 |
| odyssey4me | evrardjp: ^ | 10:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: fix ironic driver data structure usage https://review.openstack.org/589630 | 10:52 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic stable/queens: fix ironic driver data structure usage https://review.openstack.org/589838 | 10:53 |
| *** shyam89 has quit IRC | 10:53 | |
| *** shyam89 has joined #openstack-ansible | 10:53 | |
| *** dave-mccowan has joined #openstack-ansible | 10:59 | |
| *** rmart04 has joined #openstack-ansible | 11:02 | |
| evrardjp | odyssey4me: I will merge it fast | 11:03 |
| evrardjp | ok | 11:03 |
| evrardjp | so | 11:03 |
| evrardjp | with this centos in, we can recheck everything MQ related | 11:04 |
| *** cshen has joined #openstack-ansible | 11:15 | |
| odyssey4me | evrardjp: have you looked at https://review.openstack.org/#/q/topic:smart-sources+(status:open+OR+status:merged) at all? | 11:19 |
| odyssey4me | That removes our need to carry any of the templates for everything except the .conf file for the service. | 11:19 |
| odyssey4me | It was the other thing I wanted to get done, but never managed to get to. | 11:20 |
| *** vnogin has joined #openstack-ansible | 11:30 | |
| *** vnogin has quit IRC | 11:31 | |
| *** fghaas has quit IRC | 11:33 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Replace tempest with healthchecks https://review.openstack.org/587544 | 11:34 |
| *** shyam89 has quit IRC | 11:35 | |
| *** shyambiradar has joined #openstack-ansible | 11:35 | |
| *** shyambiradar has quit IRC | 11:37 | |
| *** shyambiradar has joined #openstack-ansible | 11:38 | |
| evrardjp | that looks nice odyssey4me | 11:39 |
| *** kaiokmo has joined #openstack-ansible | 11:47 | |
| *** shyambiradar has quit IRC | 11:50 | |
| *** shyambiradar has joined #openstack-ansible | 11:51 | |
| *** yolanda has joined #openstack-ansible | 11:57 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add jmespath to requirements.txt https://review.openstack.org/589861 | 12:04 |
| *** udesale has joined #openstack-ansible | 12:05 | |
| *** udesale has quit IRC | 12:05 | |
| *** diliprenkila has quit IRC | 12:05 | |
| *** shyam89 has joined #openstack-ansible | 12:05 | |
| *** udesale has joined #openstack-ansible | 12:06 | |
| *** udesale has quit IRC | 12:07 | |
| *** udesale has joined #openstack-ansible | 12:08 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic stable/queens: Add jmespath to test-requirements.txt https://review.openstack.org/589862 | 12:08 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic stable/queens: fix ironic driver data structure usage https://review.openstack.org/589838 | 12:08 |
| *** shyambiradar has quit IRC | 12:09 | |
| *** udesale has quit IRC | 12:10 | |
| hwoarang | evrardjp: one question. when you tag releases for rocky, shouldn't you also add shas in ansible-role-requiremetns for the role? right now everything points to the master branch so the tag is not so "fixed" | 12:11 |
| odyssey4me | evrardjp: looks like https://review.openstack.org/589633 has no negative side-effects for the integrated build | 12:11 |
| evrardjp | hwoarang: nothing released for stable/rocky yet. | 12:11 |
| evrardjp | hwoarang: we need to branch first. | 12:12 |
| evrardjp | then yes, freeze requirements | 12:12 |
| evrardjp | freeze a-r-r | 12:12 |
| hwoarang | ok | 12:12 |
| odyssey4me | folks, the absolute last set of things we pip install on the host is removed by https://review.openstack.org/582296 and https://review.openstack.org/589633 - reviews are welcome | 12:13 |
| *** fghaas has joined #openstack-ansible | 12:18 | |
| odyssey4me | dammit, just found another place we pip install on to the host - the utility playbook | 12:36 |
| odyssey4me | evrardjp: what do you think - put those into a venv too, then do something like symlink into /usr/local/bin so that it's available as it was? | 12:37 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Make CentOS tests non-voting https://review.openstack.org/589835 | 12:40 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Make role test-requirements.txt be additive https://review.openstack.org/589248 | 12:40 |
| odyssey4me | hooray, https://review.openstack.org/582296 finally passed | 12:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Allow tags to be used for MQ tasks https://review.openstack.org/589491 | 12:43 |
| *** shyambiradar has joined #openstack-ansible | 12:44 | |
| *** shyam89 has quit IRC | 12:48 | |
| evrardjp | odyssey4me: symlink | 12:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Allow tags to be used for MQ tasks https://review.openstack.org/589489 | 12:49 |
| odyssey4me | evrardjp: yeah, will work up a patch to do that now - it seems the only sensible way | 12:50 |
| * jrosser back - will do the rest of those mq reviews | 12:50 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Allow tags to be used for MQ tasks https://review.openstack.org/589438 | 12:53 |
| odyssey4me | evrardjp: oh, FYI was talking to cloudnull yesterday and he showed me that with http://paste.openstack.org/show/727569/ we can do an nspawn AIO with no bridges | 12:58 |
| odyssey4me | thought that might be nice to put together as a user story, and perhaps even to use that instead of our more complex current AIO config | 12:59 |
| odyssey4me | jrosser: mind adjusting https://review.openstack.org/#/c/586756/ as we discussed yesterday? | 13:07 |
| jrosser | odyssey4me: ok - on it | 13:07 |
| *** shyambiradar has quit IRC | 13:08 | |
| jrosser | odyssey4me: actaully should i just abandon that? bionic is broken anyway | 13:09 |
| odyssey4me | jrosser: well, ideally we want to have a patch making it voting - and we can add whatever bits necessary to the same patch to make it work. | 13:11 |
| odyssey4me | assuming it's localised | 13:11 |
| jrosser | it wasn't an immediatley obvious brok (~10 seconds looking) | 13:12 |
| odyssey4me | I see there's https://review.openstack.org/588274 and https://review.openstack.org/586668 too | 13:12 |
| odyssey4me | so perhaps just make a single patch that's to get bionic going, which adds the job and metadata too | 13:12 |
| odyssey4me | perhaps do that after https://review.openstack.org/589259 merges though, as I think it'll be easier | 13:13 |
| jrosser | yes that sounds like a plan | 13:13 |
| jrosser | i need to filter through all those "add metadata" review as some are not yet valid | 13:14 |
| jrosser | but a whole load of them could go in now, i've just not had time to -W the ones that arent yet ready | 13:14 |
| *** phasespace has quit IRC | 13:16 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Only test for repo availability once https://review.openstack.org/589888 | 13:18 |
| odyssey4me | evrardjp: I think I'm right with https://review.openstack.org/589888 and that could save quite a bit of time on large clusters | 13:18 |
| odyssey4me | please validate my assumption | 13:18 |
| openstackgerrit | Merged openstack/openstack-ansible-os_masakari master: Allow tags to be used for MQ tasks https://review.openstack.org/589470 | 13:19 |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 13:20 |
| *** vollman has joined #openstack-ansible | 13:20 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic stable/queens: Add jmespath to test-requirements.txt https://review.openstack.org/589862 | 13:22 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Add jmespath to requirements.txt https://review.openstack.org/589861 | 13:22 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic stable/queens: fix ironic driver data structure usage https://review.openstack.org/589838 | 13:23 |
| openstackgerrit | Merged openstack/openstack-ansible master: Set kernel logging to 'quiet' in AIOs https://review.openstack.org/589463 | 13:26 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Allow tags to be used for MQ tasks https://review.openstack.org/589490 | 13:28 |
| evrardjp | odyssey4me: yeah it goes faster when ansible_play_hosts > ansible_forks | 13:28 |
| *** spatel has joined #openstack-ansible | 13:28 | |
| spatel | Folk i want to add few more IPs in mysqlchk xinetd script in this section "only_from = 172.28.1.216 172.28.1.236 172.28.1.183 172.28.0.8 172.28.0.7 172.28.0.12 172.28.0.13 127.0.0.1" | 13:29 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca-agent master: Add support for openSUSE https://review.openstack.org/588462 | 13:29 |
| vollman | evrardjp: I fixed up https://review.openstack.org/#/c/582532/ based on your comments. Ready for review when you have time | 13:31 |
| evrardjp | vollman: thanks | 13:32 |
| evrardjp | will do when I can | 13:32 |
| openstackgerrit | Merged openstack/openstack-ansible-os_congress master: Allow tags to be used for MQ tasks https://review.openstack.org/589442 | 13:32 |
| *** spatel_ has joined #openstack-ansible | 13:32 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Add bionic to galaxy_info metadata https://review.openstack.org/588274 | 13:33 |
| *** spatel has quit IRC | 13:33 | |
| evrardjp | jrosser: can you vote on https://review.openstack.org/#/c/589464/ ? | 13:34 |
| *** jwitko__ has joined #openstack-ansible | 13:34 | |
| *** jwitko__ has quit IRC | 13:34 | |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca-agent master: Add support for CentOS 7 https://review.openstack.org/588456 | 13:34 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Allow tags to be used for MQ tasks https://review.openstack.org/589455 | 13:35 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Allow tags to be used for MQ tasks https://review.openstack.org/589454 | 13:39 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Allow tags to be used for MQ tasks https://review.openstack.org/589492 | 13:39 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Replace tempest with healthchecks https://review.openstack.org/587544 | 13:42 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Allow tags to be used for MQ tasks https://review.openstack.org/589466 | 13:43 |
| evrardjp | odyssey4me: I have even less vars on my env, but that's indeed nice. | 13:43 |
| *** bauruine has quit IRC | 13:45 | |
| evrardjp | I think for Stein we should have the restart done in a separate play. Simply as that. Record the need of a play in the role using a fact, but then only do a service restart in a later play | 13:46 |
| evrardjp | simplifies the understand for everyone, and easy to modify without spending so much brain cycles should behavior change | 13:47 |
| *** Jay_ has quit IRC | 13:47 | |
| *** spatel_ has quit IRC | 13:47 | |
| odyssey4me | evrardjp: I was thinking that we move it into the role instead. | 13:52 |
| odyssey4me | But yeah, it'd be nice to figure out a better strategy for deployment/upgrades and rolling restarts. | 13:52 |
| evrardjp | odyssey4me: I don't think it would be a good idea to move the restart into the role. | 13:55 |
| odyssey4me | evrardjp: it all depends how it's done | 13:55 |
| odyssey4me | I don't want to discuss it now - too much else going on. But we should discuss it at the PTG. | 13:56 |
| evrardjp | it's gonna be very complex in the role but I challenge you to make it simple :) | 13:56 |
| evrardjp | yeah | 13:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Allow tags to be used for MQ tasks https://review.openstack.org/589452 | 13:56 |
| evrardjp | let's add those to the topics | 13:56 |
| odyssey4me | the thing is that I think we should discuss it, but I'm not going to be able to do anything about it - so I'm not adding the topic | 13:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: Allow tags to be used for MQ tasks https://review.openstack.org/589461 | 13:57 |
| odyssey4me | it's pointless discussing it unless osmeone is going to drive the work | 13:57 |
| odyssey4me | we could perhaps have a section where we can have 'lightening ideas' raised, spending no more than 5 mins on them | 13:58 |
| odyssey4me | this might be a good way for people to prep a brief on describing the problem ,and any potential solution they can think of - then ask if anyone is keen to pick up the work | 13:59 |
| odyssey4me | maybe 5 mins is a bit hard, no more than 15 mins | 13:59 |
| odyssey4me | jrosser: got a min to review https://review.openstack.org/589473 ? | 13:59 |
| odyssey4me | oh you did already | 13:59 |
| evrardjp | vollman: I reviewed it very quickly, but I am basically opening a conversation. It seems we are doing hard work, not so sure what the problem really is | 13:59 |
| odyssey4me | nvm | 13:59 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Make CentOS tests non-voting https://review.openstack.org/589906 | 14:02 |
| odyssey4me | evrardjp: ^ looks like the distro install for centos is also broken | 14:03 |
| *** mmalchuk has quit IRC | 14:04 | |
| *** spatel has joined #openstack-ansible | 14:05 | |
| spatel | folks i want to add few more IPs in mysqlchk (sync check) script in "only_from=" section | 14:05 |
| spatel | I am seeing "only_from = {{ galera_monitoring_allowed_source }}" variable | 14:05 |
| spatel | not sure how to put multiple IPs in that array ? | 14:05 |
| vollman | evrardjp: thanks for the review | 14:05 |
| spatel | cloudnull: ^^ Any idea? | 14:06 |
| cloudnull | Congratulations mnaser! Long live the PTL! | 14:09 |
| cloudnull | evrardjp thank you for being an amazing PTL. | 14:10 |
| evrardjp | cloudnull: inspired by you (TM) | 14:10 |
| cloudnull | Sorry about that | 14:10 |
| evrardjp | haha | 14:10 |
| evrardjp | cloudnull: put your name on the etherpad: https://etherpad.openstack.org/p/osa-stein-ptg | 14:11 |
| cloudnull | spatel i believe it's a comma separated list. | 14:11 |
| cloudnull | Sadly on my phone and can't look it up to be sure. | 14:11 |
| spatel | cool! let me try | 14:12 |
| cloudnull | evrardjp done. | 14:13 |
| evrardjp | and congrats mnaser :) | 14:13 |
| cloudnull | When i make it to a computer eventually today I'll try and add some things to the etherpad | 14:13 |
| evrardjp | cloudnull: with any of the sessions you want to propose ofc :) | 14:13 |
| evrardjp | haha great | 14:13 |
| cloudnull | Sessions:.doit() | 14:14 |
| cloudnull | :p | 14:14 |
| cloudnull | Ok back to lurking | 14:14 |
| evrardjp | hahaha https://github.com/dtroyer/osc-choochoo | 14:15 |
| *** Bhujay has quit IRC | 14:18 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zun master: Allow tags to be used for MQ tasks https://review.openstack.org/589494 | 14:18 |
| kaiokmo | jrosser: can you take a look at https://review.openstack.org/#/c/588462/ now? thanks :) | 14:21 |
| *** galaxyblr has quit IRC | 14:23 | |
| *** diliprenkila has joined #openstack-ansible | 14:25 | |
| jrosser | kaiokmo: ah, i did take a look but i don't understand the part about removing python - so i would like someone more familiar with opensuse to take a look at that | 14:26 |
| *** vnogin has joined #openstack-ansible | 14:27 | |
| *** hwoarang has quit IRC | 14:28 | |
| evrardjp | kaiokmo: thanks for working on this :) | 14:29 |
| *** vnogin has quit IRC | 14:31 | |
| *** fghaas has quit IRC | 14:32 | |
| *** diliprenkila has quit IRC | 14:33 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Reduce log level on CentOS by default https://review.openstack.org/589464 | 14:33 |
| *** vnogin has joined #openstack-ansible | 14:34 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Allow tags to be used for MQ tasks https://review.openstack.org/589468 | 14:34 |
| kaiokmo | evrardjp: no problem. we will not let those monasca roles be forgotten | 14:37 |
| *** dxiri has joined #openstack-ansible | 14:38 | |
| jrosser | odyssey4me: i'm making some progress with cinder - rabbitmq connect fail now - are there any unmerged patches that i need? | 14:38 |
| evrardjp | my switch at home is doing so much noise, I want to rip it apart with a saw and switch the fans. | 14:39 |
| evrardjp | jrosser: oh | 14:39 |
| spatel | cloudnull: i have tired this but didn't work "galera_monitoring_allowed_source: 1.1.1.1,2.2.2.2" | 14:39 |
| evrardjp | let's sync | 14:39 |
| *** fghaas has joined #openstack-ansible | 14:39 | |
| jrosser | evrardjp: theres a trivial error in the cinder defaults | 14:39 |
| evrardjp | jrosser: what I have noticed is that the service restart order matters | 14:39 |
| jrosser | evrardjp: this is wrong https://github.com/openstack/openstack-ansible-os_cinder/blob/12ea9635ff5f4d4dd9eee068e7ce27fb84a52846/defaults/main.yml#L366-L367 | 14:39 |
| evrardjp | I have restarted the services in a different order, no changes in config, and everything went back to normal | 14:39 |
| evrardjp | jrosser: interesting, what do you mean? | 14:40 |
| jrosser | it needs Service: in the line before Environment: otherwise the systemd unit is brok | 14:40 |
| jrosser | it makes a [DEFAULT] section as it stands which is just plain broken | 14:40 |
| evrardjp | jrosser: mmm | 14:40 |
| jrosser | evrardjp: http://paste.openstack.org/show/727629/ | 14:41 |
| jrosser | and the [DEFAULT] section contained the Environment: setting, it wasn't in [Service], so no PATH was set | 14:42 |
| gillesMo | Hello, is there an obligation to use IP addresses instead of FQDN in openstack_user_config.yml ? | 14:43 |
| evrardjp | jrosser: let me check on my node (I am rebuilding one atm) | 14:43 |
| jrosser | ok | 14:43 |
| *** gkadam has quit IRC | 14:43 | |
| evrardjp | jrosser: I have the same thing on my machine, but managed to get cinder working without changing those systemd files. | 14:48 |
| jrosser | interesting | 14:48 |
| evrardjp | it is. | 14:48 |
| evrardjp | plot tickens! | 14:48 |
| jrosser | do you agree the unit file is brok? i will patch that if so | 14:49 |
| evrardjp | I agree | 14:49 |
| jrosser | ok, one step at a time :) | 14:49 |
| evrardjp | so what I have noticed is that the scheduler takes a while to restart, and that could lead to some conditions when tasked of restarting many services at the same time. I doubt it's the same thing here, but still. | 14:50 |
| jrosser | what i have done is leave everything else alone | 14:50 |
| jrosser | just fixed the cinder-volumes unit and restarted it | 14:50 |
| jrosser | it's now moaning about rabbitmq | 14:51 |
| *** priteau has quit IRC | 14:53 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Allow tags to be used for MQ tasks https://review.openstack.org/589473 | 14:54 |
| *** vnogin has quit IRC | 14:57 | |
| evrardjp | yeah | 15:00 |
| evrardjp | restart cinder-api, cinder-scheduler, and then cinder-volume | 15:01 |
| evrardjp | and it worked for me | 15:01 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Fix cinder_environment_overrides https://review.openstack.org/589934 | 15:01 |
| evrardjp | my new system is rebuilding now, so I have to wait a little before I can try your patches, confirm what I manually did. | 15:01 |
| evrardjp | still rebuilding* | 15:01 |
| *** chason has joined #openstack-ansible | 15:02 | |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca master: Upgrade zookeeper repository https://review.openstack.org/589151 | 15:02 |
| jrosser | i will redo mine too | 15:02 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_cinder master: Fix cinder_environment_overrides https://review.openstack.org/589934 | 15:02 |
| *** priteau has joined #openstack-ansible | 15:04 | |
| jrosser | ty for the typo fix :) | 15:04 |
| *** chason has quit IRC | 15:06 | |
| *** chason has joined #openstack-ansible | 15:07 | |
| *** chason has quit IRC | 15:07 | |
| *** chason has joined #openstack-ansible | 15:08 | |
| *** vnogin has joined #openstack-ansible | 15:10 | |
| odyssey4me | evrardjp: that's the exact opposite of how they recommend it in their docs | 15:11 |
| evrardjp | odyssey4me: I have to confirm then | 15:11 |
| evrardjp | but that's what I did | 15:11 |
| spatel | My OSA is running fine but as soon as today i run "galera-install.yml -l ostack-controller-01_galera_container-966ec0a6" to update one of config i got this error, why its trying to create user now? | 15:14 |
| spatel | http://paste.openstack.org/show/727635/ | 15:14 |
| odyssey4me | evrardjp: https://docs.openstack.org/cinder/latest/upgrade.html#during-maintenance-window shows cinder-scheduler, cinder-volume, cinder-backup, cinder-api | 15:14 |
| spatel | is this normal ? | 15:14 |
| *** vnogin has quit IRC | 15:15 | |
| *** vnogin has joined #openstack-ansible | 15:18 | |
| jamesdenton | spatel is there a reason you're limiting to the one container? Perhaps the playbook won't run properly with a limit (not sure) | 15:18 |
| *** vnogin has quit IRC | 15:19 | |
| spatel | i wanted to try my change on single node to make sure i don't make mess on every node by mistake | 15:20 |
| *** DanyC has quit IRC | 15:20 | |
| spatel | now try thing "openstack-ansible galera-install.yml" | 15:20 |
| *** DanyC has joined #openstack-ansible | 15:21 | |
| spatel | I wish we have more tags in OSAD to target set of configuration, its alway scary to run any playbook on production stuff where you don't know what its going to do | 15:21 |
| *** DanyC has quit IRC | 15:21 | |
| spatel | jamesdenton: I am try to change only_from lint in /etc/xinted/mysqlchk | 15:22 |
| spatel | it would be great if have tag to just target xinted config | 15:23 |
| jamesdenton | make it so! :D | 15:23 |
| *** mmalchuk has joined #openstack-ansible | 15:23 | |
| jamesdenton | i'll have to defer tag sprawl to one of the ansible pros here | 15:23 |
| *** vnogin has joined #openstack-ansible | 15:23 | |
| *** gkadam has joined #openstack-ansible | 15:24 | |
| cloudnull | spatel https://github.com/openstack/openstack-ansible/blob/a3bb070fd6b81c5f765e43fbd5ecfec9cc050c28/inventory/group_vars/galera_all.yml#L30-L39 | 15:24 |
| *** luksky has quit IRC | 15:24 | |
| cloudnull | looks like its a space separated list | 15:24 |
| spatel | cloudnull: yes i figured | 15:25 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-os_ironic stable/queens: fix ironic driver data structure usage https://review.openstack.org/589838 | 15:25 |
| cloudnull | so the option should be `galera_monitoring_allowed_source: "1.1.1.1/32 2.2.2.2/32" | 15:25 |
| cloudnull | I think | 15:25 |
| prometheanfire | cloudnull: evrardjp odyssey4me, backport (only one needed) https://review.openstack.org/589838 ^ | 15:25 |
| spatel | I need to do this `galera_monitoring_allowed_source: "192.168.100.246 192.168.100.239 192.168.100.88 192.168.100.3 192.168.100.2 192.168.100.1 1.1.1.1 2.2.2.2 127.0.0.1"` | 15:25 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ironic stable/queens: fix ironic driver data structure usage https://review.openstack.org/589838 | 15:26 |
| odyssey4me | prometheanfire: that had already been done, and was based on another patch it needs | 15:26 |
| odyssey4me | which in turn needs another one in the integrated repo | 15:26 |
| odyssey4me | https://review.openstack.org/589838 requires https://review.openstack.org/589862 which requires https://review.openstack.org/589861 | 15:27 |
| prometheanfire | odyssey4me: uh, did I miss something? I'm talking about the os_ironic patch | 15:27 |
| odyssey4me | prometheanfire: yes, I had already done the backport this morning and it failed because queens doesn't have jmespath in requirements | 15:28 |
| prometheanfire | oh, queens doesn't have jsmepath | 15:28 |
| spatel | jamesdenton: this is the error i got when i am running galera playbook http://paste.openstack.org/show/727641/ | 15:28 |
| odyssey4me | so it's based on a patch to add it in both tests and the integrated repo | 15:28 |
| spatel | TASK [galera_server : Create galera users] give error | 15:28 |
| prometheanfire | odyssey4me: well, thanks for doing the work there then :D | 15:28 |
| odyssey4me | no worries, happy to help | 15:28 |
| odyssey4me | I figured that this was fairly important. | 15:28 |
| prometheanfire | broken role, ya | 15:29 |
| *** cshen has quit IRC | 15:29 | |
| cloudnull | spatel does curl `<galeraIP>:9200` return anything from the haproxy nodes? | 15:29 |
| jamesdenton | it's going thru your F5 now? Maybe the members have failed. | 15:29 |
| cloudnull | ^ | 15:30 |
| spatel | This is my DEV environment i am testing there to edit mysqlchk file | 15:30 |
| spatel | Still i am on haproxy LB until i resolve every issue i am scare to go on F5 | 15:30 |
| prometheanfire | looks like I added a space (CR) to the template, but won't harm anything | 15:30 |
| spatel | I pre-stage all config on F5 | 15:30 |
| spatel | cloudnull: my issue is why i am getting this error when i am running galera-install.yml playbook in working cluster http://paste.openstack.org/show/727641/ | 15:31 |
| *** mmalchuk has quit IRC | 15:31 | |
| *** mmalchuk has joined #openstack-ansible | 15:32 | |
| spatel | cloudnull: look like its trying to setup new cluster :( | 15:32 |
| *** Leo_m has joined #openstack-ansible | 15:34 | |
| *** pcaruana has quit IRC | 15:34 | |
| prometheanfire | looks like there's another problem with the ironic stuff, it's not installing python-ironic-inspector-client in the venv | 15:37 |
| *** DanyC has joined #openstack-ansible | 15:37 | |
| odyssey4me | prometheanfire: is in in the pip list, or in the requirements for ironic? | 15:37 |
| odyssey4me | evrardjp: looking at this, it seems there may be a way for me to add '--never-download' to the pip options somehow when building the venv https://github.com/ansible/ansible/blob/stable-2.5/lib/ansible/modules/packaging/language/pip.py#L423-L425 | 15:38 |
| odyssey4me | but I can't seem to figure out how - any idea? | 15:38 |
| prometheanfire | odyssey4me: I need to check, cjloader just pinged me about it | 15:40 |
| *** Emine has quit IRC | 15:40 | |
| prometheanfire | I need to talk to ironic people about it too, since the classic driver works without it, it looks like | 15:40 |
| *** DanyC has quit IRC | 15:41 | |
| *** Leo_m_ has joined #openstack-ansible | 15:41 | |
| *** rmart04 has quit IRC | 15:42 | |
| spatel | jamesdenton: question related F5, i am going to setup external VIP on F5 but i need internal VIP also right where all component talks to each other? | 15:42 |
| *** Leo_m has quit IRC | 15:42 | |
| -spatel- haproxy_keepalived_external_vip_cidr: "{{external_lb_vip_address}}/32" | 15:43 | |
| -spatel- haproxy_keepalived_internal_vip_cidr: "{{internal_lb_vip_address}}/32" | 15:43 | |
| spatel | Where these two VIP will fit in F5 ? | 15:43 |
| spatel | jamesdenton: ^^ | 15:43 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Allow tags to be used for MQ tasks https://review.openstack.org/589441 | 15:44 |
| spatel | Oh! wait i think i understand now how it will fix in F5 | 15:45 |
| *** hamzaachi has quit IRC | 15:46 | |
| spatel | I am seeing your script using internal VIP to setup F5 VS_SERVER | 15:47 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Add test for zfs backing on bionic https://review.openstack.org/589947 | 15:47 |
| spatel | jamesdenton: look like we have to setup 2 VS_SERVER for every service, one for internal and other for external | 15:50 |
| spatel | Do you have that way in your F5? | 15:50 |
| spatel | Example: http://paste.openstack.org/show/727647/ here in haproxy we have two VIP one for external and one for internal | 15:51 |
| spatel | cloudnull: Do you have any idea why it creating users ? | 16:00 |
| spatel | http://paste.openstack.org/show/727641/ | 16:00 |
| spatel | its messed up my whole production cluster again :( | 16:00 |
| spatel | This this crazy :( | 16:01 |
| spatel | I am going to open bug this is not good to do this in production :( | 16:02 |
| spatel | https://bugs.launchpad.net/openstack-ansible/+bug/1786070 | 16:05 |
| openstack | Launchpad bug 1786070 in openstack-ansible "URGENT - Running galera-install.yml playbook re-creating users and failing" [Undecided,New] | 16:05 |
| spatel | I have open bug so please help me out if possible | 16:06 |
| *** gillesMo has quit IRC | 16:07 | |
| evrardjp | odyssey4me: where does this happen? | 16:15 |
| evrardjp | I mean where do we need to pass virtualenv options? | 16:15 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Eliminate installing pip on host/containers https://review.openstack.org/587853 | 16:15 |
| evrardjp | maybe it's time to eliminate pip.conf too : p | 16:16 |
| odyssey4me | evrardjp: once the pip conf is laid down, using the pip module doesn't work for the virtualenv creation | 16:16 |
| odyssey4me | but yes, obviously it'd be nice to eliminate that - but I'd prefer to do it as a separate step | 16:16 |
| evrardjp | odyssey4me: maybe using virtualenv_command ? | 16:17 |
| odyssey4me | tried that, it's a 'path' for ansible, so it fails when validating it | 16:17 |
| evrardjp | ok | 16:17 |
| odyssey4me | so the only way in https://review.openstack.org/587853 I could work around it was to use the command module | 16:17 |
| evrardjp | maybe an action plugin | 16:17 |
| prometheanfire | odyssey4me: it looks like ironic is missing many things from it's pip env, my question is what we want to support and how (each driver selectivly including the pip deps or just install all pip deps all the time) | 16:18 |
| evrardjp | but why do we need the lock down ? I am not sure to understand https://review.openstack.org/#/c/587853/4//COMMIT_MSG | 16:18 |
| prometheanfire | https://github.com/openstack/ironic/blob/master/driver-requirements.txt | 16:18 |
| odyssey4me | well, I think there's room for a 'virtualenv_args' argument to be added to the pip module upstream | 16:18 |
| prometheanfire | also, oneview support is dying in stein | 16:19 |
| prometheanfire | so \o/ | 16:19 |
| prometheanfire | jroll: :D | 16:19 |
| jroll | oh hi | 16:19 |
| odyssey4me | evrardjp: if we remove the pip.conf, we then have to ensure that every pip install across all roles include reference to the right index so that the repo is still used | 16:20 |
| evrardjp | odyssey4me: that's the job of a good linter? | 16:21 |
| odyssey4me | we can't drop pip.conf until we're ready to drop repo build | 16:21 |
| prometheanfire | odyssey4me: should I just add all those drivers to https://github.com/openstack/openstack-ansible-os_ironic/blob/master/defaults/main.yml#L210 ? | 16:22 |
| odyssey4me | so for now, while we're not building all that many venvs outside of repo build, I think that workaround is suitable | 16:23 |
| evrardjp | odyssey4me: it is suitable | 16:24 |
| evrardjp | hello jroll | 16:24 |
| odyssey4me | prometheanfire: can ironic be installed using extras to specify the drivers you want? | 16:24 |
| evrardjp | what brings you here? (you're always welcome!) | 16:24 |
| odyssey4me | yes, one way is to add them all there - assuming you want them all in the venv | 16:24 |
| odyssey4me | another way is to use the 'optional_pip_packages' as is done in the neutron role | 16:25 |
| prometheanfire | odyssey4me: not yet, that's something jroll will hopefully have in stein | 16:25 |
| odyssey4me | extras is much nicer, as it's far more efficient | 16:25 |
| odyssey4me | both for repo build, and for venv content | 16:25 |
| prometheanfire | odyssey4me: it would be, but that's not in ironic yet | 16:26 |
| jroll | it's something jroll would like to have soon* | 16:26 |
| jroll | I can't commit to working on it | 16:26 |
| prometheanfire | odyssey4me: I didn't know repo-build supported it :D | 16:26 |
| prometheanfire | jroll: yourwelcome :P | 16:26 |
| jroll | don't worry, I have no problem letting people down :P | 16:26 |
| prometheanfire | :D | 16:26 |
| jroll | evrardjp: heh, the "oh hi" was responding to prometheanfire summoning me :) | 16:27 |
| evrardjp | odyssey4me: want me to wait a little more for rc1? want to review things with me? | 16:27 |
| evrardjp | jroll: then I am sad to announce you that you felt into a trap | 16:27 |
| prometheanfire | odyssey4me: optional_pip_packages, I don't see that in os_neutron's code base | 16:27 |
| evrardjp | fell* | 16:27 |
| prometheanfire | odyssey4me: I'm just going to install it all the time for now, as it seems like the cleanest solution (based on lack of support for extras) | 16:28 |
| odyssey4me | evrardjp: just a few of these left: https://review.openstack.org/#/q/topic:move-mq-setup-into-roles+status:open | 16:28 |
| spatel | odyssey4me: Can you answer of my basic question if i run galera-install.yml on working cluster does it going to re-install mariaDB again ? | 16:29 |
| evrardjp | I meant after those | 16:29 |
| odyssey4me | prometheanfire: look more carefully | 16:29 |
| odyssey4me | spatel: busy here, sorry | 16:29 |
| spatel | ok | 16:29 |
| odyssey4me | spatel: no, it doesn't reinstall every time - but it does verify every time | 16:29 |
| evrardjp | odyssey4me: when these are in I request RC1 and I add you to it, so that we can learn together of this :) | 16:30 |
| evrardjp | new shinies: ) | 16:30 |
| odyssey4me | spatel: ansible works that way - it will run every task to verify state | 16:30 |
| spatel | odyssey4me: in my case it re-install mariadb | 16:30 |
| spatel | and now my whole cluster is down | 16:30 |
| odyssey4me | spatel: ok, then I don't know why it's doing that and don't have time to help you work it out | 16:30 |
| spatel | This is driving me nuts every day... | 16:30 |
| evrardjp | i have to go | 16:30 |
| evrardjp | ttyl everyone | 16:30 |
| odyssey4me | spatel: that is not normal | 16:30 |
| odyssey4me | so look at the tasks and look at the code and figure out the conditional that's going wrong | 16:31 |
| spatel | I have opened BUG also in case if you guys find out.. it seems it is BUG because i have two running environment and same thing happened on both place so this is a BUG | 16:31 |
| odyssey4me | there is an upgrade flag which, if set, will cause that - that's all that comes to mind | 16:31 |
| prometheanfire | odyssey4me: oh, it's not the literal string | 16:31 |
| odyssey4me | prometheanfire: no, there are manu different optional package sets | 16:32 |
| odyssey4me | one for each different neutron plugin | 16:32 |
| prometheanfire | ya, see them now | 16:32 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible-os_ironic master: update pip packages for ironic drivers https://review.openstack.org/589967 | 16:32 |
| prometheanfire | we'll see if this works though | 16:32 |
| prometheanfire | odyssey4me: it'd be hard to do the same because ironic is more compostable in reguards to the drivers it supports https://docs.openstack.org/ironic/latest/admin/upgrade-to-hardware-types.html | 16:33 |
| noonedeadpunk | odyssey4me: will you have some time for masakari one day? As even if problem with lxc_container_create will be resolved, it still fails on the same thing (when it tries to reinstall python-libvirt, which is linked into the venv) | 16:33 |
| prometheanfire | but open to comments in the review as always :P | 16:33 |
| odyssey4me | prometheanfire: yeah, understood - very hard now, one day when I get to complete this python build simplification work it'll be simpler | 16:34 |
| noonedeadpunk | It's not urgent, but I feel, that I can't figure this out on my own... | 16:34 |
| odyssey4me | but right now, probably easiest to just jam them all into the venv | 16:34 |
| prometheanfire | odyssey4me: probably needs the extras support? | 16:34 |
| prometheanfire | cool' | 16:34 |
| odyssey4me | prometheanfire: once the work I'm trying to get done is done, each role can build multiple venvs and have more complex package list build mechanism - we won't be relying on the py_pkgs lookup | 16:35 |
| odyssey4me | noonedeadpunk: we're prepping for rocky RC1 this week, and focusing on bugs in it for the next two weeks - features are low priority right now, so sorry - I'm somewhat distracted | 16:36 |
| odyssey4me | I'll try and grab a gap somewhere. | 16:37 |
| jroll | prometheanfire: I remember about ironic-oneviewd now, it isn't in driver-requirements.txt because it's a separate application, not something imported | 16:37 |
| prometheanfire | lol, good thing I kept it in then | 16:38 |
| prometheanfire | odyssey4me: neat | 16:38 |
| odyssey4me | logan- jrosser cloudnull d34dh0r53 I need another review for https://review.openstack.org/582296 please to finalise that body of work | 16:39 |
| prometheanfire | so, general question. how does the core for a role thing work? It kinda looks like I'm taking over the ironic role | 16:40 |
| *** DanyC has joined #openstack-ansible | 16:41 | |
| spatel | Should i use "-e 'galera_ignore_cluster_state=true'" option with galer-install.yml ? | 16:43 |
| jamesdenton | spatel sorry, i was afk | 16:43 |
| *** hamzaachi has joined #openstack-ansible | 16:44 | |
| spatel | jamesdenton: no problem, I am still dealing with my galera cluster issue, somehow its trying to re-install cluster even its installed but anyway i have open BUG for that, i believe that is CentOS 7.x stuff | 16:44 |
| spatel | Majority of people using Ubuntu here so may be not seen this issue before | 16:45 |
| spatel | jamesdenton: did you read my F5 related question i posted earlier | 16:45 |
| jamesdenton | yes | 16:46 |
| *** DanyC has quit IRC | 16:46 | |
| spatel | Do i need to create two VS for each services right ? external & internal ? | 16:46 |
| jamesdenton | we tend to have two VIPs for each service - one considered Internal (adminurl,internalurl) and the other External (publicurl) | 16:46 |
| spatel | In short i have to duplicate them | 16:47 |
| jamesdenton | whether you do that in your environment is up to you. If your 'internal' address it routable in your env, then you may not need the external | 16:47 |
| jamesdenton | potentially, yes. | 16:47 |
| spatel | This is private cloud not publicly accessible, everything inside datacenter | 16:48 |
| spatel | If i don't create external VIP then i will loose SSL | 16:48 |
| jamesdenton | is the internal VIP accessible from clients? | 16:48 |
| jamesdenton | well yes, there's that too | 16:48 |
| jamesdenton | i would have separate addresses, personally | 16:49 |
| spatel | Its not accessible but i can make it routable... | 16:49 |
| spatel | anyway let me find out how i can handle this smoothly | 16:49 |
| *** DanyC has joined #openstack-ansible | 16:50 | |
| spatel | first let me fix this galera stupid issue :( | 16:50 |
| *** gyee has joined #openstack-ansible | 16:52 | |
| spatel | I have noticed "TASK [galera_server : Remove conflicting distro packages]" task un-installing my mariadb and then re-installing back WTH!!!! | 16:53 |
| spatel | Now question is why it think its conflicting package? | 16:58 |
| *** gkadam has quit IRC | 17:01 | |
| jrosser | cloudnull: https://review.openstack.org/#/c/589947/ <- it lives | 17:02 |
| spatel | It seems i am right this is a bug https://bugs.launchpad.net/openstack-ansible/+bug/1742206 | 17:06 |
| openstack | Launchpad bug 1742206 in openstack-ansible "os-cinder-install.yml fails with conflicting mariadb packages on centos infra hosts" [Medium,Fix released] - Assigned to Markos Chandras (hwoarang) | 17:06 |
| -spatel- Removing mariadb-server and mariadb-libs leads to | 17:07 | |
| -spatel- removing MariaDB-server and MariaDB-shared and its | 17:07 | |
| -spatel- dependencies | 17:07 | |
| -spatel- | 17:07 | |
| odyssey4me | spatel: looks like https://review.openstack.org/#/c/578844/ merged in master, what branch are you using? | 17:12 |
| spatel | odyssey4me: i am using 17.0.8 Queens | 17:12 |
| odyssey4me | spatel: ok, I don't know if that's the right patch to fix it though - IIRC that list of packages comes from epel or base CentOS, and is meant to be removed because they conflict with the packages from MariaDB themselves, which we use | 17:13 |
| -spatel- # Conflicting packages with those from the MariaDB repository | 17:13 | |
| -spatel- galera_server_mariadb_distro_packages_remove: | 17:13 | |
| -spatel- - mariadb-common | 17:13 | |
| -spatel- - mariadb-config | 17:13 | |
| -spatel- - mariadb-server | 17:13 | |
| -spatel- - mariadb-libs | 17:13 | |
| -spatel- - percona-xtrabackup | 17:13 | |
| spatel | these packages are for Ubuntu not for CentOS 7.5 | 17:13 |
| odyssey4me | but meh, I'm not familiar enough with CentOS/RHEL to make an informed decision | 17:14 |
| spatel | that is why its trying to remove them | 17:14 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server stable/queens: Do not remove MariaDB-server https://review.openstack.org/589973 | 17:15 |
| spatel | In master branch i am not seeing that stanza https://github.com/openstack/openstack-ansible-galera_server/blob/master/vars/redhat-7.yml | 17:15 |
| odyssey4me | ok, I've proposed the backport - it'd be good if you try it and feed back in review whether it solves the issue you saw | 17:15 |
| spatel | so look like master branch has those fix | 17:15 |
| odyssey4me | spatel: yes, as I said, https://review.openstack.org/#/c/578844/ merged in master | 17:16 |
| odyssey4me | I've proposed the backport for queens https://review.openstack.org/589973 | 17:16 |
| spatel | odyssey4me: any chance it will get marge in Queens ? | 17:17 |
| odyssey4me | spatel: if you try it and it works, then vote +1 and say so in the comment - that will help motivate others to vote positively | 17:17 |
| spatel | I am going to do right now! This is production issue, i have to deal with this ASAP | 17:18 |
| spatel | Running playbook now after hand fix | 17:19 |
| spatel | odyssey4me: EPIC!! it fixed my issue, my cluster is back and no re-install or install of any package! | 17:22 |
| spatel | where is the Vote button? | 17:22 |
| odyssey4me | spatel: great - please provide feedback in the review | 17:22 |
| odyssey4me | it's the 'comment' button, then a +1 vote with a comment | 17:22 |
| spatel | I am on this Page, where is comment button? | 17:24 |
| spatel | https://review.openstack.org/#/c/578844/ | 17:24 |
| odyssey4me | you need to be on https://review.openstack.org/589973 | 17:24 |
| odyssey4me | sorry, the button is 'reply' in the middle top | 17:24 |
| spatel | got it | 17:24 |
| spatel | odyssey4me: done! | 17:26 |
| noonedeadpunk | odyssey4me: don't worry actually. I thought that it would be great to make it some kind of stable for rocky, but nothing critical won't happen if it won't | 17:26 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Allow tags to be used for MQ tasks https://review.openstack.org/589440 | 17:27 |
| mnaser | odyssey4me: "jmespath-for-prez-stable/queens" nice!! | 17:28 |
| spatel | odyssey4me: you can close this bug also alone with that patch https://bugs.launchpad.net/openstack-ansible/+bug/1786070 | 17:28 |
| noonedeadpunk | odyssey4me: If you'll have some ideas what to try or how to workaround a problem - I'll be right here to check it and try. The problem is that I'm almost out of ideas | 17:28 |
| openstack | Launchpad bug 1786070 in openstack-ansible "URGENT - Running galera-install.yml playbook re-creating users and failing" [Undecided,New] | 17:28 |
| *** fghaas has quit IRC | 17:29 | |
| mnaser | team: i'd like to evaluate chatting with tripleo folks about https://review.openstack.org/#/c/589129/ | 17:29 |
| odyssey4me | spatel: thanks - appreciate the feedback, hopefully that'll get merged and will be released in due course | 17:31 |
| odyssey4me | mnaser: yet another tempest role, it'd be nice if we could collaborate and use something that's more broadly maintained | 17:32 |
| mnaser | odyssey4me: i just added a comment about that, please chime in | 17:32 |
| odyssey4me | mnaser: I thought that devstack was moving towards an ansible based setup too | 17:32 |
| mnaser | it has, i think, i dunno how far out that is, but i dont think it's reasonable for this one to be ansible-role-tempest which feels like it's "the one". | 17:33 |
| mnaser | we called our openstack-ansible-os_tempest, tempest has it's own (i think?) and then this one | 17:33 |
| mnaser | odyssey4me: thanks, but also there is https://review.openstack.org/#/c/589133/ | 17:35 |
| mnaser | i left my comment there too | 17:35 |
| odyssey4me | I won't be holding my breath, but it would be nicde. | 17:36 |
| odyssey4me | *nice | 17:36 |
| *** FrankZhang has joined #openstack-ansible | 17:38 | |
| odyssey4me | cores - got some review ready for more eyes: https://goo.gl/k9cCfF | 17:41 |
| spatel | odyssey4me: thanks, that means i wasn't crazy about whining :) | 17:42 |
| odyssey4me | spatel: I would recommend you invest time learning to read the ansible code and following it. This community relies on volunteers - there is no-one paid to do support here. When working with open source software one should learn to leverage the community for directional help, not the full solution every time. The more you help yourself and others, the more the community grows and can help you back. | 17:44 |
| *** electrofelix has quit IRC | 17:44 | |
| *** gkadam has joined #openstack-ansible | 17:46 | |
| odyssey4me | Bugs are reviewed by the community once per week on Tuesdays. If you register good bug reports, you're more likely to get good results. If you participate in bug triage to help others, you'll learn more about the code base and help yourself more. It's a circle of help which benefits everyone in it. | 17:46 |
| spatel | odyssey4me: I am happy to work here and help you out and help myself too, I have spent my half life in puppet so ansible is new tool and trying to workaround it. I am learning ansible and python at present. | 17:47 |
| *** FrankZhang_ has joined #openstack-ansible | 17:47 | |
| spatel | odyssey4me: lets do it.. i am happy to review and validate bugs as much as possible, Let me know what i need to do. | 17:48 |
| *** FrankZhang has quit IRC | 17:49 | |
| odyssey4me | good show - welcome, and thanks for joining - hopefully together we can make it better | 17:49 |
| spatel | ++ | 17:49 |
| jamesdenton | odyssey4me PTL 2020 | 17:49 |
| odyssey4me | spatel: one thing that is definitely needed is more help improving CentOS builds... more hands are needed to make it work, and work well | 17:50 |
| odyssey4me | if you're a CentOS user, then that's something you need - which is a perfect motivation to help | 17:50 |
| spatel | We are very big CentOS shop so i am sure i can help you out with all CentOS stuff | 17:50 |
| spatel | 2000 server running on CentOS linux in my datacenter | 17:51 |
| *** hamzaachi has quit IRC | 17:51 | |
| odyssey4me | great stuff - I'd encourage you to build a good test environment that you can build and rebuild over and over again to test the latest master branch stuff, then find bugs, report them, triage them and submit patches to fix them if you can | 17:51 |
| odyssey4me | the sooner we catch bugs, and the more people we have doing triage, the better the final result | 17:52 |
| odyssey4me | jamesdenton: :p 2.5 cycles was enough to last me a while ;) | 17:52 |
| spatel | odyssey4me: sure, i will do that | 17:52 |
| spatel | Any specific version i should start with like rocky or queens etc.. | 17:53 |
| spatel | Got it master branch :) | 17:54 |
| odyssey4me | spatel: yep, master is rocky - and will soon be stein | 17:54 |
| spatel | odyssey4me: Do you guys do all your build test on single box with AIO or multi nodee? | 17:55 |
| spatel | node* | 17:55 |
| odyssey4me | master is always the next major release - and better to find and solve bugs before that releases | 17:55 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert playbooks into roles https://review.openstack.org/589721 | 17:55 |
| odyssey4me | spatel: I use AIO's most often, personally. They're quick, and I use a cloud to run them in so I can do many at a time. | 17:55 |
| odyssey4me | Sometimes multi-node is needed, then https://github.com/openstack/openstack-ansible-ops/tree/master/multi-node-aio is very useful | 17:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Add bionic to galaxy_info metadata https://review.openstack.org/588315 | 17:57 |
| spatel | I have few servers in datacenter so i will try to make them ready to build OSA with AIO | 17:57 |
| odyssey4me | spatel: if you're doing AIO's - better to do them virtually, because you'll need to rebuild them from scratch over and over again. | 17:57 |
| odyssey4me | being able to do snap/restore is handy if you can | 17:58 |
| spatel | That is what i am doing in my LAB ( I have big FAT VMware server and i spun up multiple VM to create multi-node environment to re-build over and over with snapshot etc..) | 17:59 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [DNM] test disabling pip_install build package install https://review.openstack.org/589986 | 18:04 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Add test for zfs backing on bionic https://review.openstack.org/589947 | 18:07 |
| *** Emine has joined #openstack-ansible | 18:09 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: Add bionic to galaxy_info metadata https://review.openstack.org/588310 | 18:19 |
| *** vnogin has quit IRC | 18:20 | |
| *** poopcat has joined #openstack-ansible | 18:23 | |
| FrankZhang_ | Just found OSA master's tempest probably not pointing to tempest's master, since we found our gating continuing failing on some parts which was already patched two months ago. The actual missing patch is this one: https://github.com/openstack/tempest/commit/7d2b636a30057ed8db8cfd4fe2248f509b3570f1#diff-a147b612e7ea0662bce254e8fbbf0e92 | 18:24 |
| FrankZhang_ | this line should make tempest pointing to master but have no idea if it works: https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2#L216 | 18:25 |
| *** FrankZhang_ has quit IRC | 18:26 | |
| *** FrankZhang has joined #openstack-ansible | 18:26 | |
| *** mattt has left #openstack-ansible | 18:28 | |
| *** mattt has joined #openstack-ansible | 18:28 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Add bionic to galaxy_info metadata https://review.openstack.org/588266 | 18:34 |
| spatel | jamesdenton: I am thinking why don't we create a iRules instead of creating lots of VS_SERVER ? | 18:35 |
| spatel | jamesdenton: that would be interesting if iRule works then it will be very easy to deploy | 18:35 |
| *** Darcidride has joined #openstack-ansible | 18:35 | |
| *** Darcidride has quit IRC | 18:38 | |
| *** hamzaachi has joined #openstack-ansible | 18:39 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/582296 | 18:41 |
| *** guhcampos has joined #openstack-ansible | 18:44 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Add bionic to galaxy_info metadata https://review.openstack.org/588264 | 18:45 |
| *** hamzaachi has quit IRC | 18:49 | |
| jamesdenton | spatel: iRules could certainly be used, but would likely over-complicate things in our case. Having multiple virtual servers is ideal for us | 18:50 |
| odyssey4me | FrankZhang: that may be best discussed with evrardjp when he's online, given that he implemented that | 18:51 |
| FrankZhang | odyssey4me thanks for the noticing! | 18:51 |
| spatel | jamesdenton: i think you right there, because in multiple virtual server you can easily find status of VIP and make changes on fly | 18:52 |
| jamesdenton | ++ | 18:54 |
| jamesdenton | the script makes it easy when the input is consistent. it's built more for us than anyone else, but easily modifiable to be used by anyone | 18:54 |
| jamesdenton | that 3307 VIP, incase you wondered, isn't used but CAN be used as a read-only mysql endpoint. Some of the services have that ability. So you can have all three members available instead of 1 | 18:55 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: Add bionic to galaxy_info metadata https://review.openstack.org/588268 | 18:56 |
| spatel | jamesdenton: yes i saw that 3307, i just ignore that | 18:57 |
| spatel | also i have noticed, you are using priority in MySQL vip to target single node | 18:57 |
| jamesdenton | yes | 18:57 |
| spatel | why not round-robin? | 18:57 |
| spatel | Galera is multi master | 18:58 |
| jamesdenton | at one point in time the recommendation was to only write to one. deadlocks maybe? Don't know if that's still a thing | 18:59 |
| spatel | hmm | 19:00 |
| noonedeadpunk | it's still recommended to write only on one server from galera cluster | 19:00 |
| mnaser | ^ i think it's still a thing | 19:01 |
| spatel | I wonder then why not use master-slave two node cluster, if galer isn't true multi-master | 19:01 |
| spatel | I think multi-master work better with shared SQL storage NDB | 19:02 |
| noonedeadpunk | spatel: as you can't write to slave even if master is down in classic mysql replication | 19:02 |
| *** FrankZhang has quit IRC | 19:04 | |
| *** fghaas has joined #openstack-ansible | 19:05 | |
| *** FrankZhang has joined #openstack-ansible | 19:05 | |
| *** fghaas has quit IRC | 19:09 | |
| evrardjp[m] | FrankZhang: that code was included because of the patch you are pointing to. without it, keystone wouldn't pass tempest. | 19:15 |
| evrardjp[m] | iirc. | 19:15 |
| FrankZhang | evrardjp[m] the bug I actually have is https://bugs.launchpad.net/openstack-ansible/+bug/1786095 OSA master's tempest is not the tempest master one | 19:16 |
| openstack | Launchpad bug 1786095 in openstack-ansible "tempest branch doesn't point to its master" [Undecided,New] | 19:16 |
| evrardjp | FrankZhang: this fix was just in gates indeed. | 19:18 |
| *** gkadam has quit IRC | 19:18 | |
| evrardjp | if you don't use bootstrap aio that's not included | 19:18 |
| evrardjp | on purpose | 19:18 |
| evrardjp | we can't presume ppl will deploy tempest in a certain way | 19:19 |
| evrardjp | but because we are close to rc1 it doesn't hurt to do things differently as we'll remember when we branch now : p | 19:20 |
| FrankZhang | evrardjp Just found this template is made for aio, and guess it won't work for mnaio test gating | 19:21 |
| evrardjp | FrankZhang: precisely | 19:22 |
| evrardjp | how come is that seen only now? | 19:22 |
| evrardjp | I am fine with patching differently now -- we are a few months forward now. | 19:22 |
| evrardjp | in fact, I'd welcome a revert and re-implementation tomorrow. | 19:23 |
| FrankZhang | evrardjp sry, the patching you mentioned is about the topic we talked about? | 19:23 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Execute service setup against a delegated host using Ansible built-in modules https://review.openstack.org/590011 | 19:26 |
| evrardjp | I've updated the bug | 19:27 |
| evrardjp | so you know what to do | 19:27 |
| evrardjp | thanks for the help FrankZhang ! | 19:27 |
| FrankZhang | evrardjp cool no problem! | 19:29 |
| *** dxiri has quit IRC | 19:32 | |
| kaiokmo | can I get a CR on https://review.openstack.org/#/c/589151/ please? :) | 19:32 |
| evrardjp | kaiokmo: I'd welcome a patch following odyssey4me 's suggestion :) | 19:36 |
| evrardjp | in the meantime, I see this as a positive improvement | 19:36 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova master: Cleanup files and templates using smart sources https://review.openstack.org/588951 | 19:37 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add bionic job to elk_metrics_6x testing https://review.openstack.org/589730 | 19:38 |
| odyssey4me | kaiokmo: yep, going through now | 19:38 |
| evrardjp | odyssey4me: why is https://review.openstack.org/#/c/579371/6 with -1? | 19:38 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Execute service setup against a delegated host https://review.openstack.org/590011 | 19:38 |
| evrardjp | I'd welcome this to be included in roles, but I guess this would have to be backported? | 19:39 |
| odyssey4me | evrardjp: because it needs changes to every single other repo before that can merge | 19:39 |
| evrardjp | we probably won't have the chance to do it for tomorrow | 19:39 |
| odyssey4me | the tox changes must go everywhere first | 19:39 |
| odyssey4me | so yeah, I can do that - if we want to go that route | 19:39 |
| odyssey4me | easy enough to script | 19:39 |
| evrardjp | well now I want to do an RC with the new method, and deal with what will appear. | 19:40 |
| evrardjp | spatel: if it works well and is simple, why would you change it? | 19:44 |
| spatel | evrardjp: you talking about Galera multi write? | 19:45 |
| *** rmart04 has joined #openstack-ansible | 19:45 | |
| odyssey4me | spatel: we've not had any compelling reason to use multi-master - the bottleneck is almost never the database | 19:45 |
| odyssey4me | the current mechanism is very, very safe and very, very simple | 19:45 |
| odyssey4me | we're using it over 100+ sites | 19:45 |
| *** rmart04 has quit IRC | 19:45 | |
| odyssey4me | we've also used it with clusters of over 500 computes | 19:46 |
| evrardjp | yeah. Rabbitmq is the hardest. | 19:46 |
| *** dxiri has joined #openstack-ansible | 19:46 | |
| odyssey4me | yes, rabbitmq and networking (if you're using Neutron L3) are your first bottlenecks | 19:46 |
| spatel | odyssey4me: i am sure you guys have more experience on this domain compare to me. i was just trying to understand reason behind it. | 19:47 |
| spatel | Initially i was planning to use L3 with Cisco ASA Router, but then i decided to go with VLAN base networking. | 19:48 |
| odyssey4me | spatel: openstack isn't all that db heavy | 19:48 |
| spatel | I have small cluster running on Packstack with DVR last 2 years so far so good no issue there. | 19:49 |
| odyssey4me | spatel: yep, vlan-based works and scales well - if you're only dealing with l2 in openstack it's simpler | 19:49 |
| spatel | In my network we get lots of DDoS and my application is bandwidth hungry so i try to stay away from L3 | 19:50 |
| odyssey4me | jamesdenton knows his way around networking in openstack though, and is the real expert ;) | 19:50 |
| jamesdenton | eeek | 19:50 |
| odyssey4me | and our friend logan- does some funky things too, and knows things | 19:51 |
| spatel | jamesdenton: I did talk to him last year related Cisco ASA and all network stuff, he gave me really good boot :) | 19:51 |
| jamesdenton | you were looking at ASR integration, right? | 19:51 |
| spatel | Yes! but finally i decided VLAN base | 19:51 |
| spatel | still i have cisco ASA hanging in my LAB and i want to run some experiment too | 19:52 |
| jamesdenton | cool. so a physical router/firewall as the gateway for some provider network(s) in neutron? | 19:52 |
| spatel | Yes i used my Cisco Nexus 9000 as my gateway for openstack and running all L2 network | 19:52 |
| *** dxiri has quit IRC | 19:53 | |
| *** Emine has quit IRC | 19:53 | |
| odyssey4me | cores, could we get another review on https://review.openstack.org/589973 to help spatel out :) | 19:53 |
| jamesdenton | yeah that'll work. we've done that too | 19:53 |
| spatel | yesterday i expend my ISP bandwidth from 40G to 80G and got new shinny /23 public IP pool for Openstack | 19:54 |
| spatel | If all goes well then i have 500 node waiting in queue for Openstack | 19:55 |
| jamesdenton | perfect! bitcoin here i come | 19:56 |
| spatel | We are already paying 300k per month to AWS so decided to build own cloud :) | 19:57 |
| spatel | we have few servers running in Rackspace too :) | 19:57 |
| jamesdenton | you're off to a good start sounds like | 19:58 |
| evrardjp | jamesdenton: haha just because he got servers in Rackspace, right? :p | 19:58 |
| jamesdenton | because he's here in @openstack-ansible! :D | 19:58 |
| evrardjp | ofc ! | 19:59 |
| evrardjp | :D | 19:59 |
| evrardjp | I'm more a juniper L2/"L3" switches myself. | 19:59 |
| evrardjp | sadly I only have cisco at home. | 19:59 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Execute service setup against a delegated host https://review.openstack.org/590011 | 20:00 |
| evrardjp | old stuff, not worth discussing, next topic! | 20:00 |
| *** Emine has joined #openstack-ansible | 20:00 | |
| jamesdenton | evrardjp when are you cutting RC1? | 20:01 |
| evrardjp | when this patch gets in: | 20:01 |
| evrardjp | https://review.openstack.org/#/c/589480/ | 20:01 |
| evrardjp | I hope tonight (for me), maybe tomorrow. | 20:02 |
| openstackgerrit | Frank Zhang proposed openstack/openstack-ansible master: Make tempest point to master https://review.openstack.org/590020 | 20:03 |
| jamesdenton | cool! what would next steps be on the OVN patches? i need to update the commit, but i answered your questions and would appreciate any feedback when you have time | 20:03 |
| evrardjp | jamesdenton: next step is merging I guess then? Except I see more stuff, but they were good IIRC. | 20:06 |
| jamesdenton | k | 20:07 |
| odyssey4me | evrardjp: it'd be good to get https://review.openstack.org/589733 in too, so we can also get https://review.openstack.org/589437 in before we cut | 20:09 |
| evrardjp | odyssey4me: have a look at the timings there: https://review.openstack.org/#/c/587544/ | 20:11 |
| *** luksky has joined #openstack-ansible | 20:13 | |
| spatel | jamesdenton: i have migrated haproxy to F5 and it seems working great! | 20:18 |
| jamesdenton | nice work! now is the time to verify failover ;) | 20:19 |
| spatel | for external VIP i just created single VIP for Horizon | 20:19 |
| spatel | mostly we will deal with GUI over port 443 | 20:19 |
| *** guhcampos has quit IRC | 20:20 | |
| jamesdenton | sounds simple enough | 20:20 |
| spatel | We are using terraform for aws, planning to use with openstack here too | 20:20 |
| spatel | what you guys use in your cloud? | 20:20 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add trusty support to the project https://review.openstack.org/590027 | 20:21 |
| odyssey4me | evrardjp: looks... similar? | 20:21 |
| jamesdenton | spatel that's really up to the users | 20:21 |
| spatel | jamesdenton: I am going to bounce node and will see how it goes :) | 20:22 |
| jamesdenton | well, that AND failover of the F5s | 20:22 |
| evrardjp | odyssey4me: yeah, but it still runs tempest. Maybe I should remove tempest things now, and see how it goes. | 20:22 |
| evrardjp | not runs, installs. | 20:22 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Replace tempest with healthchecks https://review.openstack.org/587544 | 20:23 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server stable/queens: Do not remove MariaDB-server https://review.openstack.org/589973 | 20:29 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: [DNM] test disabling pip_install build package install https://review.openstack.org/589986 | 20:32 |
| odyssey4me | evrardjp: oh yeah, no need to install tempest if it's not being run | 20:33 |
| jamesdenton | . openrc | 20:34 |
| jamesdenton | hey now | 20:34 |
| odyssey4me | logan-: FYI, I'm targeting a harder target - no pip on the hosts at all | 20:36 |
| odyssey4me | all venvs, no dirty hosts | 20:36 |
| logan- | nice | 20:36 |
| odyssey4me | we're super-close | 20:36 |
| logan- | i'm trying to get queens working in my gate | 20:36 |
| odyssey4me | I think we'll get it done for rocky | 20:36 |
| logan- | i have build packages disabled and uwsgi is blowing up because libpython2.7-dev is not installed | 20:36 |
| logan- | i guess we never realized it is a dependency for uwsgi because pip_install installs it normally | 20:36 |
| odyssey4me | logan-: some of these may help - there were some broken things: https://goo.gl/k9cCfF | 20:37 |
| cloudnull | odyssey4me ++ very nice! | 20:37 |
| odyssey4me | specifically the log collection things were broken | 20:37 |
| odyssey4me | thanks to jrosser for spotting and fixing | 20:37 |
| logan- | yeah weird @ the log things. did we rework that recently? | 20:38 |
| logan- | or has it just been broken for a while | 20:38 |
| odyssey4me | no, that's been broken for ageas | 20:38 |
| logan- | :( | 20:38 |
| odyssey4me | we just collected the raw logs, instead of compressing and renaming them | 20:38 |
| odyssey4me | so we've been bad infra customers | 20:38 |
| logan- | along those lines could you re-review https://review.openstack.org/#/c/582273/ please? :) | 20:39 |
| odyssey4me | oh yeah, I was just confused with that one because the commit msg didn't match the content | 20:39 |
| *** spatel has quit IRC | 20:40 | |
| logan- | yup, thats fixed up now, thanks | 20:40 |
| logan- | oh wait | 20:40 |
| odyssey4me | heh, I think you edited in gerrit - then reposted your fix from your fork | 20:40 |
| logan- | whoops yep | 20:41 |
| odyssey4me | without the commit fix | 20:41 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible master: Use ZUUL_SRC_PATH to pass the path to Zuul git sources https://review.openstack.org/582273 | 20:41 |
| logan- | i guess i screwed it up while i was rebasing stuff last week | 20:41 |
| logan- | thanks | 20:42 |
| odyssey4me | well, look at that - mnaser's previous vote has returned | 20:42 |
| odyssey4me | happy days | 20:42 |
| evrardjp | logan-: haha I have been lazy and this is a patch (with the path) I should have done long ago, instead of mimicing what gates are doing in my environment.... thanks for th patch! | 20:49 |
| odyssey4me | ok folks - I'm out for the night | 20:50 |
| logan- | evrardjp: :) getting there... working on un-jenkinsing my osa ci | 20:51 |
| *** aludwar has quit IRC | 20:55 | |
| *** aludwar has joined #openstack-ansible | 20:56 | |
| *** hamzaachi has joined #openstack-ansible | 20:57 | |
| *** pcaruana has joined #openstack-ansible | 21:08 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Allow tags to be used for MQ tasks https://review.openstack.org/589437 | 21:09 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build master: Use a venv to build the python wheels https://review.openstack.org/589633 | 21:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca master: Upgrade zookeeper repository https://review.openstack.org/589151 | 21:14 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert playbooks into roles https://review.openstack.org/589721 | 21:17 |
| *** priteau has quit IRC | 21:20 | |
| jrosser | cloudnull: i get a 50% memory footprint reduction in my AIO (after setup-hosts) with zfs rather than dir | 21:22 |
| jrosser | and that is entirely dependant on dedupe=on | 21:24 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Add jmespath to requirements.txt https://review.openstack.org/589861 | 21:25 |
| *** poopcat has quit IRC | 21:55 | |
| *** pcaruana has quit IRC | 22:02 | |
| *** nicolasbock has quit IRC | 22:06 | |
| *** spatel has joined #openstack-ansible | 22:07 | |
| *** hamzaachi has quit IRC | 22:07 | |
| cloudnull | jrosser that's awesome ! | 22:10 |
| *** spatel has quit IRC | 22:11 | |
| *** spatel has joined #openstack-ansible | 22:14 | |
| *** Emine has quit IRC | 22:23 | |
| *** priteau has joined #openstack-ansible | 22:25 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert playbooks into roles https://review.openstack.org/589721 | 22:25 |
| *** vnogin has joined #openstack-ansible | 22:40 | |
| *** hamzy_ has joined #openstack-ansible | 22:40 | |
| *** hamzy has quit IRC | 22:42 | |
| *** mubix_ has joined #openstack-ansible | 22:55 | |
| *** lbragstad has quit IRC | 23:01 | |
| *** Leo_m_ has quit IRC | 23:01 | |
| *** evrardjp has quit IRC | 23:02 | |
| *** SamYaple has quit IRC | 23:02 | |
| *** Tahvok has quit IRC | 23:02 | |
| *** noonedeadpunk has quit IRC | 23:02 | |
| *** mhayden has quit IRC | 23:02 | |
| *** tailDASHeF has quit IRC | 23:02 | |
| *** mwe1 has quit IRC | 23:02 | |
| *** oanson has quit IRC | 23:02 | |
| *** mubix has quit IRC | 23:02 | |
| *** chudster_ has quit IRC | 23:02 | |
| *** lhinds has quit IRC | 23:02 | |
| *** mudpuppy has quit IRC | 23:02 | |
| *** rickflare has quit IRC | 23:02 | |
| *** mubix_ is now known as mubix | 23:02 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Fix log compression https://review.openstack.org/589537 | 23:03 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Fix log compression https://review.openstack.org/589536 | 23:03 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Fix gate log collection https://review.openstack.org/589427 | 23:03 |
| *** Tahvok has joined #openstack-ansible | 23:03 | |
| *** mhayden has joined #openstack-ansible | 23:09 | |
| *** mwe1 has joined #openstack-ansible | 23:10 | |
| *** luksky has quit IRC | 23:26 | |
| *** vnogin has quit IRC | 23:26 | |
| *** vnogin has joined #openstack-ansible | 23:27 | |
| *** spatel has quit IRC | 23:33 | |
| *** vnogin has quit IRC | 23:44 | |
| *** tosky has quit IRC | 23:49 | |
| *** rmcall has joined #openstack-ansible | 23:50 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!