| *** markvoelker has quit IRC | 00:02 | |
| *** brokencycle has quit IRC | 00:02 | |
| *** macz has quit IRC | 00:30 | |
| *** dave-mccowan has quit IRC | 01:58 | |
| *** macz has joined #openstack-ansible | 03:50 | |
| *** udesale has joined #openstack-ansible | 03:53 | |
| *** macz has quit IRC | 03:55 | |
| *** gkadam has joined #openstack-ansible | 03:57 | |
| *** cshen has joined #openstack-ansible | 04:15 | |
| *** markvoelker has joined #openstack-ansible | 04:55 | |
| *** markvoelker has quit IRC | 05:00 | |
| *** chaptertoaster has joined #openstack-ansible | 05:09 | |
| chaptertoaster | having a bit of an issue after adding a new compute node (stein), neutron is giving me this error when i attempt to create an instance on that node | 05:31 |
|---|---|---|
| chaptertoaster | http://paste.openstack.org/show/773972/ | 05:31 |
| chaptertoaster | network configuration is effectively identical and i'm not sure why it would be lacking permission | 05:31 |
| chaptertoaster | identical to the other nodes i mean | 05:35 |
| *** raukadah is now known as chandankumar | 05:46 | |
| *** macz has joined #openstack-ansible | 05:51 | |
| *** macz has quit IRC | 05:56 | |
| *** kopecmartin|off is now known as kopecmartin | 05:56 | |
| *** jbadiapa has joined #openstack-ansible | 06:13 | |
| *** miloa has joined #openstack-ansible | 06:33 | |
| miloa | Morning | 06:33 |
| jrosser | chaptertoaster: can you check if you have this commit in your os_neutron ansible role? https://github.com/openstack/openstack-ansible-os_neutron/commit/f7c787974cec5871a61760838cf46be14a61acab | 06:43 |
| jrosser | evrardjp: you around? | 06:43 |
| evrardjp | jrosser: I am | 06:44 |
| jrosser | evrardjp: if you have a suse box to hand can you try this command https://zuul.opendev.org/t/openstack/build/b79d4ae2b4394e65ac8d91f0d88858c3/log/job-output.txt#12932 | 06:47 |
| jrosser | Or spot the obvious error :) | 06:47 |
| evrardjp | jrosser: if one of those isn't installed, the egrep would failed | 06:50 |
| evrardjp | that's the obvious one | 06:50 |
| evrardjp | (or if egrep isn't installed) | 06:50 |
| evrardjp | now let me check around the whole command | 06:50 |
| jrosser | All I did here was add the py3 libs | 06:51 |
| evrardjp | are those installed in the package first? | 06:52 |
| jrosser | https://review.opendev.org/#/c/680835/ | 06:52 |
| *** luksky has joined #openstack-ansible | 06:53 | |
| jrosser | Oh right, you mean the ordering might be wrong | 06:53 |
| evrardjp | correct | 06:53 |
| jrosser | Haha | 06:53 |
| evrardjp | fyi that's the output generated: http://paste.openstack.org/show/773974/ | 06:54 |
| jrosser | I did spend 10mins trying to start a suse vagrant box yesterday, looks like the images are called something different now so the OSA vagrantfile is probably out of date | 06:56 |
| evrardjp | yea, there was a move. I tried recently too | 06:56 |
| jrosser | And I didn’t find an obvious 15.1 cloud image either | 06:56 |
| evrardjp | in those cases, I use a container | 06:57 |
| evrardjp | podman run -it opensuse/leap:15.1 /bin/bash | 06:57 |
| evrardjp | and you have a playbook | 06:57 |
| evrardjp | playground* | 06:57 |
| jrosser | Cool | 06:57 |
| jrosser | Anyway, I nearly have p3 going | 06:58 |
| jrosser | Py3 | 06:58 |
| evrardjp | that's nice | 06:59 |
| evrardjp | is swift on py3? | 06:59 |
| evrardjp | or you leave it in 2? | 06:59 |
| jrosser | I only did the core stuff right now | 07:01 |
| jrosser | I’m not even sure we actually test swift to tell.... | 07:02 |
| jrosser | https://review.opendev.org/#/q/topic:osa-py3+(status:open+OR+status:merged) | 07:03 |
| *** miloa has quit IRC | 07:04 | |
| evrardjp | is everyone aware of the work you're doing there? except the fact we're talking about it here, and now :) | 07:04 |
| evrardjp | I am approving those patches, but I am not sure this is enough for moving to py3 :) | 07:06 |
| jrosser | No, I just had a hack on Friday, will bring it up on Tuesday at the meeting | 07:06 |
| evrardjp | good | 07:06 |
| jrosser | Actually I see what’s wrong on suse now | 07:06 |
| evrardjp | yes? | 07:07 |
| jrosser | I don’t think it ever explicitly installs those packages that it tries to symlink | 07:07 |
| evrardjp | (I haven't seen yet what you've done for py2/3 in the venv build role) | 07:07 |
| evrardjp | whether it's implicit or explicit it doesn't really matter, as long as they are installed. | 07:07 |
| evrardjp | to ease debugging, remove the egrep :) | 07:08 |
| jrosser | The only thing I had to do in the venv build was namespace the venv name with the python version | 07:08 |
| jrosser | Because the repo server needs a wheel build venv per python version | 07:09 |
| ChosSimbaOne | Morning | 07:09 |
| *** jawad_axd has joined #openstack-ansible | 07:09 | |
| *** trident has quit IRC | 07:10 | |
| evrardjp | jrosser: that looks good, but I haven't seen that one yet | 07:10 |
| evrardjp | jrosser: got it. Nice. | 07:11 |
| *** miloa has joined #openstack-ansible | 07:11 | |
| evrardjp | (I saw we already have the ternary on py2/3 versions already in the pkg install) | 07:13 |
| *** hamzy_ has quit IRC | 07:19 | |
| *** trident has joined #openstack-ansible | 07:21 | |
| *** southquist has joined #openstack-ansible | 07:29 | |
| *** hamzy_ has joined #openstack-ansible | 07:29 | |
| *** owalsh has quit IRC | 07:32 | |
| *** owalsh has joined #openstack-ansible | 07:33 | |
| southquist | Hello everyone, I am currently running openstack-ansible stable/pike and together with ceph-ansible luminus. I'm interested in upgrading ceph (to ceph-ansible 4.0) but I wonder about the compatibility with my version of openstack-ansible. Is there any recommendation on pairing openstack-ansible versions and ceph-ansible? | 07:41 |
| jrosser | southquist: do you deploy your own ceph cluster with ceph-ansible? or do you deploy your ceph with OSA? | 07:42 |
| *** kaiokmo has joined #openstack-ansible | 07:46 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Add python3 libs to link into the nova venv for suse https://review.opendev.org/680835 | 07:47 |
| *** pcaruana has joined #openstack-ansible | 07:49 | |
| *** markvoelker has joined #openstack-ansible | 07:50 | |
| *** sep has quit IRC | 07:51 | |
| *** sshnaidm|afk is now known as sshnaidm | 07:51 | |
| *** sshnaidm is now known as sshnaidm|ruck | 07:51 | |
| *** macz has joined #openstack-ansible | 07:52 | |
| *** markvoelker has quit IRC | 07:55 | |
| *** rpittau|afk is now known as rpittau | 07:55 | |
| *** macz has quit IRC | 07:56 | |
| *** ppetit has joined #openstack-ansible | 07:59 | |
| *** miloa has quit IRC | 08:04 | |
| southquist | jrosser: I just checked our setup and we actually use OSA to to deploy the ceph cluster. | 08:05 |
| jrosser | southquist: in that case you are limited to the version of ceph that is deployed with OSA pike | 08:05 |
| jrosser | primarily because the ansible version you are using is linked fundamentally to the OSA version, and ceph-ansible has a similar set of ansible version limitations | 08:07 |
| jrosser | seel "Releases" here https://docs.ceph.com/ceph-ansible/master/ | 08:07 |
| jrosser | *see | 08:07 |
| jrosser | if you deploy your ceph cluster independantly with ceph-ansible then you have less coupling between the versions | 08:09 |
| *** sep has joined #openstack-ansible | 08:13 | |
| southquist | Alright, well that makes sense. | 08:16 |
| southquist | I had my suspicions that the ansible versions might be a problem, well many thanks for your input! | 08:17 |
| *** miloa has joined #openstack-ansible | 08:20 | |
| ppetit | Hello | 08:30 |
| ppetit | I am experiencing package dependency problems with installing Stein release with RDO packages. That's on centos 7. | 08:31 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_swift master: Allow venv python interpreter to be overridden https://review.opendev.org/680944 | 08:33 |
| ppetit | Install of python-openstackclient is not happy :( it says python2-openstacksdk-0.27.0-0.20190405041020.4174082.el7.noarch (rdo-current-passed-ci) requires python2-cryptography >= 2.1 but available package is python2-cryptography-1.7.2-2.el7.x86_64 (core-0) | 08:33 |
| ppetit | This is a fresh install | 08:34 |
| ppetit | Has someone seen that before? | 08:34 |
| jrosser | ppetit: which playbook/openstack service is failing to install? | 08:35 |
| ppetit | jrosser: openstack-ansible setup-infrastructure.yml | 08:37 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_swift master: [DNM] test swift / py3 https://review.opendev.org/680946 | 08:37 |
| ppetit | jrosser: task is [Install distro packages] | 08:39 |
| jrosser | ppetit: for th utility container? | 08:39 |
| *** andymccr has joined #openstack-ansible | 08:39 | |
| ppetit | jrosser: yes | 08:39 |
| jrosser | well i really don't use centos myself, but the thing that i find a bit odd in your error message is "rdo-current-passed-ci" | 08:40 |
| jrosser | to me that suggests master/current version of the packages rather than a repo for stein versions | 08:41 |
| ppetit | https://pastebin.com/Y6gYxtTh | 08:41 |
| jrosser | ppetit: oh dear https://opendev.org/openstack/openstack-ansible-openstack_hosts/commit/b85ba313bee836e5ca4aad722e1593b98aa7557e | 08:43 |
| ppetit | jrosser: hmmm.... so this is broken :( | 08:45 |
| jrosser | it would seem so | 08:45 |
| jrosser | what are you building? an AIO / test lab / prod environment? | 08:46 |
| ppetit | jrosser: for now test lab but in view a prod deployment I am supposed to do next week. | 08:47 |
| jrosser | ok so you can fix this locally | 08:47 |
| *** kaiokmo has quit IRC | 08:48 | |
| ppetit | jrosser: I suppose but still I am confused ... how breaking things that big can pass unnoticed | 08:49 |
| jrosser | ppetit: i am a bit surprised because our CI is passing for stein | 08:50 |
| jrosser | but lets try to get you running first | 08:50 |
| jrosser | the list of repos is defined here https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/branch/master/vars/redhat.yml#L103 | 08:50 |
| ppetit | jrosser: yeah exactly. | 08:50 |
| jrosser | and that variable is then used here https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/branch/stable/stein/defaults/main.yml#L60 | 08:51 |
| jrosser | so on your deploy host, in /etc/openstack_deploy/user_variables.yml you can redefine the var openstack_hosts_package_repos and that will override the default one | 08:52 |
| ppetit | jrosser: okay.... I'll try that. thank you ! | 08:53 |
| jrosser | copy the original repo list, and i would guess you need openstack-stein instead of rdo-current-passed-ci | 08:53 |
| jrosser | and delete the bogus repo from your host, by hand | 08:54 |
| jawad_axd | Heat error: Any help/suggestion about this error http://paste.openstack.org/show/773976/ ? It doest show much information in log. Rest of the services working fine in environment. | 08:58 |
| *** hamzy_ has quit IRC | 09:07 | |
| jrosser | jawad_axd: can you try that again with openstack stack list --debug | 09:07 |
| jawad_axd | ok | 09:08 |
| jawad_axd | jrosser: http://paste.openstack.org/show/773978/ | 09:12 |
| jrosser | ppetit: looks like i was wrong - the repos should be set up for stein correctly https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/branch/stable/stein/defaults/main.yml#L147-L153 | 09:13 |
| jrosser | can you check that looks like what you have? | 09:13 |
| *** arxcruz_pto is now known as arxcruz | 09:27 | |
| *** jawad_axd has quit IRC | 09:51 | |
| *** markvoelker has joined #openstack-ansible | 09:51 | |
| *** macz has joined #openstack-ansible | 09:53 | |
| *** macz has quit IRC | 09:57 | |
| *** miloa has quit IRC | 09:59 | |
| *** markvoelker has quit IRC | 10:00 | |
| *** jawad_axd has joined #openstack-ansible | 10:05 | |
| *** jawad_axd has quit IRC | 10:10 | |
| *** cshen has quit IRC | 10:10 | |
| *** miloa has joined #openstack-ansible | 10:16 | |
| *** spsurya has joined #openstack-ansible | 10:21 | |
| *** cshen has joined #openstack-ansible | 10:26 | |
| jrosser | evrardjp: in the paste you gave me earlier http://paste.openstack.org/show/773974/ is it an error that the __pycache__ dir is included in the rpm? | 10:28 |
| jrosser | where error == packaging error | 10:28 |
| *** cshen has quit IRC | 10:31 | |
| *** mathlin has joined #openstack-ansible | 10:33 | |
| noonedeadpunk | mornings | 10:33 |
| *** dave-mccowan has joined #openstack-ansible | 10:52 | |
| openstackgerrit | Jonathan Rosser proposed openstack/ansible-role-python_venv_build master: Do not symlink the __pycache__ directory into the venv https://review.opendev.org/680971 | 10:56 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 10:57 |
| jrosser | evrardjp: you asked about python3 / swift, here is the first look https://zuul.opendev.org/t/openstack/build/420943e5ae8246eb94ab517f7f7975c4/log/job-output.txt#13749-13751 | 10:59 |
| *** nicolasbock has joined #openstack-ansible | 11:02 | |
| evrardjp | for the pycache let me ask if it's voluntary or not. It's done by architecture, so I don't think there would be a big problem of having it there, yet I don't see a reason to carry it either | 11:03 |
| jrosser | evrardjp: i've had to do this https://review.opendev.org/680971 | 11:04 |
| *** cshen has joined #openstack-ansible | 11:05 | |
| evrardjp | jrosser: makes sense | 11:06 |
| evrardjp | I think it's better to not symlink pycache, so even if the packages are changing, that code can stay | 11:06 |
| *** cshen has quit IRC | 11:10 | |
| *** rpittau is now known as rpittau|bbl | 11:12 | |
| *** udesale has quit IRC | 11:15 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 11:15 | |
| jrosser | evrardjp: actually it is a good idea anyway, i think there is a risk of polluting the system pycache dir from things running in the venvs if it every gets symlinked | 11:18 |
| jrosser | so we should never allow that to be linked | 11:18 |
| chandankumar | evrardjp: please have a look at this review https://review.opendev.org/#/c/680761/, thanks! | 11:23 |
| evrardjp | jrosser: exactly my point | 11:28 |
| evrardjp | :) | 11:29 |
| evrardjp | chandankumar: done | 11:29 |
| miloa | if someone can check https://review.opendev.org/#/c/671762/ and tell me if there is more things to do/change, thanks :) | 11:31 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Bump SHAs for stable/queens https://review.opendev.org/680864 | 11:32 |
| evrardjp | miloa: the title still says WIP :) | 11:34 |
| evrardjp | noonedeadpunk: I know it's not my job to follow this anymore, but where are we in this plan: https://docs.openstack.org/openstack-ansible/latest/contributor/periodic-work.html#development-cycle-checklist (we are around m3 btw) | 11:35 |
| evrardjp | jrosser: ^ | 11:35 |
| evrardjp | jrosser: I think you bumped ceph during this cycle already | 11:36 |
| evrardjp | but did someone bump galera/rabbit? | 11:36 |
| jrosser | ceph is done | 11:36 |
| jrosser | you read my mind | 11:36 |
| jrosser | i was thinging about this right now :) | 11:36 |
| evrardjp | :) | 11:36 |
| jrosser | and ansible version | 11:37 |
| jrosser | 2.9 is imminent | 11:37 |
| evrardjp | alternatively it might be the time where we stop bumping rabbitmq/galera, if we are at a "good enough" version in the OS | 11:37 |
| evrardjp | (which I doubt, because I suppose Stein is already above the OS provided versions of rabbitmq) | 11:38 |
| evrardjp | "os provided" | 11:38 |
| evrardjp | haha | 11:38 |
| miloa | evrardjp: ok :) | 11:39 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Update ansible-next experimental job to 2.9 https://review.opendev.org/680981 | 11:40 |
| openstackgerrit | Mikael Loaec proposed openstack/openstack-ansible-ceph_client master: Importing keyrings from files rather than from mons https://review.opendev.org/671762 | 11:42 |
| *** cshen has joined #openstack-ansible | 11:43 | |
| *** mathlin has quit IRC | 11:47 | |
| jrosser | logan-: would you be able to take a look at https://review.opendev.org/#/c/671762/ for miloa? | 11:47 |
| *** cshen has quit IRC | 11:48 | |
| *** ansmith has quit IRC | 11:51 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/rocky: Bump SHAs for stable/rocky https://review.opendev.org/680865 | 11:51 |
| *** macz has joined #openstack-ansible | 11:54 | |
| *** cshen has joined #openstack-ansible | 11:54 | |
| *** goldyfruit has quit IRC | 11:56 | |
| *** markvoelker has joined #openstack-ansible | 11:56 | |
| *** macz has quit IRC | 11:58 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Bump SHAs for stable/pike https://review.opendev.org/680866 | 12:01 |
| *** markvoelker has quit IRC | 12:02 | |
| *** ivve has joined #openstack-ansible | 12:10 | |
| *** hamzy_ has joined #openstack-ansible | 12:11 | |
| *** markvoelker has joined #openstack-ansible | 12:12 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_gnocchi master: Use python-rados instead of cradox https://review.opendev.org/680986 | 12:21 |
| *** jawad_axd has joined #openstack-ansible | 12:21 | |
| *** jawad_axd has quit IRC | 12:25 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Drop Cython requirements override https://review.opendev.org/680988 | 12:25 |
| *** luksky has quit IRC | 12:29 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 12:32 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/stein: Bump SHAs for stable/stein https://review.opendev.org/680867 | 12:32 |
| *** jawad_axd has joined #openstack-ansible | 12:39 | |
| jrosser | evrardjp: openstack-ansible-deploy-aio_metal-opensuse-15 passed with core services on py3 | 12:39 |
| *** jawad_axd has quit IRC | 12:44 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: Allow venv python interpreter to be overridden https://review.opendev.org/680993 | 12:44 |
| *** ansmith has joined #openstack-ansible | 12:45 | |
| *** jawad_axd has joined #openstack-ansible | 12:45 | |
| *** jawad_axd has quit IRC | 12:50 | |
| *** jawad_ax_ has joined #openstack-ansible | 12:50 | |
| *** gillesMo has joined #openstack-ansible | 12:51 | |
| *** cshen has quit IRC | 12:54 | |
| *** gkadam has quit IRC | 12:55 | |
| *** rpittau|bbl is now known as rpittau | 12:55 | |
| *** BjoernT has joined #openstack-ansible | 12:56 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Freeze roles for Train milestone https://review.opendev.org/681002 | 13:11 |
| *** dr_feelgood has joined #openstack-ansible | 13:11 | |
| *** luksky has joined #openstack-ansible | 13:14 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Add uwsgi role to role requirements https://review.opendev.org/680761 | 13:14 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: [RHEL8]Set venv_python_executable to python3 https://review.opendev.org/680344 | 13:14 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] test ansible 2.9 https://review.opendev.org/681003 | 13:18 |
| *** udesale has joined #openstack-ansible | 13:20 | |
| *** KeithMnemonic has joined #openstack-ansible | 13:24 | |
| noonedeadpunk | jrosser: regarding osa-py3 - they have parents without uwsgi (which eventually also needs passing _venv_python_executable) | 13:27 |
| noonedeadpunk | so we either need to place another patch to cover this, or rebase and modify yours | 13:28 |
| *** gillesMo has quit IRC | 13:29 | |
| *** Miouge has joined #openstack-ansible | 13:29 | |
| *** gillesMo has joined #openstack-ansible | 13:30 | |
| jrosser | noonedeadpunk: hmm?\ | 13:39 |
| Miouge | jrosser to follow up on southquist’s question, does that means that the upgrade of OSA will cover the upgrade of Ceph I suppose, but the manual-upgrade guide doesn’t say to run playbooks/ceph-install.yml it’s only since Stein that it says to run setup-infrastructure.yml | 13:40 |
| noonedeadpunk | So for example cinder https://review.opendev.org/#/c/680390/ This patch has head earlier than https://review.opendev.org/#/c/678200/ | 13:40 |
| noonedeadpunk | So that after it will be merged, cinder-api will be running python2 (since uwsgi uses python2 by default) | 13:41 |
| *** BjoernT_ has joined #openstack-ansible | 13:41 | |
| noonedeadpunk | while all other cinder services will be running python3 | 13:41 |
| noonedeadpunk | so, we either need to rebase 680390 on top of current master and pass cinder_venv_python_executable as uwsgi_python_executable | 13:43 |
| noonedeadpunk | or create another patch which will do that | 13:43 |
| jrosser | oh just a moment.... | 13:43 |
| noonedeadpunk | the same for placement | 13:43 |
| jrosser | i have dealt with that here temporarily https://review.opendev.org/#/c/680470/8/inventory/group_vars/all/all.yml | 13:44 |
| *** BjoernT has quit IRC | 13:44 | |
| jrosser | i think? | 13:44 |
| noonedeadpunk | yeah, it kinda works in terms of global override | 13:44 |
| jrosser | yes i wanted to ask you about this, what was the right thing to do | 13:45 |
| noonedeadpunk | but missing point of cinder_venv_python_executable since it won't influence api service | 13:45 |
| jrosser | and how much coupling there must be between the python version for the service venv and the uwsgi venv | 13:45 |
| noonedeadpunk | actually they are pretty independent... | 13:46 |
| jrosser | ^ maybe this shows i don't understand correctly how it is working? | 13:46 |
| jrosser | do we need both venv? | 13:46 |
| noonedeadpunk | yep, the point was that in case of metal installs we'll have uwsgi installed once inside it's venv | 13:47 |
| noonedeadpunk | vs uwsgi instalation inside every venv | 13:47 |
| jrosser | but what does that mean for the service? it's run via uwsgi isnt it? | 13:48 |
| noonedeadpunk | uwsgi runs the service inside it's venv. So we deploy uwsgi config for every service where we define serivce venv if it's not distro install https://opendev.org/openstack/ansible-role-uwsgi/src/branch/master/templates/uwsgi.ini.j2#L6 | 13:49 |
| *** prometheanfire has quit IRC | 13:49 | |
| noonedeadpunk | but I've just realized, that I didn't test the case, when on distro install some service will request python2 and another python3... | 13:49 |
| noonedeadpunk | I'm afraid that it can ruin things.... | 13:50 |
| *** prometheanfire has joined #openstack-ansible | 13:50 | |
| noonedeadpunk | or python_venv_build can handle this.... | 13:52 |
| jrosser | i have already had to deal with something similar for python_venv_build | 13:52 |
| jrosser | https://review.opendev.org/#/c/680632/ | 13:52 |
| jrosser | i am very deliberately working on this bit by bit so we find mixed environment errors | 13:52 |
| jrosser | and i think the same patch will cover the case where you need a uwsgi venv for py2 and py3 on a metal host at the same time | 13:53 |
| noonedeadpunk | Yeah, I guess so | 13:54 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 13:57 |
| *** BjoernT_ has quit IRC | 13:58 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/ansible-role-uwsgi master: Use a different venv per python executable https://review.opendev.org/681018 | 14:01 |
| *** BjoernT has joined #openstack-ansible | 14:08 | |
| *** BjoernT_ has joined #openstack-ansible | 14:12 | |
| *** jawad_ax_ has quit IRC | 14:12 | |
| *** jawad_axd has joined #openstack-ansible | 14:13 | |
| *** BjoernT has quit IRC | 14:13 | |
| *** jawad_axd has quit IRC | 14:13 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Use python3 for the ansible-runtime venv https://review.opendev.org/681026 | 14:14 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] test ansible 2.9 https://review.opendev.org/681003 | 14:25 |
| jrosser | evrardjp: this looks like an easy fix - missing python3-systemd package https://zuul.opendev.org/t/openstack/build/450311b8ae6849f69ed04eaf4774f00c/log/job-output.txt#10240 | 14:35 |
| jrosser | just a question of where the right place to install that is? | 14:35 |
| jrosser | which ansible role, i mean | 14:35 |
| cjloader | morning all | 14:40 |
| *** markvoelker has quit IRC | 14:41 | |
| *** macz has joined #openstack-ansible | 14:48 | |
| *** luksky has quit IRC | 14:48 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Use version filter instead of version_compare https://review.opendev.org/681043 | 14:51 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] test ansible 2.9 https://review.opendev.org/681003 | 14:52 |
| *** jroll has joined #openstack-ansible | 15:00 | |
| cjloader | jrosser: in response to the comment about the ironic venv, i thought we were combining the two venvs, and only install inspector stuff if the group exists | 15:02 |
| jrosser | In containers, is there one for ironic and one for inspector? | 15:03 |
| mnaser | jrosser: are you running nautilus by any chance? | 15:03 |
| jrosser | Yes | 15:04 |
| cjloader | good point, containers... | 15:04 |
| mnaser | jrosser: just got hit by a nasty bug where pre-14.2.2 the osd_memory_limit would be dynamically calculated from the cgroup | 15:04 |
| cjloader | are we planning to continue container support? | 15:04 |
| jrosser | mnaser: I see your adventure in ceph-devel..... nightmare :/ | 15:04 |
| mnaser | 14.2.3 is good but thought i'd let you know heh | 15:05 |
| cjloader | o/ mnaser | 15:05 |
| mnaser | hi cjloader | 15:05 |
| jrosser | Yeah we’re compiling 14.2.3 right now actually | 15:05 |
| mnaser | you use your own builds, any reason? | 15:05 |
| jrosser | For our keystone auth caching patch which won’t be in master until octopus | 15:06 |
| mnaser | ah ok | 15:06 |
| jrosser | Oh unwind, not packaged until octopus | 15:06 |
| jrosser | It’s in master now | 15:06 |
| *** gyee has joined #openstack-ansible | 15:07 | |
| *** goldyfruit has joined #openstack-ansible | 15:08 | |
| jrosser | cjloader: containers or not in the future you need to make it pass tests for master today, which has lxc jobs | 15:11 |
| cjloader | ok | 15:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Allow venv python interpreter to be overridden https://review.opendev.org/680462 | 15:12 |
| mnaser | jrosser: how do you make your builds, hand written process or? | 15:18 |
| jrosser | We have a Jenkins job | 15:18 |
| jrosser | It takes *hours* | 15:18 |
| jrosser | I’ll see if I can dig it out later, just tied up with other things | 15:19 |
| *** ivve has quit IRC | 15:19 | |
| *** jawad_axd has joined #openstack-ansible | 15:22 | |
| mnaser | no worries, i dont need it, it was just curiousity | 15:23 |
| *** cshen has joined #openstack-ansible | 15:23 | |
| *** southquist has quit IRC | 15:23 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add unified linters check https://review.opendev.org/679101 | 15:25 |
| *** jawad_axd has quit IRC | 15:26 | |
| *** jawad_axd has joined #openstack-ansible | 15:38 | |
| openstackgerrit | Merged openstack/ansible-role-python_venv_build master: Use a different wheel build venv per python executable https://review.opendev.org/680632 | 15:43 |
| *** cshen has quit IRC | 15:50 | |
| *** chandankumar is now known as raukadah | 15:53 | |
| *** rpittau is now known as rpittau|afk | 15:54 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 16:00 | |
| *** jawad_axd is now known as JAX | 16:02 | |
| *** JAX is now known as JAX_ | 16:03 | |
| openstackgerrit | Merged openstack/ansible-role-python_venv_build master: Do not symlink the __pycache__ directory into the venv https://review.opendev.org/680971 | 16:05 |
| *** JAX_ is now known as JAX | 16:06 | |
| *** JAX has quit IRC | 16:12 | |
| *** jawad_axd has joined #openstack-ansible | 16:12 | |
| *** miloa has quit IRC | 16:13 | |
| *** jawad_axd has quit IRC | 16:14 | |
| *** markvoelker has joined #openstack-ansible | 16:21 | |
| *** spsurya has quit IRC | 16:27 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use version filter instead of version_compare https://review.opendev.org/681062 | 16:36 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] test ansible 2.9 https://review.opendev.org/681003 | 16:37 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add nova placement to placement migration https://review.opendev.org/664867 | 16:37 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add metal upgrade jobs https://review.opendev.org/671105 | 16:38 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Use version filter instead of version_compare https://review.opendev.org/681064 | 16:40 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] test ansible 2.9 https://review.opendev.org/681003 | 16:41 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/ansible-role-uwsgi master: Use a different venv per python executable https://review.opendev.org/681018 | 16:42 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client master: Ensure python libs are linked to service venvs consistently https://review.opendev.org/680828 | 16:42 |
| *** jawad_axd has joined #openstack-ansible | 16:45 | |
| noonedeadpunk | jrosser: should I help you with replacement of version_compare? at least ansible-role-python_venv_build,lxc_container_create, lxc_hosts use it | 16:47 |
| noonedeadpunk | and nspawn_hosts | 16:48 |
| *** dr_feelgood has quit IRC | 16:49 | |
| jawad_axd | Need help with this heat related error http://paste.openstack.org/show/774336/ , http://paste.openstack.org/show/774335/ . Using stable stein and rest of the services working fine. | 16:50 |
| noonedeadpunk | and whats for heat uwsgi log? | 16:55 |
| jrosser | jawad_axd: "ERROR: <html><body><h1>504 Gateway Time-out</h1>" sometimes happens when trying to connect to http://blahblah actually hits an https endpoing | 16:55 |
| jrosser | noonedeadpunk is right - something has gone wrong in the heat service and you need the log from that | 16:55 |
| jrosser | if this is a multi-controller deployment remember that could be in any of the heat containers | 16:56 |
| jrosser | sometimes it is a good idea to either stop 2 of the containers or turn of 2 of the backends with haproxy so all the requests end up at one heat api server | 16:56 |
| jrosser | *turn off | 16:57 |
| *** trident has quit IRC | 16:58 | |
| jawad_axd | I have just one heat container, and using haproxy. I recreated container as well, still same issue. B.t.w I have another stein setup, that has issue with heat as well. Bug ? | 16:58 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-ops master: Use version filter instead of version_compare https://review.opendev.org/681068 | 17:00 |
| *** cshen has joined #openstack-ansible | 17:02 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-tests master: Use version filter instead of version_compare https://review.opendev.org/681069 | 17:02 |
| jawad_axd | noonedeadpunk: jrosser: Does this log gives some clue http://paste.openstack.org/show/774338/ maybe ? | 17:06 |
| *** trident has joined #openstack-ansible | 17:06 | |
| *** cshen has quit IRC | 17:06 | |
| noonedeadpunk | where it comes from? | 17:09 |
| noonedeadpunk | like from what file/comand output | 17:09 |
| jawad_axd | simple command " openstack stack list" | 17:10 |
| noonedeadpunk | I mean how did you retrieve http://paste.openstack.org/show/774338/ | 17:11 |
| jawad_axd | Its heat log with debug=true | 17:12 |
| noonedeadpunk | ok, can you try replacing log_file = /var/log/heat/heat.log with use_journal = True ? | 17:13 |
| jawad_axd | ok | 17:13 |
| noonedeadpunk | do you have the same by journalctl --unit heat-* ? | 17:14 |
| noonedeadpunk | after reloading heat services ofc | 17:14 |
| jawad_axd | noonedeadpunk: http://paste.openstack.org/show/774339/ | 17:17 |
| noonedeadpunk | oh, ok | 17:17 |
| *** cshen has joined #openstack-ansible | 17:18 | |
| noonedeadpunk | seems that it's missing /etc/uwsgi/heat-api.ini But I guess that file is supposed to be inside /etc/heat folder | 17:18 |
| jawad_axd | In /etc/heat are api-paste.ini and heat.conf | 17:20 |
| noonedeadpunk | let me double check smth | 17:21 |
| *** cshen has quit IRC | 17:22 | |
| noonedeadpunk | hm.. I'd say that /etc/uwsgi/heat-api.ini should be present | 17:23 |
| jawad_axd | yes it is | 17:23 |
| jawad_axd | it is present | 17:23 |
| noonedeadpunk | hmmm uwsgi[2229]: realpath() of /etc/uwsgi/heat-api.ini failed: No such file or directory | 17:24 |
| noonedeadpunk | ah, it's old log | 17:24 |
| noonedeadpunk | I'd say it's 1 day old | 17:24 |
| jawad_axd | journalctl --unit heat-* showed 1 day old log, right. | 17:28 |
| jrosser | jawad_axd: if you've got 'less' installed i think you can shift-g to get to the end of the log wuth journalctl | 17:31 |
| jawad_axd | yes, sorry for before. this is http://paste.openstack.org/show/774340/ | 17:32 |
| *** luksky has joined #openstack-ansible | 17:33 | |
| jrosser | interesting "raise RuntimeError("Systemd bindings do not exist")" | 17:34 |
| jrosser | which OS is this? | 17:34 |
| jawad_axd | centos7 | 17:34 |
| jrosser | source or distro install? containers or bare metal? | 17:34 |
| jawad_axd | source | 17:35 |
| jrosser | fwiw i have seen exactly this error today on an opensuse CI job | 17:35 |
| jawad_axd | container | 17:35 |
| jrosser | but not centos7, interestingly | 17:36 |
| jrosser | could you look and see if python-systemd package is installed on the host and in the container? | 17:36 |
| jawad_axd | ok | 17:37 |
| jawad_axd | systemd-python is not installed on container and host | 17:41 |
| jrosser | noonedeadpunk: what do you think? ^ interesting we merged all the log-to-systemd stuff with that missing? | 17:42 |
| *** spatel has joined #openstack-ansible | 17:42 | |
| spatel | Folk, i have deployed ceph cluster using openstack-ansible so my mon nodes running inside container, now i want to build 2 more mon nodes on metal machine so what i need to do? | 17:43 |
| spatel | In short trying to create 5 monitor nodes (currently i have 3 running in container) | 17:43 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: [WIP] Adds configuration and enables the Ironic Inspector https://review.opendev.org/680284 | 17:44 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: [WIP] Adds configuration and enables the Ironic Inspector https://review.opendev.org/680284 | 17:45 |
| *** udesale has quit IRC | 17:49 | |
| *** goldyfruit_ has joined #openstack-ansible | 17:51 | |
| *** goldyfruit has quit IRC | 17:53 | |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client master: Install ceph python3 libraries on debian systems https://review.opendev.org/680802 | 17:57 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use version filter instead of version_compare https://review.opendev.org/681062 | 17:58 |
| jrosser | jawad_axd: you could try installing those manually and see if the heat service will start properly | 18:04 |
| jrosser | try the container first | 18:04 |
| jawad_axd | jrosser: I installed maunally systemd-python on host and container, re ran os-heat.yml. Here is journal output http://paste.openstack.org/show/774343/ | 18:04 |
| jrosser | and is openstack stack list working now? | 18:05 |
| jawad_axd | Nope. Actually same error system binding error.I checked journal again. | 18:09 |
| jawad_axd | jrosser: http://paste.openstack.org/show/774345/ | 18:12 |
| jrosser | oh hold on | 18:13 |
| jrosser | python systemd bindings need to be in the heat venv i guess | 18:13 |
| jawad_axd | let me try | 18:25 |
| jawad_axd | jrosser: No success with installing python-systemd in venv as well. Same "Systemd bindings do not exist" | 18:34 |
| *** vesper11 has quit IRC | 18:52 | |
| *** BjoernT has joined #openstack-ansible | 18:59 | |
| *** jbadiapa has quit IRC | 19:00 | |
| *** BjoernT_ has quit IRC | 19:00 | |
| *** BjoernT_ has joined #openstack-ansible | 19:12 | |
| *** jbadiapa has joined #openstack-ansible | 19:13 | |
| *** BjoernT has quit IRC | 19:13 | |
| *** goldyfruit_ has quit IRC | 19:15 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Allow venv python interpreter to be overridden https://review.opendev.org/680466 | 19:30 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: Add python3 compatibility to horizon role https://review.opendev.org/681096 | 19:32 |
| *** goldyfruit has joined #openstack-ansible | 19:33 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 19:33 |
| *** vesper11 has joined #openstack-ansible | 19:41 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_murano master: Initial commit to os_murano role https://review.opendev.org/677830 | 19:48 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: [WIP] Adds configuration and enables the Ironic Inspector https://review.opendev.org/680284 | 19:48 |
| guilhermesp | noonedeadpunk: ^ found out the reason to be breaking in uwsgi task... there were some vars missing in defaults/main.yml | 19:48 |
| * guilhermesp but seems there are some adjustments to be done still http://paste.openstack.org/show/774350/ | 19:51 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use version filter instead of version_compare https://review.opendev.org/681062 | 19:51 |
| *** nicolasbock has quit IRC | 20:11 | |
| *** trident has quit IRC | 20:15 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Use python3 for the ansible-runtime venv https://review.opendev.org/681026 | 20:16 |
| jrosser | mnaser: https://review.opendev.org/681026 <- OMG that worked first time | 20:17 |
| jrosser | thats like all the plugins and connection management stuff, i can barely believe that | 20:17 |
| mnaser | jrosser: its one of those moments.. | 20:17 |
| *** nicolasbock has joined #openstack-ansible | 20:18 | |
| *** trident has joined #openstack-ansible | 20:27 | |
| *** trident has quit IRC | 20:32 | |
| *** trident has joined #openstack-ansible | 20:33 | |
| jrosser | lots of code review todo mnaser guilhermesp and anyone else around | 20:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Migrate aodh to use uWSGI role https://review.opendev.org/678350 | 20:37 |
| guilhermesp | jrosser: I'm around | 20:39 |
| guilhermesp | osa-py3 topic jrosser ? | 20:41 |
| jrosser | first 2-3 pages of http://bit.ly/2xA1eZC would be good to check down | 20:42 |
| *** kopecmartin is now known as kopecmartin|off | 20:45 | |
| *** ansmith has quit IRC | 20:49 | |
| *** CosmicSound has quit IRC | 21:01 | |
| *** jawad_axd has quit IRC | 21:01 | |
| *** CosmicSound has joined #openstack-ansible | 21:02 | |
| *** markvoelker has quit IRC | 21:02 | |
| *** CosmicSound has quit IRC | 21:02 | |
| *** CosmicSound has joined #openstack-ansible | 21:03 | |
| *** markvoelker has joined #openstack-ansible | 21:16 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: Add python3 compatibility to horizon role https://review.opendev.org/681096 | 21:21 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: Add python3 compatibility to horizon role https://review.opendev.org/681096 | 21:22 |
| *** ansmith has joined #openstack-ansible | 21:24 | |
| *** spatel has quit IRC | 21:28 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Use version filter instead of version_compare https://review.opendev.org/681069 | 21:44 |
| openstackgerrit | Merged openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/680863 | 21:45 |
| jrosser | guilhermesp: reason i -w some of those version_compare patches at the moment is https://1ab3f57b9e2ef73ea70d-5fceee2f007c99899c2486771c61cec7.ssl.cf2.rackcdn.com/681003/5/check/openstack-ansible-deploy-aio_metal-opensuse-15/269a66f/logs/ara-report/result/5dbac717-23bd-4d93-9598-33365afdad28/ | 21:47 |
| jrosser | which makes no sense at all - the ubuntu jobs all pass with those patches applied but all the other distros fail | 21:48 |
| jrosser | it could easily be an ansible 2.9 bug becasue i'm testing them all together with a 2.9 test job | 21:49 |
| *** BjoernT_ has quit IRC | 21:54 | |
| *** markvoelker has quit IRC | 21:59 | |
| *** markvoelker has joined #openstack-ansible | 22:01 | |
| *** goldyfruit has quit IRC | 22:01 | |
| *** markvoelker has quit IRC | 22:06 | |
| *** goldyfruit has joined #openstack-ansible | 22:08 | |
| *** luksky has quit IRC | 22:15 | |
| *** goldyfruit_ has joined #openstack-ansible | 22:21 | |
| *** goldyfruit has quit IRC | 22:24 | |
| *** markvoelker has joined #openstack-ansible | 22:36 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Drop mod_wsgi support https://review.opendev.org/679973 | 22:37 |
| *** EmilienM is now known as little_script | 22:39 | |
| *** little_script is now known as EmilienM | 22:42 | |
| *** markvoelker has quit IRC | 22:46 | |
| *** macz has quit IRC | 22:48 | |
| *** macz has joined #openstack-ansible | 22:49 | |
| *** goldyfruit_ has quit IRC | 22:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi master: Use python-rados instead of cradox https://review.opendev.org/680986 | 23:10 |
| *** owalsh has quit IRC | 23:11 | |
| *** owalsh has joined #openstack-ansible | 23:14 | |
| *** threestrands has joined #openstack-ansible | 23:14 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Add python3 libs to link into the nova venv for suse https://review.opendev.org/680835 | 23:26 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!