| *** BjoernT has quit IRC | 00:00 | |
| *** BjoernT_ has joined #openstack-ansible | 00:00 | |
| *** BjoernT_ has quit IRC | 00:01 | |
| *** BjoernT has joined #openstack-ansible | 00:01 | |
| *** BjoernT has quit IRC | 00:02 | |
| *** BjoernT_ has joined #openstack-ansible | 00:02 | |
| *** trident has joined #openstack-ansible | 00:03 | |
| *** ozzzo has quit IRC | 00:04 | |
| *** BjoernT_ has quit IRC | 00:06 | |
| *** macz has joined #openstack-ansible | 00:10 | |
| *** macz has quit IRC | 00:37 | |
| *** gyee has quit IRC | 00:39 | |
| *** markvoelker has joined #openstack-ansible | 01:00 | |
| *** markvoelker has quit IRC | 01:05 | |
| *** markvoelker has joined #openstack-ansible | 02:01 | |
| *** markvoelker has quit IRC | 02:05 | |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: [WIP] Adds configuration and enables the Ironic Inspector https://review.opendev.org/680284 | 02:27 |
|---|---|---|
| cjloader | https://review.opendev.org/#/c/680284/ | 02:32 |
| cjloader | mnaser: d34dh0r53 prometheanfire noonedeadpunk jrosser guilhermesp | 02:33 |
| cjloader | initial PR | 02:33 |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: [WIP] Adds configuration and enables the Ironic Inspector https://review.opendev.org/680284 | 02:36 |
| *** nicolasbock has quit IRC | 02:49 | |
| *** markvoelker has joined #openstack-ansible | 03:31 | |
| *** markvoelker has quit IRC | 03:41 | |
| *** gkadam has joined #openstack-ansible | 03:54 | |
| *** gkadam has quit IRC | 03:54 | |
| *** markvoelker has joined #openstack-ansible | 04:30 | |
| *** markvoelker has quit IRC | 04:35 | |
| *** raukadah is now known as chandankumar | 04:49 | |
| *** udesale has joined #openstack-ansible | 05:02 | |
| *** dr_feelgood has joined #openstack-ansible | 05:29 | |
| *** markvoelker has joined #openstack-ansible | 05:30 | |
| *** markvoelker has quit IRC | 05:35 | |
| *** radeks has joined #openstack-ansible | 05:44 | |
| *** radeks has quit IRC | 05:49 | |
| *** radeks has joined #openstack-ansible | 05:52 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 05:53 |
| noonedeadpunk | cjloader: if that's the patch you asked about for backporting - I'd say it's hardly backportable to stein, since it's completely new feature | 06:19 |
| *** kopecmartin|off is now known as kopecmartion | 06:23 | |
| *** kopecmartion is now known as kopecmartin | 06:24 | |
| *** cshen has joined #openstack-ansible | 06:41 | |
| jrosser | Morning | 06:46 |
| *** openstackgerrit has quit IRC | 06:51 | |
| *** luksky has joined #openstack-ansible | 06:56 | |
| *** udesale has quit IRC | 06:56 | |
| *** markvoelker has joined #openstack-ansible | 07:06 | |
| *** dr_feelgood has quit IRC | 07:06 | |
| *** dr_feelgood has joined #openstack-ansible | 07:07 | |
| *** markvoelker has quit IRC | 07:10 | |
| *** trident has quit IRC | 07:21 | |
| *** tosky has joined #openstack-ansible | 07:23 | |
| *** ivve has joined #openstack-ansible | 07:27 | |
| kaiokmo | morning jrosser | 07:28 |
| *** trident has joined #openstack-ansible | 07:29 | |
| jrosser | o/ | 07:29 |
| *** trident has quit IRC | 07:34 | |
| *** trident has joined #openstack-ansible | 07:43 | |
| ChosSimbaOne | morning :-) | 07:54 |
| *** openstackgerrit has joined #openstack-ansible | 08:03 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 08:03 |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 08:13 |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 08:16 |
| jrosser | chandankumar: what is this patch actually trying to do? https://review.opendev.org/617522 | 08:33 |
| chandankumar | jrosser: current we create venv then python2 env gots created but in RHEL-8, we need python3 venv so we are making changes so that based on env the venv_python_executable gets setted | 08:35 |
| jrosser | i guess i'm not understanding why you don't just override the default when you know you want a python3 venv | 08:36 |
| *** rgogunskiy has joined #openstack-ansible | 08:37 | |
| jrosser | so like here, this is a role that uses python_venv_build http://codesearch.openstack.org/?q=uwsgi_python_executable&i=nope&files=&repos= | 08:38 |
| jrosser | it's up to that role to drive if the venv is py2/3 | 08:39 |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 08:39 |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 08:39 |
| jrosser | chandankumar: look also, changing a default like that introduces a huge behaviour change for existing code? thats not good | 08:40 |
| *** markvoelker has joined #openstack-ansible | 08:45 | |
| chandankumar | jrosser: overrideing to py3 it will break other distro as stackviz is installed in venv everywhere | 08:48 |
| chandankumar | python3 does not available at all places | 08:48 |
| noonedeadpunk | so you can override or define it during particular include... | 08:49 |
| jrosser | so then the stackviz install needs to select the appropriate python | 08:49 |
| jrosser | via the distro specific vars | 08:49 |
| chandankumar | jrosser: for wsegi case, it will not affect, seems lika e nice idea | 08:50 |
| *** markvoelker has quit IRC | 08:50 | |
| chandankumar | for using distro vars in os_tempest itself | 08:50 |
| jrosser | chandankumar: look at it the other way - OSA is currently py2 everywhere and uses the python_venv_build role in many many places | 08:52 |
| jrosser | we install already on py3 capable systems but stick to py2 in the venvs | 08:52 |
| jrosser | so what happens if you change this default? | 08:52 |
| jrosser | as i understand it we will manage that transition by slowly working through and overriding venv_python_executable to migrate py2->py3 | 08:53 |
| chandankumar | jrosser: if we change default, at each palce https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/tasks/python_venv_install.yml#L54 | 08:54 |
| chandankumar | it is going to break | 08:54 |
| noonedeadpunk | oh, suse repos are back | 08:55 |
| jrosser | maybe odyssey4me has something to say on how py2->3 transition should be managed with python_venv_build? | 08:56 |
| noonedeadpunk | that's nice idea:) his input might be very valuable here | 08:57 |
| *** trident has quit IRC | 09:01 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible-os_tempest master: [RHEL8]Set venv_python_executable to python3 https://review.opendev.org/680344 | 09:03 |
| chandankumar | jrosser: noonedeadpunk ^^ | 09:04 |
| zbr | jrosser: i see nothing in the readme documenting that venv role is py2 specific by design/purpose. The only hint about its hardcoded default is around https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/defaults/main.yml#L98 | 09:04 |
| jrosser | the defaults file kind of is the documentation | 09:05 |
| zbr | and considering that python2 is going to be dropped rapidly very soon, I do think that it makes sense to have a dynamic default that respects the distro. | 09:05 |
| jrosser | imho this is more about not changing existing defaults and breaking other users of the role | 09:06 |
| jrosser | tripleo and OSA need to manage their py2-3 transition independently | 09:07 |
| zbr | so all/most openstack migrated to py3 but this rote does not? i am not saying this change shouldn not be announced | 09:07 |
| noonedeadpunk | OSA hasn't start it's migration process and we're not sure how this will influence roles. And we're using defaults everywhere | 09:07 |
| zbr | not to mention that those that really want to stick to python2 have only to add one variable. | 09:08 |
| noonedeadpunk | And CI do not include all of the roles. | 09:08 |
| noonedeadpunk | So what I'm saying that I don't know how this change would result deployments. | 09:09 |
| *** trident has joined #openstack-ansible | 09:09 | |
| noonedeadpunk | Maybe it's ok.... | 09:09 |
| zbr | usually (unit) tests are the contract, in the absence of tests around that default, it should be no issue to change it. in the end the change is not even so dramatic, is not like making default "python3". | 09:09 |
| noonedeadpunk | But I'd love to have more wide discussion | 09:09 |
| jrosser | i am pretty much -2 changing a role default that is so fundmental at such a late stage in the cycle, unless someone like odyssey4me can explain why it's ok | 09:10 |
| noonedeadpunk | but isn't zuul running ansible on python3 alread? | 09:10 |
| jrosser | right and this is orthoganol to the venv python version too? | 09:10 |
| zbr | i would be happy to add some functional testing too, but I had some rough time running tests locally on osa due to its non concentional setup, I am not even able to run tox, not even for linting but that is another issue. | 09:12 |
| zbr | noonedeadpunk: re zuul: answer is depends, depends on which version of ansible you configure the job to use, and obviously python version on the host. | 09:13 |
| jrosser | OSA installs and uses it's own ansible in CI, so thats more complicated too | 09:14 |
| jrosser | thats another transition that needs to be managed | 09:14 |
| zbr | jobs using ansible 2.8+ will default to prefer py3 when available, older ones prefer py2 *unless* someone is overriding default interpreter in their definition to point to py3. | 09:14 |
| jrosser | we are 2.8 on py2 currently | 09:15 |
| noonedeadpunk | jrosser: have we defined py2 for zuul explicitly? | 09:15 |
| jrosser | we put ansible in it's own venv in the bootstrap-ansible.sh | 09:16 |
| noonedeadpunk | ah | 09:16 |
| noonedeadpunk | So, then this patch should be kinda safe for us | 09:16 |
| jrosser | i was just trying to figure what it means for multinode | 09:16 |
| jrosser | if ansible_python is always the deploy host or is the target host version | 09:17 |
| jrosser | but i'm stuck in meetings right now | 09:17 |
| zbr | just to be sure: detection of python version is based on the *host*, it does not matter which version of ansible you have on ansible controler (it can be any) | 09:17 |
| noonedeadpunk | oh, than it's bad:) | 09:18 |
| noonedeadpunk | it really almost the same as hardcoding python3 nowdays | 09:18 |
| zbr | so there is no problem in using ansible28-py2 on one the controller and connecting to hosts that python3 only, or the other way around. | 09:18 |
| noonedeadpunk | but that means, that venv will be build against python3 | 09:19 |
| jrosser | zbr: but we want to control if the created venv are py2 or 3 depending on our own data, not whatever the host data happens to be | 09:19 |
| noonedeadpunk | ++ | 09:19 |
| jrosser | currently the python_venv role default defines this as py2 | 09:19 |
| zbr | jrosser: if you want to control which python version you want: you need to mention it. | 09:20 |
| noonedeadpunk | we should kinda find the middle ground... | 09:20 |
| jrosser | so imho the right thing for both OSA and tripleo is to override venv_python_executable where needed | 09:20 |
| jrosser | then at the beginning of U cycle maybe we switch the default? | 09:20 |
| noonedeadpunk | jrosser: dont we need to migrate to py3 until the T release? | 09:21 |
| noonedeadpunk | Since python2 become not supported during U | 09:22 |
| zbr | noonedeadpunk++ | 09:22 |
| jrosser | yes, and we do that incrementally with overrides as the code stands now | 09:22 |
| noonedeadpunk | yep, agree. We can't do that right today... | 09:22 |
| zbr | can't we make a risk assertion here? lets see how many places would be impacted from this change and see if we can fix them? | 09:23 |
| noonedeadpunk | It's nica idea but I'm not sure about our capacity right now... | 09:24 |
| noonedeadpunk | I guess I need at least mnaser to answer this question | 09:24 |
| noonedeadpunk | zbr: can you point me to code that need these changes from your side? | 09:29 |
| ChosSimbaOne | hi. the image_hosts role in the playbooks, when deploying towards an existing ceph cluster, should that be left out or still point to the infrastructure nodes? | 09:29 |
| noonedeadpunk | which uses python_venv_build I mean:) | 09:29 |
| zbr | noonedeadpunk: chandankumar already raised a patch at https://review.opendev.org/#/c/680344/1 -- not sure about other places. the role itself seems quite generic, with potential to be reused in many places. | 09:31 |
| zbr | so for me it makes sense to fix the role instead of overriding it every time we call it in order to make it work. | 09:32 |
| noonedeadpunk | the problem is, that we can't fix role right now, since we didn't test things against py3 which may result in some mess at this stage. | 09:34 |
| zbr | i do not have data to back up my theory but i suspect more users prefering a py3 env than a py2 already. | 09:34 |
| zbr | that role could easily be tested cross-platform with a single molecule job. | 09:35 |
| jrosser | zbr: the role is already used in many places | 09:35 |
| jrosser | and look at this, the pattern that jesse designed makes very heavy use of overriding defaults https://opendev.org/openstack/openstack-ansible-os_barbican/src/branch/master/tasks/barbican_install.yml#L42-L55 | 09:35 |
| jrosser | imho the pyhon version gets added across the many places we use the role | 09:36 |
| jrosser | and we use those vars to manage the transition | 09:36 |
| noonedeadpunk | Yep, we have in plans to migrate things to py3 during this cycle | 09:38 |
| noonedeadpunk | But I'm not sure if it's worth doing in such way instead of iterating one by one. | 09:38 |
| noonedeadpunk | And I'm not sure if this role isn't used somewhere outside of openstack by people | 09:38 |
| zbr | i would estimate ~30 places based on http://codesearch.openstack.org/?q=name%3A%20%22%3Fpython_venv_build&i=nope&files=.yml&repos= | 09:39 |
| noonedeadpunk | zbr: tbh I'd raise that question during meeting, since it's worth a bit wider discussion | 09:39 |
| zbr | the meeting etherpad is not even listed on the wiki page, adding it now | 09:42 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible-os_tempest master: [RHEL8]Set venv_python_executable to python3 https://review.opendev.org/680344 | 09:42 |
| chandankumar | zbr: noonedeadpunk jrosser ^^ done | 09:44 |
| chandankumar | zbr: use the wiki page | 09:44 |
| *** itandops has joined #openstack-ansible | 09:47 | |
| *** sshnaidm|ruck has joined #openstack-ansible | 09:53 | |
| evrardjp | noonedeadpunk: we're reverted the full python3 path in the path due to discrependencies between distros. | 10:00 |
| evrardjp | it was hard to debug, some projects were not full onboard with py3 | 10:00 |
| evrardjp | I think times have changed, and we could re-try | 10:00 |
| evrardjp | try again* | 10:00 |
| noonedeadpunk | I guess centos 7 nowdays should have python3.6 from some of the default repos? | 10:01 |
| evrardjp | in software collections, which suck | 10:01 |
| evrardjp | you need to wait for centos 7.7 | 10:01 |
| evrardjp | or centos 8 | 10:01 |
| noonedeadpunk | Yeah, I guess it was related to 7.7 releasenotes;) | 10:02 |
| zbr | is a known issue that current centos has py35 optional, and some tools are requiring py36 minimal, one that I know is black. 7.7/8 will address this. | 10:04 |
| *** luksky has quit IRC | 10:10 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-python_venv_build master: defaults: Dynamically determine the python version https://review.opendev.org/617522 | 10:10 |
| *** udesale has joined #openstack-ansible | 10:20 | |
| *** udesale has quit IRC | 10:28 | |
| *** udesale has joined #openstack-ansible | 10:29 | |
| *** cshen has quit IRC | 10:30 | |
| *** nicolasbock has joined #openstack-ansible | 10:41 | |
| noonedeadpunk | cores, can you kindly vote on https://review.opendev.org/#/c/679904/4 to unblock gates? | 10:46 |
| *** markvoelker has joined #openstack-ansible | 10:46 | |
| *** cshen has joined #openstack-ansible | 10:47 | |
| *** markvoelker has quit IRC | 10:52 | |
| *** cshen has quit IRC | 10:52 | |
| jrosser | noonedeadpunk: done | 10:57 |
| noonedeadpunk | evrardjp: have some time for reviewing ^ | 10:57 |
| *** luksky has joined #openstack-ansible | 10:58 | |
| *** cshen has joined #openstack-ansible | 11:00 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum master: Start using uWSGI role https://review.opendev.org/680103 | 11:05 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_magnum master: Start using uWSGI role https://review.opendev.org/680103 | 11:11 |
| chandankumar | jrosser: noonedeadpunk http://paste.openstack.org/show/771277/ | 11:17 |
| chandankumar | afte enabling cr repo in centos7.6 | 11:18 |
| noonedeadpunk | chandankumar: that's great news! | 11:18 |
| noonedeadpunk | evrardjp: ^ | 11:18 |
| chandankumar | yum-config-manager --enable cr to enable it | 11:19 |
| noonedeadpunk | We can proceed with the migration to python3 then | 11:19 |
| *** ansmith_ has quit IRC | 11:39 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_mistral master: Start using uWSGI role https://review.opendev.org/680378 | 11:46 |
| zbr | it seems the python3 from centos7.7 is missing libselinux and we all know what this means for anyone trying to use it with ansible. I raised https://bugs.centos.org/view.php?id=16389 in the hope that we will get a package for it (comment on it) | 11:48 |
| noonedeadpunk | zbr: thanks for the update | 11:53 |
| openstackgerrit | kourosh vivan proposed openstack/openstack-ansible-rabbitmq_server stable/stein: Add variable causing error on rerun for redhat OS https://review.opendev.org/680381 | 11:57 |
| *** markvoelker has joined #openstack-ansible | 12:02 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_octavia master: Start using uWSGI role https://review.opendev.org/680389 | 12:13 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Allow venv python interpreter to be overridden https://review.opendev.org/680390 | 12:17 |
| noonedeadpunk | mnaser: guilhermesp can you please take a look at https://review.opendev.org/#/c/679904/ | 12:18 |
| evrardjp | cr repo? | 12:18 |
| evrardjp | chandankumar: ^ | 12:18 |
| evrardjp | I thought it was 7.7 for having python3 | 12:19 |
| evrardjp | 7.6 and before had to deal with software collections | 12:19 |
| evrardjp | what is CR? | 12:19 |
| chandankumar | continous release | 12:20 |
| *** dr_feelgood has quit IRC | 12:20 | |
| chandankumar | evrardjp: https://wiki.centos.org/AdditionalResources/Repositories/CR | 12:21 |
| evrardjp | next point release, makes sense! | 12:21 |
| evrardjp | that's very good news chandankumar | 12:22 |
| noonedeadpunk | but it's apparently missing selinux module, which is not so great | 12:22 |
| *** mkuf_ is now known as mkuf | 12:25 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_octavia master: Start using uWSGI role https://review.opendev.org/680389 | 12:27 |
| *** ansmith_ has joined #openstack-ansible | 12:37 | |
| jrosser | noonedeadpunk: prototype for opt-in py3 https://review.opendev.org/#/c/680390/1 | 12:38 |
| redrobot | Hello friends! Could I get a +2/+A on this small change? https://review.opendev.org/#/c/680074/ | 12:42 |
| *** ansmith_ has quit IRC | 12:42 | |
| *** ansmith has joined #openstack-ansible | 12:42 | |
| evrardjp | I am annoying, but it's not on purpose :p | 12:43 |
| jrosser | redrobot: do you see the comment? | 12:44 |
| evrardjp | or should I say, I want my comments to help on improving our practices :) | 12:44 |
| noonedeadpunk | evrardjp: I'd wish you had more time for reviews :p | 12:44 |
| redrobot | jrosser, hi, yeah, just saw your comment. I understand your concern, but I'm not aware of any non-rhel interest in HSM support. | 12:46 |
| redrobot | jrosser, I would not be opposed to making this work with more distros, so if someone wants to do the needful, I'd be more than happy to review. | 12:47 |
| noonedeadpunk | evrardjp: posted error here https://review.opendev.org/#/c/679904/ don't look good... where from it goes? | 12:48 |
| *** dr_feelgood has joined #openstack-ansible | 12:55 | |
| *** dr_feelgood has quit IRC | 12:56 | |
| *** dr_feelgood has joined #openstack-ansible | 12:56 | |
| *** kupu-sama has joined #openstack-ansible | 12:57 | |
| *** kupu-sama has quit IRC | 12:59 | |
| *** pcaruana has quit IRC | 13:10 | |
| evrardjp | noonedeadpunk: sorry | 13:15 |
| evrardjp | redrobot: all the tasks look independent of the distro, it's just about adding a condition when: "ansible_os_family | lower == 'redhat'" | 13:17 |
| redrobot | evrardjp, ack, I can do that. | 13:18 |
| evrardjp | we don't need to ensure it works for all the distros, but at least it won't badly break if you try to run it on another distro | 13:18 |
| evrardjp | it encourages people to contribute :) | 13:18 |
| evrardjp | redrobot: thanks, I can review immediately if you like | 13:19 |
| *** spatel has joined #openstack-ansible | 13:22 | |
| openstackgerrit | Douglas Mendizábal proposed openstack/ansible-role-thales-hsm master: Allow barbican to access /opt/nfast when selinux is enforcing https://review.opendev.org/680074 | 13:26 |
| redrobot | evrardjp, ^^ updated. 😁 | 13:26 |
| *** pcaruana has joined #openstack-ansible | 13:40 | |
| *** spatel has quit IRC | 13:41 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_panko master: Start using uWSGI role https://review.opendev.org/680416 | 13:49 |
| cjloader | noonedeadpunk: ty for your review | 13:51 |
| cjloader | as you can tell its rough right now | 13:52 |
| *** BjoernT has joined #openstack-ansible | 13:54 | |
| *** rgogunskiy has quit IRC | 13:55 | |
| *** admin0 has joined #openstack-ansible | 13:55 | |
| openstackgerrit | Douglas Mendizábal proposed openstack/ansible-role-thales-hsm stable/stein: Allow barbican to access /opt/nfast when selinux is enforcing https://review.opendev.org/680419 | 13:56 |
| *** bhenc has joined #openstack-ansible | 13:56 | |
| openstackgerrit | Merged openstack/ansible-role-thales-hsm master: Allow barbican to access /opt/nfast when selinux is enforcing https://review.opendev.org/680074 | 13:56 |
| redrobot | evrardjp, I'm also backporting the change to stable/stein: https://review.opendev.org/#/c/680419/ | 13:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Rename cinder distro packages to match convention https://review.opendev.org/679904 | 13:58 |
| *** BjoernT_ has joined #openstack-ansible | 14:08 | |
| *** BjoernT has quit IRC | 14:09 | |
| *** spatel has joined #openstack-ansible | 14:10 | |
| spatel | I am writing doc for disaster recovery, so have question what will happened when all 3 controller/infra nodes dead? | 14:11 |
| *** pcaruana has quit IRC | 14:12 | |
| spatel | i believe all running vms will be fine but we can't add/remove/update anything. | 14:12 |
| spatel | but what about DHCP lease? | 14:12 |
| admin0 | spatel, when all controllers are dead, all you can do is reinstall and migrate/rsync from the vms that are running | 14:14 |
| spatel | admin0: i can understand that part, but lets say tonight all three infra nodes dead... how long my vms going to hold IP address from DHCP lease? | 14:15 |
| spatel | Infra nodes running neutron-dhcp-agent | 14:15 |
| noonedeadpunk | spatel: regarding dhcp lease everything is pretty bed. iirc DHCP lease is given for 5 minutes by default. So I'd recommend you to extend lease time | 14:16 |
| spatel | noonedeadpunk: oh really? | 14:16 |
| noonedeadpunk | So that you have some time for recovery | 14:16 |
| *** dr_feelgood has quit IRC | 14:16 | |
| admin0 | as soon as you know the infra nodes are dead, you need to change the IP to static | 14:16 |
| noonedeadpunk | Maybe it's changed now but that what I can recall from rocky | 14:16 |
| spatel | how do i increase lease time? how long you guys suggest? | 14:16 |
| admin0 | so that if there is an accidental restart, your vms still hold the IP address | 14:16 |
| noonedeadpunk | admin0: how do you offer to change ip inside vms? | 14:17 |
| spatel | admin0: i would like to make it 7 days... | 14:17 |
| admin0 | and to prepare for this situation, make sure you set/know the root pass so that u can login via console | 14:17 |
| admin0 | from inside the OS, u can set the ip to static .. as long as the ip/mac matches, it will work | 14:17 |
| noonedeadpunk | it can work for private cloud only | 14:17 |
| cshen | spatel: I think the default DHCP expire time in Rocky is 1 day. | 14:17 |
| spatel | I am running stein | 14:18 |
| spatel | cshen: can we change it to 7 days? | 14:18 |
| *** dr_feelgood has joined #openstack-ansible | 14:18 | |
| admin0 | one qustion .. do you guys set sometime isuses when heat deletes the VM first and not cinder, and cinder is stuck and needs manual database change to delete the instance .. like instance is deleted .. but volume was not unmounted, so cinder just waits and waits and goes into error state | 14:18 |
| admin0 | see sometimes* | 14:18 |
| cshen | spatel: of course, it's configurable. | 14:19 |
| *** luksky has quit IRC | 14:19 | |
| cshen | search this dhcp_lease_duration in neutron config. | 14:19 |
| spatel | cshen: can i do that in production, hope it will readjust without dropping any packet | 14:20 |
| cshen | when you resinstall neutron, the new cfg will be there. | 14:20 |
| cshen | we adjust dhcp lease time during our major upgrade. | 14:20 |
| noonedeadpunk | Yep, you're right, it's 1 day https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/templates/neutron.conf.j2#L90 | 14:20 |
| spatel | noonedeadpunk: thanks for confirmation.. now question is can we adjust it in production and restart services? | 14:21 |
| admin0 | yep | 14:21 |
| noonedeadpunk | +2 | 14:21 |
| admin0 | since its not in variables yet, u have to override the neutron.conf directly and rerun the neutron playbook | 14:22 |
| cshen | if all ctr* are down, there is another problem of dns in each network. if you use dns from dhcp which are also on the ctr*, then it will be a problem. | 14:22 |
| spatel | admin0: thanks... ( i am going to adjust right now ) | 14:22 |
| noonedeadpunk | Just wondering how I ended up in expiring leases... Probably that wasn't osa environment.. whatever | 14:22 |
| *** dr_feelgood has quit IRC | 14:22 | |
| spatel | cshen: we are not running DNS on infra | 14:22 |
| openstackgerrit | Merged openstack/ansible-role-thales-hsm stable/stein: Allow barbican to access /opt/nfast when selinux is enforcing https://review.opendev.org/680419 | 14:23 |
| cshen | spatel: override lease time like this http://paste.openstack.org/show/771448/ | 14:23 |
| spatel | cshen: sweet!! can i change it to 7 Days? anything wrong to make it longer lease? | 14:24 |
| cshen | I never changed it to be longer than 1 day :-D | 14:25 |
| cshen | maybe google if it's bad when dhcp lease time is longer than 1day. | 14:25 |
| spatel | cshen: hmm! look like you guys have faster recovery plan :) | 14:25 |
| spatel | in my case we have small team and in worst case if folks on vacation and take longer time... | 14:26 |
| noonedeadpunk | I can hardly iamgine when all network nodes are down at once and neither of them can't be recovered fast. | 14:26 |
| cshen | we reduce the dhcp lease time especially for pike -> queens upgrade. | 14:26 |
| noonedeadpunk | While everything is ok with all compue nodes... | 14:26 |
| spatel | noonedeadpunk: totally understand it shouldn't take long time but you never know.. which power breaker make you day.. | 14:27 |
| spatel | your* | 14:28 |
| *** tiffanie has quit IRC | 14:28 | |
| noonedeadpunk | in case of power issue you most likely found your compute nodes down as well | 14:28 |
| *** tiffanie has joined #openstack-ansible | 14:28 | |
| admin0 | i have some heat-configs that are extensively complex .. they come up fine . but when i do a cleanup, it kills the server first before killing/unmounting cinder and i end up with stuck cinder volumes with instances already deleted | 14:29 |
| spatel | We have 48 racks in different rows across datacenter.. total 500 compute nodes in various racks, but yes you never know until you see the issue.. | 14:30 |
| spatel | noonedeadpunk: anyway thanks.. i am going to adjust for 2 days.. in worst case.. 7 days would be little longer.. | 14:31 |
| cshen | 48 racks feel like not a small setup. | 14:32 |
| cshen | good luck! | 14:32 |
| spatel | cshen: :) | 14:32 |
| noonedeadpunk | especially when you have team that can take a vacation at once :P | 14:33 |
| spatel | cshen: so just change dhcp_lease_duration= in /etc/neutron/neutron.conf and restart neutron-server? | 14:33 |
| admin0 | spatel, use the override | 14:33 |
| noonedeadpunk | you kinda need to use neutron conf overrides | 14:33 |
| admin0 | else next time, it will get overwritten again if you just edit and change | 14:33 |
| noonedeadpunk | and run os-neutron-setup.yml --tags neutron-config | 14:33 |
| spatel | admin0: i will but i want to test to make sure it works before pushing config to all 3 controller nodes | 14:33 |
| *** tiffanie has quit IRC | 14:34 | |
| * noonedeadpunk hopes that neutron tags already worked for yours osa version | 14:34 | |
| *** tiffanie has joined #openstack-ansible | 14:34 | |
| admin0 | spatel, you need to test in an AIO or in your acceptance environment | 14:34 |
| spatel | admin0: i do have AIO but in bad shape after lots of poking.. i think you are right.. let me first bring it back in life and try stuff there | 14:35 |
| admin0 | or just edit manually in all 3, restart, create a new VM .. check lease time .. if new, you are good .. do the override ( but not run it ) but next time you run it, it will be the same output as well | 14:36 |
| spatel | noonedeadpunk: i believe --tags neutron-config isn't working for me last time when i tried, why did you say that? | 14:36 |
| spatel | admin0: also my existing VMs get longer DHCP time when next time they talk to DHCP for renewal right? | 14:37 |
| noonedeadpunk | spatel: Hm, I'd say they should since I don't see any task or role include for neutron in stein | 14:38 |
| spatel | i am running stein and i was having issue there, i will check again on my AIO | 14:38 |
| noonedeadpunk | but ok, maybe there might be some undefined variable errror | 14:38 |
| admin0 | in the logs, you can grep dhclient or renewal and it will show you the renewal in seconds | 14:39 |
| spatel | noonedeadpunk: also i remember --tags nova-config also throwing error | 14:39 |
| admin0 | think there is also a file in /var/lib/dhcleint where u can view the stuff | 14:39 |
| spatel | admin0: +1 /var/lib/dhcleint | 14:39 |
| -spatel- # cat /var/lib/dhclient/dhclient--ens5.lease | grep lease | 14:40 | |
| -spatel- lease { | 14:40 | |
| -spatel- option dhcp-lease-time 86400; | 14:40 | |
| *** cshen has quit IRC | 14:41 | |
| noonedeadpunk | also I guess you can check with nmap, like sudo nmap --script broadcast-dhcp-discover | 14:42 |
| admin0 | spatel, btw, lxd works/worked fine .. can make flavor that only goes to lxd nodes .. only issue is even i use ceph and the kvm uses ceph, it sticks to using local disk | 14:42 |
| admin0 | and as a user, df -h shows me the whole disk and not only what the flavor says .. apart from that, works fine\ | 14:43 |
| spatel | admin0: nice!! do you have doc or anything which i can use to make it work? | 14:44 |
| admin0 | i will have it up in 1-2 days | 14:44 |
| admin0 | i am currently stuck with why my cinder gets stuck .. maybe its a heat issue .. or my cinder backend is wrong | 14:45 |
| spatel | community doesn't support LXC but i don't care as far as it works for me.. | 14:45 |
| spatel | Are you using heat for VM deployments? | 14:45 |
| admin0 | yep | 14:45 |
| spatel | oh! okay.. did you ever configure autoscaling with heat? | 14:46 |
| spatel | my next project is to work with autoscaling | 14:46 |
| *** udesale has quit IRC | 14:46 | |
| *** udesale has joined #openstack-ansible | 14:47 | |
| admin0 | i enabled aodh,ceilometer,gnocci for autoscaling .. did not used .. forgot about it .. one day they filled up all controller disk space to 100% which led to corrupted database .. so uninstalled :) | 14:47 |
| admin0 | thankfully it was not in production | 14:47 |
| spatel | I am not running aodh,ceilometer,gnocci because they fill your signaling plan with extra traffic and choke rabbitMQ etc.. | 14:49 |
| noonedeadpunk | humm, my gnocchi in ceph don't use really much of diskspace... It creates tons of objects, but regarding diskspace it's pretty fair... | 14:49 |
| noonedeadpunk | But I'm not collecting all data every minute, so... | 14:50 |
| *** gyee has joined #openstack-ansible | 14:50 | |
| spatel | admin0: i love grafana :) i pull all information from openstack to grafana for better navigation (horizon is horrible ) https://imgur.com/a/kImeUs1 | 14:53 |
| admin0 | write a howto please :) | 14:53 |
| noonedeadpunk | spatel: yeah, nice idea btw | 14:55 |
| spatel | noonedeadpunk: check more in same link i have added VMs also so anyone can easily search for VM running on X compute node.. https://imgur.com/a/kImeUs1 | 14:56 |
| spatel | 311 compute nodes 791 VMs... :) | 14:57 |
| noonedeadpunk | yeah that's nice. I've created dashboard for resource from gnochhi to see what exact vm utilize cpu/disk/memory/etc on compute node | 14:58 |
| spatel | noonedeadpunk: can you shared? i would like to see what info it provide | 15:00 |
| noonedeadpunk | yeah, just searching for the dashboard:) | 15:00 |
| spatel | also i am pulling error codes and pushing in grafana check it out 3rd screenshot https://imgur.com/a/kImeUs1 | 15:00 |
| spatel | noonedeadpunk: interesting... i never get time to look at but look like worth | 15:02 |
| spatel | can i run gnocchi outside OSA ? | 15:02 |
| spatel | i don't want to overload infra anymore :) | 15:02 |
| admin0 | spatel, actually you can run everything ourside infra with your own mapping | 15:04 |
| admin0 | when i started first with osa, i had a special data nodes where i ran rabbit and mysql .. ( they only had SSD's ) .. but not anymore | 15:04 |
| spatel | hmm! i have never seen rabbitMQ going crazy in my environment.. what is your mesg rate on rabbitMQ ? | 15:05 |
| noonedeadpunk | for example http://i.piccy.info/i9/3a4dbec4985c01650d8219c25fe320bd/1567695978/160258/1336302/Screenshot_20190905_180155.png http://i.piccy.info/i9/7f982c7bd0dc907b858ad32ae0fd655a/1567696007/328382/1336302/Screenshot_20190905_180101.png | 15:06 |
| spatel | in my production its around 300/s | 15:06 |
| noonedeadpunk | but it can be basically any metric from docs.openstack.org/ceilometer/stein/admin/telemetry-measurements.html | 15:06 |
| spatel | noonedeadpunk: OMG!! i love second graph with all vms report.. is this gnocchi ? | 15:07 |
| noonedeadpunk | Yep | 15:07 |
| spatel | how much extra CPU it need to collect all these metrics ? | 15:07 |
| noonedeadpunk | The first one also have all vms, that had disc activity at all | 15:07 |
| spatel | noonedeadpunk: so you are running that many vms on single compute nodes?? really? | 15:08 |
| noonedeadpunk | tbh never calculated in comparison... | 15:08 |
| noonedeadpunk | It's kinda small node out of our "sandbox" since our "sandbox" contain some infra things... | 15:09 |
| noonedeadpunk | like that grafana server | 15:09 |
| noonedeadpunk | but, I mean my ex one:) Since it's not about vexxhost | 15:10 |
| spatel | hmmm! very impressive graph.. | 15:10 |
| spatel | what is the difference in collecting meterics from VMs itself and collecting from compute nodes hypervisor ? | 15:11 |
| noonedeadpunk | except that you need access to vms? | 15:11 |
| spatel | We are collecting all kind of data from VMs itself... trying to understand what would be the advantage to collect from KVM ? | 15:13 |
| *** tosky has quit IRC | 15:15 | |
| noonedeadpunk | I dont see real advanteges, especially if you're collecting data with snmp (which don't require any agents). There's no access to vms as it's kinda sandbox of public cloud | 15:15 |
| noonedeadpunk | since there's no access to vms, there's no choice except collecting from host | 15:15 |
| noonedeadpunk | But advantage might be is a gnocchi architecture itself. So if you like it and it suites your needs it might be the main advantage | 15:16 |
| noonedeadpunk | Since it's as a metric storage engine is really interesting | 15:16 |
| spatel | noonedeadpunk: totally agreed.. | 15:20 |
| spatel | we are using agent to collect data from VMs.. Zabbix etc.. | 15:20 |
| noonedeadpunk | yeah, zabbix is cool - I rly like it:) | 15:20 |
| spatel | noonedeadpunk: i am going to give it a shot anyway.. i would like to have all data in hand :) | 15:21 |
| spatel | noonedeadpunk: are you guys running public cloud or private.. (don't tell me if its NDA related question) | 15:21 |
| noonedeadpunk | actually, if you like python, or whatever, you may write a script for collection vms data from hypervisor and sending to server with zabbix sender | 15:22 |
| *** devsum has quit IRC | 15:22 | |
| spatel | I am thinking to do that with telegraf agent to hook it up with KVM | 15:22 |
| noonedeadpunk | I'd say Vexxhost is running both:) | 15:24 |
| spatel | cool! Do you guys running LinuxBridge or OVS? | 15:25 |
| spatel | because i have big discussion going on with my team about it | 15:25 |
| noonedeadpunk | I'd say OVS, but personally I like linuxbridges since they started supporting vxlans | 15:26 |
| spatel | i have zero experience with OVS but sure about performance between linuxbridge vs OVS | 15:27 |
| spatel | i have zero experience with OVS but no sure about performance between linuxbridge vs OVS | 15:27 |
| spatel | Did you guys see any noticeable difference in performance? | 15:28 |
| noonedeadpunk | Don't have an answer to this question | 15:28 |
| jamesdenton | spatel https://www.slideshare.net/slideshow/embed_code/key/vjD9JBXhU7bSYh | 15:28 |
| spatel | jamesdenton: OMG.. you are alive :) | 15:29 |
| jamesdenton | Slide 41 -> End | 15:29 |
| jamesdenton | hah yeah | 15:29 |
| noonedeadpunk | I'd say the same, yeah... I guess ovs is more about features and control over your network | 15:30 |
| noonedeadpunk | if you need them of course | 15:30 |
| spatel | noonedeadpunk: we love performance over feature :) | 15:31 |
| noonedeadpunk | then sr-iov is your choice:) And if you have not much vms per node - it can be the solution | 15:32 |
| spatel | jamesdenton: that PPT is freaking awesome!!! | 15:32 |
| cjloader | when is freeze for train? | 15:32 |
| jrosser | oh my is the suse CI still broken | 15:33 |
| jamesdenton | thanks, spatel! That was from the Denver summit | 15:34 |
| spatel | jamesdenton: i would like to have more like that if you have :) | 15:35 |
| jamesdenton | naw, that's the only one. I unloaded my 100G NICs recently, too. Looking at getting some XL710s to test the SR-IOV trunking capabilities | 15:35 |
| jamesdenton | but no time for performance comparisons right now, unfortunately | 15:35 |
| noonedeadpunk | jrosser: I guess it's just recovered | 15:35 |
| cjloader | trying to see how long I have to get my PR in. | 15:37 |
| noonedeadpunk | since https://review.opendev.org/#/c/679904/ merged | 15:37 |
| spatel | jamesdenton: 100G man!! i can see your growth :) | 15:37 |
| cjloader | noonedeadpunk: do you know how long until freeze? | 15:37 |
| ioni | spatel, i tested myself ovs vs linuxbridge and what i can say is that linuxbridge sucks on latency and bandwidth | 15:38 |
| spatel | jamesdenton: when youe new book coming out with new cool networking hacks? | 15:38 |
| noonedeadpunk | cjloader: it's better to ask evrardjp but I'd say not less than a week | 15:38 |
| jamesdenton | haha | 15:38 |
| spatel | ioni: really? what kind of load you tested on ? | 15:39 |
| cjloader | evrardjp: do you know?? | 15:39 |
| noonedeadpunk | I hope you meant before next tag... | 15:39 |
| spatel | ioni: i have tried in lab with basic UDP (pps load) and both sucks... | 15:39 |
| cjloader | no I meant for official release | 15:39 |
| ioni | spatel, only tcp with 40G cards | 15:40 |
| cjloader | i'd really like to get my PR in | 15:40 |
| spatel | ioni: i did run test on 10G nic and found linuxbridge/OVS both not able to handle UDP traffic with hight pps rate ( in 30kpps it started dropping packets) | 15:41 |
| noonedeadpunk | ah, you're about train release:) just missed that message | 15:41 |
| noonedeadpunk | I'd say that stein freeze we did at the openstack release date | 15:42 |
| cjloader | ok | 15:42 |
| cjloader | so prob at least a month? | 15:42 |
| evrardjp | what's the question? | 15:43 |
| noonedeadpunk | when we're going to freeze train | 15:43 |
| noonedeadpunk | to branch actually | 15:43 |
| evrardjp | we don't really do freezes anymore since the change of policy for us | 15:44 |
| cjloader | i'm wwondering how long I have to get my inspector config automation in Train | 15:44 |
| evrardjp | I would expect it to be around m3 though, which should be this week-end of week | 15:44 |
| evrardjp | this week/end of next week | 15:44 |
| noonedeadpunk | wow.... time goes so fast... | 15:45 |
| cjloader | oh ok | 15:45 |
| evrardjp | after that we should stabilize for rc1, which is when we branch. It's supposed to be end of september | 15:45 |
| noonedeadpunk | and upgrades are not still ready... | 15:45 |
| cjloader | evrardjp: can you try to do it end of next week? | 15:45 |
| ioni | spatel, you should look at solarflare and openonload stuff :) | 15:45 |
| ioni | spatel, the latency is 4us with that | 15:46 |
| evrardjp | we've historically done that with 2 weeks of delay, because that's what we were allowed to | 15:46 |
| evrardjp | cjloader: what can I try to do for the end of next week? | 15:46 |
| spatel | ioni: let me google... | 15:46 |
| ioni | spatel, you can use openonload with a srv-io inside the vm | 15:46 |
| cjloader | hold freeze until next week lol | 15:46 |
| cjloader | so I can get a patch in | 15:46 |
| evrardjp | sure. I don't think ppl will mind. I am not the only holder of the keys here. | 15:46 |
| cjloader | okay, i'll def keep people informed | 15:47 |
| ioni | spatel, just get two cards and test them, i was amazed | 15:47 |
| cjloader | about my patch | 15:47 |
| evrardjp | keep in mind the changes in releases allows us some leeway after official release time | 15:47 |
| evrardjp | (allowing us to produce more rcs, and release later) | 15:48 |
| spatel | ioni: sure.. i would love to see | 15:48 |
| evrardjp | that's why we are trailing | 15:48 |
| cjloader | ok | 15:48 |
| evrardjp | but we should still produce at least a milestone afaik | 15:48 |
| cjloader | i'll aim to have patch merged by next friday | 15:48 |
| ioni | spatel, https://paste.xinu.at/sUpg/ | 15:48 |
| evrardjp | ok | 15:49 |
| evrardjp | we'll discuss this on next meeting anyway | 15:49 |
| cjloader | ok | 15:49 |
| cjloader | i'll attend | 15:49 |
| spatel | ioni: interesting... | 15:50 |
| spatel | This is 40G test right? can you provide card model? | 15:50 |
| ioni | spatel, well the nodes are 2x40 bond 802.3ad | 15:51 |
| ioni | spatel, but that was only a test with a single core | 15:51 |
| jamesdenton | Using Solarflare NICs? | 15:51 |
| ioni | yes | 15:51 |
| ioni | :) | 15:51 |
| spatel | oh!! got it.. | 15:51 |
| ioni | spatel, SFN8542 - $598/unit | 15:52 |
| spatel | never heard of that NIC so got confused | 15:52 |
| ioni | spatel, i think they got 10G as well | 15:52 |
| jamesdenton | so it's like, SolarFlare's answer to DPDK? Sorta? | 15:52 |
| ioni | jamesdenton, they do support dpdk as well | 15:52 |
| ioni | srv-io | 15:52 |
| ioni | 4096 | 15:53 |
| ioni | that got my attention in the first | 15:53 |
| spatel | DPDK is not related to hardware right, its software library.. am i missing something? | 15:53 |
| ioni | jamesdenton, i did try to run onload with qemu but failed and didn't work | 15:54 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_sahara master: Start using uWSGI role https://review.opendev.org/680452 | 15:54 |
| ioni | but with srv-io and onload in the vm, you can start your app with it and they say that works out of the box | 15:55 |
| cjloader | jrosser: ty for comments | 15:59 |
| jrosser | i think you missed some files? | 15:59 |
| cjloader | yeah I saw that | 15:59 |
| noonedeadpunk | Since it's WIP I didn't comment that at all) | 15:59 |
| cjloader | I dev'ed in Stein | 15:59 |
| noonedeadpunk | As well as these tasks are not included anywhere | 16:00 |
| *** SmearedBeard has joined #openstack-ansible | 16:00 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 16:00 | |
| cjloader | I know, I'm going to update with everything later today | 16:00 |
| cjloader | my apologies | 16:01 |
| *** altlogbot_0 has quit IRC | 16:01 | |
| jrosser | i figure there is also a companion patch to the integrated repo to acutally test some of this | 16:01 |
| *** altlogbot_0 has joined #openstack-ansible | 16:01 | |
| noonedeadpunk | np at asll | 16:02 |
| *** irclogbot_0 has quit IRC | 16:02 | |
| evrardjp | spatel: there are drivers based on nics | 16:03 |
| *** irclogbot_3 has joined #openstack-ansible | 16:03 | |
| evrardjp | spatel: https://doc.dpdk.org/guides/nics/index.html | 16:03 |
| cjloader | noonedeadpunk: wanted to get something out there to iterate on | 16:04 |
| spatel | evrardjp: oh!! now i can see.. thanks | 16:05 |
| noonedeadpunk | yep, that's good, and you've placed wip to show it:) we just can't take ourselvs in hands when we see smth to review :p | 16:05 |
| *** irclogbot_3 has quit IRC | 16:07 | |
| *** irclogbot_3 has joined #openstack-ansible | 16:07 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_manila master: Updated from OpenStack Ansible Tests https://review.opendev.org/680050 | 16:15 |
| *** kopecmartin is now known as kopecmartin|off | 16:18 | |
| *** ivve has quit IRC | 16:19 | |
| *** chandankumar is now known as raukadah | 16:33 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: Allow venv python interpreter to be overridden https://review.opendev.org/680460 | 16:34 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_keystone master: Allow venv python interpreter to be overridden https://review.opendev.org/680462 | 16:37 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Allow venv python interpreter to be overridden https://review.opendev.org/680466 | 16:41 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Allow venv python interpreter to be overridden https://review.opendev.org/680467 | 16:44 |
| raukadah | jrosser: ^^ we need to put these changes for os_tempest also? | 16:45 |
| noonedeadpunk | omg, what's going on with zuul | 16:46 |
| jrosser | if we take this as the pattern across all the OSA roles | 16:46 |
| jrosser | raukadah: but i think i just do the core services for now, make a WIP patch with a bunch of depends-on for openstack-ansible and see how spectacularly it breaks with just those doing py3 :) | 16:46 |
| noonedeadpunk | zuul has 338 patches for check, and some of them (like 576027) are waiting for over 18 hours | 16:48 |
| jrosser | raukadah: you see how i'm doing it with a per role variable, and then a global one on top | 16:48 |
| *** spsurya has quit IRC | 16:48 | |
| jrosser | so we can opt services in/out if something goes bad | 16:48 |
| raukadah | jrosser: I will take a look. | 16:49 |
| noonedeadpunk | seems that someone has proclaimed a week without gates | 16:52 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_placement master: Allow venv python interpreter to be overridden https://review.opendev.org/680469 | 16:58 |
| *** itandops has quit IRC | 17:00 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 17:03 |
| *** markvoelker has quit IRC | 17:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Updated from OpenStack Ansible Tests https://review.opendev.org/680040 | 17:05 |
| *** SmearedBeard has quit IRC | 17:10 | |
| *** markvoelker has joined #openstack-ansible | 17:13 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zun master: Updated from OpenStack Ansible Tests https://review.opendev.org/680062 | 17:17 |
| *** udesale has quit IRC | 17:18 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_sahara master: Start using uWSGI role https://review.opendev.org/680452 | 17:24 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_sahara master: Start using uWSGI role https://review.opendev.org/680452 | 17:27 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_cloudkitty master: Start using uWSGI role https://review.opendev.org/680479 | 17:31 |
| noonedeadpunk | So, the last patch for https://review.opendev.org/#/q/topic:ansible-role-uwsg just landed :) | 17:32 |
| noonedeadpunk | Just needs rechecks once gates are ok in terms of their load now... | 17:32 |
| noonedeadpunk | * https://review.opendev.org/#/q/topic:ansible-role-uwsgi | 17:33 |
| *** luksky has joined #openstack-ansible | 18:02 | |
| *** devx has quit IRC | 18:02 | |
| *** devx has joined #openstack-ansible | 18:04 | |
| cjloader | :q | 18:35 |
| guilhermesp | jrosser: maybe we can also add the mgmt bind in galera_client role and then modify that file? https://github.com/openstack/openstack-ansible-galera_client/blob/master/templates/client.my.cnf.j2 | 19:07 |
| guilhermesp | just wondering if the client config file needs to be exclusively localhost | 19:07 |
| *** trident has quit IRC | 19:09 | |
| cjloader | what time is OSA meeting tuesdays? | 19:13 |
| cjloader | i forget | 19:13 |
| vakuznet | review please https://review.opendev.org/#/c/657662/ | 19:15 |
| guilhermesp | cjloader: https://wiki.openstack.org/wiki/Meetings/openstack-ansible 16 UTC | 19:19 |
| *** trident has joined #openstack-ansible | 19:20 | |
| *** itandops has joined #openstack-ansible | 19:30 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_panko master: Start using uWSGI role https://review.opendev.org/680416 | 19:41 |
| spatel | noonedeadpunk: Question related rabbitMQ, if all 3 rabbitMQ nodes down in that how to bring up cluster? | 20:05 |
| spatel | Just start any node and then join slave one by one using cluster_join command? | 20:05 |
| noonedeadpunk | iirc they remeber state and cluster members | 20:06 |
| noonedeadpunk | so once you turn on rebbitmq node it will automatically join the cluster | 20:06 |
| spatel | hmm! i had some issue with my stein rabbitMQ, when i reboot it didn't join cluster and i did by hand with "reset" option | 20:07 |
| spatel | i believe that could be a bug myside | 20:07 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally master: Updated from OpenStack Ansible Tests https://review.opendev.org/680057 | 20:16 |
| noonedeadpunk | spatel: so every problems I ever had with rabbitmq I resolved by running playbook with setting rabbitmq_upgrade=true (or some similar var, cant look for exact name now) | 20:18 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Updated from OpenStack Ansible Tests https://review.opendev.org/680039 | 20:19 |
| spatel | rabbitmq_upgrade=true is this option means upgrade ? | 20:19 |
| spatel | I have "rabbitmq_upgrade: false" in user_variables.yml | 20:21 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Updated from OpenStack Ansible Tests https://review.opendev.org/680048 | 20:21 |
| spatel | noonedeadpunk: it must be "rabbitmq_ignore_version_state: false" | 20:21 |
| -spatel- # If the user does not want to upgrade but needs to rerun the playbooks for any reason the | 20:21 | |
| -spatel- # upgrade/version state can be ignored by setting `rabbitmq_ignore_version_state=true` | 20:21 | |
| spatel | I have that option to rabbitmq_ignore_version_state=false | 20:23 |
| noonedeadpunk | so https://opendev.org/openstack/openstack-ansible-rabbitmq_server/src/branch/master/defaults/main.yml#L42 | 20:24 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Updated from OpenStack Ansible Tests https://review.opendev.org/680058 | 20:25 |
| noonedeadpunk | hm, maybe rabbitmq_ignore_version_state will work as well. The point was in re-creation of queus | 20:25 |
| spatel | i have that option false | 20:25 |
| spatel | i think i need to set that to rabbitmq_ignore_version_state: true | 20:26 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Updated from OpenStack Ansible Tests https://review.opendev.org/680049 | 20:27 |
| openstackgerrit | Merged openstack/openstack-ansible-os_congress master: Updated from OpenStack Ansible Tests https://review.opendev.org/680041 | 20:28 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 20:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Updated from OpenStack Ansible Tests https://review.opendev.org/680042 | 20:37 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: Updated from OpenStack Ansible Tests https://review.opendev.org/680059 | 20:39 |
| *** ansmith has quit IRC | 20:39 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar master: Updated from OpenStack Ansible Tests https://review.opendev.org/680037 | 20:41 |
| *** spatel has quit IRC | 20:44 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Updated from OpenStack Ansible Tests https://review.opendev.org/680043 | 20:44 |
| *** SmearedBeard has joined #openstack-ansible | 20:46 | |
| *** itandops has quit IRC | 20:48 | |
| admin0 | hi guys .. if heat is killing my instances before cinder volumes are unmounted ( leaving cinder volumes in a state that needs db changes to fix ) , is it something wrong in the backend ? | 20:52 |
| *** itandops has joined #openstack-ansible | 21:00 | |
| *** markvoelker has quit IRC | 21:05 | |
| *** markvoelker has joined #openstack-ansible | 21:11 | |
| -openstackstatus- NOTICE: Gerrit is being restarted to pick up configuration changes. Should be quick. Sorry for the interruption. | 21:12 | |
| *** markvoelker has quit IRC | 21:15 | |
| openstackgerrit | Cam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: [WIP] Adds configuration and enables the Ironic Inspector https://review.opendev.org/680284 | 21:17 |
| cjloader | noonedeadpunk: jrosser ^ | 21:19 |
| openstackgerrit | Merged openstack/openstack-ansible-os_mistral master: Updated from OpenStack Ansible Tests https://review.opendev.org/680052 | 21:24 |
| noonedeadpunk | cjloader: btw don't you want to change year in your copyright? I guess it's 2019 now:) | 21:31 |
| noonedeadpunk | and I'd love to decrease number of files for ironic. As I'd say that pre install and post install of inspector can be merged with the main part | 21:33 |
| noonedeadpunk | As they have almost the same tasks but just different lists (like for creation of directories or placing config files) | 21:34 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Updated from OpenStack Ansible Tests https://review.opendev.org/680053 | 21:39 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_panko master: Start using uWSGI role https://review.opendev.org/680416 | 21:45 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_sahara master: Start using uWSGI role https://review.opendev.org/680452 | 21:48 |
| *** mrda has joined #openstack-ansible | 21:50 | |
| *** radeks has quit IRC | 21:54 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [WIP] Test python3 https://review.opendev.org/680470 | 21:59 |
| *** itandops has quit IRC | 22:00 | |
| *** luksky has quit IRC | 22:06 | |
| *** sum12 has quit IRC | 22:14 | |
| *** sum12 has joined #openstack-ansible | 22:16 | |
| cjloader | noonedeadpunk: looking better now? | 22:18 |
| cjloader | =) | 22:19 |
| noonedeadpunk | yep but there;s still smth to improve:) | 22:20 |
| *** prometheanfire has quit IRC | 22:25 | |
| *** prometheanfire has joined #openstack-ansible | 22:26 | |
| cjloader | is this something people have wanted? noonedeadpunk | 22:30 |
| cjloader | the inspector part | 22:30 |
| *** BjoernT_ has quit IRC | 22:35 | |
| noonedeadpunk | personally I never heard any requests regarding it | 22:35 |
| cjloader | i'l combine both pre/post playbooks | 22:35 |
| noonedeadpunk | Just want less code repeating:) | 22:39 |
| noonedeadpunk | wherever possible:) | 22:41 |
| *** SmearedBeard has quit IRC | 22:56 | |
| *** ansmith has joined #openstack-ansible | 23:07 | |
| *** macz has joined #openstack-ansible | 23:24 | |
| *** macz has quit IRC | 23:26 | |
| *** macz has joined #openstack-ansible | 23:27 | |
| *** macz has quit IRC | 23:29 | |
| *** threestrands has joined #openstack-ansible | 23:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
