Tuesday, 2019-02-12

*** partlycloudy has joined #openstack-ansible00:07
*** Emine has joined #openstack-ansible00:15
*** errr_ has joined #openstack-ansible00:15
*** errr has quit IRC00:18
*** tosky has quit IRC00:18
*** TxGirlGeek has quit IRC00:25
*** Emine has quit IRC00:27
*** macza has joined #openstack-ansible00:29
*** macza has quit IRC00:34
*** chhagarw has joined #openstack-ansible00:37
admin0queens -> rocky  18.1.3 ubuntu-xenial :  Could not find a version that satisfies the requirement cryptography (from versions: ) -- in repo build step --   is this a known issue ?00:40
*** hwoarang_ has quit IRC00:48
*** chhagarw has quit IRC00:51
*** hwoarang has joined #openstack-ansible00:53
*** gyee has quit IRC00:54
*** mrhillsman has joined #openstack-ansible00:59
*** fyx has joined #openstack-ansible00:59
*** preece has joined #openstack-ansible00:59
*** FrankZhang has joined #openstack-ansible00:59
*** sdake has joined #openstack-ansible01:01
*** DanyC has quit IRC01:04
*** macza has joined #openstack-ansible01:08
*** hwoarang has quit IRC01:10
cloudnulladmin0 not that I'm aware of01:10
*** hwoarang has joined #openstack-ansible01:11
admin0i am getting it .. so kind of stuck at the failed repo-build process01:13
*** macza has quit IRC01:13
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update overlay inventory to resovle circular dep  https://review.openstack.org/63620401:15
*** mhayden has joined #openstack-ansible01:25
*** markvoelker has quit IRC01:30
*** markvoelker has joined #openstack-ansible01:31
*** markvoelker has quit IRC01:35
admin0queens -> rocky 18.1.3 -- requirements_absolute_requirements.txt 404 Not Found .. i thought this was already solved when it was reported back on 18.1.101:36
*** sdake has quit IRC01:37
*** sdake has joined #openstack-ansible01:40
*** sdake has quit IRC01:43
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Update overlay inventory to resovle circular dep  https://review.openstack.org/63620401:43
*** sdake has joined #openstack-ansible01:51
*** sdake has quit IRC01:51
cloudnulladmin0 if you curl the file is it intermittently found ?02:02
*** TxGirlGeek has joined #openstack-ansible02:04
admin0 never found ..02:04
admin0so its not a repo our of sync02:04
admin0not in the repo at all .. thought the repo build gave no errors02:04
openstackgerritMerged openstack/openstack-ansible-ops master: Update overlay inventory to resovle circular dep  https://review.openstack.org/63620402:13
*** macza has joined #openstack-ansible02:22
*** macza has quit IRC02:27
*** macza has joined #openstack-ansible02:27
*** macza has quit IRC02:31
*** markvoelker has joined #openstack-ansible02:31
*** TxGirlGeek has quit IRC02:36
openstackgerritGuilherme  Steinmuller Pimentel proposed openstack/openstack-ansible-os_heat master: Add heat user to heat domain admin role  https://review.openstack.org/63403202:38
*** hwoarang has quit IRC02:48
*** hwoarang has joined #openstack-ansible02:56
*** DanyC has joined #openstack-ansible03:05
*** markvoelker has quit IRC03:05
*** DanyC has quit IRC03:09
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add the ability to enable or disable rollups / indexes  https://review.openstack.org/63609003:12
cloudnulladmin0 sorry was having dinner.03:12
cloudnullif the repo is out of sync can you check the lsync process in the repo server?03:13
cloudnullit should be running but if it died then that could cuase it to become out of sync03:13
*** fnpanic has quit IRC03:30
*** sdake has joined #openstack-ansible03:39
*** hwoarang has quit IRC03:40
*** hwoarang has joined #openstack-ansible03:45
*** TxGirlGeek has joined #openstack-ansible03:57
*** dave-mccowan has quit IRC03:58
*** sdake has quit IRC03:59
*** markvoelker has joined #openstack-ansible04:02
*** udesale has joined #openstack-ansible04:02
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add the ability to enable or disable rollups / indexes  https://review.openstack.org/63609004:07
*** hwoarang has quit IRC04:30
*** hwoarang has joined #openstack-ansible04:31
*** markvoelker has quit IRC04:34
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add the ability to enable or disable rollups / indexes  https://review.openstack.org/63609004:50
*** shyamb has joined #openstack-ansible05:03
*** shyamb has quit IRC05:15
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add the ability to enable or disable rollups / indexes  https://review.openstack.org/63609005:15
openstackgerritKevin Carter (cloudnull) proposed openstack/ansible-role-python_venv_build master: Add venv build target hosts data structure  https://review.openstack.org/63619205:17
openstackgerritKevin Carter (cloudnull) proposed openstack/ansible-role-python_venv_build master: Add venv build target hosts data structure  https://review.openstack.org/63619205:24
*** ArchiFleKs has quit IRC05:31
*** markvoelker has joined #openstack-ansible05:32
*** ArchiFleKs has joined #openstack-ansible05:37
*** udesale has quit IRC05:37
*** hwoarang has quit IRC05:42
*** hwoarang has joined #openstack-ansible05:44
*** chandankumar is now known as chkumar|ruck05:47
chkumar|ruckguilhermesp: Thanks!05:48
*** TxGirlGeek has quit IRC05:56
*** markvoelker has quit IRC06:05
openstackgerritMerged openstack/openstack-ansible-ops master: Add the ability to enable or disable rollups / indexes  https://review.openstack.org/63609006:33
*** markvoelker has joined #openstack-ansible07:02
*** kopecmartin|off is now known as kopecmartin07:05
*** udesale has joined #openstack-ansible07:09
errr_odyssey4me: ping07:29
*** errr_ is now known as errr07:29
*** Emine has joined #openstack-ansible07:32
*** markvoelker has quit IRC07:35
*** fnpanic has joined #openstack-ansible07:36
fnpanicGood morning07:36
fnpanici am hitting the ceph-client package downgrade bug with rocky 18.1.2-1607:37
fnpanicis there a fix yet?07:37
fnpanici have seen the discussion yesterday07:37
fnpanichttp://paste.openstack.org/show/744906/07:38
*** yetiszaf has joined #openstack-ansible07:40
*** Emine has quit IRC07:49
fnpanicseems everybody is still asleep :-)07:49
*** gkadam has joined #openstack-ansible08:08
*** gkadam_ has joined #openstack-ansible08:15
*** gkadam has quit IRC08:17
*** markvoelker has joined #openstack-ansible08:32
*** tosky has joined #openstack-ansible08:40
*** mathlin has quit IRC08:41
*** priteau has joined #openstack-ansible08:58
*** markvoelker has quit IRC09:05
*** Emine has joined #openstack-ansible09:15
*** DanyC has joined #openstack-ansible09:20
*** shardy has joined #openstack-ansible09:21
CeeMacmorning09:24
fnpanichi09:24
*** Emine has quit IRC09:26
*** Emine has joined #openstack-ansible09:26
openstackgerritJonathan Rosser proposed openstack/openstack-ansible stable/rocky: [DNM] test patch  https://review.openstack.org/63628809:27
jrosserfnpanic: ^ lets see if ceph/bionic works on rocky CI09:28
*** mathlin has joined #openstack-ansible09:28
fnpanicok :-)09:28
fnpanichttps://bugs.launchpad.net/openstack-ansible/+bug/181557009:32
openstackLaunchpad bug 1815570 in openstack-ansible "cinder fails because of trying to downgrade librados2 librbd1" [Undecided,New]09:32
CeeMacfnpanic, i this is the issue you are having: https://bugs.launchpad.net/openstack-ansible/+bug/1814909 from our discussion yesterday09:37
openstackLaunchpad bug 1814909 in openstack-ansible "Ssl verify error in heat -> keystone communication" [Undecided,New]09:37
*** aedc has joined #openstack-ansible09:44
fnpanicCeeMac: yes this looks like my problem09:45
CeeMaccame across it while i was trying to troubleshoot my issue.  I don't get the same errors as is reported there but thought it might be helpful for you09:46
fnpanic:-)09:46
fnpanicit is09:46
fnpanicthanks09:46
CeeMacno problem :)09:46
jrosserthis still all feels ugly, that the internal API are expecting to talk to the external endpoint09:48
jrosseri don't feel that is correct09:49
jrosserthis is the change responsible for the currentl behaviour https://github.com/openstack/openstack-ansible-os_heat/commit/785fcfd33d29ddfee54f09cd6bf126990d64e4dd09:50
fnpanicnot sure09:51
fnpanicif going with the internal api is the correct solution09:52
CeeMacha! I've fixed my problem too :D09:57
CeeMacwasn't ssl issue, network firewall was blocking traffic from the br-mgmt network to the dmz network the HAproxy external interface is on09:57
CeeMacthat does seem a bit unnecessary, going external from an internal call.....09:58
CeeMacthat only seemed to occur when I enabled a public ssl cert on the haproxy though, curious09:58
*** markvoelker has joined #openstack-ansible10:02
*** shyamb has joined #openstack-ansible10:03
jrosserCeeMac: there are a bunch of things like this https://github.com/openstack/openstack-ansible-os_heat/blob/785fcfd33d29ddfee54f09cd6bf126990d64e4dd/templates/heat.conf.j2#L8110:07
jrosserCeeMac: and you can see how that is driven here http://git.openstack.org/cgit/openstack/openstack-ansible/tree/inventory/group_vars/all/keystone.yml#n3010:08
jrosserif you supply your own cert the assumption is that it is properly trustworthy10:08
CeeMacit is, i just wasn't expecting the change in pathing to the resource10:09
fnpanicso when i deploy a private signed cert the REQUESTS_CA_BUNDLE should be set10:11
fnpanicbecause in my case for example it is an internal cert and needs to be trusted a private ca cert10:12
CeeMacsounds about right fnpanic10:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Mark OSA version in the wrapper script  https://review.openstack.org/63376210:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Use an env lookup to determine the OSA version  https://review.openstack.org/63376310:16
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_nova master: Cleanup files and templates using smart sources  https://review.openstack.org/58895110:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Update the heat service egg name to openstack_heat  https://review.openstack.org/63629710:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_heat master: Fixed the egg name of heat to openstack_heat  https://review.openstack.org/63551810:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_heat master: [DNM] heat tempest tests finding  https://review.openstack.org/63069410:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Use the config_template module from the dedicated repo  https://review.openstack.org/63618210:26
chkumar|ruckodyssey4me: jrosser I will be not around on 13th and 14th, will see on friday directly10:28
errrodyssey4me: there?10:32
odyssey4mechkumar|ruck ok, business or pleasure?10:33
odyssey4meerrr yep10:33
chkumar|ruckodyssey4me: to attend friend's marriage :-)10:33
odyssey4mechkumar|ruck ah, nice - have a good time!10:33
chkumar|ruckodyssey4me: thanks :-)10:33
*** shyamb has quit IRC10:34
*** markvoelker has quit IRC10:35
*** udesale has quit IRC10:49
CeeMachave any of you come across the issue where member or _member_ users are unable to create an instance through horizon because of "unable to retrieve the volumes" ?11:07
CeeMacI've seen a bug that seems to fit https://bugs.launchpad.net/tripleo/+bug/1797811 but its in tripleo and related to queens, fixed for stein11:08
openstackLaunchpad bug 1797811 in tripleo "Users with _member_ or member roles cannot use cinder-volume via Horizon Dashboard" [High,Triaged]11:08
*** Emine has quit IRC11:09
*** Emine has joined #openstack-ansible11:10
CeeMachmmm, followed fix from https://lists.rdoproject.org/pipermail/users/2018-October/000375.html seems to have resolved it11:24
CeeMaccan't find any bugs on OSA related to this11:26
odyssey4meCeeMac it seems that we have a fix for that merged: https://github.com/openstack/openstack-ansible-os_cinder/commit/a4f0db735de5ad1ae5130cdc8ea32ff60647339911:32
*** markvoelker has joined #openstack-ansible11:32
odyssey4meCeeMac and that should be in the latest release of rocky11:33
CeeMacoh, thanks odyssey4me , I couldn't find anything on launchpad11:33
CeeMaclinled osa project to the bug i posted earlier, just in case11:33
odyssey4meCeeMac yeah, our fixes don't always have bugs attached11:33
CeeMacthat'll be why :)11:33
admin0hi all .. queens -> rocky 18.1.3, i am getting this :8181/os-releases/18.1.3/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt not found on all repos .. i recall similar during -> 18.1.1 and a patch ..and thought it was already fixed11:35
odyssey4meadmin0 if that file isn't found, then repo build has not been run11:35
admin0it runs, but gives all green11:36
admin0nuke all 3 and rebuild ?11:36
odyssey4meadmin0 can you check that all repo servers have the file?11:36
admin0none have11:37
CeeMacodyssey4me, is there a cherrypick url i can use for this?11:38
odyssey4meadmin0 none at all? that's odd - does the /os-releases/18.1.3 directory exist, and if so, what's in it?11:40
odyssey4meCeeMac just update your openstack-ansible checkout to 18.1.3 and run bootstrap-ansible.sh11:40
odyssey4meCeeMac cd /opt/openstack-ansible; git fetch --all; git checkout 18.1.3; ./scripts/bootstrap-ansible.sh11:41
CeeMachaha, thanks, i was struggling there11:41
odyssey4meCeeMac once that's done, run each of the main playbooks to perform the minor upgrade11:46
CeeMacas in hosts, infrastructure, openstack ones?11:47
odyssey4mecd /opt/openstack-ansible/playbooks; openstack-ansible setup-hosts.yml setup-infrastructure.yml setup-openstack.yml11:47
odyssey4meThat will perform the minor upgrade completely.11:47
CeeMacsure. thanks11:47
CeeMachopefully I've wrapped all of my other changes into my configs already11:49
odyssey4meheh, yeah - hopefully :)11:49
CeeMacwould this affect any cherry-pick fixes I've pulled down already?  if they aren't rolled up in this that is11:49
odyssey4meI don't know - it depends on what you've cherry-picked? do you have a list?11:50
odyssey4meHopefully everything you've picked down is included in the release.11:50
CeeMacI can dig them out of my notes11:50
CeeMactheres only a couple i think11:50
CeeMacfirst one was https://review.openstack.org/#/c/632505/11:53
fnpanichttps://review.openstack.org/#/c/636115/11:55
fnpanicthe problem which was fixed here in cinder also applies to nova when using ceph....11:55
fnpanicdamn11:55
CeeMacother one was https://review.openstack.org/#/c/630747/11:56
CeeMacthink it was just those two11:56
CeeMacoh, i reverted that last one as it was against master and it caused me issues, did a manual fix instead11:57
CeeMacso just the one then11:57
openstackgerritMerged openstack/openstack-ansible master: Bump SHAs for master  https://review.openstack.org/63600211:59
odyssey4meCeeMac https://review.openstack.org/632505 is included in 18.1.3 - checking the other one12:00
odyssey4meCeeMac unfortunately https://review.openstack.org/630747 is not included, so if you need that, I'd recommend using the stable/rocky checkout instead of 18.1.312:03
odyssey4meThe patch will be in the 18.1.4 release whenever that happens12:03
*** markvoelker has quit IRC12:05
jrosserodyssey4me: we're running an upgrade to 18.1.3 here right now and see similar requirements_absolute_requirements.txt missing on all but the first repo server12:07
CeeMacthanks odyssey4me, i should be fine with just https://review.openstack.org/63250512:07
jrossernot looked into why yet (need to complete the upgrade first) so just marked two of the repo servers as down in haproxy12:08
odyssey4mejrosser hmm, that sounds like some sort of lsync issue again12:08
jrosseryes, seen this before last minor upgrade iirc12:09
odyssey4mejrosser it's plausible that there may be an issue with too many open files again - IIRC lsync keeps an open tracking file for every single file in the repo :/12:11
jrosserblimey, right12:12
odyssey4meI'm quite honestly thinking that it may be better to just change the repository to have a primary up, and the others as a backup... and then just use a basic rsync cron job to sync the content.12:12
odyssey4meThe alternative, of course, is just to use a shared file system across them all - NFS or whatever.12:13
fnpanici would say +1 for the cronjob :-)12:20
*** emine__ has joined #openstack-ansible12:28
*** Emine has quit IRC12:30
CeeMac+1 for shared storage for me :)12:31
CeeMacbut i guess that adds another layer of potential issues for HA if the shared storage goes down/inaccessible12:32
fnpanicceemac: this is my fear. a/p is straight forward12:33
fnpaniceven you could easily disable the cornjob before the upgrade and keep a old copy of the repo12:34
CeeMacyeah, there is definitely something attractive in the simplicity of local a/p deployment12:34
CeeMacand simple == good12:35
fnpanic:-)12:35
fnpanicless headache12:35
CeeMacfor sure12:35
CeeMacok, I'm sold. +1 for cronjob here too :D12:35
fnpanichehe12:35
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Update the heat service egg name to openstack_heat  https://review.openstack.org/63629712:43
odyssey4mejrosser evrardjp the first part of the config_template change over is ready: https://review.openstack.org/#/q/topic:bug/179125812:45
jrosserhas anyone seen rabbitmq get upset like this? http://paste.openstack.org/show/744926/12:46
odyssey4methose take it to use the config_template repo, and remove the old plugin - the next stage is to implement the meta-dep in all the roles, then we should be able to remove the library/action plugin config12:46
jrosserit appears to be otherwise normal and working with no errors in the log12:46
evrardjpinteresting odyssey4me I would have thought that this would have failed without the depends on: https://review.openstack.org/#/c/635841/212:46
odyssey4meevrardjp no need really, because the new plugin is first in line12:47
odyssey4methe one to remove the old plugin actually depends on that one12:47
evrardjpyeah I see, but that's the issue I had before12:48
evrardjpWhen both were there, ansible started to complain about overlaps, where here it doesn't :p12:48
odyssey4meheh, I'll consider myself lucky then ;)12:49
odyssey4mewe would already have issues for that though, because ceph-ansible also has config_template12:49
*** mmercer has quit IRC12:49
*** emine__ has quit IRC12:50
*** mmercer has joined #openstack-ansible12:51
evrardjpthat was the problem I had before :p12:51
chkumar|ruckodyssey4me: ceph copies the config_template script in their repo, i think we need to convince them to switch to native action pluign from the repo12:51
evrardjpI think Ansible is better now :)12:51
evrardjpanyway12:51
evrardjpthat will be easier :)12:51
chkumar|ruckevrardjp: odyssey4me is it possible to make it as a part of ansible ?12:52
evrardjpchkumar|ruck: nope -- We tried :)12:53
odyssey4mechkumar|ruck we tried, and they rejected it, stating that everything it does can be done using existing filters...12:53
chkumar|ruckhmm12:53
evrardjpwhich is true12:53
evrardjpthen I discussed it again, to explain on a different case, due to the addition of more modules that are closer12:53
evrardjpso we could have done something like ini-config_template, json-config_template, and yaml-config_template, and had to be strict into what we passed into it. So at the end we didn't do it.12:54
evrardjpIt works "good enough" for us. It's not perfect, but it's super flexible12:54
evrardjpchkumar|ruck: Now you can use it in your role meta deps and you get it for free12:54
chkumar|ruckevrardjp: just config_template where one format can coverted it to any format12:54
*** mkuf_ has joined #openstack-ansible12:55
chkumar|ruckjust like pandoc do coverting to various format12:55
odyssey4mechkumar|ruck you can read the sordid details here: https://github.com/ansible/ansible/pull/1255512:55
evrardjpodyssey4me: there is another PR for it too :p12:55
evrardjphttps://github.com/ansible/ansible/pull/3545312:56
odyssey4meevrardjp yeah, I see it referenced there12:57
evrardjpsadly it doesn't show the community meeting conversation12:57
evrardjpanywya12:57
*** mkuf has quit IRC12:58
*** markvoelker has joined #openstack-ansible13:02
*** aedc has quit IRC13:03
*** partlycloudy has left #openstack-ansible13:03
*** aedc has joined #openstack-ansible13:04
guilhermespmornings!13:04
guilhermespchkumar|ruck: I was debugging yesterday that one https://review.openstack.org/#/c/627782/6 I have a strong feeling that the tempest workspace move is not happy13:05
guilhermespit's like it moves only the pointer of the workspace and created an empty folder.13:05
guilhermespfor example, the old workspace was in <venv>/workspace with some sub-dirs13:06
guilhermespand it is still there... and the new one is in the new location but empty13:06
*** udesale has joined #openstack-ansible13:10
*** chkumar|ruck is now known as chkumar|pto13:10
mbuilcould anybody have a look at https://review.openstack.org/#/c/622216/ please?13:10
*** partlycloudy_ has joined #openstack-ansible13:13
*** partlycloudy_ has quit IRC13:15
evrardjpguilhermesp: morning!13:16
*** partlycloudy has joined #openstack-ansible13:16
jamesdentonmbuil one sec13:16
guilhermespevrardjp: o/13:16
jamesdentonmbuil Just one note about the release notes13:21
mbuiljamesdenton: ok :)13:24
jrosserjamesdenton: thoughts on this? https://review.openstack.org/#/c/636211/13:31
*** sdake has joined #openstack-ansible13:31
jamesdentonI kinda prefer the idea of building out the appropriate bridges to provide that connectivity.13:33
*** markvoelker has quit IRC13:35
admin0nuked all repos, rebuilt .. now its moving further ..13:35
jrosserjamesdenton: relatedly https://review.openstack.org/#/c/636162/13:35
jamesdentonthere's one more in flight, i think.13:36
*** mkuf_ has quit IRC13:37
jamesdentonvollman mentioned mimicing the bridge build-out that os_nova uses13:37
*** openstackgerrit has quit IRC13:37
jrosseri'm slightly bothered that https://review.openstack.org/#/c/636162/ has other code paths which will not add the forwarding13:37
admin0after all is working . can the older vnevs be removed ?13:38
admin0is there a ansible for it, or is it a rm -rf ?13:38
vollmanjamesdenton: I abandoned that change in favor of https://review.openstack.org/#/c/636211/13:39
jamesdentonoh alright13:39
*** openstackgerrit has joined #openstack-ansible13:49
openstackgerritMichael Vollman proposed openstack/openstack-ansible-os_nova master: Avoid distro installing unused services  https://review.openstack.org/63327513:49
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron master: Use the new services names for sfc  https://review.openstack.org/62221613:58
*** mkuf_ has joined #openstack-ansible14:01
*** sdake has quit IRC14:03
openstackgerritMerged openstack/openstack-ansible-os_tempest master: Add tempest_service_available_mistral with distro packages  https://review.openstack.org/63518014:05
openstackgerritMichael Vollman proposed openstack/openstack-ansible-os_neutron master: Disable tempest router ping  https://review.openstack.org/63633814:06
*** zul has joined #openstack-ansible14:07
*** asettle has joined #openstack-ansible14:09
openstackgerritMerged openstack/openstack-ansible master: Mark OSA version in the wrapper script  https://review.openstack.org/63376214:13
*** shertel has joined #openstack-ansible14:23
*** priteau has quit IRC14:26
openstackgerritMerged openstack/openstack-ansible-tests master: Use the config_template module from the dedicated repo  https://review.openstack.org/63584114:28
shertelmnsaser, odyssey4me, evrardjp, cloudnull: Hi, can I get a review on https://github.com/ansible/ansible/pull/51827? Adding a path option so to send logs to a file like is done here https://docs.openstack.org/openstacksdk/latest/user/guides/logging.html is probably preferential but as features aren't backported this would be an intermediate fix.14:29
odyssey4mejrosser any chance for a review on https://review.openstack.org/635838 ?14:30
*** partlycloudy has quit IRC14:31
odyssey4meshertel I'm tied up with other work right now. Will try and take a look later.14:31
shertelOkay, thanks odyssey4me.14:31
*** markvoelker has joined #openstack-ansible14:32
*** kmadac has joined #openstack-ansible14:32
*** ArchiFleKs has quit IRC14:34
*** sdake has joined #openstack-ansible14:35
*** ironfoot_ is now known as ironfoot14:35
*** sdake has quit IRC14:37
*** sdake_ has joined #openstack-ansible14:37
jrosserodyssey4me: done - we should check parity of the tests between the old and new code too14:48
evrardjpshertel: I am between two meetings but I can try having a look today14:48
shertelevrardjp Thanks!14:49
*** ArchiFleKs has joined #openstack-ansible14:49
fnpanichi14:49
fnpanicCeeMac: just wanted to confirm14:50
fnpanichttps://bugs.launchpad.net/openstack-ansible/+bug/181490914:50
openstackLaunchpad bug 1814909 in openstack-ansible "Ssl verify error in heat -> keystone communication" [Undecided,New]14:50
fnpanicthis workaround fixed the heat ssl error14:50
fnpanicthanks14:50
odyssey4mejrosser we should - I've not got the time to do it though, my focus is just to do the switch14:50
CeeMacgood to know fnpanic14:50
CeeMacglad i was able to help :)14:50
jrosserodyssey4me: create a bug for it - you never know someone might look at it tomorrow :)14:51
fnpanici have a question. looks like the orchestation tab is missing under admin - system - system information14:51
fnpaniccould this be that horizon also is missing the cert?14:51
CeeMacyeah, i dont have one of those either.14:51
CeeMacis that supposed to be there?14:51
fnpanicyes?14:51
*** udesale has quit IRC14:52
CeeMacdont recall ever seeing it, but then I'm new to the game :)14:52
fnpanichold a second, looking at my pike14:52
fnpanicin pike it is there14:53
fnpanicyes14:53
CeeMaccould be one of those things that was changed?14:53
fnpaniclike this14:54
fnpanichttps://snag.gy/Hh3YkZ.jpg14:54
*** gkadam_ has quit IRC14:54
*** gkadam has joined #openstack-ansible14:54
fnpaniccan someone confirm?14:54
fnpanicnot that i am searching for something which is not supposed to be there :-)14:54
*** mathlin has quit IRC14:55
odyssey4mejrosser done: https://bugs.launchpad.net/openstack-ansible/+bug/181562014:55
openstackLaunchpad bug 1815620 in openstack-ansible "Improve testing for config_template" [Undecided,New]14:55
odyssey4mefnpanic CeeMac as I understand it, this all relates back to the change in the config for the service which makes the client use the public endpoint - so to make it work in horizon you'd also need to deploy the appropriate CA there, or disable cert verification14:56
openstackgerritMerged openstack/openstack-ansible master: Remove an extra comment about Tempest settings  https://review.openstack.org/63614814:57
fnpanicok so i need to put it in the horizon container14:57
fnpanicok14:57
fnpanicwill do and report back14:57
CeeMacodyssey4me, i'm using public trusted cert, shouldn't be an issue?14:58
CeeMaci've updated the FW rule that was blocking me earlier so the full br-mgmt network can speak to the public endpoint14:59
odyssey4meCeeMac I honestly have no clue. You may have to enable some horizon debug logs to figure out what's going on there.14:59
CeeMacyeah14:59
CeeMaci'll see what fnpanic comes back with14:59
fnpanicmhhh14:59
CeeMacjust seems weird that project | orchestration would work but admin | system | system info wouldn't14:59
fnpanicdo i need to alter the glance cors to make it work from horizon?15:00
fnpanicUnable to create the image.15:00
fnpanicin horizon15:00
fnpanic?15:00
fnpanici remember that for pike there were changes required15:00
fnpanici think setting horizon_images_upload_mode: legacy15:01
fnpanicis not a good idea right?15:02
*** udesale has joined #openstack-ansible15:02
CeeMaci had to set horizon_images_upload_mode: remote to get it to work15:03
CeeMacnot sure yet what was stopping it working in direct mode15:04
fnpanici will give it a try15:05
fnpanicthanks15:05
fnpanicchecked the cacert.pem in horizon container15:05
fnpanicit is there but a different file15:05
*** markvoelker has quit IRC15:05
fnpaniccompared the md5sums....15:05
fnpanicstrange15:05
*** ianychoi has quit IRC15:07
*** ianychoi has joined #openstack-ansible15:07
CeeMaci'll have a play around on thursday, don't want to make many more changes while its (mostly) working15:08
CeeMacBoss has a demo with the board tomorrow so I don't want to risk breaking it :|15:09
*** rgogunskiy has quit IRC15:09
*** sreejithp has joined #openstack-ansible15:11
openstackgerritMerged openstack/openstack-ansible master: nspawn: use correct network range for tempest  https://review.openstack.org/63505915:11
admin0how do i target just compute nodes in -l in osa ?15:17
admin0in limit15:17
admin0ceilometer-agent-compute is the only group i see for them..15:18
fnpanicCeeMac: then stay away from the running system ;-)15:18
CeeMacfnpanic, exactly my plan!15:18
fnpanichehe15:18
fnpanicbut what could possibly go wrong?15:18
CeeMacwhat indeed >_>15:18
fnpanicodyssey4me: i added the cert and restarted the container15:19
fnpanicthe orchestration tab is not there15:19
CeeMacadmin0, is it not --limit compute_hosts15:19
fnpanicCeeMac: add the direct upload method but still the cors error15:20
fnpanic:-(15:21
CeeMacyou can run your playbook with that and --list-hosts to verify?15:21
CeeMacfnpanic, you need to add remote15:21
fnpanicsorry i mean remote15:21
CeeMacoh15:21
CeeMacboo15:21
fnpanicalso did a os-horizon run15:21
fnpanicdamn15:21
openstackgerritJames Denton proposed openstack/openstack-ansible-tests master: Restart dnsmasq after clearing iptables  https://review.openstack.org/63635215:22
fnpanichorizon_images_upload_mode: direct15:22
fnpanicthis is it what i set15:22
CeeMacah15:22
CeeMachorizon_images_upload_mode: remote15:23
*** rpsene has joined #openstack-ansible15:23
CeeMacis what it set15:23
CeeMac*i iset15:23
CeeMac*i set15:23
* CeeMac can't type today15:23
*** TxGirlGeek has joined #openstack-ansible15:24
fnpanici will take a look at local_settings.py15:24
odyssey4mecores, could we get some eyes on https://review.openstack.org/#/q/topic:egg_fix please to unblock heat for master15:25
rpseneguys, we are trying to get openstack-ansible running on IBM Power, ppc64le and it is getting an error15:25
rpseneASK [galera_server : Prepare the package list] ********************************15:25
rpsenefatal: [infra1_galera_container-c9dc3590]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'ansible_architecture'\n\nThe error appears to have been in '/etc/ansible/roles/galera_server/tasks/galera_install.yml': line 16, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Prepare the15:25
rpsene package list\n  ^ here\n"}15:25
odyssey4meadmin0 I think the group name is nova_compute? you can see all groups using inventory-manage15:26
fnpanicHORIZON_IMAGES_UPLOAD_MODE = 'direct'15:26
odyssey4meadmin0 oh yes, I think CeeMac is right15:26
*** lcnzg has joined #openstack-ansible15:26
fnpanicit is set in the config15:26
admin0inventroy-manage -l | grep  <hostname of compute> gave me only that one group15:27
admin0compute_hosts worked fine15:27
CeeMaccool15:27
CeeMacinventory-manage.py -G15:27
CeeMacshows groups and member hosts15:27
jamesdentonrpsene This is on Ubuntu?15:27
rpsene@jamesdenton yes15:28
admin0does rocky support ovs completely ( like for br-mgmt and br-storage) as well ?15:28
odyssey4merpsene hmm, could you register a bug for that and put the verbose log into a paste service (something like https://pasted.tech)15:28
admin0anyone has a config i can peek :)15:28
odyssey4merpsene hmm, if ansible_architecture is missing that'll break OSA in all kinds of ways... which version of OSA are you trying to deploy?15:29
odyssey4merpsene typically that's a fact gathered by ansible15:29
*** priteau has joined #openstack-ansible15:30
fnpanicceemac: any idea with the glance upload?15:31
CeeMacfnpanic, switch out 'direct' for 'remote'15:31
fnpanichave you changed your cors settings?15:31
jrosserheat requiring openmpi. goodness me15:31
odyssey4mejrosser yeah, sucks a bit - and I'd like to move that into a devel package list, but that can come later15:32
CeeMacfnpanic, thats the only thing i changed15:32
fnpanicok i try15:33
CeeMack15:33
openstackgerritMichael Vollman proposed openstack/openstack-ansible-os_neutron master: Disable tempest router ping  https://review.openstack.org/63633815:34
dmsimardI found the issue that made horizon not able to query cinder and I'm not happy :(15:35
openstackgerritMerged openstack/openstack-ansible-plugins master: Remove the config_template module  https://review.openstack.org/63583815:36
dmsimardBefore instanciating an (insecure) cinder API client, horizon uses cinderclient to get the microversions: https://github.com/openstack/horizon/blob/d5b7feb5d4bf622905d717cd20fc83fd136c8a8c/openstack_dashboard/api/cinder.py#L26315:37
dmsimardBut when you look, cinderclient doesn't use insecure or verify=false so it fails: https://github.com/openstack/python-cinderclient/blob/63b36a901bfaf2508a9c3cda1d8dafb8769f2340/cinderclient/client.py#L75-L10915:38
dmsimardand then everything else cinder related in horizon fails :(15:38
*** macza has joined #openstack-ansible15:42
fnpanicjust a question15:43
fnpanicshould this not be the default for every container / host?15:43
fnpanicREQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.cr15:43
fnpanic+t15:43
fnpanicthis would make life a lot easier15:44
dmsimardfnpanic: that file is unique per container right now but that's not a bad idea15:48
jrosserwe'd have to check that was valid on all the distros, and adjust to suit if not15:48
fnpanicceemac: worked! thanks! this should be in the docs for the examples.15:48
spotzFor those who might be interested. The next OPS meetup will be back in Berlin on March 6-7, information can be found here - https://etherpad.openstack.org/p/BER-ops-meetup15:49
fnpanicdmsimard: i am looking at the docs to find if this could have negative impact. systemd can be a #*#+*= somtimes15:50
fnpanicspotz: thanks for the info15:51
dmsimardfnpanic: I sort of hacked something together to send the self signed certificate to all the containers and update the ca-cert bundle -- but it didn't seem like requests was using it15:52
spotzfnpanic My pleasure:)15:52
dmsimardso this would be an envvar set by the systemd service ?15:52
fnpanicdmsimard: you only did update-ca-certificates right?15:53
*** aedc has quit IRC15:53
dmsimardfnpanic: yeah, didn't think about the envvar15:56
dmsimardI'll try that now15:56
kmadac@fnpanic: i'm author of that bug with heat and certificate. If interested here is patch to osa stable/rocky we used in our deployment: http://paste.openstack.org/show/744947/15:58
CeeMacfnpanic, glad that worked for you too :D15:59
mnasercloudnull, DimGR, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum, darrenc, JRobinson__, colinmcnamara, thorst, adreznec, eil397,16:00
mnaserqwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, mnaser, nicolasbock, jrosser, cjloader, antonym, dcdamien, jamesdenton16:00
mnaser~meeting time~16:00
mnaser#startmeeting openstack_ansible_meeting16:00
openstackMeeting started Tue Feb 12 16:00:46 2019 UTC and is due to finish in 60 minutes.  The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
mnaser#topic rollcall16:00
mnasero/16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
guilhermespo/16:00
*** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)"16:00
openstackThe meeting name has been set to 'openstack_ansible_meeting'16:00
*** openstack changes topic to "rollcall (Meeting topic: openstack_ansible_meeting)"16:00
kmadac@fnpanic: patch basically adds environment variable REQUESTS_CA_BUNDLE to heat systemd service16:01
jrossero/16:01
evrardjpp/16:01
odyssey4merpsene something to try - clear out the fact cache, then set the fact gathering subset to 'all', then try again, ie rm -rf /etc/openstack_deploy/ansible_facts; export ANSIBLE_GATHER_SUBSET='all'; ... execute the playbooks ...16:01
jamesdentono/16:01
spotzo/16:02
mnaser#topic Last week highlights16:02
*** openstack changes topic to "Last week highlights (Meeting topic: openstack_ansible_meeting)"16:02
mnaseranyone gots some fun highlights they want to share from last week?16:02
odyssey4meFOSDEM was interesting :)16:02
*** markvoelker has joined #openstack-ansible16:03
mnaseri hear it was a mini ptg :)16:03
odyssey4memnaser haha, not quite - but there was much beer, and evrardjp and I managed to have a short chat about config_template16:03
prometheanfireo/16:04
mnaserfun times16:04
odyssey4memore progress on that in https://review.openstack.org/#/q/topic:bug/1791258 and https://etherpad.openstack.org/p/osa-config-template-migration16:04
*** jtanner has joined #openstack-ansible16:04
evrardjpand releases too ! :p16:05
mnasersounds like a fun time16:05
evrardjp(I was merely pushing my laziness agenda I guess?)16:05
mnaseri support that agenda, hah16:06
odyssey4meoh yes, he told me about https://review.openstack.org/#/q/topic:easier_osa_releases which is awesome :)16:06
mnaserawesome, we're almost landing that16:06
fnpanichttps://bugs.launchpad.net/openstack-ansible/+bug/181563116:06
openstackLaunchpad bug 1815631 in openstack-ansible "[Docs] all examples lacking correct horizon image upload mode" [Undecided,New]16:06
odyssey4mewith that merged, we'll never have to update openstack_release, ever again (from stein onwards)16:07
evrardjpI will backport those16:07
mnaser🔥16:07
mnaseri'm excited :)16:07
odyssey4meoh, and we chatted a bit about the python build simplification - I got confirmation from evrardjp that my thinking was right - I'll do some updates to the existing patches on friday and will hopefully be able to spread the pattern across other roles if everyone's happy with them... so we should be able to chat about that next week in more detail16:08
odyssey4mewith that done, I'll circle back to using the integrated repo for all role tests - I think we should be able to get it done for stein16:09
*** gkadam has quit IRC16:09
mnaserw00t, odyssey4me please let us know if you need any help with all of this :)16:09
evrardjpDuring that conversation I pointed out the fact that we should branch when we feel this is ready16:10
odyssey4meslightly tangential to that, I need another review for https://review.openstack.org/636297 to unblock the os_heat repo's integrated build tests: https://review.openstack.org/63629716:10
odyssey4meurgh, sorry - I got a little over excited with copy/paste there16:10
fnpanichow can i use the ansible inventory groups in my plays?16:11
guilhermespand this patch unblocked this https://review.openstack.org/#/c/634032/3 odyssey4me :)16:11
guilhermespit is still finishing but I see all green16:11
odyssey4meyep, great - thanks guilhermesp16:12
odyssey4mefnpanic we can discuss that after the meeting if you don't mind?16:12
fnpanicok sorry16:12
mnasercoolio16:12
mnaserthat's some awesome updates over the weekends :)16:12
fnpanicwas not awarre thar is it that late16:12
mnaseror the week rather ;)16:13
dmsimardFor the record, I filed a bug for the horizon -> cinder issue I had with my AIO https://bugs.launchpad.net/horizon/+bug/181563516:13
openstackLaunchpad bug 1815635 in OpenStack Dashboard (Horizon) "Horizon is unable to retrieve Cinder API versions when it has a self-signed SSL certificate" [Undecided,New]16:13
guilhermespaaaand some clues about this https://review.openstack.org/#/c/627782/6 the workspace move maybe is not working as expected16:13
*** gkadam has joined #openstack-ansible16:14
odyssey4meoh, I see the placement role patch merged - is that all good to go? if so, then part of my work to use the integrated build for role testing will be to work on the upgrade path.... but someone will still need to change the nova role up to use the new placement role16:14
mnaserodyssey4me: yes, my next step is doing os_nova to use os_placement16:14
odyssey4methanks dmsimard - we'll move to triage shortly and that'll likely come up16:14
dmsimardodyssey4me: I filed the bug in horizon, but it looks like there are improvement opportunities for self-signed certificates in AIO in general16:15
odyssey4memnaser ok cool - just greenfield will be fine, we'll work on the upgrade path later and test it using the integrated build, hopefully in the gate :)16:15
openstackgerritGeorgina Shippey proposed openstack/openstack-ansible-os_neutron stable/rocky: Add support for dns_domain_ports api extension  https://review.openstack.org/63636816:15
dmsimardapologies, I wasn't aware a meeting was ongoing16:16
mnaserwe need to have a job that uses in-repo placement in os_nova16:17
mnaserand then another one that uses the new os_placement role16:17
mnaser(because nova wants to test in-repo placement using our jobs)16:17
mnaserbut yes indeed odyssey4me we'll figure it out for greenfield and then upgrade will be the tricky one16:18
mnaser#topic bug squash day16:19
*** openstack changes topic to "bug squash day (Meeting topic: openstack_ansible_meeting)"16:19
odyssey4memnaser heh, sounds like a bit of fun then16:19
* mnaser #link https://etherpad.openstack.org/p/osa-bug-squash-q116:20
mnaser#link https://etherpad.openstack.org/p/osa-bug-squash-q116:20
odyssey4medmsimard most definitely, that's an itch which no-one has really scratched yet unfortunately... although some reasonable work was done in the galera_server/galera_client roles which may be worth looking at16:20
mnaseri came up with a little link to share and work on some of our progress16:21
mnaserfor tomororw hopeuflly16:21
jamesdentonright on16:22
spotzI'm working weird hours tomorrow but I'll work through the review list when I can for sure16:22
admin0is removing old facts,  # in the config and  using inventroy-manage --remove  to delete enough to remove services .. like i want to not use aodh/gnocchi anymore16:23
mnaserhow does everyone feel about a high bandwidth way of chatting16:23
mnaserzoom? hangouts? good o'l conference call? :P16:23
odyssey4meI unfortunately won't be able to actively participate in the bug triage, other than to be available to answer questions and advise where I can via IRC.16:23
guilhermespI vote zoom16:23
odyssey4memnaser I would personally prefer IRC, because it's all logged and open - but it may make sense to use vid conf for small groups who're working together...16:24
mnaserodyssey4me: yeah im thinking maybe like a subgroup to hack on a specific task if they feel like but *leaving* everything in irc16:24
odyssey4meadmin0 we can discuss that after the meeting16:25
admin0sorry .. did not realized we are in a meeting ( just walked in)16:25
spotzmnaser maybe create 2-3 zooms and have links in the ether that folks can check in and out?16:25
mnaserand log all that stuff in the etherpad to track our progress16:25
guilhermespyeah I think would be good to divide and conquer in this way...main topics on irc and small groups discussing in a conference, then registering things on etherpad16:26
mnaseri'll leave it as on-demand16:26
mnaserwe can get a good room anytime :)16:26
odyssey4memnaser yep, makes sense - and spotz' suggestion to have a few rooms to use for particular focus groups is a good one, I think.16:26
mnaserhangouts or whatever can be really on demand16:26
mnaseryeppers16:26
mnaserdoes anyone have any other suggestions for that etherpad?16:26
mnaserit's my first time running something like this so all advice is welcome16:27
spotzMaybe just a note when something is merged so those of us focusing on reviews don't keep opening things?16:27
odyssey4mezoom's a great way to get more eyes on the same console to see the same problem as the person doing the triage/fix16:27
odyssey4mespotz I think that's what gerritbot is for. ;)16:28
spotzodyssey4me: Bah, you know I tend to squirrel onto things in front of me!:)16:28
cloudnullo/16:28
mnaseri have fully carved out my day to dedicate for this :)16:29
mnaserso it'll be fun except for the fact we can't go out after we're done some good work D:16:29
cjloadero/16:30
evrardjpI see what you did there16:30
spotzNot that you managed to go out with us in Berlin mnaser.....:)16:31
guilhermesphahahah16:32
spotzhey cloudnull and cjloader16:32
odyssey4memnaser True. I think I'd like to facilitate a virtual mid-cycle in Train for OSA where we can focus on achieving our goals, so I'm interested to see how this goes. I'm thinking it may also be useful to facilitate a regular bug/review smash for the Ansible OpenStack modules too, given that those reviews and bugs aren't getting enough attention either.16:32
mnaserodyssey4me: i agree!  this will be a good attempt at seeing that16:32
cjloader+1 odyssey4me16:33
mnaseri would love for us to take a day or two to sprint and get all those super awesome things we discucsed at the ptg16:33
mnaserwe've done great progress16:33
* redrobot sneaks into the meeting 16:34
mnaserso if anyone is around does anyone have a show of hands of who might be there tomorrow? :)16:34
jamesdentoni'll do my best16:35
*** markvoelker has quit IRC16:35
jrosserwe will be there16:37
guilhermespmnaser: o/16:37
*** rjgibson has quit IRC16:37
spotzo/ but at weird times16:37
mnasercoolio16:37
*** udesale has quit IRC16:37
mnasergiven we'll be doing a bunch of bug triage tomorrow i'll skip that :)16:37
mnaser#topic open discussion16:37
*** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)"16:37
redroboto/16:38
mnaserhi redrobot :)16:38
redrobotHi mnaser!16:38
mnasersomething fun to share for our open discussion ? :P16:39
redrobotlooks like I snuck in here just in time :D16:39
redrobotyes!16:39
redrobotSo, both HSM ansible roles are part of openstack-ansible now!16:39
redrobot🎉🎉🎉16:39
redrobotThanks to everyone who +1/+2 patches16:40
redrobotThere's only one outstanding thing in my todo16:40
redrobotand that's adding myself and ade_lee (or barbican-core) to the review teams for those two roles16:40
redrobot#link https://review.openstack.org/#/admin/groups/2005,members16:41
odyssey4meredrobot personally, I'd be happy to see barbican-core in there16:41
redrobot#link https://review.openstack.org/#/admin/groups/2005,members16:41
redrobotmnaser, I believe you own both groups... so you'd be the one to add new members16:41
spotz+116:41
odyssey4meI think we're all sensible enough to review on the basis of the things we understand.16:42
redrobotodyssey4me, spotz 😁16:42
odyssey4meI'd suggest also adding the openstack-ansible-core group there too.16:42
evrardjpindeed16:42
mnaseri agree with odyssey4me.16:43
redrobotodyssey4me, +1 ... I agree sharing duties between our two teams seems ideal16:43
mnaserredrobot: i added barbican-core and osa-core to that group16:43
evrardjpgood job mnaser16:43
mnaserplease don't shy from sending reviews as you start16:43
redrobotmnaser, how about the other one? 😬16:43
mnaseri see the same link twice D:16:44
redrobotoh oops16:44
redrobotmy bad16:44
redrobot#link https://review.openstack.org/#/admin/groups/2006,members16:44
redrobot^^ that's the second one16:44
odyssey4meredrobot I'm not really all that familiar with the subject matter, so hopefully someone in the community will adopt them for their environment and start working on getting them working with our integrated build - or creating an appropriate user story and some openstack-ansible-ops playbooks to help people understand how to use them.16:44
spotzbad redrobot!16:44
mnaservoila redrobot :)16:44
redrobotodyssey4me, agreed, the challenge there is to have both HSMs readily available for test.  We're working on downstream tests for sure, but I'll try to push for upstream testing as well.16:45
jamesdentonI'm soliciting feedback on https://docs.openstack.org/openstack-ansible/latest/user/network-arch/example.html and https://docs.openstack.org/openstack-ansible/latest/user/prod/provnet_groups.html for those interested. Both new docs.16:46
odyssey4meredrobot well, if there's some sort of downstream testing via external CI to provide active feedback to changes on the role, that'd be good enough in my books16:46
redrobot(additionally, to complicate things some HSM client software may not have open source licenses. 😔16:46
evrardjptbh I am not too familiar with those, but I guess if we are a little trained with them we could review more efficiently. I can only review the ansible bits to see if they match the latest standards16:46
mnaseryou can still have 3rd party voting16:46
evrardjpredrobot: I see16:46
redrobotodyssey4me, ack, I will definitely keep that in mind.16:46
mnasers/3rd party voting/3rd party ci/16:46
redrobotmnaser, much appreciated! :D16:46
odyssey4meooh jamesdenton you drew purdy pics :)16:47
*** partlycloudy has joined #openstack-ansible16:47
redrobotok, that's all I have16:47
jamesdentonjust for you, odyssey4me16:47
spotzhehe16:47
prometheanfireAdri2000: plan on backporting I2a3e266e3066d6c271a29038e814ff8e3090fafb ? looks like nova has the same change in rocky (and queens, pike)16:48
mnaserjamesdenton: i like those docs!16:49
mnaseranyone else has any other proposed solutions?16:49
jamesdentonthanks!16:49
odyssey4mejamesdenton at a cursory glance, that's really nice work - great example configs and all16:49
odyssey4meprometheanfire anyone can propose the backport :)16:50
odyssey4memnaser proposed solutions? I'm confused16:50
cjloadermnaser: i'll be there tomorrow -- had to check w my manager16:51
mnaser`ER16:51
mnaserproposed topics16:51
mnaserim sorry16:51
mnaser:<16:51
prometheanfireodyssey4me: there are other issues it looks like, how did both https://github.com/openstack/openstack-ansible-os_nova/commit/f67e610609df9f99778de00630c7e0db9b4295ed and https://github.com/openstack/openstack-ansible-os_nova/commit/a383124336dfc74b4d652be594f7bba0197c24b1 make it in, they both deleted the same section of the template16:51
odyssey4meprometheanfire heh, I have no idea16:52
prometheanfireya, need to fix that up too16:53
odyssey4memnaser I have nothing for now.16:53
mnasercool, anyone else? :)16:53
guilhermespmnaser: I just have the upgrade jobs that we need some feedback of tempest guys16:53
guilhermespwe already discussed about it too16:53
prometheanfirebackports would be incomplete so I might make a combined (correct) one.16:53
guilhermespalso AFIK https://review.openstack.org/#/c/635579/3 depends on a fix that I don't fully remember now16:54
odyssey4meguilhermesp oh, that workspace thing is odd - so running the 'move' command doesn't actually move it? perhaps we should just move it with 'mv' after the 'move' command?16:54
odyssey4meprometheanfire yep, sounds good16:54
guilhermespmaybe odyssey4me ... what I saw in the hold is that the old workspace is in the same place and the new one is empty16:55
guilhermespit's like nothing was moved, just in the api point of view16:55
Adri2000prometheanfire: I think I didn't backport that because it's quite easy to override in the config, but feel free to submit a backport if needed16:55
guilhermespthat's why we could be experiencing these nested venv issues when running tempest run -l between the two releases16:55
odyssey4meguilhermesp I see other os_nova patches passing CI today - so perhaps that just needs a recheck now16:55
guilhermespodyssey4me: will do that now16:56
mnasershall i end? :>16:56
guilhermespfor me it's ok mnaser16:57
*** TxGirlGeek has quit IRC16:57
mnasercoolio16:57
mnaser#endmeeting16:57
*** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/2xA1eZC"16:57
openstackMeeting ended Tue Feb 12 16:57:36 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:57
openstackMinutes:        http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2019/openstack_ansible_meeting.2019-02-12-16.00.html16:57
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2019/openstack_ansible_meeting.2019-02-12-16.00.txt16:57
openstackLog:            http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2019/openstack_ansible_meeting.2019-02-12-16.00.log.html16:57
jrossernova patches are only passing today becasue of depends-on this https://review.openstack.org/#/c/636162/16:58
odyssey4mekmadac have you added the link to http://paste.openstack.org/show/744947/ in the bug you mentioned? that may be useful to whoever picks it up16:58
odyssey4meor for others looking for a solution16:58
guilhermespuow, I think I should add to and recheck again jrosser16:59
jrosserwe just need to decide what to do with the tests network setup16:59
jrosserbecasue thats not enough to fox neutron tests16:59
jrosser*fix16:59
openstackgerritGuilherme  Steinmuller Pimentel proposed openstack/openstack-ansible-os_nova master: Add nova_user_pip_packages variable  https://review.openstack.org/63557917:00
*** sdake_ has quit IRC17:01
odyssey4mefnpanic to use openstack-ansible inventory groups in your plays, you just use them in the 'hosts: <target group>' for the playbook - and then execute the playbook using 'openstack-ansible' - alternatively you can run 'ansible-playbook -i /opt/openstack-ansible/inventory -e@/etc/openstack_deploy/user_variables.yml -e@/etc/openstack_deploy/user_secrets.yml ...'17:02
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova master: remove duplicate pci section  https://review.openstack.org/63638817:02
*** sdake has joined #openstack-ansible17:02
odyssey4mefnpanic or you can cd to /opt/openstack-ansible and use ansible-playbook and it will automagically include the right CLI parameters17:03
odyssey4meadmin0 to remove a service - 1. use the container delete playbook to delete the containers; 2. remove the openstack_user_cconfig entry that allocates the host to the host group; 3. use inventory-manage to remove the containers from the inventory; 4. run the playbooks which would have integrated with those services to remove any related conf entries and integrations17:05
*** hamzy has quit IRC17:06
kmadac@oddysey4me: not yet, but I'll do it now17:09
odyssey4mekmadac fantastic, thank you17:11
odyssey4meguilhermesp could you add a link to the docs as noted by evrardjp in https://review.openstack.org/634032 ?17:11
*** partlycloudy has quit IRC17:12
guilhermespfor sure, will do now17:12
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova stable/rocky: Update deprecated option for pci passthrough  https://review.openstack.org/63639017:14
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova stable/rocky: Update the pci config for nova.  https://review.openstack.org/63639117:14
prometheanfirelooks like I can backport :D17:14
* jrosser joins the group with broken heat deployment17:18
guilhermespjrosser what is broken now? rocky?17:19
*** gkadam has quit IRC17:19
*** flaviosr has quit IRC17:24
odyssey4mejrosser guilhermesp of course it's all mnaser's fault :p17:26
mnaseruhohhhhhhh17:26
mnaseryou mean its centos fault?17:26
guilhermesphahaha17:26
mnaserodyssey4me: contesxt though?17:27
odyssey4memnaser it seems that https://github.com/openstack/openstack-ansible-os_heat/commit/1b70515960f2761ab5b98d876ae9d9854effde21 has broken most deployments, and given that jrosser has an env with it broken, he may need some help figuring out an appropriate fix17:28
openstackgerritMerged openstack/openstack-ansible master: Use an env lookup to determine the OSA version  https://review.openstack.org/63376317:29
mnaserguilhermesp: just recently did a deployment that heat works fine in + magnum as well :X17:29
guilhermespyep I can confirm that17:29
odyssey4memnaser guilhermesp using a real cert, or self-signed?17:30
odyssey4meand were any override configs required?17:31
guilhermespI think I was using real certs tho17:31
jrosserright, but you allow your mgmt net access to the external VIP?17:32
*** markvoelker has joined #openstack-ansible17:32
*** aedc has joined #openstack-ansible17:33
jrosserthe orchestration panels in horizon and the cli are all hosed17:33
jrosserthe cli errors with "'ClientManager' object has no attribute '_auth_url'"17:34
guilhermespfor now yeah, mgmt net can access the external VIP17:34
jrosserok, well i suspect that is why its working in that environment17:35
jrossertheres a bunch of other folks who are broken relatedly becasue of self signed certs17:35
jrosserwhere the internal heat components don't trust the external VIP cert17:36
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova stable/queens: Update deprecated option for pci passthrough  https://review.openstack.org/63639617:36
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova stable/queens: Update the pci config for nova.  https://review.openstack.org/63639717:36
admin0odyssey4me, " 4. run the playbooks which would have integrated with those services to remove any related conf entries and integrations" -- if they do not see it in inventory or the config or the facts, would they automatically disassociate/fix themselves ?17:37
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova stable/pike: Update deprecated option for pci passthrough  https://review.openstack.org/63639817:38
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_nova stable/pike: Update the pci config for nova.  https://review.openstack.org/63639917:38
*** spsurya has quit IRC17:40
*** idlemind has joined #openstack-ansible17:43
*** sdake has quit IRC17:44
*** sdake has joined #openstack-ansible17:47
*** gyee has joined #openstack-ansible17:47
prometheanfirecan someone confirm for me if they think nova_pci_passthrough_whitelist should be quoted like that, given that it should end up with         { "vendor_id": "8086", "product_id": "10fb" }17:52
prometheanfireAdri2000: ^17:52
prometheanfirehttps://github.com/openstack/openstack-ansible-os_nova/blob/master/templates/nova.conf.j2#L36117:52
prometheanfiresame for alias17:53
prometheanfirehttps://docs.openstack.org/nova/rocky/admin/pci-passthrough.html17:53
*** kopecmartin is now known as kopecmartin|off17:58
prometheanfireodyssey4me: if you have a sec, mind checking my logic from a few min ago? ^18:00
odyssey4meprometheanfire hmm, it would seem that the quotes wouldn't be right there... but there are folks using this stuff, so I wonder if nova translates it correctly anyway18:01
odyssey4meI would have thought that it might need something like: alias = {{ item | to_json }}18:02
odyssey4meprometheanfire https://github.com/openstack/openstack-ansible-os_nova/blob/master/defaults/main.yml#L648-L65918:03
odyssey4meit would appear that the approach has been to put the value into a string, and I would guess that nova receives it as a string and converts it into a dict... so it's fine18:04
odyssey4me.conf files only do strings as far as I know, which is why it's working18:05
*** markvoelker has quit IRC18:06
jrosserguilhermesp: I think I will revert the heat endpoint change out of my environment and follow you guys down the magnum path and see what happens18:11
jrosserIs there anything I need to know up front about getting magnum going?18:12
prometheanfireodyssey4me: the output is this which might be fine, just seems odd     "{ "vendor_id": "10de", "product_id": "15f7" }"18:16
prometheanfirevs { "vendor_id": "10de", "product_id": "15f7" }18:16
jamesdentoni guess the bigger question is, does Nova care?18:17
jamesdentonBut you're right, the example conf file does not show it quoted: https://docs.openstack.org/nova/latest/configuration/sample-config.html18:17
odyssey4meprometheanfire sure, but if you push a patch to change it then please verify that it works first18:17
prometheanfireodyssey4me: ok, ya, I'm verifying18:17
*** DanyC has quit IRC18:20
*** DanyC has joined #openstack-ansible18:21
*** DanyC has quit IRC18:25
guilhermespoh jrosser yes, you need that patch https://review.openstack.org/#/c/634032/18:26
mnaserhttps://etherpad.openstack.org/p/osa-bug-squash-q1 updated that a little bit, please add yourself to the etherpad :)18:30
odyssey4mecjloader ^18:33
openstackgerritMerged openstack/openstack-ansible master: Use the config_template module from the dedicated repo  https://review.openstack.org/63618218:33
openstackgerritMerged openstack/openstack-ansible master: Update the heat service egg name to openstack_heat  https://review.openstack.org/63629718:40
mnaseranyone can have eyes on https://review.openstack.org/#/c/634991/ ?: >18:41
mnaser:>18:41
*** hamzy has joined #openstack-ansible18:43
*** sdake has quit IRC18:44
*** DanyC has joined #openstack-ansible18:44
mgariepysure done18:45
mgariepymnaser, do you have a sec for those : https://review.openstack.org/#/c/632907/ https://review.openstack.org/#/c/632908/ ?18:46
* mnaser click18:46
mnasermgariepy: done :)18:46
jamesdenton> fatal: [infra1]: FAILED! => {"msg": "module (config_template) is missing interpreter line"}18:46
jamesdentonhum18:46
mgariepythanks :)18:47
*** priteau has quit IRC18:47
jamesdentonare there config_template things in flight at the moment?18:47
odyssey4mejamesdenton yeah, some patches just merged - where're you seeing that?18:47
jamesdentonhttps://review.openstack.org/#/c/636390/18:47
jamesdentonabout 40 min ago18:47
odyssey4mejamesdenton that's odd - that's rocky, and we've oly been changing master18:48
odyssey4melemme check something18:48
*** DanyC has quit IRC18:49
openstackgerritMerged openstack/openstack-ansible-tests master: Ensure ipforward is set on all test bridges  https://review.openstack.org/63616218:51
*** sdake has joined #openstack-ansible18:51
odyssey4mejamesdenton that's bizarre, I wonder if that'll replicate outside the gate18:53
odyssey4meit seems like the test did all the right things, and was only using stable/rocky18:54
jamesdentonThe queens and pike backports seem to be progressing OK18:56
*** markvoelker has joined #openstack-ansible19:03
*** jrosser_ has joined #openstack-ansible19:04
*** adrianreza has quit IRC19:04
*** jrosser has quit IRC19:04
*** Dantalion has quit IRC19:04
*** jroll has quit IRC19:04
*** sm806 has quit IRC19:04
*** masterpe has quit IRC19:04
*** mugsie has quit IRC19:04
*** fdegir has quit IRC19:04
*** asettle has quit IRC19:04
*** zul has quit IRC19:04
*** mwe1 has quit IRC19:04
*** tinwood has quit IRC19:04
*** med_ has quit IRC19:04
*** jrosser_ is now known as jrosser19:04
mnaserodyssey4me: i was thinking maybe we should eventually just move out things inside openstack-ansible-plugins to in-repo and the things that are shared to their own repo19:04
*** tinwood has joined #openstack-ansible19:04
*** sm806 has joined #openstack-ansible19:05
*** mugsie has joined #openstack-ansible19:05
odyssey4memnaser the advantage of having them in their own repo is that it makes implementing changes and related tests simpler.19:05
*** fdegir has joined #openstack-ansible19:05
odyssey4metoo many things in one repo means loads of tests - both long running and short, and sometimes for a simple change that's too much19:05
mnaserodyssey4me: gotcha, it does seem like (since osa's inception), a lot of these modules and filters have decreased19:06
odyssey4meyep, hopefully we'll be able to get rid of them all over time and just retire the extra repo19:06
*** mwe1 has joined #openstack-ansible19:06
*** jroll has joined #openstack-ansible19:06
mnaserand some others im not sure where we use19:07
mnaseri.e. memcached module19:07
odyssey4mejamesdenton oh dear - found the bug19:09
odyssey4meI'll push a patch up now19:09
jamesdentonoh good19:09
*** canori01 has joined #openstack-ansible19:16
openstackgerritMerged openstack/ansible-hardening stable/rocky: Switch to rtcsync for chrony  https://review.openstack.org/63290719:18
openstackgerritMerged openstack/ansible-hardening stable/queens: Switch to rtcsync for chrony  https://review.openstack.org/63290819:18
canori01I'm having a problem where the neutron role is incorrectly using the br-mgmt ip for the vxlan vtep address instead of the ip of br-vxlan. How does it figure out?  I've been going through my config, but I'm not seeing problems there19:22
*** dmsimard has quit IRC19:22
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Clone the correct plugins repo for role tests  https://review.openstack.org/63642619:22
odyssey4mejamesdenton ^ I'll need to port that back - will give that a go now19:22
jamesdentoncanori01 I've seen that before. What version are you trying to install?19:23
canori01jamesdenton: This Rocky19:23
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/rocky: Clone the correct plugins repo for role tests  https://review.openstack.org/63642719:23
jamesdentonstable/rocky? Are a particular tag?19:24
canori01jamesdenton: 18.1.2 tag off the openstack ansible repo19:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/rocky: Clone the correct plugins repo for role tests  https://review.openstack.org/63642719:28
odyssey4mejames ok, https://review.openstack.org/#/q/Iaf5ee20e534a8279b5b563c820518bb176f0f4bb works - mgariepy mnaser could I get a quick review please - this unbreaks stable/rocky role tests19:28
jamesdentonthanks odyssey4me19:28
jamesdentoncanori01 trying to find my notes on that issue.19:28
jamesdentonOk canori01 - It was due to an issue that was patched here: https://review.openstack.org/#/c/630429/19:31
jamesdentonLooks like you'll need 18.1.3 to get that19:31
canori01jamesdenton: alright, thanks!19:32
jamesdentonAnd to fix any existing vxlan networks, there's a bit of a process there that includes either a) deleting and recreating the networks or b) deleting the vxlan interfaces and restarting L2 agent19:32
*** dmsimard has joined #openstack-ansible19:32
openstackgerritMerged openstack/openstack-ansible-os_mistral master: Add initial Mistral role  https://review.openstack.org/63499119:33
canori01jamesdenton: Luckily, this was a fresh deployment, so there's no existing vxlan networks other than the one I was testing with19:34
*** markvoelker has quit IRC19:36
jamesdentonok great, should be pretty straightforward then19:37
*** sdake has quit IRC19:42
openstackgerritMerged openstack/openstack-ansible-os_heat master: Fixed the egg name of heat to openstack_heat  https://review.openstack.org/63551819:53
openstackgerritMerged openstack/openstack-ansible-os_heat master: Add heat user to heat domain admin role  https://review.openstack.org/63403219:53
*** zul has joined #openstack-ansible19:59
mnaserodyssey4me: does it work for you locally?20:05
odyssey4memnaser yep20:06
odyssey4meassuming you're speaking about https://review.openstack.org/#/q/Iaf5ee20e534a8279b5b563c820518bb176f0f4bb20:06
odyssey4meit's passed in CI too - just waiting for the slower gates20:06
*** DanyC has joined #openstack-ansible20:08
openstackgerritGuilherme  Steinmuller Pimentel proposed openstack/openstack-ansible-os_nova master: Add nova_user_pip_packages variable  https://review.openstack.org/63557920:12
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add option to disable container builds on specific hosts  https://review.openstack.org/63599720:27
cloudnullany chance I could get someone to nudge https://review.openstack.org/#/c/588951 and https://review.openstack.org/#/c/633104 forward20:27
*** markvoelker has joined #openstack-ansible20:33
openstackgerritKevin Carter (cloudnull) proposed openstack/ansible-role-python_venv_build master: Add venv build target hosts data structure  https://review.openstack.org/63619220:35
cloudnullodyssey4me ^20:35
cloudnullalso https://review.openstack.org/#/q/topic:bug/1794320+(status:open+OR+status:open) - these are super simple reviews20:36
openstackgerritMerged openstack/openstack-ansible-os_tempest master: Add option to disable router ping  https://review.openstack.org/63621120:51
*** fnpanic has quit IRC20:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_heat master: [DNM] heat tempest tests finding  https://review.openstack.org/63069420:56
*** vollman has quit IRC20:57
odyssey4mecloudnull done21:00
odyssey4meok, I'm out for the night - cheers folks21:01
*** fdegir has quit IRC21:06
*** tinwood has quit IRC21:06
*** mugsie has quit IRC21:06
*** markvoelker has quit IRC21:06
*** mugsie has joined #openstack-ansible21:06
*** tinwood has joined #openstack-ansible21:06
cloudnullthanks odyssey4me have a good night !21:09
openstackgerritMerged openstack/openstack-ansible-tests stable/rocky: Clone the correct plugins repo for role tests  https://review.openstack.org/63642721:11
*** DanyC has quit IRC21:11
*** DanyC has joined #openstack-ansible21:12
*** sdake has joined #openstack-ansible21:26
openstackgerritMerged openstack/ansible-role-systemd_networkd master: Update home-page  https://review.openstack.org/63043521:32
cjloaderstein will be bionic only correct?21:37
openstackgerritMerged openstack/openstack-ansible-tests master: Clone the correct plugins repo for role tests  https://review.openstack.org/63642621:38
*** ArchiFleKs has quit IRC21:40
openstackgerritMerged openstack/ansible-role-systemd_service master: Build out the PrivateNetwork function for services  https://review.openstack.org/63539621:45
*** hamzy has quit IRC21:46
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible-os_mistral master: Updated from OpenStack Ansible Tests  https://review.openstack.org/63644521:48
*** sdake has quit IRC21:50
*** ArchiFleKs has joined #openstack-ansible21:52
*** markvoelker has joined #openstack-ansible22:03
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Correct notification driver  https://review.openstack.org/63567922:03
openstackgerritMerged openstack/openstack-ansible-os_glance master: Correct notification driver  https://review.openstack.org/63567622:04
openstackgerritMerged openstack/openstack-ansible-os_trove master: Correct notification driver  https://review.openstack.org/63568322:05
*** NobodyCam has joined #openstack-ansible22:12
openstackgerritMerged openstack/openstack-ansible-os_nova master: remove duplicate pci section  https://review.openstack.org/63638822:13
openstackgerritMerged openstack/openstack-ansible-os_nova master: Fix default init config override for nova-compute  https://review.openstack.org/63310422:13
NobodyCamgood afternoon OSA folks22:13
openstackgerritMerged openstack/openstack-ansible-os_nova master: Cleanup files and templates using smart sources  https://review.openstack.org/58895122:14
openstackgerritMerged openstack/openstack-ansible-os_designate master: Correct notification driver  https://review.openstack.org/63567522:20
*** markvoelker has quit IRC22:36
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Correct notification driver  https://review.openstack.org/63567422:38
*** sreejithp has quit IRC22:47
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Correct notification driver  https://review.openstack.org/63568022:50
openstackgerritMerged openstack/openstack-ansible-os_heat master: venv: use inventory_hostname instead of ansible_hostname  https://review.openstack.org/63517823:24
*** markvoelker has joined #openstack-ansible23:33
*** macza has quit IRC23:42
*** macza has joined #openstack-ansible23:42
*** sdake has joined #openstack-ansible23:43

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!