Thursday, 2018-03-29

*** flemingo has joined #openstack-ansible00:01
*** flemingo_ has quit IRC00:04
*** flemingo has quit IRC00:05
openstackgerritMerged openstack/openstack-ansible-plugins master: Revert "Only su within container if container_user is provided"
*** Leo_m has quit IRC00:22
*** odyssey4me has quit IRC00:53
*** odyssey4me has joined #openstack-ansible00:53
*** chyka has joined #openstack-ansible01:01
*** chyka has quit IRC01:05
*** osnaya has quit IRC01:45
*** NWOutlier__ has quit IRC01:53
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Update and remove
*** yifei1 has joined #openstack-ansible02:18
*** strobelight has quit IRC03:08
*** NWOutlier__ has joined #openstack-ansible03:10
*** udesale has joined #openstack-ansible03:29
*** sawblade6 has joined #openstack-ansible03:32
*** sawblade_ has quit IRC03:34
*** sawblade_ has joined #openstack-ansible03:41
*** sawblade6 has quit IRC03:43
*** ianychoi__ has joined #openstack-ansible03:47
*** geb has quit IRC03:49
*** rudysp has quit IRC03:49
*** ianychoi_ has quit IRC03:50
*** goldenfri has quit IRC03:50
*** udesale has quit IRC03:51
*** geb has joined #openstack-ansible03:53
*** sawblade6 has joined #openstack-ansible03:53
*** sawblade_ has quit IRC03:55
*** dave-mccowan has quit IRC03:56
*** dave-mccowan has joined #openstack-ansible03:58
*** poopcat has quit IRC04:06
*** dave-mccowan has quit IRC04:16
*** dave-mcc_ has joined #openstack-ansible04:17
*** bhujay has joined #openstack-ansible04:21
*** ianychoi__ is now known as ianychoi04:29
*** chyka has joined #openstack-ansible04:37
openstackgerritMerged openstack/openstack-ansible-specs master: doc: source: index: Add Rocky specifications to index
*** chyka has quit IRC04:41
cloudnullah bummer. looks like with the changes we've been making in the new venv role we've caused some conflict where it was merged already.04:43
cloudnullidk if we revert or fix?04:44
cloudnulli think we're still working on the interface04:44
cloudnullevrardjp: odyssey4me?04:44
*** dave-mcc_ has quit IRC04:46
*** cjloader has quit IRC04:47
*** indistylo has joined #openstack-ansible04:55
*** cjloader has joined #openstack-ansible04:56
*** udesale has joined #openstack-ansible05:04
*** bhujay has quit IRC05:08
openstackgerritMerged openstack/openstack-ansible-os_glance master: Convert role to use a common systemd service role
*** NWOutlier__ has quit IRC05:19
openstackgerritMerged openstack/openstack-ansible-tests master: Re-use variables in
openstackgerritMerged openstack/openstack-ansible-tests master: Re-use env var for setting the test branch
openstackgerritKevin Carter (cloudnull) proposed openstack/ansible-role-python_venv_build master: Implement build delegation
*** bhujay has joined #openstack-ansible05:23
*** cjloader has quit IRC05:27
*** aruns has joined #openstack-ansible05:37
*** indistylo has quit IRC05:40
*** admin0 has joined #openstack-ansible05:42
*** chhagarw has joined #openstack-ansible06:02
*** udesale_ has joined #openstack-ansible06:13
*** cjloader has joined #openstack-ansible06:14
*** udesale has quit IRC06:16
*** vnogin has joined #openstack-ansible06:17
*** cjloader has quit IRC06:18
*** haad1 has joined #openstack-ansible06:18
*** vnogin has quit IRC06:22
*** udesale_ has quit IRC06:38
*** udesale has joined #openstack-ansible06:39
*** fghaas has joined #openstack-ansible06:50
evrardjpcloudnull: oh bummer06:58
fghaas#success I just wanted to mention that I ran for the first time last night (for a pike→queens upgrade), having only used the individual upgrade playbooks before, and holy cow it's awesome. I kicked that thing off in a screen session, and didn't touch it again until after the upgrade was complete. Awesome work everyone!06:59
openstackstatusfghaas: Added success to Success page (
*** jwitko has quit IRC06:59
*** armaan has joined #openstack-ansible07:00
evrardjpfghaas: a lot of effort was done in making sure this is well tested and working, but it's an ongoing effort.07:00
evrardjpthanks for the kind words07:00
*** udesale_ has joined #openstack-ansible07:03
*** udesale__ has joined #openstack-ansible07:05
*** haad1 has quit IRC07:07
*** udesale has quit IRC07:07
*** udesale_ has quit IRC07:09
*** haad1 has joined #openstack-ansible07:13
fghaasevrardjp: filed per your request07:27
openstackLaunchpad bug 1759784 in openstack-ansible "Gate jobs don't cover Glance image upload to file store" [Undecided,New]07:27
*** haad1 has joined #openstack-ansible07:41
*** ibmko has quit IRC07:44
fghaasActually, one question about It looks as though from pike to queens, nova-{conductor,scheduler,consoleauth} were all consolidated into one LXC container when previously they used three. That leaves me with — are the original containers (1, 2, 5) safe to discard and if so, should the script do that?07:44
*** hamza21 has joined #openstack-ansible07:44
*** udesale has joined #openstack-ansible07:45
fghaasSame for Cinder, by the way, where cinder-scheduler is now lumped in with the API container.07:45
*** udesale__ has quit IRC07:47
fghaas(Unless I missed something, happens to be silent on the above.)07:49
*** electrofelix has joined #openstack-ansible07:54
*** haad1 has joined #openstack-ansible07:57
*** bhujay has quit IRC08:07
openstackgerritKevin Lefevre proposed openstack/openstack-ansible stable/queens: Add support for Horizon Octavia Dashboard
*** DanyC has quit IRC08:09
*** DanyC has joined #openstack-ansible08:10
*** aruns__ has joined #openstack-ansible08:12
*** chyka has joined #openstack-ansible08:12
*** DanyC has quit IRC08:14
*** ibmko has joined #openstack-ansible08:14
*** aruns has quit IRC08:16
*** chyka has quit IRC08:17
*** aruns has joined #openstack-ansible08:20
*** aruns__ has quit IRC08:22
*** admin0 has quit IRC08:26
*** DanyC has joined #openstack-ansible08:34
*** bhujay has joined #openstack-ansible08:35
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Developer docs refactor
*** aruns has quit IRC08:49
*** bhujay has quit IRC08:49
*** aruns has joined #openstack-ansible08:54
niraj_singhevrardjp: masakari also need pacemaker to install along with it. So can we use this role ref: for pacemaker?08:55
evrardjpyes you can re-use external roles, as long as they match our guidelines08:55
evrardjpthis one is not matching our guidelines though08:56
niraj_singh is this one08:56
evrardjpit doesn't look multi distro08:56
evrardjplet me check that one08:56
evrardjpmaybe worth checking with redhat or suse, they may have something around08:57
evrardjpbut yes you can reuse one08:57
evrardjpmaybe use the first one at the beginning08:57
evrardjpso that you have something to start with08:58
niraj_singhyes :)08:58
evrardjpand think about how it can be re-used08:58
evrardjpplease adapt the spec08:58
evrardjpnow you have an external dependency08:58
evrardjpand new roles that need to be applied, and things like that08:59
niraj_singhooh yes. i will do that.08:59
niraj_singhi also updated github repo: this branch include masakari-api and masakari-engine installation part.09:00
*** vnogin has joined #openstack-ansible09:00
*** cjloader has joined #openstack-ansible09:06
*** aruns has quit IRC09:07
*** aruns has joined #openstack-ansible09:08
*** gkadam has joined #openstack-ansible09:09
*** kukacz_ has joined #openstack-ansible09:10
*** gkadam_ has joined #openstack-ansible09:10
*** bhujay has joined #openstack-ansible09:10
*** cjloader has quit IRC09:11
*** gkadam has quit IRC09:13
*** hamza21 has quit IRC09:19
*** bhujay has quit IRC09:27
*** bhujay has joined #openstack-ansible09:27
*** yifei1 has quit IRC09:32
*** kukacz_ has quit IRC09:41
*** kukacz_ has joined #openstack-ansible09:42
*** stuartgr has quit IRC09:43
*** stuartgr has joined #openstack-ansible09:43
evrardjpniraj_singh: so you went for a single repo doing both things at the end? It's fine for me, just wondering what is the separation09:45
evrardjpI guess I will re-read the spec as soon as I can :)09:45
niraj_singhevrardjp: sorry but i am not clear about the sigle role for both masakari and masakari monitors till now. Discussion going on here.09:46
niraj_singhbut the depepndencies is like this:    Masakari-monitors should be installed after masakari is installed as it monitors compute hosts and submits notifications to masakari. And the sequence will be same in upgrade case.09:47
niraj_singhevrardjp: till now i didn't updated the specs. i will do that as per decision.09:48
odyssey4meevrardjp do you think we should revert for now until we figure out build/storage delegation?09:49
*** kukacz_ is now known as kukacz09:52
odyssey4meevrardjp another option could be to try and change the integrated build to use online constraints instead of the repo build constraints, at least for now - we can evolve it later again... but given that the patch hasn't merged yet, we'd end up having to combine things which isn't great09:56
odyssey4mewhat's weird, though, is that the package it's looking for was in the repo :/09:57
evrardjpodyssey4me: that's exactly what I told you in the past, sometimes it might look like it's working but fetches things outside which sometimes fail09:58
evrardjpbut I haven't any evidence, so...09:58
evrardjpit's a large body of work now09:59
evrardjpthe revert will be hard, isn't it?09:59
odyssey4menope, just one merge so far09:59
evrardjpthen it's maybe wise indeed09:59
evrardjpyou -2 the others?10:00
odyssey4methey're all -w already10:00
evrardjpwhat if someone adds a +w ?10:00
evrardjpthe -w wins?10:01
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Revert "Use a common python build/install role for source-base deployment"
evrardjp-2 is explicit though10:01
odyssey4meI can -2 them if you'd feel more comfortable10:01
evrardjpbut ok10:01
evrardjpno that's fine, if nobody rebases will be good10:01
evrardjpso we have to think about what went right and what went wrong?10:01
evrardjpI'd like to finish what I am busy with, and then I can help you with that10:02
*** cjloader has joined #openstack-ansible10:06
*** cjloader has quit IRC10:11
odyssey4melogan- cloudnull I see that went in, but no explanation for the revert10:14
odyssey4meevrardjp it looks like all the nova upgrade tests are pretty much broken due to the same issue with UCA10:22
odyssey4methis is an ocata->pike:
odyssey4meI may as well make them all non-voting for now until we work out how to improve the tests10:22
evrardjpGRRRRRRRRRRRRRRRR I just did git checkout -- filename1 .10:22
evrardjpinstead of git checkout -- filename110:23
evrardjpall my work10:23
odyssey4mewow, that was a fail10:23
*** savvas has joined #openstack-ansible10:23
evrardjpat least I have the generated html10:24
evrardjp**** **** ****!10:24
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Switch upgrade test to non-voting
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Switch upgrade test to non-voting
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: Omit default_schedule_zone if its not configured
*** hamza21 has joined #openstack-ansible10:26
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: change python-libguestfs to python-guestfs for ubuntu
*** haad1 has quit IRC10:27
openstackgerritMerged openstack/openstack-ansible stable/queens: [Docs] Simplify overview page
openstackgerritMerged openstack/openstack-ansible stable/queens: [Docs] Adapt the wording for disk requirements
openstackgerritMerged openstack/openstack-ansible stable/queens: [Docs] Uniform image on top
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer master: Convert role to use a common systemd service role
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_aodh master: Convert role to use a common systemd service role
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_barbican master: Convert role to use a common systemd service role
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Developer docs refactor
*** Sha0000 has joined #openstack-ansible10:44
evrardjpodyssey4me: oh you said ALL the nova upgrade issues are UCA issues, due to the way we do things?10:45
odyssey4menewton->ocata is already non-voting due to the issue, as is pike->queens... now ocata->pike is showing the same issue10:50
odyssey4mequeens->master use the same UCA for now, so that's still voting10:50
odyssey4meI'll have to find some time to progress so that we can use that for upgrade tests in roles10:51
*** haad1 has joined #openstack-ansible10:53
*** Sha0000 has quit IRC10:55
fghaasodyssey4me, evrardjp: if one of you has thoughts on the container question I mentioned earlier, I'd be grateful for them — whenever you can spare the time of course, no rush.11:02
odyssey4mefghaas ah yes, the container removal is left up to the deployer - although quite honestly it could be automated into I think11:04
odyssey4mewe just haven't had anyone pick that work up11:04
odyssey4mecare to volunteer? ;)11:04
*** gunix has left #openstack-ansible11:05
fghaasokay but the expectation is definitely that the containers are safe to remove? If so, I would have volunteered for adding that to the Queens release notes, at least — until it's been rolled into a script or playbook.11:05
odyssey4mehmm, weird - I would have sworn that there was a reno for this11:05
* odyssey4me hunts down the commit11:06
fghaasI'm looking at — wrong place perhaps?11:06
odyssey4melooks like the first had no reno, so yeah - if you could put together a reno for that it'd be great11:07
*** ansmith has quit IRC11:07
fghaasYeah those only talk about Neutron agents though :)11:08
odyssey4mewe could figure out a run_upgrade sequence for the clean up too - that's supposed to have been done by cloudnull but hasn't happened11:08
fghaasOkay so the other services affected are Nova, Cinder, Trove, and Heat? Any others?11:09
odyssey4meneutron_agents (not server)11:09
odyssey4meironic_conductor -> ironic_server11:10
*** DanyC has quit IRC11:10
odyssey4methere may also be entirely new containers, where old api containers can be removed11:10
*** DanyC has joined #openstack-ansible11:10
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Update the old http links in docs
odyssey4meeg I think there's a new nova container whichi replaces all the old ones11:11
odyssey4meit looks like it converged to a standard default set of <service>_api_container11:12
*** DanyC has quit IRC11:15
evrardjpI have to think about planning, what do we need to land first. I am stretched in too many places, and I want stuff included before M111:25
openstackgerritMerged openstack/openstack-ansible-haproxy_server master: Options to enable SSL for backend connections
logan-fghaas odyssey4me: ya without the cleanup ive seen a huge spike in memory usage after upgrading to queens due to 2 nova api/placements, etc. running on each host. aside from the container cleanup you'll also need to cleanup all of the service agents in openstack also11:30
logan-odyssey4me: regarding the revert, cloudnull and I were looking into the failures on where it was failing after setup-hosts but before setup-infra11:32
fghaas logan- yep, true, unless you're cool with seeing a lot of down services/agents11:33
logan-if you scroll to the bottom of, you can see there is an a connection issue with the containers that causes fact collection to fail11:33
*** chyka has joined #openstack-ansible11:34
logan-fghaas: ya for the most part that is the extent of the impact, but its a little worse with neutron because I saw it scheduling extra dhcp agents and HA routers on all of these agents11:35
odyssey4melogan- aha, so that was the culprit - wonder why11:35
odyssey4melogan- an explanation in the commit message next time is a useful reference ;)11:36
logan-odyssey4me: ya not sure, interesting that it works in the plugins gate but breaks integrated. will have to do some more debugging on it and figure it out.11:36
*** hamza21 has quit IRC11:37
odyssey4melogan- fghaas it'd be really good to put together the steps to do all the clean up and get it documented in the upgrade guide11:37
*** hamza21 has joined #openstack-ansible11:38
odyssey4meI'd be happy to actually put the patch together if we could perhaps crowd-source an etherpad with the steps11:38
fghaaslogan-: you mean you were afflicted by the fact that the Neutron agents lived in a container on the network node, and now no longer do? That one issue doesn't touch me, because I've always overridden that and had the server container on the shared infra host, and the agents container-less on the gateway nodes. :)11:38
odyssey4meperhaps you guys could put together a quick off-the-cuff set of notes now?11:38
logan-correct fghaas, no worries on that then.11:38
*** hamza21 has quit IRC11:39
*** chyka has quit IRC11:39
*** hamza21 has joined #openstack-ansible11:39
fghaasodyssey4me: I'm currently in the process of updating our training course for the Queens upgrade, when the lab instructions in that are done I can C&P those into an etherpad and you can grab what you'd like11:39
logan-i have to run atm, but can help w/ that later today11:39
fghaasOr, for that matter, I can just give you access to the course :)11:39
odyssey4meawesome, thanks fghaas :)11:39
odyssey4mehaha, that'd be awesome - happy to beta test it for you ;)11:40
fghaasYou just said that in front of witnesses11:40
fghaasSo I'm definitely taking you up on that11:40
odyssey4megiven that my openstack knowldge is a bit dated, I might learn some new tricks :)11:43
*** hamza21 has quit IRC11:43
*** hamza21 has joined #openstack-ansible11:43
* fghaas is waiting for mhayden to inject a "pfffft eyeroll c'mon" gif here11:45
*** udesale has quit IRC11:46
*** gkadam_ has quit IRC11:46
*** udesale has joined #openstack-ansible11:46
*** gkadam_ has joined #openstack-ansible11:47
*** armaan has quit IRC11:48
*** armaan has joined #openstack-ansible11:48
*** Sha000000 has joined #openstack-ansible11:52
*** strobelight has joined #openstack-ansible11:59
evrardjpthat's not a bad idea for a user refresh12:03
evrardjpwe need to know what they are doing with this thing we are building :p12:03
evrardjpodyssey4me: what do you think of having an entry in tox.ini for the integrated repo, as "functional", which would call gate-check-commit ?12:05
evrardjpthis way our only interface is run_tests12:05
evrardjpand ofc I am writing the docs and it sounds weird we have something different for the integrated repo12:06
evrardjpfor later...12:07
*** armaan has quit IRC12:11
*** armaan has joined #openstack-ansible12:11
*** udesale_ has joined #openstack-ansible12:14
*** DanyC has joined #openstack-ansible12:16
*** udesale has quit IRC12:16
*** throwsb1 has joined #openstack-ansible12:16
openstackgerritMerged openstack/openstack-ansible-os_octavia stable/queens: Update systemd service for longer TimeoutStopSec
*** udesale_ has quit IRC12:19
*** haad1 has quit IRC12:19
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Revert "Use a common python build/install role for source-base deployment"
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Developer docs refactor
*** haad1 has joined #openstack-ansible12:25
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance master: Revert "Use a common python build/install role for source-base deployment"
odyssey4meany cores around - this unblocks the pike os_nova gate:
*** strobelight_ has joined #openstack-ansible12:28
*** strobelight is now known as Guest2418912:28
odyssey4mealso, this helps unblock the integrated gate:
*** strobelight_ is now known as strobelight12:29
*** tnogisto2 has joined #openstack-ansible12:32
*** Guest24189 has quit IRC12:32
*** tnogisto2 has quit IRC12:32
*** tnogisto2 has joined #openstack-ansible12:32
odyssey4meevrardjp using tox for the integrated gate doesn't work - I tried it before12:34
*** tnogisto has quit IRC12:35
odyssey4methat said, I'm working on a tool to read the zuul job config and spit out the things it specifies so that we can remove tox and just use the zuul job config as a single reference for test definition and running12:35
odyssey4meso basically it will be ansible all the way12:35
odyssey4meI have a very hacky tool done - I'm just working out the recursion bits to resolve the job parent-child hierarchy so that we have a full data set for each job.12:36
evrardjpI will just mention the current state in our docs, and hope we'll not forget to update the docs12:36
evrardjpif you need help, don't hesitate12:36
odyssey4meja, I'm trying to just hack it for now - once I have something working I'll push up a patch for iteration12:37
odyssey4methat's my weekend fun with python hobby12:37
*** haad1 has quit IRC12:38
*** electrofelix has quit IRC12:46
*** haad1 has joined #openstack-ansible12:53
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Developer docs refactor
*** odyssey4me has quit IRC12:57
*** odyssey4me has joined #openstack-ansible12:57
evrardjpok when and ^ will pass, the documentation would be fully rewritten as wizard like stuff, and should help on the onboarding. All the usual questions are now answered.12:58
*** ansmith has joined #openstack-ansible12:58
*** niraj_singh_ has joined #openstack-ansible13:00
odyssey4melooks like still needs work13:00
*** niraj_singh_ has quit IRC13:01
odyssey4meevrardjp I see that newton-eol is now done in the roles - time for a final sha bump in the integrated repo?13:05
evrardjpI planned to do this with the rest on friday13:06
odyssey4meah ok, that'll do13:06
*** kstev has joined #openstack-ansible13:09
evrardjpodyssey4me: that's why I didn't want to do the synchronize and copy tasks13:10
evrardjptalking about the two patchsets for delegations13:10
evrardjpif the venv building is just about building we can point to any host13:11
evrardjpthen we know it's producing a result on THAT host13:11
evrardjpthe orchestration should be done elsewhere13:11
odyssey4mesure, but we also have to get the venv to the target host - so somehow that has to be copied13:11
evrardjpto not mess up with complex things like we do13:11
evrardjpyes that's fine, let's say that we include this in the role, but don't put that into the role execution path13:12
*** kstev1 has joined #openstack-ansible13:12
evrardjpwe can have another series of tasks for fetching, and another series of tasks for pushing13:12
evrardjpso instead of calling the role once, we call it three times13:12
*** savvas has quit IRC13:12
evrardjpdelegation would be done not in the role, which starts to be messy, but at the orchestration level (the playbook)13:13
evrardjphere it would still be a role though, because it's a role that calls the role.13:13
odyssey4meso you're suggesting that build just builds it on whichever host, then install either downloads it from a url or copies it from the deploy host (whichever option is enabled)13:13
*** savvas has joined #openstack-ansible13:13
*** kstev has quit IRC13:13
evrardjpI'd hope it would be possible to merge those last two actions into one13:14
evrardjpdeploy host being an url with ssh:// or whatever13:15
odyssey4mewell, that can be an option - but it doesn't have to be - they're two different methods of serving the file13:15
evrardjpyeah but we can do it in a smart way, and this is far simpler to understand?13:15
odyssey4mescp/sftp would require openssh on the host serving it, which I don't think we should *require*13:15
evrardjpmaybe we should discuss the approach with other ppl?13:16
odyssey4mebut yeah, I was thinking that we could do something like that instead13:16
evrardjpI like the idea of "I build there", "I fetch from there"13:17
evrardjpwhich is simple13:17
evrardjpbut you have to split the thing in two13:17
evrardjpI should have said13:17
evrardjp"I build here", "I fetch from there"13:18
evrardjpso splitting in two would allow an easy definition of "here" and "there"13:18
odyssey4mewell, regardless of whether it's a single path or two - it can still work in a single path as long as there're two actions ultimately13:18
evrardjpI guess that if the venv build is done inside the role call, we'll have no choice than to have variables to define the target.13:20
*** hamza21 has quit IRC13:22
*** tnogisto2 has quit IRC13:23
*** savvas has quit IRC13:24
*** savvas has joined #openstack-ansible13:24
*** tnogisto has joined #openstack-ansible13:25
odyssey4mewell, clearly what I was trying to do isn't going to work - I was hoping to distribute the build process out to target nodes, but have the results collected back to a designated storage host13:27
odyssey4meI thought that this was a good way of ensuring that we didn't have to have multiple build hosts for each distro/arch combination13:28
*** esberglu has joined #openstack-ansible13:28
odyssey4meand also to distribute the computing power for the builds13:28
odyssey4meusing synchronize just doesn't work through when delegating13:28
odyssey4menor does copy/fetch13:28
evrardjpmaybe we are trying to solve too many things with ansible13:29
*** niraj_singh has quit IRC13:31
*** jwitko has joined #openstack-ansible13:34
evrardjpdelegate the build with a var is quite simple to understand13:34
evrardjplet me analyse these two patches once again13:35
*** aruns has quit IRC13:39
evrardjpI just don't get cloudnull 's patch. Maybe I am dumb.13:40
*** ibmko has quit IRC13:40
evrardjpcloudnull: why the venv_pip_build_target == inventory_hostname ?13:40
evrardjpI'd say the other way around, you have to sync when you don't have it yet13:41
matttodyssey4me evrardjp : should these SHAs not match?
mattti'm noticing differing behaviours running tempest as part of integrated repo versus as a role functional test13:43
evrardjpwe should probably remove one of those13:43
odyssey4memattt the test-vars value will override the default13:43
evrardjpmattt: the tempest sha in the integrated is in defauts/ blablabla13:44
odyssey4memattt and the integrated build will use
odyssey4meso yes, ideally they should match to the right one13:44
evrardjpwell once might wonder why we have both set, if we can just have it one in defaults :)13:45
odyssey4mewe pin the sha for stable branches so that changes upstream don't affect the stable branches as they progress development on master13:45
odyssey4meevrardjp the one in test-vars is so that all other role tests get the same, but yeah I guess we could remove it and let the defaults carry13:45
evrardjplet me remove it from the tests repo see what breaks13:45
odyssey4mein fact, I think that makes sense to do13:46
evrardjpSometimes I make sense!13:46
odyssey4meonly sometimes :p13:46
odyssey4mehopefully quite soon we'll be able to just define the sha's used in roles - and the integrated repo values will be optional13:47
odyssey4methen it'll be much, much simpler13:47
matttoh shoot, forgot about that other SHA!13:47
*** gameon has joined #openstack-ansible13:48
*** indistylo has joined #openstack-ansible13:50
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-tests stable/queens: Remove os_tempest defaults
*** gameon has quit IRC13:53
evrardjplet's see what breaks and fix it case by case ^13:53
mattti still definitely have different versions of tempest installed tho13:53
*** udesale has joined #openstack-ansible13:53
evrardjpyour host is used for different builds?13:53
evrardjpbecause a search before patching show a tangled mess of overrides13:54
matttnope, separate AIOs, one for integrated build and one for functional test of os_tempest role13:54
evrardjpand your os_tempest got 2 venvs?13:55
evrardjpor did I misunderstood you?13:55
matttintegrated: /openstack/venvs/tempest-17.0.0/lib/python2.7/site-packages/tempest-18.0.0.dist-info functional: /openstack/venvs/tempest-17.0.0/lib/python2.7/site-packages/tempest-18.0.0.dist-info13:56
matttlet me try that again :)13:56
openstackgerritMerged openstack/openstack-ansible-os_nova stable/pike: Switch upgrade test to non-voting
matttintegrated: /openstack/venvs/tempest-17.0.0/lib/python2.7/site-packages/tempest-18.0.0.dist-info functional: /opt/tempest_70ebe69cc2e4fa8a06071d48d19f61d2f01a33f6/lib/python2.7/site-packages/tempest-17.2.1.dev229-py2.7.egg-info13:56
*** savvas has quit IRC13:57
evrardjpfor the path, it's all about the venv tag, which can mean nothing13:58
evrardjpfor the integrated, we use the release version13:58
evrardjpwhich is more explicit13:58
evrardjpfor what's inside the venv, that depends on the defaults/ tempest sha13:59
*** udesale has quit IRC13:59
evrardjpand therefore I guess you have precisely the queens version13:59
evrardjpfor the functional test, you had the override which be a little older13:59
evrardjpbecause forgot to be bumped13:59
evrardjpso it means you have a version that's got 229 commits since last tag.14:00
evrardjp(at least to my knowledge of setuptools)14:00
*** savvas has joined #openstack-ansible14:00
matttok cool14:01
matttso that patch you put up should bring them in line then14:01
odyssey4meI think we should probably try to converge on using released versions of tempest by default, and having the git sourced build be an opt-in14:01
odyssey4mewe're not able to follow tempest changes closely enough to warrant always building from git source14:02
evrardjpodyssey4me: I think we can use a tempest tag instead of a tempest sha14:02
evrardjpfor the role defaults14:02
*** dave-mccowan has joined #openstack-ansible14:02
odyssey4mewhy bother using a tag when we can install from pip ;)14:02
matttit's funny, because in that slight discrepancy tempest went from using testrepository to stestr, and my role changes didn't work as expected but did in the integrated build14:03
matttso i was kinda confused14:03
odyssey4mealso, u-c can manage the constraint for us:
*** ibmko has joined #openstack-ansible14:03
*** armaan has quit IRC14:03
odyssey4mesee also
evrardjpodyssey4me: I am not sure where we differ :p14:04
evrardjpwe are already installing with pi14:04
odyssey4meevrardjp I'm saying that we make the use of a git source entirely optional fir the tempest role - we just let u-c take its course14:05
evrardjpoh no version at all?14:06
evrardjplet's talk with code, shall we?14:06
odyssey4meyes, we inherit a version from u-c so we don't need to specify it14:06
odyssey4meok, lemme propose something and we take it from there14:06
matttnice, dropping the lingua franca14:07
*** savvas has quit IRC14:07
matttevrardjp: re-running my test with your patch applied14:08
*** savvas has joined #openstack-ansible14:08
evrardjpodyssey4me: keep in mind we need the version of tempest to be able to work with plugins, but that should be alright. But there is a risk of reduced coverage if we are setting a moving target14:08
evrardjpmattt: it will probably break :p14:08
evrardjpbut let's see14:08
evrardjpalso you'd still have a difference14:09
matttevrardjp: shush, it's going to be ok14:09
evrardjpyou should add a patch on os_tempest14:09
evrardjpto bump its version14:09
evrardjpor wait for odyssey4me 's patch14:09
odyssey4memattt go ahead and make changes to the queens branch to figure out the right sha14:10
odyssey4mewhatever I do will have to go through some motions and get agreement, which takes time14:10
evrardjpfair point14:10
evrardjpI'm also changing tempest, so it might be good working all together.14:10
evrardjpTeam team team!14:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: [TEST] Disable developer mode for tempest
evrardjpodyssey4me: we seemed to be in disagreement, but your commit message is exactly what I thought we should be doing.14:15
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: [TEST] Disable developer mode for tempest
evrardjpLater we can just trim the content of the role14:16
odyssey4meI don't think we should get rid of the ability to use a later tempest - I just think our default should not use that mode.14:16
cloudnullevrardjp: [08:40] why the venv_pip_build_target == inventory_hostname? -- what questions do you have?14:16
evrardjpcloudnull: well -- I simply don't understand that condition14:17
evrardjplet's give 2 use cases14:17
evrardjp1) you run everything on your metal node.14:17
evrardjp2) you have a repo machine where you want to build14:17
evrardjpcase 1) venv_pip_build_target == inventory_hostname but you don't want to sync14:17
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [TEST] Disable developer mode for tempest
evrardjpcase 2) venv_pip_build_target != inventory_hostname and you want to sync14:18
evrardjpwhat do I get wrong?14:18
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [TEST] Disable developer mode for tempest
cloudnullthats in reverse14:18
odyssey4mecloudnull mind reviewing this quick?
cloudnullif venv_pip_build_target == inventory_hostname sync the bits14:19
evrardjpwhy ?14:19
evrardjpthat's the part I don't get14:19
evrardjpmaybe we are talking about two different things14:19
cloudnullif the build target is the inventory host name your building everything on your target host.14:20
evrardjpwhy would I want to synchronize on something I already have?14:20
cloudnullthats whats there today14:20
cloudnullIf i change the build target I dont need to sync builds will always happen elsewhere14:20
evrardjpmaybe but that's the part I don't get14:20
evrardjpbut why then?14:21
odyssey4meevrardjp no need to sync if the build target is always the same14:21
cloudnulltheres no need to push and pull bits between the deployment host and the target hsot14:21
evrardjpBecause the install then needs the venvs, so syncing should happen14:21
openstackgerritMerged openstack/openstack-ansible-os_nova stable/pike: Omit default_schedule_zone if its not configured
cloudnullthe build will always happen elsewhere14:21
odyssey4mewell yes, the venv is needed for the install - that should sync, not the wheels14:22
cloudnullthat part remains14:22
evrardjpit's just for wheels14:22
evrardjpterrible naming14:22
evrardjpthat's very confusing14:22
cloudnullits the wheels and venvs14:22
odyssey4mecloudnull what I don't get then, is why not use the playbook to target a build host, and do build only (there's a flag for it)14:22
evrardjpthis is harder than what it should14:23
odyssey4methen later target the various hosts and it should just do the install14:23
odyssey4meif we do that, there's no need to sync wheels at all14:23
odyssey4mewe go back to the simplicity14:24
odyssey4meright now we're trying to do orchestration in a role, which never ends well14:24
evrardjpagreed with odyssey4me there14:24
cloudnulltotally agree.14:24
cloudnullbut the proposed method is too slow14:25
evrardjpahah we all agree that we wrote bad things, let's make the world better with more bisounours14:25
cloudnullsync'ing between the target and the deploy host is painfully slow14:25
odyssey4meok, so shall we pull the wheel sync out?14:25
*** SimAloo has joined #openstack-ansible14:25
cloudnulland we are building the same bits on every run14:26
odyssey4meso that's a problem for the integrated build, yes14:26
cloudnullits really no change for the role tests given the use of developer mode14:26
odyssey4meso what I can do it break this into parts - and we don't merge a things until everything affecting the integrated build is ready14:27
*** armaan has joined #openstack-ansible14:27
cloudnullbut running this code on my 5 node hardware lab shows its not viable in production14:27
odyssey4mewe can perhaps reduce the integrated build test down to the base kit for all platforms for now, to make it easier to get this in and remove the repo build14:28
odyssey4methe problem here is that we're trying to run before we walk, so I think we should get all the pieces on the table before we merge anything14:28
evrardjpwhat if...14:28
cloudnulli think thats the problem, the repo build is a lot faster than this14:28
evrardjpwe buid a systemd service that does an inotify service to rsync between all the nodes of a same group?14:29
odyssey4mewe can do exactly the same in a different way by executing the repo-build in each service role playbook14:29
cloudnullhappy to take a performance hit for better usability but not one that extends a deployment by hours14:29
evrardjpwe build on the first node, and all the time it's synced.14:29
evrardjpand we don't have to care about orchestration14:29
odyssey4mecan we get this revert done? then give me time to get all the proposed pieces on the table14:30
cloudnullevrardjp: first node of a particular distro and arch14:30
odyssey4meI'll hold back any further stuff that touches the integrated build14:30
evrardjpcloudnull: what do you think of that? completely different, and will scale easily14:31
odyssey4mewe *could* merge this one as a stop-gap, and have the integrated build use the deploy host as a build host for starters:
evrardjpbecause if it doesn't scale, we can just change the tech to be better than rsync14:31
odyssey4methe wheel sync sucks, let me pull that out again and use playbooks for orchestration14:33
odyssey4methe role should just simply build/deploy the venv - we keep it at that, it'll build the wheels to help speed things up - we then use a playbook to decide where to build the venvs14:37
odyssey4meapologies, I should have actually realised that up front instead of wasting time on trying to orchestrate inside the role14:38
odyssey4me<--- dumbass14:38
*** cjloader has joined #openstack-ansible14:38
evrardjpodyssey4me: I said that in the past :p14:39
evrardjp(you pick on what it applies)14:40
evrardjphint: all of the above.14:40
odyssey4meyes, yes you told me so14:40
odyssey4meyou can thanks me later, now that you've got that picture in channel and no-one's talking :p14:46
*** hamzy has quit IRC14:46
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [Docs] Restructure inventory documentation
evrardjpodyssey4me: answered on
evrardjpshould I ping the whole world with a link to that giffy?14:50
*** bhujay has quit IRC14:56
*** chyka has joined #openstack-ansible14:58
*** chyka has quit IRC15:02
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Make docs be container agnostic
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Make docs be container agnostic
evrardjpI see there is activity in nova for the cells things15:04
*** MikeW has joined #openstack-ansible15:16
openstackgerritMerged openstack/openstack-ansible-os_glance master: Revert "Use a common python build/install role for source-base deployment"
*** hamzy has joined #openstack-ansible15:22
*** haad1 has quit IRC15:27
*** chyka has joined #openstack-ansible15:33
*** indistylo has quit IRC15:36
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [Docs] Restructure inventory documentation
*** armaan has quit IRC15:38
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Revert the use of wheel synchronizing
odyssey4mecloudnull evrardjp ^ as discussed - we pull the wheel sync bits out15:40
odyssey4menow, one more feature - add the ability to source the venv from a URL instead of from the deploy host15:41
odyssey4methen I think we're ready15:41
evrardjpI will have to take a step back to think about if all is right15:42
evrardjpbut I think it's better that way15:42
odyssey4meit does make it easier to transition, so I think this is a good approach15:43
*** haad1 has joined #openstack-ansible15:47
*** haad1 has quit IRC15:47
*** Sha000000 has quit IRC15:48
evrardjpodyssey4me: I think I have an idea on how to make it even more readable15:52
*** cjloader has quit IRC15:52
evrardjpthe python_venv_build should be triggered by a boolean, which can be set in the playbook. By default it would be true on first node of the targetted group + arch + distro15:53
evrardjpit's exactly the same thing15:53
odyssey4meit is already doign a build on the first node15:53
evrardjpbut the condition is placed elsewhere15:53
odyssey4methe boolean just forbids it from doing the install part15:53
evrardjpyeah it's exactly the same code15:54
evrardjpI am thinking as an external user right now15:54
odyssey4meI think that whole bit might just come out later, because it's probably not needed15:54
odyssey4meif you target at one host, it only does the build15:54
evrardjpOr alternatively maybe a comment15:54
odyssey4meso there's no real need to have a boolean to prevent the install15:55
evrardjpthat's not what I meant :)15:56
evrardjpIt's just a bonus for later anyway15:56
evrardjpwe have the whole cycle to figure this out15:56
*** RandomTech has joined #openstack-ansible15:59
*** vnogin has quit IRC16:03
openstackgerritMatthew Thode proposed openstack/openstack-ansible-specs master: create Designate back end provider spec
*** vnogin has joined #openstack-ansible16:05
*** MikeW has quit IRC16:10
*** vnogin has quit IRC16:10
*** openstacking_123 has joined #openstack-ansible16:11
openstacking_123Hope is well16:11
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Fix delegation to hosts not in inventory
openstacking_123Hitting an issue with with repo_build at Create OpenStack-Ansible requirement wheels on pike/stable16:13
*** savvas has quit IRC16:13
openstacking_123Mostly related to markers 'python_version == \"3.5\"' don't match your environment and Could not find a tag or branch16:14
RandomTechevrardjp: It does not seem adding compute-infrastructure_hosts orchestrastion_hosts and image_hosts fixed the issue16:15
cloudnullodyssey4me: evrardjp: sorry been in meetings, and moving to new meetings. will be back in a while.16:16
openstacking_123I put full error here if anyone has time to give a quick look. Thank you16:16
*** savvas has joined #openstack-ansible16:19
evrardjpopenstacking_123: could you try without gnocchi?16:22
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_tempest master: update for changes needed to pass refstack v2
prometheanfireevrardjp: that should clean up the image reuploading16:23
*** savvas has quit IRC16:24
logan-openstacking_123: cloudnull actually just hit this yesterday, and I think he has a fix up for review, but basically as a workaround for now try running: ansible pkg_repo -m pip -a 'name="pbr==3.1.1" state=present'16:26
logan-and then run repo build again16:26
logan-gnocchi uses pbr to build its wheel, and pbr just released 4.0.0 a few days ago, which seems to break the gnocchi build.16:27
openstacking_123Thanks guys!16:27
openstacking_123Thanks guys this a tremendous help. Will give this a try and report back16:28
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_build stable/queens: Install PBR to ensure we're not breaking builds
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_build stable/pike: Install PBR to ensure we're not breaking builds
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_build stable/ocata: Install PBR to ensure we're not breaking builds
cloudnullopenstacking_123: ^ those are the cherry-picks for the issue16:33
openstacking_123thank you!16:34
cloudnullif folks can give,n,z a shove through it'll help our stable releases16:35
logan-i still wonder how we're getting pbr in there without constraints :/16:36
*** savvas has joined #openstack-ansible16:37
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Fix delegation to hosts not in inventory
*** savvas has quit IRC16:42
*** savvas has joined #openstack-ansible16:47
*** armaan has joined #openstack-ansible16:47
throwsb1Hey, I am building a fresh install and I am using the latest 16.09 and the deploy is dying on infrastructure setup playbook on repo server.  I am not sure if there is a python mis-match version.
throwsb1I am running on ubuntu
logan-throwsb1: we were just discussing that a few minutes ago.. see the convo here:
odyssey4mecloudnull done16:50
cloudnulllogan-: ive tested this a couple times now, it doesn't look like new PBR is being pulled in, so we're not getting pbr 4.x on the host (the previous test may have been polluted from me doing other things) it looks like PBR is just not there which is causing the issue in gnocchi.16:51
cloudnullim still not 100% on why this is the case16:52
logan-but this is in the repo container not the host right?16:52
cloudnullbut I can say installing pbr 3.1.1 on the repo container fixed the issue for me16:52
logan-im just concerned about future breakage now16:52
*** savvas has quit IRC16:52
logan-because something is messed up if we're seeing pbr 4 installed in a pike repo container at any point imo16:52
throwsb1thanks locan16:52
odyssey4meit does worry me a bit that we having a chink in the constraints armor :/16:52
odyssey4meI guess this is part of what's driving the simplification work - right now we have this all happening in too many places... it'll be more transparent and easier to work out when the work is done.16:53
openstacking_123cloudnull I  already started with @logan  pip install fix. It worked great16:54
cloudnullso we should have that fix in the branches asap16:54
odyssey4melogan- cloudnull evrardjp I guess, given that we're getting combacks on it - we'll need to pull those pbr fixes into the next release... otherwise the release will likely be useless16:55
cloudnullwe might need evrardjp to cut a release16:55
odyssey4merelease is being proposed tomorrow, so we should get it merged and merged into the integrated build asap16:55
*** ibmko has quit IRC16:55
logan-ya will need to get integrated a-r-r bumps up right after they merge16:55
*** savvas has joined #openstack-ansible16:56
*** savvas has quit IRC17:05
*** savvas has joined #openstack-ansible17:05
*** gkadam__ has joined #openstack-ansible17:09
*** gkadam has joined #openstack-ansible17:11
*** gkadam_ has quit IRC17:12
*** RandomTech has quit IRC17:13
*** gkadam__ has quit IRC17:15
*** Sha000000 has joined #openstack-ansible17:26
evrardjphey sorry I was busy -- will read the scrollback17:30
evrardjpI can wait before releasing that this goes in17:30
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Allow a venv to be downloaded from a URL
evrardjpso there is a package (pbr) that wasn't explicitly listed?17:31
evrardjpisn't pbr constrained and things like that? I thought that all the OS packages including us were using it17:32
evrardjpoh I understand17:32
odyssey4meevrardjp cloudnull ^ adds the ability to install venvs from a URL - I'll now work on a set of patches for the integrated build17:32
*** openstacking_123 has quit IRC17:32
evrardjpcloudnull: odyssey4me I don't think we should backport that far back but it doesn't hurt17:34
odyssey4meevrardjp we've had reports of brokenness for pike & queens17:34
evrardjpI'm surprised we had to pull cython17:34
odyssey4methe original problem was discovered on pike17:34
evrardjpPike/Queens make sense17:34
evrardjpwell it doesn't hurt and its better17:34
evrardjpis there a patch in the integrated repos?17:35
odyssey4me'pull' cython? it's just reordered alphabetically17:35
odyssey4menot yet, none of the pbr patches have merged17:35
evrardjpI just misread then17:35
odyssey4me(except master)17:35
evrardjpyeah good17:35
evrardjpodyssey4me: I see :)17:35
evrardjplet's bump master!17:36
openstackgerritShannon Mitchell proposed openstack/openstack-ansible-ops master: Apt-cacher-ng's default ubuntu mirror list has faulty mirrors
evrardjpLet's not merge risky patches until those get in17:36
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Allow a venv to be downloaded from a URL
evrardjpthe docs build are flaky17:38
evrardjpsometimes using version x of openstackdocstheme , sometimes x+117:38
odyssey4meprobably also due to the pbr changes17:38
evrardjpwhen was this pbr released?17:39
odyssey4mea day or two ago17:39
evrardjptwo days ago17:39
evrardjpok that makes sense17:39
evrardjpyeah I saw that for two days now17:39
*** hamza21 has joined #openstack-ansible17:39
*** chyka_ has joined #openstack-ansible17:41
evrardjpwell maybe more17:41
evrardjpfirst iteration was 23/0317:41
evrardjpso no it's something different17:41
evrardjpwondering if zuul docs jobs is respecting our requirements17:42
*** armaan has quit IRC17:42
evrardjphaha no that's not that17:42
evrardjpGot it!17:42
evrardjpI will fix it17:42
*** armaan has joined #openstack-ansible17:43
*** chyka has quit IRC17:43
openstackgerritMerged openstack/openstack-ansible-repo_build stable/ocata: Install PBR to ensure we're not breaking builds
*** Sha000000 has quit IRC17:49
*** vnogin has joined #openstack-ansible17:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/ocata: Update repo_build role for PBR fix
*** fghaas has left #openstack-ansible17:52
odyssey4meevrardjp logan- cloudnull ^17:52
*** poopcat has joined #openstack-ansible17:56
odyssey4meevrardjp wtf?
*** osnaya has joined #openstack-ansible17:59
*** flemingo has joined #openstack-ansible18:00
*** gkadam has quit IRC18:01
shananigansAny way I can get this one looked at today?  Its stopping the mnaio builds during the preseed kick.18:01
evrardjpodyssey4me: opening -- just had dinner real quick18:01
*** vnogin has quit IRC18:02
evrardjpodyssey4me: yeah18:02
evrardjpa new version of openstackdocs theme is out18:02
evrardjpwhich breaks many things18:02
evrardjpit's flaky18:02
evrardjpsometimes an old version is used18:02
evrardjpbecause we don't constrain it18:02
evrardjpso master will sometimes break if he finds the 1.2018:03
evrardjpI will introduce a fix asap18:03
evrardjpthat's what I said abov18:03
odyssey4meevrardjp u-c does though:
osnayamy understanding is: when you run openstack-ansible setup-hosts.yml --syntax-check (it only generates openstack_inventory.json and openstack_hostnames_ips.yml with container addresses...(doesn't run any code on the target hosts, right?)18:04
evrardjpodyssey4me: it constraints to 1.20 for master18:04
evrardjpso it can install a bad version18:04
evrardjpdepending or not if he finds 1.2018:04
evrardjpif 1.18.1 is the only one found on the mirror, then it's working18:04
evrardjpI will do a compatibility shim18:04
evrardjpjust finishing dinner real quick18:05
odyssey4meosnaya if you just want to run the inventory, just run the python script itself: /opt/ansible-runtime/bin/python /opt/openstack-ansible/.../dynamic_inventory.py18:05
evrardjposnaya: (and odyssey4me ) this is documented in our reference18:05
evrardjpinventory section18:05
evrardjpoh sorry this patch hasn't merged yet18:06
odyssey4meevrardjp but uses constraints, so it *must* find the version set in constraints or it will fail... unless that's not being used18:06
evrardjpit's somewhere in our docs!18:06
*** flemingo_ has joined #openstack-ansible18:07
evrardjpthat was the patch to use 1.2018:07
evrardjpas you can see there, it failed18:08
evrardjplet me double check if I wasn't too tired on this patch18:08
*** flemingo has quit IRC18:09
ansmithhello, a quick tox question, I checked out openstack-ansible-os_cinder and when I run 'tox -e ansible-syntax' I get openstack-ansible-os_cinder/tests/common/ No such file or directory. Is there a step I need to run tox tests?18:10
ansmithwhen I check out openstack-ansible on its own, I can run tox fine.18:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/ansible-role-python_venv_build master: Allow a venv to be downloaded from a URL
*** sawblade_ has joined #openstack-ansible18:13
evrardjpodyssey4me: ok it seems to reliably use 1.2018:14
evrardjpon my machine I mean18:14
*** sawblade6 has quit IRC18:15
openstackgerritCorey Wright proposed openstack/openstack-ansible-pip_install stable/pike: Fix misspelled registered variable name in conditional
*** ivve has quit IRC18:28
odyssey4meevrardjp is fairly significant - I'm amazed that we didn't pick that up sooner18:29
evrardjpis that only for pike?18:29
evrardjpI see it's not a backport but significant enough18:30
evrardjplet me not +w18:30
odyssey4meevrardjp nope, once that merges it needs to go back to ocata too18:30
evrardjpqueens seems ok18:30
odyssey4methe same code is not present in queens, that's why it's not a backport18:30
evrardjpyeah I was checking in openstack-hosts18:31
evrardjpto see if that was there18:31
evrardjpbut it's not18:31
evrardjpit's different code, not copy pasta!18:31
odyssey4meok, good18:32
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [Docs] Fix docs for latest openstackdocstheme
evrardjpso that's very weird, the 23 this patch failed, but openstackdocstheme 1.20 was already released.18:35
evrardjpI should have rechecked earlier18:35
evrardjpI wonder if I shouldn't use a different version of openstackdocstheme18:39
evrardjpI do not agree with what they did18:40
evrardjpI will fix openstackdocstheme instead18:40
evrardjplet's go with this one18:40
*** esberglu has quit IRC18:43
evrardjpodyssey4me: could you merge this: 555748 ?18:44
openstackgerritMerged openstack/openstack-ansible-pip_install stable/pike: Fix misspelled registered variable name in conditional
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-pip_install stable/ocata: Fix misspelled registered variable name in conditional
*** ivve has joined #openstack-ansible18:46
throwsb1Hey, I am running the setup-openstack playbook and getting an unable to connect to DB.  I connected to the galera db and was able to connect to the mysql and was able to move around.  I also verified that I can ping keystone to galera with no issues.18:46
throwsb1here is the full error
openstackgerritMerged openstack/openstack-ansible master: [Docs] Fix docs for latest openstackdocstheme
evrardjpthrowsb1: could you check the status according to haproxy?18:48
evrardjpbefore going further18:48
evrardjpthrowsb1: also, which branch?18:48
throwsb1using systemctl it is showing active.  I am using 16.0.9.  Just pulled the latest and this is a fresh build18:49
throwsb1I did run into this with and thought it was due to a non-clean build, but this is the same error18:50
openstackgerritMerged openstack/openstack-ansible-repo_build stable/queens: Install PBR to ensure we're not breaking builds
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Disable developer mode for tempest role tests
*** Sha000000 has joined #openstack-ansible18:53
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Disable developer mode for tempest role tests
openstackgerritMerged openstack/openstack-ansible-os_aodh master: Convert role to use a common systemd service role
evrardjpthrowsb1: I am asking from the haproxy standpoint18:55
evrardjpI am thinking the healthcheck user may be busted for some weird reason when I see this message, but I need more facts18:56
throwsb1evrardjb: anything specific I could run to verify?  I am new to ha18:56
evrardjphatop -s /var/run/haproxy.stat18:56
evrardjpthat will be very helpful to you18:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Update repo_build role for PBR fix
throwsb1evrardjp: looks like galara-back is down.18:58
evrardjpthrowsb1: that's your issue18:58
evrardjpcheck on why in the logs18:58
evrardjpthere is obiously something wrong with one of your mysql of your cluster18:59
evrardjpthat's what you have to find out18:59
evrardjpby default it should work.18:59
evrardjpso it's probably something in your configuration18:59
evrardjpthrowsb1: could you show us your mysql version by any chance?19:00
evrardjp(just to make sure of something)19:00
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: [Docs] Change heading in target hosts
*** ivve has quit IRC19:07
throwsb1evrardjp: I am seeing layer4 connection refused19:07
osnayaok... genreated inventroy.json - going thru it - questions: compute1 shows up is_Metal=true and HAproxy shows isMetal=true. Rest of the OS services and Infra services created structure showing containers... For compute and HAProxy please confirm if that is expected?19:08
evrardjposnaya: ^19:08
evrardjpneutron agents are now in metal too19:09
evrardjpdepending on your branch19:09
throwsb1db version is maria 10.119:09
evrardjpthrowsb1: in more details?19:09
evrardjpwell layer 4 issue anyway :p19:09
throwsb1mariadb-server-10.1         10.1.30+maria-1~xenial19:09
ansmithevrardjp: regarding question ^, should be included in a role's tox.ini? (e.g. to setup tests/common)19:10
osnaya@evrardjp I am deploying pike... so I see inventory.json with a container entry for neutron-agent.... I guess in pike it was still container?19:13
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [Docs] Simplify docs configuration
evrardjpansmith: as of today, not anymore I'd say19:13
evrardjpansmith: you need to sync the run_tests.sh19:13
ansmithevrardjp: thanks, will resynch, I had clone yesterday19:14
evrardjposnaya: for pike that's normal to not have neutron agents as metal19:14
evrardjpansmith: are you developing your own role?19:14
evrardjpOr are you running into issues?19:14
throwsb1evrardjp: I am going to dig some more.  I had put in cidr in user vars for ha, which I thought was needed when specifying interfaces for ha.  I am going to remove them and see if that helps any.19:14
osnaya@evrardjp thanks for clarifying19:14
evrardjpthrowsb1: if you drop your user_variables and openstack_user_config maybe I will be able to help you19:14
evrardjpbut without that I cant do anything19:14
evrardjposnaya: yw19:15
ansmithevrardjp: I was starting to work on patchsets related to bp/hybrid-messaging that we discussed at ptg19:15
evrardjpoh great :)19:15
evrardjpjmccrory: ^19:15
ansmithevrardjp: it will require touching up the roles a bit19:15
evrardjpthe existing roles are currently merging this change of the tests-repo-clone19:15
evrardjpso you might be in the "in-between" state19:16
ansmithevrardjp: wanted to make sure I could test the roles out before changing things much19:16
evrardjpgood idea :)19:16
evrardjpmaster a moving target :)19:16
ansmithevrardjp: good to know19:16
evrardjpwhich role do you start with?19:16
evrardjpansmith: well we are generally stable19:16
evrardjpbut we've done major changes in the last days19:16
ansmithpatch to openstack-ansible and then roles will have a dependency on the change, wanted to do it in a way that there doesn't have to be a flag day to update the roles19:18
mhayden... major changes ...19:18
*** osnaya has quit IRC19:18
evrardjpdon't change too much19:20
spotzYou guys are quite chatty today!19:21
evrardjpansmith: don't hesitate to ping jmccrory which is closer to your timezone than I am19:21
evrardjpspotz: haha19:21
evrardjpI try to answer questions19:21
spotzmhayden!!!! I refuse to accept your resignation by the way:)19:22
evrardjpnot sure how good I am doing, but at least I got bonus points for extra trying19:22
spotzevrardjp: When I can't answer I just refer to who I think can:)19:22
mhaydenspotz: aww sorry19:22
* mhayden is neck deep in kernel land with openshift19:22
openstackgerritMerged openstack/openstack-ansible-repo_build stable/pike: Install PBR to ensure we're not breaking builds
spotzmhayden: Glad your hanging out there!19:23
ansmithevrardjp: thanks, will do19:23
spotzevrardjp: While you're still about I saw tbreeds email but git still has stable/newton?19:25
odyssey4melogan- FYI, I've thrown up an etherpad for notes on the P->Q upgrade - please add things that come to mind19:25
odyssey4mespotz the roles have the eol tags, but theintegrated repo still needs a final bump19:26
spotzodyssey4me: Should I hold off on using 3/30 as the EOL date? For some reason I've got tomorrow's date in the patch I've got prepped19:27
matttmhayden: sup buddy!19:27
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Update repo_build role for PBR fix
odyssey4mespotz I think there needs to be a final sha bump, then a release of that - so it might be another few days19:29
odyssey4meonce that final release is done, we can merge a patch giving that date as the final19:30
evrardjpmattt: and mhayden on the same channel -- OMG OMAGAD!19:30
spotzodyssey4me: will keep a hold of my docs patch:)19:30
odyssey4meor evrardjp can work it out19:30
evrardjpspotz: I will issue a release on the 30 March19:31
evrardjpand that release will bump the roles to eol19:31
openstackgerritMerged openstack/openstack-ansible stable/queens: [Docs] Change heading in target hosts
spotzevrardjp: Want me to go ahead and put up the patch then so it's there?19:31
evrardjpand then it can be tagged EOL, so I guess yes, you can plan for the 30. One day or so doesn't make a lot of difference :)19:31
mhaydeni heard someone needed gates broken19:31
evrardjpspotz: in which branch is that?19:31
evrardjpnewton itself?19:31
evrardjpif it's in newton, yes, you should land it19:32
evrardjpif it's in master, that can wait19:32
spotzevrardjp: It's master, openstack-ansible/doc/source/index.rst19:32
evrardjpno worries19:32
evrardjpthere is no urgency ther19:32
*** Sha000000 has quit IRC19:32
evrardjpyou can wait for the EOL to be officially done19:32
spotzevrardjp: Okie, I'll keep holding on to it:)19:32
*** vnogin has joined #openstack-ansible19:35
odyssey4meI trust y'all to keep an eye on to ensure they get merged today in time for release tomorrow.19:36
odyssey4meTime for me to !computer.19:36
matttodyssey4me: how does one play video games with !computer ?19:37
odyssey4meheh, one doesn't - but this time I mean !computer19:38
odyssey4mefor cores, there are plenty of reviews waiting if anyone has time - use this bookmark:*%22+status:open+label:Verified%253E%253D1%252Czuul+NOT+label:Workflow%253C%253D-1+NOT+label:Code-Review%253C%253D-1+NOT+owner:self19:38
evrardjpodyssey4me: could you vote on something real quick?19:44
*** vnogin has quit IRC19:44
*** Sha000000 has joined #openstack-ansible19:45
evrardjpthat's preventing issues on master for the deploy-guide19:45
evrardjpodyssey4me: you should update your bookmarks19:45
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_tempest master: update for changes needed to pass refstack v2
openstackgerritMatthew Thode proposed openstack/openstack-ansible-os_tempest master: update for changes needed to pass refstack v2
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: [Docs] Include OpenStack-Ansible Manifesto
*** chhagarw has quit IRC19:53
evrardjpprometheanfire: team team team!19:53
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Fix delegation to hosts not in inventory
prometheanfireevrardjp: lol, if we merge the two vars the comment needed to change it's plurality :P19:54
evrardjpyeah but now I can vote :p19:55
evrardjpyou're aware of the change, you agreed, and we can continue, because I agreed.19:55
evrardjpjmccrory: thanks for the work done there.19:55
evrardjpokay it's late for me19:56
matttevrardjp: night night!19:56
jmccroryevrardjp np, did you already have that experimental ansible-next job created somewhere?19:56
prometheanfireevrardjp: lol, sure19:56
evrardjpjmccrory: there is no 2.6 branch19:56
evrardjpso devel it is19:56
prometheanfireevrardjp: now we can backport it to queens/pike too19:56
prometheanfirethe full fix, not the partial that is19:57
evrardjpOh but we are still 2.4, so I CAN create the next19:57
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Test the main repo with next version of ansible
evrardjpjmccrory: you can probably do a test in the integrated repo, with a depends-on on your plugin patch, and check experimental.20:01
evrardjpif we merge
evrardjpor I can build a specific job if you like20:02
evrardjpto direct target from the plugins20:02
evrardjpI guess it's good enough for you to test for today20:03
jmccroryyeah, was going to do that but think we should have some test for plugins repo specifically too to find out blockers earlier20:03
evrardjpwill add that tomorrow20:03
evrardjpas a nv job or something20:03
jmccroryyep, sounds good. alright, have a good night20:04
evrardjpI am technically off tomorrow and monday , just so you know :)20:04
evrardjpwill still do the release20:04
spotzevrardjp: Have a good weekend off20:05
d34dh0r53evrardjp: got a minute before you take off?20:05
*** sawblade6 has joined #openstack-ansible20:05
*** sawblade_ has quit IRC20:08
idlemindwhere is the file where i can bump my local sha's of upstream projects?20:09
d34dh0r53idlemind: ansible-role-requirements.yml?20:10
d34dh0r53is that the file you're talking about?20:11
idlemindmight be20:11
*** strobelight has quit IRC20:12
openstackgerritMerged openstack/openstack-ansible-ops master: Apt-cacher-ng's default ubuntu mirror list has faulty mirrors
idlemindah good, i actually don't think i need to ... the reference to project (neutron-dynamic-routing in my case) is in the defaults/main.yml of each sub openstack-ansible project and for me that's targetted at the stable/pike branch ... hopefully just a re-run of os-neutron will pickup the updated repo's and the bugfix i need20:19
idlemindor do i need to do something besides the neutron play to get the git cache's to update?20:19
*** chhagarw has joined #openstack-ansible20:28
*** osnaya has joined #openstack-ansible20:29
osnayaquick q: from inventory.json I see containers are created on controller(infra) node. Are there containers created on computes? any specific applicable cases?20:31
idlemindosnaya containers are placed where you put services for me, my compute node didn't get containers until i made it a networking_host20:33
*** ansmith has quit IRC20:34
d34dh0r53idlemind: you'll probably want to run something like ansible-galaxy install -r ansible-role-requirements.yml to pull in the latest changes20:40
*** savvas has quit IRC20:41
*** haad1 has joined #openstack-ansible20:42
*** jwitko has quit IRC20:49
*** jwitko has joined #openstack-ansible20:49
*** hamzy has quit IRC20:50
*** savvas has joined #openstack-ansible20:50
*** savvas has quit IRC20:55
*** esberglu has joined #openstack-ansible20:55
*** chhagarw has quit IRC20:59
*** esberglu has quit IRC21:00
*** savvas has joined #openstack-ansible21:00
*** DanyC has quit IRC21:07
*** DanyC has joined #openstack-ansible21:11
*** hamza21 has quit IRC21:11
openstackgerritMerged openstack/openstack-ansible stable/ocata: Update repo_build role for PBR fix
*** esberglu has joined #openstack-ansible21:17
openstackgerritMerged openstack/openstack-ansible-os_nova stable/pike: change python-libguestfs to python-guestfs for ubuntu
*** esberglu has quit IRC21:22
*** esberglu has joined #openstack-ansible21:23
*** gyee has joined #openstack-ansible21:28
*** esberglu has quit IRC21:28
gyeedoes openstack-ansible support keystone fernet key rotation right now?21:29
lbragstadgyee: yessir21:29
*** ansmith has joined #openstack-ansible21:29
lbragstadgyee: it's crazy automated, too21:29
gyeewhere's that code?21:29
lbragstadit's smarter than i am21:29
gyeeit's calling keystone-manage or that in a module somewhere?21:30
lbragstadwhich isn't saying much, but still.. it's neat
lbragstadgyee: yeah - it'll call keystone-manage to establish a fernet repository on *a* keystone node21:31
lbragstadthen it goes through and lays down a rotation script that syncs the repository based on a cron job21:31
*** strobelight has joined #openstack-ansible21:31
lbragstadbut it's controlled via an ansible variable21:31
gyeeoh nice!21:31
lbragstadthe keystone nodes establish trust by sharing public keys, which allows other keystone nodes to perform the rotation if it falls outside of the rotation window (i think)21:32
spotzHey lbragstad:)21:32
lbragstads/rotation/rotation and sync/21:32
lbragstadspotz: o/21:33
gyeeso it establish a 'master' node and do the rotation there21:33
gyeethen sync the keys to the other nodes?21:33
lbragstadgyee: yeah - kinda21:33
lbragstadit will do it that way, but the master can change21:33
gyeebasically the one with the highest key number21:33
lbragstadif node-1 is the master, then goes down, node-2 should have the infra to perform a sync and push those keys to the rest of the nodes in the deployment21:33
lbragstadyeah... it's pretty cool21:34
d34dh0r53someone says fernet and lbragstad's ears start burning :)21:34
gyeelbragstad, thanks man! you just saved me a bunch of time!21:34
lbragstadi didn't implement it :)21:34
lbragstadd34dh0r53: i either run to help or run away21:34
gyeeI can reuse some of the mysql galera logic, basically finding the node with the highest number if stuff does go wrong21:35
gyeeas ansible is supposed to be idempotent :-)21:35
*** strobelight has quit IRC21:36
lbragstadyeah - i think it's done with inventory
lbragstadif the list in the inventory is consistent, that node should be the same21:37
*** jwitko_ has joined #openstack-ansible21:38
osnayaabout to run setup-hosts.yml.... is there a way to run it with parameter to create a verbose debug log?21:38
spotzd34dh0r53: Yep:)21:40
*** jwitko has quit IRC21:42
*** jwitko_ has quit IRC21:43
osnaya@cloudnull @odyssey4me @idlemind @evrardjp - is there a way to create debug log for setup-hosts.yml or setup-infrastructure.yml??21:48
osnayasimply redirecting console output for now21:55
*** SimAloo has quit IRC21:58
osnaya@cloudnull @odyssey4me @idlemind @evrardjp on the target hosts after running setup-hosts.yml how to check what containers are created?22:02
*** jwitko has joined #openstack-ansible22:10
osnayaanyone.... when I see the console output for setup-hosts.yml run, it shows a list of infra1_......._container_#### (about 20+) containers created thru play#6 Create container(s)TAGS: [lxc-containers-create]... How do I verify it on target before proceeding to the next step?22:13
*** savvas_ has joined #openstack-ansible22:17
*** savvas has quit IRC22:20
*** savvas_ has quit IRC22:21
dmsimardevrardjp, odyssey4me, logan-, cloudnull: btw
openstackgerritJimmy McCrory proposed openstack/openstack-ansible-plugins master: Fix delegation to hosts not in inventory
*** flemingo has joined #openstack-ansible22:34
*** flemingo_ has quit IRC22:37
openstackgerritMerged openstack/openstack-ansible-os_tempest master: update for changes needed to pass refstack v2
*** flemingo has quit IRC22:38
*** kstev1 has quit IRC22:39
jrosserosnaya: check the containers on the host with lxc-ls, your log is on the delpoy host in /openstack22:39
*** chyka has joined #openstack-ansible22:45
*** chyka_ has quit IRC22:48
*** Sha000000 has quit IRC22:52
osnayawas trying on the target host $lxc list (conn refused; is LXD running?) - systemctl status lxd.service says Loaded: masked (/dev/null; bad) inactive (dead)...22:55
osnayatried to restart (lxd.service and lxd-containers.service) but same rebooted the host.... but the same results ....22:56
osnayatried $service lxd restart -- Failed to restart lxd.service: Unit lxd.service is masked. Googling... any pointers?22:57
osnayado we need to do anything to disable ipv6 entries ---- any ideas?22:59
idlemindosnaya you could try prefixing the openstack-ansible command with ANSIBLE_STRATEGY=debug (for your earlier question)23:03
*** ibmko has joined #openstack-ansible23:03
idlemindosnaya lxc-ls on the host to verify the presence of the containers23:04
idlemindosnaya you can attach to one of them by doing lxc-attach --name <>23:04
osnaya@idlemind on the target host lxc list $lxc list (conn refused; is LXD running?) - systemctl status lxd.service says Loaded: masked (/dev/null; bad) inactive (dead)...23:06
idlemindosnaya if it's an ubuntu host i can't say for certain. mine are centos hosts and they have "lxc-ls"23:06
idlemindnote ... lxc not lxd23:07
osnaya@idlemind ubuntu 16.04 LTS23:07
idlemindtry "lxc-ls"23:07
idlemindmight have to "sudo lxc-ls"23:07
idleminda quick google seems to indicate ubuntu uses the same tooling23:07
osnaya@idlemind lxc-ls (command not found) on Ubuntu 16.0423:08
idlemindodd; running that on the infra node?23:09
osnayadpkg shows lxc-common (2.0.8), lxcfs (2.0.8), lxd (2.0.11-0) and lxd-client (
osnayayes on the infra1 node (there is a single controller node)... setup-hosts.yml returned successful.... so wanted to verify containers it listed as created on target host...23:11
osnayaLinux ansible5-1 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux23:11
idlemindwhat about just "lxc list"23:12
osnaya$lxc list (conn refused; is LXD running?)23:13
osnayahmmm.... systemctl status lxd-containers.service gave Warning: lxd-containers.service changed on disk. Run 'systemctl daemon-reload' to reload units. Loaded: masked (inactive dead...).23:15
*** flemingo has joined #openstack-ansible23:16
osnayajust checking there is a lxc.service, lxc-fs.service, lxd.service, lxd-containers.service, and lxd.socket23:18
openstackgerritMerged openstack/openstack-ansible-pip_install stable/ocata: Fix misspelled registered variable name in conditional
*** chyka_ has joined #openstack-ansible23:19
*** flemingo has quit IRC23:21
idlemindya one of the other folks that runs ubuntu will have to pipe in i think; it appears different commands are used on centos23:22
idlemindother than waiting for that you could try an ansible play w/the lxc module to see if it displays them23:22
*** chyka has quit IRC23:23
idlemindi also can't comment on the ipv6 disabling question becuase i use ipv6 in production23:23
*** chyka_ has quit IRC23:23
osnayawait ok was for the earlier resp23:23
osnayaso how to run the ansible play for lxc module?23:24
idlemind(i vote for using it over disabling but i suppose i'm still weird)23:24
*** flemingo has joined #openstack-ansible23:32
*** flemingo has quit IRC23:37
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: Convert role to use a common systemd service role
*** flemingo has joined #openstack-ansible23:51
*** haad1 has quit IRC23:51
*** flemingo has quit IRC23:55
*** osnaya has quit IRC23:59

Generated by 2.15.3 by Marius Gedminas - find it at!