Tuesday, 2020-04-14

*** openstack has joined #opendev09:53
*** ChanServ sets mode: +o openstack09:53
fricklerinfra-root: supy-bot seems to have been stuck in a loop waiting for a reply to a ping from yesterday, not sure how it got the idea that might ever be successful http://paste.openstack.org/show/792090/09:54
*** dpawlik has quit IRC09:55
*** ysandeep|lunch is now known as ysandeep09:55
fricklerstatusbot seems to have gotten stuck at around the same time, restarted them both09:56
*** dpawlik has joined #opendev09:56
frickler#status log restarted statusbot and meetbot which both seem to have stopped working around 2020-04-13 21:40Z09:59
openstackstatusfrickler: finished logging09:59
openstackgerritMarcin Juszkiewicz proposed openstack/diskimage-builder master: Do not try to use MBR on AArch64  https://review.opendev.org/71980510:12
*** rpittau is now known as rpittau|bbl10:23
*** ysandeep is now known as ysandeep|afk10:37
*** ysandeep|afk is now known as ysandeep|rover10:53
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: WIP: tox: allow tox to be upgraded  https://review.opendev.org/69005710:55
*** drifterza has quit IRC11:14
*** ysandeep|rover is now known as ysandeep|coffee11:39
*** ysandeep|coffee is now known as ysandeep11:51
*** ysandeep is now known as ysandeep|rover11:56
*** hashar has joined #opendev12:02
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded  https://review.opendev.org/69005712:28
*** rpittau|bbl is now known as rpittau12:31
*** lpetrut has quit IRC12:54
*** lpetrut has joined #opendev13:01
slittle1Yes, it was https://review.opendev.org/#/c/718772 that we need to get on track.13:16
slittle1and sorry for the long delay.  Stuff happens when 'work from home' collides with 'daddy daycare'.  :)13:18
openstackgerritMerged openstack/project-config master: Add kernel to StarlingX  https://review.opendev.org/71877213:50
*** ykarel is now known as ykarel|away13:50
*** rpittau has quit IRC13:50
*** rpittau has joined #opendev13:51
slittle1Please add myself to  starlingx-kernel-core.  I'll add the rest.13:57
slittle1uhmmm... is this normal ....   Zuul message "Unable to freeze job graph: Job infra-prod-remote-puppet-else depends on infra-prod-update-system-config which was not run.14:01
*** roman_g has joined #opendev14:01
AJaegermordred: ^14:02
slittle1can't see/clone the new git yet14:02
mordredAJaeger: on project-config?14:04
openstackgerritMonty Taylor proposed openstack/project-config master: Remove deps from remote-puppet-else  https://review.opendev.org/71997614:06
mordredAJaeger: ^^14:06
mordredI'll run manage-projects real quick14:06
AJaegerthanks, mordred . config-core, please +2A 71997614:11
*** lpetrut has quit IRC14:24
openstackgerritMonty Taylor proposed opendev/system-config master: Don't map known_hosts in to manage-project  https://review.opendev.org/71998614:26
*** sshnaidm has joined #opendev14:29
*** sshnaidm has quit IRC14:30
corvusmordred: did you write that change to use the same ansible.cfg in both places?14:31
corvusif not, i can write it14:32
mordredcorvus: yes - it's on my list for today14:33
corvusmordred: you want to do it, or want me to?14:33
mordred I want to do it14:33
corvusok all yours then :)14:33
mordredcorvus: I'm ... I'm having some further thoughts about the shape of how this all works based on clarkb's feedback about what to do with /opt/system-config ... and I had another idea this morning14:34
mordredtl;dr - still working on it :)14:34
openstackgerritMonty Taylor proposed opendev/system-config master: Fix the hostkey for gerrit  https://review.opendev.org/71998614:35
mordredcorvus, fungi: ^^14:35
mordredthat's broken in prod, could use a quick +A14:36
corvusmordred: i don't understand the 'sad' can you leave a comment with a quick summary of what happened?14:36
mordredcorvus: you're looking at an old patchset14:37
corvusmordred: oh so i am :)14:37
mordredcorvus: the first one was "crap, let's partially revert" - but then I was able to diagnose the actual problem14:37
corvusmy wish was granted14:37
corvusmordred: https://review.opendev.org/719685 feels like it's in service of making things easier to run manually on bridge -- but what's the process of running manually with that?14:40
mordredcorvus: so - on the run ansible from zuul cloned content ... what if we went a slightly different direction, and instead of pushing the content to /home/zuulcd/src we just pushed system-config and project-config to /opt/system-config and /opt/project-config and had the global ansible cfg always apply whether it's us or zuulcd running the playbooks. we'd need to be able to block zuul from syncing, but I'm14:40
mordredstarting to worry that we're growing 3 different contexts for running things (manual, zuulcd and run-base test)14:40
mordredcorvus: this is jus the thing - it actuallymakes the process harder - but clarkb was worried that we'd run manage-projects without remembering to git pull in /opt/project-config so thought that needing to pass -eproject_config_src=/opt/project-config to ansible-playbook would be a good reminder of that14:41
corvusmordred: "we'd need to block zuul from syncing" you mean: if we want to run something manually, we need to block zuul from doing anything on bridge during that time?14:41
mordredbut this is what got me worried that it's getting too complex and we might need to rethink the shape14:41
mordredcorvus: yeah - if we have zuulcd push refs to /opt/system-config - we might want to make sure that doesn't happen for a period14:42
mordredwe could pretty easily drop a flag file and have the job check for it and bail14:42
clarkbproject-config updates are a bit different fwiw. Because zuul and manual use a different path by default14:42
mordredright - that's what I'm getting at ... maybe that's a mistake14:43
clarkbthis means it is very easy for zuul to happily keep things moving fowrard then for a human to accidentally sync a 6 month old state14:43
clarkbmordred: ah14:43
mordredlike - I think we're creeping towards a system that's brittle when we have to do something manually - which is exactly when we don't want it to be brittle14:43
corvusyeah, i could see a flag file as a solution to that -- we could actually have the job wait in that case (under the assumption that maybe if it waits 15 minutes, we'll be done with whatever manual thing we were doing)14:44
mordredwhy don't I code that up and we can see what we think?14:44
corvusand there are also many manual thing we could do without even dropping the flag file (anything where the p-c state isn't absolutely critical)14:45
mordredthat said - what if /home/zuulcd is the new "location"14:45
mordredand we ditch /opt/system-config and /opt/project-config14:45
*** mlavalle has joined #opendev14:45
mordred(keeps the repo sync easy - and if we grow a third or fourth repo it remains consistent)14:46
mordredwe could make /opt/system-config a symlink for our muscle memory :)14:46
corvusi have a slight preference for just making it go away so we don't forget it's managed by zuul (but don't object to a symlink if others feel it's good)14:47
mordredyeah - I think that's a good point and you've convinced me14:47
mordredI think it expresses nicely "this is a process owned by zuul and manual operation is the exception"14:48
mordredrather than "zuul is driving an otherwise manual process" or "zuul and manual are equally valid"14:48
clarkbmordred: ya I think "this is a process owned by zuul and manual operation is the exception" is the perspective we should apply to these improvements14:49
* mordred will update the stack with this approach14:49
clarkbthen have some methods/tools/process/whatever that allow us to make exceptions to the automated process as necessary14:49
*** ysandeep|rover is now known as ysandeep|away14:50
mordredalso - I was thinking it would be neat if we can collapse run-production-playbook.yaml and run-base.yaml more14:50
mordrednow that we have run-production-playbook and it's what we're using, it feels like there might be opportunities to do less in run-production-playbook - but I haven't gotten all the way there yet :)14:51
mordredclarkb, corvus: what if we changed the zuulcd user on bridge to just be the zuul user?15:01
openstackgerritMerged openstack/project-config master: Remove deps from remote-puppet-else  https://review.opendev.org/71997615:02
clarkbmordred: would that change anything?15:02
mordredI say that becuase I'm currently making a symlink from /home/zuul to /home/zuulcd in the test job15:02
mordredsince the test job runs in /home/zuul and it's a little harder to change that15:03
clarkbthere shouldn't be a /home/zuul?15:03
clarkboh I see its to match the gate/check jobs to deploy15:03
mordredotherwise we need to add a variable and set it in the test job but not in prod15:03
clarkbI think at the time we called it zuulcd the idea was to differentiate it from zuul direct login15:03
clarkbbut maybe that was a mistake given the test setups15:03
mordredyeah - totally, and it was a reasonable choice at the time I think15:04
mordredbut yeah - I think maybe we've learned something15:04
mordredI can do the symlink for now for this stack15:04
mordredand we can take that as a follow on cleanup15:04
mordredALSO - I had this thought that we should make inventory/groups.yaml into inventory/groups/{foo}.yaml - so that we can better do file matchers in zuul15:05
mordred(and the gate-groups.yaml wouldn't be a special-case for gate jobs either)15:05
slittle1please add 'scott little' is first core of starlingx-kernel-coreĀ .  I'll add the others15:13
clarkbslittle1: done15:13
fungiclarkb is the fastest draw in the west15:13
clarkbits an easy task while listening to the board meeting15:14
fungithat's what i thought too, but i only got as far as the group filter page ;)15:15
fungianybody else having trouble reaching cacti.openstack.org?15:19
clarkbfungi: not me15:20
clarkbI'll be hitting it via ipv4 fwiw15:20
fungiit just started responding for me15:20
fungiahh, yep, that was a v6 timeout15:20
fungilooks like something has killed my v6 routes15:21
*** JasonF is now known as JayF15:25
fungiyeah, apparently sometimes my provider's dhcp6 server doesn't respond to my dhcpcd and it expires my delegation. i should probably force a longer grace period in an override15:26
clarkbfungi: I think a lot of dhcp renewals happen at the halfway point through a lease (so you can renew very early)15:27
fungiyeah, it starts trying according to the log and goes multiple attempts with a "no carrier"15:28
fungiit may be that the broadband connection itself was down at the time, i was sound asleep when it happened15:29
fungiin fact, my v4 dhclient was complaining about "host is down" too, so i bet that was it15:30
*** drifterza has joined #opendev15:33
openstackgerritClark Boylan proposed opendev/system-config master: DNM Test docker-compose upgrade  https://review.opendev.org/71968215:43
openstackgerritClark Boylan proposed opendev/system-config master: Install docker-compose from pypi  https://review.opendev.org/71958915:50
openstackgerritClark Boylan proposed opendev/system-config master: Use HUP to stop gerrit in docker-compose  https://review.opendev.org/71905115:50
*** rpittau is now known as rpittau|afk16:02
*** drifterza has quit IRC16:07
clarkbinfra-root ^ fwiw I think testing is showing that that is safe, but I want it to go green and not just me assume that based on what I'm seeing :) I've also got a couple general test job improvements related to that that I'll push up once the current round of results are available16:09
clarkbthis has me thinking that it might be a good idea to more properly formalize a before and after upgrade process in our test jobs. I'm not quite sure how to make that less hacky though16:09
clarkbI'll ponder on it further as it does give you that extra bit of confidence going from state A to state B won't break anything16:10
openstackgerritMonty Taylor proposed opendev/system-config master: Use project-config from zuul instead of direct clones  https://review.opendev.org/71934316:11
mordredclarkb, corvus: how does that look now ^^?16:11
mordredclarkb: I agree - a before and after upgrade would be really nice16:11
clarkbmordred: what I'm thinking is we want to run everything on HEAD^ then go ahead to HEAD and run everything at that state16:12
mordredclarkb: yeah.16:13
clarkbbut that makes the jobs run logner and I'm sure there are corner cases where doing that won't work as I expect (like if HEAD^ is just broken compared to state of the world)16:13
mordredclarkb: we'll need to special case "this patch added a completely new playbook"16:13
mordredclarkb: that's a very good pint16:13
clarkbmordred: ya it makes me wonder if we can make it opt in somehow16:14
mordredclarkb: maybe we write a general upgrade job but put it in an on-demand only pipeline16:14
mordredbecause there are plenty of changes where an upgrade test would be overkill16:14
clarkbya maybe check opendev upgrade comemnts are the way to make it opt in16:15
clarkbI had a thought we could have a "UPGRAGE" file that we edit to trigger jobs too16:15
clarkb(but thats the hacky stuff I'm talking about )16:15
mordredclarkb: I think it would be important to the design to name it UPGRAGE16:19
clarkbI didn't catch that typo but I agree it should stay :)16:20
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002916:45
mordredclarkb: ^^16:46
*** hashar has quit IRC16:49
clarkb https://review.opendev.org/#/c/719682 passes testing \o/ I think that means we are good to land the production change16:52
openstackgerritClark Boylan proposed opendev/system-config master: DNM Test docker-compose upgrade  https://review.opendev.org/71968216:53
openstackgerritClark Boylan proposed opendev/system-config master: Run jobs prod test jobs when docker images update  https://review.opendev.org/72003016:53
clarkbhowever new ps to simplify some things16:53
clarkbinfra-root https://review.opendev.org/720030 in particular is a simple job files update to ensure jobs run when we update docker images if you have a moment16:53
clarkbthat tripped me up when I started thsi testing process16:53
mordredclarkb: I'm good landing that whenver16:59
noonedeadpunkhi everyone.17:01
mordredhi noonedeadpunk17:01
noonedeadpunkhave small question about how required-projects works. Am I right that they got cloned `/home/zuul/src/` on master branch and patch being tested don't affect it?17:03
noonedeadpunkSo like if patch is for stein, cloned roles in /home/zuul/src/ will still be for master?17:03
AJaegernoonedeadpunk: no, they are for stein as well17:03
AJaegernoonedeadpunk: unless you use override-checkout17:03
AJaegerWhy do you ask?17:03
noonedeadpunkActually investigating why  https://zuul.opendev.org/t/openstack/build/3e730fa1cd7143daac1d2957270ab749 fails... So we actually linking cloned project here https://zuul.opendev.org/t/openstack/build/3e730fa1cd7143daac1d2957270ab749/log/job-output.txt#273917:05
noonedeadpunkBut it feels like os_tempest is from master branch...17:05
noonedeadpunkbut maybe issue is somewhere else then17:05
funginoonedeadpunk: does this match what you think should have been checked out there? https://zuul.opendev.org/t/openstack/build/3e730fa1cd7143daac1d2957270ab749/log/job-output.txt#1811-181217:07
mordrednoonedeadpunk: zuul will prepare the state for all branches ...17:07
mordredso that if there are depends-on or whatnot across branches, they'll all be stacked as needed into the appropriate branches - git checkout will work if you need to switch between them - but as AJaeger mentions, you can also set override-checkout in the job to control which one of them zuul checks out by default17:08
noonedeadpunkfungi: yeah, it's correct17:08
noonedeadpunkmordred: oh, it's pretty useful info thanks17:09
noonedeadpunkdidn't know that's possible17:09
mordrednoonedeadpunk: yeah - so you can test things like "does this change to stable/train still work with the upgrades currently in master"17:11
noonedeadpunkyeah, actually was fighting with the way of upgrades test17:12
noonedeadpunkwe have implemented one, but at the moment it don't respect patches17:13
funginoonedeadpunk: so if you see evidence that there's a different version installed than what zuul checked out, the usual culprits are something like a pip install not pointed at that source tree, or a distro package slipping in overwriting things, or some condition causing a script to grab its own copy of the source out across the internet17:13
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Improve 404 error message on download-logs.sh  https://review.opendev.org/72003517:13
fungii'm not spotting any of those in the job log you linked, but there's presumably a lot more going on than winds up in that log17:13
*** roman_g has quit IRC17:19
*** roman_g has joined #opendev17:19
*** roman_g has quit IRC17:20
*** roman_g has joined #opendev17:20
*** roman_g has quit IRC17:20
*** roman_g has joined #opendev17:21
*** roman_g has quit IRC17:21
*** roman_g has joined #opendev17:22
*** roman_g has quit IRC17:22
*** roman_g has joined #opendev17:23
*** roman_g has quit IRC17:23
*** roman_g has joined #opendev17:23
*** roman_g has quit IRC17:24
clarkbinfra-root ok https://review.opendev.org/#/c/719682/ has passed for a second time.  Ithink that implies https://review.opendev.org/719051 and its parent are safe to go now. https://review.opendev.org/720030 is another good one that fixes job defs related to this17:49
clarkbyall should look over the testing in that first change though in order to convince yourself that the second is good17:49
clarkbI need to get late breakfast/early lunch now before our meeting but will be back in an hour for that18:00
*** dpawlik has quit IRC18:02
openstackgerritMonty Taylor proposed opendev/system-config master: Use project-config from zuul instead of direct clones  https://review.opendev.org/71934318:06
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002918:06
*** tobiash has quit IRC18:16
*** tobiash has joined #opendev18:17
openstackgerritMonty Taylor proposed opendev/system-config master: Use project-config from zuul instead of direct clones  https://review.opendev.org/71934318:22
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002918:22
openstackgerritMonty Taylor proposed opendev/system-config master: Fix the hostkey for gerrit  https://review.opendev.org/71998618:25
mordredinfra-root: ^^ sorry - typo18:25
fungiaha, missing '18:25
fungisorry i didn't spot that earlier18:26
fricklermordred: still one ' missing in 71998618:34
mordredfrickler: doh18:41
openstackgerritMonty Taylor proposed opendev/system-config master: Fix the hostkey for gerrit  https://review.opendev.org/71998618:41
mordredfrickler, fungi : ^^ fixed (thanks!)18:42
* fungi must need glasses18:42
*** ralonsoh has quit IRC18:49
clarkbI know I need them :)18:55
openstackgerritJeremy Stanley proposed opendev/irc-meetings master: Update OpenDev meeting location and name  https://review.opendev.org/72006019:15
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002919:16
openstackgerritJeremy Stanley proposed opendev/irc-meetings master: Not all meetings are OpenStack  https://review.opendev.org/72006319:21
*** DSpider has quit IRC19:22
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002919:44
ianwfungi: could you look over https://review.opendev.org/#/c/719707/, which i noticed yesterday while debugging xenail venv failures19:50
ianwremoves an old pip install from wheel build19:50
openstackgerritMerged opendev/system-config master: Fix the hostkey for gerrit  https://review.opendev.org/71998619:51
openstackgerritMerged opendev/system-config master: Publish docs updates to docs.opendev.org  https://review.opendev.org/71876219:51
ianwinfra-root: another smallish one -> https://review.opendev.org/718600 ; this sends stats on vos release timing for mirror updates to graphite.  it will be nice to graph that, alongside the timings we track for the other repos from the periodic release script19:54
openstackgerritMonty Taylor proposed opendev/system-config master: Update install-ansible away from /opt/system-config  https://review.opendev.org/71918619:55
openstackgerritMonty Taylor proposed opendev/system-config master: Run playbooks out of zuul checkout  https://review.opendev.org/71919019:55
openstackgerritMonty Taylor proposed opendev/system-config master: Use project-config from zuul instead of direct clones  https://review.opendev.org/71934319:55
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002919:55
AJaegerconfig-core, please review https://review.opendev.org/718676 - grafana neutron update20:01
fungiokay, i need to step away to start preparing dinner, but will jump back on to get some stuff done once that's finished20:01
clarkbmordred: can you check my comment on https://review.opendev.org/#/c/719192/120:07
clarkbmordred: and comment on https://review.opendev.org/#/c/719484/220:08
openstackgerritMerged openstack/project-config master: Revert "wheel-build : temporary pip with checksum output"  https://review.opendev.org/71970720:09
AJaegermnaser: did you see the comment on https://review.opendev.org/#/c/717507/ by tobiash to rename the role?20:11
*** mtreinish has joined #opendev20:12
mnaserAJaeger: i did, but i'm a little overwhelemd with a lot of stuff going on :<20:15
openstackgerritMerged openstack/project-config master: Update Grafana dashboards for stable Neutron releases  https://review.opendev.org/71867620:16
*** fdegir4 has quit IRC20:19
*** fdegir4 has joined #opendev20:19
*** fdegir4 has quit IRC20:20
openstackgerritMonty Taylor proposed opendev/system-config master: Write out db config for root user  https://review.opendev.org/71919220:21
*** moppiner has joined #opendev20:21
mordredclarkb: ^^ fixed20:21
*** moppy has quit IRC20:21
mordredclarkb: for the other - it shouldn't matter - but maybe we should update that to ansible first just in case?20:21
mordredclarkb: also - what's the deal with the other bup server? it seems like we have one for puppet and one for ansible20:22
openstackgerritMerged opendev/gerritbot master: Build docker images  https://review.opendev.org/71563520:23
clarkbmordred: I think that is due to them being managed separately and weneed to transition from one to the other20:29
clarkbianw ^ should know for sure20:29
mordredclarkb: I'm going to have to merge teh "run from project-config" and the "Rename to zuul" changes - because otherwise it actually gets really srewey20:29
clarkboh you mean squash them together?20:29
mordredI can't make the run from project-config change work in test20:29
ianwclarkb / mordred : it's been a long while since i looked in on that.  i'll have to context switch, but we should be migrated to the new server?20:30
mordredit's gonna make the change bigger - but I dont think it's worth making the middle one work20:30
mordredianw: non-urgent, so don't know that you need to context switch this moment20:30
ianwmordred: where are you seeing references to the old server?20:32
openstackgerritMerged opendev/system-config master: mirror-update: stats for vos release of mirrors  https://review.opendev.org/71860020:33
openstackgerritMerged opendev/system-config master: Back up a single gitea backend  https://review.opendev.org/71946520:33
mordredianw: in the puppet - one sec20:33
openstackgerritMonty Taylor proposed opendev/system-config master: Rename zuulcd to zuul  https://review.opendev.org/72002920:33
mordredianw: bup::site { 'ord.rax':20:34
mordredianw: is in a bunch of puppet modules in modules/openstack_project20:34
mordredianw: while the ansible-driven bup stuff uses backup01.ca-ymq-1.vexxhost.opendev.org20:35
ianwmordred: oh yeah, i feel like that's right, the idea was to cut over as we removed puppet20:40
mordredianw: kk. in that case I'll wait on eavesdrop20:41
ianwmordred: although now i notice etherpad say isn't in the backup group20:43
mordredit is20:44
mordred  backup:20:45
mordred    - etherpad[0-9]*.opendev.org20:45
mordredinfra-root: would you mind +Aing this: https://review.opendev.org/#/c/718559/20:46
ianwoh, sorry, was on the wrong branch20:46
ianwmordred: on a related note some debug removals -> https://review.opendev.org/#/c/717874/20:47
ianwthat change is from the prehistoric era; april 720:48
mordredianw: that's so long ago20:48
clarkba whole week20:50
openstackgerritMonty Taylor proposed opendev/system-config master: Use project-config from zuul instead of direct clones  https://review.opendev.org/71934320:55
clarkbianw: couple of things on https://review.opendev.org/#/c/717882/1920:56
ianwclarkb: on the change ... yeah i feel like it's very unlikely; it's already documented wrong?20:58
clarkbianw: oh I see its documented wrong but then we still change the var20:59
clarkbya in that case I think its fine20:59
ianwclarkb: on the checking, that was somewhat deliberate, thinking it should be totally idempotent20:59
ianwi had being toying with this; in a way perhaps we should put it in the base role, and roles like this should say "assume a working pip"?21:00
clarkbianw: ya, its mostly just concern with individuals like tristanC being very concered about job runtime21:00
clarkbianw: and fetch-zuul-cloner, fetch-subunit, and ensure-tox are all likely to run in the same job21:01
ianwit feels like you can't have it both ways -- either you run a role that's supposed to ensure it and make sure it's idempotent, or leave it up to the role to assume things are there21:01
ianwbut then take the chance the role is not usable "out of the bxo"21:01
ianwbox even21:01
clarkbianw: I think the difference here is we are hiding those details in the roles and not asking jobs to deal with it21:02
clarkbwe make it idempotent anyways so it can be used that way, but since these set of roles are expected to be used together we can optimize a bit?21:02
ianwi just feel a bit icky making ansible skip like that, it feels counter to the way you're supposed to let the cfg mgmt tool figure out what to run21:04
ianwdoes it actually run it twice anyway, if no arguments have changed?21:04
clarkbianw: yes21:04
clarkbpuppet wouldn't because it generates its big graph of unique resources and collapses includes. But ansible tasks are run each time they are listed21:05
clarkbfwiw I'm happy to go with it as is and then we can optimize later if we find we need to21:06
tristanCwell it seems more efficient to have smaller role that can be combined efficiently from a play instead of a multi-purpose roles that does everything21:06
clarkbI just half expect that to be the case21:06
ianwclarkb: it definitely doesn't run duplicates twice -- https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#role-duplication-and-execution21:07
clarkbtristanC: the problem with that appraoch is then users fail to get all the bits composed in their plays21:07
ianwbut ... i don't know if across playbooks etc we are fooling it21:07
*** roman_g has joined #opendev21:08
clarkbianw: in this case we aren't using the role: directive on a play but instead are using the include_role task21:08
clarkband it is my understanding that since it is a task it runs every time21:08
openstackgerritMerged opendev/system-config master: Don't log the public loop on master-nameserver  https://review.opendev.org/71855921:08
openstackgerritMerged opendev/system-config master: Switch documentation to alabaster theme  https://review.opendev.org/71879021:08
tristanCclarkb: isn't ianw suggestion of adding assumption, like "assume a working pip", a solution to incorect play?21:10
clarkbtristanC: no that doesn't fix the play. It forces you to know to add that to your plays21:11
clarkbI think it is an option. I don't think it is a good option for most zuul users21:11
ianwtristanC: in this case, it's using a variable exported by "ensure-pip" ... so it's a harder dependency  of "ensure-pip" was run21:12
ianwso, the question is sort of -- do you want people to have to incrementally debug why the role doesn't work as they use it (it's wrong if you just try and call it, without reading the documentation in detail)21:13
ianwor, have it work out of the box21:13
clarkbianw: ya and not only that what the end user wants is `tox` to run and work because that is the tool they are interfacing with21:13
clarkbin depth debugging of ansible isn't necessarily a part of their skillset21:13
ianwi keep coming back to http://sweng.the-davies.net/Home/rustys-api-design-manifesto which i mentioned to mordred the other day21:13
ianwit's about a -7 v a +7 :)21:13
clarkbbasically I think what you've done is correct. I'm just wodnering if we should optimize it within the context of those related and controled roles21:14
ianwmaybe a 7 to 3 actually -- "read the documentation and you'll get it right." v "The obvious use is (probably) the correct one" :)21:15
clarkbI've +2'd the stack I think we can optimize later as we learn more21:15
ianwclarkb: how about i do a follow-on to gate on the variable and see how it goes?21:15
clarkbianw: sure21:16
*** tobiash has quit IRC21:18
ianwtristanC: i'd sure like it if you could look over https://review.opendev.org/#/q/topic:ensure-pip anyway ...21:18
tristanCclarkb: ianw: for what it worth, when using kubectl connection, the runtime likely already have the build requirements like tox/pip/python baked in, and in that situation i find it wasteful to have the job spend time (and fill the log with useless task) ensure things21:20
*** tobiash has joined #opendev21:20
clarkbtristanC: yes that is why I brought this up as I knew you'd not like :)21:21
tristanCbut since it's already the case with ensure-tox, then i think the only way to be really efficient is to provide custom play that goes straight to running the actual job like tox21:21
clarkbultimately I think zuul-jobs role (no pun intended) is to provide generic tools that work well together and address common problems21:21
clarkbI think that means we can't make you happy in all cases because your needs are so environment specific21:21
ianwi'm totally up for incrementally improving it ... i think this more conservative approach is probably good for the initial switch off of pip-and-virtualenv21:21
tristanCyes, thus i don't mind having "heavy" ensure-role in the current zuul-jobs that works for most case, even when the user somehow got the play wrong21:22
clarkbtristanC: in the case of docker image backed tasks I would expect that all tools are already present and the job shoudl simply just execute commands21:22
clarkbtristanC: and so zuul-jobs isn't very valuable for the runtime of the job there (but would be for setting up the use of those images)21:22
tristanCclarkb: well we still want to use the `tox` role as it takes care of sibbling installation, but it seems like we have to create new tox job without the pre-run that ensure things21:23
tristanCas long as the `tox` role doesn't include the `ensure-tox` role, we are in the clear :-)21:26
ianwthis stack has not added ensure-tox to the tox role :)21:27
ianwit has added ensure-pip to the ensure-tox role, though21:28
ianwmaybe that's a compromise, that ensure-roles include other ensure roles, but the "runtime" roles can assume the setup has been done?21:28
corvustristanC: the ensure-tox should just be a no-op if tox is already installed -- but you're saying that even the no-op wastes too much time?21:28
corvusianw: yes, i think that is or should be the understanding -- these roles are split so that they can be put in different playbooks.  ensure- roles should generally be in pre-run.  runtime roles should be in run.21:29
corvusianw: that happens to make it so that tristanC can avoid running the ensure role too if he wants to define a new tox job.  defining new jobs with different combinations of roles is also an explicit goal -- that's why we put all the work in roles and not playbooks.21:30
clarkbcorvus: I think you just said what I tried to say better. Particularly the bit about images starting with what they need (so the ensure-* goes aay)21:32
ianwi think we're in agreement.  we're not crossing runtime roles with setup roles in this stack21:34
corvusi think so too21:34
ianwhowever, if you would like to review the stack with that explicitly in mind, that would be welcome :)21:34
tristanCcorvus: i haven't looked in detail, but the existing tox job, when doing a simple flake8 linter may spend more time in pre/post than doing the actual tox command21:35
clarkbtristanC: which we identified was an odd ansible behavior that needed debugging21:36
*** roman_g has quit IRC21:53
ianwznc listservers22:32
mordredclarkb, ianw, corvus : https://review.opendev.org/#/c/719343/ is green!22:38
mordredI had to squash it with the zuulcd -> zuul change, because getting just half of it to work was more work than it was worth22:39
clarkbk Im heading out on a bike ride then will review22:41
ianwi'm still getting my head around 719186 :)22:44
openstackgerritJames E. Blair proposed opendev/system-config master: Meetpad: proxy through meetpad to etherpad.opendev.org  https://review.opendev.org/72009522:45
corvusdo we think that's the best way to use the new setup?  (still proxying through the meetpad server?) or should we have it go directly to the new server like we did earlier?22:45
corvusmordred: what happens if we "modify" the zuul user in test?22:46
corvusmordred: other than that curiosity, that looks nice :)22:47
openstackgerritGoutham Pacha Ravi proposed openstack/project-config master: Add devstack-plugin-ceph notifications to manila channel  https://review.opendev.org/72009722:56
mordredcorvus: it fails with a "user in use" error - lemme see if I can find one23:17
corvusmordred: what's it trying to do?  i'm just wondering why it doesn't noop23:17
mordredcorvus: https://84cacd386095a03eb330-320386062a8fef96051148fe5e7af6b1.ssl.cf2.rackcdn.com/720029/1/check/system-config-run-base/c6018fb/bridge.openstack.org/ara-report/result/0260d718-f9cc-48b0-8371-75d69c6304ca/23:18
mordredcorvus: it's trying to change the uid23:19
mordredcorvus: because we set uid and gid in users in system-config23:19
corvusah that'll do it23:19
openstackgerritGoutham Pacha Ravi proposed openstack/project-config master: Add devstack-plugin-ceph notifications to manila channel  https://review.opendev.org/72009723:20
*** tosky has quit IRC23:20
*** mlavalle has quit IRC23:28

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!