Wednesday, 2021-06-30

clarkbthen it should magically work if that is all there00:00
ianwyeah, it is there00:00
ianw(the buildset registry)00:00
ianwi guess i need it in the project jobs:00:01
clarkbhrm in that case I would expect it to work. You do need to set up the job ordering within the buildset too if building the image and running the job in separate jobs. But otherwise I think that would do it?00:01
opendevreviewIan Wienand proposed opendev/system-config master: [wip] paste service
corvusianw, clarkb: shouldn't that be run on changes to lodgeit, not system-config?00:04
corvusbasically, build the image in the lodgeit repo, then rely on the buildset registry to bring over the previously built image00:05
ianwi was hoping that depends-on: <lodgeit change> would test in system-config00:06
corvusit should00:06
corvusi believe PS15 is correct00:07
corvusbut it's gone now, so i can't see what was wrong with it00:07
ianwi can restore and put that back in the queue, but that wasn't running the lodgeit image build job00:07
corvusit's not supposed to00:07
corvus798419 runs that job00:08
corvus(or actually, already ran it)00:08
corvusso when run-lodgeit runs, it will fetch the previously built artifact due to the depends on00:08
ianwoh!  yeah i forgot about persistence and that it had already run00:09
corvusactually, i guess ps 15 wasn't correct because it had a dependency on the lodgeit build job00:09
corvusso basically, remove all mention of lodgeit-build-opendev-image from 79840000:10
opendevreviewIan Wienand proposed opendev/system-config master: [wip] paste service
opendevreviewIan Wienand proposed opendev/system-config master: [wip] paste service
corvusianw: couple more quick comments that aren't important now; but check 'em when the dust settles00:13
opendevreviewIan Wienand proposed opendev/system-config master: [wip] paste service
ianwcorvus: thanks.  yeah, so the "requires: lodgeit-container-image" should keep things in order if they end up in the queue at the same time, right00:15
ianwhere's another mystery00:31
ianwi feel like we should have a periodic build ~2021-06-28 06:<something> 00:31
ianwevery run of the periodic scheduler seems to put into the logs00:33
ianw2021-06-30 00:31:23,007 DEBUG zuul.Pipeline.openstack.periodic: Starting queue processor: periodic00:33
ianw2021-06-30 00:31:23,007 DEBUG zuul.Pipeline.openstack.periodic: [e: afdb7039494649c09e8bb2b64158a385] Checking for changes needed by <Branch 0x7fe343803e50 openstack/requirements refs/heads/master updated None..None>:00:33
ianw2021-06-30 00:31:23,007 DEBUG zuul.Pipeline.openstack.periodic: [e: afdb7039494649c09e8bb2b64158a385]   <class 'zuul.model.Branch'> does not support dependencies00:33
fungiianw: has openstack/requirements fixed its config errors yet?00:33
fungiyeah, i don't see it in the errors list any longer, so it's probably not that00:35
ianwahh, it probably is that causing it to be skipped00:36
ianwyeah, i'd expect todays run00:37
fungiyeah, not sure then00:37
ianwit's been a few months(!) since that release job worked :/00:37
ianwhrm, lodgeit still pulled from upstream01:10
ianw2021-06-30 00:42:24.687160 | | Pulling lodgeit ... digest: sha256:c0ff85021d4572123e...01:10
ianwthat matches
ianwohh, i guess lodgeit doesn't use the intermediate registry01:12
opendevreviewIan Wienand proposed opendev/lodgeit master: Use opendev-buildset registry
opendevreviewMerged opendev/lodgeit master: Use opendev-buildset registry
opendevreviewIan Wienand proposed opendev/system-config master: [wip] paste service
*** ykarel|away is now known as ykarel04:24
*** sshnaidm is now known as sshnaidm|afk04:39
*** marios is now known as marios|ruck05:08
*** ysandeep|away is now known as ysandeep05:10
opendevreviewIan Wienand proposed opendev/base-jobs master: Add opendev-intermediate-registry-base
ianwcorvus / clarkb: ^ re lodgeit.  i may be off-base here, but the system-config-run-paste job still isn't consuming the results of the lodgeit change.  i can't see how our current system-config jobs are supposed to pull from the intermediate registry, hence ^05:30
opendevreviewIan Wienand proposed opendev/base-jobs master: Add opendev-intermediate-registry-base
*** amoralej|off is now known as amoralej06:13
opendevreviewDmitriy Rabotyagov proposed opendev/lodgeit master: Add mariadb requirements
opendevreviewMerged opendev/lodgeit master: Allow for overriding title
*** marios is now known as marios|ruck07:16
*** jpena|off is now known as jpena07:39
*** marios is now known as marios|ruck08:09
opendevreviewBenjamin Schanzel proposed zuul/zuul-jobs master: Fix a bug in s3 log uploader with .gz files
*** ykarel is now known as ykarel|lunch09:24
*** ykarel|lunch is now known as ykarel10:33
*** sshnaidm|afk is now known as sshnaidm10:53
opendevreviewIan Wienand proposed opendev/lodgeit master: Add mariadb requirements
*** jpena is now known as jpena|lunch11:27
*** dviroel|out is now known as dviroel11:29
roman_gGood morning, team. Could you have a look at docs publishing job results for me, please? According to the logs, 'promote' job 'promote-airship-project-docs' has copied updated documentation to the AFS share. But the website still hosts old documentation structure.11:57
roman_gActual website:
*** marios|ruck is now known as marios|ruck|call12:01
roman_gFor example, new <<CHANGED>>>f+++++++++ _sources/cli/baremetal/airshipctl_baremetal.rst.txt gets copied to AFS (a bit strange that it's a .txt file), it's visible on a preview site, bit not available on actual website
*** amoralej is now known as amoralej|lunch12:07
*** jpena|lunch is now known as jpena12:27
*** gthiemon1e is now known as gthiemonge12:49
*** marios|ruck|call is now known as marios|ruck12:57
*** amoralej|lunch is now known as amoralej13:16
rosmaitawhen someone has a few minutes, it looks like merged cinder-specs are no longer being published to
rosmaitathis one is from last week:
rosmaitathis one if from earlier today:
opendevreviewHitesh Kumar proposed openstack/diskimage-builder master: Migrate from testr to stestr
roman_grosmaita looks like we have similar issue. Your logs are here:
fungiroman_g: rosmaita: the similarities in what you're reporting sounds like we may have problems with static site publication. i have to run some errands this morning but should be able to take a look in a couple of hours if nobody beats me to it13:38
roman_gfungi thank you!13:38
rosmaitafungi: no rush, thanks!13:38
*** ysandeep is now known as ysandeep|afk13:52
fricklerinfra-root: rosmaita: roman_g: that looks like some brokenness with the changed jobs after the zuul update13:55
fricklerdrwxr-xr-x 7 10012 10001 2048 Jun 28 15:07 '/afs/{zuul[project][short_name]}/'13:55
fricklerdrwxr-xr-x 5 10012 10001 2048 Jun 30 13:08 '/afs/{zuul[project][name]}/'13:55
fricklerbut I'll leave it to others to fix it, busy day here today13:56
frickleractually you can see that path also in the log that roman_g posted above (
rosmaitaok, thanks for troubleshooting13:57
roman_gfrickler I thought it is being substituted :)13:58
roman_gReplaced with actual value.13:58
roman_gfrickler then I would expect this{zuul[project][short_name]/airshipctl/cli/baremetal/airshipctl_baremetal.html or this{zuul[project][short_name]/cli/baremetal/airshipctl_baremetal.html URLs to possibly work. But it doesn't.14:01
clarkbinfra-root (particularly corvus and mordred since you indicated interest) looking like 17:00 UTC tomorrow or Friday for EMS discussion. I've suggested tomorrow so that we don't cut into the weekend, but am waiting for confirmation on that14:33
clarkbmordred: corvus: confirmed that 17:00 tomorrow (Thursday) is the time. They seem interested in using matrix for the call and I've passed along your matrix user links to help coordinate that14:38
*** ysandeep|afk is now known as ysandeep14:43
*** ysandeep is now known as ysandeep|away15:02
*** ykarel is now known as ykarel|away15:08
opendevreviewRoman Gorshunov proposed opendev/base-jobs master: Fix mistypes in variables in secrets
roman_gfrickler thank you for hints.15:17
roman_gfungi, hopefully this ^^ will fix issue, but some manual cleanup would still be needed on afs share, plus a refresh of HTTP redirects configs.15:18
fungiroman_g: no, since zuul 4.6.0 jinja templating won't work in secrets, it was an exploitable security hole. that's why corvus switched those to use variable substitution in but maybe there's still something not quite right with them, i'm back now and looking over the logs15:25
roman_gfungi thank you for the explanation.15:25
corvusclarkb: 1700utc tomorrow wfm thanks!15:27
fungiroman_g: my guess is we're missing a lookup() function in the consuming job but checking to be sure15:27
corvusfungi: maybe a playbook missing a .format() call?15:27
fungier, .format() right15:27
fungithe "Set target path" task is setting target_dir to the literal here: but it's supposed to be using .format() already according to
fungicorvus: i don't suppose you can spot the cause there? it looks consistent with what we did elsewhere... are the unset extra variables maybe causing the .format() function to silently fail?15:34
*** amoralej is now known as amoralej|off15:34
corvusoh there's a publish_site job variable15:38
fungiwe do put publish_site=publish_site in the .format() there15:40
corvusfungi: i think the promote.yaml playbook assumes publish_site is a plain string, but that's a template15:41
corvusso it would need its own format() call15:41
corvusfungi: tbh, that's not safe15:41
corvuswell, i take that back15:42
corvusit is safe because it's set in project-config and final:true15:42
corvusso should be fine to add an extra .format(zuul=zuul) to publish_site15:43
mordredclarkb: awesome. I'm free then - I've put it on my calendar15:43
corvus(re safety, we just can't let any individual projects override that; but setting final:true fixes that in place in project-config, so it's good)15:43
fungicorvus: yep, good find. in almost all cases we set publish_site to a plain string constant, promote-openstack-specs and promote-airship-project-docs are the two places in project-config we have variable substitution within the string15:46
fungiand they're the two reported problem cases15:46
clarkbfungi: there is a report in #openstack-infra about a similar issue for another job15:47
fungitarget_dir: "{{ target_dict.path.format(zuul=zuul, publish_site=publish_site.format(zuul=zuul), special_publish_directory=special_publish_directory) }}"15:47
funginested .format() calls like that?15:47
clarkbpromote-deploy-guide for openstack/charm-deployment-guide15:48
corvusfungi: nested should be fine i think15:48
corvusfungi: hrm, that may be a problem if publish_site isn't defined15:48
corvusoh, i guess publish_site is always defined?15:50
corvusyeah, i don't think publishsite=publishsite would work otherwise.  so should be fine15:50
fungimm, i thought previously you had said it looked like unset variables caused them to be ignored in the .format() call, but perhaps it is always set15:56
fungicorvus: also entirely unrelated, but extremely curious... what keeps eating your underscores?15:56
corvusfungi: probably the weechat matrix plugin15:57
fungineat, i haven't tried it yet15:57
fungijust such an odd sort of bug15:58
corvusi think it's treating them as formatting characters16:01
fungiahh, like it thinks you're doing _underline_ markup?16:03
fungicorvus: so, in order to test the nested .format() calls with base-test, are we pretty much limited to making alternative versions of some of the jobs in project-config which descend from base-test and then switching some project to using those, since this is promote pipeline stuff?16:06
corvusfungi: yeah; i'd probably just yolo it in and watch the first promote run; at least it won't break everything16:11
*** marios|ruck is now known as marios|out16:12
fungiyeah, it's a pragmatic compromise in this case, otherwise it'll take upwards of a day to get fixed and possibly cooperation of additional teams to merge some things16:13
fungii'll get the presumed fix pushed up now16:13
opendevreviewJeremy Stanley proposed opendev/base-jobs master: Format publish_site in docs promotion
fungiroman_g: rosmaita: pmatulis: frickler: clarkb: corvus: ^ that's the proposed solution16:25
clarkbfungi: great I'll take a look shortly16:26
clarkbI've +2'd the change. I've held off approving it as I think pmatulis' promote-deploy-guide job doesn't set publish_site and I'm not sure what effect that will have with fungi's fix16:37
clarkbI guess it will just fail the job bceause the var isn't set?16:37
fungii'll take a closer look at the promote-deploy-guide failures, since if it's not setting publish_site the problem with it may be separate anyway16:38
opendevreviewMartin Kopec proposed opendev/system-config master: refstack: trigger image upload
clarkbI guess we can land the change and fix everything but promote-deploy-guide then they can psuh and update to project-config to set the publish_site16:38
fungiis that the only consumer which needs it set but doesn't have it?16:39
clarkbof that I am not complete sure. It does seem like a number of those jobs do set the value though16:39
fungiyeah, the failure on is separate i think16:41
fungiit's dying when the "Set target directory if master" task tries to expand {{ afs.targets.master }}16:42
corvusyeah, if the var isn't set, it should already fail16:43
clarkbpromote-deploy-guide has a different parent too16:43
corvusadding a format to it won't change that16:43
clarkbthe publish_site using jobs all have promote-tox-docs-site-base as a parent but promote-deploy-guide's parent is opendev-promote-docs-base16:44
clarkbIs publish_site specific to promote-tox-docs-site-base ? if so romote-deploy-guide has a similar target_dir dict problem but for another reason?16:44
fungiit looks to me like it's saying the afs dict doesn't have a targets key at all16:45
fungiit should be getting set in the secret16:46
clarkbafsdocs_secret-deploy-guide specifically16:47
clarkbif I search that name I don't get any results16:47
clarkbis the secret just not set at all?16:47
clarkbI suppose it could be on a stable branch and hound isn't seeing it16:48
fungiit's in opendev/base-jobs as afsdocs_secret-deploy-guide16:48
fungiand does have a target with a master path16:49
*** jpena is now known as jpena|off16:49
clarkbthe secret needs to be in the repo defining the job though?16:50
fungier, i mean in openstack/project-9config16:50
clarkbThe error was: 'dict object' has no attribute 'targets'16:51
clarkbwe have target but not targets ?16:51
fungigreat eye16:51
fungiyep, typo. that is probably my error too16:51
fungii think i fixed all these up post-upgrade16:51
fungii'll get a fix pushed for that after i double-check it's the only one like that16:52
clarkbsounds good16:52
opendevreviewJeremy Stanley proposed openstack/project-config master: Correct targets for afsdocs_secret-deploy-guide
fungipmatulis: clarkb: ^ that ought to solve it16:54
fungiit was indeed a completely separate problem16:54
clarkbfungi: should I approve the base-jobs change now?16:55
clarkbI had only +2'd it previously but corvus  also +2'd which means if you are happy with it we can land it16:56
fungiyeah, just want to watch subsequent promote jobs as soon as it lands16:57
fungiif we have something we can approve which would trigger one of the known problem jobs that would help too16:57
clarkbroman_g: rosmaita ^ may have something?16:57
fungiwe'll of course also be equally watching for any other unrelated docs promotion failures this fix unexpectedly might result in16:59
roman_gfungi clarkb Thank you!17:01
roman_gYes, we would be merging today.17:01
roman_gI've submitted small PS for merge in airship/airshipctl repo. It should trigger docs update.17:04
*** anbanerj|rover is now known as frenzy_friday17:07
fungiroman_g: the fix hasn't merged yet, but looks like you already approved that change17:07
roman_gfungi Oops. I will find something else.17:07
fungior you could reset your workflow vote if you catch it before it merges, and then approve again after we get the fix in17:08
fungiif the workflow +1 isn't there when the jobs complete, gerrit won't let zuul merge it and the review will stay open until it gets approved again17:08
roman_gHave reset it.17:09
roman_gGate jobs are just starting.17:09
fungithey'll still run and report success or (in this case) failure, but the change won't merge without the workflow +1 present when they do17:10
roman_gI would be offline, will submit the change later today. Will read channel logs.17:10
fungii've self-approved the publish_site fix now17:11
roman_gWrite here when it gets merged.17:13
roman_gWill be back later today.17:13
fungiroman_g: i'll let you know, definitely, thanks for the help!17:13
roman_gfungi Thank you!17:13
opendevreviewMerged opendev/base-jobs master: Format publish_site in docs promotion
fungiroman_g: you should be clear to approve 797732 as a test any time now, thanks again!17:35
fungiwatching we don't have any newer results yet, but there have been a lot of promote-openstack-releasenotes failures, looking into those next17:37
fungithe field 'args' has an invalid value ({'target_dir': '{{ target_dict.path.format(zuul=zuul, publish_site=publish_site, special_publish_directory=special_publish_directory) }}'}), and could not be converted to an dict.The error was: Single '}' encountered in format string17:38
fungilooks like we missed switching double {{ }} somewhere for that17:38
fungioh, or maybe that's not it. delving deeper17:40
fungiaha, looks like maybe another typo17:41
fungi"path":"/afs/{{ zuul[project][short_name]}/"17:41
fungiyep, found it in project-config, another typo of mine!17:44
opendevreviewJeremy Stanley proposed openstack/project-config master: Correct branch path in afsdocs_secret-releasenotes
fungithat ought to address it17:45
pmatulisfungi, awesome. i'll test it out17:49
fungipmatulis: well, the deploy guide fix hasn't merged yet, but it's got a core review on it so i'll self-approve it now17:53
fungiit's a trivially obvious patch anyway17:54
fungias is 798938 above if any config-core reviewer is available to take a look17:55
opendevreviewMerged openstack/project-config master: Correct targets for afsdocs_secret-deploy-guide
fungipmatulis: ^ you should be clear to try again now18:01
pmatulisperfect thanks18:02
fungiugh, many promote-openstack-tox-docs failures now18:11
fungiThe task includes an option with an undefined variable. The error was: 'publish_site' is undefined18:11
fungithat's using afsdocs_secret-tox-docs18:13
fungiwhich indeed does not set publish_site18:14
fungicorvus: is there a trivial way to default that to an empty value in the playbook? or do i need to set empty strings for it in every consuming job?18:17
fungi(every consuming job which doesn't already set it)18:17
fungii guess that would be every job parented to opendev-promote-docs-base, or maybe we can set a fallback value there18:18
fungii guess an earlier task in the playbook can check whether publish_site is undefined and then set it to some empty value18:20
fungier, that probably runs afoul of the var freezing though18:20
fungicould a task locally set a variable based on the value of publish_site if defined with a default of some sort, and then we use that in building the path rather than directly consuming publish_site?18:23
fungifinding documentation on the .format() method is not an easy task. in part because i'm not sure if that's handled by directly jinja, ansible, or python18:30
fungihelp from anyone with more than my limited, hamfisted ansible knowledge would be most welcome18:31
fungithe word "format" turns up so frequently in things not related at all to that method, i'm having trouble finding out what's responsible for handling it18:32
fungiif ansible were handling it, looks like it would be via a filter construction such as |filter() rather than as a . attribute of the var18:34
corvusfungi: huh, i thought it would have failed before adding the format, was i wrong about that?18:36
pmatulisfungi, failure:
fungii don't know that you were wrong, all i know is that the error looks like this:
corvusoh weird18:37
fungipmatulis: yes, that's the new regression i'm trying to understand now18:37
corvusso "{foo}".format(foo=foo, bar=bar) fails if foo is not defined, but does not fail if bar is not defined18:38
corvusthere's definitely some not-quite-python magic going on there18:38
fungioh yeah i agree that's odd18:39
corvusfungi: i think you'll either need to make sure its always defined, or add a separate task18:39
fungicorvus: see above, will forcing a default when undefined be possible with the var freezing introduced in 4.6.0?18:40
fungior will we need to use it to set a value for some local proxy variable instead?18:40
corvusthe second was what i was thinking.18:42
corvus```  - set_fact:18:44
corvus      publish_site: "{{ '{x}'.format(x=publish_site) }}"18:44
corvus    when: publish_site is defined18:44
* corvus < >18:45
corvusfungi: ^ i think that would work18:45
corvuser not quite...18:45
corvus  - set_fact:18:46
corvus      publish_site: "{{ publish_site.format(zuul=zuul) }}"18:46
corvus    when: publish_site is defined18:46
fungicorvus: aha! so setting a fact will let us substitute for the original var, i guess?18:46
fungioh, shadowing it when it _is_ defined18:46
corvusfungi: there -- i think that's what's needed.  yep, so that should do the formatting if the var exists, and if it doesn't, we'll rely on the existing magic behavior that doesn't cause an error if it's not defined.18:46
opendevreviewJeremy Stanley proposed opendev/base-jobs master: Separately set publish_site for docs promotion
fungicorvus: so like that ^18:50
corvusfungi: lgtm!18:52
fungipmatulis: roman_g: 798945 will hopefully take care of the newest regression and then we can check again18:55
pmatuliswill wait to test again18:57
opendevreviewMerged opendev/base-jobs master: Separately set publish_site for docs promotion
roman_gfungi shall I wait?19:07
fungiroman_g: pmatulis: it *may* be fixed now that 798945 has merged, so worth trying19:13
fungii'm back to checking promote jobs to see if that's solved the larger regression19:13
fungiso far no new promote pipeline jobs have reported since that merged, aside from a promote-openstack-releasenotes which is separately broken (fix for that is up for review as 798938)19:15
roman_gRunning Post jobs19:18
roman_gPromote will start in about 10min19:18
fungiroman_g: there's this which just succeeded:
roman_gfungi problem solved. Works perfect.19:21
fungilooks like it wrote into /afs/ so that seems correct19:21
fungiroman_g: thanks for confirming!19:21
fungicorvus: ^ looks like that was the solution, thanks!19:21
roman_gfungi thank you! AFS probably needs some cleanup.19:21
fungiroman_g: hopefully not since we publish with rsync --delete which in theory should remove the old trees, but worth double-checking19:22
roman_gHave a great rest of the day and thank you all for your support.19:22
fungipossible the erroneous paths were entirely outside the normal sync path though, so i'll double-check and clean up any stray cruft19:23
fungiroman_g: you too, and thanks for bringing the issue to our attention119:23
fungiwe also had a successful run of promote-openstack-tox-docs for openstack/os-brick so the most recent regression is definitely addressed now19:25
opendevreviewMerged openstack/project-config master: Correct branch path in afsdocs_secret-releasenotes
*** dviroel is now known as dviroel|out20:41
clarkbinfra-root (particularly ianw and corvus I think) upstrema gerrit is discussing Depends-On and how to support it in plugins. Apparently the zuul plugin already does support it and qualcomm has another plugin that wants to support it.21:20
clarkbI don't really have super stron opinions on it. I think its probably ok to have multiple plugins support the same behaviors as long as the behaviors are similar21:21
clarkbbut others may have better ideas than "ya go for it" and want ot catch up on the upstrema thread21:21
corvusclarkb: thanks for the heads up; i don't think i have much to add to that21:42
fungiother than "great idea, wish i'd thought of it! (oh, wait...)"21:51
clarkbI did end up responding because zuul support for circular deps came up and I noted zuul does support them now21:52
fungiif memory serves, depends-on footers came about as a result of discussions we had at our mid-cycle in manhattan... is that right?21:55
mordredI know we discussed circular deps in manhattan21:57
mordredbut I have a non-substantiated feeling-ish memory that depends-on itself pre-dates that?21:57
clarkbmy memory doesn't go back that far21:58
fungiyeah, though i want to say circular deps came up because we hadn't nailed down the implementation for cross-project change dependencies yet. i'm likely misremembering21:58
mordredI feel like people discussed things, and some of them wanted things that were silly, but some of them wanted things that were reasonable21:58
fungiyes, there were opinions22:02
funginot all of them compatible22:02
corvusopinions were had22:03
corvusmordred: depends-on originally came from gerrit itself and i believe that did pre-date manhattan22:03
persiaWhile I wasn't part of any of the discussions in manhattan, my memory of when that occured suggests that depends-on predates that, but extending depends-on to generic cross-project (or cross-hosting solution) does not.22:03
fungioh, gerrit added depends-on tracking in commit footers?22:03
corvusfungi: no, but there was a patch for it22:04
fungipersia: definitely cross-hosting syntax came much later in zuul v322:04
corvusthat everyone thought would be merged; but then it wasn't.  it may have gotten caught up in the same CLA snafu as single-page-diff22:04
fungiso we thought zuul's depends-on would be compliant with a feature which ended up not being carried in gerrit. this seems like a recurring theme ;)22:05
corvusyep, and so i think our picking that up and extending it to support multiple sources is appropriate since gerrit itself abdicated it22:06
fungi was the change to implement it in zuul, first patchset uploaded new year's eve of 201422:07
fungii seem to have approved it on february 122:08
corvusi know how to par-tay22:09
ianwclarkb / corvus : came out of me trying to use speculative images yesterday.  i might be wrong, but i don't see where system-config pulls from the intermediate registry?  could you take a look at that change?22:10
clarkbneat we fixed my concern with zuulv3 :)22:10
clarkbianw: I think it relies on the parent jobs configuring docker to use the intermediate registry first then fallback to dockerhub22:11
clarkbianw: I want to say what your change is trying to do there is supposed to already happen automatically via the job hierarchy22:11
ianwclarkb: it's very possible i've got something wrong, but i couldn't seem to make that happen22:12
corvusianw: i think you just need to run opendev-buildset-registry22:12
ianwthis is what i had assumed too, but is running opendev-buildset-registry and didn't pull in the depends-on change22:14
ianw(i have a few comments with links @ 798789)22:14
corvusianw: does the run job run use-buildset-registry?22:15
ianwyes, here it is running on paste which is where we want to pull the image
corvusianw: gotcha; then i think your change may be correct22:19
corvusianw: i think that probably is the missing piece; i don't know off the top of my head if there's something else we expected to run the pull (i'm assuming you've looked and didn't find anything).22:20
ianwyeah, i could not find anywhere other than build-docker-image @
ianwi considered maybe just putting it in the opendev base job toott22:22
ianwit seemed like it should do nothing if no buildset registry defined22:22
clarkbso we add the pre playbook for image builds which populates the buildset registry in jobs that are't building imgaes?22:24
ianwclarkb: it's perhaps misnamed @
ianwit only runs the one task which is pull-from-intermediate-registry22:26
clarkbianw: ya it was added for the imgae builds so it got the name. I don't know that we need to rename it just making sure I'm follwoing along22:26
clarkbbasically we do the right thing when building images but not when consuming them?22:26
ianwyes, since that role runs as part of build-docker-image, whenever we're in the context of that running against our buildset registry things work22:27
clarkbopendev-build-docker-image-base seems to be equivalent to your chang but it adds the secret in too22:27
clarkbdo we need the secret? if yes then maybe just reuse opendev-build-docker-image-base if not then maybe we should note that in the new base job22:28
ianwi would think we only need the secret to push, which in this case (consuming lodgeit container) we don't22:29
clarkbgot it22:30
ianwhowever, perhaps something like that which also starts the opendev-buildset-registry would work?22:30
clarkband ya I agree the intermediate registry allows anonymous reads22:30
ianwthat might mean jobs with this theoretical "opendev-use-docker-image-base" parent don't have to also run opendev-builset-registry?22:31
ianwwhat happens if we have two buildset registries, though?22:32
clarkbdo the registry all in one? I think we colocate into a central paused job buildest registry to avoid extra IO to the various registries but I think that would work22:32
clarkbreally the balance is limiting bw usage to the outside world aiui22:32
ianwhrm, there might be room for both then.  one like proposed for something like system-config that runs a lot of container jobs, and a self-contained one if you have just one job that speculatively consumes another project's containers22:34
ianwi'm guessing nobody actually has a project like the latter atm22:35
clarkbya it seems to be common to grow multiple jobs. For example Zuul uses the same images all over but then runs a ton of different jobs on them22:36
clarkbI +2'd the base jobs change. There rae no secrets involved and it adds a new job without modifying existing jobs That should mean its safe to land without any additional testing or exposure concerns22:39
ianwi think statusbot from system-config would have the same problem; but i probably never tried a depends-on for any changes 22:40
clarkbianw: I'll let you approve it if you want to proceed with that method of using the buildset registry rathe rthan all in one22:40
corvusthat's still using the all-in-one22:40
clarkboh is it?22:40
corvusiiuc the idea would be to reparent the run job on that so that it pulls in the dependency22:41
corvusbut it'll still pull it in to the buildset registry22:41
clarkbya rereading the docstring on the job that seems to be the case22:41
clarkbwill it pull ot a central buildest registry too?22:41
clarkbsupporting both cases?22:41
corvus(btw, i don't think we should have more than one job run a buildset registry)22:42
corvus(it's called a buildset registry for a reason :)22:42
ianwcorvus: yeah, i think we're saying in contrast to there is no dependency on opendev-buildset-registry in the job22:43
corvusthe natural next step to evolve this would be to collect the artifacts for the whole buildset so that the buildset registry job can do the pulling for all the jobs, then this change wouldn't be needed.22:43
ianwso you have to arrange for your project to be running that22:43
corvusyep, and system-config does22:44
ianwwhereas we could *also* add a job "opendev-use-docker-image-base" that has a " parent: opendev-buildset-registry" for projects to use if they just want to consume someone else's container22:44
corvusplease don't; the buildset registry is not designed for that22:45
clarkbnow I'm confused again. The change that has been +2'd is designed to work with a central buildest registry run in a paused job right?22:46
corvusit's designed to work with a buildset registry; it doesn't care where it runs22:47
corvus(which is the right way to deal with a buildset registry :)22:47
clarkbwell if you inherit from that job and then run the buildest registry in pre-run that will be too late due to the nesting behavior of jobs22:47
clarkbwhich means you're probably looking for a buildest registry elsewhere and not an all in one setup22:47
ianwcorvus: i'm not sure i follow why has a parent of opendev-buildset-registry but a sibling "opendev-use-docker-image" would be a wrong pattern22:50
corvusthat's fine, it does the right thing; i'm just saying please don't run more than one buildset registry22:52
corvusi think there are too many ideas being discussed at once, sorry for the confusion22:52
ianwahh, right, yes we all agree that one buildset registry per ... buildset ... is the way to go (it's in the name :)22:54
ianwnow i look at i feel like i could add some relevant info on consuming images that would be helpful22:55
opendevreviewSteve Baker proposed openstack/diskimage-builder master: Add a growvols utility for growing LVM volumes

Generated by 2.17.2 by Marius Gedminas - find it at!