Wednesday, 2019-10-16

*** jamesmcarthur has joined #zuul00:06
*** jamesmcarthur has quit IRC00:18
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add support for selecting the kubernetes version
*** igordc has quit IRC01:47
*** spsurya has joined #zuul01:59
*** bhavikdbavishi has joined #zuul03:24
*** bhavikdbavishi1 has joined #zuul03:27
*** bhavikdbavishi has quit IRC03:28
*** bhavikdbavishi1 is now known as bhavikdbavishi03:28
*** raukadah is now known as chandankumar04:29
*** pcaruana has joined #zuul04:37
*** pcaruana has quit IRC05:14
*** saneax has joined #zuul05:52
*** pcaruana has joined #zuul06:13
*** hashar has joined #zuul06:19
*** tosky has joined #zuul07:17
*** themroc has joined #zuul07:19
*** jpena|off is now known as jpena07:39
*** bolg has joined #zuul07:39
*** bhavikdbavishi has quit IRC07:40
bolgtobiash: I was toying yesterday with the mandatory SQL connection ( a bit. This would need to adapt most of the tests which use ZuulTestCase to ZuulDBTest case. Just wanted to check if that is what is meant in since this would be mean significant efforts.07:45
tobiashbolg: actually I started with this already a few months ago:
openstackgerritTobias Henkel proposed zuul/zuul master: Enforce sql connections for scheduler and web
openstackgerritTobias Henkel proposed zuul/zuul master: Enforce sql connections for scheduler and web
openstackgerritTobias Henkel proposed zuul/zuul master: WIP: Add spec for scale out scheduler
bolgtohiash: ok missed that, i will look at it08:00
*** themr0c has joined #zuul08:02
*** themroc has quit IRC08:03
fboHi @zuul-maint is there any blocker that prevent the Elasticsearch reporter to be merged ?
bolgtobiash: maybe we should sync sometime on the topic of scale-out-scheduler08:09
*** jangutter has joined #zuul08:14
*** gtema has joined #zuul08:21
*** themr0c has quit IRC08:38
*** themr0c has joined #zuul08:38
*** avass has joined #zuul08:48
*** hashar has quit IRC08:50
openstackgerritFelix Schmidt proposed zuul/zuul master: Build GitHub URLs for tags correctly
*** hashar has joined #zuul09:01
*** themr0c has quit IRC09:09
*** panda is now known as panda|drappt09:23
*** hashar has quit IRC09:57
openstackgerritFelix Schmidt proposed zuul/zuul master: Build GitHub URLs for tags correctly
*** gtema has quit IRC10:08
*** themroc has joined #zuul10:13
*** fsvsbs has joined #zuul10:14
*** fdegir has quit IRC10:16
*** fdegir has joined #zuul10:17
*** themr0c has joined #zuul10:17
*** themroc has quit IRC10:18
*** themr0c has quit IRC10:20
*** themr0c has joined #zuul10:20
*** hashar has joined #zuul10:29
*** themroc has joined #zuul10:34
*** armstrongs has joined #zuul10:34
*** themr0c has quit IRC10:34
*** bhavikdbavishi has joined #zuul10:35
*** bhavikdbavishi1 has joined #zuul10:38
*** bhavikdbavishi has quit IRC10:39
*** bhavikdbavishi1 is now known as bhavikdbavishi10:39
*** themr0c has joined #zuul10:44
*** themroc has quit IRC10:45
*** themr0c has quit IRC10:57
*** rfolco|ruck is now known as rfolco|brb11:01
*** jpena is now known as jpena|lunch11:27
*** markw18 has joined #zuul11:36
markw18Hi there,11:37
markw18Apologies still typing I just hit enter by mistake11:37
markw18I was just wondering what the rational was around this commit: we are currently using the hashicorp_vault plugins to lookup secrets within our Zuul jobs, however after updating to the latest version of Zuul we found the plugin to be blocked.11:38
markw18We would really love it if there was a way of having a whitelist of allowed lookup plugins as it's really only the vault lookup plugins that we depend upon11:40
markw18Failing that are there any plans on the cards for directly offering support for looking up secrets from vault actually within the Zuul app itself ?11:41
AJaegermarkw18: better look at
AJaegerSo, shouldn't this *Fix* using hashi_vault?11:42
markw18No this actually blocks the usage of the vault plugin, so we now see messages like the following:11:43
markw18An unhandled exception occurred while running the lookup plugin 'hashi_vault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Use of lookup modules that perform local actions on the executor is forbidden.11:43
AJaegerah - so better go back to the orginal broken prevent11:43
AJaegermarkw18: is the change you want to look at11:44
markw18Yes thanks :)  we are able to do that for now but we were just wondering what the plans were long term ? I.E is there plans to add a whitelist of allowed plugins and or adding the support for looking up secrets from hashicorp vault directly in Zuul ?11:45
AJaegermarkw18: sorry, can't answer that, you might need to wait until others are awake11:45
markw18No worries thanks anyway much appreciated !11:46
fungimarkw18: if you look at the other plugins allowed, you'll see many of them need to stub out dangerous methods which allow things like writing files to disk. would probably need to audit that plugin for similar risks11:47
fungi is one i did not long ago because we wanted to be able to use the password lookup plugin to generate random passwords in jobs11:48
openstackgerritMerged zuul/zuul master: Remove python-path auto release note
markw18Ok, in the case of the hasicorp vault plugin it shouldn't be doing anything other than looking up the secrets and not writing to disk etc, thanks for the PR link I will have a read11:50
fungimarkw18: may be an easier way to read it11:52
fungifor some reason gerrit isn't displaying the zuul/ansible/base/lookup/ addition (it replaces a symlink, so that's probably why)11:54
*** rfolco|brb is now known as rfolco|ruck11:55
fungithe password lookup module had functions which allowed specifying a file path, and writing the result to a temporary file11:55
fungithose seem innocuous, but could in theory be used to replace or read arbitrary files as the user running ansible11:56
markw18Perfect thanks :)  much appreciated11:56
fungiso just need to read through the vault plugin and make sure the things its able to do are tightly scoped and that we don't expose functions or parameters which could cause it to inadvertently do more11:57
fungithough we've had discussions about dropping the current plugin filtering entirely and just relying on bubblewrap's sandboxing as the only layer of security there11:58
*** themr0c has joined #zuul11:59
fungiit would be easier on everyone, but requires we put an increased amount of faith in the process and filesystem isolation measures bwrap provides11:59
markw18Indeed, although I'd say if possible then yes that would be great12:00
zbris there a generic way to define an env variable for a zuul job?12:12
*** jamesmcarthur has joined #zuul12:13
*** jamesmcarthur has joined #zuul12:13
*** rlandy has joined #zuul12:14
zbrnevermind, i found one way of doing it, i was confused by tox_environment variable with is an shell env, not the tox env :p12:16
zbrvery confusing name12:17
*** jamesmcarthur has quit IRC12:26
*** jpena|lunch is now known as jpena12:31
jangutter(wall-of-text-start) Found two interesting entertaining digressions yesterday and today just posting them here for chuckle values.12:41
jangutterFirst one: we're running software-factory and zuul, and due to the scl python there are persistent ansible warnings for long unix domain socket paths.12:41
jangutterSymlinks don't help, since the full path is resolved. The workaround: set job_dir=/var/lib/zuul/b to save 3 extra characters.12:41
jangutterSecond one: we've been scratching our head with Ansible unreachable errors. Turns out, if your script returns 255, it looks like an ssh error.12:41
jangutterWorkaround, _don't return -1 as an error, return 1_. (wall-of-text-end)12:42
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: Pull minikube log data
Shrewsjangutter: yeah, i've encountered the too-long-socket-path error in zuul in the recent past. I forget what problem I was seeing, but it was not obvious based on the error12:52
*** hashar has quit IRC12:52
*** jamesmcarthur has joined #zuul12:52
jangutterShrews: to be honest, I think it's a warning, not a problem - but since it was the last error we saw, we thought that was the cause.12:53
Shrewsit created some sort of error for me. like i said though, don't recall the context  :/12:54
jangutterShrews: Ansible seems to fall back to some other working strategy. It makes a bit of noise in the logs though.12:54
Shrewsswest: we seem to have a race in your new test_static_waiting_handler_order test:
Shrewsswest: any chance you can look into that?13:24
*** michael-beaver has joined #zuul13:28
*** saneax has quit IRC13:40
*** bhavikdbavishi has quit IRC13:41
*** jangutter_ has joined #zuul13:57
*** brendangalloway has joined #zuul13:58
*** jangutter_ has quit IRC14:00
*** jangutter has quit IRC14:00
*** jangutter has joined #zuul14:01
*** swest has quit IRC14:13
corvusi'll tag 88708660f774e506681921bb315c07c25d78e0e5 as zuul 3.11.0 -- sound good?14:28
*** mgoddard has quit IRC14:28
*** mgoddard has joined #zuul14:30
fboI'm getting zuul-build-image and zuul-quick-start job failure on a patch that seems to not be related . Is there an issue with those jobs atm ?14:34
corvusfbo: could be an issue with the intermediate registry; i'll take a look in a minute14:34
*** chandankumar is now known as raukadah14:35
fbocorvus: thanks14:36
*** jangutter has quit IRC14:39
*** jangutter has joined #zuul14:40
*** jangutter_ has joined #zuul14:44
*** jangutter has quit IRC14:47
corvusfbo: 679938 had a post-failure on zuul-build-image, and it looks like the later changes were trying to get the images from the failed build; i think if we recheck them now, they'll find the new successful build.  i've done that.14:49
corvus(this is probably something about the provides/requires that we could improve)14:49
*** avass has quit IRC14:52
fbook thanks we'll see if the jobs pass14:52
*** jangutter has joined #zuul14:59
clarkbcorvus: yes looks like opendev is running b768ece and the only changes after that to 88708660f774e506681921bb315c07c25d78e0e5 are bindep and release note changes15:00
clarkbcorvus: 88708660f774e506681921bb315c07c25d78e0e5 as 3.11.0 lgtm15:01
*** jangutter_ has quit IRC15:03
corvuscool, i'll push it then15:03
*** panda|drappt is now known as panda15:09
*** mattw4 has joined #zuul15:20
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: WIP: Cleanup k8s playbook
*** hashar has joined #zuul15:43
*** rfolco|ruck has quit IRC15:49
*** themr0c has quit IRC15:52
Shrewshrm, the nodepool k8s job seems to always fail in ovh15:58
*** jpena is now known as jpena|off16:00
ShrewstristanC: can you glean any useful info from ?16:00
clarkbFailed to list *v1.Namespace: Get dial tcp connect: no route to host16:01
clarkba networking problem I think16:01
clarkbdumping ip addr and ip route output might help16:02
Shrewscoredns (whatever that is) seems to be crashing?16:02
corvusclarkb: this is from the start of the job
corvusclarkb: i guess you mean after the minikube crash?16:02
clarkbcorvus: yes after k8s has modified things16:03
tristanCShrews: coredns was also crashing in successfull logs16:03
*** bolg has quit IRC16:03
ShrewstristanC: oh neat  :)16:03
tristanCShrews: looking for the pod id, this line seems suspicious: kubelet[7911]: W1016 15:49:12.115521    7911 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for main-0000000001/pod-fedora through plugin: invalid network status for16:05
clarkbis it possible the /32 network for the host confuses k8s?16:05
tristanCunfortunately it seems like the line is truncated right before valuable information :)16:05
clarkb(that is a known network difference on ovh)16:05
tristanCShrews: iiuc it is a docker related issue, do we collect docker logs too?16:06
ShrewstristanC: doesn't appear we do16:07
corvusShrews, tristanC: there's some code in system-config to grab all docker logs16:09
Shrewscorvus: oh excellent. i'll see about adding that to the job. thx16:09
corvusShrews: playbooks/zuul/run-base-post.yaml16:10
*** gtema has joined #zuul16:11
*** markw18 has quit IRC16:12
*** rfolco has joined #zuul16:15
Shrewsi think that's probably doing what 'minikube logs' is already doing? just pulling the container logs16:15
*** rfolco has quit IRC16:18
tristanCShrews: is the FN ipv6 issue solved for the openshift integration test?16:18
*** rfolco has joined #zuul16:18
ShrewstristanC: not sure. i haven't noticed any failures and haven't actively been looking for successes16:19
*** hashar has quit IRC16:19
*** rfolco is now known as rfolco|ruck16:20
clarkbShrews: tristanC we believe the underlying network issue is fixed16:20
clarkbwhether or not that was the only issue with that job on that platform I don't know16:20
tristanCclarkb: ianw: thanks a lot for working on the fedora ipv6 issue!16:28
*** gtema has quit IRC16:28
Shrewsah, the entire /var/logs/docker contents are pulled in the system-config playbook. i think that's what we need16:29
openstackgerritJames E. Blair proposed zuul/zuul master: Fix skipping child jobs with soft dependencies
corvusclarkb, tobiash: ^ that bug was introduced in the just-release 3.11.0 -- maybe we want to land that and cut 3.11.1 ?16:33
corvusalso, you can see the bug in action now at
corvus(system-config changes in check)16:33
clarkbreviewing it now16:36
*** hashar has joined #zuul16:38
toskyuhm, my heavily conflicts with tristanC's
corvustosky: yesterday we were discussing reworking tristanC's patches... it sounds to me like that may be the approach we take, so i think you may be clear to proceed, and the new versions of tristanC's patches should be simpler16:42
tobiashcorvus, clarkb: lgtm and ++ for release16:42
corvustobiash, clarkb: thanks -- i'll wait for that to land -- should we restart opendev zuul before releasing 3.11.1, or do we think this is tested enough?16:43
clarkbthe test case should cover it pretty well I think. But happy to restart opendev zuul as long as openstack release team is done with their release jobs16:44
clarkb(I think they are but we should confirm)16:44
tobiashsince we have test cases for soft deps, hard deps and now this case I guess the test cases are probably enough16:45
tobiashbut if you feel more comfortable testing it in opendev go ahead16:45
*** mattw4 has quit IRC16:57
*** ryanpetrello has joined #zuul17:00
ryanpetrelloo/ fungi17:00
*** mattw4 has joined #zuul17:00
ryanpetrelloany chance you have control over the contents of ?17:00
ryanpetrellospecifically, "Tower, on the other hand, is meant to trigger arbitrary Ansible playbooks on demand via arbitrary human inputs."17:01
clarkbryanpetrello: we know the people that can update that17:01
ryanpetrelloas of very recently, this isn’t true, AWX/Tower has an HTTP API you can integrate with, and it also has direct API support for triggering playbook runs via webhook events from github and gitlab17:01
clarkbryanpetrello: what would be a more appropriate differentiation?17:01
ryanpetrellozuul's control over this is much more robust and configurable via source control like other zuul things17:02
clarkb(also I totally solicited feedback on that on the zuul mailing list :) )17:02
ryanpetrelloAWX/Tower is controlled by an API (you wouldn't put the config in source control, for instance)17:02
ryanpetrelloyea, it's a _very_ recent change we've made to AWX17:02
ryanpetrellolike, in the past month17:02
ryanpetrellobut it *is* a bit of functionality we're proud of, and it's a use case AWX/Tower users have wanted for some time, so I'm just interested in having accurate info out there17:03
ryanpetrellozuul definitely has the upper hand here in terms of robustness17:03
ryanpetrellobut AWX/Tower *does* have some support for webhook integration with both Github and Gitlab today17:03
ryanpetrello(we're zuul users ourselves, because it's awesome :D)17:04
fungias far as our faq in the zuul docs is concerned, we could likely do a better job there of suggesting where zuul/tower integration opportunities lie, and how they can be complimentary, to defuse the competitive image some folks might imagine17:05
clarkbryanpetrello: what about "Historically Tower has primarily been used to trigger Ansible playbooks via arbitrary human inputs. Recently Tower has grown an HTTP API and support for GitHub and Gitlab webhooks. While these new features overlap with Zuul you will get a more robust CI experience through Zuul as it has built in support for features like gating." is that better?17:05
fungi"but don't these do the same thing? nope, in fact here's what you can do if you combine them..."17:05
*** michael-beaver has quit IRC17:08
clarkbryanpetrello: basically I'm happy to provide edits to the superuser team, but am not quite sure what the best way to position this is. Zuul and Tower are different ( as evidenced by tower using zuul )17:08
clarkbunrlated to tower
*** jamesmcarthur has quit IRC17:09
*** calebb has joined #zuul17:09
corvusand in fact, we have discussed having opendev's systems managed by awx and having that triggered by zuul -- that's the kind of 'use both systems for complete coverage' that i think is suggested by fungi's comment17:10
corvussometimes you need to compare apples and oranges, and it's hard.  :)17:13
fungii often compare them while eating fruit salad17:14
fungiapples and oranges can go very well together17:14
*** jangutter has quit IRC17:17
ryanpetrello@clarkb I think if it mentioned how zuul had more robust support for handling events from source control, that would be most accurate17:26
ryanpetrellobut AWX/Tower does have *some* support, so the current text blurb is inaccurate17:26
ryanpetrello@clarkb your suggestion above looks good to me17:26
clarkbryanpetrello: ok let me make an etherpad with the entire paragraph just to confirm and then I'll get that to the superuser team17:27
ryanpetrelloRecently Tower has grown HTTP API support for GitHub and Gitlab webhooks17:27
ryanpetrello(we've had an HTTP API _itself_ for years)17:27
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: WIP: k8s logging
clarkbryanpetrello: how does that look17:31
clarkbryanpetrello: I tried to incorporate the bit about source code events too17:36
ryanpetrelloyep, I think that's a great summary17:37
ryanpetrellothanks for putting that together17:37
clarkbnp passing that along now17:37
fungithanks for the update ryanpetrello!17:39
fungiit's unfortunately inevitable that articles are already slightly outdated by the time they go to print17:39
fungiso errata to bring them current at time of publication is always appreciated17:40
fungithat's just a point-in-time article though, so longer-term we very much appreciate whatever adjustments you can propose to the faq we're trying to maintain as a living document here: (granted that question isn't included in it yet)17:41
raukadahfungi: is it possible to get a feature wise comparison with jenkins vs zuul in faq, it is common question always got asked17:42
*** michael-beaver has joined #zuul17:43
fungiraukadah: i think we're working out who can get zuul entries added to so that will probably be something useful to start from17:44
fungiwe also want to get updated to mention zuul, i think17:44
fungimostly came down to working out who we know that's already a wikipedia editor17:45
clarkbryanpetrello: update made to the article17:46
ryanpetrellothx @clarkb17:46
raukadahfungi: thanks :-)17:46
*** jangutter has joined #zuul17:49
openstackgerritJames E. Blair proposed zuul/zuul master: Fix skipping child jobs with soft dependencies
corvusclarkb, tobiash: ^ fixed pep8 and added a release note (due to our informal 'no releases without release notes' rule)17:53
*** jangutter has quit IRC17:53
*** spsurya has quit IRC17:58
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: Grab docker logs in k8s job
*** mattw4 has quit IRC18:13
*** mattw4 has joined #zuul18:15
*** hashar has quit IRC18:18
*** hashar has joined #zuul18:18
*** jangutter has joined #zuul18:22
*** jamesmcarthur has joined #zuul18:23
*** mattw4 has quit IRC18:23
*** mattw4 has joined #zuul18:25
corvustristanC: is passing now (yay) -- what do you think we should do about ?  it looks like crio has major.minor version packages, but minikube wants major.minor.micro if you specify the version.  should we just leave it hardcoded and update occasionally?  or maybe use latest k8s, do a package listing to find the latest crio, and use that?18:26
Shrewstobiash: i don't see swest around anymore, but i've seen a couple failures today for the test introduced in should we revert or wait for swest to have a look?18:28
tobiashShrews: if it's urgent, revert, id it can wait until tomorrow, he'd probably fix it18:30
Shrewstobiash: ok, let's wait then. wasn't sure of his availability18:31
Shrewstobiash: thx18:31
tobiashI'll notify him18:31
tristanCcorvus: fedora package is also using major.minor... We could leave the hardcoded but this may break in the future. Though since we pull k8s and cri-o from different source, there isn't much we can do about it18:32
tristanCand perhaps the cri interface is getting more and more stable and mixing an older cri-o with a newer kubelet may also work well18:33
tristanCcorvus: i'll add a quick sanity check (e.g. spawn a pause pod) for cri-o to make sure it's working18:35
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: install-kubernetes: add sanity check for crio
openstackgerritMerged zuul/zuul-registry master: Implement namespacing
openstackgerritMerged zuul/zuul-registry master: Run docker and podman push/pull tests
openstackgerritMerged zuul/zuul master: Fix skipping child jobs with soft dependencies
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: Remove unused k8s log config
ShrewstristanC: corvus: oh, heh, i just realized that the minikube logs also contain the docker service logs19:03
clarkbwith all this k8s activity it might be a good time to take a look at my understanding from mnaser's comments is that this is necessary if you want workload in k8s to be able to resolve dns records outside of the cluster19:04
clarkbtristanC: ^ you may have thoughts on that19:04
corvusyeah i stacked my change on that; lgtm19:05
AJaeger#status log openSUSE 42.3 images have been removed from Zuul19:11
openstackstatusAJaeger: finished logging19:11
clarkbAJaeger: you can #success that one too if you want :)19:11
AJaeger#success openSUSE 42.3 images have been removed from Zuul19:13
openstackstatusAJaeger: Added success to Success page (
Shrewslgtm too19:13
AJaegerargh, should have done this in #openstack-infra... sorry for the noise here19:13
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: install-kubernetes: add sanity check for crio
tristanCShrews: then we may need a lower log level... the current logs doesn't seem to tell why the main-0000000001 pod didn't transition from Pending to Running. beside that kubelet warning about plugin/docker network failure, which doesn't seem to correlate with docker logs19:26
openstackgerritDavid Shrewsbury proposed zuul/nodepool master: Remove unused playbook
ShrewstristanC: which logs do you want to see?19:27
tristanCShrews: i guess docker, kube-apiserver and kubelet19:28
ShrewstristanC: those are all there in the minikube.txt19:28
tristanCShrews: yes but it's too quiet, docker should at least tell when it start or stop a container19:29
openstackgerritMerged zuul/zuul-jobs master: Allow for overriding dns resolvers in install-kubernetes
*** tosky has quit IRC19:30
Shrewslooks like minikube has a -v option for logging level, but i don't know if that affects docker. there is very little documentation on it19:32
tristanCShrews: iiuc, pod creation goes through kube-apiserver <-> kubelet <-> docker, in the minikube.txt there is currently only one reference of the nodepool pod id in apiserver and one in kubelet. Debug log about container creation, status, etc... would help figure out why the job failed.19:33
*** armstrongs has quit IRC19:34
tristanCiirc the kube services argument should be --v=219:34
Shrewsi guess --v=7 is preferable19:35
Shrewsstill no idea if that affects docker itself19:35
Shrewscorvus: would you want to add that option to 688578 to see what effect it has? or shall i try it seperately?19:36
corvusShrews: i think 578 is ready to land as-is (further work around versions i discussed with tristanC can be a followup) so why don't you stack on top of that?19:39
openstackgerritDavid Shrewsbury proposed zuul/zuul-jobs master: Increase minikube logging output to maximum
corvusShrews: i'm restarting right now, so that may need a recheck ^19:44
mnaserdid we wanna look at and ill try to test it out locally after and figure out some basics tests ?19:47
*** michael-beaver has quit IRC19:53
corvusmnaser: lgtm -- let's put a note in #openstack-infra real quick since it's an opendev project19:53
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: install-kubernetes: add sanity check for crio
mnasercorvus: thanks for that, my next ideal thing is getting tagged releases of zuul19:57
mnaserinside dockerhub19:57
corvusmnaser: ++ i think we just need to add plain old "build an image on tag" jobs -- i don't think we should try to use the promoted images for that (it's too complicated)19:58
mnaserokay, ill try to get into doing that and then ask nicely to enqueue that job on a tag19:59
fungimapping the merge commit from the branch back to an artifact from a change merging in the gate pipeline is presumably easier once opendev's zuul pushes merges to gerrit, right?20:00
fungithen we can in theory tie the git tag target commit to an artifact from when a particular change merged20:01
corvusfungi: yeah, that may make the problem easier to solve (it *is* tractable now, but i don't think we need to let perfect be the enemy of good here)20:01
*** hashar has quit IRC20:01
fungimore thinking it's probably not worth having a precise solution to the problem now if we know it's likely to get a good deal easier in the future20:02
corvus(zuul is back up)20:02
fungiso post pipeline build as an interim stop-gap seems fine20:02
mnaseri might be able to get away with some neat workaround20:07
* mnaser hack20:07
openstackgerritMohammed Naser proposed zuul/zuul master: DNM: Tag both 'latest' and version
*** jamesmcarthur has quit IRC20:14
mnaser^ here is an attempt that might just crash and burn terribly20:14
*** igordc has joined #zuul20:15
*** jamesmcarthur has joined #zuul20:16
*** jamesmcarthur has quit IRC20:18
*** mattw4 has quit IRC20:19
mnaserwith python we kinda rely on pbr to help us do the versioning there20:21
*** mattw4 has joined #zuul20:22
tristanCcorvus: seems like cri-o is working as expected, well done! ( )20:22
corvustristanC: \o/  next i'll add a job to zuul-registry to use that, then openshift after that20:23
mnaserwild idea: how do we feel about adding a `zuul_return` with the version from the build-python job20:25
corvusmnaser: that sounds clever20:26
clarkbanother option is to ask pbr what the version is20:26
clarkb(download the image then pbr freeze in it)20:26
mnaserright but that means i would have to make a custom "build-python-docker-job"20:27
mnaserand i guess the upload jobs are trusted so i wouldn't be able to modify those in repo20:27
clarkbeither way tagged versions would have to be handled separately I think (but the tag itslef is that data so that is easy)20:27
mnaseri have a bit of a very "oblivious" attempt here
*** pcaruana has quit IRC20:28
corvusmnaser: i'm not sure what your idea is -- but originally i was just thinking that we stick build-docker-image in to the release pipeline and the tag is {{zuul.tag}} (or whatever it's called)20:28
mnaserbut its naive, with the other one, we'd actually get an image for every single version (say like 3.11.0dev10 or what not)20:28
clarkbcorvus: ya that is sort of what I figured too20:29
clarkb(thats the special tag case I mention)20:29
mnaseryep thats pretty much exactly what i did in there, but i was hoping `zuul.project.checkout` might get me a branch *or* tag20:29
mnaseri mean we've always historically had master only but, yeah20:29
mnasermainly to avoid the copy pasta of all of that docker_images section20:29
clarkbmnaser: I think the problem with it is even the jobs that run against master will now be tagged "master"20:30
corvuswhat about something like {{ defined(zuul.tag) | ternary(zuul.tag, 'latest') }}  ?20:30
clarkb(I think you may be right that the tag is the checkout on release jobs)20:30
mnaseryeah from an openstack world i was like "we get tip of branches for free!"20:31
mnaserand yeah that's what i was thinking, but i was wondering "{{ zuul.tag | default('latest') }}" works20:31
corvusyeah that's shorter :)20:31
corvuswe could even make that the default in the base docker jobs i think20:31
mnaserbut that would imply zuul always has tag20:31
* mnaser checks20:32
corvuszuul.tag should only exist for tag items:
mnaseryeah zuul.tag isn't defined at all so likely that would fail for "missing in dict" -- i think20:32
mnaserso my fanciness might not work20:32
corvuswell, there's definitely a way to do that :)20:33
openstackgerritMohammed Naser proposed zuul/zuul master: Tag both 'latest' and version
mnaserbased on my small testing this might work20:35
corvushrm, i think we're still missing something, since the upload job is meant to work with promote -- it's going to try to put a change number in the tag and fail...20:38
corvusi'm not sure we have a plain docker-push role...20:39
mnasercorvus: im wondering if we can just push :latest + :<short-commit-id> in gate and and reuse the promote image bits to just retag :<short-commit-id> to :<version-id>20:44
corvusmnaser: that would be great, but that's the hard bit -- mapping tag to short commit20:50
mnaser{{ zuul.tag }} + {{ zuul.newrev }} + some introspection ?20:50
mnaser"If the item was enqueued as the result of a tag being created, the new git sha of the tag will be included here." unless i'm missing something20:51
clarkbif you want the version to show up correctly in the dashboard you need to rebuild the image on tag rather than promoting20:51
corvusmnaser: we know the sha of the tag -- that may be a commit that zuul has never seen (this is why fungi was saying that zuul push would help with this)20:51
corvusmnaser: because merge commits are generated by gerrit20:52
clarkbThe simplest thing is likely to do a build and publish on tag20:52
*** jamesmcarthur has joined #zuul20:53
*** brendangalloway has quit IRC20:54
corvusShrews, ianw: zuul 3.11.1 is out -- we can have the nodepool release note point to that and then cut a nodepool release21:03
mnasercorvus: i think we might be ok regarding the issue you mentioned21:05
mnaserah wait, i see what you mean21:06
* mnaser goes to find the jobs that consume the images21:07
mnaserwell, when we run this in check/gate -- it tries to pull 'latest' -- which is still the right thing, and in 'release' when it will run with the tag, well, we dont have any jobs that consume that.. so it should be ok(tm)21:08
mnaserso my non-gating brain says: SUCCESS21:08
*** igordc has quit IRC21:19
*** igordc has joined #zuul21:29
*** jangutter has quit IRC21:40
Shrewscorvus: so i guess we just need updated with that release note modification, then we can merge it and release?21:47
ShrewsI expect we might want to run that in production for a day or so first.21:48
corvusShrews: that's my understanding -- and yeah, maybe we want to exercise that in opendev?21:49
ShrewsI think that would be best21:50
*** mattw4 has quit IRC21:55
*** mattw4 has joined #zuul21:56
mordredfbo: since the pagure driver has landed - I wonder if it would make sense to add a pagure connection to opendev zuul so that we could add a functional test between zuul and pagure - and so that you could use depends-on footers on patches that are still waiting on upstream patches to land?22:01
mordredfbo: was looking at - and there was a moment where it was marked WIP waiting on the upstream patches to land, which is what made me think of it22:02
mordredalthough at this point the list of outstanding changes needed in pagure seems to be smaller - so maybe it's not worth it anymore22:03
corvuswould still be cool :)22:03
openstackgerritIan Wienand proposed zuul/nodepool master: Set default python-path to "auto"
ianwcorvus: ^ thanks ... updated releasenotes -- i think the reflects the reality now22:18
corvusianw: lgtm -- tobiash ^22:23
mordredcorvus: that lgtm- do you want to wait for an ack from tobiash ?22:28
corvusmordred: yeah let's22:32
*** jamesmcarthur has quit IRC22:34
openstackgerritMerged zuul/zuul master: Build GitHub URLs for tags correctly
mordredcorvus: fwiw - I think the gerrit job is starting to make me grok what zuul might want to do with submodules - assuming there is a good way to get the submodule info from python in the mergers22:34
mordredit seems that if a repo has a submodule that's also on a connection zuul knows about - it should imply a required-projects entry - and if there is a branch listed, it should imply an override-checkout entry on the required-projects entry - and then finally when repos are being prepared on disk, putting the zuul prepared repo into the submodule target location22:37
mordredthen - doing the equiv of git submodule update --init for any submodules zuul *doesn't* know about the source repos of when preparing the repo22:38
openstackgerritLuigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try)
*** sgw has quit IRC22:51
openstackgerritMerged zuul/zuul master: Add the process environment to zuul.conf parser
SpamapSmordred: +1 for how submodules would play nice with zuul22:56
corvusmordred: the zuul-internal part of that sounds good -- i think i need to think a bit more about the submodule --init part (and look into what existing submodule roles/jobs do)23:17
*** sgw has joined #zuul23:18
mordredcorvus: ++23:19
mordredcorvus: doing submodule --init on unknown submodules could also be skipped - and maybe we say if you want to do zuul + submodules you need all the submodules to be in zuul's config (the main thing is that the repo doesn't have a valid remote, so you can't do submodule init in job itself if the submodule has a relative path)23:20
mordredbut yeah - bears more thinking on the specifics23:20
*** rfolco has joined #zuul23:25
*** rfolco|ruck has quit IRC23:25
*** mattw4 has quit IRC23:26
*** rfolco has quit IRC23:34
*** rfolco has joined #zuul23:34
corvusyeah, it's those edge cases i'm not trusting my brain to come up with right now.  :)23:40
mordredcorvus: ++23:44
*** rlandy has quit IRC23:48
*** rfolco has quit IRC23:48
SpamapSLots of little details to work out, but I think the mode of comparing the submodule remotes to the remotes in connection configs, and using those as required-projects, is a good start.23:49
SpamapSPerhaps with an additional step of having zuul's git prep code take care of putting those required projects dirs in the right places.23:49

Generated by 2.15.3 by Marius Gedminas - find it at!