| *** rcarrillocruz has quit IRC | 00:23 | |
| *** rcarrillocruz has joined #zuul | 00:38 | |
| *** xinliang has quit IRC | 00:43 | |
| *** xinliang has joined #zuul | 00:58 | |
| *** threestrands_ has joined #zuul | 03:12 | |
| *** threestrands_ has quit IRC | 03:12 | |
| *** threestrands_ has joined #zuul | 03:12 | |
| *** threestrands has quit IRC | 03:16 | |
| *** dkranz has quit IRC | 03:23 | |
| tobiash | pabelanger: do you mean trusted or untrusted? In trusted playbooks local shell scripts aren't constrained, | 04:23 |
|---|---|---|
| tobiash | And running git in an untrusted shell has probably a big attack vector | 04:25 |
| *** bhavik has joined #zuul | 04:30 | |
| *** bhavik has quit IRC | 04:37 | |
| *** sshnaidm|ptg has joined #zuul | 06:27 | |
| *** sshnaidm|ptg has quit IRC | 06:31 | |
| *** jaianshu has joined #zuul | 06:36 | |
| *** threestrands_ has quit IRC | 06:57 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Normalize hostname for statsd https://review.openstack.org/547309 | 07:39 |
| tobiash | jhesketh: ^ | 07:40 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Normalize hostname for statsd https://review.openstack.org/547309 | 07:52 |
| *** snapiri has quit IRC | 08:17 | |
| *** snapiri has joined #zuul | 08:22 | |
| *** jaianshu has left #zuul | 08:45 | |
| *** chrnils has joined #zuul | 08:46 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Set wheels-mirror for openSUSE Tumbleweed https://review.openstack.org/549181 | 09:18 |
| *** zigo has quit IRC | 09:18 | |
| *** jhesketh has quit IRC | 09:20 | |
| *** hashar has joined #zuul | 09:21 | |
| *** jhesketh has joined #zuul | 09:22 | |
| *** zigo has joined #zuul | 09:23 | |
| *** hashar has quit IRC | 09:32 | |
| *** electrofelix has joined #zuul | 09:34 | |
| *** hashar has joined #zuul | 09:41 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Normalize hostname for statsd https://review.openstack.org/547309 | 09:42 |
| electrofelix | I recall there was a dedicated site setup for zuul recently, but can't seem find it? Is there also a separate mailing list? | 10:00 |
| AJaeger | electrofelix: zuul-ci.org | 10:04 |
| openstackgerrit | Antoine Musso proposed openstack-infra/zuul master: Ensure that Change number and patchets are string https://review.openstack.org/545768 | 10:12 |
| electrofelix | AJaeger: thanks | 10:16 |
| electrofelix | Maybe the topic could be updated to include it? | 10:18 |
| *** yolanda has joined #zuul | 10:19 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/zuul master: Import Zuul modules at top of files https://review.openstack.org/549730 | 10:28 |
| kklimonda | is there an official way of passing variables between pre/run/post playbooks? | 12:04 |
| kklimonda | for some reason I recall zuul_return not exposing variables to other playbooks, but I could be wrong? | 12:06 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: configloader: add variant-description https://review.openstack.org/549748 | 12:22 |
| pabelanger | tobiash: yah, untrusted. Doesn't need to be shell, could be command (is the same I think). But untrusted git commands on executor fail obviously today | 12:25 |
| tobiash | pabelanger: so do you need untrusted git commands on the executor? | 12:26 |
| pabelanger | tobiash: right now yes, only because I am working around issues with base jobs in project-config. eg: http://git.openstack.org/cgit/openstack/windmill/tree/tests/base-minimal.yaml | 12:27 |
| pabelanger | tobiash: as you see, all the roles only run on a single node (bastion) however, my nodeset is multiple nodes. Today, we cannot do that with our base jobs, I have ideas how to fix them, but need to discuss: http://git.openstack.org/cgit/openstack/windmill/tree/tests/base-minimal.yaml | 12:28 |
| pabelanger | err | 12:28 |
| pabelanger | https://review.openstack.org/546474/ | 12:28 |
| tobiash | if the issue is a missing origin, https://review.openstack.org/#/c/544964/ adds one (with the state of the speculative parent) | 12:30 |
| tobiash | I fear allowing git operations for the command module will open an attack vector we cannot really estimate due to the huge featureset of git | 12:31 |
| pabelanger | tobiash: no, it is about only git cloning repos from executor to a specific host, not all hosts | 12:31 |
| pabelanger | tobiash: but origin is also a bug, I think in ansible git module | 12:32 |
| tobiash | ah, so you just want to filter the hosts where you want to push the repos to | 12:32 |
| pabelanger | tobiash: yes, this is because how my nodeset is setup: http://git.openstack.org/cgit/openstack/windmill/tree/.zuul.d/jobs.yaml#n50 I'm using ansible host aliases. Basically 1 node from nodepool, but ansible sees 7 | 12:33 |
| tobiash | pabelanger: is the git push playbook just doing the git stuff? | 12:34 |
| tobiash | if yes, it could run against {{ filtered_hosts | default('all') }} | 12:35 |
| tobiash | and if a job defines the filtered_hosts variable it might be possible to limit pushing to a specific host | 12:35 |
| pabelanger | tobiash: yes, that is right. But filtered_hosts needs to be an extra-vars, not host or group | 12:36 |
| pabelanger | tobiash: which we don't have a way to setup in zuul.yaml today | 12:36 |
| tobiash | ok, and the only extra vars we currently have are sitevars | 12:36 |
| pabelanger | tobiash: we could use a secret, but then it gets more complicated as they are dicts | 12:36 |
| tobiash | maybe also secrets... | 12:37 |
| pabelanger | yah, but we'd need a more complicated playbooks for secrets, as secret.filtered_hosts | default('all') doesn't work out of box. We'd need to include more if logic into the jinja2 template | 12:38 |
| pabelanger | doable, just more tricky | 12:38 |
| tobiash | pabelanger: can we already define host or group vars in the nodeset? | 12:40 |
| pabelanger | tobiash: yah, I think that landed 2 weeks ago? | 12:40 |
| tobiash | maybe the git push should skip per host if required | 12:40 |
| tobiash | so make it push with when: no_source is not defined | 12:41 |
| tobiash | then you could mark some hosts in the nodeset to not receive git repos if desired | 12:42 |
| pabelanger | tobiash: yah, we could do that too. But opted not to do so, because it isn't just the git push. It really is above running the roles as they are today, only on 1 or specific filtered hosts | 12:42 |
| tobiash | pabelanger: well you should not be able to opt out of the ssh key stuff | 12:43 |
| tobiash | so that leaves the streaming daemon, git push | 12:43 |
| tobiash | am I missing further stuff? | 12:44 |
| pabelanger | tobiash: remember, I just have a single ansible_host (shared IP) between all the ansible hosts. So, even if I say 1 nodes in inventory, all that are using the shared_ip, get the tasks run | 12:44 |
| pabelanger | so, not opting out, just not running the same tasks multiple times on the same host | 12:45 |
| pabelanger | SSH keys, zuul_stream do work today | 12:45 |
| tobiash | so you're trying to solve races with serial | 12:45 |
| tobiash | I see | 12:45 |
| pabelanger | yah, serial would also work, but again, need extra-vars support | 12:46 |
| pabelanger | because, cannot be setup via host / group vars | 12:46 |
| pabelanger | Idempotence is really the issue | 12:47 |
| pabelanger | some of our tasks are, others are not | 12:47 |
| pabelanger | and racy | 12:47 |
| tobiash | yah, the extra-vars will be useful | 12:48 |
| tobiash | also for doing CD with deployment scripts which use extra-vars | 12:48 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add known hosts for all users https://review.openstack.org/548642 | 12:58 |
| *** dkranz has joined #zuul | 13:10 | |
| andreaf | tobiash pabelanger is there a tool in zuul that give a job name will build a list of plays that will be invoked for that job? | 13:20 |
| tobiash | andreaf: no | 13:21 |
| andreaf | tobiash pabelanger what I'm trying to do is to run a job locally and for the test to be meaningful I need to run all the pre plays of all parent jobs first | 13:21 |
| andreaf | tobiash ok too bad | 13:22 |
| tobiash | andreaf: such a tool is kind of on the future roadmap in the future | 13:22 |
| pabelanger | yah, for now, you'd have to parse debug logs and see what ansible-playbook commands are run. Which is a lot harder, if you are not a zuul admin | 13:22 |
| *** rlandy has joined #zuul | 13:23 | |
| tobiash | andreaf: no one has taken this task yet so if you want to work on that tool nobody will complain ;) | 13:24 |
| andreaf | tobiash heh not likely to happen but I will consider it | 13:26 |
| *** myoung|ruck|off is now known as myoung|ruck | 13:27 | |
| *** rlandy_ has joined #zuul | 15:00 | |
| *** rlandy has quit IRC | 15:01 | |
| *** rlandy__ has joined #zuul | 15:02 | |
| *** rlandy_ has quit IRC | 15:05 | |
| *** rlandy__ is now known as rlandy | 15:23 | |
| kklimonda | Is there an example of passing some variables between playbooks? Is it even possible with current implementation? If not, I'd like to extend behavior of `zuul_return` so that variables returned are then made available to other playbooks | 15:32 |
| clarkb | eaxh playbook is its own ansible-playbook process | 15:32 |
| clarkb | so I dont think that is currently possible | 15:32 |
| kklimonda | mhm, and I probably can't easily load variables from executor in an untrusted playbook? | 15:33 |
| kklimonda | hmm, unless include_vars accept any paths, and doesn't have to be relative to the role | 15:34 |
| *** openstackgerrit has quit IRC | 15:34 | |
| kklimonda | if so, I could probably write yaml file with variables as part of a trusted playbook, and load it back from untrusted | 15:34 |
| tobiash | kklimonda: I think this is not implemented yet but such a use case has been discussed in the past | 15:56 |
| tobiash | so feel free to propose an implementation | 15:56 |
| tobiash | during implementation keep in mind that you have to make sure that this doesn't overwrite secrets or sitevars | 15:57 |
| *** openstackgerrit has joined #zuul | 16:02 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Allow updates from untrusted mirrors https://review.openstack.org/549162 | 16:02 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: bindep: use shell instead of command with executable https://review.openstack.org/546869 | 16:02 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: ara: check for return code instead of succeed https://review.openstack.org/546892 | 16:02 |
| kklimonda | tobiash: the idea I had was to reuse zuul_return for that, making returned variables available for other playbooks - assuming that ansible is smart about that, as long as I include variables from results.json (or really results.yaml) before secrets and sitevars, it should work. I think that's how site-variables are implemented now, having precedence over any other source. | 16:02 |
| tobiash | kklimonda: I'm not sure if we maybe want to put these vars under a root var | 16:03 |
| tobiash | But that probably doesn't change the approach | 16:03 |
| pabelanger | yah, we've discussed zuul_return for that before | 16:04 |
| pabelanger | not sure where that ended up on the discussion list at PTG | 16:04 |
| tobiash | Unfortunately not | 16:04 |
| tobiash | But zuul_return is the right thing for that | 16:05 |
| kklimonda | tobiash: is there any particular reason to "namespace" those variables, or are you just playing it safe? | 16:06 |
| tobiash | This idea is just for discussion | 16:06 |
| tobiash | In that regard i'd like to hear corvus thought but he is afk this week | 16:07 |
| tobiash | The idea is just to be safe | 16:08 |
| tobiash | But anyway, namespacing doesn't really change the overall approach | 16:09 |
| *** electrofelix has quit IRC | 16:10 | |
| openstackgerrit | Krzysztof Klimonda proposed openstack-infra/zuul master: [WIP] Pass variables returned by zuul_return to other playbooks https://review.openstack.org/549807 | 16:21 |
| SpamapS | tobiash: do you use the review API from github to send things into your gate pipelines? | 16:30 |
| SpamapS | tobiash: I'm having trouble with it.. the permissions don't seem to be right, so I can't actually get it to send things in because reviews don't come back with the 'write' permission, even though people do in fact have write permissions on the repo. | 16:30 |
| SpamapS | kklimonda: as a workaround, you can always use `slurp` and `from_json` to grab the bits that zuul_return leaves behind in the work directory. | 16:32 |
| kklimonda | can I slurp on executor? | 16:33 |
| kklimonda | (from an untrusted code) | 16:34 |
| pabelanger | kklimonda: I'm not sure. I'd guess not, but lets us know :) | 16:34 |
| tobiash | SpamapS: you mean to triggee the gate? | 16:34 |
| tobiash | The review analysis is still buggy (e.g. when formatting the change message) | 16:36 |
| SpamapS | kklimonda: you should be able to slurp from the work dir, which is where zuul_return drops its json. | 16:37 |
| SpamapS | Or did we get slick and make zuul_return special enough to write outside work dir? | 16:37 |
| pabelanger | can you slurp on localhost? I've never tired myself | 16:38 |
| SpamapS | yep | 16:38 |
| pabelanger | cool | 16:38 |
| SpamapS | Though I'd guess include_vars is the better choice there *if* it works. | 16:38 |
| tobiash | SpamapS: that is a possibility to share data between playbooks in the same job | 16:39 |
| tobiash | For data sharing between dependent jobs the zuul_return method would be needed | 16:39 |
| SpamapS | Yeah I thought kklimonda was talking about playbooks in the same job. | 16:40 |
| SpamapS | zuul_return already works fine for passing variables to child jobs. | 16:40 |
| tobiash | That got already implemented? | 16:40 |
| tobiash | Cool, didn't notice that | 16:41 |
| kklimonda | yes, passing variables to child jobs work fine with zuul_return | 16:41 |
| tobiash | kklimonda: then forget all I said and trust SpamapS ;) | 16:41 |
| SpamapS | tobiash: regarding reviews and pipelines, what I mean is, I have this pipeline and it doesn't ever trigger: | 16:42 |
| tobiash | Yeah, I ty | 16:42 |
| kklimonda | SpamapS: I indeed want to pass variables between playbooks in the same job - `include_var` sounds like a nice workaround until that can be done by zuul | 16:42 |
| kklimonda | I'll see if it actually works in a minute ;) | 16:42 |
| SpamapS | http://paste.openstack.org/show/691974/ | 16:42 |
| tobiash | Yeah, that could be broken | 16:42 |
| tobiash | We trigger only on merge label due to this | 16:43 |
| SpamapS | and the reason is the require section | 16:43 |
| kklimonda | (I wouldn't be surprised if both slurp and include_vars refuse to run in untrusted "context") | 16:43 |
| SpamapS | nobody seems to have the 'write' permission. | 16:43 |
| SpamapS | kklimonda: they should just be scoped down to safe paths | 16:43 |
| tobiash | SpamapS: what does the api say? | 16:44 |
| SpamapS | tobiash: Oh I keep forgetting you wrote us a nice tool for this. | 16:44 |
| * SpamapS fires it up | 16:45 | |
| tobiash | SpamapS: remove the write from the pipeline | 16:45 |
| tobiash | Afaik zuul only checks reviews from people with write acced | 16:46 |
| tobiash | But have to double check that | 16:46 |
| tobiash | SpamapS: like https://docs.openstack.org/infra/zuul/admin/drivers/github.html#requirements-configuration | 16:47 |
| tobiash | Zuul should only take reviews from write access people into account | 16:48 |
| tobiash | SpamapS: yea, used that tool too | 16:50 |
| tobiash | Github was a minute behind which broke zuul auth... | 16:51 |
| tobiash | SpamapS: so if you use app auth you should definitely make sure that both zuul and github have ntp configured... | 16:52 |
| SpamapS | No I am still on webhooks | 16:54 |
| SpamapS | haven't had time to migrate to apps | 16:54 |
| SpamapS | pipeline.require.<github source>.review.permission | 16:55 |
| SpamapS | If present, the author of the code review must have this permission (or permissions). The available values are read, write, and admin. | 16:55 |
| SpamapS | tobiash: ? | 16:55 |
| SpamapS | that looks like what I'm aiming for. | 16:55 |
| tobiash | Hrm, then that may not work quite yet | 16:56 |
| tobiash | Does it trigger on any approval if you remone the write? | 16:57 |
| tobiash | remove | 16:57 |
| tobiash | Gah, my mobile keyboard is too small | 16:58 |
| kklimonda | SpamapS: thanks, looks like include_vars will work as a workaround - slurp however has no zuul-specific implementation and so won't work | 17:06 |
| SpamapS | tobiash: I haven't tried that yet. | 17:22 |
| SpamapS | kklimonda: indeed, was just poking at that. | 17:22 |
| tobiash | SpamapS: at lease what I read from the code requesting reviews with write permission won't match reviews with admin permission | 17:32 |
| tobiash | but it looks like currently you cannot require with permission write OR admin | 17:33 |
| tobiash | SpamapS: I think it would also be better to implement review checks in canMerge in the github connection | 17:39 |
| tobiash | that way you could trigger on any review, remove reviews from the required section and rely on the branch protection settings | 17:39 |
| tobiash | this could be implemented on top of https://review.openstack.org/#/c/535680/ | 17:40 |
| tobiash | which does that for build statuses (which I use in production for quite some time now) | 17:40 |
| tobiash | it's still wip due to lacking tests | 17:40 |
| SpamapS | tobiash: I like that. | 17:40 |
| *** rlandy is now known as rlandy|biab | 17:42 | |
| pabelanger | figure k8s people might be interested: http://logs.openstack.org/68/549768/11/check/kubespray-base/dc9118b/job-output.txt.gz | 17:49 |
| pabelanger | while not a native zuulv3 job, I did manage to install kubespray with it being in zuul tenant configuration | 17:50 |
| pabelanger | https://review.openstack.org/549768/ now to start cleaning it up | 17:50 |
| AJaeger | great, pabelanger ! | 17:51 |
| pabelanger | yah, the cool thing, is their playbooks / roles worked as expected | 17:52 |
| Shrews | oh hey, where is the next PTG? | 17:55 |
| Shrews | i assume that was announced last week | 17:55 |
| pabelanger | I didn't hear anything myself | 17:55 |
| tobiash | Shrews: not yet | 17:55 |
| pabelanger | I also hear talks at board meeting to cancel PTG | 17:56 |
| tobiash | I just heard that it will probably be in north america | 17:56 |
| tobiash | pabelanger: ? | 17:56 |
| pabelanger | yah, wasn't there discussions on monday board meeting about if PTG is needed any more? That's atleast what I heard from people | 17:56 |
| tobiash | oh | 17:58 |
| *** myoung|ruck is now known as myoung|ruck|food | 17:59 | |
| rcarrillocruz | eugh | 18:05 |
| rcarrillocruz | wasn't aware of that | 18:05 |
| *** hashar is now known as hasharAway | 18:10 | |
| pabelanger | don't quote me. But that is what I heard from some people | 18:11 |
| *** myoung|ruck|food is now known as myoung|ruck | 18:26 | |
| *** rlandy|biab is now known as rlandy | 18:42 | |
| *** harlowja has joined #zuul | 18:49 | |
| SpamapS | tobiash: indeed, require write was the problem, using require admin is my current workaround, but I think I have a simple solution to try in code | 18:50 |
| tobiash | SpamapS: ok | 18:50 |
| SpamapS | cancel PTG and just move it back to summit? | 18:50 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul master: Match github model of granting admins write https://review.openstack.org/549853 | 19:24 |
| SpamapS | tobiash: ^ | 19:24 |
| tobiash | lgtm | 19:26 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Save the ARA sqlite database in a specific folder https://review.openstack.org/513874 | 19:32 |
| *** chrnils has quit IRC | 19:47 | |
| *** hasharAway is now known as hashar | 19:57 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Save the ARA sqlite database in a specific folder https://review.openstack.org/513874 | 20:18 |
| *** dkranz has quit IRC | 20:30 | |
| pabelanger | is depends-on for github working today? | 20:33 |
| pabelanger | I haven't tried, but wasn't sure | 20:33 |
| clarkb | I think so iirc monty tested it with shade/ansible | 20:34 |
| pabelanger | k, that's what I figured | 20:35 |
| *** dkranz has joined #zuul | 20:35 | |
| pabelanger | trying it with kubernetes-incubator/kubespray now | 20:35 |
| rbergeron | spray alllllll the kubes | 20:35 |
| pabelanger | yah, it does work | 20:36 |
| rbergeron | with that sweet zuul pixiedust | 20:36 |
| pabelanger | which is cool | 20:36 |
| pabelanger | and worked as expected, nice | 20:38 |
| tobiash | pabelanger: it works even cross source with gerrit :) | 20:47 |
| *** kmalloc has joined #zuul | 20:49 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: DNM - test base-test https://review.openstack.org/549867 | 20:55 |
| *** dkranz has quit IRC | 21:20 | |
| *** myoung|ruck is now known as myoung|ruck|bbl | 21:41 | |
| *** threestrands_ has joined #zuul | 21:45 | |
| *** threestrands_ has quit IRC | 21:45 | |
| *** threestrands_ has joined #zuul | 21:45 | |
| *** hashar has quit IRC | 22:03 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add known hosts for all users https://review.openstack.org/548642 | 22:08 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add known hosts for all users https://review.openstack.org/548642 | 22:30 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Add known hosts for all users https://review.openstack.org/548642 | 23:00 |
| *** jappleii__ has joined #zuul | 23:09 | |
| *** jappleii__ has quit IRC | 23:12 | |
| *** threestrands_ has quit IRC | 23:12 | |
| *** threestrands has joined #zuul | 23:12 | |
| *** threestrands has quit IRC | 23:12 | |
| *** threestrands has joined #zuul | 23:12 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!