| *** sdake has quit IRC | 00:01 | |
| clarkb | tobiash: comments on https://review.openstack.org/#/c/616306/13 maybe corvus can check my note about tenant scoping projects to see if that is worth a 01 | 00:16 |
|---|---|---|
| clarkb | er -1 | 00:16 |
| corvus | clarkb: well, it's perfectly legit (expected, even encouraged) to share projects across tenants | 00:20 |
| corvus | i have to run now, i'll try to think more about that tomorrow | 00:21 |
| clarkb | corvus: right so collisions are likely to be a thing | 00:22 |
| corvus | yes, but depending on both the details of implementation, and the questions we're trying to answer, "collisions" might be okay | 00:22 |
| corvus | (like, even though a project is in 2 tenants, it may only be in pipelines in one, and therefore there's no confusion. or, if it is in pipelines in multiple tenants, maybe the aggregate is what you want -- project foo is responsible for X cpu hours, even though X/2 comes from this tenant, and X/2 comes from that... | 00:24 |
| corvus | that's where my thinking will start tomorrow when i review that :) | 00:24 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: config: add playbooks to job.toDict() https://review.openstack.org/621343 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Add API endpoint to get frozen jobs https://review.openstack.org/607077 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: implement prep-workspace https://review.openstack.org/607082 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: use connections endpoint for prepare-workspace https://review.openstack.org/631704 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add execute sub-command https://review.openstack.org/630944 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 00:31 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add quick-start integration test https://review.openstack.org/635701 | 00:31 |
| *** sdake has joined #zuul | 00:45 | |
| openstackgerrit | Paul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have an .updated attribute. https://review.openstack.org/633259 | 00:46 |
| mnaser | anyone seen 404 github exceptions when using a github app when a webhook is recieved ? | 01:03 |
| mnaser | hmm this might be a misconfig | 01:04 |
| *** sdake has quit IRC | 01:06 | |
| pabelanger | in zuul logs? | 01:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add quick-start integration test https://review.openstack.org/635701 | 01:22 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 01:22 |
| *** rlandy has quit IRC | 01:30 | |
| mnaser | pabelanger: yeah it was a misconfig | 02:03 |
| mnaser | i'm wondering if anyone knows what caps are needed for bwrap to allow creating new namespace | 02:04 |
| *** sdake has joined #zuul | 02:07 | |
| *** sdake has quit IRC | 02:13 | |
| clarkb | mnaser bwrap documents it. It needs userns or to run as root iirc | 02:17 |
| mnaser | clarkb: looks like i can only run it in a privileged docker container, not ideal but liveable | 02:17 |
| clarkb | that is hoe tobiash does it but I dont think it is required if your kernel supports userns and ylu set up caps right | 02:18 |
| clarkb | are you on centos? | 02:19 |
| clarkb | their kernel doesnt do user ns by default | 02:19 |
| mnaser | clarkb: this is docker on mac so it boots a vm running $something | 02:19 |
| mnaser | so i have this tenant config right now -- http://paste.openstack.org/show/744716/ -- however it looks like it's complaining that it can't find `add-build-sshkey` and `fetch-output` which are both in zuul-jobs. they are referenced in zuul.yaml as `zuul: git.zuul-ci.org/zuul-jobs` and im wondering if the mismatch is the cause of this? | 02:25 |
| mnaser | https://github.com/openstack-infra/zuul-jobs/blob/8ed7cf4c525c0356098c8e25fb2958eff00143b2/doc/source/install.rst | 02:26 |
| mnaser | aaand i should learn to read alright | 02:26 |
| *** saneax has joined #zuul | 02:28 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: implement prep-workspace https://review.openstack.org/607082 | 02:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /connections route https://review.openstack.org/631703 | 02:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: use connections endpoint for prepare-workspace https://review.openstack.org/631704 | 02:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add execute sub-command https://review.openstack.org/630944 | 02:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 02:52 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add quick-start integration test https://review.openstack.org/635701 | 02:54 |
| *** bhavikdbavishi has joined #zuul | 03:45 | |
| *** ianychoi has quit IRC | 03:49 | |
| *** daniel2 has quit IRC | 03:49 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 03:53 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: zuul-runner: add quick-start integration test https://review.openstack.org/635701 | 03:53 |
| *** ianychoi has joined #zuul | 03:55 | |
| *** bjackman has joined #zuul | 05:06 | |
| *** chandankumar is now known as chkumar|ruck | 05:41 | |
| *** bjackman has quit IRC | 06:02 | |
| *** swest has joined #zuul | 06:04 | |
| *** bjackman has joined #zuul | 06:06 | |
| *** bjackman has quit IRC | 06:12 | |
| *** bjackman has joined #zuul | 06:13 | |
| badboy | is there a way to change the color of the failed build in the Builds tab of Zuul's dashboard to red? | 06:24 |
| tobiash | badboy: don't you like the yellow? | 06:26 |
| tobiash | badboy: you'd need to patch the css if you want to change that | 06:27 |
| badboy | tobiash: not really ;) | 06:27 |
| tobiash | zuul-web is not skinnable ;) | 06:27 |
| badboy | tobiash: red is more suitable for failure as green is for success | 06:27 |
| tobiash | badboy: but permanently chan ging this to red can be problematic for all the people that are red-green colorblind | 06:28 |
| badboy | tobiash: probably you're right | 06:28 |
| *** bjackman has quit IRC | 06:28 | |
| badboy | tobiash: never thought about it since I've never met anyone who's colorblind | 06:29 |
| tobiash | badboy: most people won't tell you unless you ask for it ;) | 06:29 |
| badboy | tobiash: true | 06:30 |
| badboy | tobiash: can you help me with apache config for Zuul? | 06:30 |
| badboy | tobiash: I would like to have multiple tenants on one machine | 06:30 |
| tobiash | badboy: do you already have an apache config? | 06:31 |
| badboy | yes, I do | 06:32 |
| badboy | tobiash: scratch that, I don't | 06:32 |
| badboy | tobiash: I was thinking about apache conf for logs | 06:33 |
| tobiash | badboy: how do you want to have the urls? | 06:33 |
| *** bjackman has joined #zuul | 06:34 | |
| badboy | tobiash: zuul.example.com/<tanant_1> | 06:34 |
| *** pleia2 has quit IRC | 06:35 | |
| tobiash | badboy: then it's simple, just forward all zuul.example.com to zuul-web | 06:35 |
| *** pleia2 has joined #zuul | 06:35 | |
| tobiash | having multi tenant is basically the default in zuul-web | 06:35 |
| badboy | tobiash: and rewriting everything from port 9000 to 80? | 06:36 |
| tobiash | yes, I guess the most simple way is to just use mod_proxy | 06:37 |
| badboy | tobiash: like this? https://github.com/CiscoSystems/third-party-ci-installer/blob/master/roles/apache/tasks/main.yaml#L89 | 06:37 |
| tobiash | badboy: http://paste.openstack.org/show/744726/ | 06:38 |
| tobiash | that's how mine looks like | 06:39 |
| tobiash | and you'll need to enable the modules mod_proxy, mod_proxy_html and mod_proxy_wstunnel (for log streaming) | 06:41 |
| badboy | tobiash: thanks, that should be in the docs with a sample config or maybe an ansible playbook | 06:42 |
| tobiash | badboy: it is: https://zuul-ci.org/docs/zuul/admin/installation.html?highlight=apache#reverse-proxy | 06:42 |
| tobiash | (with the rewrite method, but I like proxypass more ;) | 06:43 |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Retrieve full list of jobs with details per tenant via API https://review.openstack.org/635714 | 06:49 |
| *** chkumar|ruck has quit IRC | 06:51 | |
| *** quiquell|off is now known as quiquell|rover | 06:52 | |
| *** chandankumar has joined #zuul | 06:52 | |
| *** chandankumar is now known as chkumar|ruck | 06:52 | |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Retrieve full list of jobs with details per tenant via API https://review.openstack.org/635714 | 06:53 |
| *** goern has joined #zuul | 07:04 | |
| *** daniel2 has joined #zuul | 07:14 | |
| *** remi_ness has joined #zuul | 07:15 | |
| *** saneax has quit IRC | 07:20 | |
| *** pcaruana has joined #zuul | 07:23 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add initial driver and event https://review.openstack.org/555153 | 07:32 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add web route and rpclistener https://review.openstack.org/554839 | 07:32 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build modal button to trigger job https://review.openstack.org/635716 | 07:32 |
| *** quiquell|rover is now known as quique|rover|brb | 07:35 | |
| *** gtema has joined #zuul | 08:02 | |
| openstackgerrit | Felix Schmidt proposed openstack-infra/zuul master: Retrieve full list of jobs with details per tenant via API https://review.openstack.org/635714 | 08:04 |
| *** quique|rover|brb is now known as quiquell|rover | 08:05 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add initial driver and event https://review.openstack.org/555153 | 08:24 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add web route and rpclistener https://review.openstack.org/554839 | 08:24 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build modal button to trigger job https://review.openstack.org/635716 | 08:24 |
| tristanC | corvus: could you please revisit your -2 on https://review.openstack.org/555153 | 08:28 |
| *** saneax has joined #zuul | 08:36 | |
| *** jpena|off is now known as jpena | 08:45 | |
| zbr | can zuul trigger jobs based on commit message pattern? like if "risk" mentioned in it to trigger jobs that are usually not run? | 08:48 |
| zbr | i know that in theory we could use a placeholder file for this, but it would be more convenient to be able to use magic words. | 08:49 |
| *** ianychoi has quit IRC | 08:50 | |
| *** panda|off is now known as panda | 09:08 | |
| tobias-urdin | pabelanger: do you have some time over to check https://review.openstack.org/#/c/632699/ | 09:25 |
| tobias-urdin | fungi: do you have any feedback for ^ i'm a little stuck on that right now and it is blocking m2 release that still needs requeue and the newly requested https://review.openstack.org/#/c/635541/ | 09:25 |
| *** manjeets_ has joined #zuul | 09:55 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 09:56 |
| *** manjeets has quit IRC | 09:57 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: URLTrigger driver time based - artifact change jobs triggering driver https://review.openstack.org/635567 | 09:59 |
| *** remi_ness has quit IRC | 10:07 | |
| *** bhavikdbavishi has quit IRC | 10:18 | |
| *** sdake has joined #zuul | 11:34 | |
| *** gtema has quit IRC | 11:35 | |
| *** gtema has joined #zuul | 11:38 | |
| *** sdake has quit IRC | 11:41 | |
| *** sdake has joined #zuul | 11:45 | |
| *** bjackman has quit IRC | 11:48 | |
| mhu | Hello, when I try to run unit tests for zuul-web on my environment (ubuntu 18.04 + tools/test-setup.sh) on master, I get these errors: https://pastebin.com/6mTzjeMe Any idea what I can do? | 11:50 |
| *** bjackman has joined #zuul | 11:50 | |
| *** bjackman has quit IRC | 12:09 | |
| *** sdake has quit IRC | 12:11 | |
| *** sdake has joined #zuul | 12:13 | |
| *** bjackman has joined #zuul | 12:19 | |
| *** jpena is now known as jpena|lunch | 12:37 | |
| *** sdake has quit IRC | 12:48 | |
| *** Diabelko has quit IRC | 12:53 | |
| *** Diabelko has joined #zuul | 12:54 | |
| *** bjackman has quit IRC | 12:59 | |
| mordred | zbr: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/pipelines.yaml#n254 | 13:27 |
| *** ianychoi has joined #zuul | 13:27 | |
| mordred | zbr: yes. that's an example of a pipeline liek that we have in openstack ^^ | 13:27 |
| mnaser | tobiash: do you use nodepool-builder inside docker as well? | 13:28 |
| mnaser | the image seems to be missing........ sudo | 13:29 |
| zbr | mordred: i was referring to commit-message content, not comment content. | 13:31 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 13:31 |
| zbr | they both start with comm :) | 13:31 |
| tobiash | mnaser: yes, but still using my own images | 13:33 |
| zbr | i guess the only way to do it is to always trigger the job and to introspect inside it and decide to exit quickly if the message has no matching pattern, but this sucks from resource maintenance point of view. especially if I want to trigger so special fat jobs in "risk" case. | 13:33 |
| mnaser | tobiash: is it possible to share the docker files and i can slowly try to get them to converge? | 13:34 |
| *** rlandy has joined #zuul | 13:37 | |
| mordred | zbr: ah - gotcha. yeah - you're better off doing that. you may want to check out: https://zuul-ci.org/docs/zuul/user/jobs.html#skipping-child-jobs | 13:38 |
| *** chkumar|ruck is now known as chkumar|out | 13:39 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/zuul master: Escape jinja2 template from commit message https://review.openstack.org/633930 | 13:39 |
| mordred | zbr: you could make a parent job that doesn't use any nodes from nodepool, have that job do the commit message examination, and then skip expensive child jobs | 13:39 |
| *** jpena|lunch is now known as jpena | 13:39 | |
| zbr | mordred: thanks, I will be looking on this. mainly what I want to do is to find an alternative to trigger additional jobs on changes that are risk. Currently we use an approach of creating a DNM change on another repository which has the jobs that may be affected. Instead of spamming gerrit with zuul DNM with depends-on, it would be easier to have all in the same change. sounds ok? | 13:41 |
| mordred | zbr: yeah, makes sense - you could totally do that with the child jobs filter | 13:44 |
| *** avass has joined #zuul | 14:03 | |
| avass | Can you override the checkout for a job to not check out anything? | 14:07 |
| *** EmilienM is now known as EvilienM | 14:15 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 14:16 |
| tobiash | mnaser: I can check later, currently not @laptop | 14:19 |
| *** TheJulia is now known as needssleep | 14:25 | |
| *** sdake has joined #zuul | 14:38 | |
| *** sdake has quit IRC | 14:42 | |
| *** sdake has joined #zuul | 14:52 | |
| *** pwhalen has joined #zuul | 14:54 | |
| Shrews | tobiash: the ansible spec is looking good! i just had one question about deprecation and module overrides (left a note) | 15:04 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 15:17 |
| *** quiquell|rover is now known as quiquell|off | 15:22 | |
| *** avass has quit IRC | 15:23 | |
| *** remi_ness has joined #zuul | 15:27 | |
| *** pcaruana has quit IRC | 15:31 | |
| Shrews | mnaser: i think i found the problem with https://review.openstack.org/635584 noted inline | 15:35 |
| pabelanger | tobias-urdin: fungi: just reading 632699 seems like corvus was suggesting not change revoke-sudo and do a local gem install into executor side, so we don't publish secrets to nodepool node | 15:48 |
| fungi | pabelanger: trick is, that means also using what those gems provide to run arbitrary code from the repository being released | 15:51 |
| *** sdake has quit IRC | 15:52 | |
| fungi | because it's not just a static upload tool from what i gather, it potentially runs turing-complete routines from the repository in the process of uploading | 15:52 |
| *** sdake has joined #zuul | 15:53 | |
| corvus | that's a bad upload tool then | 15:53 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 15:54 |
| corvus | (why would a tool do that?) | 15:54 |
| corvus | for a while, before twine, we had a simple script to upload things to pypi. | 15:55 |
| *** sdake has quit IRC | 15:56 | |
| fungi | tobias-urdin: can you confirm/deny my (mis)understanding there? | 15:56 |
| clarkb | corvus: ya I had written a small thing that just curl POSTed to pypi iirc | 15:57 |
| fungi | i need to pop out to grab lunch, but will remove my +2 while we discuss | 15:57 |
| clarkb | and when dstufft found out we were doing that was like oh ya we should have a better tool for that and wrote twine | 15:58 |
| *** sdake has joined #zuul | 15:58 | |
| pabelanger | unfortunately haven't followed along much on puppet publishing refactor myself | 16:03 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/nodepool master: docker: don't daemonize when starting images https://review.openstack.org/635584 | 16:03 |
| tobias-urdin | fungi: so right now module is cloned to a nodepool instance -> build tarball -> copy them to executor -> rename tarball -> upload | 16:09 |
| tobias-urdin | what is happening now would be: nodepool instance -> build tarball -> upload to puppetforge -> copy to executor -> rename tarball -> upload | 16:09 |
| tobias-urdin | it runs as a rake task inside the git repo for the module and requires the puppet build output i.e it must be named openstack-neutron-<version>.tar.gz but we would rename to puppet-neutron-<version>.tar.gz | 16:10 |
| mnaser | is there any docs around the 'opendevorg/python-builder' ? | 16:10 |
| mnaser | trying to see how i can add 'sudo' to the nodepool image | 16:11 |
| tobias-urdin | if we can have ruby and all required build gcc/g++ make to build native ruby gems we could move the "upload to puppetforge" step after "copy to executor" but before rename tarball | 16:11 |
| tobias-urdin | on the executor that is | 16:11 |
| mordred | mnaser: python-builder basically just provides for installing the things in bindep.txt and python requirements | 16:12 |
| mnaser | mordred: so if i drop sudo into bindep.txt and i should be in a good place | 16:12 |
| mordred | mnaser: to add sudo, I think just adding an apt-get update && apt-get install -y sudo && apt-get clean to the dockerfile | 16:12 |
| mordred | mnaser: yeah - or add sudo to the bindep.txt :) | 16:12 |
| tobias-urdin | fungi: oh and regarding "it potentially runs turing-complete routines from the repository in the process of uploading" that not really true | 16:14 |
| tobias-urdin | it's drops a Rakefile in the git repo root which is then executed, so there is no code by the module that could run in the trusted executor context | 16:14 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/nodepool master: bindep: add sudo https://review.openstack.org/635876 | 16:15 |
| mnaser | mordred: ^ lets see how that goes.. | 16:16 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul-jobs master: Add the skip_bindep option to the tox job https://review.openstack.org/635877 | 16:16 |
| pabelanger | mnaser: mordred: there will be a bunch more dependencies required for DIB, eg: http://git.openstack.org/cgit/openstack/diskimage-builder/tree/bindep.txt rather then adding them into nodepool, can we not have docker builds use bindep from diskimage-builder? | 16:17 |
| mnaser | that seems like a good idea too | 16:18 |
| mordred | heh. that gets complicated :) | 16:18 |
| * mordred is on the phone - but will say more words about that getting complicated in just a minute | 16:18 | |
| pabelanger | or, can we do a dedicated dib container and somehow link that to nodepool-builder? | 16:19 |
| pabelanger | (my docker knowledge is weak here) | 16:19 |
| clarkb | pabelanger: you could layer nodepool on top of a dib container potentially | 16:19 |
| mordred | yeah - but even that is going to get complicated quickly based on how the nodepool containers are currently being built | 16:20 |
| mordred | actually - no - it might not be that bad ... | 16:20 |
| * mordred has a thought | 16:20 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: WIP Base nodepool-builder on diskimage-builder https://review.openstack.org/635879 | 16:24 |
| mordred | clarkb, pabelanger, mnaser: ^^ there's a quick sketch - assumes we make a dib docker image in the same way we're building the nodepool and zuul images | 16:25 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 16:25 |
| *** saneax has quit IRC | 16:25 | |
| mordred | (which obviously wouldn't be hard- main question would wind up being whether we make it as openstack/diskimage-builder or opendevorg/diskimage-builder | 16:25 |
| corvus | mordred: dib is an infra project, so i assume it would be opendevorg | 16:28 |
| mordred | corvus: ++ | 16:28 |
| corvus | though, what would be swell is if it were packaged :) | 16:28 |
| mordred | we'll need to do a couple of things with the bindep file in dib | 16:29 |
| tobiash | Shrews: thanks, thinking about this maybe we should instead add a meta file to each module override with information about the target versions | 16:32 |
| tobiash | and then do the copy on the executor startup more intelligent | 16:33 |
| tobiash | (re ansible spec) | 16:33 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul-jobs master: Add the skip_bindep option to the tox job https://review.openstack.org/635877 | 16:40 |
| tobiash | https://docs.python.org/3/library/os.html#os.makedirs | 16:41 |
| tobiash | Changed in version 3.7: The mode argument no longer affects the file permission bits of newly-created intermediate-level directories. | 16:41 |
| tobiash | really? | 16:41 |
| clarkb | that is an interesting choice | 16:42 |
| mordred | what? | 16:44 |
| mordred | that's | 16:44 |
| *** bhavikdbavishi has joined #zuul | 16:44 | |
| mordred | I think that documentation is a bug. I think it should say "Changed in version 3.7: os.makedirs was made useless" | 16:44 |
| mordred | but I might just be being reactionary | 16:45 |
| mordred | https://bugs.python.org/issue19930 is the issue/patch that did that, btw | 16:46 |
| mordred | ah. the change is to make it work like mkdir -p | 16:46 |
| clarkb | tobiash: did you see my comments on https://review.openstack.org/#/c/616306/ curious to know what you think about it | 16:46 |
| tobiash | clarkb: yes, but had no time yet to read them, will read in a sec | 16:47 |
| corvus | clarkb, tobiash: heh, i just finished reviewing that. i responded to the accounting question. i don't have thoughts on the other one, so i left only a +1 until you sort that out. | 16:47 |
| SpamapS | As a umask person, I was always surprised that makedirs mode worked that way. | 16:49 |
| clarkb | corvus: with graphite you could do tenant value of * to get total cost for a project too, though if you have enough tenants that may be costly. I'm fine with it as is I guess since we can always add the tenant scoped project later too | 16:50 |
| corvus | clarkb: yeah, the thing i've learned from graphite is if you want a roll-up value, emit it, don't try to calculate it. because that doesn't scale past 10 metrics. | 16:51 |
| tobiash | clarkb: I'm not sure I completely understood your comment | 16:52 |
| clarkb | tobiash: which one? | 16:52 |
| tobiash | clarkb: the 'if node.resources' is inside a loop | 16:52 |
| clarkb | tobiash: oh right. In that case we should do if build_set and resources | 16:53 |
| clarkb | tobiash: mostly that block is a noop if resources if {} but we execute it anyway | 16:53 |
| clarkb | (or do we want to emit the zero value stats? I don't think they produce much value but maybe I'm missing a use case) | 16:54 |
| tobiash | clarkb: I think you're right, we can do if build_set and resources | 16:54 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 16:55 |
| tobiash | emitStatsResources emits the gauges so if nothing changes there is no point in updating it | 16:56 |
| clarkb | tobiash: mostly my concern was doing unnecessary work for executor only jobs and then I tricked myself into thinking we could further simplify it | 16:56 |
| tobiash | clarkb: regarding the other question, are you satisfied with corvus' response of should we discuss this further? | 16:57 |
| clarkb | tobiash: I'm good with corvus' comment. We can always add tenent.project later if we find we need it too | 16:57 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Report tenant and project specific resource usage stats https://review.openstack.org/616306 | 17:00 |
| *** manjeets_ is now known as manjeets | 17:00 | |
| tobiash | Shrews: responded on the ansible spec https://review.openstack.org/623927 | 17:10 |
| tobiash | corvus, SpamapS: what do you think? ^ | 17:10 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Report tenant and project specific resource usage stats https://review.openstack.org/616306 | 17:14 |
| corvus | tobiash, Shrews, SpamapS: i've been thinking about suggesting that we stop overriding ansible modules and only rely on bubblewrap. i haven't thought everything through yet -- don't consider this a formal proposal. but should we pause and discuss that before we move forward on multi-ansible? | 17:16 |
| corvus | mordred: ^ | 17:17 |
| corvus | tobiash: (and, fwiw, i was thinking we'd do the next-higher version with symlinks) | 17:18 |
| corvus | that seems like the easiest to implement | 17:18 |
| tobiash | Shrews, mnaser: responded on 635584 | 17:19 |
| tobiash | corvus: we also rely on these modules for log streaming and command module, so I think we still need them (maybe not all of them) even with bubblewrap | 17:20 |
| corvus | tobiash: okay, so we need to pick symlinks or metadata either way. | 17:21 |
| tobiash | corvus: I'm happy with either and think metadata might be even more flexible and less hacky | 17:21 |
| corvus | (and therefore, multi-ansible doesn't need to depend on this decision, but we should still probably decide before we put too much work into making new overrides) | 17:21 |
| tobiash | as we don't have to create a fancy directory structure with symlinks, but just attach versions | 17:22 |
| corvus | tobiash: yeah, i don't have a strong preference and am happy to say it's the implementor's choice :) | 17:22 |
| tobiash | then I'd try the metadata and avoid the subdir mess | 17:22 |
| mnaser | tobiash: mordred mentioned that there has been changes which enable usage of github checks api? how is that any different than the current state of things? | 17:27 |
| tobiash | mnaser: not in zuul, but github3.py now supports that so we finally could add this to zuul | 17:28 |
| mnaser | i guess individual jobs can report directly to the PR rather than 'check' passed? | 17:28 |
| tobiash | and this yes | 17:28 |
| mnaser | and then the 'pipeline' is really the 'check suite' | 17:28 |
| tobiash | but there are some conceptual questions to solve | 17:29 |
| mnaser | so we have 'check' or 'gate' and a 'check suite' and then individual jobs as 'checks' | 17:29 |
| mnaser | gotcha | 17:29 |
| tobiash | because there is no concept of non-voting checks e.g. | 17:29 |
| mnaser | i see a 'neutral' result | 17:29 |
| tobiash | each check is voting and automatically required | 17:29 |
| mnaser | in conclusions | 17:29 |
| tobiash | yes, but you cannot state failed but nonvoting | 17:29 |
| mnaser | ah right so the non voting job result is not visible | 17:30 |
| mnaser | interesting | 17:30 |
| tobiash | and I think that doesn't really fit into check and gate pipeline concept | 17:30 |
| mnaser | okay, that's super interesting. i'd love to be part of that effort both discussions and writing code afterwards if you're going to be working on that | 17:30 |
| tobiash | so there is some discussion needed how that could be used by zuul in a useful way without breaking all concepts | 17:30 |
| mnaser | yep, i see that | 17:31 |
| tobiash | swest had a deeper look into that some weeks ago | 17:31 |
| tobiash | and it looks like status api and checks api are not really integrated with each other | 17:32 |
| tobiash | that are just two different things | 17:32 |
| * SpamapS thinking | 17:41 | |
| SpamapS | I'm a fan of symlinks, and agree with corvus that this feels like an implementor's choice. | 17:43 |
| *** sdake has quit IRC | 17:47 | |
| *** jpena is now known as jpena|off | 17:48 | |
| *** remi_ness has quit IRC | 17:50 | |
| tobiash | SpamapS: yeah I like symlinks too, but I fear that this will get a complicated link mess ;) | 17:53 |
| Shrews | tobiash: not sure i follow the "self.args" does not exist. you mean the attribute? b/c self.args is definitely defined | 17:54 |
| tobiash | but maybe I'll try both ways and we can then decide what's better | 17:54 |
| SpamapS | Perhaps a hybrid approach: metadata that makes symlinks? Though, that sounds like "an ansible playbook". ;) | 17:54 |
| tobiash | Shrews: not in this class, this method has args and returns them | 17:54 |
| Shrews | tobiash: ah, the return value. ok, was trying to debug the test failures in my head | 17:56 |
| tobiash | Shrews: the result of mnaser's edit: http://logs.openstack.org/84/635584/4/check/tox-py35/c90fa67/testr_results.html.gz | 17:56 |
| Shrews | i believe you | 17:57 |
| tobiash | Shrews: the question is why the functional tests failed | 17:58 |
| tobiash | Shrews: http://logs.openstack.org/84/635584/3/check/nodepool-functional-py35/b103b89/controller/logs/screen-nodepool-builder.txt.gz | 18:00 |
| Shrews | tobiash: they failed b/c the builder and launcher would not start b/c of a pid issue, which depends on that nodaemon attribute, which led me to suspect that portion of code i pointed out | 18:00 |
| tobiash | ah I see | 18:00 |
| tobiash | hrm | 18:01 |
| tobiash | needs more digging | 18:01 |
| Shrews | but i only debugged in my head, so was mostly a guess | 18:01 |
| tobiash | Shrews: is the devstack nodepool expected to run daemonized or in foreground? | 18:01 |
| Shrews | i think it runs in foreground, which is why we missed the builder image build timeout issue when daemonized | 18:02 |
| mnaser | mordred: looks like your idea was ambitious about sourcing from diskimage-builder except it looks like we don't actually publish any diskimage-builder images :) | 18:03 |
| Shrews | corvus pointed that out for me, actually | 18:03 |
| *** gtema has quit IRC | 18:03 | |
| corvus | mnaser: yeah, i think mordred intended that as a sketch of what it would look like if we did publish images; meanwhile, i believe he's investigating what it would actually take to do so | 18:04 |
| corvus | mnaser: 635892 and 635896 are part of that | 18:05 |
| mnaser | corvus: ooo i didn't see those | 18:06 |
| tobiash | Shrews: I think I've spotted the bug | 18:06 |
| mnaser | ok i guess diskimage-builder doesnt feed notification in here, there ya go | 18:07 |
| mnaser | i was confused where those patches were coming from :) | 18:07 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: docker: don't daemonize when starting images https://review.openstack.org/635584 | 18:11 |
| tobiash | Shrews, mnaser: that should fix the error and hopefully the confusion args vs self.args ^ | 18:11 |
| clarkb | is there a bug in zuul/tools/encrypt_secret.py reading from stdin? | 18:12 |
| clarkb | it seems to be doing a normal read not a readline so not sure how we are expected to end that read | 18:12 |
| tobiash | clarkb: I'd expect it to read until eof | 18:12 |
| clarkb | tobiash: seems to require a newline then an EOF | 18:17 |
| clarkb | but I'm not sure I want the newline in my secret /me checks if it chomps it | 18:18 |
| clarkb | ok it does | 18:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Return artifacts as dicts and add metadata https://review.openstack.org/634825 | 18:19 |
| corvus | clarkb: if you want to terminate stdin and you haven't just hit a newline, you may need to hit ctrl-d twice | 18:20 |
| *** remi_ness has joined #zuul | 18:20 | |
| clarkb | corvus: ah is that the trick? in any case script does strip the whitespace so we are good | 18:20 |
| corvus | oh fascinating | 18:24 |
| corvus | https://stackoverflow.com/questions/2162914/why-do-i-have-to-press-ctrld-twice-to-close-stdin | 18:24 |
| corvus | til what ctrl-d *actually* does | 18:24 |
| corvus | (it flushes the buffer) | 18:24 |
| corvus | enter also flushes the buffer | 18:25 |
| corvus | so a ctrl-d entered with an empty buffer causes read() to return 0 bytes, which is interpreted by most programs as EOF | 18:25 |
| corvus | that trick only works if the buffer is empty, which happens either after an enter, or after a previous ctrl-d. | 18:26 |
| tobiash | every day something new to learn :) | 18:28 |
| corvus | tobiash: i just realized i forgot to post a draft comment in reply to you on 634825. i updated the patch based on your comment, so you probably came to the same conclusion, but i had meant to reply. sorry. it's posted now. | 18:31 |
| corvus | latest ps should fix the test failures | 18:31 |
| *** saneax has joined #zuul | 18:38 | |
| fungi | tobias-urdin: thanks for the clarification... so catching up, it sounds like we could safely run the upload task on the executor so long as ruby and c++ toolchains are present to be able to compile... what exactly? is the tool which performs the uploading recompiled at each invocation? | 18:43 |
| fungi | and is there no precompiled binary package of whatever it needs suitable for installation on ubuntu? | 18:44 |
| fungi | just trying to wrap my head around what about the uploader requires presence of a compiler | 18:45 |
| *** electrofelix has quit IRC | 18:50 | |
| tobias-urdin | fungi: it's the dependencies on these three gems that contains native ruby extensions that is compiled upon installation | 18:51 |
| tobias-urdin | if the build tools and these gems are installed on the executors | 18:51 |
| tobias-urdin | https://github.com/openstack-infra/zuul-jobs/blob/master/roles/upload-puppetforge/tasks/main.yaml#L25 | 18:51 |
| tobias-urdin | we can skip all install parts, or just install the ruby part and install the gems on each run, which is probably preferable if the gems need to be updated | 18:52 |
| tobias-urdin | think about it as if you do "pip install <something>" and it requires puppet-dev and some libxml-dev as example :) | 18:53 |
| fungi | partly asking because the tasks which run on the executor happen inside bubblewrap and so potentially need additional bits of the outer filesystem mappen in or may be hamstrung by capability filters | 18:53 |
| fungi | so i guess the idea is that we gem install the uploader separately outside bubblewrap and map its files into the bwrap context, then the rake task doesn't actually need to download and compile anything? | 18:55 |
| tobias-urdin | hm, isn't PATH or /usr/local/bin or similar exposed inside the bwrap? | 18:57 |
| tobias-urdin | fungi: would it be easier if i investigated if i could build some python tooling that did the upload? | 18:59 |
| fungi | maybe, but hoping to get more input from other zuul maintainers too who might have better suggestions before you go writing something from scratch | 19:01 |
| tobias-urdin | the issue here is that we need to install stuff inside executor, since ansible is there, python is there and it probably very simple reading a tarball and doing some HTTP code to upload | 19:02 |
| fungi | it doesn't seem to me like uploading a file using an interpreted language like ruby should require more than the interpreter, so curious to know why a c++ toolchain is needed | 19:02 |
| tobias-urdin | fungi: does this make more sense? http://paste.openstack.org/show/744759/ | 19:07 |
| tobias-urdin | a ruby gem can have native c/c++ code in modules that is not implemented in ruby but in native code that is compiled upon installation | 19:08 |
| tobias-urdin | sorry i suck at trying to articulate what i mean sometimes | 19:09 |
| corvus | tobias-urdin, fungi: we can ask that folks either install the gems on the executor, or ask that they install c++ so that the job can build the gems. but it would be friendlier to zuul operators if we didn't need to do either of those things (because it keeps the executors simple). on the other hand, we it would be better to do that than to implement a complicated tool from scratch, especially if the upload | 19:10 |
| corvus | api might change over time and we'd have to maintain it. | 19:10 |
| fungi | it tells me that you expect puppet-blacksmith to be available for some reason, and that puppet-blacksmith in turn has some non-pure-ruby extensions | 19:10 |
| fungi | what is puppet-blacksmith used for? why does the upload task need it instead of just relyig on pure ruby implementations of things? | 19:11 |
| corvus | so there's a trade-off there. i *think* i would say that if the upload api is very simple, let's look into implementing something from scratch. we might even be able to do it with pure ansible using the uri module. but if it's more than a simple authentication/http post roundtrip, then we should install things on the executor. | 19:11 |
| * fungi checks to see if puppet-blacksmith is packaged for distros directly already in binary form | 19:11 | |
| tobias-urdin | i just went with it since it already implemented the upload logic https://github.com/voxpupuli/puppet-blacksmith/blob/master/lib/puppet_blacksmith/forge.rb | 19:12 |
| tobias-urdin | and was already in use here http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/puppet-httpd-forge-upload/run.yaml | 19:12 |
| fungi | yeah, no sign of puppet-blacksmith in ubuntu, so we'd need to build it(s extensions) from source i guess if it's required | 19:12 |
| tobias-urdin | but yeah i think we can replicate it with some super simple python or even ansible | 19:13 |
| *** sdake has joined #zuul | 19:29 | |
| *** daniel2 has quit IRC | 19:37 | |
| *** daniel2 has joined #zuul | 19:40 | |
| dkehn | clarkb: in https://zuul-ci.org/docs/zuul/admin/components.html#components the executor.finger_port default is 7900 and the fingers.port default is 79, I'm assuming one of them needs to change? | 19:50 |
| clarkb | dkehn: on the executor side it listens on a hgih port so you don't need root | 19:50 |
| clarkb | dkehn: the finger gateway service listens on 79 by default which is finger procol default and proxies to the high port on executors | 19:51 |
| dkehn | clarkb: ok, so no change required | 19:51 |
| clarkb | shouldn't be | 19:51 |
| dkehn | thanks | 19:51 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: docker: don't daemonize when starting images https://review.openstack.org/635584 | 19:53 |
| dkehn | clarkb: in the docker-come.yaml is there a necessity to define ports: ? | 19:53 |
| dkehn | for the executor | 19:54 |
| dkehn | these are running on different containers | 19:54 |
| dkehn | or should the fingergw be running on the same as the executor | 19:54 |
| tobiash | dkehn: fingergw runs in its own container | 19:55 |
| tobiash | so yes, you probably need to declare the finger_port | 19:55 |
| tobiash | this is also necessary for websocket streaming via zuul-web | 19:55 |
| dkehn | tobiash: I've defined in for the fingergw contain as 79 , I'm guessing then I should define the excutor"s port 7900, does that sound correct? | 19:56 |
| tobiash | ++ | 19:56 |
| tobiash | dkehn: and you also need to make sure that the executor knows its hostname (or configure it in zuul.conf) | 19:57 |
| dkehn | ack | 19:57 |
| tobiash | zuul-web and fingergw will use this hostname to connect to the executor | 19:57 |
| dkehn | tobiash: can you point to an example setup | 19:58 |
| tobiash | dkehn: in case the executor cannot resolve its hostname you can configure it here: https://zuul-ci.org/docs/zuul/admin/components.html#attr-executor.hostname | 19:59 |
| tobias-urdin | fungi: corvus tried to implement it in ansible, i can do all authenticate and try to upload using the uri module | 19:59 |
| *** sdake has quit IRC | 19:59 | |
| tobias-urdin | but there is no way to read the binary data of the tarball and send it, so i guess i'll need to revert to using python | 19:59 |
| *** sdake has joined #zuul | 20:00 | |
| fungi | oh, so it needs to unpack the tarball? can we possibly just archive the tarball content it needs for that step separate from the tarball itself? | 20:00 |
| fungi | like, if there's some metadata file in the tarball, just retrieve a bare copy of that from the build along with the tarball itself? | 20:01 |
| tobias-urdin | i need to sent a "file" field in the POST body with the tarball binary data, just a plain read of the file | 20:01 |
| tobias-urdin | but i can't read binary blobs from ansible iiuc | 20:02 |
| tobias-urdin | i can't use "src" for uri module because i can't map that loaded file to the "file" body field | 20:02 |
| fungi | oh... got it. ansible is actually incapable of reading a non-text file into a variable? | 20:03 |
| tobias-urdin | can't use lookup('file', 'path/to/tarball.tar.gz') atleast | 20:03 |
| fungi | ansible can also call arbitrary python functions, right? | 20:03 |
| tobias-urdin | tries to parse it and gives | 20:03 |
| tobias-urdin | Error was a <type 'exceptions.UnicodeDecodeError'>, original message: 'utf8' codec can't decode byte 0x8b in position 1: invalid start byte | 20:03 |
| fungi | ahh, it decides the file content should be utf-8 i guess, when it's really just raw binary data | 20:04 |
| *** sdake has quit IRC | 20:09 | |
| *** sdake has joined #zuul | 20:11 | |
| *** sdake has quit IRC | 20:13 | |
| *** sdake has joined #zuul | 20:14 | |
| *** bhavikdbavishi has quit IRC | 20:14 | |
| *** saneax has quit IRC | 20:37 | |
| *** sdake has quit IRC | 20:44 | |
| *** sdake has joined #zuul | 20:48 | |
| tobias-urdin | fungi: we can prob do this with ansible now | 21:06 |
| tobias-urdin | here is a module http://paste.openstack.org/show/744763/ | 21:06 |
| tobias-urdin | here is usage http://paste.openstack.org/show/744764/ | 21:06 |
| tobias-urdin | just requires python requests module to be installed on the executor node | 21:06 |
| corvus | tobias-urdin: the slurp module might work | 21:12 |
| corvus | tobias-urdin: it base64 encodes the data, but i think there's a base64 decode filter | 21:13 |
| corvus | but yeah, a module should be fine | 21:13 |
| corvus | since you've already written it :) | 21:13 |
| tobias-urdin | not sure were i should place it though | 21:13 |
| corvus | tobias-urdin: in the 'library' dir underneath the forge upload role | 21:14 |
| tobias-urdin | yeah but you think i should just keep the upload-puppetforge role in infra/zuul-jobs? | 21:14 |
| corvus | yep | 21:15 |
| tobias-urdin | okok, on it | 21:16 |
| corvus | this should be useful to anyone | 21:16 |
| *** rlandy has quit IRC | 21:33 | |
| openstackgerrit | Tobias Urdin proposed openstack-infra/zuul-jobs master: Rework upload-puppetforge role to use module https://review.openstack.org/635941 | 21:34 |
| fungi | and much, much cleaner, thanks! | 21:38 |
| tobias-urdin | think i shot myself in the foot though since it takes the full path to a tarball | 21:48 |
| *** sdake has quit IRC | 21:58 | |
| *** krasmussen has joined #zuul | 22:07 | |
| *** remi_ness has quit IRC | 22:09 | |
| openstackgerrit | Tobias Urdin proposed openstack-infra/zuul-jobs master: Rework upload-puppetforge role to use module https://review.openstack.org/635941 | 22:11 |
| krasmussen | Hey, has anyone seen a `AttributeError: 'MergeJob' object has no attribute 'updated'` error on startup of the zuul scheduler before? Trace can be found here: https://pastebin.com/daYyXBUk | 22:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Remove "--verbose" from yarn commands https://review.openstack.org/635973 | 22:23 |
| corvus | krasmussen: yes, there's a patch in review to fix that: https://review.openstack.org/633259 | 22:26 |
| *** remi_ness has joined #zuul | 22:26 | |
| *** klindgren has joined #zuul | 22:27 | |
| dkehn | ls | 22:29 |
| corvus | no such file or directory | 22:30 |
| dkehn | haha | 22:30 |
| dkehn | wrong window | 22:30 |
| krasmussen | I guess you could have gone with `irc: ls: command not found` | 22:32 |
| krasmussen | Also thanks corvus I'll take a look :) | 22:32 |
| *** remi_ness has quit IRC | 22:35 | |
| *** panda has quit IRC | 22:46 | |
| *** panda has joined #zuul | 22:49 | |
| openstackgerrit | Tobias Urdin proposed openstack-infra/zuul-jobs master: Rework upload-puppetforge role to use module https://review.openstack.org/635941 | 22:56 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove "--verbose" from yarn commands https://review.openstack.org/635973 | 23:08 |
| *** sdake has joined #zuul | 23:33 | |
| krasmussen | @corvus sadly I am still looking at the same stacktrace after applying that patch locally :( any other thoughts? | 23:40 |
| corvus | krasmussen: ha, that patch has a logic error | 23:42 |
| krasmussen | Well, I dealt with the `if not hasattr(job, 'updated') and not job.updated` and seperated those out already. | 23:43 |
| corvus | krasmussen: but yeah, that patch should only produce a more sane error message, there's still probably an underlying problem | 23:43 |
| corvus | krasmussen: i think you may need to check your merger or executor logs to find out what actually happened | 23:44 |
| krasmussen | I do see a `Exception: Cat job <gear.Job 0x7f129021a438 handle: b'H:10.32.55.158:2' name: merger:cat unique: 83e940b9166149fda0a4ccc7ce6d9bb4> failed ` | 23:44 |
| krasmussen | But I'm also new enough to zuul to where that is greek to me. | 23:44 |
| corvus | krasmussen: at that point, the scheduler is asking mergers (executors contain an internal merger, so they participate in this too) to fetch files from git. | 23:44 |
| corvus | krasmussen: if you look at the merger or executor logs, you should see an actuall error in them about why they couldn't clone or fetch from some git repo | 23:45 |
| corvus | krasmussen: (thanks for testing that patch, i glossed right over the boolean error) | 23:45 |
| corvus | krasmussen: that unique id is also used on the mergers, so grepping for "83e940b9166149fda0a4ccc7ce6d9bb4" should get you near the right spot | 23:46 |
| krasmussen | Hum... my merger logs are pretty much empty. | 23:47 |
| corvus | try the executor(s) then | 23:48 |
| krasmussen | Still nothing... kinda just adding debug info and making my way through. | 23:56 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!