| *** altlogbot_2 has quit IRC | 00:10 | |
| *** altlogbot_0 has joined #zuul | 00:13 | |
| *** mattw4 has quit IRC | 00:20 | |
| *** mattw4 has joined #zuul | 00:20 | |
| *** mattw4 has quit IRC | 00:25 | |
| *** saneax has joined #zuul | 01:30 | |
| *** rlandy has joined #zuul | 02:06 | |
| *** rlandy has quit IRC | 02:15 | |
| *** jamesmcarthur has joined #zuul | 02:50 | |
| *** saneax has quit IRC | 03:06 | |
| *** bhavikdbavishi has joined #zuul | 03:08 | |
| *** bhavikdbavishi1 has joined #zuul | 03:31 | |
| *** bhavikdbavishi has quit IRC | 03:32 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:32 | |
| *** jamesmcarthur has quit IRC | 03:45 | |
| *** jamesmcarthur has joined #zuul | 04:04 | |
| *** jamesmcarthur has quit IRC | 04:06 | |
| *** nickx-intel has joined #zuul | 04:28 | |
| *** nickx-intel is now known as nickx | 04:30 | |
| *** nickx is now known as nicks | 04:31 | |
| *** nicks is now known as nick_x | 04:31 | |
| *** nick_x has quit IRC | 04:33 | |
| *** nick_x has joined #zuul | 04:41 | |
| openstackgerrit | Merged zuul/zuul master: pass-to-parent: Fix passing multiple secrets to parent https://review.opendev.org/660105 | 04:51 |
|---|---|---|
| *** nick_x has quit IRC | 05:05 | |
| *** raukadah is now known as chandankumar | 05:44 | |
| *** saneax has joined #zuul | 06:05 | |
| *** themr0c has joined #zuul | 06:50 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: remove SafeLoader left-over from ZuulJSONEncoder https://review.opendev.org/659026 | 07:03 |
| *** hashar has joined #zuul | 07:13 | |
| *** pcaruana has joined #zuul | 07:17 | |
| openstackgerrit | Merged zuul/zuul-jobs master: add-build-sshkey: remove previously authorized build-sshkey https://review.opendev.org/632620 | 07:36 |
| *** saneax has quit IRC | 07:36 | |
| *** jpena|off is now known as jpena | 07:44 | |
| *** bhavikdbavishi has quit IRC | 07:44 | |
| *** hashar has quit IRC | 07:48 | |
| *** hashar has joined #zuul | 07:48 | |
| *** jangutter has joined #zuul | 07:56 | |
| *** jangutter has quit IRC | 08:03 | |
| *** themr0c has quit IRC | 08:07 | |
| *** jangutter has joined #zuul | 08:08 | |
| *** jangutter has quit IRC | 08:13 | |
| *** swest has joined #zuul | 08:18 | |
| *** jangutter has joined #zuul | 08:20 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: stage-output: fix dotfiles handling https://review.opendev.org/648658 | 08:23 |
| *** swest has quit IRC | 08:23 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: stage-output: fix dotfiles handling https://review.opendev.org/648658 | 08:25 |
| *** bhavikdbavishi has joined #zuul | 08:31 | |
| *** gtema has joined #zuul | 08:32 | |
| *** panda is now known as panda|rover | 08:50 | |
| *** markwork__ has joined #zuul | 08:52 | |
| *** electrofelix has joined #zuul | 09:00 | |
| *** saneax has joined #zuul | 09:03 | |
| *** markwork__ is now known as ofosos | 09:10 | |
| *** hashar has quit IRC | 09:46 | |
| *** flaper87 has quit IRC | 09:48 | |
| *** bhavikdbavishi has quit IRC | 10:23 | |
| *** bhavikdbavishi has joined #zuul | 10:23 | |
| *** bhavikdbavishi has quit IRC | 10:28 | |
| *** bhavikdbavishi has joined #zuul | 10:43 | |
| *** gtema has quit IRC | 10:58 | |
| *** toabctl has joined #zuul | 10:58 | |
| *** panda|rover is now known as panda|rover|eat | 11:16 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Upgrade formatting of the patch series. https://review.opendev.org/660683 | 11:27 |
| *** jpena is now known as jpena|lunch | 11:31 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:35 | |
| *** rfolco is now known as rfolco|dentist | 11:39 | |
| *** themroc has joined #zuul | 11:41 | |
| *** bhavikdbavishi has quit IRC | 11:49 | |
| *** bhavikdbavishi has joined #zuul | 11:51 | |
| *** gtema has joined #zuul | 11:55 | |
| *** bhavikdbavishi has quit IRC | 11:56 | |
| *** rlandy has joined #zuul | 11:57 | |
| *** panda|rover|eat is now known as panda|rover | 12:06 | |
| *** hashar has joined #zuul | 12:15 | |
| *** jpena|lunch is now known as jpena | 12:33 | |
| *** persia has quit IRC | 12:34 | |
| *** persia has joined #zuul | 12:35 | |
| *** bhavikdbavishi has joined #zuul | 12:52 | |
| *** jamesmcarthur has joined #zuul | 12:54 | |
| *** flepied has joined #zuul | 12:58 | |
| *** themroc has quit IRC | 13:05 | |
| *** bhavikdbavishi has quit IRC | 13:10 | |
| *** rfolco|dentist is now known as rfolco | 13:29 | |
| *** jamesmcarthur has quit IRC | 14:16 | |
| *** jamesmcarthur has joined #zuul | 14:18 | |
| *** jamesmcarthur has quit IRC | 14:20 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure driver - https://pagure.io/pagure/ https://review.opendev.org/604404 | 14:29 |
| *** hashar has quit IRC | 14:30 | |
| dmsimard | heads up: I've somehow missed a significant regression in ARA with Ansible 2.8, documented the issue here while I work on a fix: https://github.com/ansible-community/ara/issues/46 | 14:36 |
| dmsimard | Due to the nature of the change, there's a good chance that the Zuul callback(s) are impacted as well | 14:36 |
| dmsimard | tl;dr, some things are no longer None or [], they're now some new Sentinel class objects: http://paste.openstack.org/raw/751936/ | 14:37 |
| *** jamesmcarthur has joined #zuul | 14:41 | |
| *** jamesmcarthur has quit IRC | 14:41 | |
| *** jamesmcarthur has joined #zuul | 14:42 | |
| *** saneax has quit IRC | 14:44 | |
| *** saneax has joined #zuul | 14:44 | |
| pabelanger | dmsimard: thanks for heads up, I actually wanted to start using 2.8 for zuul.opendev.org soon, cc clarkb. I have some spare cycles to help with that | 14:59 |
| fungi | good to know, since we haven't upped our default yet | 15:02 |
| *** saneax has quit IRC | 15:21 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Annotate builds with event id https://review.opendev.org/658895 | 15:25 |
| mrhillsman | is there already a way to see which node a job is running on? | 15:28 |
| corvus | mrhillsman: as a user, only if the job includes a role that outputs that info (in opendev, ours do -- there's a standard role for that in zuul-jobs) | 15:28 |
| mrhillsman | from the dashboard | 15:28 |
| mrhillsman | ok cool, ty sir | 15:29 |
| *** hashar has joined #zuul | 15:32 | |
| *** gtema has quit IRC | 15:34 | |
| *** nhicher has quit IRC | 15:35 | |
| *** fbo_ has quit IRC | 15:37 | |
| *** flepied has quit IRC | 15:41 | |
| *** jpena is now known as jpena|off | 16:02 | |
| *** pwhalen has joined #zuul | 16:03 | |
| *** jangutter has quit IRC | 16:05 | |
| *** themroc has joined #zuul | 16:16 | |
| pabelanger | mordred: in clouds.yaml, for nodepool, is there a way to not ask for a FIP, if a network is able to assign one? | 16:21 |
| pabelanger | for example, we have a cloud where they can give us one, but because this is a 2nd network we don't actually need to use it | 16:21 |
| *** jpena|off is now known as jpena | 16:22 | |
| *** themroc has quit IRC | 16:23 | |
| pabelanger | mordred: I think nat_destination: false, is the setting? | 16:25 |
| mordred | pabelanger: I'd use floating_ip_source: None | 16:26 |
| pabelanger | mordred: ack, thanks | 16:26 |
| *** hashar has quit IRC | 16:30 | |
| *** mattw4 has joined #zuul | 16:39 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure driver - https://pagure.io/pagure/ https://review.opendev.org/604404 | 16:40 |
| openstackgerrit | Merged zuul/zuul master: Install latest git-review from PyPI in quickstart https://review.opendev.org/659674 | 16:57 |
| openstackgerrit | Merged zuul/zuul-jobs master: Delete files in dest that don't exist https://review.opendev.org/648815 | 17:01 |
| pabelanger | so, I'm not sure if github.com is having issues with their event system, we've seen a large lag in the last few days (upwards of 10mins) | 17:05 |
| pabelanger | or if related to adding our github app to ansible/ansible repo | 17:05 |
| pabelanger | but actions on PRs seem to be lagging for sure | 17:06 |
| *** electrofelix has quit IRC | 17:10 | |
| *** chandankumar is now known as raukadah | 17:14 | |
| pabelanger | okay, looks to be github issue | 17:15 |
| pabelanger | sorry for noise | 17:15 |
| *** bhavikdbavishi has joined #zuul | 17:15 | |
| Shrews | mordred: back to your autohold admin complaints, would the api proposed in http://paste.openstack.org/show/751950/ make it a more pleasant experience? | 17:15 |
| *** jpena is now known as jpena|off | 17:17 | |
| *** themroc has joined #zuul | 17:23 | |
| mordred | Shrews: yes - I think I would like that | 17:25 |
| Shrews | mordred: cool. that would definitely require storing autohold info in zookeeper rather than in memory. corvus should perhaps weigh in on such a change. | 17:30 |
| corvus | Shrews: that looks nice | 17:34 |
| corvus | mordred: going back to the convo from earlier -- one other thing that can be helpful is to set the hold expiration duration -- so you can set it to a day or something and they will automatically be deleted | 17:34 |
| mordred | ++ | 17:35 |
| corvus | (i think opendev should update our systems to default to 1 day, we currently default to no expiration) | 17:35 |
| Shrews | corvus: agreed. that would at least force folks to think about how long they are around | 17:35 |
| Shrews | rather than set and forget | 17:36 |
| *** themroc has quit IRC | 18:02 | |
| *** nhicher has joined #zuul | 18:05 | |
| *** jamesmcarthur has quit IRC | 18:38 | |
| *** bhavikdbavishi has quit IRC | 18:41 | |
| fungi | pabelanger: be aware of the lag in processing github events for pull requests on projects with many open pull requests, since the scheduler has to basically query the entire set to figure out which one has a match with a given git ref | 18:43 |
| *** flepied has joined #zuul | 18:43 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure driver - https://pagure.io/pagure/ https://review.opendev.org/604404 | 18:43 |
| fungi | pabelanger: those can pile up too if there are a lot of events in a short timeframe | 18:43 |
| *** sshnaidm is now known as sshnaidm|afk | 18:45 | |
| tobiash | corvus: do you want to wait with 658889 until there is the nodepool part? | 18:51 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Log github requests with annotated events https://review.opendev.org/660800 | 18:57 |
| tobiash | pabelanger: this might help analysing github request lag ^ | 18:57 |
| pabelanger | fungi: yah, so far we haven't seen that. Mostly delayed events, at first I was thinking because there was much more events coming in, due to ansible/ansible but seems just to be github issue atm | 19:00 |
| pabelanger | tobiash: looking | 19:00 |
| fungi | pabelanger: the github event processing has its own queue, separate from the scheduler's normal event queue | 19:03 |
| tobiash | and it serially processes each event | 19:03 |
| tobiash | so if the event processing is slow the events can queue up | 19:04 |
| tobiash | and 660800 could help to find further potential optimizations in that area | 19:04 |
| * SpamapS wonders how long github's webhook queue is | 19:09 | |
| pabelanger | I'd guess 2-3 hours | 19:10 |
| pabelanger | had some old events show up for zuul.a.c | 19:11 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Log github requests with annotated events https://review.opendev.org/660800 | 19:12 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Annotate logs around build completion and cancellation https://review.opendev.org/660806 | 19:12 |
| *** EmilienM is now known as EmilienM|bbl | 19:15 | |
| *** EmilienM|bbl is now known as EmilienM | 19:16 | |
| *** flaper87 has joined #zuul | 19:21 | |
| flaper87 | in the docker-compose example, I see the executor container is run with `priviledge: true`. Can someone provide more details on why that's needed? | 19:22 |
| fungi | the executor wants to sandbox ansible forks inside bubblewrap, and needs to bindmount some trees into the bubblewrap chroot... also probably sets some capabilities on it? | 19:23 |
| pabelanger | yah, because of bubble wrap | 19:24 |
| fungi | basically we don't want arbitrary ansible plugins to be able to do things in the greater context of the executor itself | 19:24 |
| pabelanger | flaper87: which means, in some COE (eg: openshift) it is harder to run zuul-executor | 19:24 |
| pabelanger | you have to disable some security stuff on openshift side | 19:24 |
| pabelanger | but is doeable, tobiash does it | 19:25 |
| pabelanger | doable* | 19:25 |
| fungi | i guess the alternative was to design the executor to interact with a coe to fork ansible processes inside coe-managed containers, but that would add lots of complexity and probably lock-in to some specific coes | 19:25 |
| flaper87 | pabelanger: that's why I'm asking | 19:26 |
| flaper87 | I'm running this thing on k8s now and I was wondering if I could avoid that | 19:26 |
| flaper87 | sounds like I can't | 19:26 |
| * flaper87 runs this container as priviledge | 19:26 | |
| fungi | basically, the zuul executor is a very basic container orchestration engine | 19:27 |
| fungi | so trying to layer it onto another coe would take a bit of work | 19:27 |
| pabelanger | flaper87: yah, for now, can't. however, I believe there is a longer term plan to see how we can drop privileged. I believe mordred talked with some k8s folks about it | 19:27 |
| pabelanger | or coreos | 19:27 |
| tobiash | flaper87: theoretically you cpuld avoid it if you enable unprivileged user namespace s in your cluster | 19:27 |
| *** jamesmcarthur has joined #zuul | 19:39 | |
| corvus | tobiash: we can merge 658889 now. i only didn't +3 because i was on the plane | 19:42 |
| flaper87 | In the docker-compose example, the playbooks dir is bindmounted in the container. IIUC, that is only needed for the example and it shouldn't be needed for a production environment. Is that correct? | 19:45 |
| tobiash | corvus: thanks, how was your flight? | 19:45 |
| corvus | tobiash: bleh. | 19:45 |
| corvus | tobiash: it's over and i'm home, which is nice. :) | 19:46 |
| tobiash | :) | 19:48 |
| fungi | home is always better than being on an airplane, in my experience | 19:55 |
| pabelanger | tobiash: heh, managed to just see this github exception: http://paste.openstack.org/show/751955/ | 19:58 |
| tobiash | whoops | 19:59 |
| fungi | neat! | 19:59 |
| fungi | i think we knew that could happen, but yeah hard to know how to deal with it | 19:59 |
| tobiash | but this cannot be avoided | 19:59 |
| pabelanger | trying to see which event it was from | 20:00 |
| fungi | gerrit actually won't allow two changes to have patchsets with the same commit hash | 20:00 |
| pabelanger | yah, can't tell from scheduler log | 20:02 |
| flaper87 | tobiash: did you store the db credentials in a secret? I was thinking to create a secret with a zuul.conf file that contained the db section and pass that to the zuul binaries (assuming multiple config files can be passed) | 20:03 |
| pabelanger | would need to look at github app | 20:03 |
| corvus | flaper87: only one zuul.conf is supported | 20:03 |
| pabelanger | SpamapS: looks like still about 1hr delay on github events | 20:04 |
| tobiash | flaper87: we compose the zuul.conf from multiple configmaps and secrets in the boot script during container startup | 20:04 |
| corvus | flaper87: you may also be interested in this effort which we are starting: https://review.opendev.org/659180 | 20:05 |
| tobiash | corvus: thanks for the reminder, I should give that a review too :) | 20:05 |
| flaper87 | corvus: oh, I am. Hope to be able to contribute to that soon. | 20:06 |
| flaper87 | tobiash: is that script public? :P | 20:06 |
| tobiash | flaper87: not yet as it's tailored specifically to our deployment, but it's very easy to do so using configparser | 20:08 |
| tobiash | or we might want to add config dir support in zuul (which is easy too) | 20:09 |
| flaper87 | adding config dir support would be awesome! | 20:11 |
| pabelanger | yah, I'd review that patch :) | 20:15 |
| pabelanger | right now my zuul.conf is also templated a lot | 20:15 |
| *** pcaruana has quit IRC | 20:20 | |
| corvus | tobiash, clarkb: http://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=64792&rra_id=2&view_type=&graph_start=1558024599&graph_end=1558555158&graph_height=120&graph_width=500&title_font_size=12 | 20:29 |
| tobiash | corvus: that looks bad | 20:30 |
| corvus | that's since the last restart -- so something happened at 14:00ish on 21 may *without* a restart | 20:30 |
| corvus | (the last restart was on 16 may) | 20:30 |
| openstackgerrit | Dirk Mueller proposed zuul/zuul-jobs master: Switch to TOX_CONSTRAINTS_FILE https://review.opendev.org/657886 | 20:31 |
| corvus | Shrews: ^ fyi | 20:31 |
| pabelanger | eep (cacti) | 20:36 |
| *** mattw4 has quit IRC | 20:39 | |
| *** mattw4 has joined #zuul | 20:40 | |
| dmsimard | ara 0.16.4 is tagged to address the regression with ansible 2.8 | 20:57 |
| dmsimard | flaper87: oh hai, fancy seeing you here <3 | 20:58 |
| flaper87 | dmsimard: hey hey! yeah, zuuling my way through this world <3 | 20:58 |
| * mordred zuuls flaper87 and dmsimard | 21:02 | |
| mordred | flaper87, pabelanger: having now read scrollback - my takeaway from taking to people about the priv/unpriv thing is that we essentially need the same things that projects like buildah and podman need to be able to run rootless inside of a container (user namespaces as tobiash mentioned - as well as some other capability change somewhere) ... which is to say - there are a set of people working on a parallel | 21:13 |
| mordred | problem who want/need to solve it and who I believe are working through the issues | 21:13 |
| mordred | SO | 21:13 |
| mordred | we **should** end up in a place where we can drop privileged needs with little work of our own :) | 21:13 |
| *** flepied has quit IRC | 21:22 | |
| *** jamesmcarthur has quit IRC | 21:26 | |
| *** mattw4 has quit IRC | 21:41 | |
| *** mattw4 has joined #zuul | 21:42 | |
| *** jamesmcarthur has joined #zuul | 21:45 | |
| *** rlandy is now known as rlandy|bbl | 21:59 | |
| openstackgerrit | Merged zuul/zuul master: Annotate node request processing with event id https://review.opendev.org/658889 | 22:04 |
| *** jamesmcarthur has quit IRC | 22:26 | |
| *** mattw4 has quit IRC | 23:26 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Skip file matcher for pipeline using timer trigger https://review.opendev.org/660856 | 23:33 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: ansible-config: pin ara to <1.0.0 https://review.opendev.org/660859 | 23:38 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!