| corvus | was there perhaps a tenant reconfig? | 00:00 |
|---|---|---|
| corvus | (ie, did we merge a change shortly before that?) | 00:00 |
| fungi | for the zuul tenant? | 00:00 |
| corvus | yeah -- to any project in the tenant (ie, could be an opendev or openstack project) | 00:00 |
| fungi | oh, right | 00:01 |
| corvus | i'm wondering if there's some race with the web ui when the scheduler is reconfiguring or something | 00:01 |
| fungi | corvus: 777511 maybe? | 00:02 |
| fungi | changed zuul.d/projects.yaml in openstack/project-config when it merged at 23:24:22 which was right before i looked again | 00:03 |
| *** harrymichal has quit IRC | 00:04 | |
| fungi | so maybe we were in some transient incomplete state or something which the merge of that ended up resetting | 00:10 |
| *** tosky has quit IRC | 00:17 | |
| corvus | yeah. that sounds plausible, but i don't immediately see the mechanism for that happening. | 00:18 |
| corvus | i don't think i'm going to pull on that thread right now, but i'll keep it in mind as we work through the config-in-zk changes | 00:18 |
| fungi | yep, i'm really not sure how to work out at what point we got into that state, but will keep an eye out for recurrences in case there's some sort of pattern to it | 00:24 |
| openstackgerrit | Merged zuul/zuul master: Prune git tags on fetch https://review.opendev.org/c/zuul/zuul/+/761756 | 00:37 |
| *** hamalq has quit IRC | 00:41 | |
| *** rlandy has quit IRC | 00:43 | |
| *** ikhan has quit IRC | 00:52 | |
| *** ikhan has joined #zuul | 00:56 | |
| *** iurygregory has quit IRC | 01:07 | |
| *** saneax has quit IRC | 01:16 | |
| *** iurygregory has joined #zuul | 01:30 | |
| *** ikhan has quit IRC | 01:54 | |
| *** vishalmanchanda has joined #zuul | 03:45 | |
| *** saneax has joined #zuul | 04:03 | |
| *** Tahvok has quit IRC | 04:03 | |
| *** ykarel|away has joined #zuul | 04:09 | |
| *** Tahvok has joined #zuul | 04:14 | |
| *** ykarel|away is now known as ykarel | 04:14 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** jfoufas1 has joined #zuul | 05:45 | |
| *** piotrowskim has joined #zuul | 07:02 | |
| *** ykarel_ has joined #zuul | 07:27 | |
| *** ykarel has quit IRC | 07:30 | |
| *** ykarel_ is now known as ykarel | 07:51 | |
| *** jcapitao has joined #zuul | 07:52 | |
| openstackgerrit | Dong Zhang proposed zuul/zuul master: Display branch of queue in status page https://review.opendev.org/c/zuul/zuul/+/777613 | 08:05 |
| *** nils has joined #zuul | 08:09 | |
| *** rpittau|afk is now known as rpittau | 08:22 | |
| *** tosky has joined #zuul | 08:35 | |
| *** harrymichal has joined #zuul | 08:46 | |
| *** hashar has joined #zuul | 08:56 | |
| *** jpena|off is now known as jpena | 08:57 | |
| *** SaifAddin has joined #zuul | 08:59 | |
| openstackgerrit | Dong Zhang proposed zuul/zuul master: Display branch of queue in status page https://review.opendev.org/c/zuul/zuul/+/777613 | 09:33 |
| *** piotrowskim has quit IRC | 09:44 | |
| *** jangutter has joined #zuul | 10:41 | |
| *** jangutter has quit IRC | 10:42 | |
| *** jangutter has joined #zuul | 10:43 | |
| *** jangutte_ has quit IRC | 10:44 | |
| *** jangutter_ has joined #zuul | 11:30 | |
| *** jangutter has quit IRC | 11:33 | |
| *** saneax has quit IRC | 12:00 | |
| *** jcapitao is now known as jcapitao_lunch | 12:03 | |
| *** saneax has joined #zuul | 12:20 | |
| *** jpena is now known as jpena|lunch | 12:31 | |
| *** rlandy has joined #zuul | 12:39 | |
| *** jcapitao_lunch is now known as jcapitao | 13:00 | |
| *** rlandy has quit IRC | 13:01 | |
| *** rlandy has joined #zuul | 13:02 | |
| *** rlandy is now known as rlandy|mtg | 13:12 | |
| tobiash | corvus: I've posted a question on 673840 (Optionally allow zoned executors to process unzoned jobs) | 13:27 |
| *** harrymichal has quit IRC | 13:29 | |
| *** jpena|lunch is now known as jpena | 13:33 | |
| *** jangutter has joined #zuul | 13:33 | |
| *** jangutter_ has quit IRC | 13:37 | |
| *** rlandy|mtg is now known as rlandy | 13:56 | |
| corvus | tobiash: thx will fix | 14:09 |
| corvus | tobiash: 2 questions: | 14:10 |
| corvus | 1) in circular deps, does the entire bundle run with the same merged zuul configuration of every item of the bundle (ie, if A<->B, do both A and B run with .zuul.yaml from A+B)? | 14:10 |
| corvus | 2) global repo state doesn't depend on the circular deps change; when we merge it, will it dtrt with circular deps? | 14:12 |
| *** jangutter has quit IRC | 14:16 | |
| *** jangutter has joined #zuul | 14:16 | |
| *** fdegir has joined #zuul | 14:19 | |
| *** jangutter_ has joined #zuul | 14:38 | |
| tobiash | corvus: we don't run global repo state yet in prod so I think we should rebase it on top of it. Regarding 1): every item in the bundle will do its own merge but with a stable sorting of the dependencies | 14:38 |
| tobiash | corvus: so yes I think global repo state could differ between the items based on the target branch state during initial merge | 14:39 |
| tobiash | corvus: so global repo state only enforces the whole repo state within an item | 14:40 |
| *** jangutter has quit IRC | 14:41 | |
| tobiash | the question is, do we accept this at first as an improvement with todo to cover cdep or do we want to add cdep handling within global repo state? | 14:41 |
| tobiash | one solution could be to deduplicate the initial merge in a cycle | 14:42 |
| corvus | tobiash: that actually doesn't sound that concerning to me. i agree that if the entire bundle had the same state for required-projects it would be better, but i don't think it's wrong for different items in the bundle to have different states for required projects; as long as they have the same state for all the items in the pipeline. | 14:42 |
| Open10K8S | Hi team. How could the commit be merged with not merged depends-on commit? https://review.opendev.org/c/openstack/openstack-helm/+/777983 has depends-on https://review.opendev.org/openstack/openstack-helm-infra/777980. Any policy changed OR I was wrong till now? | 14:43 |
| tobiash | I think in most cases this will be true but for cycles I think global repo state would be not guaranteed over all those items | 14:43 |
| corvus | tobiash: and regarding 1 -- with your answer and another look at the code, it looks like that does what i would expect; i'm happy there. | 14:44 |
| corvus | tobiash: hrm, i don't understand the last thing you wrote | 14:44 |
| tobiash | forget my last sentence | 14:45 |
| corvus | ok | 14:45 |
| tobiash | corvus: let me rephrase. I think the initial merge which is done prior to job freezing is done independently for each item within the cycle but with a deterministic ordering of the items | 14:46 |
| tobiash | so the repo state of repos that are not part of the dependency queue can be different between the items | 14:47 |
| corvus | tobiash: that makes sense; here is my own attempt to summarize: with cdep we expect every item in a bundle to have the same repo state for all of the projects in the pipeline ahead of it plus the projects in the bundle. with the global state change, we also fix the required-projects state for every project involved, but the state of required-projects may vary between items. | 14:48 |
| tobiash | correct | 14:48 |
| corvus | ok. sounds good to me. i think we could improve that but i don't think it's "wrong". | 14:49 |
| tobiash | and if we want to improve the last part we would need to put deduplicaton of the merges on top | 14:49 |
| tobiash | ++ | 14:49 |
| Open10K8S | corvus: tobiash: anyway, depends-on is working in CI but not freezing now, you mean? | 14:51 |
| corvus | tobiash: i was thinking there might be a moderate performance improvement if we send out our "best guess" for what the extra merge jobs will be to get the repo state during the initial merge phase (right now, the change does the "extra" repos after the merge of the primary repos). our best guess will usually be right (unless there's a config change). so then the later merge would only need to happen if a | 14:51 |
| corvus | new project was added. | 14:51 |
| corvus | Open10K8S: we're discussing changes for new features in review | 14:51 |
| corvus | Open10K8S: 685354 and 738603 to be specific | 14:52 |
| tobiash | corvus: how would you generate the best guess? | 14:52 |
| corvus | Open10K8S: regarding your question, the depends-on link in 777983 is invalid (try following it) | 14:53 |
| corvus | tobiash: from the current layout (before we get the potentially updated dynamic layout) | 14:53 |
| tobiash | ok, that makes sense | 14:55 |
| corvus | tobiash: i suppose that would come with an extra job freeze | 14:55 |
| corvus | tobiash: so it might be a trade off with adding a little more cpu up front in order to save some time waiting on merge jobs | 14:55 |
| corvus | tobiash: i don't think it's something we have to do now; maybe an optimization to keep in mind if it starts to feel sluggish | 14:56 |
| tobiash | yeah | 14:57 |
| tobiash | atm our bottleneck is scheduler cpu actually :/ | 14:57 |
| corvus | yeah, i think there are some things that are going to be more attractive after 5.0 :) | 14:57 |
| tobiash | at some point before 5.0 we might need to dig into optimizing job freezing one more time | 14:58 |
| tobiash | I think we still had an issue that the schema things are slow to instanciate | 14:58 |
| tobiash | but we need to profile this again | 14:58 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Optionally allow zoned executors to process unzoned jobs https://review.opendev.org/c/zuul/zuul/+/673840 | 15:02 |
| corvus | tristanC: are you interested in reviewing "--validate-tenants" option? https://review.opendev.org/542160 i don't see that you've reviewed it before. i don't know if that would be useful to you or not but i figured you should know about it. | 15:06 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: web UI: user login with OpenID Connect https://review.opendev.org/c/zuul/zuul/+/734082 | 15:07 |
| *** ykarel has quit IRC | 15:17 | |
| tobiash | corvus: commented on 673840 | 15:26 |
| corvus | tobiash: ah yep. those tests are going to take a long time to time out :) | 15:30 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Optionally allow zoned executors to process unzoned jobs https://review.opendev.org/c/zuul/zuul/+/673840 | 15:30 |
| *** openstackgerrit has quit IRC | 15:35 | |
| *** wuchunyang has joined #zuul | 16:08 | |
| *** wuchunyang has quit IRC | 16:12 | |
| *** openstackgerrit has joined #zuul | 16:19 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add authentication-realm attribute to tenants https://review.opendev.org/c/zuul/zuul/+/735586 | 16:19 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to dequeue a change https://review.opendev.org/c/zuul/zuul/+/734850 | 16:19 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: web UI: allow a privileged user to re-enqueue a change https://review.opendev.org/c/zuul/zuul/+/736772 | 16:19 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Web UI: allow a privileged user to request autohold https://review.opendev.org/c/zuul/zuul/+/768115 | 16:20 |
| *** jfoufas1 has quit IRC | 16:25 | |
| tobiash | zuul-maint: I'd appreciate a review on a small change that adds support for the yaml merge operator ('<<'): https://review.opendev.org/c/zuul/zuul/+/763532 | 16:33 |
| avass | tobiash: now if yaml only supported merging lists as well :) | 16:40 |
| fungi | zbr: i'm curious, why are you adding me as a requested reviewer on changes like that? ^ i also pay attention in irc | 16:49 |
| *** sduthil has quit IRC | 16:50 | |
| *** sduthil has joined #zuul | 16:51 | |
| zbr | fungi: because you are one of the few that can made a decision on that, irc channel should not be seen an implied/required part of notifying possible reviewers. | 16:51 |
| zbr | or to rephrase, is my way of saying that I do think that this review worth your attention | 16:52 |
| fungi | we have 13 (last time i counted) zuul maintainers, so "few" is relative | 16:52 |
| corvus | and they were all pinged by tobiash with zuul-maint | 16:52 |
| corvus | and i have confidence that every one of them can make good decisions on a change like that | 16:53 |
| fungi | i appreciate that you think so highly of me, but this is basically like when you were previously "assigning" reviews to me (and others) which i also asked you to stop doing, at least until we worked out how to disable that "feature" | 16:53 |
| corvus | (there are centainly changes that suggest reviews from folks with particular areas of expertise, but that isn't one of them) | 16:53 |
| zbr | it is possible to disable add reviewer feature, but i doubt it would send the right signal other contributors | 16:55 |
| fungi | it's not a huge nuisance in my case, because i don't rely on gerrit's default dashboard and i heavily filter its e-mail notifications already, but generally adding reviewers to a change is somewhere between ineffective and annoying depending on the reviewer | 16:56 |
| zbr | apparently there is a very low chance for a review to get noticed unless manual pinging is done either by ird or adding someone as reviewer. | 16:59 |
| *** sshnaidm is now known as sshnaidm|afk | 17:00 | |
| fungi | i generally don't notice when people add me as a requested reviewer in gerrit, because i'm already a reviewer on so many changes it gets lost in the noise | 17:00 |
| zbr | i am now aware of a guidance against adding reviewers, and I personally find it less intrusive than manual mentions on irc | 17:01 |
| fungi | (in this particular case it was pointed out to me by another reviewer that i had been added) | 17:01 |
| corvus | that was me; i try to make sure everyone's review comments are addressed before merging a change, and i could not find fungi's review comments, so i asked why he was a reviewer | 17:02 |
| corvus | i found the situation confusing | 17:02 |
| corvus | i asked clarkb the same thing | 17:03 |
| zbr | i will keep in mind to not add extra reviewers for zuul project changes i look at, that is something new for me as on tripleo it was required. | 17:06 |
| fungi | thanks for understanding. different communities with different review volumes and different team sizes are going to have varied norms | 17:08 |
| *** saneax has quit IRC | 17:18 | |
| *** rlandy is now known as rlandy|lunch | 17:23 | |
| *** jcapitao has quit IRC | 17:25 | |
| *** rpittau is now known as rpittau|afk | 17:42 | |
| *** rlandy|lunch is now known as rlandy|brb | 17:58 | |
| *** jpena is now known as jpena|off | 18:02 | |
| openstackgerrit | Merged zuul/zuul master: Add --validate-tenants option to zuul scheduler https://review.opendev.org/c/zuul/zuul/+/542160 | 18:06 |
| openstackgerrit | Merged zuul/zuul master: Handle the yaml merge operator https://review.opendev.org/c/zuul/zuul/+/763532 | 18:06 |
| *** harrymichal has joined #zuul | 18:18 | |
| *** hashar has quit IRC | 18:21 | |
| pabelanger | working on enabling TLS on zookeeper. I'm guessing it is a hard cut-over, or can zookeeper be configured to accept both tls / non-tls? | 18:26 |
| pabelanger | so I can do say nodepool first, then zook | 18:27 |
| pabelanger | err | 18:27 |
| pabelanger | zuul | 18:27 |
| corvus | pabelanger: can do both | 18:27 |
| pabelanger | tyty | 18:28 |
| corvus | pabelanger: there are separate lines in the zoo.cfg file for each, and they're on different ports | 18:29 |
| clarkb | pabelanger: in fact we continue to confogure it to allow both because it makes local interaction much simpler | 18:29 |
| clarkb | the firewall prevents external access via not ssl, but admins can interact with the server easily this way (we may end up changing it though not sure yet) | 18:29 |
| pabelanger | great idea | 18:29 |
| corvus | yeah, i think that's okay as long as we firewall it down to just the zk servers | 18:29 |
| corvus | definitely don't allow non-tls connections from the executors :) | 18:30 |
| *** rlandy|brb is now known as rlandy | 18:37 | |
| *** hamalq has joined #zuul | 18:38 | |
| *** nils has quit IRC | 19:07 | |
| *** hashar has joined #zuul | 20:30 | |
| *** ikhan has joined #zuul | 20:50 | |
| *** bhagyashri|rover has quit IRC | 21:06 | |
| *** bhagyashris has joined #zuul | 21:07 | |
| ianw | corvus: not sure if you saw https://groups.google.com/g/repo-discuss/c/_oFBJXBLv8s/m/-E9kcxpjAwAJ about the checks api | 21:16 |
| *** SaifAddin has quit IRC | 21:17 | |
| ianw | i was wondering what our thoughts are around that being declared stable v the zuul plugin, which is receiving some work to be configurable and have a rest interface etc. | 21:17 |
| ianw | https://sourcegraph.com/github.com/GerritCodeReview/gerrit/-/blob/polygerrit-ui/app/api/checks.ts is probably the most useful reference | 21:17 |
| ianw | it feels like there's a bit of a mis-match there, but not so much it couldn't be papered over. you've have to know what tenant to query; and integrating the status of running jobs v completed would be ... interesting | 21:20 |
| corvus | ianw: unfortunately, i haven't had a chance to really study it :( | 21:32 |
| tristanC | could we provide a new /change endpoint that provides running and completed build for a given (change, patchset) tuple? | 21:32 |
| tristanC | create* a new /change endpoint | 21:33 |
| ianw | tristanC: yeah, otherwise running builds would seem tohave to query the status.json | 21:35 |
| corvus | tristanC, ianw: i suspect we're going to need a new endpoint, probably provided by the gerrit driver, to return data specifically for this | 21:35 |
| corvus | either that, or encode a lot of mapping logic in js, right? | 21:36 |
| corvus | at a high level, we need something that turns buildsets into check runs; that could either be in python code in a new gerrit-zuul-driver endpoint, or in java/js in the gerrit zuul plugin? | 21:37 |
| corvus | the tenant mapping is the thing i'd be most concerned about | 21:38 |
| tristanC | corvus: or perhaps we could re-use the /status/change/ endpoint to also return completed build | 21:39 |
| corvus | tristanC: yeah could be an option | 21:39 |
| ianw | yeah, i was thinking that would probably require something like what is being written for the zuul-summary plugin, where you set a config option for projects giving their URL/tenant | 21:39 |
| corvus | ianw: yeah i think that might be the way | 21:39 |
| corvus | ianw: maybe a list for that value | 21:39 |
| corvus | ie project -> [url/tenant, url/tenant] | 21:39 |
| tristanC | corvus: instead of a list, could the url either be whitelabel or global, and for later we could return build for all tenants? | 21:40 |
| corvus | example use case: plugins/zuul may want to have checks from one tenant on ci.gerritcodereview.com and two tenants on opendev.org | 21:40 |
| corvus | tristanC: i don't think so -- tenants are meant to be completely separate installations and data shouldn't bleed between them | 21:41 |
| ianw | (https://gerrit-review.googlesource.com/c/plugins/zuul-results-summary/+/298465 is what i'm referring to for reference) | 21:41 |
| ianw | anyway, seeing as there was a offer to help build the front-end on the list, i can take an action item to follow up on that in a thread if we'd like | 21:42 |
| tristanC | corvus: but if you have a global /api access, then that's just a convenience over querying each tenant scoped endpoint | 21:42 |
| corvus | tristanC: there are [almost] no global /api endpoints | 21:43 |
| corvus | tristanC: i think you're suggesting we implement a new global api endpoint in zuul to perform the convience of repeating the query on each tenant? | 21:44 |
| tristanC | corvus: e.g. https://zuul.opendev.org/api | 21:44 |
| corvus | right, almost everything there is tenant-specific | 21:45 |
| openstackgerrit | lotorev vitaly proposed zuul/zuul master: Fix artifacts example in documentation https://review.opendev.org/c/zuul/zuul/+/778238 | 21:45 |
| corvus | tristanC: i think we should stick with our "tenants are separate" model here. i think the zuul plugin is going to need to handle multiple installations regardless (opendev's zuul and gerrit's zuul are two separate systems which will want to report on the same project) so configuring another tenant should be about the same | 21:46 |
| corvus | ianw: have you seen any suggestion about how systems are supposed to decide to run jobs? | 21:47 |
| corvus | (previously, gerrit was responsible for creating check run instances and you could query the api to find check runs that had not started yet) | 21:47 |
| tristanC | corvus: alright, i guess we'll have to manually configure all the tenant which run jobs for a given gerrit | 21:48 |
| corvus | tristanC: if we wanted to have the checks plugin iterate over all the tenants, that might be fine. i'm just really wary of publishing a zuul api model that rolls up tenants | 21:51 |
| tristanC | corvus: that makes sense, but the list can be long then | 21:53 |
| corvus | we can do both; have the checks plugin iterate if you give it a global url, and if you don't want it to, give it a tenant url | 21:54 |
| corvus | "Copyright (C) 2020 The Android Open Source Settings" looks like a search/replace mixup :) | 21:54 |
| corvus | ianw: is it possible the backend side of this is staying the same? | 21:56 |
| corvus | if they are keeping that, then that might help with the tenant mapping | 21:58 |
| corvus | i think i need a summary of how the whole system (not just the ui layer) is expected to wokr | 21:58 |
| corvus | work | 21:58 |
| ianw | corvus: hrm, i don't think this is responsible for triggering jobs? it's only querying? | 22:03 |
| corvus | "Gerrit requests check runs and results from the plugin by change number and patchset number." makes me think no. which means the conversation we just had is relevant. but it also makes me wonder how they're going to handle triggering jobs. | 22:03 |
| corvus | ianw: yeah | 22:03 |
| corvus | so if we wanted to use this on opendev, we use stream-events to trigger jobs, then query some api by change/patchset to get check runs (buildsets or builds) and results. | 22:05 |
| corvus | on gerrit's zuul we use (???) to trigger jobs. :) | 22:05 |
| *** ikhan has quit IRC | 22:05 | |
| corvus | regarding builds/buildsets -- we could consider mapping a checkrun to either. the main reason we could only do buildsets in the orig checks api was because they had to be defined in advance, but since we can return them after zuul calculates the jobs, we could return builds. if we use builds, the organizing them (by pipeline) is going to be tricky. if we do that, we may have to prefix each build name with | 22:08 |
| corvus | the pipeline name. | 22:08 |
| *** vishalmanchanda has quit IRC | 22:54 | |
| *** harrymichal has quit IRC | 23:01 | |
| *** hashar has quit IRC | 23:10 | |
| *** ikhan has joined #zuul | 23:22 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!