openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 00:16 |
---|---|---|
*** brinzhang has joined #opendev | 00:22 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: refstack: Edit URL of public RefStackAPI https://review.opendev.org/c/opendev/system-config/+/776292 | 00:28 |
*** tosky has quit IRC | 00:36 | |
*** avass has quit IRC | 00:41 | |
*** yourname has joined #opendev | 00:42 | |
*** mlavalle has quit IRC | 01:27 | |
*** iurygregory_ has joined #opendev | 01:30 | |
*** mattmceuen_ has joined #opendev | 01:31 | |
*** dviroel_ has joined #opendev | 01:31 | |
*** iurygregory has quit IRC | 01:38 | |
*** mordred has quit IRC | 01:38 | |
*** dviroel has quit IRC | 01:38 | |
*** ttx has quit IRC | 01:38 | |
*** mattmceuen has quit IRC | 01:38 | |
*** dviroel_ is now known as dviroel | 01:39 | |
*** mattmceuen_ is now known as mattmceuen | 01:39 | |
*** ttx has joined #opendev | 01:47 | |
*** mordred has joined #opendev | 01:54 | |
*** hamalq has quit IRC | 02:14 | |
*** ysandeep|away is now known as ysandeep | 03:04 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: refstack: Edit URL of public RefStackAPI https://review.opendev.org/c/opendev/system-config/+/776292 | 03:09 |
*** ysandeep is now known as ysandeep|afk | 03:50 | |
*** stevebaker has quit IRC | 03:58 | |
*** stevebaker has joined #opendev | 04:02 | |
*** ykarel has joined #opendev | 04:16 | |
*** ysandeep|afk is now known as ysandeep | 05:20 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 05:29 |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 05:35 |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 06:00 |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 06:02 |
frickler | seems ovh is having some major issues, but not at the sites we're using. good luck to them anyway http://travaux.ovh.net/?do=details&id=49484& | 06:18 |
*** whoami-rajat has joined #opendev | 06:23 | |
*** marios has joined #opendev | 06:28 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 06:42 |
frickler | ouch https://twitter.com/abonin_DNA/status/1369538028243456000 also, did they really just put up a huge stack of shipping containers there? | 06:57 |
ianw | yeah :( | 07:02 |
*** ralonsoh has joined #opendev | 07:21 | |
*** slaweq has quit IRC | 07:42 | |
*** eolivare has joined #opendev | 07:46 | |
*** slaweq has joined #opendev | 07:46 | |
*** yourname is now known as avass | 07:48 | |
*** slaweq has quit IRC | 07:52 | |
*** sboyron has joined #opendev | 07:53 | |
*** slaweq has joined #opendev | 07:54 | |
*** ysandeep is now known as ysandeep|lunch | 07:58 | |
*** fressi has joined #opendev | 08:01 | |
*** lpetrut has joined #opendev | 08:06 | |
*** hashar has joined #opendev | 08:15 | |
*** brinzhang_ has joined #opendev | 08:20 | |
*** gothicserpent has quit IRC | 08:21 | |
*** brinzhang has quit IRC | 08:23 | |
*** gothicserpent has joined #opendev | 08:24 | |
*** arxcruz|off is now known as arxcruz | 08:24 | |
*** rpittau|afk is now known as rpittau | 08:27 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 08:30 |
*** ysandeep|lunch is now known as ysandeep | 08:37 | |
*** DSpider has joined #opendev | 08:43 | |
*** tosky has joined #opendev | 08:46 | |
*** hashar has quit IRC | 08:55 | |
*** jpena|off is now known as jpena | 08:58 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 09:00 |
*** andrewbonney has joined #opendev | 09:03 | |
*** hashar has joined #opendev | 09:04 | |
openstackgerrit | Benjamin Schanzel proposed zuul/zuul-jobs master: ensure-helm: skip helm init for helm v3 and newer https://review.opendev.org/c/zuul/zuul-jobs/+/779698 | 09:39 |
*** ykarel is now known as ykarel|lunch | 10:00 | |
openstackgerrit | Benjamin Schanzel proposed zuul/zuul-jobs master: ensure-helm: skip helm init for helm v3 and newer https://review.opendev.org/c/zuul/zuul-jobs/+/779698 | 10:03 |
*** DSpider has quit IRC | 10:07 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 10:34 |
*** Eighth_Doctor has quit IRC | 10:36 | |
*** mordred has quit IRC | 10:36 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 10:39 |
*** klonn has joined #opendev | 10:41 | |
*** Eighth_Doctor has joined #opendev | 10:46 | |
*** ysandeep is now known as ysandeep|away | 10:50 | |
*** hashar has quit IRC | 10:53 | |
*** mordred has joined #opendev | 11:03 | |
*** dtantsur|afk is now known as dtantsur | 11:07 | |
*** ysandeep|away is now known as ysandeep | 11:09 | |
*** toomer has joined #opendev | 11:15 | |
*** ykarel|lunch is now known as ykarel | 11:18 | |
*** iurygregory_ is now known as iurygregory | 11:33 | |
openstackgerrit | Moshiur Rahman proposed openstack/diskimage-builder master: Fix: IPA image buidling with OpenSuse. https://review.opendev.org/c/openstack/diskimage-builder/+/778723 | 11:56 |
*** eolivare_ has joined #opendev | 12:16 | |
*** ykarel_ has joined #opendev | 12:17 | |
*** eolivare has quit IRC | 12:19 | |
*** ykarel has quit IRC | 12:20 | |
*** jpena is now known as jpena|lunch | 12:34 | |
*** klonn has quit IRC | 13:05 | |
openstackgerrit | Sorin Sbârnea proposed zuul/zuul-jobs master: Bits to keep ansible-lint happy https://review.opendev.org/c/zuul/zuul-jobs/+/779276 | 13:11 |
*** ykarel_ is now known as ykarel | 13:15 | |
*** brinzhang_ has quit IRC | 13:23 | |
*** brinzhang_ has joined #opendev | 13:24 | |
priteau | Hello. FYI we just had an error in the promote-openstack-releasenotes job in Kayobe. | 13:38 |
priteau | rsync: close failed on "/afs/.openstack.org/docs/releasenotes/kayobe/_static/css/.font-awesome.css.wrAMwd": Input/output error (5) | 13:38 |
*** jpena|lunch is now known as jpena | 13:40 | |
*** klonn has joined #opendev | 13:44 | |
priteau | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_64d/775936/4/promote/promote-openstack-releasenotes/64d8728/job-output.txt | 13:47 |
*** klonn has quit IRC | 13:49 | |
*** auristor has quit IRC | 14:03 | |
*** auristor has joined #opendev | 14:06 | |
*** jrollen is now known as jroll | 14:13 | |
fungi | priteau: looks like that ran from ze05, i've checked out the executor and it doesn't seem to have any persistent issues communicating with the afs infrastructure. it's more likely that build raced another build of the same job and one rsync tried to remove the tempfiles the other rsync was creating. i'll see if i can find stronger evidence | 14:38 |
priteau | There were multiple Kayobe patches merging at the same time so it's plausible | 14:39 |
fungi | usually that happens if there are changes merging for different branches of the same project at the same time, since the release notes all want to write to the same unversioned place, and we only supersede promote refs when they're for the same branch | 14:39 |
priteau | Yep, makes sense | 14:39 |
priteau | It's very likely this then, no need to dig more | 14:39 |
fungi | well, the up side is that the next releasenotes build to run will update it all again anyway | 14:40 |
*** ysandeep is now known as ysandeep|away | 15:11 | |
*** toomer has quit IRC | 15:37 | |
openstackgerrit | Merged openstack/project-config master: [magnum] Add Backport-Candidate and Review-Priority labels https://review.opendev.org/c/openstack/project-config/+/778629 | 15:43 |
*** lpetrut has quit IRC | 15:44 | |
*** klonn has joined #opendev | 15:48 | |
*** dmsimard8 has joined #opendev | 15:52 | |
*** dmsimard has quit IRC | 15:54 | |
*** dmsimard8 is now known as dmsimard | 15:54 | |
openstackgerrit | Sofia Enriquez proposed opendev/irc-meetings master: Add Cinder bug squad meeting https://review.opendev.org/c/opendev/irc-meetings/+/779818 | 15:54 |
*** toomer has joined #opendev | 15:57 | |
*** ysandeep|away is now known as ysandeep | 16:08 | |
*** marios is now known as marios|out | 16:37 | |
*** marios|out has quit IRC | 16:43 | |
*** hashar has joined #opendev | 16:54 | |
*** toomer has quit IRC | 17:04 | |
*** rpittau is now known as rpittau|afk | 17:05 | |
*** mlavalle has joined #opendev | 17:06 | |
*** dtantsur is now known as dtantsur|afk | 17:15 | |
openstackgerrit | Antoine Musso proposed opendev/gear master: wakeConnections: Randomize connections before scanning them https://review.opendev.org/c/opendev/gear/+/747119 | 17:15 |
hashar | hi, is there still any interest in opendev/gear ? There are a few pending changes that seems straightforward :] | 17:19 |
fungi | hashar: it's seemed at least close to "feature-complete" for some years, but if there are fixes or in-scope improvements then we'll try to review them | 17:20 |
fungi | though also its importance to us may shrink with zuul 5.0.0 and the distributed scheduler work moving that coordination into zookeeper | 17:22 |
hashar | there are a few trivial ones though that should not take too much time. The one I hhave rebased above better balances the workload among worker and addressed some issue we had on our setup ;) | 17:22 |
fungi | i'll try to take a look after i get out of my current meeting | 17:22 |
hashar | there are a few others that sounds trivial enough, I have +1 ed anything that looked sane/straightforward | 17:23 |
hashar | some others are a bit more challenging I guess or super old. So I guess they will end up being abandoned eventually | 17:23 |
fungi | thanks for the heads up, and for going through the open reviews! | 17:24 |
fungi | that's a big help | 17:24 |
hashar | I felt like if I need my change approved, I might as well spend sometime reviewing others patches ;≠ | 17:24 |
hashar | happy meeting fungi! | 17:24 |
fungi | same to you! | 17:25 |
hashar | (the open patches I have +1ed https://review.opendev.org/q/project:opendev/gear+is:open+label:Code-Review%252B1%252Chashar ) | 17:27 |
*** akrpan-pure has joined #opendev | 17:40 | |
*** klonn has quit IRC | 17:45 | |
*** ykarel has quit IRC | 17:50 | |
*** jpena is now known as jpena|off | 17:58 | |
*** SWAT has quit IRC | 17:58 | |
*** ysandeep is now known as ysandeep|holiday | 18:03 | |
openstackgerrit | Sorin Sbârnea proposed zuul/zuul-jobs master: Bits to keep ansible-lint happy https://review.opendev.org/c/zuul/zuul-jobs/+/779276 | 18:04 |
*** SWAT has joined #opendev | 18:11 | |
corvus | clarkb: what's your zuul component restart status? | 18:14 |
fungi | all mergers and executors have been rebuilt with ubuntu focal servers, and the old ones cleaned up. he's working on nodepool launchers next, then builders i think? | 18:17 |
*** klonn has joined #opendev | 18:17 | |
fungi | scheduler was going last | 18:18 |
fungi | oh, also zk servers are in there somewhere | 18:18 |
fungi | they're not done yet | 18:18 |
corvus | ok; i'd like to restart all of zuul today if conditions warrant; i'm thinking after my lunch based on the node request graph | 18:19 |
corvus | we merged a whole pile of changes and it'd be good to exercise them in prod | 18:19 |
*** andrewbonney has quit IRC | 18:19 | |
fungi | sounds good to me. he and i are in a meeting which is running over | 18:20 |
fungi | but i'll be freed up soon | 18:20 |
clarkb | ya should be fine | 18:20 |
corvus | cool, i'll check back around 2100 | 18:21 |
clarkb | next up is launchers I want to see if I can get https://review.opendev.org/c/zuul/nodepool/+/779616 in to make that easier | 18:21 |
clarkb | my plan is land that change, restart current launchers on it and make sure it is happy. Then build a new nl01.opendev.org and do a switchover | 18:21 |
clarkb | builders are already rebuilt. After launchers is zk or scheduler. I haven't sorted out which I'd prefer there yet (though suspect it is zk) | 18:22 |
*** akrpan-pure has quit IRC | 18:22 | |
corvus | clarkb: +2 from me; maybe if someone +3s it soon we can roll that into a restart this afternoon | 18:23 |
clarkb | corvus: ok. I can also do it later without much trouble so don't hold up restarts on that | 18:23 |
corvus | k | 18:24 |
clarkb | that user that ended up in a confused state with gerrit accounts has confirmed that the account was never properly used for anything and they deleted the corresponding ubuntu one account. That means we should be able to retire it and remove all email external ids so that they can start over again. | 18:27 |
clarkb | is any other infra-root interested in working through that process? We've got all the tooling for the preexisting account cleanups ready to go and thought it might be useful to have someone else work through one too | 18:28 |
clarkb | might also help grow confidence in the tooling (then we can land that change?) | 18:32 |
*** eolivare_ has quit IRC | 18:36 | |
*** tbarron has joined #opendev | 18:37 | |
*** ralonsoh has quit IRC | 18:41 | |
*** SWAT has quit IRC | 18:42 | |
*** SWAT has joined #opendev | 18:45 | |
openstackgerrit | Merged opendev/gear master: Modify connection timeout process https://review.opendev.org/c/opendev/gear/+/531059 | 19:05 |
clarkb | re the gerrit user situation, I don't think gerrit got this entirely wrong in this case. The user deleted the source account and tried again. THis is why the email addrs didn't change but the openid did | 19:09 |
clarkb | however, it definitely would be nice if gerrit had an easy mode way to say "this external id belongs to this account" | 19:10 |
clarkb | I suppose we get half easy once the other external id conflicts are sorted and can push to gerrit for this stuff | 19:10 |
openstackgerrit | Merged opendev/gear master: remove unicode from code https://review.opendev.org/c/opendev/gear/+/769671 | 19:26 |
*** brinzhang0 has joined #opendev | 19:33 | |
*** redrobot2 has joined #opendev | 19:34 | |
*** Dmitrii-Sh4 has joined #opendev | 19:34 | |
*** dpawlik6 has joined #opendev | 19:34 | |
*** stevebaker_ has joined #opendev | 19:36 | |
*** mugsie__ has joined #opendev | 19:37 | |
*** tosky_ has joined #opendev | 19:37 | |
*** whoami-rajat has quit IRC | 19:40 | |
openstackgerrit | Merged opendev/gear master: Move handleDisconnect into BaseClientServer https://review.opendev.org/c/opendev/gear/+/714709 | 19:41 |
openstackgerrit | Merged opendev/gear master: wakeConnections: Randomize connections before scanning them https://review.opendev.org/c/opendev/gear/+/747119 | 19:41 |
*** paladox_ has joined #opendev | 19:41 | |
*** artom has quit IRC | 19:42 | |
*** SWAT has quit IRC | 19:42 | |
*** dmsimard has quit IRC | 19:42 | |
*** brinzhang_ has quit IRC | 19:42 | |
*** tosky has quit IRC | 19:42 | |
*** stevebaker has quit IRC | 19:42 | |
*** dpawlik has quit IRC | 19:42 | |
*** Dmitrii-Sh has quit IRC | 19:42 | |
*** paladox has quit IRC | 19:42 | |
*** mugsie has quit IRC | 19:42 | |
*** redrobot has quit IRC | 19:42 | |
*** dpawlik6 is now known as dpawlik | 19:42 | |
*** Dmitrii-Sh4 is now known as Dmitrii-Sh | 19:42 | |
*** dmsimard has joined #opendev | 19:43 | |
*** redrobot2 is now known as redrobot | 19:43 | |
*** irclogbot_0 has quit IRC | 19:44 | |
*** paladox_ is now known as paladox | 19:45 | |
*** mugsie__ is now known as mugsie | 19:47 | |
*** SWAT has joined #opendev | 19:48 | |
*** stevebaker_ is now known as stevebaker | 20:00 | |
*** artom has joined #opendev | 20:01 | |
*** tosky_ is now known as tosky | 20:16 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 20:23 |
openstackgerrit | Steve Baker proposed openstack/diskimage-builder master: Support secure-boot bootloader where possible https://review.opendev.org/c/openstack/diskimage-builder/+/779107 | 20:37 |
*** SWAT has quit IRC | 20:37 | |
*** SWAT has joined #opendev | 20:40 | |
openstackgerrit | Clark Boylan proposed openstack/project-config master: Startup nl01.opendev.org idle https://review.opendev.org/c/openstack/project-config/+/779863 | 20:46 |
openstackgerrit | Clark Boylan proposed openstack/project-config master: Flip nl01.openstack.org to nl01.opendev.org https://review.opendev.org/c/openstack/project-config/+/779864 | 20:46 |
clarkb | that second change I'll WIP now but I think that captures roughly how we can do the flip over once https://review.opendev.org/c/zuul/nodepool/+/779616 lands and a new instance is deployed | 20:46 |
clarkb | the first one should be safe to land whenever though I just have to wait on that for the new nl01.opendev.org | 20:47 |
clarkb | (as well as the nodepool change) | 20:47 |
*** tosky has quit IRC | 20:49 | |
*** tosky has joined #opendev | 20:49 | |
*** hashar has quit IRC | 20:52 | |
*** klonn has quit IRC | 20:57 | |
clarkb | re zuul restarts, I remembered that it is feature freeze week. Queues don't look terrible but they aren't super small either | 20:59 |
clarkb | it does look like a good chunk of changes should finish gating in the next 20 minutes though and that might be a good opportunity? | 20:59 |
*** mtreinish has quit IRC | 21:09 | |
fungi | i'm about to eat dinner, but am around to help | 21:10 |
*** mtreinish has joined #opendev | 21:11 | |
fungi | looks like we've still got a hefty node request backlog (>1k) | 21:12 |
fungi | and yeah, looking at the openstack gate pipeline graph, it's way shorter than it was a few hours ago | 21:14 |
*** hashar has joined #opendev | 21:14 | |
corvus | clarkb: do you want to veto the restart until next week? | 21:18 |
corvus | (or this weekend) | 21:20 |
fungi | clarkb: looks like those two integrated gate changes just merged if we want to do it now | 21:20 |
clarkb | corvus: I think its probably ok based on the size of the queues and their recent resets. However, I don't know what sorts of changes we want to validate (is it your zk work?) | 21:20 |
clarkb | corvus: I think feature freeze officially ends tomorrow so we could do it on friday maybe | 21:21 |
clarkb | (if we wanted to be more careful) | 21:21 |
clarkb | put another way I'd probably be more concerned that a restart will be buggy and prevent changes from merging than the delay in the queues right now. I just don't have a good way of evaluating the risk for that | 21:24 |
clarkb | though I am pulling up the git log now | 21:24 |
clarkb | looks like the big chagne is the cycle support? | 21:26 |
openstackgerrit | Steve Baker proposed openstack/diskimage-builder master: Support secure-boot bootloader where possible https://review.opendev.org/c/openstack/diskimage-builder/+/779107 | 21:28 |
clarkb | for the zk changes we can probably restart a single executor after asking it to gracefully stop and use that as a canary | 21:28 |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 21:30 |
corvus | clarkb: the cycle changes will be new to opendev but have seen use elsewhere; i think the global state has never been in production. so there's definitely a risk of error, and the restart is explicitly to surface errors. | 21:33 |
clarkb | corvus: 'Make repo state buildset global' that change? | 21:34 |
corvus | yep | 21:34 |
corvus | clarkb: however, depending on exactly when you restarted things, that may already be in [partial] production | 21:34 |
corvus | there is an executor/merger aspect to that change | 21:34 |
corvus | i'm unsure if it needs a scheduler restart to be activated though | 21:34 |
corvus | (at least, that's my recollection, i haven't looked) | 21:35 |
clarkb | corvus: I believe all mergers and executors were started on code prior to the merge timestamp for that change | 21:36 |
corvus | the node request backlog is higher than i expected; i'd be fine deferring until friday | 21:37 |
clarkb | the last zuul executor would've been started on march 8th local time and gerrit says that change landed march 9 morning local time | 21:37 |
clarkb | corvus: ya I think if we're looking for errors then waiting for friday is probably best. I'll dobule check on the feature freeze timing now too | 21:38 |
clarkb | I've asked the release team to confirm, but lets say friday for now | 21:41 |
*** TheJulia has quit IRC | 21:45 | |
*** TheJulia has joined #opendev | 21:45 | |
*** johnsom has quit IRC | 21:46 | |
*** johnsom has joined #opendev | 21:46 | |
openstackgerrit | Gage Hugo proposed openstack/project-config master: Add create ref acl for osh release groups https://review.opendev.org/c/openstack/project-config/+/779872 | 21:47 |
*** SWAT has quit IRC | 21:49 | |
*** SWAT has joined #opendev | 21:52 | |
*** slaweq has quit IRC | 21:55 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 22:11 |
openstackgerrit | Steve Baker proposed openstack/diskimage-builder master: Only add rhel base repos when REG_REPOS is not set https://review.opendev.org/c/openstack/diskimage-builder/+/779874 | 22:15 |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 22:34 |
*** hashar has quit IRC | 22:45 | |
*** sboyron has quit IRC | 22:56 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 23:18 |
*** rpittau|afk has quit IRC | 23:37 | |
*** rpittau|afk has joined #opendev | 23:37 | |
*** dmellado_ has joined #opendev | 23:47 | |
*** dmellado has quit IRC | 23:48 | |
*** dmellado_ is now known as dmellado | 23:48 | |
openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] kerberos ansible https://review.opendev.org/c/opendev/system-config/+/778840 | 23:56 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!