| Clark[m] | Ya I doubt they are updating that quickly but they will probably tell us | 00:53 |
|---|---|---|
| opendevreview | Merged openstack/project-config master: Retire os-win: end project gating https://review.opendev.org/c/openstack/project-config/+/894407 | 08:09 |
| opendevreview | Merged openstack/project-config master: Retire compute-hyperv: end project gating https://review.opendev.org/c/openstack/project-config/+/894408 | 08:10 |
| opendevreview | Merged openstack/project-config master: Retire networking-hyperv: end project gating https://review.opendev.org/c/openstack/project-config/+/894409 | 08:10 |
| opendevreview | Merged openstack/project-config master: Retire oswin-tempest-plugin: end project gating https://review.opendev.org/c/openstack/project-config/+/894410 | 08:11 |
| opendevreview | Bernhard Berg proposed zuul/zuul-jobs master: prepare-workspace-git: Add ability to define synced pojects https://review.opendev.org/c/zuul/zuul-jobs/+/887917 | 10:43 |
| *** amoralej is now known as amoralej|lunch | 11:03 | |
| opendevreview | Piotr Parczewski proposed openstack/diskimage-builder master: Include linux-firmware in Rocky linux element https://review.opendev.org/c/openstack/diskimage-builder/+/894905 | 11:19 |
| opendevreview | Alex Kavanagh proposed openstack/project-config master: Add the Pure Storage Flashblade charm-manila subordinate https://review.opendev.org/c/openstack/project-config/+/893910 | 12:06 |
| opendevreview | Mayank Patel proposed openstack/project-config master: Add node-interface-metrics-exporter app to StarlingX https://review.opendev.org/c/openstack/project-config/+/894910 | 12:13 |
| *** amoralej|lunch is now known as amoralej | 12:31 | |
| opendevreview | Matthias Runge proposed openstack/project-config master: Add python-observabilityclient https://review.opendev.org/c/openstack/project-config/+/894541 | 13:01 |
| opendevreview | Alex Kavanagh proposed openstack/project-config master: Add the Pure Storage Flashblade charm-manila subordinate https://review.opendev.org/c/openstack/project-config/+/893910 | 13:48 |
| Perdu | Hi, I'm following the Getting Started page to send change to a project. When running the 'git review -s' command for the first time, ssh prompts me to validate the fingerprint for '[review.opendev.org]:29418. Is this fingerprint written somewhere so that I can cross-check for security? | 14:16 |
| fungi | Perdu: excellent question! yes, we should add that to the documentation but it can be found in gerrit's webui. let me get you a direct link | 14:18 |
| fungi | mmm, or it used to be. just a sec | 14:20 |
| SvenKieske | Perdu: glad you found your way here :) I linked you to the getting started guide on github (@artificial-intelligence, over there) | 14:22 |
| SvenKieske | fwiw, that is what I got in my known_hosts file: [review.opendev.org]:29418 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIncSZ8k+c0YJTO6Wux/Kpu2jnktFkepoU3jBkbIzXve | 14:23 |
| fungi | yeah, we've got it published in git, but i'm trying to figure out what happened to the part of the gerrit webui that used to display it | 14:24 |
| SvenKieske | mhm https://review.opendev.org/ssh_info just has the port number.. | 14:26 |
| fungi | yeah, the settings page where you upload your ssh keys used to show the host key fingerprint right there too | 14:27 |
| fungi | i think that must have disappeared in a recent upgrade, so i'll probably need to hound gerrit upstream devs in their matrix/discord channel to get clarification. in the meantime i'll add something to the infra-manual | 14:28 |
| Perdu | Another missing step in the documentation is the necessity to select a contributor agreement (and what are these) | 14:29 |
| fungi | openstack's onboarding documentation covers that (not all projects hosted in opendev require an individual contributor license agreement, but official openinfra projects do for legal reasons, basically asserting that the patches you upload are your own work or you have permission of the copyright holder, e.g. your employer) | 14:30 |
| fungi | guilhermesp_____: have you heard anything about the routing loop in ca-ymq-1? | 14:31 |
| fungi | Perdu: for the cla reference, looks like maybe we could do with referring to the developer's guide chapter earlier in the quick setup guide, but this is the relevant bit: https://docs.opendev.org/opendev/infra-manual/latest/developers.html#openstack-individual-contributor-license-agreement | 14:35 |
| fungi | trying to find a good balance between "your setup documentation has too many steps, i'm not going to bother" and "your setup documentation omits important details i wanted more information on" | 14:36 |
| fungi | and also not duplicating too much information between chapters | 14:37 |
| SvenKieske | well, it clearly has too many steps :D I remember myself reading it the first time, some years ago and I needed multiple attempts to resolve everything :D | 14:37 |
| SvenKieske | but it _is_ fairly complete, tbf. and many steps would not get that simpler, even if we'd switch to different infra, like sometimes is suggested. | 14:38 |
| guilhermesp_____ | fungi: not yet sorry, im trying to find a slot to take a look at it today. Is this a blocker for you so we may raise the priority? | 14:42 |
| fungi | guilhermesp_____: nah, i went ahead and added a /etc/hosts entry on my workstation so that everything will just resolve exclusively ipv4 for the gerrit server so i don't have to wait for v6 connections to it to time out | 14:43 |
| fungi | i don't know how many other people are using my isp's ipv6 support (it's a fairly popular broadband isp in the usa, but what percentage of traffic that amounts to i couldn't guess) | 14:43 |
| fungi | i haven't heard from any of our users about problems connecting to our servers in ca-ymq-1 so it may be a very small number (perhaps just me) impacted | 14:44 |
| guilhermesp_____ | yeah ideally would be good for me to try and reproduce with my isp here | 14:45 |
| fungi | guilhermesp_____: probably you won't see any problems unless there are more routes missing than just the one for my provider | 14:45 |
| Perdu | fungi: You get an error when you run 'git review -s' and you haven't selected the CLA in gerrit's setting, so there should be a line about this | 14:46 |
| fungi | Perdu: right, it's worth thinking about how to cover that in the documentation since you'll only get that if you're trying to set up to contribute to a project that has cla enforcement (not all projects hosted in opendev do) | 14:46 |
| fungi | every additional "you might see this and if so do this or look here" we add to the quick start guide makes it more cumbersome for people that doesn't apply to | 14:47 |
| guilhermesp_____ | so fungi the idea here is that you are trying to reach the review02 and irror01.ca-ymq-1 ipv6 addrs from your local computer right? ping6 <ipv6> which is never giving you a reply? | 14:47 |
| Perdu | fungi: Oh, I see | 14:48 |
| fungi | guilhermesp_____: correct, and in the other direction (trying to reach my home address from the server) i see what looks like a routing loop between your gateways | 14:48 |
| Perdu | I think any step that leads to an error when you're trying to set things up is pretty frustrating and should be handled in some way | 14:49 |
| fungi | Perdu: the way we've tried to approach it is to suggest that projects have their own getting started guides, and are free to copy from or refer to ours however helps (it's published under an open license so can be freely reused and remixed_ | 14:49 |
| fungi | so for example, trying to follow the opendev manual for contributing to openstack may miss some key details that you would find in the openstack contributor guide here: https://docs.openstack.org/contributors/common/setup-gerrit.html | 14:50 |
| fungi | there they cover the cla up front, because all their projects require it | 14:50 |
| fungi | the idea is that new contributors to, say, kolla-ansible would spot https://opendev.org/openstack/kolla-ansible/src/branch/master/CONTRIBUTING.rst and follow the link to the openstack contributors guide first | 14:51 |
| fungi | the "get started" link up in the top navbar at opendev.org is a bit of a nuisance in that users are likely to spot it first and that will take them to our more generic manual for opendev's infrastructure | 14:52 |
| fungi | might be worth a feature request to gitea to allow the get started link to vary by project being browsed (like the "issues" and "proposed changes" links) rather than being global for the entire site | 14:53 |
| fungi | clarkb: ^ what do you think? | 14:53 |
| guilhermesp_____ | hum yeah ok from a vm inside ca-ymq-1 dc i can ping6 ... however from my home | 14:53 |
| guilhermesp_____ | https://www.irccloud.com/pastebin/fRMMGNAV/ | 14:53 |
| fungi | guilhermesp_____: trying to ping6 my home address from one of our servers, i eventually (after seeing a very long wait) get "From 2604:e100:1::2 icmp_seq=12 Time exceeded: Hop limit" which is indicative of a routing loop | 14:57 |
| fungi | guilhermesp_____: and attempting to traceroute6 to anything in 2600:6c60::/24 just bounces hops between 2604:e100:1::1 and 2604:e100:1::2 | 14:58 |
| fungi | guilhermesp_____: it's hard to hazard a solid guess since you don't operate a public bgp looking glass that i'm aware of, but it's acting just like i would expect if the announcement for 2600:6c60::/24 was being filtered or otherwise not making it into your tables | 14:59 |
| guilhermesp_____ | yeah sadly i dont have access to our core network infra... im trying to get some confirmations with mnaser to see if thats in our domain | 15:00 |
| fungi | also i put most of this detail into the e-mail i sent to the support address last week | 15:00 |
| guilhermesp_____ | yeah i saw it.. i have an internal chat with mnaser to discuss that but he is not available until thuersday since he is traveling | 15:02 |
| *** blarnath is now known as d34dh0r53 | 15:02 | |
| guilhermesp_____ | so trying to do the checks as far as i can go | 15:02 |
| fungi | no worries, i have workarounds, i just don't know how many of our (or your) users are impacted. a /24 is pretty huge from ipv6 standards ;) | 15:03 |
| fungi | if you have any questions or anything you want me to test, let me know | 15:03 |
| fungi | if that route isn't in your tables, then there's a good chance there are others missing too (maybe need to increase the memory you've allowed for v6 tables, for example) | 15:04 |
| guilhermesp_____ | sure yeah, we havent heard back from other customers but what you showed us is in fact something we should keep an eye to validate | 15:04 |
| guilhermesp_____ | but this is in my radar fungi i will try to move forward with him and update you asap | 15:06 |
| fungi | i was running the external networking group for a large isp back in the days when v4 exhaustion was nearing criticality and the global v6 route count started to explode, we had to fiddle with our memory configs multiple times to accommodate the ever growing table sizes (also because of v4 fragmentation) | 15:06 |
| fungi | i saw precisely these sorts of misbehaviors when routers would run out of memory | 15:07 |
| fungi | but of course, there are countless possible causes for what i'm observing | 15:08 |
| guilhermesp_____ | thats helpful from the man that has authority to :) | 15:08 |
| fungi | indeed | 15:08 |
| guilhermesp_____ | i appending the suggestions here into our internal discussion | 15:08 |
| opendevreview | Jeremy Stanley proposed opendev/infra-manual master: Document Gerrit service SSH host key fingerprints https://review.opendev.org/c/opendev/infra-manual/+/894934 | 15:24 |
| fungi | Perdu: that ^ will add gerrit's ssh host key fingerprints to the infra manual once it gets approved, but you can see a rendered preview here in the meantime: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d73/894934/1/check/opendev-tox-docs/d735361/docs/gettingstarted.html#setting-up-your-gerrit-account | 15:31 |
| Perdu | Nice! | 15:38 |
| fungi | i've also asked in #gerritcodereview:matrix.org about the bit in the webui that used to display the fingerprints | 15:38 |
| fungi | and i just remembered that clarkb is gone fishing, so i'll pick his brain tomorrow about the gitea feature request for differentiating the "get started" link by project | 15:39 |
| fungi | thanks for bringing these things up. it's the sort of stuff that we can do better, but established users tend not to notice, so it needs a new user who's willing to spend time pointing them out | 15:41 |
| fungi | like the ssh host keys... i have no clue how many gerrit upgrades ago those might have disappeared. could have been years, but nobody brought it up | 15:42 |
| fungi | chatter in the gerrit channel confirms it disappeared sometime between 2.11 and 3.5 (we're on 3.7), best guess is it never got added to the new polygerrit interface that showed up in 3.0 | 15:43 |
| Perdu | Yea, that what my guess: no established developer is going to check the 'Getting started' page as part of their daily routine | 15:47 |
| fungi | headed out to run some quick errands, shouldn't hopefully be more than an hour | 16:34 |
| opendevreview | Michal Nasiadka proposed opendev/irc-meetings master: Rename Containers Team meeting to Magnum https://review.opendev.org/c/opendev/irc-meetings/+/894941 | 17:10 |
| fungi | looks like nothing fell apart in that hour | 17:35 |
| opendevreview | Merged opendev/irc-meetings master: Rename Containers Team meeting to Magnum https://review.opendev.org/c/opendev/irc-meetings/+/894941 | 17:42 |
| frickler | not sure whether that is some issue with my browser or my connections, but I've had multiple times now where a comment in gerrit is only shown as "[object Object]". after a reload the real comment is shown | 19:51 |
| fungi | sounds like it might be some lag on the backend, or yeah maybe failed rest api queries | 20:14 |
| TheJulia | is Gerrit okay? page loads seem to be stalling out from my end | 21:17 |
| fungi | looking. i wonder if frickler's earlier observations were a precursor | 21:52 |
| fungi | TheJulia: is it sometimes or always? random? comes and goes? | 21:53 |
| fungi | i just pulled up https://review.opendev.org/c/opendev/irc-meetings/+/894941 and it displayed near-instantly for me | 21:53 |
| fungi | noting that i'm connecting over ipv4 for the moment since there are problems with my isp's v6 space being unreachable in vexxhost ca-ymq-1 | 21:54 |
| fungi | TheJulia: do you see similar timeouts/stalling trying to browse https://mirror.ca-ymq-1.vexxhost.opendev.org/ ? | 21:55 |
| TheJulia | it was pretty much consistent and caused me to stop and shift what I was working on | 21:56 |
| TheJulia | but it now loads | 21:56 |
| fungi | okay, so whatever it was may have stopped. i'll check cacti graphs | 21:56 |
| fungi | nothing on the system resource graphs jumps out at me around the time you reported problems | 21:58 |
| TheJulia | weird | 21:58 |
| fungi | might have been something going on somewhere on the internet or inside that vexxhost region maybe | 21:58 |
| fungi | but whatever it was doesn't seem to have either prevented snmp queries nor shown up as a spike in cpu/memory/io/bandwidth/packetrate | 21:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!