| *** liuxie is now known as liushy | 02:40 | |
| opendevreview | Bernhard Berg proposed zuul/zuul-jobs master: prepare-workspace-git: Add allow deleting current branch https://review.opendev.org/c/zuul/zuul-jobs/+/917539 | 07:27 |
|---|---|---|
| zigo | fungi: FYI, because of D. Pococ spamming every list all the time, all important lists (like debian-project@ and debian-devel@) are moderated (but only for spam, not for content). Though there was very few spam (as in: adds, not as in: daniel pocok type of messages). | 09:15 |
| opendevreview | Bernhard Berg proposed zuul/zuul-jobs master: prepare-workspace-git: Add allow deleting current branch https://review.opendev.org/c/zuul/zuul-jobs/+/917539 | 11:27 |
| slittle_ | We are having audio issues with https://meetpad.opendev.org/stx-build | 15:08 |
| slittle_ | wher ecan I get help with that? | 15:08 |
| clarkb | slittle_: this is an appropriate venue. Can you be more specific about what sorts of problem you are experiencing? | 15:09 |
| slittle_ | No one can hear anyone else | 15:09 |
| slittle_ | not muted | 15:09 |
| slittle_ | audio device in/out is correctly selected | 15:10 |
| clarkb | historically that problem has been due to browsers not allowing auto playing media, but I think we worked around that by adding the landing page so that probably isn't the issue | 15:10 |
| clarkb | I am in a meeting right now so can't tryand reproduce right now but can take a closer look after this meeting. Have you tried another browser in order to rule out browser compatibility issues? I generally use Chrome/Chromium but if the problem is there then maybe trying firefox is worth a shot | 15:11 |
| slittle_ | tried with Firefox and Chrome from linux host | 15:11 |
| slittle_ | under the audio settings, the microphone volume level indicator moves when I speak | 15:13 |
| slittle_ | so it's getting a signal | 15:13 |
| slittle_ | it does seem to auto-mute me rather frequently. | 15:13 |
| clarkb | the only time it should auto mute anyone is when you go from 1 meeting attendee to 2 | 15:14 |
| tonyb | slittle_: Are you "Scott" Looks like it's just the two of us on that meetpad? | 15:14 |
| slittle_ | Yes | 15:15 |
| slittle_ | I'm Scott | 15:15 |
| tonyb | Okay can did you hear me earlier? | 15:15 |
| slittle_ | the auto mute thing is kicking in a lot now | 15:15 |
| fungi | auto mute thing? | 15:16 |
| fungi | the service is configured to mute all users when they first join, and expects them to unmute themselves if they want to talk. keeps noise to a minimum when you have larger calls with most people listening rather than speaking | 15:17 |
| slittle_ | lots of notifications wizzing by... 'you are starting the conversation muted' | 15:17 |
| fungi | makes me wonder if you're connecting through a firewall that interferes with websockets connections | 15:18 |
| tonyb | slittle_: How are people on the call? As far as I can tell it's only you and I | 15:19 |
| fungi | i heard you were having similar issues with meet.jit.si, so it could be something to do with connections from browsers getting broken by a middlebox/proxy | 15:19 |
| tonyb | slittle_: but I see 'unnamed' in the etherpad | 15:19 |
| slittle_ | We are firewalled for sure | 15:20 |
| fungi | going to guess you have zero control over nor insight into that firewall (not work-from-home) | 15:21 |
| slittle_ | it seems to be de-selecting my microphone | 15:22 |
| slittle_ | fungi you are correct | 15:22 |
| slittle_ | corp firewall is a black box to me | 15:22 |
| tonyb | slittle_: Have you heard me any time I spoke? | 15:23 |
| slittle_ | no | 15:23 |
| tonyb | slittle_: Trying to work out if it's full audio issue or your mic? | 15:23 |
| tonyb | Okay | 15:23 |
| clarkb | I can hop in in about 40 minutes and see if this is a geneal problem or spceific to specific clients | 15:24 |
| tonyb | slittle_: If you can hang in there for 5 more mins Im looking at the serverside logs for clues | 15:25 |
| slittle_ | sure | 15:25 |
| slittle_ | when I keep the audio-settings dialog up, what I see is that my microphone selection is unchecked and a flurry of notifications are shown on the left hand side ... too fast to read, | 15:26 |
| slittle_ | this happens 10-15 secs after I renew my microphone selection | 15:27 |
| tonyb | You said you'd tried chrome and firefox, which one are you using now? | 15:28 |
| slittle_ | Chrome | 15:28 |
| slittle_ | when the microphone is de-selected, it mutes me. | 15:30 |
| fungi | my other call ended early, i'll try joining | 15:32 |
| slittle_ | mayby it's the other way around. When I mute deliberately, the audio control microphone selection looses it's check mark | 15:34 |
| clarkb | the three of us are in the room and able to hear each other. Looking at your connection status info in the meeting no codec is selected and it has no bitrate info | 15:36 |
| clarkb | which would probably be evidence to fungi's theory | 15:37 |
| clarkb | one way to test that would be to use a different internet connection if possible to get around the firewall and at least confrim (or deny) if the local browsers are working | 15:37 |
| fungi | another possibility could be corporate browser policy enforced on everyone's work machines disabling features (webrtc or specifici hardware access) | 15:38 |
| clarkb | slittle_: if test is also you then the connection stats look very similar to your older connection | 15:44 |
| clarkb | another thing you might be able to do is open your local browser network debugger and see what connections (if any) are being rejected/fail | 15:45 |
| slittle_ | not sure how to do that in chrome | 15:47 |
| tonyb | slittle_: You'd use chrome developer tools | 15:48 |
| clarkb | open the three dot menu -> more tools -> developer tools then there is a network tab | 15:48 |
| slittle_ | are there specific ports i need to ask our IT to open to enable https://meetpad.opendev.org/ audio | 15:48 |
| clarkb | it should start to populate with network requests made by the page you open that on | 15:48 |
| slittle_ | got it. What am i looking for | 15:50 |
| fungi | as best we can tell, all client communication with jitsi-meet is tunneled through https on port 443/rcp | 15:51 |
| clarkb | looking for http requests that fail. I've opened it locally and there are a number of https requests to http-bind?room=stx-build and they are all 200 success responses | 15:51 |
| slittle_ | BridgeChannel.js:91 getting 101 all the time | 15:51 |
| slittle_ | all else is 200 | 15:51 |
| slittle_ | type websocket | 15:52 |
| fungi | though if your connections are going through an https proxy or some sort of web inspection firewall that doesn't understand/handle websocket connections, i wouldn't be surprised | 15:52 |
| fungi | also, when said 443/rcp earlier that was a typo, i meant tcp of course | 15:53 |
| slittle_ | k, I'll open a ticket with our IT. See if they can open this up | 15:55 |
| fungi | well, 443/tcp is the standard https port, but yeah it could be some deeper connection inspection they're doing that assumes normal https traffic on that port | 15:56 |
| fungi | and freaks out when it encounters audiovisual streams tunneling through there | 15:56 |
| fungi | slittle_: we're seeing the same incomplete a/v connection state from michel too | 15:58 |
| tonyb | slittle_: From the logs on the server I just found "<candidate component='1' foundation='1' generation='0' id='2ef737cd2271d0b2031dbe602' network='0' priority='2130706431' protocol='udp' type='host' ip='10.209.128.133' port='10000'/" | 16:01 |
| tonyb | So it looks that could be the firewall "magic" needed, | 16:02 |
| tonyb | I don't know how dynamic that is | 16:02 |
| fungi | tonyb: 10.209.128.133 is the eth1 interface on our meetpad01 server | 16:03 |
| tonyb | Ahh nevermind then | 16:03 |
| fungi | that log entry looks like it could be related to server-to-server communication, which we configured to go over those interfaces | 16:03 |
| fungi | so probably not having anything to do with a client-to-server connection | 16:04 |
| tonyb | Yeah Sorry | 16:05 |
| slittle_ | any idea who 'owns' the https://zoom.us/j/342730236?pwd=N21CUXNXVlJXMlcyZjZ0SE96cVNjQT09 link used by most of the StarlingX meetings? Perhaps I can book a time slot on that bridge. | 16:11 |
| fungi | ildikov: ^ maybe you know? | 16:11 |
| ildikov | fungi: yes, it's me | 16:45 |
| tonyb | clarkb: is 907902: Don't run mirror jobs when roles/ updates | https://review.opendev.org/c/opendev/system-config/+/907902 still desirable? | 17:38 |
| clarkb | tonyb: yes I think so | 18:08 |
| tonyb | clarkb: okay cool | 18:15 |
| opendevreview | Merged opendev/system-config master: roles/openafs-client: small overhaul of rpm builds https://review.opendev.org/c/opendev/system-config/+/906013 | 18:21 |
| fungi | 23.253.166.211 is the held etherpad upgrade node | 18:28 |
| fungi | testing it with https://etherpad.opendev.org/p/isitbroken | 18:29 |
| clarkb | seems to be working from here. Even chrome isn't disconnecting like it has in the past | 18:32 |
| fungi | yeah, no issues for me yet | 18:32 |
| clarkb | fungi: you might want to login to the test node and perform an admin task or three just to make sure that is working in a way that is workable. Oh and I think we need to add the new secret for that to vars before merging | 18:33 |
| fungi | oh, good call | 18:33 |
| clarkb | pretty sure I didn't add it since all the stuff previously was pre release testing | 18:33 |
| clarkb | the process is documented in the change's doc updates | 18:33 |
| fungi | yeah, i just pulled it up | 18:34 |
| fungi | does leave me wondering whether settings.json should be world-readable | 18:35 |
| clarkb | probably shouldn't be | 18:36 |
| fungi | annoying that curl suddenly gets verbose any time stdout is a pipe instead of a tty | 18:47 |
| clarkb | fungi: I think I switched to curl bceause I thought it had/has special jwt handling but then it ended up not or I didn't use it. Could probably use wget just as easily | 18:49 |
| clarkb | though wget writes to fiels by default which is a bit annoying when you need the token value | 18:49 |
| fungi | yeah, `wget -qO-` is deeply etched into my finger memory, just trying to work out how to get curl to behave the same way | 18:52 |
| fungi | curl -H "Authorization: Bearer $(curl --data grant_type=client_credentials --data client_id=api_admin --data client_secret="$(grep '"client_secret":' /etc/etherpad/settings.json | sed -e 's/\s\+"client_secret": "\(.*\)",/\1/')" http://localhost:9001/oidc/token | sed -e 's/.*"access_token":"\([^"]*\)".*/\1/')" "http://localhost:9001/api/1.2.11/restoreRevision?padID=isitbroken&rev=104" ; echo | 18:59 |
| fungi | that returned {"code":0,"message":"ok","data":null} and observably rolled the pad's content back to match what was in revision 104 | 19:00 |
| fungi | curl "http://localhost:9001/api/1.2.11/restoreRevision?padID=isitbroken&rev=104" ; echo | 19:00 |
| fungi | that returns {"code":4,"message":"no or wrong API Key","data":null} | 19:00 |
| fungi | so this seems to be working as intended | 19:00 |
| clarkb | I think the email from openmetal that we just got is them announcing the stuff we were waiting for before redeploying that cloud | 20:04 |
| clarkb | so now we can add that to the todo list :) | 20:04 |
| tonyb | yay | 20:10 |
| tonyb | popping out for an hour or so | 20:10 |
| fungi | oh awesome | 20:15 |
| clarkb | I've got to do a school run in a bit and then there are a few zuul changes I want to review. But let me knwo if there is anything I'm missing on the opendev side of things. | 20:57 |
| clarkb | I've got our topic limit questions posted over to Thursday's gerrit community meeting agenda. I'll do my best to attend and have that discussion with them | 21:02 |
| fungi | thanks! | 21:14 |
| fungi | heading out to dinner, but should be back soon | 21:28 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!