Tuesday, 2020-10-06

openstackgerritClark Boylan proposed opendev/system-config master: WIP: Add commentlinks for polygerrit  https://review.opendev.org/75617700:27
clarkbfungi: ^ I quickly put that together and will test it tomorrow00:28
clarkbinfra-root fungi and I successfully upgraded review-test today to 2.16 (pre notedb for changes)00:28
clarkbfirst thing I've noticed is taht commentlinks don't work for polygerrit which I've starting hacking on at ^ but have run out of time and need to do evening things now00:28
clarkbI'll continue to poke at that tomorrow. Once we're generally happy with it the next thing I want to do is the notedb migration and possibly a 3.0 upgrade00:29
clarkbI'm beginnig to think our upgrade process might be something like Day N: do the upgrade to 2.16 without notedb. Then Day N + M do notedb migration + 3.x upgrade00:30
clarkbI'm also beginning to realize that we likely won't get every integration working00:30
fungii'm beginning to realize i need a drink00:31
clarkbin part because if we make 2.16 integrate well we'll likely just throw all that away when we get to 3.x but its hard to make the huge leap to 3.x in one go00:31
fungiokay, that's a lie, i realized hours ago i needed a drink00:31
clarkbbut I think that gets us to a better spot that is sustainable then we can build on top of that from there00:31
ianwi think that sounds sane, especially if we're talking days between00:31
clarkbanyway still plenty to learn but early progress is promising00:31
ianwi'm imagining things like storyboard/irc are the pain points?00:32
clarkbianw: ya00:32
clarkbwell actually irc should be fine00:32
clarkbstoryboard and the ci summary system are problems00:32
clarkbci summary system because checks plugin is in limbo?00:32
fungiand maybe anything else which uses hook scripts00:32
fungior alters the ui javascript00:32
ianwyeah, i'd rate the upgrade more important than the ui overlay at this point00:33
clarkbok food was just waved in front of me to prove it is dinner time00:33
clarkbI'm going to go do that00:33
clarkbianw: ya and I'm hoping that we'll be able to build new once there in a sustainable way00:33
clarkbianw: since polygerrit seems to be sitcking around etc00:33
clarkbwe can discuss in tomorrows meeting too00:33
*** hamalq has quit IRC00:41
funginot entirely certain i'll be around for the meeting, but will try00:54
clarkbok I did more fiddling with the commentlinks. There were some bugs in my string, fixed those. Then still it wasn't working. Put things in a javascript regex tester, took the output of its substitutions and pasted it into the firefox inspect element rewrite tool and it works then :/01:36
clarkbwhich means I'm really stumped at what the problem is now01:36
clarkbit does make me wonder if commentlinks are not handled by polygerrit01:36
openstackgerritClark Boylan proposed opendev/system-config master: WIP: Add commentlinks for polygerrit  https://review.opendev.org/75617701:40
clarkbthats the simplified version that based on javascript regex tester should work01:40
clarkbhttps://gerrit.googlesource.com/gerrit/+/stable-2.16/Documentation/config-gerrit.txt#1431 may be a clue01:52
clarkbok I give up for the night.  Itried fiddling with what I thought was the raw data but that isn't working either :/02:16
clarkbI've turned the server off again and will look at this again tomorrow02:16
*** fressi has joined #opendev04:02
openstackgerritMerged openstack/project-config master: Add Review-Priority flag for cloudkitty  https://review.opendev.org/75607204:05
*** ykarel|away has joined #opendev04:31
*** ykarel|away is now known as ykarel04:36
openstackgerritIan Wienand proposed zuul/zuul-jobs master: ensure-docker: validate network connectivity  https://review.opendev.org/75550504:42
openstackgerritIan Wienand proposed zuul/zuul-jobs master: ensure-docker: < 1500 MTU workaround  https://review.opendev.org/74706304:42
*** marios has joined #opendev05:06
*** fressi has quit IRC05:12
*** ysandeep|away is now known as ysandeep05:26
*** Topner has joined #opendev05:46
*** Topner has quit IRC05:51
*** cloudnull2 has joined #opendev06:03
*** gouthamr has quit IRC06:05
*** bbezak has quit IRC06:05
*** jbryce has quit IRC06:05
*** dviroel has quit IRC06:05
*** weshay|ruck has quit IRC06:05
*** cloudnull has quit IRC06:05
*** openstackgerrit has quit IRC06:05
*** ttx has quit IRC06:05
*** cloudnull2 is now known as cloudnull06:05
*** bbezak has joined #opendev06:05
*** dviroel has joined #opendev06:05
*** gouthamr has joined #opendev06:05
*** jbryce has joined #opendev06:05
*** weshay has joined #opendev06:06
*** ttx has joined #opendev06:10
*** slaweq has joined #opendev06:10
*** eolivare has joined #opendev06:35
*** ralonsoh has joined #opendev06:36
*** fressi has joined #opendev06:46
*** ttx has quit IRC06:50
*** ttx has joined #opendev06:51
*** slaweq has quit IRC07:03
*** andrewbonney has joined #opendev07:21
*** ralonsoh has quit IRC07:21
*** ralonsoh has joined #opendev07:23
*** fressi has quit IRC07:42
*** tosky has joined #opendev07:44
*** rpittau|afk is now known as rpittau07:45
*** mnasiadka has joined #opendev07:47
*** Topner has joined #opendev07:47
*** fressi has joined #opendev07:48
*** Topner has quit IRC07:52
noonedeadpunkianw: can I ask you to look at https://review.opendev.org/#/c/751614/ when you have a chance?07:52
*** moppy has quit IRC08:01
*** moppy has joined #opendev08:01
*** ykarel_ has joined #opendev08:06
*** ykarel has quit IRC08:09
*** ykarel_ is now known as ykarel08:11
*** danpawlik has quit IRC08:38
*** danpawlik has joined #opendev08:39
*** fressi has quit IRC08:40
*** dtantsur|afk is now known as dtantsur08:41
*** fressi has joined #opendev08:43
*** openstackgerrit has joined #opendev08:45
openstackgerritCarlos Goncalves proposed zuul/zuul-jobs master: configure-mirrors: add CentOS 8 Stream  https://review.opendev.org/73478708:45
*** hashar has joined #opendev08:46
*** slaweq has joined #opendev08:47
*** priteau has joined #opendev08:54
*** fressi has quit IRC09:04
*** fressi has joined #opendev09:12
*** ysandeep is now known as ysandeep|lunch09:28
*** fressi has quit IRC09:48
*** priteau has quit IRC09:54
*** DSpider has joined #opendev10:01
*** bbezak has quit IRC10:33
*** dviroel has quit IRC10:33
*** tkajinam has quit IRC10:33
*** dviroel has joined #opendev10:34
*** bbezak has joined #opendev10:34
*** tkajinam has joined #opendev10:34
*** fressi has joined #opendev10:46
*** ysandeep|lunch is now known as ysandeep11:05
*** roman_g has joined #opendev11:10
*** fressi has quit IRC11:13
openstackgerritzbr proposed zuul/zuul-jobs master: ensure-docker: < 1500 MTU workaround  https://review.opendev.org/74706311:26
*** eolivare has quit IRC11:35
*** eolivare has joined #opendev11:42
*** lpetrut has joined #opendev11:54
openstackgerritMichal Nasiadka proposed openstack/diskimage-builder master: Disable growpart in cloud-init-disable-resizefs  https://review.opendev.org/74840212:04
*** ykarel has quit IRC12:06
*** ykarel has joined #opendev12:08
*** fressi has joined #opendev12:25
*** elod is now known as elod_afk13:12
*** ykarel_ has joined #opendev13:17
*** ykarel has quit IRC13:18
*** ykarel_ is now known as ykarel13:30
openstackgerritRiccardo Pittau proposed openstack/diskimage-builder master: Convert multi line if statement to case  https://review.opendev.org/73447913:38
*** hashar has quit IRC13:40
*** mlavalle has joined #opendev13:59
openstackgerritMerged opendev/irc-meetings master: Add Rafael Weingärtner to CloudKitty meeting chairs  https://review.opendev.org/75608714:12
fungijust saw someone use this to share a traceback in #zuul... very compelling alternative to lodgeit! https://privatebin.info/14:20
frickler"As a user you have to trust the server administrator not to inject any malicious javascript code." I'd trust myself in that case, but I wouldn't want to force that trust for others. though otoh we do so with zuul anyhow14:24
fungiyes, i think the selling point there is for the admin of the service, not the user14:25
fungion the other hand, that would make abuse much harder to detect, i suppose14:26
*** ykarel is now known as ykarel|away14:27
fungiat least right now if somebody uuencodes a multi-gb file and splits it across thousands of pastes, we can find and clean that up14:27
fungii suppose large-scale abuse could be countered to some degree with throttling/rate limiting14:28
openstackgerritTobias Henkel proposed openstack/diskimage-builder master: Fix running in a venv  https://review.opendev.org/75632514:30
*** ykarel|away has quit IRC14:47
AJaegermnaser: Vexxhost CI has post failures, see https://review.opendev.org/#/c/755505/ for example14:53
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Pin openstacksdk to <0.48.0  https://review.opendev.org/75633115:09
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Pin openstacksdk to <0.48.0  https://review.opendev.org/75633115:12
*** elod_afk is now known as elod15:15
*** ysandeep is now known as ysandeep|away15:18
*** sgw has quit IRC15:22
*** sgw has joined #opendev15:22
*** fressi has quit IRC15:27
clarkbupdate on the commentlinks. I can match the job name like "tempest-full-py3" as a literal string and that works. I can add in the space after that word "tempest-full-py3 ", but if I add the h in https which is part of the next word like "tempest-full-py3 h" it breaks. I have tried doing a .* and also break15:31
clarkbI have a hunch that the url may be represented as a javascript object by the point we run the commentlink matchers so we can't match it, I'm trying to work around the edges of that idea and will do a commentlink for the suffix after the url next and see if that works15:32
clarkbI'm also cloning the gerrit source to see if I can make sense of the polygerrit commentlink implementation that way15:32
clarkbhttps://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.16/polygerrit-ui/app/elements/shared/gr-linked-text/link-text-parser.js#268 unfortunately I think that confirms my suspicion15:39
clarkbif polygerrit recognizes text as a link it renders it as is and skips the commentlink parsing which is called in the else block15:39
clarkbwe may need to liev with this as a limitation of the polygerrit ui and figure out some other method of making things pretty (though honestly they aren't that bad currently without commentlink substitution)15:40
* clarkb checks newer gerrits to see if that branch changes15:40
*** lpetrut has quit IRC15:43
clarkbya that behavior exists through 3.2 at least. 3.1 (and 3.2) added the ability to have a url prefix I think so you can do things like git+https:// type stuff15:43
clarkbbut nothing that would match our current zuul comments15:44
clarkbcorvus: ^ are you still using the checks plugin with upstream gerrit zuul integration? Wondering if you've got any ideas on how we might approach this if not15:44
*** rpittau is now known as rpittau|afk15:48
clarkbgiven that I'm going to stop digging into this as I think its basically not doable without changing our zuul comment format. We could do something like - zuuljobcomment jobname builduuid SUCCESS $time zuuljobcomment <- and write out a commentlink that would convert that to what we are used to I think15:49
clarkbbut I think the output zuul emits is probably good enough and we can roll with it for now15:50
clarkbnext I'm going to do a db backup at 2.16 pre notedb then do a snapshot of the cinder volume so that we can test the 2.16 and beyond upgrade path now without needing to fully go back to 2.13 if we do it multiple times15:52
*** hashar has joined #opendev15:55
*** marios has quit IRC16:00
*** hamalq has joined #opendev16:12
clarkbI'm unmounting /home/gerrit2 on review-test now to do the snapshotting16:14
*** dtantsur is now known as dtantsur|afk16:18
openstackgerritMerged zuul/zuul-jobs master: ensure-docker: validate network connectivity  https://review.opendev.org/75550516:19
clarkbsnapshotting is not quick but I expect this to be done before our meeting and I'll have the server running again so peopel can poke at it and leave more feedback16:24
clarkbbut then I think tomorrow I'll try the notedb conversion as well as the 3.x upgrades and see what that looks like16:24
clarkbthen we can poke at it again16:25
fungisounds great16:25
openstackgerritMerged zuul/zuul-jobs master: Pin openstacksdk to <0.48.0  https://review.opendev.org/75633116:53
clarkbsnapshot is done and I've remounted the cinder volume17:04
clarkbhttps://review-test.opendev.org is up and running for people to interact with. I'll leave it up for a while now.17:04
clarkbI hvae not yet pushed a change which would also eb a good test17:06
clarkbas far as zuul itnegration goes I expect the zuul testing has us largely covered there so am not too worried about it.17:07
clarkbgertty users might want to check that reviwe-test works for them? I think api auth changes slightly?17:08
clarkb(that might be a bit we need to consider for zuul?)17:08
clarkbya we need to change zuul's gerrit auth_type to digest17:09
clarkber basic from digesty17:10
openstackgerritMerged zuul/zuul-jobs master: ensure-docker: < 1500 MTU workaround  https://review.opendev.org/74706317:10
*** eolivare has quit IRC17:41
*** sgw has left #opendev17:48
*** ralonsoh has quit IRC17:52
openstackgerritClark Boylan proposed opendev/system-config master: Build images for gerrit 3.1 and 3.2  https://review.opendev.org/75637217:56
clarkbI half expect ^ to fail but the way we do image builds is we don't build gerrit in the docker image so the easiest way to find out is to have zuul do it17:57
clarkbI realized I need those images to test beyond a 3.0 upgrade though17:57
*** andrewbonney has quit IRC18:22
clarkbhrm that fails trying to find jgit18:26
clarkbDo we need to predownload jgit?18:27
fungii added a note on the upgrade pad observing that 2.16 seems to have dropped rel="nofollow" (potentially making it a more attractive target for linkspam again)18:27
clarkbya I noticed that too ,its noopen? something like that18:28
clarkbmy hunch is we need to fetch jgit into the workspace of the job that builds the war18:28
clarkbI'm told lunch is ready though so will haev to look at that after the meeting and a bike ride18:28
funginoopen is not similar nor does it help that18:29
fungier, noopener18:29
fungiwhat that does is tell the browser not to allow the link to cause anything new to load in the same tab18:29
fungiin effort to thwart "tabnapping" exploits at the target18:30
funginofollow instructs search engines to not index or at least not improve page ranking of link targets, though the newer ugc attribute is starting to get used similarly to indicate hyperlinks supplied as a part of user-generated content18:31
fungiwe asked gerrit upstream to add nofollow to user comment and commit message hyperlinks a few years back after a linkspamming scare18:32
fungiit seems to have regressed or been intentionally undone18:32
clarkbI bet polygerrit just didjt do it18:34
*** diablo_rojo has joined #opendev18:58
*** roman_g has quit IRC19:01
openstackgerritClark Boylan proposed opendev/system-config master: Build images for gerrit 3.1 and 3.2  https://review.opendev.org/75637219:47
ianwfungi:  could you poke at https://review.opendev.org/#/c/754070/ which fixes the apache gitea proxy, just to clear that19:52
fungiyou bet!19:52
*** tosky has quit IRC19:52
fungiianw: yeah, that graphite failure was odd... i couldn't make heads or tails of it either19:57
ianwi'll see if it repeats ... it should be sending some stats over localhost that graphite should show.  it doesn't seem those packets could go missing19:58
fungimakes me wonder if we just don't exercise that job often and it has a non-negligible failure race19:59
ianwyeah, i exercised it a lot during bring-up and never saw it ... that's why i'm actually hoping a recheck gives me something to debug20:01
clarkbI'm discovering that "Removing the origin causes git to use relative local filesystem paths." is git magic20:07
openstackgerritMerged opendev/base-jobs master: Add centos-8-stream nodeset  https://review.opendev.org/73478820:09
openstackgerritIan Wienand proposed zuul/zuul-jobs master: Add CentOS 8 Stream testing  https://review.opendev.org/73478720:14
openstackgerritClark Boylan proposed opendev/system-config master: Build images for gerrit 3.1 and 3.2  https://review.opendev.org/75637220:22
clarkbcorvus: ^ fwiw the end state I'd like to get to is using a copy of that role you pointed out, I'm sort of working backwards frmo what mordred built to that to learn things20:22
clarkbfor example that git magic about claering out the git origin remote is wow20:22
clarkband now I'm going to pop out for a bike ride20:23
clarkbthey say its the last warm day of the year, need to take advantage :)20:23
openstackgerritMerged opendev/system-config master: gitea: fix 3081 proxy  https://review.opendev.org/75407020:55
*** slaweq has quit IRC21:07
*** slaweq has joined #opendev21:12
ianwmnaser: fyi https://zuul.vexxhost.dev/t/opendev/builds is looking like a lot of POST_FAILURE21:20
*** slaweq has quit IRC21:44
openstackgerritMerged opendev/system-config master: Cleanup graphite01  https://review.opendev.org/75515022:03
*** hashar has quit IRC22:04
openstackgerritIan Wienand proposed opendev/system-config master: doc: update Zuul restart instructions  https://review.opendev.org/75515522:35
openstackgerritClark Boylan proposed opendev/system-config master: Build images for gerrit 3.1 and 3.2  https://review.opendev.org/75637222:37
clarkbI think ^ might actually work. The last ps built a 3.2 image22:37
clarkbbefore we land ^ I'll do some local verification of those images. Probably tomorrow morning then hopefully do the notedb migration and upgrades on test in the afternoon22:38
corvusclarkb: i'm having trouble seeing any commentlink work22:38
clarkbcorvus: I got it to work if I just did a job name as the regex22:38
clarkbcorvus: eg match = tempest-full-py3 and link = https://some.url22:39
corvusthat's what i'm trying, among other even simpler things and it's not happening22:39
corvusi'm using html instead of link22:39
clarkbah I didn't get far enough to html since I couldn't get the simple case to work22:39
corvusno joy with link either22:39
clarkbalso I was restarting the service, I didn't try the more dynamic reload22:40
corvushrm, let me try restarting then22:40
clarkbmaybe the dynamic reload isn't working properly? (not sure if you are restarting or not)22:40
corvusthe reload says it noticed the change but maybe doesn't take effect22:40
corvusugh.  yeah that did it.22:41
corvusit takes a long time to restart :(22:41
clarkbwe're learning things at least :(22:41
*** qchris has quit IRC22:41
clarkbya the restart isn't very quick22:41
corvusclarkb: do you happen to know if the regex matcher is multiline mode?22:49
clarkbcorvus: I don't, but I can go look in that source a bit more22:49
corvusclarkb: i can look (do you have a source link handy though?)22:50
clarkbya one sec22:50
corvus(incidentally "docker restart gerrit-compose_gerrit_1; sleep 12; /etc/init.d/apache2 restart" ) is about right timing wise22:51
clarkbcorvus: https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.16/polygerrit-ui/app/elements/shared/gr-linked-text/link-text-parser.js#27722:51
corvusclarkb: thx22:52
*** qchris has joined #opendev22:53
corvus(so that's a negative)22:55
clarkbcorvus: if I read that correctly the 'g' at https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.16/polygerrit-ui/app/elements/shared/gr-linked-text/link-text-parser.js#304 is where you set the modes and m is multiline22:55
clarkbso ya not multiline22:55
*** mlavalle has quit IRC23:03
corvusclarkb: it iterates over text chunks, apparently after markdown processing and url processing.  given the original message format of "- jobname jobname url : result comment", an individual execution of a commentlink regex is going to get "jobname jobname ".  then the next execution will get " : result comment"23:19
corvusthe initial "- " is hidden from us, as is the url23:19
corvus(and also the trailing newline)23:20
corvusi should say pseudo markdown processing :/23:20
corvusif it were real md the solution would be obvious23:20
fungiwell, to the extent that there's such a thing as "read markdown" given all the competing post-hoc standards which are floating around for it23:21
clarkbya so we never see the url at all to match on it? even if it is wrapped in {} or similar?23:21
corvusclarkb: correct23:21
corvusi've learned this by stepping through in the debugger23:21
clarkbin that case probably what zuul does is the best option for us? otherwise we'd have to do some weird zuul comment that is expected to be rewritten with comment links23:26
clarkblike - jobname jobuuid status time then render that to a url ourselves?23:26
corvusi think https://gerrit.googlesource.com/gerrit/+/refs/heads/stable-2.16/polygerrit-ui/app/elements/shared/gr-formatted-text/gr-formatted-text.js is the text formatting bit23:29
corvusi don't see anything about handling urls specially23:29
* clarkb writes the email about testing review-test now23:30
corvusand http://benalman.com/code/projects/javascript-linkify/docs/files/ba-linkify-js.html is what does the chunking/linkify stuff23:30
corvusdoesn't look like it has any magic syntax either23:30
corvusclarkb: so yeah, i'm not seeing any good options.23:31
corvusclarkb: i think the new-world gerrit answer would be a polygerrit plugin23:32
clarkbah interesting. And that is something we can owrk on post upgrade problably23:32
clarkbcorvus: I wonder if the mediaiwki folks have already done some of that?23:40
clarkbpaladox: ^23:40
corvusmaybe could be added to https://gerrit.googlesource.com/plugins/zuul/ ?23:45
clarkbooh ya I think that could be a good fit23:50
clarkbheh the first 3.3.0 rc for gerrit shows up just as soon as I make a 3.2 image change23:51
clarkbits ok we're making progress23:51
fungii expect our upgrade to 3.x can target 3.3 then depending on their release timeline23:53
clarkbpossibly, we'll see what we learn when we try the 3.x upgrades next. Supposedly they are easier than the old 2.x upgrades have been23:54
clarkbseparately I'm really happy that we've managed to convince people submodules are a bad idea23:57
clarkbmy brain has melted thinking about the gerrit job setup :)23:57
*** DSpider has quit IRC23:58

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!