Tuesday, 2021-05-18

*** mlavalle has quit IRC00:19
*** brinzhang_ has quit IRC00:23
*** zbr has quit IRC03:21
*** zbr0 has joined #opendev03:21
*** zbr0 has quit IRC03:30
*** zbr has joined #opendev03:30
*** ykarel has joined #opendev04:25
*** hamalq has quit IRC04:27
*** marios has joined #opendev05:00
*** vishalmanchanda has joined #opendev05:32
openstackgerritHitesh Kumar proposed openstack/diskimage-builder master: Migrate from testr to stestr  https://review.opendev.org/c/openstack/diskimage-builder/+/78924605:56
*** amoralej|off is now known as amoralej06:04
*** sboyron has joined #opendev06:06
*** slaweq has joined #opendev06:22
*** ralonsoh has joined #opendev06:30
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [wip] Use containers for functional testing  https://review.opendev.org/c/openstack/diskimage-builder/+/79188806:36
openstackgerritIan Wienand proposed opendev/system-config master: [DNM] Use new docker ipv6tables option to map haproxy ports  https://review.opendev.org/c/opendev/system-config/+/79163306:51
*** ysandeep|away is now known as ysandeep07:02
*** fressi has joined #opendev07:03
*** hashar has joined #opendev07:07
*** hashar is now known as hasharAway07:07
*** andrewbonney has joined #opendev07:13
*** rpittau|afk is now known as rpittau07:27
*** auristor has quit IRC07:32
*** hasharAway is now known as hashar07:43
*** tosky has joined #opendev07:46
*** jpena|off is now known as jpena07:57
*** lucasagomes has joined #opendev08:05
openstackgerritMatthias Runge proposed openstack/project-config master: Retire panko, python-pankoclient and puppet-panko  https://review.opendev.org/c/openstack/project-config/+/79190508:28
openstackgerritMatthias Runge proposed openstack/project-config master: Retire panko, python-pankoclient and puppet-panko  https://review.opendev.org/c/openstack/project-config/+/79190508:53
*** tkajinam has quit IRC09:02
*** tkajinam has joined #opendev09:03
openstackgerritMatthias Runge proposed openstack/project-config master: Retire panko, python-pankoclient and puppet-panko  https://review.opendev.org/c/openstack/project-config/+/79190509:05
*** hrw has quit IRC09:07
*** ysandeep is now known as ysandeep|lunch09:10
*** artom has quit IRC09:43
*** artom has joined #opendev09:43
*** artom has quit IRC09:43
*** artom has joined #opendev09:44
*** klonn has joined #opendev10:07
*** dtantsur|afk is now known as dtantsur10:14
*** ysandeep|lunch is now known as ysandeep10:17
*** gtema has joined #opendev10:20
gtemadevstack just landed EOL for bionic. Since bionic is still a default image in base-jobs some jobs started to fail.10:22
gtemashould the base-job be modified or the jobs be switched explicitly to focal?10:22
toskygtema: http://lists.opendev.org/pipermail/service-announce/2021-May/000019.html10:26
gtemabuh, thanks tosky10:28
gtemaI have overseen this email10:28
toskyand the date is today! I didn't remember that10:29
gtemajust landed some hours ago10:30
gtemaand the mess starts ;-)10:30
toskyuh, but https://review.opendev.org/c/opendev/base-jobs/+/789098/ is still WIP10:32
*** klonn has quit IRC10:38
*** gtema has quit IRC10:56
*** whoami-rajat has joined #opendev11:06
*** auristor has joined #opendev11:20
*** jpena is now known as jpena|lunch11:26
*** ysandeep is now known as ysandeep|brb11:52
*** amoralej is now known as amoralej|lunch11:58
*** jpena|lunch is now known as jpena12:13
*** klonn has joined #opendev12:15
fungiwip was to prevent it from merging early12:20
fungii can un-wip and approve it now12:20
openstackgerritMerged opendev/base-jobs master: Switch the base jobs to use ubuntu-focal nodes  https://review.opendev.org/c/opendev/base-jobs/+/78909812:28
*** ysandeep|brb is now known as ysandeep12:30
fungigtema: tosky: ^ i'm replying to mailing lists now12:34
fungihopefully it fixes more things than it breaks12:45
fungii was planning to merge it later today, but with devstack dropping bionic support i guess it's better to move forward asap12:47
mrungehi there, is there a way to tell project-config to stop gating on master (in prep for retirement of a project)?12:48
fungimrunge: yes, is it an openstack project? i think there are special instructions12:48
mrungefungi: yes, it is. I am currently working on https://review.opendev.org/c/openstack/project-config/+/79190512:49
mrunge(panko removal)12:49
fungimrunge: https://docs.openstack.org/project-team-guide/repository.html#retiring-a-repository12:49
mrungeyes, I have looked at that12:49
mrungebut that just removes the project completely12:49
mrungeI wouldn't want to remove it for stable right now12:49
fungisee the section right after it then, https://docs.openstack.org/project-team-guide/repository.html#deprecating-a-repository12:50
mrungeapparently, that doesn't do the job for panko then12:50
fungistep 2 there which switches to noop jobs?12:51
fungiwhat problem are you running into?12:51
fungiwhen did your openstack/project-config change from step 2b merge?12:51
mrungethis patch e.g depends on the one https://review.opendev.org/c/openstack/puppet-panko/+/79189012:52
mrungenothing was merged yet, I just added a depends-on12:52
mrungethere is a couple of patches piled up, all depending on each other12:52
fungiahh, yeah openstack/project-config is a trusted config project in zuul, so speculative configuration changes aren't applied for security reasons. changes to openstack/project-config have to merge before any job alterations in them take effect12:53
mrungeah! thank you, that explains it12:53
fungionce the openstack/project-config change merges you should be able to recheck the changes depending on it12:54
fungiand hopefully see them pass12:54
mrungeokay, that's good to know, thank you!12:54
fungiyou're welcome12:54
fungithe openstack project team guide could probably do a better job of mentioning that fact12:55
*** hashar has quit IRC13:07
*** hashar has joined #opendev13:08
*** amoralej|lunch is now known as amoralej13:10
*** klonn has quit IRC13:16
*** klonn has joined #opendev13:22
openstackgerritRiccardo Pittau proposed openstack/project-config master: Retire sushy-cli - Step 1  https://review.opendev.org/c/openstack/project-config/+/79196313:31
*** mnasiadka has quit IRC13:35
*** mnasiadka has joined #opendev13:37
openstackgerritRiccardo Pittau proposed openstack/project-config master: Retire sushy-cli - Step 3  https://review.opendev.org/c/openstack/project-config/+/79196713:43
*** bhagyashris has quit IRC13:52
*** bhagyashris has joined #opendev13:54
*** openstack has joined #opendev14:08
*** ChanServ sets mode: +o openstack14:08
*** ysandeep is now known as ysandeep|mtg14:10
*** openstackstatus has joined #opendev14:11
*** ChanServ sets mode: +v openstackstatus14:11
*** artom has joined #opendev14:13
*** dpawlik7 is now known as dpawlik14:19
*** fressi has quit IRC14:22
*** ricolin has quit IRC14:41
fungiclarkb: per https://bugs.chromium.org/p/gerrit/issues/detail?id=13721 i guess we can drop our patch if we update to 3.1.1514:43
clarkbfungi: note we don't want to downgrade to 3.1.x but ya I think that the same fixes should end up on 3.2 and we could drop it if we rebuild our 3.2 images?14:44
clarkbfungi: we have functional testing of that so I guess I can push up a change that checks if that is already present in 3.2 and 3.314:44
fungioh, right, we're already on 3.214:45
*** hashar is now known as hasharAway14:48
openstackgerritClark Boylan proposed opendev/system-config master: Remove special x/ handling patch in gerrit  https://review.opendev.org/c/opendev/system-config/+/79199514:48
clarkbfungi: ^ that should functionaly test things for us14:49
clarkbsince the test job for gerrit does clone from x/test-project or similar14:49
fungioh yep14:49
clarkbI realize this morning that I didn't detach zuul01 from ua. Oh well14:56
clarkbif it becomes a problem later we can sort that out14:56
fungithere's probably a dashboard somewhere which lets you unregister stuff directly14:57
*** ysandeep|mtg is now known as ysandeep15:01
*** ykarel has quit IRC15:23
mordredalso- looks like they released 3.415:25
*** amoralej is now known as amoralej|off15:35
*** hasharAway is now known as hashar15:45
*** ysandeep is now known as ysandeep|dinner15:46
*** tosky_ has joined #opendev15:49
*** tosky has quit IRC15:52
*** lucasagomes has quit IRC16:02
clarkb"ERROR: /home/zuul/src/gerrit.googlesource.com/gerrit/lib/bouncycastle/BUILD:3:13: //lib/bouncycastle:bcprov depends on @bcprov//jar:jar in repository @bcprov which failed to fetch. no such package '@bcprov//jar': Argument 0 of execute is neither a path, label, nor string."16:07
clarkbdidn't get far enough to test the handling of x/foo projects16:07
clarkbmordred: ^ do you know if thati s a broken dependency or if that is something we control?16:07
clarkbhrm does that mean maybe that bcprov is now a local dep that we need to ensure gets checked out (git submodule etc?)16:08
clarkbgerrit/lib/bouncycastle/BUILD hasn't updated in a very long time16:09
mordredclarkb: it's entirely possible it has become a submodule - lemme look16:16
mordredclarkb: that's your 3.2 patch yeah?16:17
clarkbmordred: ya, I've got a local checkout of stable-3.2 and in the gerrit/WORKSPACE dir they define the maven_jar for bcproc16:17
mordredyeah - it doesn't seem to be submodule16:17
clarkbI think it isn't expected to be locally built based on ^ and instead we failed to fetch it?16:17
*** klonn has quit IRC16:18
clarkbhttps://search.maven.org/artifact/org.bouncycastle/bcprov-jdk15on/1.61/jar I think that is what the WORKSPACE directive is saying gerrit wants16:19
*** marios is now known as marios|out16:19
clarkbyup and the sha1 mtaches16:19
clarkbI think I'll recheck and we can see if it ws the internet being sad16:20
clarkbalso today turned into a day of a million meetings so I need to eat food while I've got time. Back in a bit16:21
mordred++ to recheck16:22
fungiyes, i ended up with a 6-hour block of meetings scheduled16:23
funginot sure why today turned into the meeting dumping ground16:23
*** rpittau is now known as rpittau|afk16:45
clarkbit failed again this time on bcpkix16:50
clarkbmaybe maven is having trouble?16:50
clarkbhttps://status.maven.org/ says it should be happy though16:50
clarkbbcpkix comes after bcprov in the WORKSPACE file16:51
*** mikidep has joined #opendev16:51
clarkbassuming it evaluates in order that would imply it got bcprov successfully16:51
clarkbI've got to context switch to otherthings which may be for the best. we can rechec kin a few hours and see if it changes behavior16:51
mikidepHi everybody, I found a bug in the tosca-parser project, how can I submit it or contact someone who worked on the project?16:52
clarkbmikidep: https://opendev.org/openstack/tosca-parser/src/branch/master/CONTRIBUTING.rst has pointers for contributing directly. Essentially you need to create an account, sign the CLA and then push to gerrit16:53
mikidepclarkb: thanks, the thing is, the code is a bit complicated and I don't understand exactly the purpose of each piece. I think I found the line where the issue happens but I can't understand why it's there and what it accomplishes.16:58
*** jpena is now known as jpena|off16:58
mikidepI would be glad to submit it to the attention of someone who understands the code, maybe the author of that piece of code, to avoid making a mess.16:59
fungimikidep: that repository is part of the heat project, and it looks like they hang out in the #heat irc channel here on freenode17:00
mikidepfungi: Ok thanks17:01
clarkbyup your best bet is probably to reach out the the heat developers17:01
*** klonn has joined #opendev17:15
mordredclarkb: if we re-kick in a few hours and it still is borked, we can ping luca and see if he sees anything17:16
mordredclarkb: oh!17:18
mordred    python = ctx.which("python")17:18
mordred    args = [python, script, "-o", binjar_path, "-u", binurl]17:18
mordred    out = ctx.execute(args)17:19
mordredso - the error is: "Argument 0 of execute is neither a path, label, nor string."17:19
mordredwhatcha wanna bet there's no bare python on that node, only a python317:19
mordreddo we have a role somewhere that makes a python symlink?17:20
clarkboh this could be related to the base image update :)17:20
clarkbya I bet that is it17:20
clarkbfungi: ^ fyi since you were doing base nodeset change17:20
*** ralonsoh has quit IRC17:21
fungiooh, fun. we can try setting the nodeset in the job to see what happens17:22
clarkbor install python2 as part of the job17:24
openstackgerritMonty Taylor proposed opendev/system-config master: DNM Test whether a python symlink fixes gerrit  https://review.opendev.org/c/opendev/system-config/+/79202017:24
mordredoh - whoops, didn't see those suggestions :)17:25
mordredI tried sledgehammer a17:25
clarkbmordred: if yours works we should suggest that gerrit make that change upstream17:25
clarkband use python317:25
mordredwell - the script they're running at least has print()17:27
*** marios|out has quit IRC17:27
*** andrewbonney has quit IRC17:32
*** ysandeep|dinner is now known as ysandeep17:36
*** dtantsur is now known as dtantsur|afk17:43
*** ysandeep is now known as ysandeep|away17:46
*** tiotags has joined #opendev17:46
*** mikidep has quit IRC17:48
clarkbmordred: looks like the 3.3 build worked in your cases at least. the 3.2 is still running. However the system-config-run jobs failed for both arleady (how does 3.2 fail before the image is even built?)18:00
clarkbcorvus: ^ you might want to look at that to double check it isn't related to the recent zuul updates?18:00
corvusclarkb: ack18:01
*** hashar has quit IRC18:02
*** hamalq has joined #opendev18:05
mordredyeah - I wouldn't expect those to have run18:07
clarkbI think they are supposed to trigger off of edits to the docker and image build stuff but they have dependencies and should wait I thought18:08
clarkbrequires: gerrit-3.2-container-image in the run job and provides: gerrit-3.2-container-image in the build job18:09
*** lpetrut has joined #opendev18:10
*** lpetrut has quit IRC18:10
clarkbzuul reported back what it thought the issue was to gerrit. I think we need to squash the two changes18:15
clarkbmordred: ^ do you want to do that or should I?18:15
clarkbbasically the parent broke and it wants to push that down the chain18:15
mordredoh - gotit18:17
mordredI can squash18:17
clarkbwe might also need to do the job dependencies explicitly for ordering within a buildset18:18
openstackgerritMonty Taylor proposed opendev/system-config master: Remove special x/ handling patch in gerrit  https://review.opendev.org/c/opendev/system-config/+/79199518:18
*** zbr has quit IRC18:48
*** zbr has joined #opendev19:00
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [wip] Use containers for functional testing  https://review.opendev.org/c/openstack/diskimage-builder/+/79188819:07
*** whoami-rajat has quit IRC19:16
*** tosky_ is now known as tosky19:25
mordred clarkb ^^ all green19:36
clarkbmaybe we should see if upstream gerrit want to update their things to use python3 before we land the symlink hack?19:37
mordredclarkb: hah. it's fixed in master19:38
mordredthey fixed it as part of a larger "Migrate to python3" patch: https://gerrit-review.googlesource.com/c/gerrit/+/29890319:41
mordredit's not in 3.4 either19:41
fungibetter late than never19:41
mordredso I think we probably need our hack until 3.519:41
mordredshouldI update the patch and add a comment saying that?19:41
clarkbmordred: ++ also looking at stable-3.2 894638f24f7b7f648eb911d514ea90fc993fdb87 seems to have added the git serving fix for x/19:42
clarkbI think the commit message can be updated to indicate that it is present now and our testing just confirms it19:42
mordreddo you want to hold/test something?19:43
mordredor are our fundtional tests sufficient?19:43
clarkbmordred: it should already be tested by our functional testing19:43
clarkband worst case I guess we just revert to the older image?19:43
mordredcool - yeah19:43
clarkbmordred: but we can also split the changes again if we want19:44
clarkband land the python fix first then worry about x/19:44
mordrednah - I think landing both seems fine19:44
clarkbI'm happy with that too if we want to keep things simple and revertable19:44
mordredoh - hrm19:44
mordredthat's a good point19:44
openstackgerritMonty Taylor proposed opendev/system-config master: Remove special x/ handling patch in gerrit  https://review.opendev.org/c/opendev/system-config/+/79199519:47
openstackgerritMonty Taylor proposed opendev/system-config master: Symlink python3 to python for gerrit image build  https://review.opendev.org/c/opendev/system-config/+/79204119:47
*** cloudnull has quit IRC19:48
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [wip] Use containers for functional testing  https://review.opendev.org/c/openstack/diskimage-builder/+/79188820:17
*** klonn has quit IRC20:19
clarkbLooking at the mailman stuff again there are a couple of things we should pay attention to when landing the ansiblification (and mostly I'm writing this here so others can see it but it is largely for myself). We want to check the /etc/mailman/sites file is equivelant to what we had before (I expect entry order to change but otherwise be equivalent). We also want to check that the crontab entries20:42
clarkbare good. The puppet ones are unlikely to be modified/removed by the ansible ones so there will be some cleanup there20:42
*** tiotags has quit IRC20:48
*** auristor has quit IRC20:52
*** auristor has joined #opendev20:52
mordredyeah - I never came up with a great way to have ansible replace puppet crontab entries - other than updating puppet to remove them20:55
mordredthis is a good argument for cron.d files rather than cron: entries20:56
*** slaweq has quit IRC20:57
*** sboyron has quit IRC21:07
*** cloudnull has joined #opendev21:13
*** auristor has quit IRC21:17
*** auristor has joined #opendev21:17
openstackgerritMonty Taylor proposed opendev/system-config master: Start building gerrit 3.4  https://review.opendev.org/c/opendev/system-config/+/79204721:24
mordredclarkb: since I had it up - I thought I'd make one of those ^^21:25
*** cloudnull has quit IRC21:30
corvusinfra-root: some off-topic food for thought based on stuff i've been working on recently: i think a prometheus server with snmp_exporter would be a good replacement for cacti (we could connect grafana to it for graphing).   we would also get a web dashboard of alerts for free.  it seems likely that zuul will grow some prometheus support as well, so we could start feeding that into the prometheus server if21:36
corvusthat happens.  zuul would probably continue to emit statsd metrics for a while; we could either move those into prometheus or leave the graphite server to handle that.  grafana can talk to both backends simultaneously, so it's not a big deal.21:36
corvus(i have used prometheus with statsd_exporter, but i have not used it with snmp_exporter, so i don't have direct experience with that; so i'm not vouching for it.  :)21:37
fungisounds interesting. what sort of data storage backend does it use? rrd files or some other aggregative/roll-up format?21:37
corvus(i have also recently set up an influxdb2+telegraf system for snmp; that's working fine, but influxdb2+grafana is not in a happy place right now and i would not recommend it -- influxdb appears to want to move into the graphing dashboard space; if they succeed, then it might be interesting in that it could supercede grafana, but right now that's nowhere near as good, so i'd rather keep grafana and also21:39
corvushave a backend system that's easy to use with it)21:39
corvusfungi: it has its own time series db format: https://prometheus.io/docs/prometheus/latest/storage/21:39
fungiinfluxdb has/had licensing problems? i can't recall now21:40
corvusfungi: i don't remember influx licensing issues...21:41
fungii'm likely confusing it with mongo or cockroach or something21:41
corvusmongo def.21:42
corvus(they made their own agpl)21:42
fungiahh, influxdb is open-core and clustering requires you buy a license21:43
fungithat's what it was21:43
corvusah :)21:43
fungiso as long as we don't want a redundant cluster we're probably fine using their community edition21:43
corvusi found that it requires a phd to use their new query language with grafana.  prometheus+grafana is much easier.21:44
corvusi will probably replace the influxdb+telegraf stuff i did with prometheus+snmp_exporter; if/when i do, i'll have a better user report :)21:44
openstackgerritAde Lee proposed zuul/zuul-jobs master: Add role to enable FIPS on a node  https://review.opendev.org/c/zuul/zuul-jobs/+/78877821:59
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [wip] Use containers for functional testing  https://review.opendev.org/c/openstack/diskimage-builder/+/79188822:31
clarkbno opposition from me to use more modern metrics gaterhing tools22:56
clarkband ya influxdb's issue was the open core stuff. I know for a lot of openstack users that was problematic. I don't think it will be a problem for us22:57
*** tosky has quit IRC23:05
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [wip] Use containers for functional testing  https://review.opendev.org/c/openstack/diskimage-builder/+/79188823:17
clarkbmordred: looks like we might also need to bump the timeout on those gerrit image builds. The python fix's 3.3 build job timed out23:30
clarkbmordred: and a couple of missed s/3.3/3.4/ replacements on the 3.4 builds change23:31
clarkb(I left a review on that one)23:31
*** hamalq has quit IRC23:37

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!